1. A REPORT ON NED 203 Presented by: CARMINA FELISILDA – GURREA NED 203 Graduate Student
2. TOPICS: Criterion – Referenced Test Rubrics (Components and steps to creating a Rubric) Develop a sample rubric for evaluating an essay test
3. OBJECTIVES:1. Define Criterion Referenced Test2. Differentiate Criterion Referenced Test from Norm Referenced Test3. List the steps in the preparation of Criterion – Referenced Measure4. Define Rubrics5. Determine the uses of Rubrics in Evaluation6. Identify the types of Rubrics7. Identify the steps in the Design of Scoring Rubrics8. Apply Rubrics in evaluating an Essay Test
4. CRITERION REFERENCED TESTDefinition: Criterion – referenced evaluation is that which compares the learner with well – defined performance criteria rather than comparing him or her with other learners. (Deyoung 2009) Criterion – referenced evaluation tool defines the behaviour expected at each level of performance.
5. Criterion – referenced tests are tests used to determine the learners’ mastery of a skill, knowledge, or any subject matter taught to them with reference to a criterion established which is usually an absolute standard. (Calderon et.al 1993) The status of the individual learner’s score is not dependent upon the scores of other individual learners who take the same test, but it is dependent upon the established standard or criterion. Criterion referenced instruments are constructed to provide a measure that is interpreted in terms of specific performance criteria. It serves to identify on what extent the individual performance has met a given criteria rather than with other individuals. The meaningfulness of an individual’s score
6. Advantages: Learners are informed of the behaviours expected of them to pass or achieve a certain grade, and if they either attain that level of performance or not. Grading is less subjective when criteria are spelled out and each learner is held to that standard. Since criterion referenced measure identifies weak and strong points in an individual’s performance, it helps to identify the high achievers but not intelligent students or vice versa and those who have been achieving progressively at their own rate. Hence, this measure could be both diagnostic and prognostic in nature.
7. CRITERION REFERENCED TEST VS NORM – REFERENCED TEST CRITERION – REFERENCED TESTS NORM – REFERENCED TESTCriterion – referenced tests are used to Norm – referenced tests are used todetermine the achievement of individuals in determine the achievements of individuals incomparison with a criterion, usually an comparison with the achievement of otherabsolute standard. Suppose the criterion set individuals who take the same test. Supposefor a test is 75%. All those obtaining 75% and a student obtains 65 in a test. This student isabove pass the learning task and all those better than those below 65 and all studentsbelow fail, or at least need reteaching. with scores above 65 are better than he is.In criterion – referenced test, a student is not In norm – referenced tests, learners may besupposed to tackle a higher learning task if he allowed to tackle a higher level of learning taskhas not passed the standard set for the although they have not mastered very well the preceding learning task. Some students arepreceding learning task. The ideal of failure is promoted to the next higher level although theynot emphasized but the student is supposed have not mastered very well the lessons of theto progress according to his own rate of previous level. This is the result of gradinglearning. according to the normal probability curve.In criterion referenced tests, an individual’s score In norm referenced tests, relative placementis simple above or below the standard or indices are used to describe the relativecriterion. If there are things to be done arising placement of scores. Such indices are absolutefrom the result, then these are done, as for ranks, percentile ranks, quartiles, means,instance, if there is a need for reteaching. medians and the like.
8. STEPS IN THE PREPARATION OFCRITERION – REFERENCED MEASUREStep 1. Clearly defining the instructional objective in behavioural terms.Step 2. Outlining the content.Step 3. Preparing the table of specifications.Step 4. Constructing relevant test items
9. Step 1. Clearly defining the instructional objective inbehavioural terms.• Instructional objectives which are specific, observable, measurable, achievable and interpretable serve as the criteria in selecting what is to be learned and the order in which it is to be learned. It helps to identify and select relevant media for each learning activity as well as selecting methods for evaluating the extent of learning.• Below is an example of an objective that is stated in behavioural terms:• Following a discussion on hypertension, the student will be able to state three out of four causes of high blood pressure.
10. Step 2. Outlining the content.• A teacher should make an outline of the content to be covered by the test because an achievement test should adequately sample the subject matter included in the instruction. Similar content outline should be developed for teaching and testing purpose.
11. Step 3. Preparing the table of specifications • A table of specifications is a one – way table that relates the instructional objectives to the course content. To simplify the table, only the general instructional objectives and the major areas of the content are included. The relative stress on the number of items in the table of specifications should reflect the emphasis given in the instructional objective.
12. Step 4. Constructing relevant test items.• In constructing the relevant test items, the instructional objective and the criterion measure are closely related with each other. The test measures the learning outcome mentioned in the objective.
13. Instructional Objective:• Recognizes basic concepts (Concepts on hypertension and its causes)Learning Outcome:Identifies concepts (The student needs to know the causes)Test Item:Which of the following can cause hypertension?A. Active lifestyleB. SmokingC. Low salt dietThe learning outcome on the foregoing example is theidentifying of concepts, so the test item calls for the samebehaviour. If the student recognizes the concept abouthypertension, he will choose option B.
14. WHAT ARE RUBRICS? A Scoring Guide A tool that lists a set of criteria required for a piece of work. A Working Guide They help students to work out aspects/ concepts for examination Authentic Assessment Tool Specifies level of performance expected for different levels of quality Provide students with an indication of how they may revise their work.
15. RUBRICS• Rubrics are rating scales used with performance assessments. They are formally defined as scoring guides, consisting of specific pre-established performance criteria, used in evaluating student work on performance assessments. Rubrics are typically the specific form of scoring instrument used when evaluating student performances or products resulting from a performance task.
16. FEATURES OF RUBRICSIdentify learning criteriaIdentify levels of performanceUsually identify four levels of qualityDescribe problems and difficulties students encounterIdentify thinking skills
17. ADVANTAGES1. Rubrics are powerful tools for both teaching and assessment.2. Rubrics are useful is that they help students become more thoughtful judges of the quality of their own and other’s work.3. Rubrics reduce the amount of time teachers spend evaluating student work.4. Rubrics are easy to use and explain.5. Teachers appreciate rubrics because their “accordion” nature allows them to accommodate heterogeneous classes.
18. TYPES OF RUBRICSHolistic RubricsAnalytic Rubrics
19. ANALYTIC RUBRICS• The teacher scores separate, individual parts of the product or performance first, then sums the individual scores to obtain a total score.• Analytic rubrics are usually preferred when a fairly focused type of response is required; that is, for performance tasks in which there may be one or two acceptable responses and creativity is not an essential feature of the students’ responses. Furthermore, analytic rubrics result initially in several scores, followed by a summed total score – their use represents assessment on a multidimensional level.• The advantage to the use of analytic rubrics is quite substantial. The degree of feedback offered to student and to teachers is significant. Students receive specific feedback on their performance with respect to each of the individual scoring criteria, something that does not happen when using holistic rubrics.
20. HOLISTIC RUBRICS Holistic Rubrics requires the teacher to score the overall process or product as a whole, without judging the component parts separately. Holistic Rubrics are customarily utilized when errors in some part of the process can be tolerated provided the overall quality is high. Nitko (2001) states that use of holistic rubrics is probably more appropriate when performance tasks require students to create some sort of response and where there is no definitive correct answer. Use of holistic rubrics can result in a somewhat quicker scoring process than use of analytic rubrics (Nitko, 2001). This is basically due to the fact that the teacher is required to read through or otherwise examine the student product or performance only once, in order to get an “overall” sense of what the student was able to accomplish. (Mertler, 2001) Since assessment of the overall performance is the key, holistic rubrics are also typically, though not exclusively, used when the purpose of the performance assessment is summative in nature. At most, only limited feedback is provided to the student as a result of scoring performance tasks in this manner.
21. STEPS IN THE DESIGN OF SCORING RUBRICSStep 1:Re – examine the learning objectives to beaddressed by the task. This allows you tomatch your scoring guide with your objectivesand actual instruction.
22. STEPS IN THE DESIGN OF SCORING RUBRICSStep 2:Identify specific observable attributes that youwant to see (as well as those you don’t want tosee) your students demonstrate in theirproduct, process, or performance. Specify thecharacteristics, skills, or behaviours that youwill be looking for, as well as common mistakesyou do not want to see.
23. STEPS IN THE DESIGN OF SCORING RUBRICSStep 3:Brainstorm characteristics that describe eachattribute. Identify ways to describe aboveaverage, average, and below averageperformance for each observable attributeidentified in Step 2.
24. STEPS IN THE DESIGN OF SCORING RUBRICSStep 4a:For holistic rubrics, write thorough narrativedescriptions for excellent work and poor workincorporating each attribute into the description.Describe the highest and lowest levels ofperformance combining the descriptors for allattributes.
25. STEPS IN THE DESIGN OF SCORING RUBRICSStep 4b:For analytic rubrics, write thorough narrativedescriptions for excellent work and poor workfor each individual attribute. Describe thehighest and lowest levesl of performance usingthe descriptors for each attribute separately.
26. STEPS IN THE DESIGN OF SCORING RUBRICSStep 5a:For holistic rubrics, complete the rubric bydescribing other levels on the continuum thatranges from excellent to poor work for thecollective attributes. Write descriptions for allintermediate levels of performance.
27. STEPS IN THE DESIGN OF SCORING RUBRICSStep 5b:For analytic rubrics, complete the rubric bydescribing other levels on the continuum thatranges from excellent to poor work for eachattribute. Write descriptions for all intermediatelevels of performance for each attributeseparately.
28. STEPS IN THE DESIGN OF SCORING RUBRICSStep 6:Collect samples of student work that exemplifyeach level. These will help you score in thefuture by serving as benchmarks.
29. STEPS IN THE DESIGN OF SCORING RUBRICSStep 7:Revise the rubric, as necessary. Be prepared toreflect on the effectiveness of the rubric andrevise it prior to its next implementation.
30. TERMS TO USE IN MEASURING RANGENeeds improvement, Satisfactory, Good, ExemplaryBeginning, Developing, Accomplished, ExemplaryNeeds work, Good, ExcellentNovice, Apprentice, Proficient, DistinguishedNumeric scale 1-5
31. CONCEPT WORDSDepth, Breadth, Quality, Scope, Extent, Com plexity, Degrees, Accuracy – Examples: – Presence – Absence – Complete – Incomplete – Many – Some – None – Major – Minor – Consistent – Inconsistent – Always – RarelyAvoid subjective words such as boring, rather use lacks structure
32. ESSAY TYPE QUESTIONS• Essay types questions lend themselves to testing the highest levels of knowing, especially analysis, synthesis and evaluation. However, they are time consuming for the test takers to answer and they are also time consuming to score.
33. Types of ESSAY TYPE QUESTIONS Restricted response questions• are short answer questions; they placed limitations on the type of response requested (Oermann, 1999).• Some examples are as follows:• Explain in a few sentences why patients with lymphoma are susceptible to infection.• Describe three major pathological processes involve in multiple myeloma.• List two infection prevention measures a nurse should teach a patient who is going home with an ileal conduit.
34. Extended response questions• are full essay questions (Oermann, 1999); they permit the test taker to select all pertinent information, organize it as desired, and express the thesis in a clear manner.• Example:• Compare and contrast two theories of death and dying, and describe how the nurse’s role in supporting a dying patient might differ depending on which of the theories the nurse subscribes to.
35. How to Score Extended Response Questions1) Point method –• the instructor lists the elements that must appear in the answer and assigns points to these elements depending on their importance.2) Rubric method (Brookhart, 1999) –• This system is most useful if the teacher is just as concerned about the overall quality of the answer and the writing style as he or she is about the facts that are included in the essay. The learner is evaluated on whether the points of the argument are clear, logical, and defensible; whether the writing is clear, organized, and grammatically correct; and whether the relevant facts are included.
36. Example:• Compare and contrast two theories of death and dying, and describe how the nurse’s role in supporting a dying patient might differ depending on which of the theories the nurse subscribes to. Point method• In the example below, considering that the entire essay is worth 20 points within the entire examination, the elements and points might look like this:• Compare two theories, covering all the important components. (5pts)• Contrasts the theories, pointing out major differences (5pts)• Describe several aspects of the nurse’s role. (5pts)• Contrasts the nurses role as it depends on the two theories. (5pts)
37. Example:• Compare and contrast two theories of death and dying, and describe how the nurse’s role in supporting a dying patient might differ depending on which of the theories the nurse subscribes to.
38. Example:• Compare and contrast two theories of death and dying, and describe how the nurse’s role in supporting a dying patient might differ depending on which of the theories the nurse subscribes to.
39. Directions: Score the essay answer using the Rubrics scoring method for evaluating essay questions.• ESSAY QUESTION:• IF YOU WERE A NURSE ADMINISTRATOR OF A HOSPITAL HOW DO YOU PLAN TO CARRY OUT YOUR ROLE(S) TO THE FULLEST TO ADVANCE QUALITY PATIENT CARE? • Answer: • The nurse administrator has been described as a “registered nurse whose primary responsibility is the management of health care delivery services and who represents nursing service.” (Roussel, et.al 2006) • As a nurse administrator I would draw from the best and most applicable theories of management to create an individual management style and performance. In order to do this, I should add to the knowledge and skills acquired from experience and from the learning experiences gained during graduate school because solving managerial problems requires a contingency approach since no single approach works for all managerial situations. Also as a nurse administrator one should act with the assumption that clinical nurses and other health care providers want to be competent and that with right managerial support they will be motivated to achieve competence and greater levels of productivity. • In addition I need to increase knowledge of and sensitivity to other health care individuals providing clinical services because these services are integrated into the client’s overall experience of health care, of which nursing is a critical component.
40. ESSAY QUESTION:DESCRIBE THE ORGANIZATIONAL CLIMATE AND ORGANIZATIONALCULTURE OF YOUR WORK SETTINGS(S). IF YOU ARE CURRENTLYUNEMPLOYED, PLEASE BUILD SCENARIOS OR CONCRETEEXAMPLES. ESSAY ANSWER: Organizations differ a great deal. Some are very traditional, preserving their customary ways of doing things even when these processes no longer work well. Others are very progressive, eternally chasing the newest management fad or buying the very latest high-tech equipment. Some seem to be warm, friendly, and open to new people and new ideas. Others are cold, defensive, and indifferent or even hostile to the outside world. These very different organizational climates have a considerable effect on the employees and the people served by the organization. The climate shapes people’s behavior, especially their responses to each other, a very important factor in health care. (Tappen, 2001). In the current workplace, I would say that it is very traditional and rigid. Seniority is usually emphasized although there are some instances that administration has shown openness to new ideas particularly in the delivery of new teaching methods and techniques which is a welcome change.
41. • WHAT ARE THE TRAITS OR QUALITIES OF A NURSE ADMINISTRATOR AND RELATE THIS/THESE QUALITIES WITH MEETING THE REQUIREMENTS FOR WHOLESOME INTERNAL AND EXTERNAL HEALTH ENVIRONMENT.