Download to read offline
Measurement and evaluation final-7.pptx
- 1. Measurement and evaluation AbuneGorgorios schools 2015 E.C
- 2. Objectives 1. To helpteachers share their measurment and evaluation experience. 2. To get common understanding on how to conduct measurement and evaluation in our school system 3. To inspire teachers to explore the area further for the sake of professional development.
- 3. Basic concepts • Measurement •Assessment • Test • Evaluation
- 4. Measurement • It oftenconnotes a broader concept • It is the process by which the attributes or dimensions of some physical objects are determined. • It often involves : 1. Identification and definition of quantity, attribute or variable that is to be measured 2. Determining the set of operations
- 5. Evaluation • It isthe act or process that assigns ‘VALUE’ to a measure • It is a continuous process and is concerned with more than just the formal academic achievement of students against the set social cultural …
- 6. Assessment • A processof gathering evidence of students’ performance over a period of time to determine learning and mastery of kills • Goal: to improve student learning
- 7. TEST Test isusually considered the narrowest terms it is standard set of questions to be answered most commonly used method of making measurements in education. It is also an instrument or systematic procedures for measuring sample of behavior by posing a set of questions in a uniform manner. It designed to measure any ability, skill or knowledge.
- 8. Test Development Process •1. Statement of Goals • 2. Content Outline • 3. Table of Specifications • 4. Item Selection • 5. Item Construction • 6. Composition of Instructions • 7. Development of Answer Sheets • 8. Construction of Answer Keys • 9. Test Administration • 10. Test Revision
- 9. Today’s Focus • 3.Table of Specifications • 4. Item Selection • 5. Item Construction
- 10. Two Important Characteristicsof Tests • Reliability – consistency • free from extraneous sources of error • Validity – how well a test measures what it is supposed to measure
- 11. Formative vs. SummativeTests • Formative – monitor progress toward goals within a course of study • Summative – assess overall achievement of course goals
- 12. A.Table of Specifications •Blue print for test • Purpose –ensure proper emphasis given to all elements of a course coverage. –content validity • Guide for writing items
- 13. Table of Specifications(2) • Start with Instructional Objectives – Biology 30: Circulation Unit Test 1. Identifies parts of circulation system. 2. Explains function of parts in relation to whole system. 3. Distinguishes between circulation system and other bodily systems.
- 14. A FORMULA FORTHE NUMBER OF ITEMS let N.I = 𝑃𝑆 𝑋 𝑇.𝑄 𝑃𝑇 N.I - Number of items from each unit or sub-unit Ps- time spent to teach the specific topics or sub-topics T.Q - Total number of questions to be prepared as a whole PT- The total number of periods assigned to teach the course
- 15. Application of theformula Unit Main topics Period allotted Number of items calculation 1 Gen human biology 27 ? 27 𝑥 60 52 = 31 2 Human and diseases 22 ? 22 𝑥 60 52 = 25 3 Flowering plant 3 ? 3 𝑥 60 52 = 4
- 16. Biology 30: Circulation LevelsOf Understanding Content Knowledge Comprehension Application Parts 10 5 0 15 Functio n 5 5 5 15 Relation to other systems 0 5 10 15 15 15 15 45
- 17. B.Item Selection • Typesof Items – Objectively Scored (Selection) • true/false • completion • matching • multiple choice – Subjectively Scored (Supply) • interpretive exercises • essay
- 18. C. Item Construction •Selected Response Test Items – Item construction skills for valid and reliable measures of student achievement. • Guidelines which apply to all types • Specific Suggestions for writing each type • Advantages and Disadvantages of Each
- 19. Guidelines for WritingObjective Items 1. Construct at appropriate level of difficulty for examinees 2. Include Items at appropriate level of difficulty for purpose of test. 3. Test significant elements of a course. 4. Write independent items.
- 20. 5. Construct questionsfree from extraneous reasons for problems. 6. Communicate the question in clear, concise language. 7. In the correct alternatives, paraphrase statements from the text. 8. Exclude clues to correct answer. II. Guidelines for Writing Objective Items
- 21. III. Guidelines forWriting Objective Items 9. Provide one correct answer. 10. Edit the Items.
- 22. Alternate Response Items •Involves the selection of one of two alternatives – true / false – yes / no – right / wrong – fact / opinion • Mainly for Knowledge & Comprehension – Can be written at higher levels
- 23. True / False •Word statements clearly. Vague or ambiguous wording will confuse students. • Avoid overgeneralizing. – Poor: Heavy smoking causes lung cancer.T F – Better: Heavy smoking often causes lung cancer. T F • Avoid Trick questions. – i.e., General Wolseley led Canadian troops to Manitoba in 1870. – 3 tricks: Colonel, British, Not yet a province • Do not use trivial statements to “pad out” the number of questions and marks to arrive at a predetermined level.
- 24. True / False •Statements should be entirely true, or entirely false: – Unacceptable: • In King Lear, Regan ordered Gloucester’s eyes to be plucked out and Gloucester died when he jumped off the cliff of Dover. – Acceptable: • In King Lear, Regan ordered Gloucester’s eyes to be plucked out. (T) • In King Lear, Gloucester died when he jumped off the cliff at Dover. (F)
- 25. True / False •Avoid using universal descriptors such as “never”, “none”, “always”, and “all”. – Testwise students will recognize that there are few absolutes. • Avoid negative words, as they are often overlooked by students. – Poor: It was not unheard of for Henry VIII to close monasteries in England. T F – Better: Henry VIII closed some monasteries in England. (T)
- 26. True / False •Do not include two ideas in one statement unless you are evaluating student’s understanding of cause and effect relationships. – Poor: Porpoises are able to communicate because they are mammals. T F – Better: Porpoises are mammals. T F Porpoises are able to communicate.T F • Provide a “T” and “F” beside each statement and ask students to circle correct answer. – Avoids problem of students writing illegible letters.
- 27. True / False& Variations • Include more false than true statments in any given test and vary the number of false statements from test to test. – tendency to mark more statements true than false. – discrimination between those who know the content and those who do not is greater for false expressions. • Avoid using negative statements. – Under the demands of the testing situation, students may fail to see the negative qualifier.
- 28. Matching Items • Consistof – a column of premises – a column of responses – directions for matching the two. • Similar to multiple choice, but easier and more efficient to construct • Can be written to assess Knowledge, Comprehension, Application, Analysis level behaviors
- 29. Guidelines for WritingMatching Items • Provide clear instructions on how to indicate the correct answers. • Indicate whether the same response can be used more than once. • Maintain grammatical consistency within and between columns. – within a column: either sentence or point form – between columns: one or the other • Ensure that any matching question appears entirely on one page.
- 30. 2. Guidelines forWriting Matching Items • Provide an unequal number of premises and responses – reduces guessing and elimination – increases measure of comprehension • Avoid designing questions which require students to draw lines between premise and response. – confusing for student and marker – provide space for letter or number answers
- 31. 3. Guidelines forWriting Matching Items • Make sure lists are homogeneous. – i.e., do not include items testing names, dates, and events. – Instead, make every response plausible • Make the wording of the premises longer than the wording of the responses. • Identify the items in one list with numbers and those in the second list with letters.
- 32. Example: Directions: 1. Onthe line to the right of each phrase in Column I, write the letter for the word in Column II that best matches the phrase. 2. Each word in Column II may be used once, more than once, or not at all. Column I Column II 1. Name of the answer in addition problems. 2. Name of the answer in subtraction problems. 3. Name of the answer in multiplication problems. 4. Name of the answer in division problems. A. Difference B. Dividend C. Multiplicand D. Product E. Quotient F. Subtrahend G. Sum
- 33. Short Answer TestItems • Typically, the student is asked to reply with a word, phrase, name, or sentence, rather than a more extended response. – Direct Questions / Short Answer • Who is the current Prime Minister of Canada? – Incomplete Sentences / Fill In the Blanks • The current Prime Minister of Canada is _____? • Items are fairly easy to construct and mark • Assess mainly knowledge, comprehension, and some application.
- 34. Guidelines for WritingShort Answer Items • Questions must be carefully worded so that all students understand the specific nature of the question asked and the answer required. – Poor: Wellington defeated Napoleon in _____ ? – Better: In what battle fought in 1815 did Wellington defeat Napoleon? OR In what year did Wellington defeat Napoleon at Waterloo?
- 35. II. Guidelines forWriting Short Answer Items • Word completion or fillin questions so that missing information is at, or near the end of, the sentence. Makes reading and responding easier. – Poor: In the year ______ , Canada turned 100 years old. – Better: Canada turned 100 years old in the year _____ . • Instructions and teacher’s expectations about filling in blanks should be made clear. Indicate whether each blank of equal length represents one word or several words, whether long blanks require sentences or phrases, and whether synonymous terms are accepted.
- 36. III. Guidelines forWriting Short Answer Items • When an answer is to be expressed in numerical units, the unit should be stated. – Poor: If a room measures 7 metres by 4 metres, the perimeter is _____ . – Better: If a room measures 7 metres by 4 metres, the perimeter is _____ metres (or m). • Do not use too many blanks in completion items. The emphasis should be on knowledge and comprehension, not mind reading! – Consider: In the year _____ , Prime Minister _________ signed the __________ , which led to a ___________ which was __________ .
- 37. Multiple Choice Items InterpretiveExercise Guidelines for Writing Advantages & Disadvantages
- 38. CHARACTERISTICS • The mostflexible of the o bjective item types • used to appraise the achievement of any of educational objecticves that can be measured by paper and pencil test except those relating to skill in written expression
- 39. 1. The capitalcity of Canada is a. Vancouver b. Montreal c. Toronto *d. Ottawa Terminology: Multiple Choice 1. Stem: presents the problem 2. Keyed Response: correct or best answer 3. Distracters: appear to be reasonable answers to the examinee who does not know the content 4. Options: include the distracters and the keyed response. 4 1 2 3
- 40. Types of MultipleChoice Items • Correct Answer* – Only one correct response • Best Answer – requires examinee to select alternative closest to being correct – fine distinctions • Multiple Answer – More than one correct or best answer
- 41. Guidelines for Writing: MultipleChoice Items • State stem in the form of a question. – Weak – Canada is • a) a country • b) where you live • c) between the Pacific and Atlantic oceans • d) all of the above – Better – Between which two oceans is Canada located? • a) Atlantic and Indian • b) Atlantic and Pacific • c) Pacific and Indian
- 42. Guidelines for Writing:Stems • Place most of the subject matter in the Stem – ensures full statement of problem • Eliminate extraneous material from the Stem – goal is to measure student achievement, not to present new material – maximize use of time for demonstrating understanding, not reading ability • Avoid Negatively phrased Stems – students may miss the qualifier – use only when learning outcome requires this type of differentiation
- 43. I. Guidelines forWriting: MC • Ensure similarity among alternatives with regard to: – grammatical structure – length – mode of expression • Grammatical errors provide unintentional clues to the answer • When in doubt, students will select the longest alternative as the correct answer
- 44. Example: Length ofAlternatives Neurotics are more likely than psychotics to a. be dangerous to society b. have delusional symptoms c. be dangerous to themselves *d. have insight into their own inappropriate behavior but nevertheless feel rather helpless in terms of dealing with their difficulties
- 45. II. Guidelines forWriting: MC • Make one of the alternatives the most clearly correct or best answer – exception: multiple answer form – reduces intrinsic ambiguity – reduces frustration during test • Make distracters plausible – desire to attract students who really do NOT know the answer to the question – create distracters from elements of the correct response – improves reliability of item
- 46. Example: Undemanding Distracters MickeyMouse’s two nephews are a) Huey, Dewey and Louie b) Clarabelle Cow *c) Morty and Ferdy d) Abbott and Costello
- 47. III. Guidelines forWriting: MC • Avoid parallel language between the Stem and the Correct Response – gives clues to keyed response – emphasizes testwiseness, not knowledge • Randomly distribute answers across the alternative positions – inexperienced test writers emphasize “b” and “c” alternatives (hide the answer!!) – do NOT use an interpretable order of keyed responses
- 48. IV. Guidelines forWriting: MC • Use qualifiers such as “all of the above” and “none of the above” sparingly – testwise students will use process of elimination to select answer – do NOT use to “pad out” the distracters because you cannot think of another one.
- 49. Advantages of MultipleChoice Items • allow more adequate sampling of content. • tend to more effectively structure the problem to be addressed • items can be more efficiently and reliably scored than supply items • different response alternatives can provide diagnostic feedback (item analysis) • items can be constructed to address various levels of cognitive complexity
- 50. Disadvantages of Mult.ChoiceItems • difficult & time consuming to construct good items – leads to emphasis on other selected response item types • can lead the instructor to favour simple recall of facts • high degree of dependence on student’s reading and instructor’s writing ability – can be difficult to achieve clarity of expression • measuring synthesis and evaluation can be difficult • inappropriate for measuring outcomes that require skilled performance
- 51. Subjective Tests Essay questions .Theessay question is defined as the test item which requires a response composed by the examinee. • Essay test is characterized by the freedom of response permitted to the learner. The learner/test taker is free to select his/her own ideas and organize them in his/her own manner.
- 52. Characteristics of Essaytests • The student is free to organize his/her responses • The student answers small number of questions • The examinee uses his/her own words and usually his own handwriting. • Depending on the freedom it provides, essay questions can be of two types; a. Restricted response questions b. Extended response questions
- 53. Advantages of essaytests 1) Ease of construction 2) Measure complex learning outcome 3) Desirable influence on students’ study habit Disadvantages: i. Simple learning outcomes can not be reliably measured ii. Scoring may be highly subjective iii. Very small aspects of the course are measured iv. Takes long time to score.
- 54. Suggestions to improveessay tests • Restrict the use of essay tests to those learning outcomes that can not be satisfactorily measured by objective tests. • Paraphrase each question so that student’s task is clearly indicated. • Avoid the use of optional questions • Indicate an approximate time limit for each question
- 55. scoring essay questgions Preparean outline of the expected answers Use scoring method that is most appropriate Decide how to handle factors that are irrelevant to the learning outcomes. Evaluate all the answers to one question before going to the next one. Evaluate the answers without looking at the name of the student Obtain two or more independent ratings if important decisions are to be based on the results.
- 56. Interpretive Exercise • Usuallybegins with verbal, tabular or graphic information which is the basis for 1 or more multiple choice questions. – map, passage from a story, a poem, a cartoon • Can challenge students at various levels of understanding – application, analysis, synthesis, evaluation • Exercise contains all information needed to answer questions • Readily adaptive to the more important outcomes of disciplines.
- 57. Interpretive Exercises (con’t) •Examples • If student answers incorrectly it is because they have not mastered the thinking or reasoning required by the question, NOT because they failed to memorize background information. • Math questions: give students the formulas, test ability to apply concepts, rather than ability to memorize formulas.
- 58.