Power point for the techniques for constructing exam items


Published on

Tips on How To Construct Valid and Reliable Assessment Instruments

Published in: Education, Technology

Power point for the techniques for constructing exam items

  1. 1. Techniques In Constructing Examination Questions Presented By William M. Kapambwe - Senior Research Officer at the 2003 Grade 7 Item Writers’ Training Workshop River Motel-Kafue 14th to 20th December, 2003
  2. 2. “ The Great Aim of Education is Not Knowledge But Action”
  3. 3. Presentation Outline <ul><li>Part 1: Introduction-Item/Objective Congruency. </li></ul><ul><li>Part 2: Assessment Options for Different Learning Domains. </li></ul><ul><li>Part 3: Constructing Objective Test Items- Types and Tips. </li></ul><ul><li>Part 4: Constructing Subjective Test Items- Types and Tips. </li></ul><ul><li>Part 5: Conclusion-Item/Objective Congruency and Item Review. </li></ul>
  4. 4. Introduction: <ul><li>Different techniques or test item formats can be used to construct questions. </li></ul><ul><li>Learning objective being tested determines type of technique to be used. </li></ul><ul><li>Matching test items to the syllabus = item/objective congruence . </li></ul>
  5. 5. Assessment Options for Different Learning Domains <ul><li>The three learning domains: cognitive , affective and psychomotor promote different learning objectives. </li></ul><ul><li>Consequently, different assessment techniques are utilised in order to assess the acquisition of the different learning objectives. </li></ul>
  6. 6. Assessment Options for the Cognitive Domain <ul><li>Assessed with tests, especially for measuring knowledge, comprehension, analysis and evaluation. </li></ul><ul><li>Written or Oral Tests. </li></ul><ul><li>Synthesis and Application: Specific tests such as essay tests are used. </li></ul><ul><li>Application: Listen to and observe pupil undertake performance. </li></ul>
  7. 7. Assessment Options for the Affective Domain <ul><li>Difficult to assess due to the personal and internal qualities of affect. </li></ul><ul><li>Inquiry/problem-solving commonly used. </li></ul><ul><li>Receiving : observations of learners’discussions and questionnaires. </li></ul><ul><li>Responding : observation of learners’ participation and interviews. </li></ul><ul><li>Valuing : Interviews, questionaiires and essays. </li></ul><ul><li>Organisation : Observation of learners’choices. </li></ul><ul><li>Characterisation : Learners’ responsibilities,projects and debates. </li></ul>
  8. 8. Assessment Options for the Psychomotor <ul><li>Hierarchy of difficulty levels ranging from reflex movements to skilled movements. </li></ul><ul><li>Levels generally assessed by observation of either behaviour or performance. </li></ul><ul><li>Observational data can be recorded as an anecdotal record, or by using ckecklists or rating scales. </li></ul>
  9. 9. School-Based Continuous Assessment:Best Alternative for Affective and Psychomotor <ul><li>Not all learning outcomes can be assessed by pencil and paper and within a short time. </li></ul><ul><li>Most learning outcomes in the affective and psychomotor can be observed by observations and recording over a long time. </li></ul><ul><li>Performance tasks are also assessed by observation. </li></ul>
  10. 10. Constructing Objective Tests <ul><li>Objective test questions are also referred to as Selected Response Questions . </li></ul><ul><li>Objective/Selected Response: Every marker or teacher will arrive at the same marks for each of the pupils who took the test. </li></ul><ul><li>Used when teacher wishes to test a relatively large amount of syllabus objectives. </li></ul><ul><li>Used to cover a large number of learners. </li></ul><ul><li>Each test item to be matched to the verb in the instructional objective you wish to test. </li></ul>
  11. 11. Examples of Objective Tests <ul><li>( i) Multiple Choice </li></ul><ul><li>e.g. What is the Capital of Zimbabwe? </li></ul><ul><li> (a) Bulawayo </li></ul><ul><li> (b) Gaborone </li></ul><ul><li> (c) Harare </li></ul><ul><li> (d) Lusaka </li></ul><ul><li>True and False </li></ul><ul><li>e.g. Beans are a good source of protein. True or False? </li></ul>
  12. 12. Examples of Objective Tests <ul><li>Matching </li></ul><ul><li>Functions of parts of a flowering plant and the names of the parts are given. Match the part with its correct function by placing the corresponding letter on the line provided: </li></ul><ul><li>Function of Part Name of part </li></ul><ul><li>1. Attracts insects A. Pollen grain </li></ul><ul><li>2. Contains male eggs B. Filament </li></ul><ul><li>3. Supports the anther C. Stigma </li></ul><ul><li>4. Contains female eggs D. Petal </li></ul><ul><li>(iv) Filling In the Blanks </li></ul><ul><li>The President of Zambia is ____________. </li></ul><ul><li>(a) Kaunda (b) Chiluba (c) Mwanawasa (d) Miyanda </li></ul>
  13. 13. Tips and Checklist for Writing & Reviewing Multiple Choice Items <ul><li>Question should be related to the curriclum </li></ul><ul><li>objective. </li></ul><ul><li>Focus the question on one clearly stated problem. Test one problem with one question. </li></ul><ul><li>Choices should be brief. </li></ul><ul><li>Do not make the stem unnecessarily long by window dressing or by including instructional material. </li></ul><ul><li>List the choices in alphabetical order. </li></ul>
  14. 14. Tips and Checklist for Writing & Reviewing Multiple Choice Items <ul><li>Use clear and simple langauge. Avoid vagueness. Make it easy for the pupil to read and understand material. </li></ul><ul><li>Only one correct choice. </li></ul><ul><li>Avoid “none of the above”. </li></ul><ul><li>Avoid “all of the above”. </li></ul><ul><li>The stem should be in positive form. When negative forms are used, underline and capitalise the negative words. </li></ul><ul><li>Avoid verbal clues to the answer and distractors. </li></ul>
  15. 15. Tips and Checklist for Writing & Reviewing Good Multiple Choice Items <ul><li>Incorrect choices should be reasonable and not wildly off. </li></ul><ul><li>Numerical alternatives should be in numerical order. </li></ul><ul><li>Important objectives may have more than one question. </li></ul>
  16. 16. Tips and Checklist for Writing and Reviewing Matching Items <ul><li>Provide clear instructions or directions. </li></ul><ul><li>Avoid long directions. </li></ul><ul><li>Use numbers to identify the premises and letters to identify the responses. </li></ul><ul><li>Avoid using incomplete sentences and premises. </li></ul><ul><li>Entire set of matches should appear on one page. </li></ul><ul><li>Tell how many times a response may be used. </li></ul><ul><li>Arrange lists in alphabetical order (or numerical order if they are numbers). </li></ul><ul><li>Keep lists short. </li></ul><ul><li>Correct grammar and simple clear language to be used. </li></ul>
  17. 17. Tips and Checklist for Writing and Reviewing True-False <ul><li>The statement should be stated positively. </li></ul><ul><li>The statement must be entirely true or false. </li></ul><ul><li>Avoid tricky negatives. </li></ul><ul><li>Items should be based on important ideas only. </li></ul><ul><li>Statement should discriminate . </li></ul><ul><li>The statement should be based on single major idea. </li></ul><ul><li>Use simple and precise language. </li></ul><ul><li>Use short statements. </li></ul>
  18. 18. Tips and Checklist for Writing and Reviewing True-False <ul><li>Attach the source of reference if the statement contains an opinion, value or attitude. </li></ul><ul><li>Avoid verbal clues (also known as specific determiners) such as “always”, “never”, and “every”. </li></ul><ul><li>Avoid copying sentences directly from text books. </li></ul><ul><li>Avoid presenting answers in a manner that form a pattern; e.g. TFTF or TFFTFF </li></ul><ul><li>Have an approximately equal number of true and false items. </li></ul>
  19. 19. Constructing Subjective Test Formats <ul><li>Also called Brief Constructed Response Items. </li></ul><ul><li>Learner provides from memory a word or phrase that completes a sentence. </li></ul><ul><li>Learner does not choose from a list of answers. </li></ul><ul><li>Learner makes the response without the help of lists or phrases. </li></ul><ul><li>Different markers give different marks for the same test. </li></ul>
  20. 20. Examples of Subjective Tests <ul><li>Subjective test format tests: short answer and essay or composition. </li></ul><ul><li>SHORT ANSWER FORMAT </li></ul><ul><li>(i) Completion item : Supply of one word, number or a phrase as a response to question. </li></ul><ul><li>e.g: (Fill in the Blank) </li></ul><ul><li>Zambia gained independence in the year:____ </li></ul><ul><li>(ii) Structured Question (Restricted Response): </li></ul><ul><li>Short Answer question; Student to write a phrase, a sentence or sentences . </li></ul><ul><li>e.g: Give the name of each of the following symbols: </li></ul><ul><li>+______, = _______, < ______, > _________ </li></ul>
  21. 21. Examples of Subjective Tests <ul><li>(iii) Labelling a Drawing </li></ul><ul><li>e.g: Given a drawing of a fish, label the parts on </li></ul><ul><li> the drawing of the fish below: </li></ul>
  22. 22. Tips and Checklist for Writing and Reviewing Completion/Short Answer Items <ul><li>Statement should be related to the curriculum objectives. </li></ul><ul><li>Make item direct question and not an incomplete statement. </li></ul><ul><li>Question or statement to require a single unique answer. e.g. number, symbol, word, or brief phrase. </li></ul><ul><li>Blank should come at the end of statement where phrase is needed to complete the item. </li></ul><ul><li>Avoid too many blanks in the item. </li></ul><ul><li>Blank item should represent a key word or phrase. </li></ul>
  23. 23. Tips and Checklist for Writing and Reviewing Completion Items <ul><li>Statements should be clear. </li></ul><ul><li>Don’t lift items directly from the text book. </li></ul><ul><li>The key to the item must be definitely correct. </li></ul><ul><li>If numerical response is needed, indicate unit of measure. </li></ul><ul><li>Blanks should be long enough for written answers. </li></ul><ul><li>Blanks are equal in length. </li></ul><ul><li>Blanks are arranged for easy scoring. </li></ul><ul><li>Statements should be stated positively. </li></ul>
  24. 24. Tips and Checklist for Writing and Reviewing “Label a Drawing” Questions. <ul><li>Drawing should be clear. </li></ul><ul><li>Lines should point clearly to intended part. </li></ul><ul><li>The drawing used should vary from original ones studied by learners (should be new examples). </li></ul>
  25. 25. Examples of Subjective Tests <ul><li>ESSAY OR COMPOSITION FORMAT </li></ul><ul><li>Used to measure complex learning outcomes such as writing ability, ability to explain and reason using knowledge. </li></ul><ul><li>Words essay and composition used interchangeably. </li></ul><ul><li>Extended response item and not restricted. </li></ul><ul><li>Letter and composition in English: </li></ul><ul><li>e.g: Write a one page letter. </li></ul><ul><li>Essay/Composition: </li></ul><ul><li>e.g Learners tell or write a story about what they did </li></ul><ul><li>during the holiday. </li></ul>
  26. 26. Tips and Checklist for Writing and Reviewing Compositions & Essays <ul><li>Match question to important learning objective. </li></ul><ul><li>Structure question very clearly; to avoid confusion. </li></ul><ul><li>For composition/essay provide guidelines on what to help and the amount of material to write. </li></ul><ul><li>Design question to measure higher level thinking skills or performance skills. </li></ul>
  27. 27. Assessment Techniques for Continuous Assessment <ul><li>Constructed Response Tasks </li></ul><ul><li>(i) Performance assessment : Learners are asked </li></ul><ul><li>to demonstrate or perform what they know. </li></ul><ul><li>(ii) Product Assessment: Ask learners to use </li></ul><ul><li>their knowledge from what they have learned to make something. </li></ul>
  28. 28. Examples of Performance Assessment Tasks for CA <ul><li>Oral presentation. </li></ul><ul><li>Dance/movement. </li></ul><ul><li>Science activity. </li></ul><ul><li>Athletic skill. </li></ul><ul><li>Dramatic reading. </li></ul><ul><li>Role play. </li></ul><ul><li>Debate. </li></ul><ul><li>Song/poem. </li></ul><ul><li>Practical test. </li></ul><ul><li>Interviews of learners. </li></ul>
  29. 29. Examples of Product Assessment Tasks for CA <ul><li>Illustration or drawing. </li></ul><ul><li>Invented dialogue. </li></ul><ul><li>Models. </li></ul><ul><li>Essay/composition/story. </li></ul><ul><li>Reports. </li></ul><ul><li>Projects. </li></ul><ul><li>Journals </li></ul>
  30. 30. Conclusion <ul><li>Principle of Item/Objective Congruence : Appropriate test techniques should be used to suit selected objectives. </li></ul><ul><li>Item Review: The purpose of the item review is to make test items as clear as possible to the candidates. </li></ul><ul><ul><li>Item review allows for the detection of ambiquities and carelessness which might not have been detected. </li></ul></ul><ul><li>Guidelines on item reviews should be used by item writers to write good items for tests and exams. </li></ul>
  31. 31. End Quote <ul><li>“ A vision without a task is but a dream. </li></ul><ul><li>A task without a vision is drudgery. </li></ul><ul><li>A vision and a task is the hope of the world” </li></ul><ul><li>1730, Church in Sussex </li></ul>