Week3 Planning Tests


Published on

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Week3 Planning Tests

  1. 1. Assessment in Schools Planning Classroom Tests Chapter 6
  2. 2. Quality Assessment <ul><li>Good Assessments </li></ul><ul><li>are NOT accidents! </li></ul><ul><li>They are the product of careful, detailed planning. </li></ul>
  3. 3. Assessment Plan <ul><li>Remember, a test is only one evidence that might be used in a Course Evaluation. </li></ul><ul><ul><li>Work samples </li></ul></ul><ul><ul><ul><li>Homework and Projects, Papers and Reports </li></ul></ul></ul><ul><ul><li>Structured Examinations </li></ul></ul><ul><ul><ul><li>Achievement and Performance tests </li></ul></ul></ul><ul><ul><li>Systematic Observation of Typical Performance </li></ul></ul><ul><ul><ul><li>Checklists, Rating scales, Informal anecdotal records </li></ul></ul></ul><ul><ul><li>Student Self-Evaluations </li></ul></ul><ul><ul><ul><li>Reflective Critiques, Oral Interviews </li></ul></ul></ul>
  4. 4. Failure to Plan <ul><li>Unplanned Course Evaluations and Tests are more likely to </li></ul><ul><ul><li>be focused on an unrepresentative sample of learning tasks and curricular content, </li></ul></ul><ul><ul><li>focus heavily on low level learning outcomes, and </li></ul></ul><ul><ul><li>lower the validity of decisions & recommendations </li></ul></ul><ul><ul><li>[see blooms taxonomy] </li></ul></ul>
  5. 5. Test Planning = Decision Making <ul><li>What is the main purpose the test is intended to serve? What will the result be used for? [see 3-3] </li></ul><ul><li>What content or subject-matter will be included? Which topics will be excluded? </li></ul><ul><li>What kinds of item formats will be used? </li></ul><ul><li>How many problems should be included? </li></ul><ul><li>How difficult should the tasks be? </li></ul><ul><li>What criteria and procedures will be used to score the responses? </li></ul><ul><li>[see 3-4] </li></ul>
  6. 6. Test Specifications should be <ul><li>Written, Detailed and Explicit </li></ul><ul><li>Focused on the Following Issues </li></ul><ul><ul><li>Primary Purpose </li></ul></ul><ul><ul><li>Learning outcomes to be tested </li></ul></ul><ul><ul><li>Content Area to be sampled </li></ul></ul><ul><ul><li>How many questions, in what areas (importance) </li></ul></ul><ul><ul><li>Intended level of difficulty, format, and item forms </li></ul></ul><ul><ul><li>[see 3-5] </li></ul></ul>
  7. 7. Task 1 <ul><li>Create a plan for an objectively scored test </li></ul><ul><li>Task specification on Blackboard </li></ul><ul><li>Due Feb 11 th </li></ul><ul><li>[Examples text, 3-6, 3-14 to 3-16] </li></ul>
  8. 8. Instructional Objective Example <ul><li>GIO – Students will </li></ul><ul><ul><li>Prepare written plans for constructing instruments & procedures for assessing instructional outcomes </li></ul></ul><ul><li>SLO – Students will </li></ul><ul><ul><li>Create written specifications for constructing an achievement test </li></ul></ul><ul><ul><li>Provide a well written description of the specific purposes for an assessment </li></ul></ul><ul><ul><li>Identify appropriate instructional objectives </li></ul></ul><ul><ul><li>Outline content to be covered in an assessment </li></ul></ul><ul><ul><li>Create a table of specifications for a test </li></ul></ul>
  9. 9. How to make Good Tests <ul><li>A plan doesn’t ensure the quality of the test. </li></ul><ul><li>The quality of a test is determined by the Quality of the test items </li></ul><ul><ul><ul><li>Appropriate types of questions (given the GIO, SLOs) </li></ul></ul></ul><ul><ul><ul><li>Representative Sample of relevant items (content) </li></ul></ul></ul><ul><ul><ul><li>Appropriate level of difficulty (given the purpose) </li></ul></ul></ul><ul><li>Implications: To improve a test </li></ul><ul><ul><li>make better test items, and plan the test properly for the purposes you decide on. </li></ul></ul>
  10. 10. Test Items determine Test Quality <ul><li>Test Items </li></ul><ul><li>Easy </li></ul><ul><li>Difficult </li></ul><ul><li>Ambiguous </li></ul><ul><li>Representative sample </li></ul><ul><li>Overall Test Quality </li></ul><ul><li>High scores, results negatively skewed </li></ul><ul><li>Low scores, results positively skewed </li></ul><ul><li>Unreliable results </li></ul><ul><li>Valid Results </li></ul>
  11. 11. Objective vs Performance <ul><li>An objective test (or test item) is one where individual scorers would likely agree on the result </li></ul><ul><li>(highly structured, single right or best answer) </li></ul><ul><li>A performance test (or test item) is one that requires expert judgment to score properly </li></ul><ul><li>(less structured but purposive, variety of responses might be considered excellent) </li></ul><ul><li>BOTH Types </li></ul><ul><li>can provide valuable evidence of student achievement </li></ul>
  12. 12. Types of Test Items <ul><li>Selected Response </li></ul><ul><li>Multiple Choice </li></ul><ul><li>Matching </li></ul><ul><li>Alternative Response (T/F) </li></ul><ul><li>Supply Response </li></ul><ul><li>Essay Questions </li></ul><ul><li>Short Answer </li></ul><ul><li>Completion </li></ul>
  13. 13. Underlying Inferences <ul><li>If a student MISSES a questions </li></ul><ul><ul><li>We want to be able to infer that the response indicates he/she did not know the answer, NOT </li></ul></ul><ul><ul><li>That the items lack of clarity or presentation led the student to answer incorrectly </li></ul></ul><ul><li>If a student ANSWERS CORRECTLY </li></ul><ul><ul><li>We want to be able to infer that the response indicates he/she knew the answer, NOT </li></ul></ul><ul><ul><li>That some inadvertent clue led to the selection of the correct answer which he/she did not know </li></ul></ul>
  14. 14. Eliminating Barriers to Performance <ul><li>Eliminate construct irrelevant factors </li></ul><ul><ul><li>Complex sentence structure </li></ul></ul><ul><ul><li>Difficult vocabulary </li></ul></ul><ul><ul><li>Unclear expectations or ambiguous intent </li></ul></ul><ul><li>Ensure students have prerequisite enabling skills and abilities </li></ul><ul><ul><li>Reading skills, understanding of testing practices </li></ul></ul><ul><li>Avoid Bias </li></ul><ul><ul><li>PC issues, but more importantly cultural dependence </li></ul></ul>Utah Assessment and Accountability http://www.schools.utah.gov/assessment/default.aspx
  15. 15. Consider the following text <ul><li>After the early demise of Zunaed who went out for a duck, Mushfiq went out for a golden duck. Sunaed may be a rabbit but Mushfiq is defiantly a ferret when it comes to striking. Ashraful had a moderate start but gave away his wicket. As usual Hasan played with maturity and patience scoring a century. Even so Alam was awarded the Man of the Match. Alam knocked it all over the place. Nafees didn’t open the innings today but got a maiden in his first three overs showing his marvelous bowling. Sumsi’s only run was by a leg bye. </li></ul>
  16. 16. Consider the following text <ul><li>One of the most dramatic moments came in the 5 th over of the 2 nd inning. As the striker stood before his wicket, near the popping crease, in the batting stance. The bowler took his run-up from behind the non-striker's wicket. He passed to one side of the wicket then, as he reached the non-striker's popping crease, bowled the ball towards the striker, bouncing the ball once on the pitch. The striker attempted a reverse sweep but the ball was a googly. The striker swung mightily. The ball lodges in his pads and the umpire declared it dead. At that point the bowler had completed his six balls so the over was complete and a new hurler is brought in. </li></ul>
  17. 17. Question about the text <ul><li>This text describes … </li></ul><ul><ul><ul><li>hunting </li></ul></ul></ul><ul><ul><ul><li>cricket </li></ul></ul></ul><ul><ul><ul><li>a fight </li></ul></ul></ul><ul><ul><ul><li>bowling </li></ul></ul></ul><ul><li>What did Mushfiq do? </li></ul><ul><ul><ul><li>Caught an animal </li></ul></ul></ul><ul><ul><ul><li>Went shopping for eggs </li></ul></ul></ul><ul><ul><ul><li>Stuck out on his first swing </li></ul></ul></ul><ul><ul><ul><li>Performs an amazing animal trick </li></ul></ul></ul>
  18. 18. Another Question <ul><li>The most effective reprimands are those that are </li></ul><ul><ul><ul><li>Soft and Public </li></ul></ul></ul><ul><ul><ul><li>Soft and Private </li></ul></ul></ul><ul><ul><ul><li>Sharp and Public </li></ul></ul></ul><ul><ul><ul><li>Sharp and Private </li></ul></ul></ul>
  19. 19. Suggestions for Writing Items <ul><li>Use a Plan </li></ul><ul><li>Write more items than needed </li></ul><ul><li>Write items well in advance of exam </li></ul><ul><li>Clearly define the task to elicit evidence that the desired learning outcome was accomplished </li></ul><ul><li>Write at appropriate reading level </li></ul><ul><li>Avoid providing clues </li></ul><ul><li>Use items that are relevant </li></ul>