How to improve test reliability


Published on

Published in: Education, Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

How to improve test reliability

  1. 1. Making a test more reliable By Miguel Angel Carranza, MsE.
  2. 2. On Reliability.. How to improve a test <ul><li>Provide clear and explicit instructions </li></ul><ul><li>Write unambiguous items.. </li></ul><ul><li>What is a table? </li></ul><ul><li>The expected answer which table? </li></ul><ul><li>Do not allow students too much freedom </li></ul><ul><li>Ensure that tests are well laid out and perfectly legible: if you include drawings, they should be clear. </li></ul>
  3. 3. How to Make Tests More Reliable (cont.) <ul><li>Students should be familiar with format and testing technique </li></ul><ul><li>Use items that permit scoring which is as objective as possible </li></ul><ul><li>Agree acceptable responses and appropriate scores </li></ul><ul><li>Train scorers </li></ul><ul><li>Identify candidates by number, not name. </li></ul>
  4. 4. More Tips for Test design <ul><li>Distractors should be parallel in form. </li></ul><ul><li>Don’t allow one item to give a clue to another one. </li></ul><ul><li>All distractors should be about the same length in terms of number of words. </li></ul><ul><li>The use of True-False items is not recommended. </li></ul><ul><li>Subjectively scored items should be scored by a sample of raters. /ideally </li></ul>
  5. 5. More Tips for Test design <ul><li>Review items before testing. There shouldn`t be any typos (typoes?)!!! </li></ul><ul><li>If you write the test questions, have a qualified colleague read through them at least once to ensure accuracy (pilot test if possible) </li></ul><ul><li>Make the distractors plaussible and attractive to the uninformed . </li></ul>
  6. 6. Analyzing test items (cont.) <ul><li>The teacher is usually silent in this method. </li></ul><ul><li>Suggestopedia b) Direct Method c)Silent Way </li></ul><ul><li>The type of standardized test used to measure </li></ul><ul><li>Academic achievement is called an </li></ul><ul><li>a) Case study </li></ul><ul><li>b) Special aptitude test </li></ul><ul><li>c) Test of intelligence </li></ul><ul><li>d) Achivement test because formal classroom learning is measured </li></ul>
  7. 7. Analizing multiple choice items <ul><li>-- Which of the following persons is the best composer? </li></ul><ul><li>a) Bach b) Beethoven c) Brahms d) Mozart </li></ul><ul><li>This item measures opinion. Bad item </li></ul><ul><li>-- The fribbled breg will minter best with a </li></ul><ul><li>a)dignu elma b) erst pinret c) ornt farg d)urtar bink </li></ul><ul><li>- The purpose of the cluss in furmpaling is to remove </li></ul><ul><li>a) Tremalis b) cloughs c) cluss-prags d) plumots. </li></ul><ul><li>The answer is included in the stem. Bad item. </li></ul>
  8. 8. Critiquing Items <ul><li>All but one of the following is not an element. Which is not? </li></ul><ul><li>A) carbon b) salt c)sugar d)plastic </li></ul><ul><li>The most important city in the southeast is_________ </li></ul><ul><li>Better: </li></ul><ul><li>The main transportation center for train and airplane traffic in the Southeast is_____ </li></ul><ul><li>Atlanta B) miami c) New Orleans d) Orlando </li></ul><ul><li>- A figure that has 8 sides is called an____ </li></ul><ul><li>a) pentagon b) quadrilateral c) octagon d) ogive </li></ul>
  9. 9. Another example.. Is anything wrong with it? <ul><li>Who was George Washington? </li></ul><ul><li>Ernest Hemingway wrote_______________. </li></ul><ul><li>Write an essay in which you consider the future of atomic energy. </li></ul><ul><li>Describe what happened to art during the Renaissance </li></ul>
  10. 10. <ul><li>Billi´s mother wanted to bake an apple pie for his aunt and uncle, who were coming for a visit. Billy had not seen them for many months. When Billy`s mother saw that she had no apples in the house, she sent Billy to the store to buy some. Her recipe called for 8 apples to make a pie. If apples at the store cost 30 cents for two, how much money would Billy need to buy eight apples? </li></ul><ul><li>A) $.30 B) $.90 C)$1.20 D) $2.40 </li></ul>
  11. 11. <ul><li>Switzertand </li></ul><ul><li>A) is located in Asia </li></ul><ul><li>B) produces large quantities of gold. </li></ul><ul><li>C) has no direct access to the ocean </li></ul><ul><li>D) is a flat, arid plain. </li></ul><ul><li>As compared to autos of the 1960`s, autos in the 1980s </li></ul><ul><li>a) more horsepower. </li></ul><ul><li>b) to use more fuel </li></ul><ul><li>c) contain more safety features </li></ul><ul><li>d) was less often constructed in foreing countries </li></ul>
  12. 12. Analyzing test items <ul><li>What method or approach do the </li></ul><ul><li>following principles correspond to? </li></ul><ul><li>Accuracy is emphasized______ </li></ul><ul><li>Grammar is taught inductively______ </li></ul><ul><li>Choose the method for the corresponding principle </li></ul><ul><li>Teacher motivates the student and makes them feel comfortable </li></ul><ul><li>a) Community L. L. b)Suggestopedia c) Silent Way </li></ul>
  13. 13. What is a Table of Specifications ? <ul><li>A Table of Specifications is a two-way chart which describes the topics to be covered by a test and the number of items or points which will be associated with each topic. Sometimes the types of items are described, as well. </li></ul><ul><li>Example </li></ul>
  14. 14. Designing a Table of Specifications <ul><li>Tables of Specification typically are designed based on the list of course objectives, the topics covered in class, the amount of time spent on those topics, textbook chapter topics, and the emphasis and space provided in the text. </li></ul><ul><li>In some cases a great weight will be assigned to a concept that is extremely important, even if relatively little class time was spent on the topic. </li></ul>
  15. 15. How can the use of a Table of Specifications benefit your students? <ul><li>A Table of Specifications benefits students in two ways. First, it improves the validity of teacher-made tests. Second, it can improve student learning as well. </li></ul><ul><li>A Table of Specifications helps to ensure that there is a match between what is taught and what is tested. Classroom assessment should be driven by classroom teaching which itself is driven by course goals and objectives. </li></ul>
  16. 16. Formative and Summative Evaluation <ul><li>Formative evaluations, </li></ul><ul><li>are ongoing, they provide information to improve a product or process. </li></ul><ul><li>Summative evaluations </li></ul><ul><li>provide short-term effectiveness or long-term impact information to decide whether or not to adopt a product or process. Summative evaluation can occur just after new course materials were implemented in full (i.e., effectiveness evaluation), or several months to years after the materials were implemented in full. </li></ul><ul><li>OR How about if I don’t give you grades until the end of the semester? A passing or failing. </li></ul>