Stages of test development and common test techniques (1)

5,119 views

Published on

Testing

Published in: Education
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,119
On SlideShare
0
From Embeds
0
Number of Embeds
119
Actions
Shares
0
Downloads
237
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

Stages of test development and common test techniques (1)

  1. 1. STAGES OF TEST DEVELOPMENT & COMMON TEST TECHNIQUES DANIEL F. CALI. O.
  2. 2. STAGES OF TEST DEVELOPMENT
  3. 3. OUTLINE STAGES OF TEST DEVELOPMENT SPECIFICATIONS WRITING AND MODERATING ITEMS ITEMS ON NATIVE SPEAKERS VALIDATION CALIBRATION OF SCALES ANALYSIS OF RESULTS ON NON-NATIVE SPEAKERS HANDBOOKS FOR TEST TAKERS TRAINING STAFF STATING THE PROBLEM
  4. 4. • SHOULD BE A TEST DESIGNED BY A SINGLE PERSON OR A TEAM? TEAM! • IT IS DIFFICULT TO DESIGN A TEST BECAUSE OF… …OBJECTIVITY …CRITICISMS …NATIVE COMMAND OF THE LANGUAGE
  5. 5. STEPS…
  6. 6. STATING THE PROBLEM PROFICIENCY FINAL DIAGNOSTIC PLACEMENT KIND? PROGRESS CONSTRAINTS? PURPOSE? EXPERTISE QUESTIONS FACILITIES WASHBACK? ABILITIES? TIME RESULTS HOW ACCURATE? HOW DETAILED?
  7. 7. STATING THE PROBLEM • ONCE THE PROBLEM IS CLEAR, IT IS ALSO IMPORTANT TO: GATHER INFORMATION ON ALREADY EXISITING TEST DESIGNED FOR SIMILAR SITUATIONS!
  8. 8. WRITING SPECIFICATIONS FOR THE TEST • A) CONTENT • B) STRUCTURE, TIMING, MEDIUM/CHANNEL AND TECHNIQUES • C) CRITERIAL LEVELS OF PERFORMANCE • D) SCORING PROCEDURES
  9. 9. A) CONTENT • INCLUDE CLEAR SPECIFICATIONS REGARDING: • • • • • • • • • • SKILLS (SUB-SKILLS) TYPE OF TEXTS ADDRESSEES LENGTH OF TEXTS TOPICS (READABILITY) STRUCTURAL RANGE VOCABULARY RANGE DIALECT, ACCENT AND STYLE SPEED OF PROCESSING
  10. 10. B) STRUCTURE, TIMING, MEDIUM/CHANNE L AND TECHNIQUES • TEST STRUCTURE • NUMBER OF ITEMS • NUMBER OF PASSAGES • TIMING • MEDIUM • KIND OF TEST TECHNIQUE(S)
  11. 11. C) CRITERIAL LEVELS OF PERFORMANCE • ACCURACY • APPROPRIACY • RANGE • FLEXIBILITY • SIZE
  12. 12. D) SCORING PROCEDURES • WHAT RATING SCALE WILL BE USED? • HOW MANY PEOPLE WILL RATE EACH PEACE OF WORK? • WHAT HAPPENS IF TWO OR MORE RATERS DISAGREE ABOUT A PIECE OF WORK?
  13. 13. WRITING AND MODERATING ITEMS • A) SAMPLING • B) WRITING ITEMS • C) MODERATING ITEMS
  14. 14. A) SAMPLING HOW THE TEXTS ARE GOING TO BE CHOSEN?
  15. 15. B) WRITING ITEMS • TRY TO LOOK AT THE TEST THROUGH THE EYES OF TEST TAKERS! • AN ITEM WITHOUT A KEY IS INCOMPLETE! • “THE BEST WAY TO IDENTIFY ITEMS THAT HAVE TO BE IMPROVED OR ABANDONED IS THROUGH THE PROCESS OF MODERATION”.
  16. 16. C) MODERATING ITEMS • INTERVATION OF TWO COLLEAGUES:
  17. 17. INFORMAL TRIALLING OF ITEMS ON NATIVE SPEAKERS • 20 OR MORE… • THEY SHOULD BE SIMILAR TO THE GROUP BEING TESTED IN TERMS OF: AGE EDUCATION GENERAL BACKGROUND
  18. 18. TRIALLING OF THE TEST ON A GROUP OF NON-NATIVE SPEAKERS SIMILAR TO THOSE FOR WHOM THE TEST IS INTENDED • PROBLEMS IN ADMINISTRATION AND SCORING CAN BE NOTED.
  19. 19. ANALYSIS OF RESULTS OF THE TRIAL; MAKING OF ANY NECESSARY CHANGES • STATISTICAL AND QUALITATIVE ANALYSIS. HOW DIFFICULT ARE THE ITEMS? DISCOVER MISINTERPRETATIONS (TO BE MODIFIED OR DROPPED)
  20. 20. CALIBRATION OF SCALES • IT IS IMPORTANT TO COLLECT SAMPLES OF PERFORMANCE AND ASSIGN EACH OF THEM TO A POINT ON THE RELEVANT SCALE.
  21. 21. VALIDATION TOEFL HIGH STAKES TEST LOW STAKES TEST POP QUIZES
  22. 22. WRITING HANDBOOKS FOR TEST TAKERS, TEST USERS AND STAFF
  23. 23. TRAINING STAFF INTERVIEWERS PROCTORS NEED TO BE TRAINED COMPUTER OPERATORS SCORERS RATERS
  24. 24. COMMON TEST TECHNIQUES
  25. 25. OUTLINE DEFINITION MULTIPLE CHOICE GAP FILLING COMMON TEST TECHNIQUES TRUE OR FALSE SHORTANSWERS
  26. 26. WHAT ARE TEST TECHNIQUES? • MEANS OF ELICITING BEHAVIOR
  27. 27. MULTIPLE CHOICE ITEMS • STEM • DISTRACTOR
  28. 28. MULTIPLE CHOICE ITEMS ADVANTAGES DISADVANTAGES DIFFICULT TO WRITE ITEMS GREAT DEMAND ON TIME AND EXPERTISE
  29. 29. YES/NO AND TRUE/FALSE ITEMS Reliability and validity
  30. 30. SHORT-ANSWER ITEMS READING TESTS LISTENING TESTS
  31. 31. GAP FILLING ITEMS • IT IS A VALUABLE TECHNIQUE AS LONG AS THE CONTEXT IS PROVIDED! • A: WHAT WILL HE DO? • B: I THINK HE ______ RESIGN. BUT: A: I WONDER WHO THAT IS. B: IT ______ BE THE DOCTOR. A: HOW CAN YOU BE SO CERTAIN? WILL, MIGHT, COULD MAY, ETC.
  32. 32. • Now you know what a teacher should do when preparing tests, what advices would you give? • (Each one should only mention only 1 advice) Names Advices Carolina Think of the applicability to real situations ar real life situations. Paola Put in your students´ shoes. Mauricio Consider multiple intelligences.
  33. 33. REFERENCES • Huges, Arthur. Testing for Language Teachers. New York: Cambridge UP, 2003.

×