Testing

1. STAGES OF TEST
DEVELOPMENT
&
COMMON TEST TECHNIQUES
DANIEL F. CALI. O.

2. STAGES OF
TEST
DEVELOPMENT

3. OUTLINE
STAGES OF TEST DEVELOPMENT
SPECIFICATIONS
WRITING AND
MODERATING
ITEMS
ITEMS ON
NATIVE
SPEAKERS
VALIDATION
CALIBRATION
OF SCALES
ANALYSIS OF
RESULTS
ON NON-NATIVE
SPEAKERS
HANDBOOKS
FOR TEST TAKERS
TRAINING STAFF
STATING THE
PROBLEM

4. • SHOULD BE A TEST DESIGNED BY A SINGLE PERSON OR A TEAM?
TEAM!
• IT IS DIFFICULT TO DESIGN A TEST BECAUSE OF…
…OBJECTIVITY
…CRITICISMS
…NATIVE COMMAND OF THE LANGUAGE

5. STEPS…

6. STATING THE PROBLEM
PROFICIENCY
FINAL
DIAGNOSTIC
PLACEMENT
KIND?
PROGRESS
CONSTRAINTS?
PURPOSE?
EXPERTISE
QUESTIONS
FACILITIES
WASHBACK?
ABILITIES?
TIME
RESULTS
HOW
ACCURATE?
HOW
DETAILED?

7. STATING THE PROBLEM
• ONCE THE PROBLEM IS CLEAR, IT IS ALSO IMPORTANT TO:
GATHER INFORMATION ON ALREADY EXISITING TEST DESIGNED FOR SIMILAR
SITUATIONS!

8. WRITING SPECIFICATIONS FOR THE TEST
• A) CONTENT
• B) STRUCTURE, TIMING, MEDIUM/CHANNEL AND TECHNIQUES
• C) CRITERIAL LEVELS OF PERFORMANCE
• D) SCORING PROCEDURES

9. A) CONTENT
• INCLUDE CLEAR SPECIFICATIONS REGARDING:
•
•
•
•
•
•
•
•
•
•
SKILLS (SUB-SKILLS)
TYPE OF TEXTS
ADDRESSEES
LENGTH OF TEXTS
TOPICS
(READABILITY)
STRUCTURAL RANGE
VOCABULARY RANGE
DIALECT, ACCENT AND STYLE
SPEED OF PROCESSING

10. B)
STRUCTURE, TIMING, MEDIUM/CHANNE
L AND TECHNIQUES
• TEST STRUCTURE
• NUMBER OF ITEMS
• NUMBER OF PASSAGES
• TIMING
• MEDIUM
• KIND OF TEST TECHNIQUE(S)

11. C) CRITERIAL LEVELS OF
PERFORMANCE
• ACCURACY
• APPROPRIACY
• RANGE
• FLEXIBILITY
• SIZE

12. D) SCORING PROCEDURES
• WHAT RATING SCALE WILL BE USED?
• HOW MANY PEOPLE WILL RATE EACH PEACE OF WORK?
• WHAT HAPPENS IF TWO OR MORE RATERS DISAGREE ABOUT A PIECE OF
WORK?

13. WRITING AND MODERATING ITEMS
• A) SAMPLING
• B) WRITING ITEMS
• C) MODERATING ITEMS

14. A) SAMPLING
HOW THE TEXTS ARE GOING TO BE CHOSEN?

15. B) WRITING ITEMS
• TRY TO LOOK AT THE TEST THROUGH THE EYES OF TEST TAKERS!
• AN ITEM WITHOUT A KEY IS INCOMPLETE!
• “THE BEST WAY TO IDENTIFY ITEMS THAT HAVE TO BE IMPROVED OR
ABANDONED IS THROUGH THE PROCESS OF MODERATION”.

16. C) MODERATING ITEMS
• INTERVATION OF TWO COLLEAGUES:

17. INFORMAL TRIALLING OF ITEMS ON
NATIVE SPEAKERS
• 20 OR MORE…
• THEY SHOULD BE SIMILAR TO THE GROUP BEING TESTED IN TERMS OF:
AGE
EDUCATION
GENERAL BACKGROUND

18. TRIALLING OF THE TEST ON A GROUP
OF NON-NATIVE SPEAKERS SIMILAR
TO THOSE FOR WHOM THE TEST IS
INTENDED
• PROBLEMS IN ADMINISTRATION AND SCORING CAN BE NOTED.

19. ANALYSIS OF RESULTS OF THE TRIAL;
MAKING OF ANY NECESSARY
CHANGES
• STATISTICAL AND QUALITATIVE ANALYSIS.
HOW DIFFICULT ARE THE ITEMS?
DISCOVER MISINTERPRETATIONS (TO BE MODIFIED OR DROPPED)

20. CALIBRATION OF SCALES
• IT IS IMPORTANT TO COLLECT SAMPLES OF PERFORMANCE AND ASSIGN EACH
OF THEM TO A POINT ON THE RELEVANT SCALE.

21. VALIDATION
TOEFL
HIGH
STAKES TEST
LOW
STAKES TEST
POP
QUIZES

22. WRITING HANDBOOKS FOR TEST
TAKERS, TEST USERS AND STAFF

23. TRAINING STAFF
INTERVIEWERS
PROCTORS
NEED
TO BE
TRAINED
COMPUTER
OPERATORS
SCORERS
RATERS

24. COMMON
TEST
TECHNIQUES

25. OUTLINE
DEFINITION
MULTIPLE
CHOICE
GAP
FILLING
COMMON
TEST
TECHNIQUES
TRUE
OR
FALSE
SHORTANSWERS

26. WHAT ARE TEST TECHNIQUES?
• MEANS OF ELICITING BEHAVIOR

27. MULTIPLE CHOICE ITEMS
• STEM
• DISTRACTOR

28. MULTIPLE CHOICE ITEMS
ADVANTAGES
DISADVANTAGES
DIFFICULT TO
WRITE ITEMS
GREAT DEMAND
ON TIME AND
EXPERTISE

29. YES/NO AND TRUE/FALSE ITEMS
Reliability and validity

30. SHORT-ANSWER ITEMS
READING
TESTS
LISTENING
TESTS

31. GAP FILLING ITEMS
• IT IS A VALUABLE TECHNIQUE AS LONG AS THE CONTEXT IS PROVIDED!
• A: WHAT WILL HE DO?
• B: I THINK HE ______ RESIGN.
BUT:
A: I WONDER WHO THAT IS.
B: IT ______ BE THE DOCTOR.
A: HOW CAN YOU BE SO CERTAIN?
WILL, MIGHT,
COULD
MAY, ETC.

32. • Now you know what a teacher should do when preparing tests, what
advices would you give?
• (Each one should only mention only 1 advice)
Names
Advices
Carolina
Think of the applicability to real situations ar real life situations.
Paola
Put in your students´ shoes.
Mauricio
Consider multiple intelligences.

34. REFERENCES
• Huges, Arthur. Testing for Language Teachers. New York: Cambridge UP,
2003.