presentation for the 2015 TESOL Asia / Asian EFL Journal 14th Annual International Conference at Clark Freeport Zone, Pampanga, Philippines August 21-23, 2015

1. Assessment in language
learning classrooms:
More questions than
answers
Robert J. Dickey
Keimyung University, S. Korea

2. Defining Assesment…
?

3. Defining Assesment…
• the wide variety of methods that educators use to evaluate, measure, and
document the academic readiness, learning progress, and skill acquisition
of students
• the use of empirical data on student learning to refine programs and
improve student learning
• the process of gathering and discussing information from multiple and
diverse sources in order to develop a deep understanding of what students
know, understand, and can do with their knowledge as a result of their
educational experiences; the process culminates when assessment results
are used to improve subsequent learning
• the systematic basis for making inferences about the learning and
development of students --. the process of defining, selecting, designing,
collecting, analyzing, interpreting, and using information to increase
students’ learning and development
• the systematic collection, review, and use of information about educational
programs undertaken for the purpose of improving student learning and
development

4. Opening Quiz!
Rank the following in determining student grades.
__ performance (as indicated in formal tests, quizzes, etc)
__ (oral) participation in class
__ improvement over the course
__ behavior in class
__ effort
__ motivation
__ assignments
__ punctuality and attendance
__ intuitive, informal assessments of class performance
(Most important = 1, Least/Not important = 9)

5. Who are we testing?
?Elicitation is both testing and lesson confirmation

6. Who are we testing?
• The Students
• The Teacher

7. What are we testing?
?

8. What are we testing?
• Overall English
–competence? proficiency? skill?
• Mastery of course content
• Needs of students
• Improvements during the course
• Teaching performance
• Coursebook / Curriculum

9. Learning Objectives
Bloom’s Taxonomy

10. Learning Objectives

11. When do we test?
?

12. When do we test?
• Pre-test
• Midterms
• Finals
• Quizzes
• More Frequent > > >
–Elicitation
–Homework/Assignments
–“heat of the action”

13. Why do we test?
?

14. Why do we test?
• Required (from above)
• Duty (teacher’s sense)
• Tradition (always been testing)
• Expectation (parents, others)
• Feedback (students, others) (request)
• Ease (following coursebook pattern)

15. How do we test?
?

16. How do we test?
• 4 skills
• “forms” : vocabulary,
grammar, spelling…
• fluency/accuracy
• paper & pencil
• multiple-choice
• fill-the-blank
• open-ended
• interview
• group-tests
• rubrics
• standardized
• textbook-source
• commercial (generic)
• curriculum-based
• standards-based
• teacher-created
• computer-based

17. Types of tests
• Proficiency (regardless of training)
• Achievement (final and progress)
• Diagnostic
• Formative / Pedagogic
• Placement
• Selection/Gate-keeping
• Aptitude
• Encouragement !!!

18. Oral & Written test forms

19. Types of tests

20. Traditional & Alternative
• One-shot, standardized exams
• Timed, multiple-choice format
• Decontextualized test items
• Scores suffice for feedback
• Norm-referenced scores
• Focus on the "right" answer
• Summative
• Oriented to product
• Non-interactive performance
• Fosters extrinsic motivation
• Continuous long-term assessment
• Untimed, free-response format
• Contextualized communicative
tasks
• Individualized feedback and
washback
• Criterion-referenced scores
• Open-ended, creative answers
• Formative
• Oriented to process
• Interactive performance
• Fosters intrinsic motivation

21. Other Assessment Instruments
• Portfolio / Journal / Notebook
• Projects
• Systematic set of observations
• Communicative tests through real-
world tasks
• “personal chats” with instructor

22. How do we score?
?

23. How do we score?
• Normative
• Relative
• “pure” scores
• “Good Job!” / “good scolding”

24. “The Curve”

25. “The Curve”

26. How do we feedback scores?
?

27. How do we feedback scores?
• Public / Private
• Personalized / Relative to the class
• Commentary / Numbers
• Constructive

28. Principles in Assessment
• Practicality
• Reliability
• Validity

29. Practicality in Assessment
• Expense
• Within time constraints
–preparation
–implementation
–grading and review
• Scoring system is specific and time-
efficient

30. Reliability in Assessment
• Consistent & Dependable
• Students (a bad day?)
• Rater (inter-rater, intra-rater)
• Conditions (noise, heat, etc)
• Is a timed-test more fair? To whom?
• More than one possible answer?
• Too easy/difficult (grade distinctions)

31. Validity in Assessment
• Rampantly misunderstood !
1.Face Validity
2.Content Validity
3.Criterion Validity (compare to other test)
(concurrent, predictive)
concurrent, predictive
4.Construct Validity (test assumptions match
matter under investigation)
(speaking test must include vocabulary?)

32. Face Validity
Mostly Under-appreciated
•“looks right” to ALL users
•Familiar tasks
•Familiar format
•Clearly do-able within allowed time
•Clear and uncomplicated items
•Clear directions
•Difficulty is “a reasonable challenge”

33. Authenticity
No one agrees exactly what this means
•Language as natural as possible
•Contextualized testing
•Meaningful topics
•Thematic organization of materials
•Tasks approximate “real-world” tasks

34. Washback
• Pedagogic (formative) testing
INTENDS a positive washback
• Information flows to students and/or
teacher
• Affects future studies:
–Lesson plan designs
–Student motivation and focus

35. Choices we make…

36. Aims of Assessment
• Assessment for Learning
• Assessment as Learning
• Assessment of Learning

37. Assessment for Learning
• to give teachers information
• to modify the learning activities
• to differentiate & focus how individual students
approach their learning
• emphasis is on teachers using the information
from carefully-designed assessments
• to determine what students know
• to gain insights on how, when, & whether
students use what they know
• so instruction can be targeted

38. Assessment as Learning
• a process of developing & supporting
metacognition for students
• focuses on the role of the student as the critical
connector between assessment and learning
– Students, as active, engaged and critical assessors
make sense of information, relate it to prior knowledge,
and use it for new learning
• the vehicle for helping students develop, practice
and become comfortable with reflection and with
critical analysis of their own learning

39. Assessment of Learning
• to confirm what students know
• to demonstrate whether or not the students have
met the standards and/or show how they are
placed in relation to others
• Focus on creating accurate and sound
statements of proficiency or competence for
students, so that the recipients of the information
can use the information to make reasonable and
defensible decisions

40. Integrating Assessment

41. A quick survey
• Instructors for English Conversation
courses
• Universities in S. Korea
• June 2015
• 58 respondents
• Online collection (solicited through
Facebook groups)

42. Included in final course grade
• attendance 94.83%
• participation 91.38%
• final examination 91.38%
• midterm examination 86.21%
• assignments 53.45%
• presentation (oral) 41.38%
• quiz 36.21%
• report (written) 12.07%
• notebook 5.17%
• Other includes
-- online assignments -- TOEIC test
-- transcription of a conversation -- video assignment
-- writing lab

43. Included in final examination
• Oral-based open response 61.11%
• paper-based multiple-choice 40.74%
• paper-based vocabulary assessment 40.74%
• paper-based fill-in the blank/complete the sentence 37.04%
• paper-based grammar assessment 35.19%
• oral-based vocabulary assessment 33.33%
• paper-based open response 31.48%
• oral-based grammar assessment 29.63%
• listening (accuracy) 29.63%
• pronunciation 24.07%
• oral-based taught phrases 24.07%
• oral-based "content" (non-linguistic materials) 24.07%
• paper-based taught phrases 14.81%
• reading (accuracy) 11.11%
• paper-based "content" (non-linguistic materials) 9.26%
• oral-based fill-in the blank/complete the sentence 3.70%
• other

45. Questions?
Rob Dickey
rjdickey@content-english.org
File available at http://tinyurl.com/q7pm5vt