Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Assessments
Principles and Ideas
Resources:
● Bracey, G. W. (2002). Put to the test: An
educator’s and consumer’s guide to
standardized testing (Revised Ed...
Two Problems
1. Construct Validity
a. Doesn't test what it claims to test
2. Face Validity
a. Doesn't seem to be an accura...
Why this matters
Problems with Construct Validity and/or Face
Validity of your assessments makes everyone
upset -
● Studen...
5Principles for Assessments
1. Relevant & Appropriate
The assessment should (1) measure
the student learning objectives or
sub-objectives of the cours...
1. Relevant & Appropriate
(1) Make sure you can connect what's on your
assessment to one of the SLOs on your syllabus
--->...
1. Relevant & Appropriate
(2) Make sure that your assessment actually
assesses the individual student's ability to meet
th...
Listening D
● Students will practice listening to
level-appropriate materials for
main ideas and details.
● ...predicting,...
"Relevant and Appropriate" Assessment
Advanced Academic Listening
● Students will be able to demonstrate
understanding of ...
2. Biased for Good
The cognitive skills involved in test
taking extend beyond merely answering
a question. Make sure the "...
2. Biased for Good
The hidden cognitive challenges of an
assessment mean that it is typically "biased for
bad."
Ex: A Fill...
2. Biased for Good
The main idea: Make "everything else" (besides
what you are testing) as easy as possible.
● Super Clear...
"Biased for Bad" Assessment (L.I. Vocabulary)
"Biased for Good" Assessment (L.I. Everyday English)
3. Fair
The assessment should be fair both in
(1) construction, and (2) appearance.
This is where written Point Values and...
3. Fair
Construct: Rubrics help with Construct Validity because it
is easier to analyse where the final grade is coming fr...
"Unfair" Assessment (L.I. Everyday English/Conversation)
More "Fair" Assessment (H.I. Conversation)
**Still problematic
Most "Fair" Assessment (Int. Conversation)
4. Washback
The assessment should provide useful
feedback to the student and further
instructional value to the teacher.
4. Washback
A simple letter grade or number doesn't really
give any indication of strength and weakness.
Washback increase...
"Washback" Assessment (Int. Pronunciation & Fluency)
"Washback" Grade Sheet (L.I. Everyday English)
5. Triangulated
You should use multiple measures
(different kinds of assessments) to
determine student performance on
cour...
5. Triangulated
Assessment is like data collection. Research
studies (esp. in Education and Linguistics) use
triangulation...
5. Triangulated
Alternatives to Pen-and-Paper Assessments:
● Recorded Speaking Tasks
https://moodle.eli.ucsd.edu/mod/assig...
5. Triangulated
Alternatives to Pen-and-Paper Assessments:
● Journals / Logs
L.I Reading 102
- Reading Log
5. Triangulated
Alternatives to
Pen-and-Paper
Assessments:
● Portfolios
5. Triangulated
Alternatives to Pen-and-Paper Assessments:
● "Real World" Projects
● Multiple-Intelligence Based Ideas
● O...
Some CEA Issues
CEA Problems
● Point Values and/or Rubrics
● Meets the Course Objectives and/or tests the
actual skill
● In-Class Work (in...
My Design Process
in 5 easy steps
My Design Process
1. Put the objective (or sub-objective) on the
assessment
2. Design a (related, meaningful) task that
st...
Communicative
Application
Activity
yaaaaaay!
Assessment Evaluation
Look at the assessment that your group has,
and answer the following questions about it.
Elect a spe...
Upcoming SlideShare
Loading in …5
×

Assessments

321 views

Published on

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

Assessments

  1. 1. Assessments Principles and Ideas
  2. 2. Resources: ● Bracey, G. W. (2002). Put to the test: An educator’s and consumer’s guide to standardized testing (Revised Edition). ● Brown, H. D. (2004). Language assessment: Principles and classroom practices. ● Hadley, A. O. (2000). Classroom testing. In Teaching Language in Context, (3rd ed.) ● Popham, W. J. (2001). The truth about testing: An educator’s call to action.
  3. 3. Two Problems 1. Construct Validity a. Doesn't test what it claims to test 2. Face Validity a. Doesn't seem to be an accurate measure of the student's ability
  4. 4. Why this matters Problems with Construct Validity and/or Face Validity of your assessments makes everyone upset - ● Students ● CEA Committee ● You
  5. 5. 5Principles for Assessments
  6. 6. 1. Relevant & Appropriate The assessment should (1) measure the student learning objectives or sub-objectives of the course, (2) without measuring other non-objective skills.
  7. 7. 1. Relevant & Appropriate (1) Make sure you can connect what's on your assessment to one of the SLOs on your syllabus ---> Goes back to the instruction EX: Intermediate Pronunciation - IPA Quiz ● Students will be able to correctly position their lips, tongue, and mouth in order to properly pronounce both consonants and vowels in English. ● Students will also be able to identify word stress, rhythm, and intonation in native speaker sentences... ● Students will be able to give 3-5 minute presentations and produce dialogues, role plays, and short conversations focusing on the specific sounds and aspects of English pronunciation...
  8. 8. 1. Relevant & Appropriate (2) Make sure that your assessment actually assesses the individual student's ability to meet those objectives you've chosen (or that you only grade the target skills) Example: ● Speaking assessment with a written prompt ● Reading test with a written response where grammar and punctuation are graded ● Listening test with written questions and/or answers
  9. 9. Listening D ● Students will practice listening to level-appropriate materials for main ideas and details. ● ...predicting, making inferences, and identifying points of view and meaning of intonation. ● ...recognizing contractions, thought groups and stress, and stress shift changes. ● Students will practice distinguishing between formal and informal language. ● Students will practice taking notes. Not "Relevant & Appropriate" Test
  10. 10. "Relevant and Appropriate" Assessment Advanced Academic Listening ● Students will be able to demonstrate understanding of a variety of formal oral presentations (e.g. lectures, broadcasts, verbal directions) ● Students will demonstrate understanding of live or recorded informal conversations among speakers of English without much repetition, ● Students will understand, follow, and take notes on academic lectures.
  11. 11. 2. Biased for Good The cognitive skills involved in test taking extend beyond merely answering a question. Make sure the "everything else" of your test is as cognitively undemanding as possible.
  12. 12. 2. Biased for Good The hidden cognitive challenges of an assessment mean that it is typically "biased for bad." Ex: A Fill-in-the-blank Grammar Test ● Understanding the directions ● Reading time (time limit?) ● Affect: Imagining failure, stress ● Distractions: Trying to ignore the next student reading out loud
  13. 13. 2. Biased for Good The main idea: Make "everything else" (besides what you are testing) as easy as possible. ● Super Clear Directions ● Clean and Intuitive Design ● Expected Format ● Familiar Tasks Make the test so that a student who knows the answer isn't stopped by the "everything else"
  14. 14. "Biased for Bad" Assessment (L.I. Vocabulary)
  15. 15. "Biased for Good" Assessment (L.I. Everyday English)
  16. 16. 3. Fair The assessment should be fair both in (1) construction, and (2) appearance. This is where written Point Values and Rubrics are very helpful.
  17. 17. 3. Fair Construct: Rubrics help with Construct Validity because it is easier to analyse where the final grade is coming from. ● weighted sections in one, condensed area ● helps promote objective testing practices Face: Rubrics help with Face Validity because the students can also see where their grade will come from. ● fosters student responsibility (rubric before test) ● helps with preparation ● minimizes possible "Teacher-doesn't-like-me-so-I-got- a-bad-grade" ideas
  18. 18. "Unfair" Assessment (L.I. Everyday English/Conversation)
  19. 19. More "Fair" Assessment (H.I. Conversation) **Still problematic
  20. 20. Most "Fair" Assessment (Int. Conversation)
  21. 21. 4. Washback The assessment should provide useful feedback to the student and further instructional value to the teacher.
  22. 22. 4. Washback A simple letter grade or number doesn't really give any indication of strength and weakness. Washback increases autonomy in student learning and gives the assessment a "use" beyond the grade. Many Forms: ● Personalized Feedback on Writing ● Short "Good" and "Bad" points ● Break down the grade
  23. 23. "Washback" Assessment (Int. Pronunciation & Fluency)
  24. 24. "Washback" Grade Sheet (L.I. Everyday English)
  25. 25. 5. Triangulated You should use multiple measures (different kinds of assessments) to determine student performance on course objectives.
  26. 26. 5. Triangulated Assessment is like data collection. Research studies (esp. in Education and Linguistics) use triangulation to make sure that a certain method of data collection doesn't overlook something. Classroom assessments should do the same thing.
  27. 27. 5. Triangulated Alternatives to Pen-and-Paper Assessments: ● Recorded Speaking Tasks https://moodle.eli.ucsd.edu/mod/assignment/submissions.php?id=9914
  28. 28. 5. Triangulated Alternatives to Pen-and-Paper Assessments: ● Journals / Logs L.I Reading 102 - Reading Log
  29. 29. 5. Triangulated Alternatives to Pen-and-Paper Assessments: ● Portfolios
  30. 30. 5. Triangulated Alternatives to Pen-and-Paper Assessments: ● "Real World" Projects ● Multiple-Intelligence Based Ideas ● Other ideas?
  31. 31. Some CEA Issues
  32. 32. CEA Problems ● Point Values and/or Rubrics ● Meets the Course Objectives and/or tests the actual skill ● In-Class Work (instead of at home, as much as possible) - possible cheating issues
  33. 33. My Design Process in 5 easy steps
  34. 34. My Design Process 1. Put the objective (or sub-objective) on the assessment 2. Design a (related, meaningful) task that students can do to prove that they have met this objective 3. Make the rubric 4. Design the washback 5. Work on the presentation issues and the directions
  35. 35. Communicative Application Activity yaaaaaay!
  36. 36. Assessment Evaluation Look at the assessment that your group has, and answer the following questions about it. Elect a speaker to share your answers with the group. 1. What are the major (obvious) flaws in this assessment? 2. Without ditching the whole assessment and starting over, what can you do (specifically) to improve it?

×