Sound Assessment Design
Chapters 4-7
Sunday, October 13, 13
Chapter 4:
Sound Design
Learning Targets
Select appropriate method(s) to assess specific
learning targets.
Follow steps in ...
Key 3: Sound Design
Do assessment methods match learning targets?
Does the sample represent learning appropriately?
Are it...
Assessment Methods
Selected response

Performance assessment

Multiple choice

Performance task

True/False

Performance c...
Target-Method Match
Strong - The method works for all learning targets of this type
Good - The method works for many learn...
Target-Method Match
Selected
Written Performance
Response Response Assessment

Personal
Communication

Knowledge

Good

St...
Activity 4.2 TargetMethod Match Template
Target Type
Learning Target
1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

Sunday, Octo...
Assessment
Development Cycle
Planning Stage
Determine who will use the assessment results and how they will use them.
Iden...
Test Blueprint
List/classify the learning targets
Write learning targets into appropriate spaces
depending on blueprint fo...
Unit:

Test Blueprint (MultipleMethod)
Learning Targets

Sunday, October 13, 13

Target Type

Assessment Method Percent Im...
Test Blueprint (Selected
Response)
Learning Targets

Sunday, October 13, 13

Problem #s

Total Points
Learning Unit Activities
Use Target-Method Match template to determine
proper assessment strategies for learning targets
A...
Assessment Methods
Selected response
Multiple choice
True/False
Matching
Fill-in-the-blank questions

Sunday, October 13, ...
When do we need
Selected Response?
Use selected response assessments when...
Broad topic requiring wide-range coverage
Dia...
Choosing Selected
Response Types
Item

Used when...

Advantage

Limitations

MultipleChoice

1 correct/several
plausible

...
Writing Quality Items
KISS - Keep wording simple and focused. Aim for the lowest possible
reading level.
Ask a full questi...
Multiple-Choice
Ask a complete question to the the item started, if you can.
Don’t repeat the same words within each respo...
True/False

Make the entire item entirely true or entirely false!!

Sunday, October 13, 13
Matching
Provide clear directions for making the match.
Keep the list of things to be matched short.
Keep the list of thin...
Fill-in-the-Blank
Ask respondents a question and provide space for
an answer.
Try to stick to one blank per item.
Don’t le...
Review for Quality
Match to blueprint
List/number targets to be covered
On test...mark each item with learning target
numb...
Review for Quality

Does the item test what you intended?
Is the item well-written?

Sunday, October 13, 13
Activity 5.2 Selected Response Test Quality Checklist

Classroom Assessment for Student Learning: Doing It Right—Using It ...
Reviewing/Analyzing
Results
Problem #

Sunday, October 13, 13

Learning Target #

Confident

Unsure

Right

Wrong

Simple M...
Activities for Selected
Response Assessment
Develop an Assessment for Learning Activity
Prepare a Quiz or Test for Formati...
Assessment Methods
Written response
Short answer items
Extended written response items

Sunday, October 13, 13
Written Response
Use when...
Students are capable of writing in English (or
desired language)
Less time-efficient methods ar...
Sample Size
Prioritization should parallel the amount of time
and emphasis give the various targets or topics in
teaching....
Written Assessment
Blueprint
Learning Target

Sunday, October 13, 13

Points
Written Assessment
Blueprint
Pattern of Reasoning

Content
Target 1
Target 2
Target 3
TOTAL

Sunday, October 13, 13

Know
...
Devising Extended
Written Response Items
Knowledge Mastery: Assess factual and
conceptual knowledge
Knowledge Mastery AND ...
Devising Extended
Written Response Items
Knowledge Mastery
Set the Context
Tell what to describe or explain
Point the way ...
Devising Extended
Written Response Items
Knowledge Mastery AND Reasoning
Set the Context
Describe the reasoning task
Point...
Devising Extended
Written Response Items
Interpretive
Set the Context
Describe the reasoning task
Point the way to an appr...
Scoring Guides
Scoring List - Simply lists required tasks and the
point allotment for each task
Scoring Rubric - Rates res...
Creating Task-Specific
Rubric
Begin with correct statement of conceptual
understanding
Identify characteristics of partial ...
Creating General Rubric

Same as task-specific rubric, but in place of
content-specific statements, use general
descriptions...
Quality Control
Can you write an outline of a high-quality
response?
Have a qualified colleague write a response and
discus...
Sunday, October 13, 13
Activities for Written
Response Assessment
Create a Short Answer Item and Scoring Guide
Create an Extended Written Respons...
Assessment Methods
Performance assessment
Performance task
Performance criteria

Sunday, October 13, 13
Performance
Assessment
Use it when...
Assessing skills, products, and some forms of
reasoning
Working with young/primary s...
Determining Sample
Size
Complexity of the Target: Complex = Larger;
Simple = Smaller
Decision the Evidence will Inform: Ho...
Characteristics of a
Good Task
Content
Target Alignment (does the task hit the mark)
Authenticity (realistic context)
Choi...
Characteristics of a
Good Task
Structure
Knowledge students are to use (What should they know?)
What students are to accom...
Characteristics of a
Good Task
Sampling
Use of information (How many tasks assigned?
Does the task sample adequately?)
Cov...
Characteristics of a
SEE P241
Good Task

Activity 7.5 Rubric for Rubrics

Sunday, October 13, 13

1
Tasks to Elicit Good
Writing
RAFTS
Role: What is my role?
Audience: Who is my audience?
Format: What is the appropriate fo...
Tasks to Elicit Good
Writing
Strong verbs
Role: What is my role?
Audience: Who is my audience?
Format: What is the appropr...
Quality Control
Activity 7.4 Rubric for Tasks

Sunday, October 13, 13

FULL CHART P222
1
Structure of a Rubric
Criteria
Content
Organization
Delivery
Language Use
Indicators (for each Criterion)
Content
Organiza...
Structure of a Rubric
Content
Clear main topic
All information important to understanding
Facts, details, anecdotes, and/o...
Structure of a Rubric
Organization
Opening introduces topic and catches attention
Sequence of ideas supports meaning and i...
Structure of a Rubric
Delivery
Eye contact
Voice
Articulation
Pace
No fillers
Gestures/movements (where appropriate)
Notes ...
Structure of a Rubric
Organization
Opening introduces topic and catches attention
Sequence of ideas supports meaning and i...
Process for Developing
a Rubric
Establish a knowledge base
Gather samples of student performances or
products
Sort student...
7 Strategies for Using Rubrics as
Instructional Tools in the Classroom
Where am I going?
1. Provide students with a clear ...
Activities for
Performance Task
Create a Rubric
Evaluate a Rubric for Quality
Create a Student-friendly Version of a Rubri...
Assessment Methods
Personal communication
Questions during instruction
Interviews and conferences
Participation
Oral exams...
Upcoming SlideShare
Loading in …5
×

Sound%20 design%20%28ch%204 7%29

663 views

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
663
On SlideShare
0
From Embeds
0
Number of Embeds
81
Actions
Shares
0
Downloads
18
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Sound%20 design%20%28ch%204 7%29

  1. 1. Sound Assessment Design Chapters 4-7 Sunday, October 13, 13
  2. 2. Chapter 4: Sound Design Learning Targets Select appropriate method(s) to assess specific learning targets. Follow steps in the Assessment Development Cycle. Create an assessment blueprint. Use an assessment blueprint with students as an assessment for learning Sunday, October 13, 13
  3. 3. Key 3: Sound Design Do assessment methods match learning targets? Does the sample represent learning appropriately? Are items, tasks, and scoring rubrics of high quality? Does the assessment control for bias? Sunday, October 13, 13
  4. 4. Assessment Methods Selected response Performance assessment Multiple choice Performance task True/False Performance criteria Matching Personal communication Fill-in-the-blank questions Written response Questions during instruction Interviews and conferences Short answer items Extended written response items Sunday, October 13, 13 Participation Oral exams Student journals and logs
  5. 5. Target-Method Match Strong - The method works for all learning targets of this type Good - The method works for many learning topics of this type Partial - The method works in some instances for learning targets of this type Poor - The method never works for learning targets of this type See chart p94 Sunday, October 13, 13
  6. 6. Target-Method Match Selected Written Performance Response Response Assessment Personal Communication Knowledge Good Strong Partial Strong Reasoning Good Strong Partial Strong Skill Partial Poor Strong Partial Product Poor Poor Strong Poor Sunday, October 13, 13
  7. 7. Activity 4.2 TargetMethod Match Template Target Type Learning Target 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Sunday, October 13, 13 K R S Assessment Method P SR WR PA PC
  8. 8. Assessment Development Cycle Planning Stage Determine who will use the assessment results and how they will use them. Identify the learning targets to be assessed. Select the appropriate assessment method or methods. Determine the sample size. Development Stage Develop or select items, exercises, tasks, and scoring procedures. Review and critique the overall assessment for quality before use. Use Stage Conduct and score the assessment. Revise as needed for future use. Sunday, October 13, 13
  9. 9. Test Blueprint List/classify the learning targets Write learning targets into appropriate spaces depending on blueprint format If using multiple methods, identify appropriate method for each target Weight importance of each target Sunday, October 13, 13
  10. 10. Unit: Test Blueprint (MultipleMethod) Learning Targets Sunday, October 13, 13 Target Type Assessment Method Percent Importance
  11. 11. Test Blueprint (Selected Response) Learning Targets Sunday, October 13, 13 Problem #s Total Points
  12. 12. Learning Unit Activities Use Target-Method Match template to determine proper assessment strategies for learning targets Audit existing assessment(s) for clear purpose Audit existing assessment(s) for clear learning targets Sunday, October 13, 13
  13. 13. Assessment Methods Selected response Multiple choice True/False Matching Fill-in-the-blank questions Sunday, October 13, 13
  14. 14. When do we need Selected Response? Use selected response assessments when... Broad topic requiring wide-range coverage Diagnose student misconceptions and flaws in reasoning Students can read/understand language well enough to know what is being asked Sunday, October 13, 13
  15. 15. Choosing Selected Response Types Item Used when... Advantage Limitations MultipleChoice 1 correct/several plausible Variety; Easy score; Coverage; Can diagnose Guessing True/False Large Content/Many items Time; Easy score Trivial/misleading; Guessing Matching Related thoughts/ facts; Association Coverage; Several MC in one (others are distractors) Elimination Fill-in-theBlank Sunday, October 13, 13 Clear/short answer; Response Answer known rather production; Reduced than picked from list guessing; Coverage Scoring time
  16. 16. Writing Quality Items KISS - Keep wording simple and focused. Aim for the lowest possible reading level. Ask a full question in the stem. Eliminate clues to the correct answer either within the question or across questions within a test. Do not make the correct answer obvious to students who have not studied the material. Highlight critical, easily overlooked words. Have a qualified colleague read your items to ensure their appropriateness. Double-check scoring key for accuracy before scoring. Sunday, October 13, 13
  17. 17. Multiple-Choice Ask a complete question to the the item started, if you can. Don’t repeat the same words within each response option; rather, reword the item stem to remove the repetitive material from below. Be sure there is only one correct or best answer. Choose distractors carefully. Word response options as briefly as possible and be sure they are grammatically parallel. Make all response options the same length. Don’t use “all of the above” or “none of the above” merely to fill space. Use “always” or “never” in your answer choices with caution. It’s okay to vary the number of response options presented as appropriate to pose the problem you want your students to solve Sunday, October 13, 13
  18. 18. True/False Make the entire item entirely true or entirely false!! Sunday, October 13, 13
  19. 19. Matching Provide clear directions for making the match. Keep the list of things to be matched short. Keep the list of things to be matched homogenous. Keep the list of response options brief in their wording and parallel in construction. Include more response options than stems and permit students to use response options more than once when appropriate. Sunday, October 13, 13
  20. 20. Fill-in-the-Blank Ask respondents a question and provide space for an answer. Try to stick to one blank per item. Don’t let the length of the line to be filled in be a clue as to the length or nature of the correct response. Put the blank toward the end of the sentence. Sunday, October 13, 13
  21. 21. Review for Quality Match to blueprint List/number targets to be covered On test...mark each item with learning target number and points worth Tally the points and write on target list Compare with blueprint Sunday, October 13, 13
  22. 22. Review for Quality Does the item test what you intended? Is the item well-written? Sunday, October 13, 13
  23. 23. Activity 5.2 Selected Response Test Quality Checklist Classroom Assessment for Student Learning: Doing It Right—Using It Well, 2e Learning Teams Facilitation Guide and Activities & Resources Copyright © 2012 by Pearson Education, Inc., All rights reserved. Sunday, October 13, 13 1
  24. 24. Reviewing/Analyzing Results Problem # Sunday, October 13, 13 Learning Target # Confident Unsure Right Wrong Simple Mistake Don’t Get It
  25. 25. Activities for Selected Response Assessment Develop an Assessment for Learning Activity Prepare a Quiz or Test for Formative Use Sunday, October 13, 13
  26. 26. Assessment Methods Written response Short answer items Extended written response items Sunday, October 13, 13
  27. 27. Written Response Use when... Students are capable of writing in English (or desired language) Less time-efficient methods are not plausible Scoring guides are of high quality (consistency) Learning targets are Knowledge or Reasoning Sunday, October 13, 13
  28. 28. Sample Size Prioritization should parallel the amount of time and emphasis give the various targets or topics in teaching. If standards referenced then point distributed evenly. Sunday, October 13, 13
  29. 29. Written Assessment Blueprint Learning Target Sunday, October 13, 13 Points
  30. 30. Written Assessment Blueprint Pattern of Reasoning Content Target 1 Target 2 Target 3 TOTAL Sunday, October 13, 13 Know Compare Evaluate Total
  31. 31. Devising Extended Written Response Items Knowledge Mastery: Assess factual and conceptual knowledge Knowledge Mastery AND Reasoning: Use factual and conceptual knowledge to form wellreasoned logical argument Interpretive: Assess mastery of specific reasoning patterns disentangled from prerequisite knowledge Sunday, October 13, 13
  32. 32. Devising Extended Written Response Items Knowledge Mastery Set the Context Tell what to describe or explain Point the way to an appropriate response Sunday, October 13, 13
  33. 33. Devising Extended Written Response Items Knowledge Mastery AND Reasoning Set the Context Describe the reasoning task Point the way to an appropriate response Sunday, October 13, 13
  34. 34. Devising Extended Written Response Items Interpretive Set the Context Describe the reasoning task Point the way to an appropriate response Sunday, October 13, 13
  35. 35. Scoring Guides Scoring List - Simply lists required tasks and the point allotment for each task Scoring Rubric - Rates responses according to predetermined hierarchy of quality Task-specific - Rates each task General - Rates the whole response Sunday, October 13, 13
  36. 36. Creating Task-Specific Rubric Begin with correct statement of conceptual understanding Identify characteristics of partial understanding Identify characteristics of misunderstanding Determine levels of rubric Sunday, October 13, 13
  37. 37. Creating General Rubric Same as task-specific rubric, but in place of content-specific statements, use general descriptions Sunday, October 13, 13
  38. 38. Quality Control Can you write an outline of a high-quality response? Have a qualified colleague write a response and discuss the aspects of a high-quality response. Sunday, October 13, 13
  39. 39. Sunday, October 13, 13
  40. 40. Activities for Written Response Assessment Create a Short Answer Item and Scoring Guide Create an Extended Written Response Item and Scoring Guide Sunday, October 13, 13
  41. 41. Assessment Methods Performance assessment Performance task Performance criteria Sunday, October 13, 13
  42. 42. Performance Assessment Use it when... Assessing skills, products, and some forms of reasoning Working with young/primary students, or students who cannot read/write in English (or desired language You cannot get the information you need through written assessments Sunday, October 13, 13
  43. 43. Determining Sample Size Complexity of the Target: Complex = Larger; Simple = Smaller Decision the Evidence will Inform: How do you intend to use the results? Consistency of Performance: Can they do it or did they just get lucky? Proximity to Cutoff Mark: Regression to the mean Sunday, October 13, 13
  44. 44. Characteristics of a Good Task Content Target Alignment (does the task hit the mark) Authenticity (realistic context) Choice (all options are equivalent) Scaffolding (points to success without overhelping) Interference (does not depend on unrelated or cultural background) Availability of Resources (do they have what they need) Sunday, October 13, 13
  45. 45. Characteristics of a Good Task Structure Knowledge students are to use (What should they know?) What students are to accomplish (What should they do with what they know?) Performance or product students are to create (What does the finished product look like?) Materials to be used (What materials should they use?) Timeline for completion (How long will they have?) Conditions (What will the conditions of the performance be?) Help allowed (Who can help and how?) Criteria (What will be the focus of the assessment?) Sunday, October 13, 13
  46. 46. Characteristics of a Good Task Sampling Use of information (How many tasks assigned? Does the task sample adequately?) Coverage of the Target (Does the task match the target in terms of breadth?) Sunday, October 13, 13
  47. 47. Characteristics of a SEE P241 Good Task Activity 7.5 Rubric for Rubrics Sunday, October 13, 13 1
  48. 48. Tasks to Elicit Good Writing RAFTS Role: What is my role? Audience: Who is my audience? Format: What is the appropriate format? Topic: What is the topic? Strong verb: Strong verbs set the tone of a piece (see p220) Sunday, October 13, 13
  49. 49. Tasks to Elicit Good Writing Strong verbs Role: What is my role? Audience: Who is my audience? Format: What is the appropriate format? Topic: What is the topic? Strong verb: Strong verbs set the tone of a piece Sunday, October 13, 13
  50. 50. Quality Control Activity 7.4 Rubric for Tasks Sunday, October 13, 13 FULL CHART P222 1
  51. 51. Structure of a Rubric Criteria Content Organization Delivery Language Use Indicators (for each Criterion) Content Organization Delivery Language Use Sunday, October 13, 13
  52. 52. Structure of a Rubric Content Clear main topic All information important to understanding Facts, details, anecdotes, and/or examples make topic come alive Sunday, October 13, 13
  53. 53. Structure of a Rubric Organization Opening introduces topic and catches attention Sequence of ideas supports meaning and is easy to follow Transition words guide audience Conclusion wraps up topic and leaves audience satisfied Sunday, October 13, 13
  54. 54. Structure of a Rubric Delivery Eye contact Voice Articulation Pace No fillers Gestures/movements (where appropriate) Notes as reminders Visual aids Sunday, October 13, 13
  55. 55. Structure of a Rubric Organization Opening introduces topic and catches attention Sequence of ideas supports meaning and is easy to follow Transition words guide audience Conclusion wraps up topic and leaves audience satisfied Sunday, October 13, 13
  56. 56. Process for Developing a Rubric Establish a knowledge base Gather samples of student performances or products Sort student work by level of quality Cluster descriptors into traits Identify samples that illustrate each level Test the rubric and revise as needed Sunday, October 13, 13
  57. 57. 7 Strategies for Using Rubrics as Instructional Tools in the Classroom Where am I going? 1. Provide students with a clear and understandable vision of the learning target 2. Use examples and models of strong and weak work Where am I now? 3. Offer regular descriptive feedback 4. Teach students to self-assess and set goals How can I close the gap? 5. Design lessons to focus on one learning target or aspect of quality at a time 6. Teach students focused revision 7. Engage students in self-reflection and let them keep track of and share their learning Sunday, October 13, 13
  58. 58. Activities for Performance Task Create a Rubric Evaluate a Rubric for Quality Create a Student-friendly Version of a Rubric Sunday, October 13, 13
  59. 59. Assessment Methods Personal communication Questions during instruction Interviews and conferences Participation Oral exams Student journals and logs Sunday, October 13, 13

×