Assessment literacy 2007

981 views

Published on

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
981
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Sub-set is like a political survey / you only ask 250 people instead 25000
  • Assessment literacy 2007

    1. 1. Basic A ssessment Literacy Everything You W anted to Know A bout Testing But W as N ever T hat Interested
    2. 2. T est builders in empowered desktopFocus just on MC & D BQ s
    3. 3. Smartest kid in class
    4. 4. Sticky Ideas
    5. 5. Q uality data requires a quality testQ uality tests are not easy to create
    6. 6. A ssessment literacy means:5 ld e & n B Mn a in E pat l hn e ?
    7. 7. A ssessment literacy means:G t rg eedb in r a n b u a e dpnal f mt aot h in e o io s dn ah vmn t etciee et uU in t a es et rcst po oe sgh s s n poeso rm t e s m mx u ah vmn aimm ciee et
    8. 8. W hat’s a “ good test?”“ The most basic and obvious answer to that question is that good tests measure what you want to measure, and bad tests do not.” Ben C lay Is This a Trick Question?
    9. 9. Q uestions to ask W hy? W hat? U sers/U ses H ow? T argets Method H ow Much? Sample A ccurate? A void Bias
    10. 10. W hy? W hat?U sers/U ses H ow? T argets Method H ow Much? Sample A ccurate? A void Bias
    11. 11. W hy?Is the Purpose for the assessment clear? oW hy are these targets being assessed? oW ho will use the information? oW hat will the information be used for?
    12. 12. W hy? W hat?U sers/U ses H ow? T argets Method H ow Much? Sample A ccurate? A void Bias
    13. 13. W hat?A re the Targets clear and appropriate? oSpecific & clearly defined expectations oC enter on the core of the discipline oA ligned to program / course / unit level curriculum
    14. 14. Four target typesKnowledge oG eorge W ashington / 1787 / U S C onstitutionReasoning oEvaluatedifferences between government types
    15. 15. Skills oA nalyze a primary source documentProducts oC reate a reasoned argument for allowing immigration reform
    16. 16. A n assessment that accurately reflects its intended target is Valid
    17. 17. W hy? W hat?U sers/U ses H ow? T argets Method H ow Much? Sample A ccurate? A void Bias
    18. 18. H ow?D ifferent targets require different methods oMatch appropriate method with intended purpose and target
    19. 19. G eneral categoriesO bjective oStudent selects correct response from several alternatives / supplies a word or short phrase to answer a question oEasy to grade / hard to create
    20. 20. Subjective oStudent organizes / presents an original answer oH ard to grade / easy to create
    21. 21. O bjective vs. Subjective o“ Bydefinition, no test can be truly objective”
    22. 22. Specific categoriesSelected Response oMultipleC hoice, T rue/False, Matching, Fill-in-the-blankEssays oShort A nswer, Extended ResponsePerformance T asksPersonal C ommunication
    23. 23. Mark Pett ©2001
    24. 24. Let’s try it!N ame the parts of the human skeleton oSelected responseA ppraise a composition on the basis of its organization oShort answer essayD emonstrate safe lab skills
    25. 25. C ite four examples of satire in Huckleberry Finn oShort answer essayD esign a logo for a web page oPerformance taskD escribe the impact of a Bull Market oExtended response essay
    26. 26. D iagnose a physical ailment oExtended response essayList important mental attributes necessary in an athlete oShort response essayC ategorize great A merican fiction writers
    27. 27. W hy? W hat?U sers/U ses H ow? T argets Method H ow Much? Sample A ccurate? A void Bias
    28. 28. H ow much?H ow can we Sample appropriately? o“ A ll assessments are just a small part of an ‘ideal’ assessment of infinite length.”
    29. 29. “ H ave I gathered enough information of the right kind so that I can draw confident conclusions about student achievement?”
    30. 30. W hy? W hat?U sers/U ses H ow? T argets Method H ow Much? Sample A ccurate? A void Bias
    31. 31. Mark Pett ©2001
    32. 32. A ccurate?W hat are possible sources of Interference or Bias?
    33. 33. A ssessment oPoorly written, poor target-method match, culturally biased, inappropriate reading levelA ssessor oIncorrectly scored
    34. 34. Student oTest anxiety, cheat, guess, interpretation different than intentEnvironment oC old, hot, poor light, time restraints, noise, lunch, recess
    35. 35. Eliminating errors due to bias increase the Reliability of our assessments
    36. 36. W hat was new?W hat was re-new?
    37. 37. Steps to Structured ResponsetestPrepare a blueprintSelect material to assessBuild test items from propositions
    38. 38. Step 1Prepare a blueprint
    39. 39. Cnn ot t e Ko & nw Poes rcs& Tt oa l Tr t ae gs U drad ne t sn Ap a n pl t ic ioFr s fom o 1 0 5 1 5gvrmnoe et nS ut e f St c r oU r u 7 5 1 2gvrmnoe et nRh & igts 8 5 1 3r p nibitse os il s ieo cizn fitesTt oa l 2 5 1 5 4 0
    40. 40. Step 2Select material to assess
    41. 41. “ It is always tempting to emphasize the parts of the course that are easiest to test, rather than the parts that are important to test.” Ben C lay Is This a Trick Question?
    42. 42. So what’s worthassessing?
    43. 43. Review the C ase StudyW hich test items, if any, need to be changed?C an you think of a test that included items you felt weren’t worth learning or assessing?H ow do you make decisions about what’s worth assessing?
    44. 44. So . . . ?A select sub-set of all possible important elementsProfessional judgment of relative importance
    45. 45. Step 3Build test items from propositions o“ C learlystated sentences that reflect elements of content aligned to C ourse / U nit EU s & EQ s.” oC reate more than blueprint requires
    46. 46. ExamplesIn a monarchy, the right to govern is secured through birth.The executive and legislative branches differ in that the latter is elected directly by the people.
    47. 47. T rue or FalseIn a monarchy, the right to govern is secured through birth.orIn a monarchy, the right to govern is secured through the approval of those governed.
    48. 48. Fill in the BlankH ow is the right to govern secured in a monarchy?
    49. 49. Multiple C hoiceH ow is the right to govern secured in a monarchy? A . W ith military power B. T hrough birth C . By popular vote D . Through purchase
    50. 50. T rue or FalseThe executive and legislative branches differ in that the latter is elected directly by the people.orMembers of executive and legislative branches are both elected directly by the people.
    51. 51. Fill in the BlankElection of members of the executive and legislative branches differ in what way?
    52. 52. Multiple C hoiceElection of members of the executive and legislative branches differ in what way? A . Legislators are restricted by term limits; presidents are not B. Legislators are elected directly; presidents are not. C . O ne must register to vote for legislators; not for president D . O ne must register to vote for presidents;
    53. 53. Franzipanics Q uiz
    54. 54. Specific assessment criteria
    55. 55. Multiple choiceA dvantages oEffectivemeasurement oReduces guessing oC an cover wide range of contentD isadvantages oD ifficult to create
    56. 56. Multiple C hoice suggestionsStem is a clear, explicit & singular questionU se at least four alternativesA void grammatical / sentence structural cuesH ighlight the negative in negatively stated stems
    57. 57. A lternatives are the same format and lengthA ll alternatives should be plausibleA void answer patterns
    58. 58. Is the question any good?Student who knows the content will know the answer before reading the alternatives
    59. 59. “ Your goal is not to trick students or require them to make difficult judgments about two nearly equal options.”
    60. 60. “ Your goal is to write questions that students who understand the material will answer correctly and students who do not understand will answer incorrectly.” D r. John Johnson Penn State U niversity
    61. 61. Bill W atterson ©1995
    62. 62. T rue or falseA dvantages oC anmeasure large amounts of basic knowledge oStudents can process many questions quicklyD isadvantages oG uessing works o
    63. 63. T rue & false suggestionsStatements are absolutely true or falseExpress item as simply & clearly as possibleMeasure only one idea per itemU se possible misconceptions as false items
    64. 64. MatchingA dvantages oMeasures large amounts of knowledge level content oU seful for factual content areasD isadvantages oC an be time consuming for students oD oes not measure higher learning
    65. 65. Matching suggestionsU se items that share same foundation of knowledgeA void grammatical cuesKeep stem list to10-15 itemsResponse list should be larger than stem list
    66. 66. Fill-in-the-BlankA dvantages oG ood for 5 W s and H oH elps minimize guessingD isadvantages oD ifficult to assess high levels of learning oMay have more than one correct
    67. 67. Fill-in-the-Blank suggestionsO mit only significant items from stemInsure only one correct answerA void grammatical cuesBlank at stem’s endShould count for more points
    68. 68. T hings to discussPossible accommodationsStudent evaluations of testPilot version / full versionW ho takes the testW ho administers the test

    ×