Games as Formative Assessment Environments

  • 7,996 views
Uploaded on

G4LI Games for Learning Day at G4C 2011

G4LI Games for Learning Day at G4C 2011

More in: Design , Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
7,996
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
8
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • At CRESST, we’ve been using the approach of creating an assessment architecture in our assessment design, and we employ the same process for the design of games. Now what is this assessment architecture? It is basically the blueprint for building an environment that is designed to help students learn. It is made up of three parts: Eva Baker’s model-based assessment approach calla for domain-independent descriptions of the relevant aspects to learning such as content understanding, problem solving, metacognition, communication, and teamwork and collaboration.
  • The scored events are chosen because they capture the targeted math knowledge and game knowledge… Choosing the coil size requires that students understand the idea of a unit, how it is denoted, and how to determine the size of the interval/fraction
  • The act of adding coils represent understanding of how quantities can be added together.
  • Reaching the goal represents successful culmination of all of the targeted knowledge.
  • Research tells us that the criteria of performance can be communicated by the external representations of learning objectives, in traditional learning environments: Rubrics/Standards Game environments: Explanations of scoring rules
  • The first hypothesis I wanted to test is whether or not externally representing both the learning objectives and the assessment criteria by giving students explanation of the scoring rules in the tutorials, in the help menu, and as part of the feedback.
  • The second aspect of my study addressed the idea that both social and game norms that aret to be valued is communicated by
  • In terms of the within-game outcomes The combination of the explanation and incentive was superior, with those students -adding fewer coils with different denominators Having fewer number of resets
  • In terms of the within-game outcomes The combination of the explanation and incentive was superior, with those students -adding fewer coils with different denominators Having fewer number of resets

Transcript

  • 1. Presentation at the 8 th Annual Games for Change Festival New York, New York—June 22, 2011 Girlie C. Delacruz Games as Formative Assessment Environments: Making Assessment Criteria Explicit and Incentivizing Use of Feedback
  • 2. Background
    • Games as context for formative assessment
    • Two issues
      • Make assessment criteria explicit
      • Motivate student use of instructional feedback and help
  • 3. What is Formative Assessment? Formative Assessment: Use and interpretation of task performance information with intent to adapt learning, such as provide feedback. (Baker, 1974; Scriven, 1967)
  • 4. Same Concept, Different Context Formative Assessment: Use and interpretation of task performance information with intent to adapt learning, such as provide feedback. (Baker, 1974; Scriven, 1967) Games: Use and interpretation of game performance information with intent to adapt learning, such as provide feedback.
  • 5. Assessment Architecture: General Model / 27
  • 6. Assessment Architecture: Save Patch / 27
  • 7. The Game: Save Patch
    • The size of a fraction is relative to a whole unit
  • 8. The Game: Save Patch
    • Like whole number integers, fractions with like denominators can be added together to produce a given quantity
  • 9. Scored Events: Choosing the Coil Size Math Knowledge Required Game Knowledge Required In mathematics, one unit is understood to be one of some quantity (intervals, areas, volumes, etc.). In our number system, the unit can be represented as one whole interval on a number line. Positive integers are represented by successive whole intervals on the positive side of zero. The interval between each integer is constant once it is established. The vertical red bars denote the whole unit. Positive non-integers are represented by fractional parts of the interval between whole numbers. Grid: The spaces between the green dots are the parts of the whole unit. Coil: The coil pieces are parts of a whole unit coil. The denominator of a fraction represents the number of identical parts in one whole unit. That is, if we break the one whole unit into “x” pieces, each piece will be “1/x” of the one whole unit. Grid: The number of spaces between the green dots is the denominator. Coil: The number of coil pieces the whole unit is broken into is the denominator.
  • 10. Scored Events: Adding Coils Math Knowledge Required Game Knowledge Required Only identical (common) units can be added to create a single numerical sum. If given different coils with different units, the coils must be changed so that they are the same unit before they can be added together.
  • 11. Scored Events: Patch Reaches the Goal Math Knowledge Required Game Knowledge Required Positive integers can be broken (decomposed) into parts that are each one unit in quantity. The length of the jump is the number of pieces between the blocks. All rational numbers can be represented as additions of integers or fractions. To add quantities, the units (or parts of units) must be identical. Identical (common) units can be added to create a single numerical sum. Add the correct number of coils that match the length of the jump. The numerator of a fraction represents the number of identical parts that have been combined. For example, ¾ means three pieces that are each ¼ of one whole unit. Grid: The top number of the jump distance equals the total number of spaces to jump over. Coils: The top number of the sum of the coil pieces on the trampoline represents the number of coil pieces that have been added together.
  • 12. Making Assessment Criteria Explicit
  • 13. Hypothesis A
    • Externally representing the learning objectives and the criteria by which students will be assessed can support learning
  • 14. Incentivizing Use of Feedback
  • 15. Hypothesis B
    • Incentivizing the use of additional feedback will have a positive impact on math achievement measures, game performance measures, and use of feedback
  • 16. Research Questions
    • Are there differential effects based on the amount of explanation of the scoring rules provided and incentivizing the use of feedback on
      • Math achievement performance
      • Game performance measures
      • Access of feedback help
  • 17. Full Explanation of Scoring Rules Click here for help
  • 18. Scoring Rules Explanation Plus Incentive To earn back 50 points, click here to get help on choosing the right denominator!
  • 19. Points-Only Feedback Click here for help Click here for help
  • 20. No Scoring Rule Information Click here for help
  • 21. Example of Feedback
  • 22. Independent Variables
  • 23. Dependent Measures
    • Math Achievement:
      • Pretest (31 items)
      • Posttest (44 items, additional items in game context)
    • Game Performance Measures
      • Number of coils added together, wrong-sized unit coils, resets, and failed attempts
      • Maximum level reached
    • Help-Seeking Behavior
      • Proportion of times feedback was accessed
      • Accessing general help menu
  • 24. Methodology
    • Participants
      • Data collected from 112 4th-6th grade students
      • After-school context
    • Treatment
      • 20-minute interval
      • Randomly assigned to treatment conditions
    • Procedure
      • Pretest  Game play  Posttest and Survey
  • 25. Math Outcomes: Overall Sample
    • Incentive + Scoring Explanation condition had higher normalized change scores ( d = .53)
    • Interaction between pretest scores and treatment variation on game-like item scores
      • For students with:
        • low prior knowledge, Incentive + Scoring Explanation more beneficial
        • high prior knowledge, just the Scoring Explanation was more effective
  • 26. Math Outcomes: Low Academic Motivation
    • For students with low self-efficacy:
      • Both (1) Incentive + Scoring Explanation and (2) Incentive + Scoring Explanation had higher posttest scores than minimal scoring rules information ( d = .88)
    • For students with low game experience, low math self-concept, and low preference for cooperative learning:
      • Incentive + Scoring Explanation resulted in higher change scores ( d = .88 – 1.17)
  • 27. Game Outcomes
    • Incentive + Scoring Explanation was superior to minimal scoring information for the within-game outcomes
      • Added fewer coils with different denominators
      • Had lower number of resets
  • 28. Use of Feedback by Incentive Group
    • Accessed feedback less overall and spent less time on feedback screen
    • However… accessed the same topic in general help menu more often, and when feedback was accessed, solved the level more quickly and with fewer mistakes
  • 29. Summary: Making Assessment Criteria Explicit
    • Providing just the scoring rules explanation resulted in better game performance
    • Providing just the explanation of scoring rules did not lead to better performance on math outcomes
      • Except for students with low self-efficacy
  • 30. Summary: Incentivizing Use of Feedback
    • Incentive + Scoring Information is superior to minimal scoring information, with better performance on:
      • Math achievement measures
      • Game play
  • 31. Next Steps: Theoretical Explanations
    • Examine cognitive and motivational processes that underlie relation between incentivizing use of feedback and learning
    Cognitive Engagement: Deeper processing of information Utility Value: Increases perceived value of feedback What is incentivized, rewarded or punished signals the norms that are valued or to be avoided Accesses feedback Achievement: Improves performance
  • 32. Next Steps: Game Design Studies
    • Feedback modifications
      • Increase opportunities to receive treatments
        • Prior to re-attempting solution rather than after a mistake
      • Provide feedback on a randomized schedule
    • Incentive modifications
      • Make incentives more relevant to the game
      • Rewards for performance rather than contingent on behavior
  • 33. CONTACT INFORMATION cats.cse.ucla.edu Girlie Delacruz [email_address]