Games as Formative Assessment Environments

Presentation at the 8 th Annual Games for Change Festival New York, New York—June 22, 2011 Girlie C. Delacruz Games as Formative Assessment Environments: Making Assessment Criteria Explicit and Incentivizing Use of Feedback

Background
Games as context for formative assessment
Two issues
Make assessment criteria explicit
Motivate student use of instructional feedback and help

What is Formative Assessment? Formative Assessment: Use and interpretation of task performance information with intent to adapt learning, such as provide feedback. (Baker, 1974; Scriven, 1967)

Same Concept, Different Context Formative Assessment: Use and interpretation of task performance information with intent to adapt learning, such as provide feedback. (Baker, 1974; Scriven, 1967) Games: Use and interpretation of game performance information with intent to adapt learning, such as provide feedback.

Assessment Architecture: General Model / 27

Assessment Architecture: Save Patch / 27

The Game: Save Patch
The size of a fraction is relative to a whole unit

The Game: Save Patch
Like whole number integers, fractions with like denominators can be added together to produce a given quantity

Scored Events: Choosing the Coil Size Math Knowledge Required Game Knowledge Required In mathematics, one unit is understood to be one of some quantity (intervals, areas, volumes, etc.). In our number system, the unit can be represented as one whole interval on a number line. Positive integers are represented by successive whole intervals on the positive side of zero. The interval between each integer is constant once it is established. The vertical red bars denote the whole unit. Positive non-integers are represented by fractional parts of the interval between whole numbers. Grid: The spaces between the green dots are the parts of the whole unit. Coil: The coil pieces are parts of a whole unit coil. The denominator of a fraction represents the number of identical parts in one whole unit. That is, if we break the one whole unit into “x” pieces, each piece will be “1/x” of the one whole unit. Grid: The number of spaces between the green dots is the denominator. Coil: The number of coil pieces the whole unit is broken into is the denominator.

Scored Events: Adding Coils Math Knowledge Required Game Knowledge Required Only identical (common) units can be added to create a single numerical sum. If given different coils with different units, the coils must be changed so that they are the same unit before they can be added together.

Scored Events: Patch Reaches the Goal Math Knowledge Required Game Knowledge Required Positive integers can be broken (decomposed) into parts that are each one unit in quantity. The length of the jump is the number of pieces between the blocks. All rational numbers can be represented as additions of integers or fractions. To add quantities, the units (or parts of units) must be identical. Identical (common) units can be added to create a single numerical sum. Add the correct number of coils that match the length of the jump. The numerator of a fraction represents the number of identical parts that have been combined. For example, ¾ means three pieces that are each ¼ of one whole unit. Grid: The top number of the jump distance equals the total number of spaces to jump over. Coils: The top number of the sum of the coil pieces on the trampoline represents the number of coil pieces that have been added together.

Making Assessment Criteria Explicit

Hypothesis A
Externally representing the learning objectives and the criteria by which students will be assessed can support learning

Incentivizing Use of Feedback

Hypothesis B
Incentivizing the use of additional feedback will have a positive impact on math achievement measures, game performance measures, and use of feedback

Research Questions
Are there differential effects based on the amount of explanation of the scoring rules provided and incentivizing the use of feedback on
Math achievement performance
Game performance measures
Access of feedback help

Full Explanation of Scoring Rules Click here for help

Scoring Rules Explanation Plus Incentive To earn back 50 points, click here to get help on choosing the right denominator!

Points-Only Feedback Click here for help Click here for help

No Scoring Rule Information Click here for help

Example of Feedback

Independent Variables

Dependent Measures
Math Achievement:
Pretest (31 items)
Posttest (44 items, additional items in game context)
Game Performance Measures
Number of coils added together, wrong-sized unit coils, resets, and failed attempts
Maximum level reached
Help-Seeking Behavior
Proportion of times feedback was accessed
Accessing general help menu

Methodology
Participants
Data collected from 112 4th-6th grade students
After-school context
Treatment
20-minute interval
Randomly assigned to treatment conditions
Procedure
Pretest  Game play  Posttest and Survey

Math Outcomes: Overall Sample
Incentive + Scoring Explanation condition had higher normalized change scores ( d = .53)
Interaction between pretest scores and treatment variation on game-like item scores
For students with:
low prior knowledge, Incentive + Scoring Explanation more beneficial
high prior knowledge, just the Scoring Explanation was more effective

Math Outcomes: Low Academic Motivation
For students with low self-efficacy:
Both (1) Incentive + Scoring Explanation and (2) Incentive + Scoring Explanation had higher posttest scores than minimal scoring rules information ( d = .88)
For students with low game experience, low math self-concept, and low preference for cooperative learning:
Incentive + Scoring Explanation resulted in higher change scores ( d = .88 – 1.17)

Game Outcomes
Incentive + Scoring Explanation was superior to minimal scoring information for the within-game outcomes
Added fewer coils with different denominators
Had lower number of resets

Use of Feedback by Incentive Group
Accessed feedback less overall and spent less time on feedback screen
However… accessed the same topic in general help menu more often, and when feedback was accessed, solved the level more quickly and with fewer mistakes

Summary: Making Assessment Criteria Explicit
Providing just the scoring rules explanation resulted in better game performance
Providing just the explanation of scoring rules did not lead to better performance on math outcomes
Except for students with low self-efficacy

Summary: Incentivizing Use of Feedback
Incentive + Scoring Information is superior to minimal scoring information, with better performance on:
Math achievement measures
Game play

Next Steps: Theoretical Explanations
Examine cognitive and motivational processes that underlie relation between incentivizing use of feedback and learning
Cognitive Engagement: Deeper processing of information Utility Value: Increases perceived value of feedback What is incentivized, rewarded or punished signals the norms that are valued or to be avoided Accesses feedback Achievement: Improves performance

Next Steps: Game Design Studies
Feedback modifications
Increase opportunities to receive treatments
Prior to re-attempting solution rather than after a mistake
Provide feedback on a randomized schedule
Incentive modifications
Make incentives more relevant to the game
Rewards for performance rather than contingent on behavior

CONTACT INFORMATION cats.cse.ucla.edu Girlie Delacruz [email_address]

http://edte.ch	3740
http://flavors.me	493
http://www.scoop.it	139
http://feeds.feedburner.com	100
http://www.edte.ch	10
http://www.xfruits.com	3
http://paper.li	3
http://translate.googleusercontent.com	3

Games as Formative Assessment Environments

by Games for Learning Institute (G4LI) on Jul 27, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

8 Embeds 4,491

Statistics

Games as Formative Assessment Environments — Presentation Transcript