The document discusses retrospective pretests, which are used to measure learning and performance. A retrospective pretest asks participants to recall their behavior or knowledge before an intervention by having them rate themselves on a scale after the intervention has occurred. This helps address response-shift bias by distinguishing the pretest from the intervention. Retrospective pretests work well when participants don't know what they don't know, there is insufficient time for a traditional pretest, or no prior benchmarking was performed. They provide a more accurate measure of actual learning by drawing on a subject's memory of their pre-intervention state.

1. Learning and Performance through InnovatioLearning and Performance through Innovatio1
RetrospectiveRetrospective
Analysis:Analysis:
A PrimerA Primer
Jeff M. AllenJeff M. Allen

2. Learning and Performance through InnovationLearning and Performance through Innovation2
When is the last time
you learned a
newnew
research
design?

3. Learning and Performance through InnovationLearning and Performance through Innovation3
Retrospective PretestRetrospective Pretest

Campbell and Stanley (1963)
offered retrospective pretest as a
tool to enhance experimental
designs.

Used and promoted in the field of
behavioral science research, since
the 1950s as a moderator for the
threat of response-shift bias
(post-then, pre-post, then)

4. Learning and Performance through InnovationLearning and Performance through Innovation4
Solution: Retrospective PretestSolution: Retrospective Pretest

Howard (1980) began prescribing
this research tool as a remedy for
response-shift bias.

Distinguishes itself from the
traditional pretest by its
relationship to the intervention.

5. Learning and Performance through InnovatioLearning and Performance through Innovatio5
When and HowWhen and How

6. Learning and Performance through InnovationLearning and Performance through Innovation6
When does it work?When does it work?

Learners don’t know …
what they don’t know

There is insufficient time for pre-test.

There as been no benchmarking
performed.

7. Learning and Performance through InnovationLearning and Performance through Innovation7
Why does it work?Why does it work?

Administered post-intervention,
asking the subjects to recall
behavior

Can provide a more accurate
measure of actual learning

Can provide more differentiation
in learning (instrument scaling)

8. Learning and Performance through InnovationLearning and Performance through Innovation8
Early ExamplesEarly Examples
1951

Effects of racially mixed housing on
prejudice
1956

Measurement of fear

9. Learning and Performance through InnovationLearning and Performance through Innovation9
Strengths LimitationsStrengths Limitations

Detects response-
shift effect

Extends posttest only
designs

Can curb threats to
history, selective
mortality, and shift in
subject perceptions

Draws on a subject’s
memory

Best used when
added to other
designs

Inappropriate
utilization for skill
measurement

10. Learning and Performance through InnovationLearning and Performance through Innovation10
Retrospective PretestRetrospective Pretest
Law Firm

11. Learning and Performance through InnovationLearning and Performance through Innovation11
Retrospective PretestRetrospective Pretest
Professional Development

12. Learning and Performance through InnovationLearning and Performance through Innovation12
Benchmark

13. Learning and Performance through InnovationLearning and Performance through Innovation13
ScalingScaling
PrePre
    
PostPost
 
ThenThen
    
NowNow
    

14. Learning and Performance through InnovatioLearning and Performance through Innovatio14
RetrospectiveRetrospective
Analysis:Analysis:
A PrimerA Primer
Jeff M. AllenJeff M. Allen