Formative evaluation is used to improve instructional materials and presentations by collecting feedback from subject matter experts, learners, and others. The document outlines various stages of formative evaluation including one-on-one sessions, small group evaluations, and field trials. Data is collected through tests, questionnaires, observations, and interviews to identify errors, understand what is working well and identify opportunities for improvement before finalizing and distributing materials. The goal is to refine materials through an iterative design process incorporating user feedback.
2. Background
Instructional materials have not always been effective.
Studies have shown that thousands of the instructional
materials sold in the US each year have not been evaluated
with learners and revised prior to distribution. Today, the idea
of formative evaluation is in the use of a variety of settings
and in these applications the idea of evaluating learners’
performance and using resulting data to improve the
instruction and curriculum.
3. Objectives
Describe the purpose for the various stages of formative
evaluation of instructor-developed materials, instructor-
selected materials, and instructor-presented instruction
Describe the instruments used in a formative evaluation
Develop an appropriate formative evaluation plan and
construct instruments for a set of instructional materials or an
instructor presentation
Collect data according to a formative evaluation plan for a
given set of instructional materials or instructor presentation
4. Formative
Evaluation
Designs
Pinpoint specific errors in materials in order to correct them
Instructional strategy as framework
Five questions related to decisions made
Types of data you will want to collect
5. Formative
Evaluation
Designs
Five Questions
Are materials appropriate for
type of learning outcomes?
Do materials include
adequate instruction on
subordinate skills and are
sequenced logically?
Are materials clear and
understood by target group?
What is the motivational
value of materials?
Can materials be managed
efficiently in the manner they
are mediated?
Types of Data
Reactions of subject-matter
expert
Reactions of manager or
supervisor
Test data collected on entry
skills, pretests and posttests
Comments or notations
made by learners
Data collected on attitude
questionnaires
Time required for learners to
complete instruction
6. Role of
Subject-
Matter,
Learning, and
Learner
Specialists in
Formative
Evaluation
Instruction reviewed by specialists
“Forest and trees” problem
Subject-Matter Expert (SME)
comment on accuracy and currency of instruction
Designer is not obligated to use suggestions made
Designer is sensitized to potential problems
7. One-to-One
Evaluation
with Learners
Identify and remove errors in instruction
Direct interaction between designer and individual learner
Designer works with 3 or more learners representing target
population
8. Continued
Three main criteria: clarity, impact, and feasibility
Help verify whether designers’ and developers’ hunches were
correct
10. Small-Group
Evaluation
Determine effectiveness of changes made following one-to-
one
Identify remaining learning problems
Determine whether learners can use instruction
Instructor should intervene as little as possible
11. Field Trial
Determine whether changes in institution were effective
See whether instruction can be used which it was intended
Desired levels of learner achievement and attitudes are
reached
12. Formative
Evaluation in
the
Performance
Context
Determine whether skills that have been taught are used in
performance context
Questionnaire can be used to assess learners’ perceptions of
training outcomes
Determine strengths and weaknesses in instruction
13. Formative
Evaluation of
Selected
Materials
Preparations for field trial of existing materials should be
made as they would for a field trial of original materials
Analysis should be made of existing documentation on
development of materials
15. Data
Collection for
Selected
Materials and
Instructor-Led
Instruction
Much of the information dealing with collection of data in field
trial of original instructional materials applies equally will to
data collection procedures used in evaluation of selected
materials and instructional procedures
18. Continued
Purpose is to make decision about what to do with a particular unit
of instruction
Designer could collect enough data in formative research about
illustrations to make a tentative decision about continued use of
instruction
19. Summary
Formative evaluation instructional materials is conducted to
determine the effectiveness of the materials and revise them
in areas where they are ineffective. Formative evaluations
should be conducted on newly developed materials as well as
existing materials that are selected based on the instructional
strategy
