Formative evaluation is used to improve instructional materials and presentations by collecting feedback from subject matter experts, learners, and others. The document outlines various stages of formative evaluation including one-on-one sessions, small group evaluations, and field trials. Data is collected through tests, questionnaires, observations, and interviews to identify errors, understand what is working well and identify opportunities for improvement before finalizing and distributing materials. The goal is to refine materials through an iterative design process incorporating user feedback.

1. Designing and
Conducting Formative
Evaluations
Alyssa Singleterry

2. Background
 Instructional materials have not always been effective.
Studies have shown that thousands of the instructional
materials sold in the US each year have not been evaluated
with learners and revised prior to distribution. Today, the idea
of formative evaluation is in the use of a variety of settings
and in these applications the idea of evaluating learners’
performance and using resulting data to improve the
instruction and curriculum.

3. Objectives
 Describe the purpose for the various stages of formative
evaluation of instructor-developed materials, instructor-
selected materials, and instructor-presented instruction
 Describe the instruments used in a formative evaluation
 Develop an appropriate formative evaluation plan and
construct instruments for a set of instructional materials or an
instructor presentation
 Collect data according to a formative evaluation plan for a
given set of instructional materials or instructor presentation

4. Formative
Evaluation
Designs
 Pinpoint specific errors in materials in order to correct them
 Instructional strategy as framework
 Five questions related to decisions made
 Types of data you will want to collect

5. Formative
Evaluation
Designs
Five Questions
 Are materials appropriate for
type of learning outcomes?
 Do materials include
adequate instruction on
subordinate skills and are
sequenced logically?
 Are materials clear and
understood by target group?
 What is the motivational
value of materials?
 Can materials be managed
efficiently in the manner they
are mediated?
Types of Data
 Reactions of subject-matter
expert
 Reactions of manager or
supervisor
 Test data collected on entry
skills, pretests and posttests
 Comments or notations
made by learners
 Data collected on attitude
questionnaires
 Time required for learners to
complete instruction

6. Role of
Subject-
Matter,
Learning, and
Learner
Specialists in
Formative
Evaluation
 Instruction reviewed by specialists
 “Forest and trees” problem
 Subject-Matter Expert (SME)
 comment on accuracy and currency of instruction
 Designer is not obligated to use suggestions made
 Designer is sensitized to potential problems

7. One-to-One
Evaluation
with Learners
 Identify and remove errors in instruction
 Direct interaction between designer and individual learner
 Designer works with 3 or more learners representing target
population

8. Continued
 Three main criteria: clarity, impact, and feasibility
 Help verify whether designers’ and developers’ hunches were
correct

9. Selecting
Learners
 Most critical decisions by
the designer
 Designer works on
individual basis with learner

10. Small-Group
Evaluation
 Determine effectiveness of changes made following one-to-
one
 Identify remaining learning problems
 Determine whether learners can use instruction
 Instructor should intervene as little as possible

11. Field Trial
 Determine whether changes in institution were effective
 See whether instruction can be used which it was intended
 Desired levels of learner achievement and attitudes are
reached

12. Formative
Evaluation in
the
Performance
Context
 Determine whether skills that have been taught are used in
performance context
 Questionnaire can be used to assess learners’ perceptions of
training outcomes
 Determine strengths and weaknesses in instruction

13. Formative
Evaluation of
Selected
Materials
 Preparations for field trial of existing materials should be
made as they would for a field trial of original materials
 Analysis should be made of existing documentation on
development of materials

14. Formative
Evaluation of
Instructor-Led
Instruction
 In preparing for field trial of
instructor-led instruction,
instructor should be
concerned with entry skills
and prior knowledge

15. Data
Collection for
Selected
Materials and
Instructor-Led
Instruction
 Much of the information dealing with collection of data in field
trial of original instructional materials applies equally will to
data collection procedures used in evaluation of selected
materials and instructional procedures

16. Concerns
Influencing
Formative
Evaluation
 Formative evaluation component distinguishes instructional
design process from a theosophical approach
 Ensure any technical equipment is operating effectively

17. Problem
Solving
during
Instructional
Design
 Designer often faced with questions that are best answered with
data
 Designer must develop two versions of instruction to use in small
group evaluation.

18. Continued
 Purpose is to make decision about what to do with a particular unit
of instruction
 Designer could collect enough data in formative research about
illustrations to make a tentative decision about continued use of
instruction

19. Summary
 Formative evaluation instructional materials is conducted to
determine the effectiveness of the materials and revise them
in areas where they are ineffective. Formative evaluations
should be conducted on newly developed materials as well as
existing materials that are selected based on the instructional
strategy

20. Contact
Information
 Alyssa Singleterry
 asingletery4436@myasu.alasu.edu