This presentation discusses formative evaluations and how to design and conduct them. Formative evaluations are used during instructional design to improve effectiveness and identify problems. They involve subject matter experts, learning specialists, and learners. One-to-one and small group evaluations identify obvious errors. Field trials test the instruction in its intended context. Data is collected to evaluate clarity, impact, feasibility, and whether objectives are met. Concerns that influence formative evaluations include the context, learners, and outcomes. Problem solving is used to improve weak units based on evaluation data.

1. DESIGNING AND
CONDUCTING
FORMATIVE
EVALUATIONS
Presented by Larry F. Cobb
April 30, 2014

2. BACKGROUND
 In this presentation, we will address formative evaluations, how
they should be used, and how to design and conduct them

3. OBJECTIVE
 To discuss and give clarity to the following subjects:
 Formative Evaluation Designs
 Role of Subject-Matter, Learning, and Learner Specialists in Formative Evaluation
 One-to-One Evaluation with Learners
 Small-Group Evaluation
 Field Trial
 Formative Evaluation in the Performance Context
 Formative Evaluation of Selected Materials
 Formative Evaluation of Instructor-Led instruction
 Data Collection for Selected Materials and Instructor-Led instruction
 Concerns Influencing Formative Evaluation
 Problem Solving during Instructional Design

4. DICK AND CAREY’S MODEL

5. WHAT IS FORMATIVE
EVALUATION?
 A formative evaluation is the collection of data and information
during the development of instruction that can be used to improve
the effectiveness of the instruction.
 The process is used by designers to make their instructions more
effective and effiecient.
 They were originally used as a process to improve instruction
after the first draft of instruction was developed.
 It was later found better to try out earlier components of the
design process, thereby avoiding a lot of problems that would
otherwise not be discovered until after the draft of the instruction
was complete.

6. FORMATIVE EVALUATION DESIGNS
 The evaluation design needs to yield information about the
location of and the reasons for any problems. This prevents the
evaluation that is too narrow or too broad.
 The instructional strategy is the framework for the design. The
components will help you create questions that will allow you to
evaluation instruction better.

7. FORMATIVE EVALUATION DESIGNS
CON’T
These are questions that can be asked to evaluate the components
of the instructional strategy:
1. Are the materials appropriate for the type of learning outcome?
2. Do the materials include adequate instruction on the subordinate skills, and
are these skills sequenced and clustered logically?
3. Are the materials clear and readily understood by representative members
of the target group?
4. What is the motivational value of the materials? Do the learners find the
materials relevant to their needs and interests? Are they confident as they
work through the materials? Are they satisfied with what they have
learned?
5. Can the materials be managed efficiently in the manner they are mediated?

8. ROLE OF SUBJECT-MATTER,
LEARNING, AND LEARNER
SPECIALISTS IN FORMATIVE
EVALUATIONIt’s important to have the instruction reviewed by specialists.
SMEs may be able to comment on the accuracy and currency of
the instruction.
Learning specialist may be able to critique your instruction related
to what is known about enhancing that particular type of learning.
Learner specialist may be able to provide insights into the
appropriateness of the material for the eventual performance context.

9. ONE-TO-ONE EVALUATION WITH
LEARNERS
The purpose of one-to-one evaluation is to identify and remove the
most obvious errors in the instruction and to obtain initial
performance indications and reactions to the content by learners
There are three criteria:
1. Clarity – Is the message, or what is being presented, clear to individual
target learners
2. Impact – What is the impact of the instruction on individual learner’s attitude
and achievement of the objectives and goals
3. Feasibility – How feasible is the instruction given the available resources
(time/context)

10. SMALL GROUP EVALUATION
Small group evaluation have two primary purposes. The first one is
to determine the effectiveness of changes made following the one-to-
one evaluation and identify any remaining learning problems that
learners may have. Secondly, small group evaluation is used to
determine whether learners can use the instruction without
interacting with the instructor.

11. FIELD TRIAL
The field trial is the third stage in formative evaluation, referring to
the evaluations of the program and product in the setting in which it
is intended to be used. It is used to determine whether the
changes/revisions in the instruction made after the small group stage
were effective and to also see whether the instruction can be used in
the context for which it was intended.

12. FORMATIVE EVALUATION IN THE
PERFORMANCE CONTEXT
The purpose of the performance context formative evaluation is to
make three fundamental determinations
1. Do the learners find that it is appropriate to use their new skills in the
workplace and have they been doing so
2. If they have been used, what has been the impact on the organi_ation
3. What suggestions do the learners and other that they work with have for
improving the instruction

13. FORMATIVE EVALUATION OF
SELECTED MATERIALS
 Selected materials formative evaluation concludes that the
instructor chooses existing materials be there is seldom time to or
resources to conduct all faces of the evaluation. In this case, the
instructor should proceed directly to the field trial.
 The instructor who conducts a field trial is able to observe the
progress and attitudes of learners using a set of adopted or
adapted materials.

14. FORMATIVE EVALUATION OF
INSTRUCTOR-LED INSTRUCTION
 Formative evaluation of instructor-led instruction is done to
determine whether the instruction is effective and decide how to
improve it. It is functions just like the field trial.
 To prepare, the instructor must be concerned with the entry skills
and prior knowledge, the posttest knowledge, and that attitudes of
learners.
 This is an opportunity to evaluate instructional procedures.

15. DATA COLLECTION FOR SELECTED
MATERIALS AND INSTRUCTOR-LED
INSTRUCTION
 Information dealing with collection of data in a field trial of original
instructional materials applies equally well to date collection
procedures used in the evaluation of selected materials and
instructional procedures.

16. CONCERNS INFLUENCING FORMATIVE
EVALUATION
There are several concerns about the formative evaluation context
and the learners who participate in the evaluation that the
designer should keep in mind when planning and implementing
data-collection procedures
1. Context concerns
2. Concerns about Learners
3. Concerns about Formative Evaluation Outcomes
4. Concerns with Implementing Formative Evaluation

17. PROBLEM SOLVING DURING
INSTRUCTIONAL DESIGN
 The purpose is to make a decision about what to do with a
particular unit of instruction. The designer could collect enough
date in the formative evaluation about the illustrations to make at
least a tentative decision about their continued use in instructoin.

18. LIFELONG LEARNER SUMMARY
 As a change agent, formative evaluations must be done regularly
to ensure that information and content is kept current.

19. LARRY F. COBB
 Email: lfcobb@alasu.edu