Designing and Conducting Formative Evaluation

1. Presented By James Henderson
Designing and Conducting Formative Evaluations

2. The concept underlying this chapter is formative
evaluation, which is the process designers use to
obtain data for revising their instruction to make
it more efficient and effective. There are three
basic phases of formative evaluation.
Background

3. OBJECTIVE
To discuss and give clarity to the following subjects:
Formative Evaluation Designs
Role of Subject-Matter, Learning, and Learner Specialists in Formative Evaluation
One-to-One Evaluation with Learners
Small-Group Evaluation
Field Trial
Formative Evaluation in the Performance Context
Formative Evaluation of Selected Materials
Formative Evaluation of Instructor-Led instruction
Data Collection for Selected Materials and Instructor-Led instruction
Concerns Influencing Formative Evaluation
Problem Solving during Instructional Design

4. Formative Evaluation
The process is used by designers to make their instructions more effective
and efficient.
A formative evaluation is the collection of data and information during the
development of instruction that can be used to improve the effectiveness
of the instruction.
 It was later found better to try out earlier components of the design
process, thereby avoiding a lot of problems that would otherwise not be
discovered until after the draft of the instruction was complete.
They were originally used as a process to improve instruction after the first
draft of instruction was developed.

5. FORMATIVE EVALUATION DESIGNS
These are questions that can be asked to evaluate the components of the
instructional strategy:
1. Are the materials appropriate for the type of learning outcome
2. Do the materials include adequate instruction on the subordinate skills,
and are these skills sequenced and clustered logically
3. Are the materials clear and readily understood by representative members
of the target group
4. What is the motivational value of the materials? Do the learners find the
materials relevant to their needs and interests? Are they confident as they
work through the materials? Are they satisfied with what they have learned
5. Can the materials be managed efficiently in the manner they are mediated

6. • It’s important to have the instruction reviewed by specialists.
• (SMEs) may be able to comment on the accuracy and currency of the
instruction.
• Learning specialist may be able to critique your instruction related to what
is known about enhancing that particular type of learning.
• Learner specialist may be able to provide insights into the appropriateness
of the material for the eventual performance context.
ROLE OF SUBJECT-MATTER, LEARNING, AND
LEARNER SPECIALISTS IN FORMATIVE EVALUATION

7. • The purpose of the first stage of the formative evaluation, is to identify and
remove the most obvious errors in the instruction and to obtain initial
performance indications and reactions to the content by learners There are
three criteria:
• 1. Clarity – Is the message, or what is being presented, clear to
target learners
• 2. Impact – What is the impact of the instruction on individual
attitude and achievement of the objectives and goals
• 3. Feasibility – How feasible is the instruction given the available
resources (time/context)?
ONE-TO-ONE EVALUATION WITH LEARNERS

8. • Small group evaluation have two primary purposes.
• The first one is to determine the effectiveness of changes made following
the one-to- one evaluation and identify any remaining learning problems
that learners may have.
• Secondly, small group evaluation is used to determine whether learners
can use the instruction without interacting with the instructor.
SMALL GROUP EVALUATION

10. • Final stage of formative evaluation
• Instructor attempts to use a learning context that closely resembles the
intended context
• Purpose is to determine whether that changes in the instruction made after
the small-group we effective
FIELD TRIAL

11. • The purpose of the performance context formative evaluation is to make
three fundamental determinations
• Do the learners find that it is appropriate to use their new skills in the
workplace and have they been doing so
• If they have been used, what has been the impact on the organization
• What suggestions do the learners and other that they work with have for
improving the instruction
FORMATIVE EVALUATION IN THE PERFORMANCE CONTEXT

12. • Selected materials formative evaluation concludes that the instructor
chooses existing materials be there is seldom time to or resources to
conduct all faces of the evaluation. In this case, the instructor should
proceed directly to the field trial.
• The instructor who conducts a field trial is able to observe the progress
and attitudes of learners using a set of adopted or adapted materials.
FORMATIVE EVALUATION OF SELECTED MATERIALS

13. • Formative evaluation of instructor-led instruction is done to determine
whether the instruction is effective and decide how to improve it. It is
functions just like the field trial.
• To prepare, the instructor must be concerned with the entry skills and prior
knowledge, the posttest knowledge, and that attitudes of learners.
• This is an opportunity to evaluate instructional procedures.
• Many instructions already use these types of formative evaluation in their
instructions
FORMATIVE EVALUATION OF INSTRUCTOR-LED
INSTRUCTION

15. • Information dealing with collection of data in a field trial of original
instructional materials applies equally well to date collection procedures
used in the evaluation of selected materials and instructional procedures.
• When an instructor evaluates self-instructional materials, selected
materials, or an instructor’s guide, existing rapport with learners can be a
great advantage.
• When the instructors selects materials to implement an instructional
strategy, a number of unique concern arise.
DATA COLLECTION FOR SELECTED MATERIALS AND
INSTRUCTOR-LED INSTRUCTION

16. • Distinguishes the instructional design process from a philosophical or
theoretical approach
• There are several concerns about the formative evaluation context and the
learners who participate in the evaluation that the designer should keep in
mind when planning and implementing data-collection procedures
• 1. Context concerns
• 2. Concerns about Learners
• 3. Concerns about Formative Evaluation Outcomes
• 4. Concerns with Implementing Formative Evaluation
CONCERNS INFLUENCING FORMATIVE EVALUATION

17. • The purpose is to make a decision about what to do with a particular unit
of instruction. The designer could collect enough date in the formative
evaluation about the illustrations to make at least a tentative decision
about their continued use in instruction.
• The same methodology can be used to answer a wide array of questions
that will inevitably arise during the design process.
PROBLEM SOLVING DURING INSTRUCTIONAL
DESIGN

18. Evaluations are important on any level, in any profession, to make sure that
the overall goals of an organization are accomplished. Evaluations also help
determine areas of improvement.
Life Long Learner

19. James Henderson
James.henderson178@gmail.com
Contact Information