Designing and Conducting Summative Evaluations

1. Designing and Conducting
Summative Evaluations
By Teri Miles
MCOM 530 ~ East Stroudsburg University

2. The Purpose of Summative Evaluation
• Make “go-no-go” decisions
• Keep current materials?
• Look for something better suited to
meet organization’s specific instruction
needs?
• Summative evaluations are used to judge
the impact of a plan of instruction on the
organization’s initial problem.

3. Evaluators – Why they should be External
• The primary evaluator in a summative
evaluation is rarely the designer or developer
of the instruction.
• Evaluator is often unfamiliar with the
materials, the organization requesting the
evaluation, or the setting in which the
materials are evaluated.
• Preferred because they have no personal
investment and will likely be more objective

4. Two Main Phases of Summative Evaluation
2.Expert Judgment Phase
2. Field Trial Phase

5. Expert Judgment Phase (Table 12.1)
Purpose:
Do the materials have the potential for
meeting this organization’s needs?
1. Congruence Analysis: Are the needs and goals of the
organization congruent with those in the instruction?
2. Content Analysis: Are the materials complete, accurate,
and current?
3. Design Analysis: Are the principles of learning, instruction,
and motivation clearly evident in the materials?
4. Feasibility Analysis: Are the materials convenient, durable,
cost-effective, and satisfactory for current users?

6. Field Trial Phase
Purpose:
Are the materials effective with
target learners in the prescribed
setting?
Outcomes Analysis:
1.Impact on Learners
2.Impact on Job
3.Impact on Organization

7. Field Trial Phase (continued)
Management Analysis:
2. Are instructor and manager attitudes
satisfactory?
3. Are recommended implementation
procedures feasible?
4. Are costs related to time, personnel,
equipment, and resources reasonable?

8. Comparison of Formative and Summative
Evaluation (Table 12.3)
Formative Evaluation Summative Evaluation
Document strengths and
Locate weaknesses in weaknesses in instruction in
Purpose instructor in order to revise it order to decide whether to
maintain or adopt it
On-to-one
Expert judgment
Phases or Stages Small group
Field trial
Field Trial
Systematically designed in- Produced in-house or
Instructional
house and tailored to the elsewhere not necessarily
Development History needs of the organization following a systems approach
One set of materials or several
Materials One set of materials
competing sets
Member of design and
Position of Evaluator development team
Typically an external evaluator
A report documenting the
A prescription for revising design, procedures, results,
Outcomes instruction recommendations, and
rationale

9. See Rubric for
Evaluating Summative Evaluations
Then, take the quiz to test your knowledge!