2. Emphasizes the necessity of gathering data from
members of the target population about the use
and effectiveness of materials and using the
information to make the materials even more
effective.
Background
3. Objectives
Describe the purposes for formative evaluation for
instructor-developed materials, instructor-selected
materials, and instructor-presented materials
Describe the instruments used in a formative evaluation
Develop an appropriate formative evaluation plan for a
set of instructional material and presentation.
Collect data according to a formative evaluation plan
for a given set of instructional materials .
4. Formative Evaluation Design Decision on
Development
Are the materials apropropriate for the type of learning
outcomes?
Do the materials include adequate instruction on the
subordinate skills and are these skills sequenced and clustered
logically?
Are the materials clear and readily understood by representative
members of the target group
What is the motivational value of the materials?
Can the materials be managed efficiently in the manner they are
mediated?
5. Main Components
of Materials
Types of
Learning
Content Clarity Motivation Management
Preinstructional
Presentation
Participation
Assessment
Who Judge? Leaning
Specialists
Content Expert Target Learners Target
Learners
Target
Learners/
Instructors
How are data
gathered?
Checklist
Interview
Checklist
Interview
Observation
Interviews
Test
Materials
Observations
Interviews
Surveys
Observations
Interviews
Main Areas of Questions about Material
EXAMPLE FRAMEWORK FOR DESIGNING A FORMATIVE EVALUATION
6. Used to identify and remove the most
obvious errors in the instruction and to
obtain initial performance indications
and reactions to the content by
learners.
One to One Evaluation
with Learners
1st Stage
7. Clarity: Is the message, or what is being presented,
clear to individual target learners?
Impact: What is the impact of the instruction on
individual learner’s attitudes and achievement of
objectives and goals?
Feasibility: How feasible is the instruction given the
available resources( time/context)?
Criteria
8. Designer selects learners with a range of ability.
The designer then work works on an individual basis with
each learner.
Designers should also be aware of learner
characteristics.(attitudes and previous experiences)
At least one learner who is above average in ability(but
not the top student)
At least one learner who is average.
At least one learner who is below average.
Selecting Learners
9. Criteria Message Links Procedures
Clarity of
Instruction
Vocabulary Level
Sentence complexity
Message complexity
Introductions
Elaboration
Conclusions
Transitions
Contexts
Examples
Analogies
Illustrations
Demotions
Reviews
Summaries
Sequence
Segment size
Transitions
Pace
Variations
Impact of
Learner
Attitudes Achievement
Utility of the information and
skills(relevance)
How easy /difficult the
and skills are to learn
Satisfaction with skills learned
Clarity of direction and
items for posttest
Scores on posttests
Feasibility Learner Resources
Maturity
Independence
Motivation
Time
Equipment
Environment
Data Collection
10. • Used to determine the
effectiveness of changes made
following the one-to-one
evaluation. Identify any remaining
learning problems that learners
may have
• Used to determine whether
learners can use the instruction
without interacting with the
instructor.
Small Group Evaluation
2nd Stage
11. Approximately eight to twenty learners
Representatives of your target population.
Low, average and high achieving students
Learners with various native languages
Learners who are familiar with a particular procedures and learners who are not
Younger or inexperience as well as more mature learners.
Small group evaluation refers to the number of learners and not the setting in which the learners
actually use the material.
Selecting Learners
Instructor administers the material in the manner they intend for the final
form
Instructor intervenes as little as possible
Learners difficulty and solutions should be noted for revision data.
Procedures
12. Was the instruction interesting
Did you understand what your were suppose to learn.
Were the materials directly related to the objectives?
Were sufficient practices exercised included?
Were the practice exercise relevant?
Did the test really measure your knowledge of the objectives?
Did you receive sufficient feedback on your practice exercises?
Did you feel confident when answering questions on the
tests?
Assessments and Questionnaires
13. Quantitative Data : test scores , time requirements and
cost projections.
Descriptive information: comments collected from
questionnaires, interviews, or evaluator’s notes.
Data Summary and Analysis
Outcomes
Refined Instructions such as: changing examples;
vocabulary in test items, increasing the amount of
time allocated for study, or modifications.
14. • Used to determine whether the
changes in the instruction made
after small group stage are
effective
• Used to determine if the
instructions can be used in the
context for which it is intended.
Field Trial
3rd Stage
15. Approximately thirty learners
Multiple trials may be necessary.
Selecting Learners
Similar to small group with exception to
Designer only observes the process
Instructor should administered the instruction
Reduction in testing.
Procedures
16. Same as small group.
Data Summary and Interpretation
Outcomes
Yields desired levels of learner achievement and
attitudes and that functions as intended in the
learning setting.
17. Performance Context: the site where
the skills are ultimately required.
Formative Evaluation in
the Performance Context
18. 1. Do the learners find that it is appropriate to use
their new skills in the workplace and have they
been doing so?
2. If they have been, what has been the impact on the
organization?
3. What suggestions do the learner and others that
work with have for improving the instructions?
Purpose of Performance Context Formative
Evaluation
19. Questions Data Sources Methods
1. Did the skills transfer? Learners Interviews
2. How are the skills used? Colleagues/peers of learners Questionnaires
3. What physical, social,
managerial factors enhanced
transfer and use of skills?
Subordinates of learners Observations
4. What physical, social,
managerial factors, inhibited
transfer and use of skills?
Supervisors Records analysis
Does using the skills help
resolve the original need?
Customers
How might training be refined
or improved?
Company records
Criteria and Data
20. One to one learners should participate.
Selecting Respondents
Learners should be told that they will be contacted
sometime in the future to discuss the instruction and
it usefulness.
Procedures
Outcomes
Strength and weaknesses in the instruction.
Areas in which transfer and use of skills can be
supported
Suggestions for revision instruction.
21. The instructor should administer the
pretest unless it is known that the
learner already has entry kills of what
is being taught.
A posttest and an attitude
questionnaire should be available to
evaluate learners' performance and
their opinion so the materials
Formative Evaluation
of Selected Materials
22. • Used to determine whether the
instruction is effective and decides
how to use it.
• The instructor should be
concerned with the entry skills and
prior knowledge, posttest
knowledge, and the attitudes of
learners
Formative Evaluation of
Instructor Led
Instructions
23. • Equipment must be in good
running order
• Environment must be conducive
for learning
• Instructor has knowledge of the
learners entry skills.
• Instructor should not rely entirely
on predictions
Data Collection for
Selected Materials and
Instructor Led
Instructions
24. • Context Concerns
• Concerns about Learners
• Concerns about Formative
Evaluation Outcomes
• Concerns with Implement
Formative Evaluation
Concerns Influencing
Formative Evaluation
25. Purpose is to make a decision about
what to do with a particular unit of
instruction.
Problem Solving during
Instructional Design
