Conducting Material
Evaluation
Puja Shrivastav
Purposes of Learning Material
 Motivation: materials to catch the learners’
attention
 Participation: materials to familiarize learners
with content through hands-on activity
 Instructional: materials to teach necessary
information
 Follow-up: materials to reinforce knowledge
and provide additional information; also to
maintain literacy skills.
Objectives for Material Evaluation
 Are the objectives set in the curricular unit
achieved after using the material?
 Are the contents set in the curricular unit
relevant to the learners’ context and needs?
 Are the words used in the material appropriate
to the learners’ literacy level?
 Should also focus on its Visuals, Formats and
Teaching – learning process set as per
curriculum.
What should be evaluated?
 1. To evaluate quality of the material some
indicators are
 Achievement of objectives
 Relevance of contents
 Attractiveness of visuals
 Appropriateness of language level
 Appropriateness of teaching-learning process
 Others (format, durability, portability, size,
etc.)
Requirements for Material Evaluation
 Agenda- Designing of pre-test to get
maximum feedback.
 Draft material- Planning a course and syllabus
development
 Curricular units-
 Questionnaire- Conducting discussion
 Exams and assignments
 Field experiences
What is missing?
Material Evaluation
Pre testingPost testing
Material Evaluation
Material Evaluation
Pre testing of
material/ content
Post testing of
material/ content
Peer pre
testing
Learner
pre testing
Peer post
testing
Learner
post
testing
Pre-Test
 Learners pre- test
 Invite learners or visit another learning centre
to have clientele
 Provide a copy of the curricular unit of the
material to the facilitator, and ask her/him to
use the draft material according to the teaching
learning process set in the curricular unit
 Observe the performance of the facilitator and
reactions of the learners.

Continue….
 Distribute the questionnaire and ask the
learners to fill them out.
 After conducting pre-test, analyze the result of
the pre-test aggregating the remarks in the
questionnaire.
 Find out and list up the points to be improved
based on the analysis and observation.
Pre-Test
 Peer pre- test
 Invite material developers/ facilitators and
school teachers as clientele.
 Ask one facilitator to demonstrate the draft
according to the teaching learning process set
in the curricular unit
 Observe the performance of the demonstrator
and reactions of the clientele.
Continue…
 Distribute the questionnaire to all the clientele
and ask them to fill out.
 Discuss general impression of the draft among
clientele. Take note all the points raised.
 After conducing pre-test, analyze the result of
the pre-test aggregating the remarks in the
questionnaire.
 Find out and list up the points to be improved
based the analysis and
Purpose of the Evaluation
 To produce the highest possible quality of
student learning
 To help students learn more effectively and
efficiently than they otherwise would on their
own
 To develop a good quality material for
curriculum.
Revision and Finalization
 After the pre-test, we should revise and finalize the
material in the following way:
 Read through the questionnaires from learners and/or
peers one by one.
 Analyze both the summary data of the questionnaires
and comments.
 Interpret the data and list up the points for
improvement of the materials.
Continue..
 Transfer all the improvement points onto a
master copy of the draft, along with comments
and suggestions made by the learners and
peers, if any.
 If the material is developed in a group, discuss
with group members (illustrators, writers, etc.)
points to revise and how to revise.
 Revise the curricular unit, especially the
teaching-learning process following the
changes to be made with the material
Post testing
 Evaluation is a measurement of achievements
in relation to objectives set in the curricular
unit.
 The major part of material evaluation is
included in post testing of the finished product.
 Getting feedback from the users, including
facilitators and learners, through evaluation is
very helpful for us to develop better material
next time and to improve our material
development skill.
Formative vs. summative evaluation
 Formative evaluation is an evaluation conducted
during the development stage of the material.
 The result of the evaluation tells the developer where
and how to make necessary changes.
 In contrast, a summative evaluation is one that
provides information necessary to make a decision at
the end, whether to disseminate, re-develop, modify,
adopt, or adapt the material.
Formative Evaluation
 It help individual teachers to obtain useful feedback
on what, how much, and how well their students are
learning
 Teachers use this information to refocus their
teaching to help students learn more effectively
 Formative evaluation is Learner-centered, teacher-
directed, mutually beneficial,context-specific,
ongoing, and firmly rooted in good practice.
Formative
 Improve quality of learning, not provide
evidence for evaluation or grading
 Rarely graded, almost always anonymous
 Aim is to provide faculty with information
what, how much, and how well students are
learning
Learner-centered
 Focuses attention of students and teachers on
observing and improving learning, not
teaching
 In the end, if students become more
independent, lifelong learners, students must
take full responsibility of their learning
Teacher Directed
 It respects the autonomy, academic freedom,
and professional judgment of teacher
 Individual teacher decides when, what and
how to assess and how to respond to material
gathered.
Mutually beneficial
 Since focused on learning, requires active
participation of the students
 By cooperating in assessment, students reinforce their
understanding of course content and strengthen their
own skills at self-assessment
 Students can participate more actively in their
learning (taking ownership in process)
 Increased motivation when realize faculty are
invested in their success
Context-specific
 You need the right tool to do the right job
 Different content areas,
 Types of activities, and
 Even class dynamics warrant different
assessment/evaluation techniques
Ongoing
 Consider it like a classroom feedback loop
 Teachers get feedback from students on their
learning
 Faculty then complete the loop by providing
students with feedback on the results of
assessment and suggestions
 Loop continues
How do students directly benefit
from Evaluation?
 Meta-cognitive skills – describe students’ awareness
and understanding of their own learning skills,
performance and habits
 Understand what they are learning and why they are
learning it
 Good learners engage in more metacognitive
activities than poor learners do
 “The more time and effort students invest in the learning
process, the more intensely they engage in their own
education, the greater their educational experiences,
persistence in college, and continuation of learning”
 Process and content based feedback
How do teachers directly benefit
from classroom assessment?
 Benefits to students = benefit to teachers
 Develop a culture of openness and consistent feedback
 Constantly monitor student views on process and
understanding of content
 Answer difficult questions such as:
 “How can I find out whether my students are learning the essential
skills and knowledge I am trying to teach”
 “How can I help students learn better?”
 Reduce uncertainty of student learning between exams
 Improve teaching skills and gain new insights
 Increase overall learning
Conducting Material Evaluation

Conducting Material Evaluation

  • 1.
  • 2.
    Purposes of LearningMaterial  Motivation: materials to catch the learners’ attention  Participation: materials to familiarize learners with content through hands-on activity  Instructional: materials to teach necessary information  Follow-up: materials to reinforce knowledge and provide additional information; also to maintain literacy skills.
  • 3.
    Objectives for MaterialEvaluation  Are the objectives set in the curricular unit achieved after using the material?  Are the contents set in the curricular unit relevant to the learners’ context and needs?  Are the words used in the material appropriate to the learners’ literacy level?  Should also focus on its Visuals, Formats and Teaching – learning process set as per curriculum.
  • 4.
    What should beevaluated?  1. To evaluate quality of the material some indicators are  Achievement of objectives  Relevance of contents  Attractiveness of visuals  Appropriateness of language level  Appropriateness of teaching-learning process  Others (format, durability, portability, size, etc.)
  • 5.
    Requirements for MaterialEvaluation  Agenda- Designing of pre-test to get maximum feedback.  Draft material- Planning a course and syllabus development  Curricular units-  Questionnaire- Conducting discussion  Exams and assignments  Field experiences What is missing?
  • 6.
  • 7.
    Material Evaluation Material Evaluation Pretesting of material/ content Post testing of material/ content Peer pre testing Learner pre testing Peer post testing Learner post testing
  • 8.
    Pre-Test  Learners pre-test  Invite learners or visit another learning centre to have clientele  Provide a copy of the curricular unit of the material to the facilitator, and ask her/him to use the draft material according to the teaching learning process set in the curricular unit  Observe the performance of the facilitator and reactions of the learners. 
  • 9.
    Continue….  Distribute thequestionnaire and ask the learners to fill them out.  After conducting pre-test, analyze the result of the pre-test aggregating the remarks in the questionnaire.  Find out and list up the points to be improved based on the analysis and observation.
  • 10.
    Pre-Test  Peer pre-test  Invite material developers/ facilitators and school teachers as clientele.  Ask one facilitator to demonstrate the draft according to the teaching learning process set in the curricular unit  Observe the performance of the demonstrator and reactions of the clientele.
  • 11.
    Continue…  Distribute thequestionnaire to all the clientele and ask them to fill out.  Discuss general impression of the draft among clientele. Take note all the points raised.  After conducing pre-test, analyze the result of the pre-test aggregating the remarks in the questionnaire.  Find out and list up the points to be improved based the analysis and
  • 12.
    Purpose of theEvaluation  To produce the highest possible quality of student learning  To help students learn more effectively and efficiently than they otherwise would on their own  To develop a good quality material for curriculum.
  • 13.
    Revision and Finalization After the pre-test, we should revise and finalize the material in the following way:  Read through the questionnaires from learners and/or peers one by one.  Analyze both the summary data of the questionnaires and comments.  Interpret the data and list up the points for improvement of the materials.
  • 14.
    Continue..  Transfer allthe improvement points onto a master copy of the draft, along with comments and suggestions made by the learners and peers, if any.  If the material is developed in a group, discuss with group members (illustrators, writers, etc.) points to revise and how to revise.  Revise the curricular unit, especially the teaching-learning process following the changes to be made with the material
  • 15.
    Post testing  Evaluationis a measurement of achievements in relation to objectives set in the curricular unit.  The major part of material evaluation is included in post testing of the finished product.  Getting feedback from the users, including facilitators and learners, through evaluation is very helpful for us to develop better material next time and to improve our material development skill.
  • 16.
    Formative vs. summativeevaluation  Formative evaluation is an evaluation conducted during the development stage of the material.  The result of the evaluation tells the developer where and how to make necessary changes.  In contrast, a summative evaluation is one that provides information necessary to make a decision at the end, whether to disseminate, re-develop, modify, adopt, or adapt the material.
  • 17.
    Formative Evaluation  Ithelp individual teachers to obtain useful feedback on what, how much, and how well their students are learning  Teachers use this information to refocus their teaching to help students learn more effectively  Formative evaluation is Learner-centered, teacher- directed, mutually beneficial,context-specific, ongoing, and firmly rooted in good practice.
  • 18.
    Formative  Improve qualityof learning, not provide evidence for evaluation or grading  Rarely graded, almost always anonymous  Aim is to provide faculty with information what, how much, and how well students are learning
  • 19.
    Learner-centered  Focuses attentionof students and teachers on observing and improving learning, not teaching  In the end, if students become more independent, lifelong learners, students must take full responsibility of their learning
  • 20.
    Teacher Directed  Itrespects the autonomy, academic freedom, and professional judgment of teacher  Individual teacher decides when, what and how to assess and how to respond to material gathered.
  • 21.
    Mutually beneficial  Sincefocused on learning, requires active participation of the students  By cooperating in assessment, students reinforce their understanding of course content and strengthen their own skills at self-assessment  Students can participate more actively in their learning (taking ownership in process)  Increased motivation when realize faculty are invested in their success
  • 22.
    Context-specific  You needthe right tool to do the right job  Different content areas,  Types of activities, and  Even class dynamics warrant different assessment/evaluation techniques
  • 23.
    Ongoing  Consider itlike a classroom feedback loop  Teachers get feedback from students on their learning  Faculty then complete the loop by providing students with feedback on the results of assessment and suggestions  Loop continues
  • 24.
    How do studentsdirectly benefit from Evaluation?  Meta-cognitive skills – describe students’ awareness and understanding of their own learning skills, performance and habits  Understand what they are learning and why they are learning it  Good learners engage in more metacognitive activities than poor learners do  “The more time and effort students invest in the learning process, the more intensely they engage in their own education, the greater their educational experiences, persistence in college, and continuation of learning”  Process and content based feedback
  • 25.
    How do teachersdirectly benefit from classroom assessment?  Benefits to students = benefit to teachers  Develop a culture of openness and consistent feedback  Constantly monitor student views on process and understanding of content  Answer difficult questions such as:  “How can I find out whether my students are learning the essential skills and knowledge I am trying to teach”  “How can I help students learn better?”  Reduce uncertainty of student learning between exams  Improve teaching skills and gain new insights  Increase overall learning