The document discusses various topics related to evaluation processes and test construction, including: - The purposes of tests such as assigning grades, measuring progress, and assessing teaching effectiveness. - Steps in test construction such as developing a table of specifications and determining test format. - Types of test questions like multiple choice, short answer, and essay questions. Guidelines for writing different types of questions are provided. - Item analysis which is used to select appropriate test items based on difficulty value and discrimination power. Formulas for calculating difficulty value and discrimination index are outlined. - Advantages and disadvantages of different question types are compared. Best practices for writing multiple choice questions, short answer questions, and essay questions are discussed

1. Dr.P.S.Sreedevi
Assistant Professor & Project Coordinator
School of Education under PMMMNMTT
Gandhigram Rural Institute – Deemed to be University
Dindigul 624 302, Tamil Nadu.

13. Evaluation Process

14. Purposes of Tests
• Assigning A Grade / Summative Evaluation
• Measuring Progress / Formative Feedback
• Providing Feedback To Students
• Motivating Students
• To Learn The Material Tested
• Assessing Teaching / Instructional
Effectiveness

15. • Step 1: Construct Table of Specifications
• Step 2: Decide Nature of Test / Type of
Questions
• Step 3: Construct Test Items (Questions)
• Step 4: Determine Test Format &
Administrative Details
Test Construction

16. Achievement Tests

17. Types of Questions
Open questions
• Such type of questions often begin with What, Why,
When, Who
• Sometimes they are statements: “tell me about”, “give me
examples of”.
• They can provide you with a good deal of information.
 Closed questions
• These are questions that require a yes or no answer and
are useful for checking facts. They should be used with
care.

18. Advantages
Closed
Easy for respondents, easier to compare,
choices can clarify meaning, less literate can
also answer, fewer confused answers.
Open
Unlimited answers, can answer in detail,
unanticipated findings, creativity and self
expression

19. Disadvantages
Closed
Respondent without knowledge can
answer, might not find desired answer,
confusing if many choices, mis interpretation
can be unnoticed.
Open
Different degree of answer, may be
irrelevant, difficult to compare, high literate
have advantage, greater amount of time and
intimidating.

20. Cont…
 Specific questions
• These are used to determine facts. For example “How
much did you spend on that”
 Probing questions
• These check for more detail or clarification.
• To explore specific areas.
• However be careful because they can easily make people
feel they are being interrogated

21. Hypothetical questions
These pose a theoretical situation in the future.
 For example, “What would you do if…?’
These can be used to get others to think of new
situations.
They can also be used in interviews to find out
how people might cope with new situations.
Cont…

22. Cont…
Reflective questions
 Helps students to reflect on their own or other’s
practices.
Leading questions
Used to gain acceptance of your view –
They are not useful in providing honest views and
opinions.
If you say to someone ‘you will be able to cope,
won’t you?’ they may not like to disagree

23. OBJECTIVE TYPE TEST

24. Revised Bloom’s Taxonomy

25. MCQ in the Light of Bloom’s Taxonomy

26. Level Definition
Remembering Recalling information
Understanding Identifying examples of a given term,
concept, or principle.
Interpreting the meaning of an idea, concept
or principle.
Applying Using information, rules and procedures in
concrete situations.
Analyzing Breaking information into parts to explore
patterns and relationships. Analyzing charts,
data to support conclusions.
Evaluating Justifying a decision or a course of action.
Creating Generating new ideas or products.

27. Remembering
Bloom’s Definition
Exhibit memory of previously learned material by
recalling facts, terms, basic concepts, and answer.
Verbs
Choose, Compare, Find, How, Label, List,
Match, Name, Omit, Recall, Relate, Select, Show,
Spell, Tell…
Sample Learning Objectives
Students will list the steps needed for
distillation.

28. Understanding
Bloom’s Definition
Demonstrate understanding of facts and ideas by
organising, comparing, translating, giving descriptions
and stating main ideas.
Verbs
Classify, Compare, Contrast, Demonstrate,
Explain, Extend, Illustrate, Infer, Outline, Relate,
Rephrase, Show, Summarise, Translate
Sample Learning Objectives
Students teachers will explain he importance of
lesson planning

29. Applying
Bloom’s Definition
Solve problems to new situations by applying
acquired knowledge, facets, techniques and rules in a
different way.
Verbs
Apply, Build, Choose, Construct, Develop,
Interview, Make use of model, Organize, Plan,
Select, Solve, utilise
Sample Learning Objectives
Students teachers will plan a lesson to teach 8 years
olds about photosynthesis

30. Analysing
Bloom’s Definition
Examine information into parts by identifying
motives or causes.
Make inferences and find evidence to support
generalisations.
Verbs
Analyse, Categorise, Classify, Compare, Contrast,
Dissect, Distinguish, Divide, Examine, Relationships,
Simplify, Survey, Take part in, Test for, Theme
Sample Learning Objectives
Students teachers will compare and contrast 3 different
cooperative learning strategies teachers can use in a science
class.

31. Evaluating
Bloom’s Definition
Present and defend opinions making judgements about
information, validity of ideas, or quality of work based on a set of
criteria
Verbs
Agree, Appraise, Assess, Award, Conclude, Criteria,
Criticise, Decide, Deduct, Defend, Determine, Disprove, Estimate,
Evaluate, Judge, Justify, Perceive, Prioritise, Prove, Recommend,
value
Sample Learning Objectives
Students teachers will appraise two teaching strategies, giving
reasons for their positive evaluation of one over the other..

32. Creating
Bloom’s Definition
Compile information together in a different way by
combining elements in a new pattern or proposing alterative
solutions.
Verbs
Adapt, Build, Change, Combine, Compile, Construct,
Create, Design, Develop, Elaborate, Estimate, Formulate,
Imagine, Improve, Invent,Make up, Modify, Originate, Plan,
Predict, propose
Sample Learning Objectives
Students teachers will appraise two teaching strategies,
giving reasons for their positive evaluation of one over the
other..

33. “A mother beats up her daughter because she
was drunk”
• Question
Who was drunk?

34. Characterises of a Good question

35. Principle of good question writing
Avoid ambiguity, confusion and vagueness
Eg. What is your income
Do you jog regularly
 Avoid emotional language and prestige bias
barrelled questions?
Do you rate the teacher higher if the teacher tells
many jokes?
Avoid leading questions?
You don’t smoke, do you?
Avoid asking about future intentions and beyond
respondents’ capabilities
Avoid double negatives

36. Elements of Multiple Choice Question
According to Clegg and Cashin (1986),
Item stem: This is the problem statement. It
contains all the information necessary to
answer the multiple choice question.
Correct option: This is the correct answer to
the multiple choice question.
Distractors/Alternatives: These are the
incorrect answers presented as alternatives
to the correct answer.

37. Example:
1. Which instrument is used to measure the
power of electric circuit ?
A. Voltmeter
B. Wavemeter
C. Viscometer
D. Wattmeter
Item
Stem
Distractors

39. Designing Stems
• Express the full problem in the stem
• Put all relevant material in the stem
• Eliminate excessive wording and
irrelevant information from the stem

40. Designing alternatives
• Limit the number of alternatives
• Make sure there is only one best answer
• Make the alternatives appealing and plausible
• Make the choices grammatically consistent with the stem
• Place the choices in some meaningful order
• Randomly distribute the correct response
• Avoid using “all of the above”& “none of the above”
• Refrain from using words such as always, never, all, or
none
• Avoid overlapping choices

41. Designing MCQ Strategies
• Write questions throughout the term
• Instruct students to select the “best
answer” rather than the “correct answer”
• Use familiar language
• Avoid giving verbal association clues
from the stem in the key
• Avoid trick questions
• Avoid negative wording

42. Advantages of MCQs
• Easy to score
• Can tap in to different cognitive levels
• Useful for diagnosing student difficulties
(e.g.,when you use common errors as
distractors)
• Provides an excellent basis for post-test
discussion (you should discuss why
distractors are wrong)
• Requires relatively little time to answer

43. Drawback of MCQs
• Items are open to misinterpretation by
students
• It can be time-consuming to construct “good”
test Items
• It is difficult to create questions that tap in to
higher levels of learning (i.e., Evaluation)
• Creativity cannot easily be tested
• Students may find that items are too picky
• They encourage guessing

44. Short Answer Questions (SAQs)
• Short-answer questions are open-ended.
• Questions that require students to create an answer.
• Commonly used in examinations to assess the basic
knowledge and understanding.
• Response restricted to point-wise answers
• More objective as point-wise answers make
agreement on key easier.
- Norman GR, Van der Vleuten (eds.)

45. Steps for Writing SAQs
Identify topic/concept that requires exploring
of different aspects such as diagnosis along
with reasons to arrive to the diagnosis.
 Construct scientific picture on the concept and
link questions
Frame 3-4 questions on specific tasks you want
candidates to perform
Select appropriate wordings for each question

46. Characteristics of Short Answer
Questions
Comes between objective and essay type.
 Can be answered in few sentences.
Is highly thought provoking.
Can cover a wide range of content.
It contains objectives individually like knowledge,
understanding, synthesis, application, analysis and
evaluation.

47. Short Answer Advantages
Question can be addressed towards the testing of
a specific objective or thinking skill.
Avoids cueing by requiring students to supply
an answer, rather than selecting or guessing from
options supplied
Short Answer Disadvantages
Individual questions for individual objectives
fragment the subject and
Reduce the likelihood that students will look for
the relationship between objectives.
It is difficult to address reasoning tasks, such as
decision making and critical thinking because
students don't have the opportunity to elaborate.

49. Essay Type Question
An essay type test presents or more
questions or other tasks that require
extended written responses from the person
being tested.
- Robert L.E & David A.F

50. Features of Essay type question
Both formative and summative assessments.
It require a great deal of thought and planning.
Students prepare their own answers.
Evaluation on knowledge areas alone.
Students handwriting, spelling, neatness,
organization and way of expressing ideas may be
considered while scoring the items.
No single answer is permitted freedom of response.
Examinee is permitted freedom of response.
Answer vary in their degree of quality or
corrections.

51. Principles for Construction of Essay Questions
Principles for
Construction
of ETQ
Survey the
landscape
Budget
your time
Scan all
the
resources
Don't
waste time
Don't
introduce
Don't
gesture.
Write
quickly
and neatly
Keep it
Real

52. Uses of Essay Test
Assess the ability to recall, organize, and
integrate ideas.
Assess the ability to express oneself in
writing.
Ability to supply information.
Assess student understanding of subject
matter.
Measure the knowledge of factual
information.

53. Advantages of Essay Questions
Freedom of response
Measures complex learning outcomes
It promotes the development of problem-solving
skills.
Improve the writing skills such as writing speed.
Encourages creativity by their unique way.
Disadvantages of Essay Questions
Scoring is not reliable
Grading of essay tests is time-consuming.
Subjective scoring of essay questions.
Didn’t cover the course content and objectives as
comprehensively as possible.

54. ITEM ANALYSIS
Item analysis is a statistical technique
which is used for selecting and rejecting the
items of the test on the basis of their difficulty
value and discriminated power

55. OBJECTIVES OF ITEM ANALYSIS
To select appropriate items for the final draft
To obtain the information about the difficulty
value(D.V) of all the items
To provide discriminatory power (D.I) to
differentiate between capable and less capable
examinees for the items
To provide modification to be made in some of
the items
To prepare the final draft properly ( easy to
difficult items)

56. STEPS OF ITEM ANAYSIS
Arrange the scores in descending order
 Separate two sub groups of the test papers
Take 27% of the scores out of the highest scores
and 27% of the scores falling at bottom
 Count the number of right answer in highest
group (R.H) and count the no of right answer in
lowest group (R.L)
 Count the non-response (N.R) examinees

57. Item analysis is done for obtaining:
Difficulty
value (D.V)
Discriminative
power (D.P)

58. DIFFICULTY VALUE (D.V)
“The difficulty value of an item is defined as the
proportion or percentage of the examinees who have
answered the item correctly”
- J.P. Guilford
The formula for difficulty value (D.V) D.V = (R.H +
R.L)/ (N.H + N.L)
R.H – rightly answered in highest group
R.L - rightly answered in lowest group
N.H – no of examinees in highest group
N.L - no of examinees in lowest group

59. DISCRIMINATION INDEX (D.I)
“Index of discrimination is that ability of
an item on the basis of which the discrimination
is made between superiors and inferiors” - Blood
and Budd (1972)

60. Types of Discrimination Index
Negative Discrimination
Index
Positive Discrimination
Index
Zero Discrimination or
No discrimination

61.  Zero Discrimination or No discrimination
The item of the test is answered correctly or know the
answer by all the examinee’s
An item is not answered correctly any of the examinee
Positive Discrimination Index
 An item is correctly answered by superiors and is not
answered correctly by inferiors. The discriminative power
range from +1 to -1.
Negative Discrimination Index
An item is correctly answered by inferiors and is not
answered correctly by superiors.

62. Discrimination Index
The formula for discrimination index (D.I)
D.I = (R.H - R.L)/ (N.H or N.L)
R.H – rightly answered in highest group
R.L - rightly answered in lowest group
N.H – no of examinees in highest group
N.L - no of examinees in lowest group

65. 3/14/2020 65

66. References
• www.u.arizona.edu/~jag/POL602/Designing-Managing-MCQs.pdf
• Dewey, R. A. (1998, January 20). Writing multiple choice items
which require comprehension. Retrieved November 3, 2003 from
• http://www.psywww.com/selfquiz/aboutq.html.
• Carneson J, Delpierre G and Masters, K (1996). Designing and
managing multiple choice questions. Retrieved from Centre for
Educational Technology, University of Capetown Web site:
http://web.uct.ac.za/projects/cbe/mcqman/mcqchp3.html.
• Green, K. (n.d.). Sample multiple choice questions that test higher
order thinking and application [PDF document]. Retrieved from
Washington State University Office of Assessment of Teaching and
Learning Web site:
http://oai.wsu.edu/large_classes/Sample%20multiple%
20choice%20questions%20for%20higher%20order%20thinking_atl.
pdf.

67. Discussion…
3/14/2020 67