This document discusses the importance of marking schemes and provides guidance on developing effective marking schemes. It explains that marking schemes should use criteria and performance levels to evaluate exam responses. Criteria should assess the thinking skills required by the exam question. Performance levels should distinguish levels of mastery based on membership and fellowship guidelines. Weighting criteria according to importance helps ensure marking is reliable and fair. Developing clear descriptions of each performance level, especially around the pass/fail cutoff, is also important.

1. Massey University | massey.ac.nz | 0800 MASSEY
Marking schemes
Liz Norman BVSc MVM EdD MANZCVS
Director of Teaching and Learning, College of Sciences
Massey University

2. Outline
What's going on? by hedera.baltica,
Attribution-ShareAlike Licence
• Why marking schemes are important
• Types of marking scheme
• Identifying criteria to use
• Deciding on performance levels

3. Validity (measuring what we want to measure)
Task setting:
– We want the candidate to perform a task that is
important and uses skills (including cognitive
skills) from the domain of practice
Marking:
– Markers need to reward features we want to
evaluate and not reward features we do not
want to evaluate
– Different markers need to award similar/scores
for the same candidate response

4. Purpose of marking schemes
• To help you during Q writing
– What content is important
– Whether the Q asks what you intended it to ask
– Whether it is do-able in the time available
• To help you during Q marking
– helps you decide how to award marks to any particular candidate
response
– facilitates reliable and fair marking

5. Types of marking scheme
1. Model answer – “ideal” answer
2. Point-based schemes
3. Criteria- & level-based schemes
4. Schemes with incorporated principle(s) for discriminating
levels

6. Model answers
• Not recommended as a sole component of a marking scheme
– Usually more than would be expected to be given by any candidate
– No guidance on how to assess alternative answers to the model
provided
– No guidance on how to award marks
• Can be a useful adjunct to a marking scheme

7. Point-based schemes
• Points for each objectively identifiable content point
• Does not indicate the relative importance of the points it
awards
• Sum can be more or less than the whole
• Treats knowledge and understanding as quantitative and
therefore does not help us evaluate and reward
higher-order thinking.

8. Criteria- & level-based schemes
• Criteria – different dimensions of performance
• Level – different quality/standards of performance on a given
criteria
Criteria
Descriptions of levels of performance
High level performance Mid-level performance Low level performance
Correctness of
knowledge
All facts correct Many facts correct but some
errors
Errors of fact or content
Sufficiency of
coverage
Comprehensive coverage Some missing but most
important areas covered
Missing important
information
Rationale and
justification
Fully supported and justified
by argumentation and citing
of evidence.
Supported and justified by
argumentation but evidence
not cited.
Limited or no support for
facts presented

9. Incorporating principles
• Calais has a warmer winter and a cooler summer than
Wroclaw. Explain why. (3 marks)
• Marking scheme:
Looking for answers related to distance from the sea therefore
latitude is not credited.
– Land heats up quicker than sea (1 mark)
– A clear distinction between land and sea heating (2 marks)
Ahmed & Pollitt (2011) Improving marking quality through a taxonomy of mark schemes. Assessment in Education: Principles, Policy & Practice 2011;18:259-278.

10. Identifying criteria to use
• Think about what you might see in the response that illustrates
the thinking processes that have gone on
• Many of the tasks used in our exams are similar, therefore the
criteria to use may be similar

11. some possible criteria to use
task criterion Generic descriptor
recall factual knowledge complete and correct
analysis
connections and
association
identifies and differentiates components, explains their relationships, provides
an organising structure for components, considers the whole picture, relates to
wider concepts and principles
analysis
grasp of subtleties
and nuances
explains or illustrates subtle connections, distinctions and associations, and
considers mitigating circumstances; identifies inconsistencies
reasoning reasoning
support, logic, rationale and justification provided; explanation of
counterarguments
judgement judgement appropriate and well-justified decisions
evaluation
implications and
significance
interprets or analyses significance, importance, & implications for thinking or
actions
evaluation contextualisation addresses the specifics of the task or scenario; selective answer
evaluation insight Illuminates tacit or overlooked assumptions, implications, conclusions.
synthesis planning systematic approach; plan is complete, appropriate, well justified

12. Activity – Choosing criteria
• There is an activity document for
you to download at the link in the
chat box.
• There are 8 examination questions
for you to consider in your groups.
– Discuss which marking scheme
criteria might apply.
– If you have time you may also reflect
on whether the questions provide
higher order tasks and whether they
belong in Paper 1 or Paper 2.
• I am going to run through the first
example together before we start

13. Discuss, based on the current understanding of the pathophysiology of post-operative
ileus, the pharmacological options for the treatment of post-operative ileus. (10 marks)
task criterion generic descriptor marking scheme high level descriptor
recall factual knowledge complete and correct
analysis
connections and
association
identifies and differentiates components, explains their
relationships, provides an organising structure for
components, considers the whole picture, relates to wider
concepts and principles
analysis
grasp of subtleties
and nuances
explains or illustrates subtle connections, distinctions and
associations, and considers mitigating circumstances;
identifies inconsistencies
reasoning reasoning
support, logic, rationale and justification provided;
explanation of counterarguments
judgement judgement appropriate and well-justified decisions
evaluation
implications and
significance
interprets or analyses significance, importance, &
implications for thinking or actions
evaluation contextualisation
addresses the specifics of the task or scenario; selective
answer
evaluation insight
Illuminates tacit or overlooked assumptions, implications,
conclusions.
synthesis planning
systematic approach; plan is complete, appropriate, well
justified

14. Discuss, based on the current understanding of the pathophysiology of post-operative
ileus, the pharmacological options for the treatment of post-operative ileus. (10 marks)
task criterion generic descriptor marking scheme high level descriptor
recall factual knowledge complete and correct
Includes all essential pharmacological options (see model
answer), correctly relates pathophysiological mechanisms,
no major misconceptions
analysis
connections and
association
identifies and differentiates components, explains their
relationships, provides an organising structure for
components, considers the whole picture, relates to wider
concepts and principles
analysis
grasp of subtleties
and nuances
explains or illustrates subtle connections, distinctions and
associations, and considers mitigating circumstances;
identifies inconsistencies
reasoning reasoning
support, logic, rationale and justification provided;
explanation of counterarguments
judgement judgement appropriate and well-justified decisions
evaluation
implications and
significance
interprets or analyses significance, importance, &
implications for thinking or actions
evaluation contextualisation
addresses the specifics of the task or scenario; selective
answer
evaluation insight
Illuminates tacit or overlooked assumptions, implications,
conclusions.
synthesis planning
systematic approach; plan is complete, appropriate, well
justified

15. Discuss, based on the current understanding of the pathophysiology of post-operative
ileus, the pharmacological options for the treatment of post-operative ileus. (10 marks)
task criterion generic descriptor marking scheme high level descriptor
recall factual knowledge complete and correct
Includes all essential pharmacological options (see model
answer), correctly relates pathophysiological mechanisms,
no major misconceptions
analysis
connections and
association
identifies and differentiates components, explains their
relationships, provides an organising structure for
components, considers the whole picture, relates to wider
concepts and principles
Explains pharmacological options in relation to
pathophysiology
analysis
grasp of subtleties
and nuances
explains or illustrates subtle connections, distinctions and
associations, and considers mitigating circumstances;
identifies inconsistencies
reasoning reasoning
support, logic, rationale and justification provided;
explanation of counterarguments
judgement judgement appropriate and well-justified decisions
evaluation
implications and
significance
interprets or analyses significance, importance, &
implications for thinking or actions
evaluation contextualisation
addresses the specifics of the task or scenario; selective
answer
evaluation insight
Illuminates tacit or overlooked assumptions, implications,
conclusions.
synthesis planning
systematic approach; plan is complete, appropriate, well
justified

16. Discuss, based on the current understanding of the pathophysiology of post-operative
ileus, the pharmacological options for the treatment of post-operative ileus. (10 marks)
task criterion generic descriptor marking scheme high level descriptor
recall factual knowledge complete and correct
Includes all essential pharmacological options (see model
answer), correctly relates pathophysiological mechanisms,
no major misconceptions
analysis
connections and
association
identifies and differentiates components, explains their
relationships, provides an organising structure for
components, considers the whole picture, relates to wider
concepts and principles
Explains pharmacological options in relation to
pathophysiology
analysis
grasp of subtleties
and nuances
explains or illustrates subtle connections, distinctions and
associations, and considers mitigating circumstances;
identifies inconsistencies
reasoning reasoning
support, logic, rationale and justification provided;
explanation of counterarguments
Explanations make sense, discussion of options well
supported by explanation of pathophysiology
judgement judgement appropriate and well-justified decisions
evaluation
implications and
significance
interprets or analyses significance, importance, &
implications for thinking or actions
evaluation contextualisation
addresses the specifics of the task or scenario; selective
answer
evaluation insight
Illuminates tacit or overlooked assumptions, implications,
conclusions.
synthesis planning
systematic approach; plan is complete, appropriate, well
justified

17. Discuss, based on the current understanding of the pathophysiology of post-operative
ileus, the pharmacological options for the treatment of post-operative ileus. (10 marks)
task criterion generic descriptor marking scheme high level descriptor
recall factual knowledge complete and correct
Includes all essential pharmacological options (see model
answer), correctly relates pathophysiological mechanisms,
no major misconceptions
analysis
connections and
association
identifies and differentiates components, explains their
relationships, provides an organising structure for
components, considers the whole picture, relates to wider
concepts and principles
Explains pharmacological options in relation to
pathophysiology
analysis
grasp of subtleties
and nuances
explains or illustrates subtle connections, distinctions and
associations, and considers mitigating circumstances;
identifies inconsistencies
reasoning reasoning
support, logic, rationale and justification provided;
explanation of counterarguments
Explanations make sense, discussion of options well
supported by explanation of pathophysiology
judgement judgement appropriate and well-justified decisions
evaluation
implications and
significance
interprets or analyses significance, importance, &
implications for thinking or actions
Discussion gives a sense of the relative value of the options,
appropriately contextualised
evaluation contextualisation
addresses the specifics of the task or scenario; selective
answer
evaluation insight
Illuminates tacit or overlooked assumptions, implications,
conclusions.
synthesis planning
systematic approach; plan is complete, appropriate, well
justified

18. Discuss, based on the current understanding of the pathophysiology of post-operative
ileus, the pharmacological options for the treatment of post-operative ileus. (10 marks)
task criterion generic descriptor marking scheme high level descriptor
recall factual knowledge complete and correct
Includes all essential pharmacological options (see model
answer), correctly relates pathophysiological mechanisms,
no major misconceptions
analysis
connections and
association
identifies and differentiates components, explains their
relationships, provides an organising structure for
components, considers the whole picture, relates to wider
concepts and principles
Explains pharmacological options in relation to
pathophysiology
analysis
grasp of subtleties
and nuances
explains or illustrates subtle connections, distinctions and
associations, and considers mitigating circumstances;
identifies inconsistencies
reasoning reasoning
support, logic, rationale and justification provided;
explanation of counterarguments
Explanations make sense, discussion of options well
supported by explanation of pathophysiology
judgement judgement appropriate and well-justified decisions
evaluation
implications and
significance
interprets or analyses significance, importance, &
implications for thinking or actions
Discussion gives a sense of the relative value of the options,
appropriately contextualised
evaluation contextualisation
addresses the specifics of the task or scenario; selective
answer
Pharmacological options chosen and explanations of
pharmacology contextualised to post-op ileus; specific (non-
generic) answer
evaluation insight
Illuminates tacit or overlooked assumptions, implications,
conclusions.
synthesis planning
systematic approach; plan is complete, appropriate, well
justified

19. Activity – Choosing criteria
• There is an activity document for
you to download at the link in the
chat box.
• There are 8 examination questions
for you to consider in your groups.
– Discuss which marking scheme
criteria might apply.
– If you have time you may also reflect
on whether the questions provide
higher order tasks and whether they
belong in Paper 1 or Paper 2.
• I am going to run through the first
example together before we start

20. Activity – discussion

21. Reconsidering question wording
• Writing marking schemes can alert you to question problems
• The question may not elicit a higher order response from the
candidate.
• Candidates cannot demonstrate higher order thinking unless
you ask them to
• Think about what aspects of their answer will alert you to their
higher order skills

22. What's going on? by hedera.baltica, Attribution-ShareAlike Licence
What about levels
of performance?

23. Criteria- & level-based schemes
• Criteria – different dimensions of performance
• Level – different quality/standards of performance on a given
criteria
Criteria
Descriptions of levels of performance
High level performance Mid-level performance Low level performance
Correctness of
knowledge
All facts correct Many facts correct but some
errors
Errors of fact or content
Sufficiency of
coverage
Comprehensive coverage Some missing but most
important areas covered
Missing important
information
Rationale and
justification
Fully supported and justified
by argumentation and citing
of evidence.
Supported and justified by
argumentation but evidence
not cited.
Limited or no support for
facts presented

24. New graduate between Membership between Fellowship
Levels of performance
• Difference between Membership and Fellowship
• Difference between passing level and failing level within
Membership and within Fellowship
Clear fail level (0-
54%)
Marginal fail (55-
70%)
Marginal pass (70-
79%)
Clear pass level
(80-100%)

25. What levels should we use?
• Subject guidelines for membership and fellowship give
guidance on levels
• When comparing membership and fellowship, bear in mind the
relative expected learning experiences
a question that is recall for a fellow who has been reading primary research
literature might be a very high level high-order question for a member who has
not read the primary research
• Pay attention to the pass-fail boundary. Ensure the descriptors
for each side of the boundary can be distinguished

26. Weighting the criteria
• Think about how important each criterion is – this dictates how
they should be weighted
• Remember that factual knowledge is encompassed in higher
order criteria so doesn’t need high weighting

27. Key points

28. Key points – writing marking schemes
• Use criterion and level marking schemes
• Select and organise 1-4 criteria for each question
• Choose criteria based on the type of thinking tasks the question is asking the
candidate to do
• Choose levels based on the subject guidelines for membership and fellowship, and
bearing in mind the question difficulty.
• Think about how important each criterion is – how they should be weighted
• Develop clear descriptions for each level/standard
of each criteria, especially to distinguish above and below the pass fail boundary
https://www.slideshare.net/liznorman

29. What's going on? by hedera.baltica, Attribution-ShareAlike Licence
Questions??

30. Key points – writing marking schemes
• Use criterion and level marking schemes
• Select and organise 1-4 criteria for each question
• Choose criteria based on the type of thinking tasks the question is asking the
candidate to do
• Choose levels based on the subject guidelines for membership and fellowship, and
bearing in mind the question difficulty.
• Think about how important each criterion is – how they should be weighted
• Develop clear descriptions for each level/standard
of each criteria, especially to distinguish above and below the pass fail boundary
https://www.slideshare.net/liznorman