Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

2 marking

499 views

Published on

ANZCVS Examiners Workshop Feb 2016

Published in: Education
  • Be the first to comment

  • Be the first to like this

2 marking

  1. 1. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Grading criteria and marking schemes Liz Norman Massey University http://www.slidehare.net/liznorman Sharanne Raidal, BVSc, PhD, FANZCVS, GradDipEd Australian and New Zealand College of Veterinary Scientists Examiner Workshop, 6 – 7th February 2016 https://geogteacher.files.wordpress.com/2012/09/markscheme.jpg
  2. 2. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Validity • The questions must elicit the behaviour we want to evaluate • Different markers award similar marks for the same candidate response • Markers reward features we want to evaluate and not reward irrelevant features Johnson 2013
  3. 3. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Purpose of marking schemes •To help you during question writing  What content is important  Whether the Q asks what you intended it to ask  Whether it is do-able in the time available •To help you during question marking  Helps you decide whether an answer is good enough to be awarded a mark  Facilitates reliable and fair marking
  4. 4. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Types of marking scheme • Model answer – “ideal” response • Point based schemes • Criteria and level (standards) based schemes • Schemes with incorporated principle(s) for discriminating levels
  5. 5. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Model answers • Not recommended as a sole component of a marking scheme  Usually more than would be expected to be given by any candidate  No guidance on how to assess alternative answers to the model provided  No guidance on how to award marks • Can be a useful adjunct to a marking scheme
  6. 6. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Point-based schemes • Points for each objectively identifiable content point • Do not indicate the relative importance of the points awarded • Sum can be more or less than the whole • Rewards quantity not quality
  7. 7. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Criteria and level (standards) bases schemes • Criteria = different dimensions of performance • Level = different quality / standards of performance on a given criterion • Generic vs specific for the question • Explicit vs implicit weighting
  8. 8. SCHOOL OF ANIMAL AND VETERINARY SCIENCES
  9. 9. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Grade Description of grade High Distinction This essay commands attention because of its insightful development and mature style. The response to the text is convincing and elaborated upon with well-chosen and correctly referenced examples. It is written with aptly chosen words, effectively constructed sentences and a keen observation of the conventions of written English. Distinction This essay provides a thoughtful and well reasoned response to the text with appropriate and correctly referenced examples. The sentences are constructed and words chosen to communicate clearly to the reader. The conventions of written English have been well observed. Credit This essay provides a competently reasoned response to the text with some appropriate and mostly correctly referenced examples. The sentence structure and choice of words have sufficient precision to communicate the message to the reader. The conventions of written English have been observed. Pass This essay is satisfactory. It provides an adequate response to the text with sufficient examples and adequate reasoning. The examples are mostly adequately referenced. The sentence structure and choice of words communicates adequately to the reader. The conventions of written English need to be observed more closely. Fail This essay fails to respond appropriately to the text. The responses are simplistic or incoherent and suggest some significant misunderstanding of the text. The writing lack appropriate structure and has a pattern of errors in word choice with poor grammatical expression. Correctly referenced examples are absent or poorly presented. More attention needs to be paid to the conventions of written English.
  10. 10. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Grade Description of grade High Distinction This response commands attention because of its insightful development and mature style. The response to the question is convincing and elaborated upon with well-chosen and correctly referenced examples. It is written with aptly chosen words, [effectively constructed sentences] and a keen observation of the conventions of written English. Distinction This essay provides a thoughtful and well reasoned response to the text with appropriate and correctly referenced examples. The sentences are constructed and words chosen to communicate clearly to the reader. The conventions of written English have been well observed. Credit This essay provides a competently reasoned response to the text with some appropriate and mostly correctly referenced examples. The sentence structure and choice of words have sufficient precision to communicate the message to the reader. The conventions of written English have been observed. Pass This essay is satisfactory. It provides an adequate response to the text with sufficient examples and adequate reasoning. The examples are mostly adequately referenced. The sentence structure and choice of words communicates adequately to the reader. The conventions of written English need to be observed more closely. Fail This essay fails to respond appropriately to the text. The responses are simplistic or incoherent and suggest some significant misunderstanding of the text. The writing lack appropriate structure and has a pattern of errors in word choice with poor grammatical expression. Correctly referenced examples are absent or poorly presented. More attention needs to be paid to the conventions of written English.
  11. 11. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Analytic vs holistic schemes • Both are valid • Analytical (criteria scored separately)  Better agreement between examiners  Insufficient criteria  Overlapping criteria  Really just lots of smaller holistic decisions • Holistic (scored as a whole)  Don’t straight jacket examiners  Challenging, especially for longer answers  Less agreement between examiners
  12. 12. SCHOOL OF ANIMAL AND VETERINARY SCIENCES
  13. 13. SCHOOL OF ANIMAL AND VETERINARY SCIENCES The verbs distinguishing levels of performance are italicised. PASS The answer has most of the following characteristics: •Correctly describes relevant facts from the case, but... •May or may not identify the relevant theoretical framework •May or may not describe the relevant parts of the theory •Does not or incorrectly applies the theory to the object of the Q •Does not or incorrectly relates the theory to the case evidence. Answers at this level will make simple and obvious connections, but their significance may not be fully articulated. This type of response only meets one part of the task. Sometimes, this type of response deals with terminology but doesn’t develop further. CREDIT The answer has most of the following characteristics: •Correctly and proficiently describes relevant facts from the case, and •Correctly identifies the relevant theoretical framework •Correctly describes the relevant parts of the theory •May or may not apply the theory to the object of the question •May or may not relate the theory to the case evidence, but uses case evidence well to support argument. Responses at this level outline the influences of a number of factors, but may not bring together and balance their influences. Straight-forward connections between theory and evidence sets may be made, but the meta-connections between them are missed, as is their significance for the whole.
  14. 14. SCHOOL OF ANIMAL AND VETERINARY SCIENCES DISTINCTION The answer has most of the following characteristics: •Correctly and proficiently describes relevant facts from the case, and •Correctly identifies the relevant theoretical framework •Correctly describes the relevant parts of the theory •Correctly applies the theory to the object of the Q •Correctly relates the theory to the case evidence., and uses case evidence well to support their argument. A response at this level describes the case evidence and the theoretical frameworks and demonstrates good appreciation of how they are integrated. The answer demonstrates an appreciation of the significance to the parts of the question in relation to the whole. The overall answer has a coherent and appropriate structure where the parts of the answer and the connections between them are clearly demonstrated. HIGH DISTINCTION A response at this level has all the characteristics of the previous level (Distinction) and in addition demonstrates further insight, extending concepts and theoretical ideas into new but clearly related areas – across a number of areas. The response demonstrates connections not only within the given subject area of the question, but also beyond it . Answers at this level generalise and transfer the principles and ideas underlying the specific instance to a number of other contexts. Source: Sara Denize, UWS
  15. 15. SCHOOL OF ANIMAL AND VETERINARY SCIENCES % Learning outcome Fail Pass Credit Distinction High Distinction 25 Knowledge and understanding of the selected industry, position & competencies Little or no knowledge Basic knowledge Good knowledge Detailed knowledge Sophisticated knowledge 25 Demonstrated critical analytical skills & integrative thinking Good assessment of research findings & application to .... Methodically assesses research findings & applies it to personal career choices and career decisions 25 Problem solving 25 Effective communication, creative & innovative thought A below average presentation - little demonstrati on of involvement from all team members
  16. 16. SCHOOL OF ANIMAL AND VETERINARY SCIENCES SOLO taxonomy Prestructural Question may be rephrased at the answer; almost completely misses the point of the question Unistructural Able to identify, list, name, enumerate, but does not describe, explain, relate or elaborate multiple aspects of a response Multistructural Able to list as well as describe distinct aspects of a response (such as being able to describe aetiology, clinical features, management) but unable to explicitly explain causes for observations; unable to present cause-effect relationships Relational Able to describe multiple aspects of a process and additionally explain or elaborate observations into cause-effect relationships; able to compare similarities and differences between apparently distinct phenomena; this level is taken as suggestion the learner has understood Extended abstract Highly developed; able to explain mechanisms of phenomena and apply this information to a novel context – able to develop novel hypotheses, theories and deduce principles; creative thinking
  17. 17. SCHOOL OF ANIMAL AND VETERINARY SCIENCES SOLO levels in marking schemes Prestructural The task may be engaged, but the student is distracted or misled by irrelevant aspects Unistructural The student focuses on the relevant domain and works with a single aspect Multistructural The student provides correct material with discrete, separate pieces of information that may be combined to provide a composite picture Relational The student offers an integrated understanding of the information. The whole has a coherent structure and meaning Extended abstract Abstract general principles or hypotheses are provided
  18. 18. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Question examples at different SOLO levels Inviting a unistructural response: • Identify descending motor pathways that primarily mediate upright posture and reflex postural adjustments • List four examples of long acting calcium channel blockers Inviting a multistructural response: • Describe the structure of the eukaryotic cell membrane. • Describe the clinical features of ischemic stroke. Inviting a relational response: • Explain the mechanisms of each of the following in a patient with cirrhosis of the liver: ascites, splenomegaly, haematemesis and neurologic deterioration.
  19. 19. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Incorporating principles / rules • Ideal is a specific criteria- and level-bases scheme with incorporated principle(s) for discriminating levels
  20. 20. SCHOOL OF ANIMAL AND VETERINARY SCIENCES AQAGCSEScienceAPhysics1FoundationTierPhysics1FSpecimenMarkScheme http://www.aga.org.uk/subjects/science/gcse/physics-4403/past-papers-and-mark-scheme
  21. 21. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Incorporating principles Calais has a warmer winter and a cooler summer than Wroclaw. Explain why. (3 marks) Marking scheme: Looking for answers related to distance from the sea, therefore latitude is not credited. •Land heats up quicker than sea (1 mark) •A clear distinction between land and sea heating (2 marks) Ahmed & Pollitt (2011) Improving marking quality through a taxonomy of mark schemes. Assessment in Education: Principles, Policy & Practice 18: 259-278
  22. 22. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Incorporating principles Marking scheme excerpt: Discussion should focus on strategies adopted to ensure sustainability and an evaluation of these with regard to whether or not, or to what extent, the Sahel can be sustainably managed. The discussion will depend on the content and whether the overall view is optimistic or pessimistic. AQA (2013) General Certificate of Education (A-level) Geography Unit 1: Physical and Human Geography http://www.aga.org.uk/subjects/geography/a-level/geography-2030/past-papers-and-mark-schemes
  23. 23. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Quality vs quantity • More complex and unstructured questions allow better assessment of quality over quantity • In very constrained tasks, only judging how correct the answer is • In very open tasks, ‘correctness’ is less important and quality is judged instead
  24. 24. SCHOOL OF ANIMAL AND VETERINARY SCIENCES SOLO taxonomy Prestructural Question may be rephrased at the answer; almost completely misses the point of the question Unistructural Able to identify, list, name, enumerate, but does not describe, explain, relate or elaborate multiple aspects of a response Multistructural Able to list as well as describe distinct aspects of a response (such as being able to describe aetiology, clinical features, management) but unable to explicitly explain causes for observations; unable to present cause-effect relationships Relational Able to describe multiple aspects of a process and additionally explain or elaborate observations into cause-effect relationships; able to compare similarities and differences between apparently distinct phenomena; this level is taken as suggestion the learner has understood Extended abstract Highly developed; able to explain mechanisms of phenomena and apply this information to a novel context – able to develop novel hypotheses, theories and deduce principles; creative thinking
  25. 25. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Content vs connections Lucander et al (2010) The structure of observed learning outcome (SOLO) taxonomy: a model to promote dental students’ learning. Euro J Dent Ed 14(3): 145-150
  26. 26. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Writing marking schemes • Select and organise the criteria/dimensions • Develop clear descriptions for each level/standard of each criteria • Need to think about poor answers as well as good ones
  27. 27. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Revising mark schemes in use • Hopefully all types of answer are anticipated • Sometimes though it is not – can indicate unanticipated problems with the Q • Marking schemes might need revising after first few candidates marked • HSE should moderate marking and should encourage team members to report marking issues early in the piece
  28. 28. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Key points
  29. 29. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Marking schemes • An indication of the key criteria for discriminating passing answers from failing ones • Then add in what would be a better performance and what would be a lesser performance • Think about both quantitative and qualitative aspects of an answer
  30. 30. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Marking schemes Quantitative: • Which parts of the answer are essential • How many errors in these parts are tolerated Qualitative • Thinking processes you want to examine • What connections are essential / important • How the structure of an answer will help you differentiate a structured, related, connected thought process from a multistructural list of unrelated facts
  31. 31. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Marking schemes • Thinking about how a candidate who includes all / most of the right stuff, but also includes wrong stuff will be graded • What it indicates about the knowledge structure if there are contradictions / irrelevancies in the answer
  32. 32. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Marking schemes • Keep them brief • Think about them as specifying principles to apply rather than exhaustive detail • Concentrate on specifying the differentiating characteristics and how much of this there needs to be • Don’t use them to educate examiners about the topic 
  33. 33. SCHOOL OF ANIMAL AND VETERINARY SCIENCES Next session’s task: For each of your questions: • Think about how you will discriminate a good from a poor answer • Form this into a marking scheme Aim is to have a full set of drafted Qs and outlined marking schemes by 5:30pm

×