The document discusses principles of effective assessment, including ensuring validity through accurate representation of course content and appropriate difficulty levels, and reliability through consistent question formats, language, and marking schemes. It also covers types of assessments, levels of learning outcomes, and how to develop an assessment plan and test blueprint to align assessments with learning objectives and standards. The goal is to design 21st century assessments that evaluate higher-order skills through real-world, collaborative tasks rather than isolated performances.

1. Assessment : Principles & Planning

2. What is the Point of Assessment?

5. Harrow School, London

6. Common complaints about assessments?

7. Validity / Accuracy

8. Reliability / Consistency

9. How do we ensure reliabilty? Set questions/paper in an expected form Be consistent for all classes Use clear language / stimulus Have a good number of test items Use a standard marking scheme

10. How do we ensure validity? Require subject mastery by setters and vetters Examination / Coursework must accurately represent syllabus Learning outcomes must be tested Fair and correct level of difficulty must be applied

11. Which is more important? Accuracy or Consistency? (SHOW VIDEO)

12. “ In 21 st century learning environments, decontextualised drop-in-from-the-sky assessments consisting of isolated tasks and performances will have zero validity as indices of educational attainment” (Pellegrino, 1999)

13. Types of Assessments

14. Formal Assessment

15. Informal Assessment

16. Quantitative Assessment

17. Qualitative Assessment

18. Formative Assessment

19. Summative Assessment

20. Structure Levels of learning outcomes Understanding KDU Standards Understand Asssessment principles Validity Reliability Credibility Realising Standards Produce Assessment Plan Produce Test Blueprint (for exams)

21. MQA: Learning Outcomes Competencies should include mastery of: A body of knowledge Practical skills Social skills and responsibility Ethics and professionalism Scientific thought; critical thinking and problem-solving Communication skills and teamwork Information management and life long learning Entrepreneurship and management

22. Taxonomy: Levels of Learning Outcome VII VI V IV III II I Organization VALUES Affective (Krathwohl) Adaptation SKILLS Psychomotor (Simpson) Analysis KNOWLEDGE Cognitive (Bloom) Levels Domain of Learning

23. Standards 40% 60% Degree Year 3 20% 50% 30% Degree Year 2 Synthesis Evaluation Application Analysis Knowledge/ Comprehension 50% 50% Degree Year 1 50% 50% Diploma Year 2 30% 70% Diploma Year 1 Learning Outcomes (Bloom’s) Context (Syllabus)

24. How do we realise the standards?

25. Assessment Plan

26. Assignments / Coursework Examinations What proportion of your assessment would you give to examinations and coursework?

27. Test Blueprint

28. The Test Blueprint 1. List topics to be tested 2. Decide number of Qs per topic 3. Decide on the level e.g. Bloom’s Taxonomy (Knowledge, Comprehension, Application, An 4. Decide on the number of Qs’ per level 5. Decide on difficulty level for each Q Analysis Synthesis Evaluation Application Knowledge/ Comprehension 1 7 2 Total Q9-M Q10-D Q8-M Q6-M Q5-M Q7-M Q2-E, Q3-M Q4-D Q1-E Total Learning Outcomes (Bloom’s) Topics (Syllabus)

29. The questions will change, not the blueprint!

30. Example of A Test Blue Print Attitude Skills Knowledge 10 1 7 2 Total 3 Q9-M Q10-D Q8-M Topic 3 3 Q6-M Q5-M Q7-M Topic 2 4 Q2-E, Q3-M Q4-D Q1-E Topic 1 Krathwohl Simpson Bloom Total CONSTRUCT/ Learning Outcomes CONTEXT (Syllabus)

31. “ Factual knowledge is valueless in a culture where Google and Wikipedia are a mouse-click away” (Elliot, 2008)

32. A Blueprint ensures… Congruence with teaching & learning plan Content validity - test what has been taught Assist test setter especially the uninitiated Maintain standard of test Comparable to other test(s)

33. 21 st Century Assessment?

34. Peer-to-Peer Assessments

35. Collaboration

36. Multi-Media / Digital

37. Distributed

38. Real-world

41. Traditional Emerging Preparing students for professorship / De-contextualised Training students for real-world action Lower Order Thinking Higher Order Skills Teacher-Assessed Peer-Assessed Individual Collaborative Conformity expected Creativity demanded