Stephen Russell 2008


Published on

Session A - H6-09

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Stephen Russell 2008

  1. 1. Assessment in Australian universities: What they say they do to engage students Stephen Russell Defence And Systems Institute, UniSA
  2. 2. Objectives <ul><li>To determine: </li></ul><ul><li>The principles of best practice in using assessment to engage students recommended by Australian universities </li></ul><ul><li>The degree of support by the universities for each principle </li></ul>ATN Assessment 2008 Adelaide
  3. 3. Why? <ul><li>Marton, Säljö, Biggs, Entwistle & Ramsden started questioning good learning and assessment in 70s and 80s </li></ul><ul><li>We KNOW about good assessment practice </li></ul><ul><li>But knowing is different from practicing good assessment practice </li></ul><ul><li>Race (2003) asserts that assessment practice in the UK is ‘broken’ </li></ul><ul><li>Angelo (1996) asserts serious deficiencies in US assessment practice </li></ul>ATN Assessment 2008 Adelaide
  4. 4. Initial Assumptions <ul><li>Australian universities are aware of and concerned about good assessment practices </li></ul><ul><li>Australian universities publicise their particular view of the principles of good assessment practices </li></ul><ul><li>These principles are available on the Internet </li></ul><ul><li>University web-sites are user friendly </li></ul>ATN Assessment 2008 Adelaide
  5. 5. Methodology <ul><li>An initial set of principles was derived from the literature: </li></ul><ul><ul><li>Assessment should measure learning outcomes </li></ul></ul><ul><ul><li>Criteria should be clear, explicit, consistent, justifiable, evaluated and revised </li></ul></ul><ul><ul><li>Assessment should help students to learn </li></ul></ul><ul><ul><li>Feedback should be explicit, objective, and timely </li></ul></ul><ul><ul><li>Workloads should be appropriate </li></ul></ul><ul><ul><li>A variety of assessment tools should be used </li></ul></ul><ul><ul><li>Assessment purposes should be clearly explained </li></ul></ul>ATN Assessment 2008 Adelaide
  6. 6. <ul><li>University views of good assessment practice were derived from their websites ( Australian Education Network , 2008) </li></ul><ul><li>Minimum of 5 principles </li></ul><ul><li>Policy statements were not included </li></ul><ul><li>28 out 39 university guidelines included </li></ul><ul><li>Policies translated in systems engineering requirements (-like) statements – </li></ul><ul><ul><li>concise, simple, specific, ‘should’ imperitive </li></ul></ul>Methodology ATN Assessment 2008 Adelaide
  7. 7. Results ATN Assessment 2008 Adelaide No. Principle 1. Objectives Rqt.1.1 Assessment should help students learn Rqt.1.2 Assessment should be criterion-referenced Rqt.1.3 Assessment should assist students achieve professional standards of work Rqt.1.4 Assessment should judge achievement of learning outcomes Rqt.1.5 Assessment should help students to improve generic (graduate qualities) 50%
  8. 8. Results ATN Assessment 2008 Adelaide 2. How students learn Rqt.2.1 Assessment should be progressive over time Rqt.2.2 A diverse range of assessment methods should be used Rqt.2.3 Assessment tasks should be appropriate, fair and equitable Rqt.2.4 Students should be involved in their own assessment 50%
  9. 9. Results ATN Assessment 2008 Adelaide 3. Assessment as part of learning Rqt.3.1 Assessment should be an integral to course design Rqt.3.2 Assessment tasks should allow demonstration of deep learning 4. Assessment Tasks Rqt.4.1 Assessments tasks should be clear Rqt.4.2 Assessments tasks should be specific Rqt.4.3 Assessment tasks should be meaningful (real life) Rqt.4.4 Assessment tasks should address a variety of learning outcomes Rqt.4.5 Assessment tasks should minimise chances of plagiarism 50%
  10. 10. Results ATN Assessment 2008 Adelaide 5. Assessment Criteria Rqt.5.1 Assessment criteria should be clear Rqt.5.2 Assessment criteria should be specific (describe required standards of – knowledge, skills, competencies and capabilities) Rqt.5.3 Assessment criteria should be valid (measure what is intended) Rqt.5.4 Assessment criteria should be justifiable 50%
  11. 11. Results ATN Assessment 2008 Adelaide 6. Assessment Outcomes Rqt.6.1 Assessment outcomes should align with educational values Rqt.6.2 Assessment outcomes should align with the students' goals Rqt.6.3 Assessment outcomes should align with community goals Rqt.6.4 Assessment outcomes should be justifiable (i.e. valid, consistent, reliable ) Rqt.6.5 Assessment outcomes should align with government goals 50%
  12. 12. Results ATN Assessment 2008 Adelaide 7. Feedback Rqt.7.1 Students should receive specific, constructive and developmental feedback on their learning and performance Rqt.7.2 Feedback should be timely Rqt.7.3 Feedback should be supportive Rqt.7.4 Assessment instruments and processes should continuously be evaluated and adjusted 50%
  13. 13. Results ATN Assessment 2008 Adelaide 8. Workloads Rqt.8.1 Assessment should be sufficient to judge a student's learning, but not excessive Rqt.8.2 Assessment should avoid excessive workload on lecturers Rqt.8.3 The effort involved in an assessment task should match the value awarded 9. Standards Rqt.9.1 Assessment should uphold academic standards ( accredited) Rqt.9.2 Assessment should uphold ethical standards Rqt.9.3 Assessment outcomes should be confidential 50%
  14. 14. Discussion <ul><li>Many of the principles of good assessment discussed in the literature (summarised by Suskie, 2006) are included here </li></ul><ul><li>It provides a framework for universities to question their own principles of assessment </li></ul><ul><li>It provides a framework for future refinement </li></ul>ATN Assessment 2008 Adelaide
  15. 15. <ul><li>The top ten principles are focussed on enhancing student engagement – </li></ul><ul><li>e.g. Assessment should… </li></ul><ul><ul><li>return ‘specific and constructive’ feedback </li></ul></ul><ul><ul><li>be fair and equitable </li></ul></ul><ul><ul><li>judge learning outcomes </li></ul></ul><ul><li>The next 2 principles describe most important constraints – </li></ul><ul><ul><li>continuous evaluation and revision of assessment </li></ul></ul><ul><ul><li>uphold academic standards </li></ul></ul>Discussion ATN Assessment 2008 Adelaide
  16. 16. <ul><li>The next 2 (13 th and 14 th ) address student engagement, but may be thought implicit: </li></ul><ul><ul><li>assessment should be progressive </li></ul></ul><ul><ul><li>assessment tasks should be clear </li></ul></ul><ul><li>Only at 15 th do we have – assessment should address Graduate Qualities (43% support) </li></ul><ul><li>The next 5 address student engagement – deep learning, marks aligned with effort, valid criteria, participatory assessment, and learning outcome variety </li></ul>Discussion ATN Assessment 2008 Adelaide
  17. 17. <ul><li>Lowest support (11% to 29%) addressed: </li></ul><ul><ul><li>technical stuff – values, standards, goals </li></ul></ul><ul><ul><li>workloads </li></ul></ul><ul><ul><li>clarity & meaningfullness of tasks </li></ul></ul><ul><ul><li>relation between criteria and learning outcomes </li></ul></ul><ul><ul><li>supportive feedback </li></ul></ul><ul><ul><li>design to minimise plagiarism </li></ul></ul>Discussion ATN Assessment 2008 Adelaide
  18. 18. Conclusions & Open Questions <ul><li>We present a framework of university supported principles for good assessment </li></ul><ul><li>Australian universities do tend to promote principles of student engagement through assessment. </li></ul><ul><li>But what do the differences in support mean? </li></ul><ul><ul><li>Inertia? </li></ul></ul><ul><ul><li>Differences in expressing similar concepts? </li></ul></ul><ul><ul><li>Assumptions on implicit or explicit principles? </li></ul></ul><ul><ul><li>Branding? </li></ul></ul>ATN Assessment 2008 Adelaide
  19. 19. <ul><li>What direction are Australian universities taking in development of assessment guidelines towards better student engagement? </li></ul><ul><li>How does practice follow guidelines? </li></ul><ul><li>Are assessment practices in Australian universities ‘broken’? </li></ul>Conclusions & Open Questions ATN Assessment 2008 Adelaide