eAssessment in practice


Published on

This presentation is an introduction to various aspects of eAssessment.

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

eAssessment in practice

  1. 1. Why assess? • To support and improve student learning • To test learning outcomes
  2. 2. Assessment – supporting student learning (1) • Quantity and distribution of student effort • Condition 1 Assessed tasks capture sufficient study time and effort • Condition 2 These tasks distribute student effort evenly across topics and weeks • Quality and level of student effort • Condition 3 These tasks engage students in productive learning activity • Condition 4 Assessment communicates clear and high expectations to students • Quantity and timing of feedback • Condition 5 Sufficient feedback is provided, both often enough and in enough detail • Condition 6 The feedback is provided quickly enough to be useful to students
  3. 3. Assessment – supporting student learning (2) • Quality of feedback • Condition 7 Feedback focuses on learning rather than on marks or students themselves • Condition 8 Feedback is linked to the purpose of the assignment and to criteria • Condition 9 Feedback is understandable to students, given their sophistication • Student response to feedback • Condition 10 Feedback is received by students and attended to • Condition 11 Feedback is acted upon by students to improve their work or their learning Source: Gibbs, G and Simpson, C. (2004). Conditions under which assessment supports students' learning. Learning and Teaching in Higher Education vol.1 pp.3-31.
  4. 4. eAssessment – overview Key players 4 3 Key issues Types of eAssessment Question & Test Design eAssessment at PolyU Online Tests 1 2 Pros & Cons
  5. 5. eAssessment – definitions (1) • The Joint Information Systems Committee (JISC) defines eAssessment as: the end-to-end electronic assessment processes where ICT is used for the presentation for assessment activity, and the recording of responses. • The Scottish Centre for Research into On-Line Learning and Assessment (SCROLLA) (2006) defines eAssessment as: an entirely automated process of delivering and marking assessments using web or intranet resources.
  6. 6. eAssessment – definitions (2) • The University of Oregon defines eAssessment as: the use of digital information and communication technologies to gather and analyse information from multiple and diverse sources to develop a deep understanding of what students know, understand, and can do with their knowledge as a result of their educational experiences
  7. 7. Questions to think about… • Do you recognise eAssessment as a separate category of assessment? • Is eAssessment just about multiple choice? • Can eAssessment assess learning levels appropriate to study in HE? • Can eAssessment be used to assess relevant knowledge and skills? • Can I use (more) eAssessment in my work?
  8. 8. Activity 1 What do you think are some of the main eAssessment issues for the key players?
  9. 9. Key Issues for Organisations, 1 Qualification Boards & Governments • What should be assessed online – high stake? Should it be institution/country-wide? • National frameworks and guidelines • JISC (E-Learning Framework, Good E-Assessment Practice guide), Higher Education Academy, EDNA, Australian Learning & Teaching Council Fellowship e-Assessment, European - Leonardo da Vinci project • Institutional projects, policies and codes of practice • Dundee University, Open University (UK), Deakin University, University of Adelaide • Can the institution handle the necessary academic cultural change?
  10. 10. Key Issues for Organisations, 1 Qualification Boards & Governments Source: www.scotland.gov.uk
  11. 11. Key Issues for Organisations, 1 Qualification Boards & Governments • 7 million adults lack functional numeracy and 5 million lack functional literacy. The proportion of adults aged 25 - 64 with low or no qualifications is more than double that in countries like Sweden, Japan and Canada. Source: Leitch Report, 2007 • ….up to half of the 12 000 recruits entering the Army each year have literacy or numeracy skills at levels at or below those expected of a primary school leaver.
  12. 12. Key Issues for Learning Designers, 2 Publishers & Developers • Enhance the validity & quality of assessment • Promote good practice in assessment • Marking & Grading assessments (in LMS) • (Intelligent) Feedback • Engage learners and teachers in assessment • Equity and diversity of assessment resources (e.g. gender, language, computer anxiety) • Improve cost-effectiveness of assessment • Keep up-to-date with latest technology and pedagogical developments in assessment • Design challenges
  13. 13. eAssessment – design challenges • symbolic systems • free text processing • computer-assisted validation of human marked scripts • mathematical input • diagrammatic reasoning • free-hand graphical input
  14. 14. 3 Key Issues for teachers, authors & subject matter experts • What to assess – type? problem-based? project-based? • Administration within LMS (+ Security) • Marking & Grading assessments • Question content – paper-base to electronic • Will it help to meet learning outcomes? • Integration into curriculum • Lack of familiarity with the medium • Perceptions – just multiple choice? • Will it ease the workload? • Will it help the weaker learners (remedial)?
  15. 15. 4 Key Issues for learners & other stakeholders • Why am I being assessed? Will it help me get my degree? • Are the right skills being tested? • Is it good preparation for the “real” exam? • Will it help to meet the learning outcomes? • Quick, accurate & detailed feedback • Security • Remedial work • Self-assessment • Graduate attributes
  16. 16. eAssessment – a rationale? • Many assessment strategies focus on what is easier to measure rather than what is important - measuring lower level cognition • But if we want students to have 21century skills we must focus on higher order outcomes. Who decides? The institution? Individual departments? Teachers? Learners?
  17. 17. eAssessment - types diagnostic tests student’s prior knowledge formative assessment to promote learning by (self- providing feedback. Usually not assessment; counted towards final grade. peer assessment) summative assessment which counts towards final grade. adaptive students have more control over the authored assessment. personal response quick way to assess understanding system
  18. 18. Assessment – part of the learning Journey
  19. 19. Activity 2 • How many kinds of activities can be eAssessed? • How do you assess? • What criterion do you use when assessing your activities?
  20. 20. Forms of eAssessment Form of assessment Examples Traditional assessment submitted online Essays, reviews, reports, case studies Automated assessment Multiple choice, short answer, matching, calculations Automated assessment – advanced options Multiple choice, short answer, matching/label matching Calculations/randomly generated answers, drag & drop Invigilated online exams – (mid/final semester) Range of formats, multiple choice/ short answer, automated Longer essay type Online discussions Forums, online debates/role plays, invited online guests Group projects PowerPoint presentations, CD-ROMs, multimedia projects, group online projects laboratory reports, networked collaborative learning, wikis Authentic assessment Simulations, critical incident analysis, case studies, access to external databases Critical reflection and meta-cognition Electronic portfolios, online journals, blogs, embedded reflective activities Advanced problem-solving Problem-based learning scenarios, learning contracts Table: Forms of eAssessment (adapted from a study by Peter Donnan, University of Canberra: Conducting assessment online. (2004)
  21. 21. Assessment Methods & Learning Outcomes • Matching learning outcomes with assessment methods Learning Outcomes Assessment methods Higher level •Reflect •Portfolios Thinking •Evaluate •Reflective writing •Predict •Practicum/workplace assessment •Argue •Group projects •Apply •Problem or case-based assessment •Relate •Extended essay •Explain •Essay exam •Describe •Objective test (e.g. MCQ) •Compute •Identify •Recall
  22. 22. Question & Test Design – more than just multiple choice! • Objective Tests • Question types
  23. 23. Online tests • PolyU examples • Examples • eAssessment resources
  24. 24. eAssessment @ PolyU • Moodle – English Language • Clinical Education • Anatomy
  25. 25. eScholars Group Statistics Simulators Self-test Mobile- Program based Glossary Tool
  26. 26. Examples • PRINCE 2 • English Language Exams • OpenMark – formative • OpenMark – diagnostic
  27. 27. Online testing – planning process Source: Sheffield Hallam University, UK
  28. 28. eAssessment Resources (1)
  29. 29. eAssessment Resources (2)
  30. 30. eAssessment – other resources • QuestionMark Perception • Pearson • Maple TA
  31. 31. Activity 3 • Can you think of any advantages and disadvantages of eAssessment?
  32. 32. eAssessment - pros • Immediate feedback can be given to students and to staff • Assessments can be linked to other online course materials being used • Ease of marking/consistency/accuracy • On demand delivery • Geographic flexibility • Time flexibility/convenience • Cost effectiveness • Ease of administration
  33. 33. eAssessment – cons • Staff need (more) time and extra skills to create eAssessment resources • Testing higher order skills requires greater effort • More difficult to assess written expression or creativity • Restrictions of multiple choice • Accessibility • Authentication/candidate validity • Security/confidentiality • Cost • Technical failure/issues & lack of support
  34. 34. Questions to think about • Do you recognise eAssessment as a separate category of assessment? • Is eAssessment just about multiple choice? • Can eAssessment assess learning levels appropriate to study in HE? • Can eAssessment be used to assess relevant knowledge and skills? • Can I use (more) eAssessment in my work?
  35. 35. Goodbye!!! Contact us: Ian etian@inet.polyu.edu.hk Jenny etjeni03@inet.polyu.edu.hk