Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduction to e-Assessment


Published on

Introduction to e-Assessment

  1. 1. Introduction to e-Assessment Designing e-Assessments City of Glasgow College 23rd November 2012
  2. 2. Overview• Designing e-Assessments• Effective Practice in e-Assessment• Coffee break• Activity – SOLAR• Additional Considerations and Emerging Trends
  3. 3. What is effective assessment?"Effective assessment and feedback can bedefined as practice that equips learners to studyand perform to their best advantage in thecomplex disciplinary fields of their choice, and toprogress with confidence and skill as lifelonglearners, without adding to the assessmentburden on academic staff." Effective Assessment in a Digital Age, JISC 2010
  4. 4. What is e-Assessment?“eAssessment is the end-to-end electronicassessment processes where ICT is used for thepresentation of assessment activity, and therecording of responses. This includes the end-to-end assessment process from the perspective oflearners, tutors, learning establishments,awarding bodies and regulators, and the generalpublic.” Effective Practice with eAssessment, JISC 2007
  5. 5. Categories of assessment • Assessment of skill / level / understanding • Used to identify the student’s current knowledge andDiagnostic skill level • Allows learning activities to match student requirements • Might be taken before a course has commenced • Tests knowledge / ability during the course • Can be used to provide feedback to individual studentsFormative at critical points in the learning process • Gives teachers an opportunity to review class results and address gaps in learning • Used to grade and judge the student’s level ofSummative understanding and skill development for progression or certification • Usually takes place at the end of a course or module
  6. 6. Assessment for LearningAssessment of How am I Learning doing? Where do I have to focus What is really my revision? making me think?An archive of evidence + What are my evaluation Is this the strengths and What are best way for weaknesses? my me to learn?Assessor determines and targets? evaluates evidence Work submitted formanual marking or to anassessment management systemResults in Accreditation
  7. 7. Blooms Taxonomy Convergent Divergent Knowledge Comprehension Application Analysis Synthesis Evaluation• Arrange • Classify • Apply • Analyse • Arrange • Argue• Define • Describe • Choose • Appraise • Assemble • Assess• Duplicate • Discuss • Demonstrate • Compare • Compose • Choose• List • Explain • Employ • Contrast • Construct • Compare• Memorise • Identify • Illustrate • Critique • Create • Defend• Name • Indicate • Interpret • Differentiate • Design • Judge• Recall • Report • Practice • Distinguish • Develop • Predict• Recognise • Select • Solve • Examine • Formulate • Rate• Repeat • Translate • Use • Test • Organise • Select• State • Plan • Evaluate • Prepare
  8. 8. Blooms Revised TaxonomyHOTS – Higher Order Thinking Skills Creating •Designing, constructing, planning, producing, inventing, devising, making Evaluating •Checking, hypothesising, critiquing, experimenting, judging, testing, detecting, monitoring Analysing •Comparing, organising, deconstructing, attributing, outlining, finding, structuring, integrating Applying •Implementing, carrying out, using, executing Understanding •Interpreting, summarising, inferring, paraphrasing, classifying, comparing, explaining, exemplifying Remembering •Recognising, listing, describing, identifying, retrieving, naming, locating, findingLOTS – Lower Order Thinking Skills
  9. 9. Question types Marking typesConvergent (LOTS) and feedback • Multiple Choice • Multiple Response • Automated marking • Matching / Drag & and tutor feedback Drop • Human based • Sequencing marking and tutor • Fill in the blank feedback • Likert scale • Peer review / • Assertion / Reason assessment • Confidence based marking with tutorDivergent (HOTS) feedback
  10. 10. Polling activity Text • Standard texting rates • We have no access to your phone number • Capitalization doesn’t matter, but spaces and spelling do BrowserCODE • Capitalization doesn’t matter, but spaces and spelling do Twitter • Capitalization doesn’t matter, but spaces and spelling do • Since @poll is the first word, your followers will not receive this tweet
  11. 11. How To Vote via Texting Text to: +447624806527 CODE
  12. 12. How To Vote via Submit to: CODE
  13. 13. How To Vote via Twitter @poll CODE
  14. 14. Multiple choice exampleA colleague is thinking about using video whenteaching, but realises she should to ensure the content isaccessible to all, what would you advise? A. Dont use video. Find alternative teaching strategy. B. Provide transcript for individual students who might need it. C. Make transcript of video available to all students. D. Only use video that has subtitles. Poll Everywhere
  15. 15. True False exampleIf a request for confidentiality is made by a disabledstudent, it may mean reasonable adjustment arecompromised because the reasonable adjustments areless important than the confidentiality. Under thesecircumstances no reasonable adjustment needs to bemade. A. True A. False Poll Everywhere
  16. 16. Assertion / Reason exampleWhen formatting digital documents you should structurethe text with coloured inbuilt heading styles because it isimportant that a range of devices or assistive technologiescan interpret the structure of digital content. A. Assertion and reason are true and the reason explains the assertion. B. Assertion and reason are true but the reason does not explain the assertion. C. Assertion is true but the reason is false. D. Assertion is false but the reason is true. E. Assertion is false and the reason is false. Poll Everywhere
  17. 17. Tips: Constructing quiz questionsUse straightforward languageAvoid using double negatives if possibleConsider the amount of text and graphicsStem should contain an unambiguous question or statementDistractors should be plausibleDistractors should be significantly different from each other3 or 4 distractor options are sufficientDon’t give the answer to one question in the wording of another one
  18. 18. Convergent (LOTS) Question types Marking types • Multiple Choice and feedback • Multiple Response • Automated marking • Matching / Drag & and tutor feedback Drop • Human based • Sequencing marking and tutor • Fill in the blank feedback • Likert scale • Peer review / • Assertion / Reason assessment • Confidence basedDivergent (HOTS) marking with tutor feedback
  19. 19. Online Assessment- Effective Practice
  20. 20. Some Theorists involved in assessment & feedback • Director of Re-Engineering Assessment Practices (REAP) and Peer Evaluation in Education Review (PEER) projects. Focuses on principles ofDavid Nicol good assessment and feedback design • • Focuses on developing students’ own skills in assessment, self-David Boud evaluation, authentic assessment, assessment in the workplace • • Focuses on the transition from feedback to self-monitoring and (David) developing independence in learningRoyce Sadler • sadler David • Focuses on learning oriented assessment (LOA), sustainable feedback practices, feedback timing Carless •
  21. 21. Good feedback practice: • Helps clarify what good performance is (goals, criteria, expected standards) • Facilitates the development of self-assessment (reflection) in learning • Delivers high quality information to students about their learning • Encourages teacher and peer dialogue around learning • Encourages positive motivational beliefs and self-esteem • Provides opportunities to close the gap between current and desired performance • Provides information to teachers that can be used to help shape the teachingNicol, D.J. & Macfarlane-Dick, D. (2004). Rethinking formative assessment in HE: a theoretical model and seven principles of good feedback practice. Higher Education Academy
  22. 22. Assessment Flexibleshould support responsive / JITT Self- Part of evaluation curriculum Autonomous Self-regulated Peer Linked to review Learners criteria Provide a Ensure Feedback Academic loop standards
  23. 23. Approaches• EVS • e-Portfolios, • e-Portfolios • Written, audio, video• Mobile polling incorporating digital • Blog posts tagged blog posts software content, blogs, files, against standards • Twitter• Twitter feeds • Virtual worlds• Quizzes • Wikis Showcasing Demonstrating Blogging /Diagnostic material and competency / meeting standards micro-blogging influences• Video / audio / text / • Online / mobile • Online / mobile • E-Portfolios animation quizzes quizzes • Wikis• Digital storytelling • Games • Free text assignment • Peer review software software submission • Virtual worldsDigital Automated Human marking Peer reviewStorytelling marking
  24. 24. Polling• Used with students at the University of the West of Scotland and Cumbernauld College• Using Poll Everywhere and Moodle EasyVoter, tutors created questions and students responded via browser, text or twitter in class• Supported active learning in class as the questions were discussed in small groups prior to responding• Helped student engagement with larger groups JISC RSC Scotland case study: Using Web 2.0 Tools to Develop and Support Multi-Campus Class, with UWS JISC RSC Scotland case study: Using Moodle easyVoter Student Response System at Cumbernauld College
  25. 25. Integrating formative, peer andsummative assessment• 1st year Computing Systems students at the University of the West of Scotland• Using Peerwise system, students composed own MC tests – created a large bank of tests• Students rated other students’ tests• Some student questions were used as part of summative assessment of the module JISC RSC Scotland case study: Using Web 2.0 Tools to Develop and Support Multi-Campus Class, with UWS JISC RSC Scotland case study: Using Moodle easyVoter Student Response System at Cumbernauld College
  26. 26. Improving academic standards• Issues highlighted at UWS and City of Glasgow College around poor academic standards and information literacy• Plagiarism detection software (Turnitin) used with Nursing students at UWS (adult returners and post-registration)• OriginalityCheck tool used to: • Enable students to learn how to analyse their own work • Consider how to integrate research into their writing • Ensure they referenced correctly• Useful to include guidance on information literacy when introducing plagiarism detection - contextualises plagiarism JISC RSC Scotland case study: Using Turnitin as a teaching tool and for summative assessment with UWS JISC RSC Scotland case study: An Information Literacy and Plagiarism Pilot Project with City of Glasgow College
  27. 27. Efficiencies in Marking• Turnitin used with Nursing students at UWS (adult returners and post-registration)• Grademark tool used to store assignments, speed up marking process, use pre-set comments• Marking consistency improved by using marking grid (rubric) to set marking criteria• Quick turnaround of feedback via the gradebook provided time for students to use comments to improve future work JISC RSC Scotland case study: Using Turnitin as a teaching tool and for summative assessment with UWS
  28. 28. Closing the feedback loop /creating a learning community• HNC Creative Industries students at Cumbernauld College and Royal Conservatoire of Scotland using ePortfolio for formative & summative assessments• Mixture of written work (theory) put in context with student generated content such as videos, blogs, images etc• Provides a showcase, archive of student work and enables students to view their progress during course• Feedback transparent and linked to learning outcomes. Feedback conversations between tutor to student, student to tutor and student to student. JISC RSC Scotland case study: Immersive Use of Mahara for Creative Industry Courses, with Cumbernauld College
  29. 29. Activity• Access OpenAssess on SOLAR website• Choose some tests and work through them.
  30. 30. Online Assessment- Additional Considerations and Emerging Trends
  31. 31. Overview • Inclusive assessment • Quality assurance • Considerations for summative assessment – case studies • Developing trends in e- Assessment - examples
  32. 32. Model of Accessibility Maturity: Diminishing Level of Risk JISC TechDis Single Equality Duty guidance booklet
  33. 33. Quality Assurance• Ensure appropriate safeguards are in place• Adhere to institutional quality assurance policies and procedures and guidelines from awarding bodies• Feed lessons learnt into future design and delivery
  34. 34. Considerations for summative assessment - case studies • Consider • Providing mock tests using system to be used for summative exam • Invigilation procedures and communication between invigilators in different geographic locations • Security and authentication procedures and links between assessment systems and student information systems • Procedures for multiple sittings of the one exam JISC RSC Scotland case study: Embedding e-Assessment in module delivery using Asessment21’s ABC at UWSJISC RSC Scotland case study: Implementing e-Assessment and Building an e-Asessment Centre at Edinburghs Telford College
  35. 35. Developing trends in e-Assessment - examples • Mobile assessment • Monitoring and documenting in situ assessment • Language learning • QR codes • Games Based assessment • xGames • GamesSpace • Augmented Reality • A framework for holistic assessment • Open Badges
  36. 36. Mobile assessment - observational assessments Observational assessment using tablet PCs • Use of tablet PCs at South Lanarkshire College with Construction students. • Assessment templates created and used to record assessment activity. • Comments recorded on template and a digital copy made available to students. • Immediate feedback for learners and access to their own portfolio of work • Assessment readily and easily available to external awarding bodies JISC RSC Scotland case study The use of tablet PCs for e-assessment, at South Lanarkshire College
  37. 37. Mobile assessment – using QR codes in assessment• Students scan QR codes using smartphone• Launches resources such as case studies, YouTube videos, quizzes and other formative assessments• Activities created using SoftChalk software and hosted on the web• QR code generator provides the image, containing access to the resources.Benefits:• Students have flexible access via their phones to content which they can use at any time• Has promoted discussion and reflection on activities• No need to pre-book computer labs• Provides spontaneity around the learning experience• Engages students in activities out-with class, promoting self- directed learning JISC RSC Scotland case study Using QR Codes with Hairdressing Students, at Perth College
  38. 38. Games based assessment - xGames• JISC funded project which aimed to use collaborative games to improve attainment levels of vocational learners• Project outputs included games templates and a bank of example questions (for use with the xBox)Project outcomes included:• Increased student motivation• Increased student to student interaction• Increased learner engagement and participation• Increased use of ICT for learning and teaching• Reduced barriers to using ICT for learning and teaching• Evidence of the pedagogical advantages of using games in an educational context• Prompted institutional decision makers and teaching staff to think about alternative methods of formative assessment JISC RSC Scotland case study xGames: Using Educational Computer Games for Group Learning, at Reid Kerr College
  39. 39. Games based assessment – SQA GamesSpace• SQA games based assessment platform• Provides games based assessments (NABs) for selected Skills for Work courses• Simulated 3d environment• Learners interact with characters and objects, perform tasks and carry out assessments• Results of the process are currently human marked Further information on GamesSpace page on SQA website
  40. 40. Framework for holistic assessment- Open Badges
  41. 41. Introducing e-Assessment• Start with low stakes formative assessment• Opportunity to innovate• Consider using devices and tools students are using• Make assessment part of the learning process
  42. 42. References & Useful Links• Principles of effective assessment and feedback practice: David Nicol, Gibbs and Simpson, 2004 David Boud, Royce Sadler, D. Carless, McDowell et al., 2006, ESCAPE project, 2010,• JISC RSC Scotland Local Case Studies http://www.rsc-• JISC RSC Scotland e-Assessment blog• JISC RSC Scotland e-Assessment magazine• Effective Assessment in a Digital Age, JISC 2010 ess.aspx• SQA eAssessment Resource• JISC Techdis e-Assessment Staff Pack Icons by