Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Formative Use Of E Assessment


Published on

Taken from

Published in: Education, Technology, Business
  • Be the first to comment

  • Be the first to like this

The Formative Use Of E Assessment

  1. 1. The formative use of e-assessment Some early implementations, and Suggestions for how we might move on Andrew Boyle
  2. 2. Initial stuff – rationale, definitions, etc.
  3. 3. Introduction <ul><li>Early predictions re e-assessment </li></ul><ul><ul><li>e-assessment will lower barriers between formative and summative assessment </li></ul></ul><ul><ul><li>Early years of e-assessment coincided with renewed interest in formative assessment (FA) </li></ul></ul><ul><ul><li>Now e-assessment needs to move to ‘mature’ phase </li></ul></ul><ul><li>Literature review </li></ul><ul><ul><li>Important role in FA research </li></ul></ul><ul><ul><li>Attempt to summarise where we have got to with formative e-assessment (eFA) </li></ul></ul><ul><ul><li>Group implementations to establish coherence </li></ul></ul><ul><ul><li>Move towards critique </li></ul></ul>
  4. 4. Definitions <ul><li>FA </li></ul><ul><ul><li>All those activities undertaken by teachers, and/or by students, which provide information to be used as feedback to modify the teaching and learning activities in which they are engaged </li></ul></ul><ul><ul><li>Contrasted with summative assessment </li></ul></ul><ul><ul><ul><li>Summarises learning </li></ul></ul></ul><ul><ul><ul><li>Used for recording and reporting the amount of learning but not for feeding back into learning </li></ul></ul></ul><ul><ul><li>Important strand in both school and HE assessment research </li></ul></ul><ul><ul><ul><li>Slightly different emphases </li></ul></ul></ul><ul><li>e-assessment includes </li></ul><ul><ul><li>Tests that are delivered on-screen </li></ul></ul><ul><ul><li>e-portfolios </li></ul></ul><ul><ul><li>Electronic discussion boards, forums and so on </li></ul></ul>
  5. 5. Research aims <ul><li>Identify types of implementations used frequently by researchers and developers working in the field </li></ul><ul><li>Suggest ways in which the body of research evidence might be expanded </li></ul><ul><li>Underlying aim to delineate those areas where eFA provides a distinctive input </li></ul><ul><li>Balance ‘e-assessment as transformative’ and more sceptical views </li></ul>
  6. 6. Method and scope <ul><li>Review is inclusive, rather than excluding </li></ul><ul><ul><li>‘ Best evidence synthesis’ </li></ul></ul><ul><ul><li>Seeks authentic, faithful and convincing results </li></ul></ul><ul><ul><li>Does not comply with one or more ‘objective’ criteria </li></ul></ul><ul><li>A thorough review of eFA literature </li></ul><ul><ul><li>Backed up by a selective review of FA literature </li></ul></ul><ul><ul><li>Not a general review of e-assessment </li></ul></ul>
  7. 7. Background findings re ‘plain’ FA
  8. 8. FA research is about several things <ul><li>Interaction </li></ul><ul><ul><li>Teacher-learner; learner-learner </li></ul></ul><ul><li>Feedback is central </li></ul><ul><ul><li>Speech – questions and answers </li></ul></ul><ul><ul><li>Comment-only marking on written work </li></ul></ul><ul><ul><ul><li>Focus on features of the work, not the individual </li></ul></ul></ul><ul><ul><ul><li>Permit learner to ‘close the gap’ between current and desired performance </li></ul></ul></ul><ul><li>Features of FA items and tests relatively little studied </li></ul><ul><ul><li>Can have more (or less) than one correct answer </li></ul></ul><ul><ul><li>Distractors explicitly connected to incorrect or incomplete conceptions </li></ul></ul><ul><ul><li>Item responses provide clues to effective action </li></ul></ul>
  9. 9. Claims about FA <ul><li>Associated with learning gains </li></ul><ul><ul><li>Claimed to be one of the strongest associations between educational intervention and learning gains </li></ul></ul><ul><ul><li>Some question about whether all reported gains are caused by the FA intervention </li></ul></ul><ul><li>Disseminating messages to practitioners key </li></ul><ul><ul><li>Evidence that changes in practice have been patchy </li></ul></ul>
  10. 10. Organisation of eFA findings
  11. 11. Findings Areas for further work Key issues
  12. 12. Finding 1 <ul><li>Electronic technologies provide a range of new tools that classroom teachers can use to create formative assessments to suit their and their students’ needs . </li></ul><ul><li>Variations on the theme of MCQ </li></ul><ul><li>Sophisticated tasks – rich in interactivity and multimedia </li></ul><ul><li>Test designs specific to e-assessment </li></ul><ul><li>e-portfolios to closely integrate formative and summative assessment </li></ul><ul><li>Communications tools such as electronic discussion boards and forums for self- and peer feedback in e-learning courses </li></ul>
  13. 13. Characteristics of formative & summative items <ul><li>eFA implementations take item and task types from summative assessment </li></ul><ul><ul><li>Cf. Wiliam’s characteristics of good formative items </li></ul></ul><ul><li>Need to build on research into FA item types </li></ul><ul><li>Establish distinctive characteristics of ‘good’ eFA items/tasks </li></ul>
  14. 14. eFA as exam revision <ul><li>FA research emphasises several facets </li></ul><ul><li>Many eFA implementations equate FA with revision or practice testing </li></ul><ul><ul><li>Revision not a bad thing per se </li></ul></ul><ul><ul><ul><li>Can reduce test anxiety </li></ul></ul></ul><ul><ul><ul><li>Support distance learners </li></ul></ul></ul><ul><ul><li>But is a reduced concept compared to totality of FA </li></ul></ul>
  15. 15. Need to write own test questions <ul><li>Early implementations of eFA often involve innovators developing own questions </li></ul><ul><ul><li>Realistic for all teachers to write their own questions? </li></ul></ul><ul><ul><li>If teachers select from pre-written banks of questions, is anything lost? </li></ul></ul><ul><ul><ul><li>Would they be able to tailor the formative assessment to their learners’ needs? </li></ul></ul></ul><ul><ul><ul><li>What sort of skill is selecting balanced formative tests from an item bank? </li></ul></ul></ul>
  16. 16. Finding 2 <ul><li>e-assessment functionality permits formative feedback to be given in variety of ways not possible in plain FA . </li></ul><ul><li>Variety of feedback methods in eFA test applications </li></ul><ul><li>Advantages of e-portfolios for providing feedback </li></ul><ul><ul><li>Encourages teachers and learners to interact about drafts </li></ul></ul><ul><ul><li>Process of generating work forefronted, rather than merely concentrating on the final product </li></ul></ul><ul><ul><li>Communications tools can give feedback varied by: channel, recipient, formality, etc. </li></ul></ul><ul><ul><ul><li>Can target learners of different styles </li></ul></ul></ul><ul><li>E-learning technologies can facilitate giving of feedback to distance learners </li></ul><ul><ul><li>Contribute to online discussions (peer feedback) </li></ul></ul><ul><ul><li>Reflective journal (self-assessment) </li></ul></ul><ul><ul><li>Evaluate peers’ work (either formally or informally) </li></ul></ul>
  17. 17. Principles for feedback in e-tests <ul><li>Need to systematise understanding of e-test feedback </li></ul><ul><ul><li>Does ‘comments not grades’ apply to eFA? </li></ul></ul><ul><ul><ul><li>If so, one of the most obvious benefits of e-assessment (rapid right/wrong info) is removed. </li></ul></ul></ul><ul><ul><li>Is engagement with rich media and interactivity synonymous with deep learning? </li></ul></ul><ul><ul><ul><li>Could learners be distracted – clicking through sites without really truly processing content? </li></ul></ul></ul><ul><ul><li>Could we relate feedback to students’ learning styles? </li></ul></ul>
  18. 18. Practicality of e-portfolio feedback <ul><li>Need logistical studies that investigate practicality of using e-portfolios </li></ul><ul><ul><li>Ensure that teachers can provide high-quality feedback and potential is not lost </li></ul></ul><ul><ul><li>ICT elements of portfolios should reduce burden </li></ul></ul>
  19. 19. Impact of cultural factors in e-communication <ul><li>Students giving and receiving feedback need to understand cultural norms </li></ul><ul><ul><li>Students working at distance may bring different prior assumptions to feedback </li></ul></ul><ul><ul><li>Culture can also mean academic culture </li></ul></ul><ul><ul><ul><li>Concern if students have not internalised academic norms </li></ul></ul></ul><ul><ul><ul><li>Requirement to style switch between varying e-discourses could make it harder for some to internalise ways of writing </li></ul></ul></ul>
  20. 20. Finding 3 <ul><li>eFA applications can be used remotely . This provides a resource which is not easily replicated via pencil-and-paper materials. </li></ul><ul><li>Advantages of asynchronicity for undergraduates: </li></ul><ul><ul><li>Reduced examination stress </li></ul></ul><ul><ul><li>Popular and motivating </li></ul></ul><ul><ul><li>Free up teacher time </li></ul></ul><ul><ul><li>Allow students to increase self-regulation for tertiary study </li></ul></ul><ul><ul><li>Asynchronous online discussions facilitates enhanced reflection </li></ul></ul><ul><li>Areas needing further clarification </li></ul><ul><ul><li>Effect of learning styles and motivation of usage </li></ul></ul><ul><ul><ul><li>Already self-regulating students benefit more </li></ul></ul></ul><ul><ul><ul><li>Usage differs between intrinsically interested & ‘pragmatists’ </li></ul></ul></ul><ul><ul><li>Clarify most common usage patterns </li></ul></ul>
  21. 21. Better studies of learning gains <ul><li>Many papers assert attainment benefits of eFA </li></ul><ul><ul><li>Esp. for self-access use </li></ul></ul><ul><li>Claims undermined by research design </li></ul><ul><ul><li>Small cohorts </li></ul></ul><ul><ul><li>Test difficulty not equated </li></ul></ul><ul><ul><li>Variables confounded </li></ul></ul><ul><li>Does eFA provide attainment benefits over and above ‘plain’ FA? </li></ul><ul><ul><li>It might – there are also claims that use of ICT in schooling is associated with improved attainment </li></ul></ul>
  22. 22. Self-access for primary and secondary students? <ul><li>eFA as self-access strongly associated with tertiary </li></ul><ul><li>What issues for self-access use by school pupils? </li></ul><ul><ul><li>Greater obligation on teachers to moderate feedback </li></ul></ul><ul><ul><li>Role of parents in online learning </li></ul></ul><ul><ul><li>Effective use to support ‘personalised learning’? </li></ul></ul>
  23. 23. Superordinate ‘key issues’
  24. 24. <ul><li>Define how the ‘e’ goes beyond plain FA, AND </li></ul><ul><li>Define genuine distinctiveness of eFA as opposed to summative e-assessment </li></ul><ul><li>Use of instruments by practitioners </li></ul><ul><li>Provision of feedback </li></ul><ul><li>Need to demonstrate learning gains </li></ul><ul><li>eFA must not be a reduced version of ‘plain’ FA </li></ul><ul><li>eFA as exam revision </li></ul><ul><li>Provision of feedback </li></ul><ul><li>Beware of how eFA may impose novel burdens on teachers and students </li></ul><ul><li>Requirement of teachers to write own test questions </li></ul><ul><li>Need to provide manageable systems for feeding back via e-portfolios </li></ul><ul><li>Need to understand how teachers and students adapt to novel roles that eFA requires </li></ul><ul><li>Cultural demands of using e-communication tools </li></ul><ul><li>Strong element of independent working and self-assessment </li></ul>
  25. 25. <ul><li>Thanks for listening. </li></ul><ul><li>Questions/comments … </li></ul>
  26. 26. Features of FA items and tests <ul><li>Can have more (or less) than one correct answer </li></ul><ul><li>Distractors explicitly connected to incorrect or incomplete conceptions </li></ul><ul><li>Item responses provide clues to effective action </li></ul>