The formative use of e-assessment Some early implementations, and Suggestions for how we might move on Andrew Boyle
Initial stuff – rationale, definitions, etc.
Introduction <ul><li>Early predictions re e-assessment </li></ul><ul><ul><li>e-assessment will lower barriers between form...
Definitions <ul><li>FA </li></ul><ul><ul><li>All those activities undertaken by teachers, and/or by students, which provid...
Research aims <ul><li>Identify types of implementations used frequently by researchers and developers working in the field...
Method and scope <ul><li>Review is inclusive, rather than excluding </li></ul><ul><ul><li>‘ Best evidence synthesis’ </li>...
Background findings re ‘plain’ FA
FA research is about several things <ul><li>Interaction </li></ul><ul><ul><li>Teacher-learner; learner-learner </li></ul><...
Claims about FA <ul><li>Associated with learning gains </li></ul><ul><ul><li>Claimed to be one of the strongest associatio...
Organisation of eFA findings
Findings Areas for further work Key issues
Finding 1 <ul><li>Electronic technologies provide a range of  new tools  that  classroom teachers  can use to create forma...
Characteristics of formative & summative items <ul><li>eFA implementations take item and task types from summative assessm...
eFA as exam revision <ul><li>FA research emphasises several facets </li></ul><ul><li>Many eFA implementations equate FA wi...
Need to write own test questions <ul><li>Early implementations of eFA often involve innovators developing own questions </...
Finding 2 <ul><li>e-assessment functionality permits formative  feedback  to be given in  variety  of ways  not possible i...
Principles for feedback in e-tests <ul><li>Need to systematise understanding of e-test feedback </li></ul><ul><ul><li>Does...
Practicality of e-portfolio feedback <ul><li>Need logistical studies that investigate practicality of using e-portfolios <...
Impact of cultural factors in e-communication <ul><li>Students giving and receiving feedback need to understand cultural n...
Finding 3 <ul><li>eFA applications can be used  remotely .  This provides a  resource  which is  not easily replicated  vi...
Better studies of learning gains <ul><li>Many papers assert attainment benefits of eFA </li></ul><ul><ul><li>Esp. for self...
Self-access for primary and secondary students? <ul><li>eFA as self-access strongly associated with tertiary </li></ul><ul...
Superordinate ‘key issues’
<ul><li>Define how the ‘e’ goes beyond plain FA, AND </li></ul><ul><li>Define genuine distinctiveness of eFA as opposed to...
<ul><li>Thanks for listening. </li></ul><ul><li>Questions/comments … </li></ul>
Features of FA items and tests <ul><li>Can have more (or less) than one correct answer </li></ul><ul><li>Distractors expli...
Upcoming SlideShare
Loading in...5
×

The Formative Use Of E Assessment

452

Published on

Taken from
http://www.caaconference.com/pastConferences/2007/presentationFiles/index.html

Published in: Education, Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
452
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

The Formative Use Of E Assessment

  1. 1. The formative use of e-assessment Some early implementations, and Suggestions for how we might move on Andrew Boyle
  2. 2. Initial stuff – rationale, definitions, etc.
  3. 3. Introduction <ul><li>Early predictions re e-assessment </li></ul><ul><ul><li>e-assessment will lower barriers between formative and summative assessment </li></ul></ul><ul><ul><li>Early years of e-assessment coincided with renewed interest in formative assessment (FA) </li></ul></ul><ul><ul><li>Now e-assessment needs to move to ‘mature’ phase </li></ul></ul><ul><li>Literature review </li></ul><ul><ul><li>Important role in FA research </li></ul></ul><ul><ul><li>Attempt to summarise where we have got to with formative e-assessment (eFA) </li></ul></ul><ul><ul><li>Group implementations to establish coherence </li></ul></ul><ul><ul><li>Move towards critique </li></ul></ul>
  4. 4. Definitions <ul><li>FA </li></ul><ul><ul><li>All those activities undertaken by teachers, and/or by students, which provide information to be used as feedback to modify the teaching and learning activities in which they are engaged </li></ul></ul><ul><ul><li>Contrasted with summative assessment </li></ul></ul><ul><ul><ul><li>Summarises learning </li></ul></ul></ul><ul><ul><ul><li>Used for recording and reporting the amount of learning but not for feeding back into learning </li></ul></ul></ul><ul><ul><li>Important strand in both school and HE assessment research </li></ul></ul><ul><ul><ul><li>Slightly different emphases </li></ul></ul></ul><ul><li>e-assessment includes </li></ul><ul><ul><li>Tests that are delivered on-screen </li></ul></ul><ul><ul><li>e-portfolios </li></ul></ul><ul><ul><li>Electronic discussion boards, forums and so on </li></ul></ul>
  5. 5. Research aims <ul><li>Identify types of implementations used frequently by researchers and developers working in the field </li></ul><ul><li>Suggest ways in which the body of research evidence might be expanded </li></ul><ul><li>Underlying aim to delineate those areas where eFA provides a distinctive input </li></ul><ul><li>Balance ‘e-assessment as transformative’ and more sceptical views </li></ul>
  6. 6. Method and scope <ul><li>Review is inclusive, rather than excluding </li></ul><ul><ul><li>‘ Best evidence synthesis’ </li></ul></ul><ul><ul><li>Seeks authentic, faithful and convincing results </li></ul></ul><ul><ul><li>Does not comply with one or more ‘objective’ criteria </li></ul></ul><ul><li>A thorough review of eFA literature </li></ul><ul><ul><li>Backed up by a selective review of FA literature </li></ul></ul><ul><ul><li>Not a general review of e-assessment </li></ul></ul>
  7. 7. Background findings re ‘plain’ FA
  8. 8. FA research is about several things <ul><li>Interaction </li></ul><ul><ul><li>Teacher-learner; learner-learner </li></ul></ul><ul><li>Feedback is central </li></ul><ul><ul><li>Speech – questions and answers </li></ul></ul><ul><ul><li>Comment-only marking on written work </li></ul></ul><ul><ul><ul><li>Focus on features of the work, not the individual </li></ul></ul></ul><ul><ul><ul><li>Permit learner to ‘close the gap’ between current and desired performance </li></ul></ul></ul><ul><li>Features of FA items and tests relatively little studied </li></ul><ul><ul><li>Can have more (or less) than one correct answer </li></ul></ul><ul><ul><li>Distractors explicitly connected to incorrect or incomplete conceptions </li></ul></ul><ul><ul><li>Item responses provide clues to effective action </li></ul></ul>
  9. 9. Claims about FA <ul><li>Associated with learning gains </li></ul><ul><ul><li>Claimed to be one of the strongest associations between educational intervention and learning gains </li></ul></ul><ul><ul><li>Some question about whether all reported gains are caused by the FA intervention </li></ul></ul><ul><li>Disseminating messages to practitioners key </li></ul><ul><ul><li>Evidence that changes in practice have been patchy </li></ul></ul>
  10. 10. Organisation of eFA findings
  11. 11. Findings Areas for further work Key issues
  12. 12. Finding 1 <ul><li>Electronic technologies provide a range of new tools that classroom teachers can use to create formative assessments to suit their and their students’ needs . </li></ul><ul><li>Variations on the theme of MCQ </li></ul><ul><li>Sophisticated tasks – rich in interactivity and multimedia </li></ul><ul><li>Test designs specific to e-assessment </li></ul><ul><li>e-portfolios to closely integrate formative and summative assessment </li></ul><ul><li>Communications tools such as electronic discussion boards and forums for self- and peer feedback in e-learning courses </li></ul>
  13. 13. Characteristics of formative & summative items <ul><li>eFA implementations take item and task types from summative assessment </li></ul><ul><ul><li>Cf. Wiliam’s characteristics of good formative items </li></ul></ul><ul><li>Need to build on research into FA item types </li></ul><ul><li>Establish distinctive characteristics of ‘good’ eFA items/tasks </li></ul>
  14. 14. eFA as exam revision <ul><li>FA research emphasises several facets </li></ul><ul><li>Many eFA implementations equate FA with revision or practice testing </li></ul><ul><ul><li>Revision not a bad thing per se </li></ul></ul><ul><ul><ul><li>Can reduce test anxiety </li></ul></ul></ul><ul><ul><ul><li>Support distance learners </li></ul></ul></ul><ul><ul><li>But is a reduced concept compared to totality of FA </li></ul></ul>
  15. 15. Need to write own test questions <ul><li>Early implementations of eFA often involve innovators developing own questions </li></ul><ul><ul><li>Realistic for all teachers to write their own questions? </li></ul></ul><ul><ul><li>If teachers select from pre-written banks of questions, is anything lost? </li></ul></ul><ul><ul><ul><li>Would they be able to tailor the formative assessment to their learners’ needs? </li></ul></ul></ul><ul><ul><ul><li>What sort of skill is selecting balanced formative tests from an item bank? </li></ul></ul></ul>
  16. 16. Finding 2 <ul><li>e-assessment functionality permits formative feedback to be given in variety of ways not possible in plain FA . </li></ul><ul><li>Variety of feedback methods in eFA test applications </li></ul><ul><li>Advantages of e-portfolios for providing feedback </li></ul><ul><ul><li>Encourages teachers and learners to interact about drafts </li></ul></ul><ul><ul><li>Process of generating work forefronted, rather than merely concentrating on the final product </li></ul></ul><ul><ul><li>Communications tools can give feedback varied by: channel, recipient, formality, etc. </li></ul></ul><ul><ul><ul><li>Can target learners of different styles </li></ul></ul></ul><ul><li>E-learning technologies can facilitate giving of feedback to distance learners </li></ul><ul><ul><li>Contribute to online discussions (peer feedback) </li></ul></ul><ul><ul><li>Reflective journal (self-assessment) </li></ul></ul><ul><ul><li>Evaluate peers’ work (either formally or informally) </li></ul></ul>
  17. 17. Principles for feedback in e-tests <ul><li>Need to systematise understanding of e-test feedback </li></ul><ul><ul><li>Does ‘comments not grades’ apply to eFA? </li></ul></ul><ul><ul><ul><li>If so, one of the most obvious benefits of e-assessment (rapid right/wrong info) is removed. </li></ul></ul></ul><ul><ul><li>Is engagement with rich media and interactivity synonymous with deep learning? </li></ul></ul><ul><ul><ul><li>Could learners be distracted – clicking through sites without really truly processing content? </li></ul></ul></ul><ul><ul><li>Could we relate feedback to students’ learning styles? </li></ul></ul>
  18. 18. Practicality of e-portfolio feedback <ul><li>Need logistical studies that investigate practicality of using e-portfolios </li></ul><ul><ul><li>Ensure that teachers can provide high-quality feedback and potential is not lost </li></ul></ul><ul><ul><li>ICT elements of portfolios should reduce burden </li></ul></ul>
  19. 19. Impact of cultural factors in e-communication <ul><li>Students giving and receiving feedback need to understand cultural norms </li></ul><ul><ul><li>Students working at distance may bring different prior assumptions to feedback </li></ul></ul><ul><ul><li>Culture can also mean academic culture </li></ul></ul><ul><ul><ul><li>Concern if students have not internalised academic norms </li></ul></ul></ul><ul><ul><ul><li>Requirement to style switch between varying e-discourses could make it harder for some to internalise ways of writing </li></ul></ul></ul>
  20. 20. Finding 3 <ul><li>eFA applications can be used remotely . This provides a resource which is not easily replicated via pencil-and-paper materials. </li></ul><ul><li>Advantages of asynchronicity for undergraduates: </li></ul><ul><ul><li>Reduced examination stress </li></ul></ul><ul><ul><li>Popular and motivating </li></ul></ul><ul><ul><li>Free up teacher time </li></ul></ul><ul><ul><li>Allow students to increase self-regulation for tertiary study </li></ul></ul><ul><ul><li>Asynchronous online discussions facilitates enhanced reflection </li></ul></ul><ul><li>Areas needing further clarification </li></ul><ul><ul><li>Effect of learning styles and motivation of usage </li></ul></ul><ul><ul><ul><li>Already self-regulating students benefit more </li></ul></ul></ul><ul><ul><ul><li>Usage differs between intrinsically interested & ‘pragmatists’ </li></ul></ul></ul><ul><ul><li>Clarify most common usage patterns </li></ul></ul>
  21. 21. Better studies of learning gains <ul><li>Many papers assert attainment benefits of eFA </li></ul><ul><ul><li>Esp. for self-access use </li></ul></ul><ul><li>Claims undermined by research design </li></ul><ul><ul><li>Small cohorts </li></ul></ul><ul><ul><li>Test difficulty not equated </li></ul></ul><ul><ul><li>Variables confounded </li></ul></ul><ul><li>Does eFA provide attainment benefits over and above ‘plain’ FA? </li></ul><ul><ul><li>It might – there are also claims that use of ICT in schooling is associated with improved attainment </li></ul></ul>
  22. 22. Self-access for primary and secondary students? <ul><li>eFA as self-access strongly associated with tertiary </li></ul><ul><li>What issues for self-access use by school pupils? </li></ul><ul><ul><li>Greater obligation on teachers to moderate feedback </li></ul></ul><ul><ul><li>Role of parents in online learning </li></ul></ul><ul><ul><li>Effective use to support ‘personalised learning’? </li></ul></ul>
  23. 23. Superordinate ‘key issues’
  24. 24. <ul><li>Define how the ‘e’ goes beyond plain FA, AND </li></ul><ul><li>Define genuine distinctiveness of eFA as opposed to summative e-assessment </li></ul><ul><li>Use of instruments by practitioners </li></ul><ul><li>Provision of feedback </li></ul><ul><li>Need to demonstrate learning gains </li></ul><ul><li>eFA must not be a reduced version of ‘plain’ FA </li></ul><ul><li>eFA as exam revision </li></ul><ul><li>Provision of feedback </li></ul><ul><li>Beware of how eFA may impose novel burdens on teachers and students </li></ul><ul><li>Requirement of teachers to write own test questions </li></ul><ul><li>Need to provide manageable systems for feeding back via e-portfolios </li></ul><ul><li>Need to understand how teachers and students adapt to novel roles that eFA requires </li></ul><ul><li>Cultural demands of using e-communication tools </li></ul><ul><li>Strong element of independent working and self-assessment </li></ul>
  25. 25. <ul><li>Thanks for listening. </li></ul><ul><li>Questions/comments … </li></ul>
  26. 26. Features of FA items and tests <ul><li>Can have more (or less) than one correct answer </li></ul><ul><li>Distractors explicitly connected to incorrect or incomplete conceptions </li></ul><ul><li>Item responses provide clues to effective action </li></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×