Scoping a vision of formative e-assessment Key issues from the literature (and do we want consensus…)?

Some consensus… Formative (e)-assessment is concerned with learners making progress towards measurable attributes/knowledge/skills/understanding It is about working with mechanisms which allow the gap to close between what they are currently able to achieve and what they might be able to achieve Increasing learners’ active responsibility for their part in the learning process is a main feature across a range of contexts and technologies … but not much…differences tend to focus around whether ‘assessment’ is treated as an ‘event’ or a ‘process’.

Formative (e)-assessment is concerned with learners making progress towards measurable attributes/knowledge/skills/understanding

It is about working with mechanisms which allow the gap to close between what they are currently able to achieve and what they might be able to achieve

Increasing learners’ active responsibility for their part in the learning process is a main feature across a range of contexts and technologies

… but not much…differences tend to focus around whether ‘assessment’ is treated as an ‘event’ or a ‘process’.

Definitions and claims – teacher & learner roles Black et al 2003 (applied FA theory in case study school settings): ‘ An assessment activity can help learning if it provides information to be used as feedback by teachers, and by their students in assessing themselves and each other, to modify the teaching and learning activities in which they are engaged. Such assessment becomes formative assessment when the evidence is used to adapt the teaching work to meet learning needs ’ (p. 2). Although it may involve several methods 'it has to be in the control of the individual teacher and, for this reason, change in formative assessment practice is an integral and intimate part of a teacher's daily work ' (p. 2) What if… the teacher/tutor is ‘monitoring’ rather than ‘changing’? assessment processes are purely for self-assessment? the responses are automated? Queries for criterion and case development : What must be defined about teacher roles in FA processes? What must be defined about learner roles in FA processes? Cox et al 2008 (practice-based element of dentistry): 'a feedback process that provides information that can be used to fine-tune or modify what has already been done ‘….but by whom?

Black et al 2003 (applied FA theory in case study school settings):

‘ An assessment activity can help learning if it provides information to be used as feedback by teachers, and by their students in assessing themselves and each other, to modify the teaching and learning activities in which they are engaged. Such assessment becomes formative assessment when the evidence is used to adapt the teaching work to meet learning needs ’ (p. 2). Although it may involve several methods 'it has to be in the control of the individual teacher and, for this reason, change in formative assessment practice is an integral and intimate part of a teacher's daily work ' (p. 2)

What if…

the teacher/tutor is ‘monitoring’ rather than ‘changing’?

assessment processes are purely for self-assessment?

the responses are automated?

Queries for criterion and case development :

What must be defined about teacher roles in FA processes?

What must be defined about learner roles in FA processes?

Cox et al 2008 (practice-based element of dentistry):

'a feedback process that provides information that can be used to fine-tune or modify what has already been done ‘….but by whom?

feedback Bull & McKenna 2004 (computer-assisted assessment CAA) ‘… assessments which assist learning by giving feedback which indicates how the student is progressing in terms of knowledge, skills and understanding of a subject. In CAA this often takes the form of objective questions with feedback given to the student either during or immediately after the assessment. Formative assessment may be monitored by the tutor, used purely for self-assessment, or used to contribute marks to a module grade’. (p. xiv) What if… feedback is complex (Shute 2008), or a two-way or multi-way process? we contrast Bull with Nicol & Macfarlane-Dick’s 2006 criticisms of transmission-focused feedback: ‘ feedback messages are invariably complex and difficult to decipher’ in transmission contexts - students find it hard to take appropriate actions to improve feedback tends to emphasis only cognition, not motivation and beliefs which are vital to internalization of knowledge and understanding transmission-based feedback is used in highly inventive ways, e.g. to direct learners within a carefully structured test environment towards appropriate pathways which enhance motivation and self-regulation (Winkley) Queries for criterion and case development: What types of knowledge, skills and understanding are to be included in the domain? What relationships between ‘assisting learning’ and ‘giving feedback’ are there in automated and other feedback contexts, and how can they be made explicit? How do feedback patterns fit into broader conceptual frameworks for learning and teaching? What is the contribution of more/faster/more frequent automated feedback to formative assessment ( Conole & Warburton, 2005 )?

Bull & McKenna 2004 (computer-assisted assessment CAA)

‘… assessments which assist learning by giving feedback which indicates how the student is progressing in terms of knowledge, skills and understanding of a subject. In CAA this often takes the form of objective questions with feedback given to the student either during or immediately after the assessment. Formative assessment may be monitored by the tutor, used purely for self-assessment, or used to contribute marks to a module grade’. (p. xiv)

What if…

feedback is complex (Shute 2008), or a two-way or multi-way process?

we contrast Bull with Nicol & Macfarlane-Dick’s 2006 criticisms of transmission-focused feedback:

‘ feedback messages are invariably complex and difficult to decipher’ in transmission contexts - students find it hard to take appropriate actions to improve

feedback tends to emphasis only cognition, not motivation and beliefs which are vital to internalization of knowledge and understanding

transmission-based feedback is used in highly inventive ways, e.g. to direct learners within a carefully structured test environment towards appropriate pathways which enhance motivation and self-regulation (Winkley)

Queries for criterion and case development:

What types of knowledge, skills and understanding are to be included in the domain?

What relationships between ‘assisting learning’ and ‘giving feedback’ are there in automated and other feedback contexts, and how can they be made explicit?

How do feedback patterns fit into broader conceptual frameworks for learning and teaching?

What is the contribution of more/faster/more frequent automated feedback to formative assessment ( Conole & Warburton, 2005 )?

evidence Elliott (Web 2.0) ‘ Assessment is the process of generating evidence of student learning and then making a judgement about that evidence’ Russell et al 2006 (VLE/CMC distance education) ‘ The purpose of formative assessment is to help with learning , while that of summative assessment is to judge what has been learnt . It is possible for an assessment to have both formative and summative aspects’ (p. 467). Roos and Hamilton 2005 : formative teacher assessment is often 'essentially summative' ‘taking 'snapshots of where the children have 'got to', rather than where they might be going next’ (p. 9) What if… ‘ formative assessment’ practices are really examples of evidence being generated and used for serial summative assessment? It is ‘meaningless’ (DW) to try to locate the distinction between formative and summative in the assessment event itself? Queries for criterion and case development: what counts as evidence in formative e-assessment processes? what is the difference (if any) between this evidence and evidence used for summative assessment? What processes generate assessment evidence which is for formative purposes?

Elliott (Web 2.0)

‘ Assessment is the process of generating evidence of student learning and then making a judgement about that evidence’

Russell et al 2006 (VLE/CMC distance education)

‘ The purpose of formative assessment is to help with learning , while that of summative assessment is to judge what has been learnt . It is possible for an assessment to have both formative and summative aspects’ (p. 467).

Roos and Hamilton 2005 : formative teacher assessment is often 'essentially summative' ‘taking 'snapshots of where the children have 'got to', rather than where they might be going next’ (p. 9)

What if…

‘ formative assessment’ practices are really examples of evidence being generated and used for serial summative assessment?

It is ‘meaningless’ (DW) to try to locate the distinction between formative and summative in the assessment event itself?

Queries for criterion and case development:

what counts as evidence in formative e-assessment processes?

what is the difference (if any) between this evidence and evidence used for summative assessment?

What processes generate assessment evidence which is for formative purposes?

Some pointers for the case studies ‘ E-assessment is starting to deliver measurable and successful improvements; but there is still much work to be done. One of the most important messages that can be drawn from this work is simply the variety and scope for imagining new forms of assessment – and of using technology to support it in imaginative ways ’ (Whitelock and Watt 2008, p. 153) ‘ Modernising assessment’ involves blurring the boundaries between formative and summative processes (Elliott 2008) Dylan Wiliam’s observation at the July PED that, rather than thinking in terms of ‘formative assessment’, it might be more appropriate to think in terms of how assessment can be used ‘formatively’

‘ E-assessment is starting to deliver measurable and successful improvements; but there is still much work to be done. One of the most important messages that can be drawn from this work is simply the variety and scope for imagining new forms of assessment – and of using technology to support it in imaginative ways ’ (Whitelock and Watt 2008, p. 153)

‘ Modernising assessment’ involves blurring the boundaries between formative and summative processes (Elliott 2008)

Dylan Wiliam’s observation at the July PED that, rather than thinking in terms of ‘formative assessment’, it might be more appropriate to think in terms of how assessment can be used ‘formatively’