• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
E assessment Josie Taylor

E assessment Josie Taylor






Total Views
Views on SlideShare
Embed Views



9 Embeds 189

http://mdxaltc.blogspot.co.uk 124
http://mdxaltc.blogspot.com 50
http://mdxaltc.blogspot.it 7
http://mdxaltc.blogspot.nl 2
http://mdxaltc.blogspot.ae 2
http://www.slideshare.net 1
http://mdxaltc.blogspot.com.es 1
http://mdxaltc.blogspot.in 1
http://mdxaltc.blogspot.com.au 1



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • One of the challenges for tertiary education will be the drive toward openness resulting from web 2.0/3.0 technologies. These are becoming a more and more familiar part of our lives now, and will increasingly be so in the future. This is not technological determinism – the key point is that social media are optional. You can choose not to use them if you wish, and many other means will still be available to people to achieve their goals. However institutions will be challenged by populations of students who want to use these media to serve both formal and informal learning.
  • An example of things to come …
  • What sorts of people are using this system?
  • If the people using the system are not registered with an institution, how are they defining their learning goals?
  • This is not a comprehensive view – just a few random points on a trajectory. But who is driving this progression, and how? If the institution gets involved, is it compromising learner autonomy? Or can we support that autonomy in appropriate ways?
  • But the positioning of the learner with regard to the academy, or the institution, is interestingly balanced. Who has the last say in what is valid for study? This issue is picked up in the literature around digital literacies.
  • Learners do not come as empty vessels – constructivist approaches. Texts have traditionally been the means by which institutional authority has been sustained, coupled with assessment processes. However, not only are learners breaking free from the academy, so are texts.
  • Lea and Jones highlight the fact that although we have studies of students’ position with regard to technology, we have very little in the way of detailed study of engagement.
  • The ecological approach to media use can help to an extent. At the OU we have been looking at the cultural communities that grow around our technologies (OpenLearn, iSpot, Cloudworks), and observing the activities they support. We are now doing it now for ITunesU and UTube
  • The nature of the community and the flattening of authority
  • Consistent with affinity spaces, a concept developed by Gee.
  • Informal learning in an affinity space is much more optional, much more dependent on self motivation. Students may be equally keen on some form of assessment to keep themselves motivated.
  • Things good assessment practice supports! Lets push a bit deeper into that pedagogy
  • Whitelock points us to the work of Elliott, who identifies the characteristics of Web 2.0 assessment. These are very consistent with the work of Gee and Jenkins mentioned earlier. Note the collaborative, personalised nature of the activity.
  • Bartlett-Bragg elaborates a pedagogy for self-publishing as a means of assessment focused around reflection and peer review. Whitelock advises that frameworks such as this require a supportive infrastructure because its not just a case of developing tools and promoting collaboration. Staff need to know how to capitalise on the affordancies of web 2.0. to reconceptualise their assessment practice. Students also need to know what to do next.
  • Hence the 4Ts pyramid. As tutors realise how to transform assessment to take advantage of web 2.0, so learners will be able to development their independence. However, we need to support that transfer.
  • Whitelock’s notion of ‘Advice for Action’ emphasises that if students don’t know what to do next, the value of assessment feedback/feedforward is limited. We can be more explicit about this. E.g. Open Comment
  • Whitelock and Watt have developed both Open Mentor (which supports tutors in their assessment activity) and Open Comment (which assists students in submitting better assessed pieces of writing).
  • This approach of providing open feedback is being trialled in the Arts faculty – students are able to benefit from comments prior to formal submission of assessed work.
  • What students expect to do in any learning activity is related to their conception of learning – broadly ‘passive’ or ‘active’ – and the perceived assessment requirements.
  • In this context, motivation should not be a problem.
  • Across many of the social networking applications, we can now support a much wider range of audiences on the long tail – who were increasingly discovering, recommending, and linking the OU on a range of platforms and media outlets. Its not all about what the institution can do by itself…

E assessment Josie Taylor E assessment Josie Taylor Presentation Transcript

  • Assessment, technology and learning: who is in the driving seat? Prof Josie Taylor Director, Institute of Educational Technology, The Open University Middlesex, 2011
  • The Open University, UK
    • Higher education needs to embrace a more open future. This will entail changes that are likely to be profound
    • We haven’t yet fully understood what these changes are, or what the impact will be on organisations, staff or students
  • OpenLearn at The Open University
    • 2006 – William and Flora Hewlett foundation provided the OU with funds to investigate sharing educational resources and more open approaches
  • Open Educational Resources
    • Our definition of OER:
    • “ The open provision of educational resources, enabled by information and communication technologies, for consultation, use and adaptation by a community of users for non-commercial purposes.”
  • OpenLearn
    • Designed on a model analogous to the open source software movement
    • >14 million unique visitors have used OpenLearn since 2006
    • Gradual build of user base
  • Studies by OLNet: Patrick McAndrew
    • Undertaking analysis of user behaviour.
    • The results from one of these studies ( n = 2,011) highlighted two distinct clusters of learners:
    • "volunteer" students
    • "social" learners
    McAndrew, Scanlon & Clow (2010)
  • Volunteer students
    • Volunteer students sought the content they wanted to learn from, and they expected to work through it. These learners were most interested in:
      • more content
      • tools for self-assessment
      • ways to reflect on their individual learning.
    McAndrew, Scanlon & Clow (2010)
  • Social Learners
    • Social learners were less motivated to work through the content. Rather, they seem to see learning as a way to meet people with shared interests. These learners were most interested in:
      • communication tools
      • advanced features on the website.
    McAndrew, Scanlon & Clow (2010)
  • What are these informal learners trying to do?
    • How might they frame their tasks?
    • How will they know when they have succeeded? i.e. what ‘counts’ as success?
    • What will be the quality of the experience?
    • How can we best support them?
  • Process of emancipation for new learners as confidence builds Lots of other stops along the way...
  • Learner emancipation & institutional authority
    • For learners:
      • Not ‘just’ skill/meta-skill acquisition
      • Profound developmental stages for the individual
    • For the academy:
      • what is a university for?
      • In an open world, who determines what is (or should be) of value?
      • Who holds the power to say ‘this is worthy’ or ‘this is valid’?
  • Digital Literacies Mary Lea & Robin Goodfellow
    • Learners bring a wealth of experience to bear – some appropriate, some not
    • Learners are engaged in meaning-making
    • Recognition of the central role of texts in construction of knowledge and practice of learning
    • Potential shifts of power between learners, communities and institutions
    • Role of the institution is critically important
    • Boundaries of ‘texts’ are more fluid and unstable than in previous times
  • Improving our understanding of student behaviour?
    • ‘Rich accounts in the literature of students’ use of technology ’
    • ‘No detailed or in depth examination of what students actually do in contexts when using different applications, or how meanings are being made from, and through, engagement with digital technology’
    • ‘Recognition of the central nature of texts both in the construction of knowledge and the practice of learning’
    • Lea and Jones (2011)
  • Ecological approach
    • Interrelationship among all the different communication technologies and
      • the cultural communities that grow up around them
      • the activities they support.
    • ‘ Interactivity is a property of the technology, while participation is a property of culture.’
    • Jenkins (2004)
  • Ecological Space in which learning happens Members feel some connection – they care Contribute when you want
  • Affinity Spaces Gee (2009)
  • Distinctions between formal and informal learning spaces
    • Formal education system
    • Conservative
    • Static
    • Structures to sustain are institutional
    • Remain little changed over long periods of time
    • Communities are bureaucratic and often national
    • Does not allow for easy movement in and out
    • Informal affinity space
    • Experimental
    • Innovative
    • Structures to sustain are provisional
    • Can respond to short-term needs and temporary interests
    • Communities are ad hoc and localised
    • Allows for easy moves in and out of informal learning communities
    Gee (2009)
  • Web 2.0 Technology supports open pedagogy: peer support, communication and sharing
    • Issues for Institutions
    • Blurring boundaries between formal and informal learning
    • Ways of learning are coming out of the academy
    • Emancipation is in the hands (and feet!) of learners
    • Wider sociological issues
    • Need to prepare learners for future information overload as the Web grows exponentially
    • Need to sharpen critical awareness, critical skills, and concepts of authorial voice
  • The e-Assessment Challenge Slide courtesy of D.Whitelock, 2011 … Pull of institutional reliability and accountability … Push of Constructivist Learning
  • Assessment 2.0 Denise Whitelock Characteristics of Assessment 2.0, Elliott (2008) in Whitelock (2010) Characteristic Descriptor Authentic Involving real-world knowledge and skills Personalised Tailored to the knowledge, skills and interests of each student Negotiated Agreed between the learner and the teacher Engaging Involving the personal interests of the students Recognition of existing skills Willing to accredit the student’s existing work Deep Assessing deep knowledge – not memorization Problem orientated Original tasks requiring genuine problem solving skills Collaboratively produced Produced in partnership with fellow students Peer and self assessed Involving self reflection and peer review Tool supported Encouraging the use of ICT
  • Pedagogical Framework for self-publishing with social software
    • 1. Establishment
    • Learners actively create personalised learning environments with social software e.g. weblogs, wikis, social bookmarking and aggregation .
    • 2. Interpretation
    • Learners develop a structure and adapt it to their perceived needs.
    • 3. Reflective Monologues
    • Learners publish to their software platform and establish their identity
    • 4. Reflective Dialogues
    • Learners extend their learning environment by developing social networks.
    • 5. Distributed knowledge artefacts
    • Learners collaborate with others, distribute their work, and gather artefacts for review and reflection.
    Bartlett-Bragg (2007) in Whitelock (2010)
  • The 4Ts pyramid to facilitate moving forward with Assessment Frameworks and Web 2.0 Tools Tool development Adapted from Whitelock (2010) Transformation of Assessment tasks Tool development Training of Staff Transfer of learning from assessment tasks which include Advice for Learning
  • Advice for Action
    • The role of socio-emotive content in feedback is critically important and cannot be ignored (e.g. Draper, 2009b).
    • Assessment practices that focus on self assessment and peer feedback need to develop towards ‘Advice for Action’ , i.e. stimulus advice for transformational change in students to get them to:
      • think differently
      • to reconceptualise the way they respond
      • to engage actively in the discourse
      • Whitelock (2010)
  • Open Comment
    • Automated formative assessment tool
    • Free text entry for students
    • Automated feedback and guidance
    • Open questions, divergent assessment
    • No marks awarded
    • For use by Arts Faculty
    • Whitelock & Watt (2008)
  • Stages of analysis of students’ free text entry for Open Comment:
    • Advice with respect to content (socio-emotional support stylised example):
    • STAGE 1a: DETECT ERRORS E.g. Incorrect dates, facts. (Incorrect inferences and causality is dealt with below)
      • Instead of concentrating on X, think about Y in order to answer this question
    • Recognise effort (Dweck) and encourage to have another go
      • You have done well to start answering this question but perhaps you misunderstood it. Instead of thinking about X which did not…….. Consider Y
    Whitelock & Watt (2008)
  • Computer analysis continued
      • Consider the role of Z in your answer
    • Praise what is correct and point out what is missing
      • Good but now consider the role X plays in your answer
      • Consider the role of P in your answer
    • Praise what is correct and point out what is missing
      • Yes but also consider P. Would it have produced the same result if P is neglected? … and so on
    • Several other stages of analysis not discussed here
    Whitelock & Watt (2008)
  • Role of technology Adrian Kirkwood and Linda Price
    • Student behaviour is not driven by technology per se, but by the way in which technology is used to support learning and teaching.
      • ‘ If academic staff genuinely want their students to be analytical and critical thinkers, and able to apply their learning to novel situations and transfer their learning to solve real problems … then their assessment methods should firstly, encourage the development of such abilities; and secondly, provide students with the opportunity to demonstrate that they have developed these higher order abilities.’ Scouller (1998)
    • This is as true for Web 2.0 as it was in 1998
    Kirkwood & Price (2008)
  • Student Expectations of Learning
      • Quantitative change (passive):
      • Learning as the increase in knowledge.
      • Learning as memorisation.
      • Learning as the acquisition of facts, procedures, and so on, that can be retained and/or utilised in practice.
      • Qualitative change (active):
      • Learning as the abstraction of meaning.
      • Learning as an interpretive process aimed at the understanding of reality.
      • Learning as personal development.
    Kirkwood & Price (2008)
  • Learners are emancipating themselves
    • Web 2.0 technology has the potential to enable many activities to be mediated on-line, thus making the establishment of communities much easier, much cheaper and possibly more rewarding for participants
    • Affinity spaces can be spontaneously formed around any topic, so can be rapid, responsive and flexible means to effective learning
  • openlearn KNOWLEDGE World Service Global News
  • Can assessment practice keep up?
    • Appropriate socio-emotive feedback
    • Automated, but highly relevant, feedback/forward
    • Focus on meta-level skills and their assessment
    • Showing students how to value their own work and that of peers – peer critique is highly valuable both as a giver and receiver
    • The academy no longer holds sway
  • References
    • Beaty, E., Dall’Alba, G., & Marton, F. (1997). The personal experience of learning in higher education: Changing views and enduring perspectives. In P. Sutherland (Ed.), Adult learning: A reader (pp. 150–165). London: Kogan Page.
    • Draper, S. (2009). Catalytic assessment: understanding how MCQs and EVS can foster deep learning. British Journal of Educational Technology, 40 (2), 285-293.
    • Elliott, B. (2008). Assessment 2.0: Modernising assessment in the age of Web 2.0. Scottish Qualifications Authority. Retrieved June 22, 2009, from http://www.scribd.com/doc/461041/Assessment-20
    • James Gee, Situated Language and Learning: A Critique of Traditional Schooling . New York: Routledge, 2004.
    • Henry Jenkins’ blog (accessed 21 December 2009) http://henryjenkins.org/2006/10/confronting_the_challenges_of.html
    • Kirkwood, A. and Price L. 2008. Assessment and Student Learning – a fundamental relationship and the role of information and communication technologies, Open Learning , 23 (1): 5-16 .
  • References 2
    • Lea, M., & Jones, S., (2011) Digital Literacies in Higher Education: exploring textual and technological practice, Studies in Higher Education , 36 (3)
    • McAndrew, P., Scanlon, E. and Clow, D., (2010). An Open Future for Higher Education. Educause Quarterly, 33(1)
    • Säljö, R. (1979). Learning about learning. Higher Education, 8 (4), 443–451.
    • Henry Jenkins’ blog (accessed 21 December 2009) http://henryjenkins.org/2006/10/confronting_the_challenges_of.html
    • Scouller, K. (1998). The influence of assessment method on students’ learning approaches: Multiple choice question examination versus assignment essay. Higher Education, 35 (4), 453–472.
    • Whitelock, D. and Watt, S. (2008). Reframing e-assessment: adopting new media and adapting old frameworks. Learning, Media and Technology, Vol. 33, No. 3, September 2008, pp.153–156 Routledge, Taylor & Francis Group. ISSN 1743-9884