Transforming assessment in higher education:A participatory approach to the development of a good practice framework for a...
Outline of presentation•   The project – what, who•   Issues in the literature•   Project aims and phases:    1.   Documen...
About the project      ALTC-funded priorities project:Web 2.0 authoring tools in higher education learning and teaching: n...
Project teamJenny Waycott (project manager), Faculty of Medicine, Dentistry and Health    Sciences, University of Melbourn...
Web 2.0 for learning, teaching & assessment in               higher education?O’Reilly & Battelle                        “...
Web 2.0 for learning, teaching & assessment in               higher education?Kakutani                               “jump...
Web 2.0 for learning, teaching & assessment in              higher education?• Social web activities can be substantially ...
Project aimsParticipatory approach to supporting good practice inassessment of students’ social web (Web 2.0) activities:1...
Project aimsParticipatory approach to supporting good practice inassessment of students’ social web (Web 2.0) activities:1...
1. Documenting Web 2.0 assessment practices• Online survey:  – 64 Australian academics who have assessed    students’ Web ...
Documenting Web 2.0 assessment practices        Field of Study           Number of respondents                            ...
Documenting Web 2.0 assessment practices     Type of Web 2.0 activity   Number of responsesWiki writing                   ...
Documenting Web 2.0 assessment practices      Number of students     Number of responses       enrolled in subject Less th...
Documenting Web 2.0 assessment practices   How much the assignment is   Number of responses            worth            01...
Documenting Web 2.0 assessment practices       Intended learning outcomes               Number of                         ...
What staff have said about Web 2.0 and assessment ...                     Open publishingIt’s not unusual for the musician...
What staff have said about Web 2.0 and assessment ...           Informal writing / communication stylesit’s not a formal w...
What staff have said about Web 2.0 and assessment ...              Personal identity and experienceThere a process that go...
What staff have said about Web 2.0 and assessment ...                   Co-authoring contentStudents found it challenging ...
What staff have said about Web 2.0 and assessment ...   Designing, managing, marking, reviewing the assignment[There is a ...
What staff have said about Web 2.0 and assessment...   Designing, managing, marking, reviewing the assignmentI found the b...
What staff have said about Web 2.0 and assessment ...   Designing, managing, marking, reviewing the assignmentThe assessor...
What staff have said about Web 2.0 and assessment ...                     Protecting studentsI tell the students over and ...
What staff have said about Web 2.0 and assessment ...                     Protecting studentsI certainly do what I can to ...
Project aimsParticipatory approach to supporting good practice inassessment of students’ social web (Web 2.0) activities:1...
2. Identifying principles of good practice• International advisory group: 30 members• National roundtable:  – participants...
What would good practice look like?                  AffordancesAffordances checklist ...      • Open publishing          ...
What would good practice look like?                   ProcessesProcesses checklist ...How do teachers use Web 2.0         ...
What would good practice look like?                     PoliciesPolicies checklist ...          • disability              ...
Project aimsParticipatory approach to supporting good practice inassessment of students’ social web (Web 2.0) activities:1...
3. Field-testing guidelines / improving practice                                      Cinema Studies / Criminal Law17 case...
Field-testing guidelines / improving practiceCase studies involved...• Introductory workshops• Meetings with researchers, ...
Staff reflections“I think that being involved in something like this gives you an    opportunity to step back and look at ...
Staff reflections“I separated the technology from the assessment and I think the    basics around good assessment practice...
Staff reflections“[I found] tension between wanting to promote the benefits of    Web 2.0 in terms of innovative and flexi...
4. Producing and sharing resources• Watch this space:  http://web2assessment.blogspot.com/• Bookmarks:  www.citeulike.org/...
Publications and activitiesGray, K., Thompson, C., Clerehan, R., Sheard, J., & Hamilton, M. (2008). Web   2.0 authorship: ...
AcknowledgementsProject Advisory Group•   Matthew Allen, Bill Anderson, Greg Battye, Robyn Benson, Tracey Bretag, Jenny Bu...
Upcoming SlideShare
Loading in …5
×

Waycottand thompsonascilite2010 slideshare

402 views

Published on

Slides from a presentation given by Jenny Waycott and Celia Thompson at the Ascilite Conference in Sydney in December 2010.
From the ALTC-funded project "Web 2.0 Authoring Tools in Higher Education: New Directions for Assessment and Academic Integrity"

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
402
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Waycottand thompsonascilite2010 slideshare

  1. 1. Transforming assessment in higher education:A participatory approach to the development of a good practice framework for assessing student learning through social web technologies Jenny Waycott, Kathleen Gray & Celia Thompson, The University of Melbourne Judithe Sheard & Rosemary Clerehan, Monash University Joan Richardson & Margaret Hamilton, RMIT University
  2. 2. Outline of presentation• The project – what, who• Issues in the literature• Project aims and phases: 1. Documenting Web 2.0 assessment practices 2. Identifying principles of good practice 3. Field-testing guidelines / improving practice 4. Producing and sharing resources 2
  3. 3. About the project ALTC-funded priorities project:Web 2.0 authoring tools in higher education learning and teaching: new directions for assessment and academic integrity.
  4. 4. Project teamJenny Waycott (project manager), Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne.Celia Thompson, School of Languages and Linguistics, University of Melbourne.Margaret Hamilton, School of Computer Science and IT, RMIT University.Joan Richardson, School of Business Information Technology, RMIT University.Kathleen Gray (project leader), Faculty of Medicine / Department of Information Systems, University of Melbourne.Rosemary Clerehan, Faculty of Medicine, Nursing and Health Sciences, Monash University.Judithe Sheard, Faculty of Information Technology, Monash University.
  5. 5. Web 2.0 for learning, teaching & assessment in higher education?O’Reilly & Battelle “One of the fundamental ideas underlying(2009, p. 2) Web 2.0 [is] that successful network applications are systems for harnessing collective intelligence ... a large group ofO’Reilly, T., & Battelle, J. (2009). WebSquared: Web 2.0 Five Years On. people can create a collective workSpecial Report for the Web 2.0Summit, 20-22 October , San Francisco whose value far exceeds that providedCA.http://assets.en.oreilly.com/1/event/2 by any of the individual participants”8/web2009_websquared-whitepaper.pdf
  6. 6. Web 2.0 for learning, teaching & assessment in higher education?Kakutani “jump to the summary, the video clip, the sound bite — never mind if context and nuance are lost(2010, in the process; never mind if it’s our emotions, moreparas 13-14) than our sense of reason, that are engaged; never mind if statements haven’t been properly vetted and sourced” “tweet and text one another during plays andKakutani, M. (2010, 17 movies, forming judgments before seeing the arc of the March). Texts without entire work” context. [Book review]. New York Times. “power-search for nuggets of information that http://www.nytimes.co m/2010/03/21/books/ might support their theses, saving them the time of 21mash.html?ref=book s wading through stacks of material that might prove marginal but that might have also prompted them to reconsider or refine their original thinking”
  7. 7. Web 2.0 for learning, teaching & assessment in higher education?• Social web activities can be substantially different from assessment tasks students and lecturers are used to.• Much has been written about pedagogical affordances of social web technologies.• What about assessment?
  8. 8. Project aimsParticipatory approach to supporting good practice inassessment of students’ social web (Web 2.0) activities:1. Documenting how academics are assessing students’ Web 2.0 activities:  Survey and interview teaching academics (September 2009)2. Identifying principles of good practice  Advisory group and national roundtable (November 2009)3. Field-testing guidelines / improving practice  17 case studies in learning and teaching settings (February to June 2010)4. Producing and sharing resources  Watch this space...
  9. 9. Project aimsParticipatory approach to supporting good practice inassessment of students’ social web (Web 2.0) activities:1. Documenting how academics are assessing students’ Web 2.0 activities:  Survey and interview teaching academics (September 2009)2. Identifying principles of good practice  Advisory group and national roundtable (November 2009)3. Field-testing guidelines / improving practice  17 case studies in learning and teaching settings (February to June 2010)4. Producing and sharing resources  Watch this space...
  10. 10. 1. Documenting Web 2.0 assessment practices• Online survey: – 64 Australian academics who have assessed students’ Web 2.0 activities• Follow up interviews with 22 respondents – further exploration of issues around Web 2.0 assessment.
  11. 11. Documenting Web 2.0 assessment practices Field of Study Number of respondents 16Humanities / Society & Culture 15Education 11Information Technology 9Medicine & Health 6Management & Commerce 3Other
  12. 12. Documenting Web 2.0 assessment practices Type of Web 2.0 activity Number of responsesWiki writing 32Blogging/microblogging 31Social networking 17Audio/video podcasting 16Virtual world activities 12Social bookmarking 11
  13. 13. Documenting Web 2.0 assessment practices Number of students Number of responses enrolled in subject Less than 50 21 50-100 10 101-200 9 More than 200 7 69% undergraduate and 31% postgraduate subjects
  14. 14. Documenting Web 2.0 assessment practices How much the assignment is Number of responses worth 01-10% 7 11-20% 11 21-30% 9 31-40% 6 41-50% 9 51-60% 2 61-70% 0 71-80% 3 81-90% 2 91-100% 4
  15. 15. Documenting Web 2.0 assessment practices Intended learning outcomes Number of responsesGeneric or graduate skills or attributes 35Specialised knowledge or skills required in a 29discipline or professionFoundation knowledge or skills preparatory to 28a discipline or profession
  16. 16. What staff have said about Web 2.0 and assessment ... Open publishingIt’s not unusual for the musician or his manager or someone to make a comment on the blog and to correct misinformation or thank them for an opinion or whatever and I think that is a really important lesson for [students] to learn that whatever they write they’re writing for an audience and if they’re writing for more than an audience of one that has implications
  17. 17. What staff have said about Web 2.0 and assessment ... Informal writing / communication stylesit’s not a formal writing exercise, the idea is to let them express their thoughts, reflections, interests in the different topics rather than focusing on good grammar and formal sentence structure, which I think tends to constrain a lot of essays.
  18. 18. What staff have said about Web 2.0 and assessment ... Personal identity and experienceThere a process that goes into them finding their different voices, how to share appropriately, how to write with authority. A lot of them say ‘but I’m just a student’.
  19. 19. What staff have said about Web 2.0 and assessment ... Co-authoring contentStudents found it challenging to co-create content and collaborate with other studentsHow do you mark assignments when students can change/overwrite each other’s work! Many students who contributed early found that their work was completely lost. How do you manage this process of overwriting and still contributing to the same content?
  20. 20. What staff have said about Web 2.0 and assessment ... Designing, managing, marking, reviewing the assignment[There is a lot of] work involved in setting it up and making sure all the students know how to do it. If you ask them to write an essay they just go off and write it, you don’t have to spend the first three weeks of the course teaching them about essays
  21. 21. What staff have said about Web 2.0 and assessment... Designing, managing, marking, reviewing the assignmentI found the bottom third of the class had difficulty thinking about what to post on when it was left completely up to them. ... This time around I’ll try giving them a specific topic each week that they can discuss
  22. 22. What staff have said about Web 2.0 and assessment ... Designing, managing, marking, reviewing the assignmentThe assessor is not assessing a written document, they’re assessing a page which ... is a whole labyrinth of choices and connections, so they’ve got to actually work their way through
  23. 23. What staff have said about Web 2.0 and assessment ... Protecting studentsI tell the students over and over again, that it is on the WWW, it’s not associated with the university, be careful what you put up there, make sure you are comfortable with this.
  24. 24. What staff have said about Web 2.0 and assessment ... Protecting studentsI certainly do what I can to protect [students]. I wouldn’t publish critical comments on their blogs, I don’t let other students know which ones I think are good, bad or indifferent. ... I protect their privacy to that extent.
  25. 25. Project aimsParticipatory approach to supporting good practice inassessment of students’ social web (Web 2.0) activities:1. Documenting how academics are assessing students’ Web 2.0 activities:  Survey and interview teaching academics (September 2009)2. Identifying principles of good practice  Advisory group and national roundtable (November 2009)3. Field-testing guidelines / improving practice  17 case studies in learning and teaching settings (February to June 2010)4. Producing and sharing resources  Watch this space...
  26. 26. 2. Identifying principles of good practice• International advisory group: 30 members• National roundtable: – participants included academics from diverse disciplines, educational developers, and students. – Discussions aimed to gather recommendations for good practice guidelines• Proceedings available at: http://web2assessmentroundtable.pbworks.com
  27. 27. What would good practice look like? AffordancesAffordances checklist ... • Open publishing • Communication styles and What is an appropriate fit texts between what assessment • Personal identity and is trying to achieve and experience what Web 2.0 can do? • Co-creation, collaboration, crowd-sourcing • Content management
  28. 28. What would good practice look like? ProcessesProcesses checklist ...How do teachers use Web 2.0 Designto support student, self- andorganisational learning Review Implementthroughout the cycle ofactivities involved in theassignment? Feedback Mark
  29. 29. What would good practice look like? PoliciesPolicies checklist ... • disability • access to IT services orHow can assessment using equipmentWeb 2.0 be made safe and fair • appropriate conductfor students and staff? • identity and privacy • academic honesty and integrity • special consideration • moral rights and copyright
  30. 30. Project aimsParticipatory approach to supporting good practice inassessment of students’ social web (Web 2.0) activities:1. Documenting how academics are assessing students’ Web 2.0 activities:  Survey and interview teaching academics (September 2009)2. Identifying principles of good practice  Advisory group and national roundtable (November 2009)3. Field-testing guidelines / improving practice  17 case studies in learning and teaching settings (February to June 2010)4. Producing and sharing resources  Watch this space...
  31. 31. 3. Field-testing guidelines / improving practice Cinema Studies / Criminal Law17 case studies: Blogging Cultural Studies / Media StudiesDraft guidelines Socialpilot-tested Education bookmarkingin 17 subjects Social networking Languagesat 5 universities Video sharing Business / Economicsin Victoriaduring Semester 1, Photo sharing Communication Design2010 Virtual worlds Languages Accounting / Education Wiki writing Information Technology Languages / Science Combined Information Management Web 2.0 tools Information Technology
  32. 32. Field-testing guidelines / improving practiceCase studies involved...• Introductory workshops• Meetings with researchers, class observations• Examples of marked student work, assessment artefacts, etc.• Focus groups – Staff reflecting on experience – Students’ perspective on using Web 2.0 for assessment in HE
  33. 33. Staff reflections“I think that being involved in something like this gives you an opportunity to step back and look at what you’re doing, through an outsider’s lens. In many ways our discussions with you and some of the questions you were asking gave us something to think about. There was perhaps a little bit of tinkering that could be done around the edges. I think that was really useful for us. We tend to use our intuition a lot with this type of thing. To actually have to sit down and explain to someone and justify what we’re doing and why we think it’s working – it reinforced for us that we were on the right track.”
  34. 34. Staff reflections“I separated the technology from the assessment and I think the basics around good assessment practice are still the same principles. What new technology does is just make you review things and think about it differently in terms of implementation ... We wanted to review our process because, you know, we’re putting ourselves out there. We’re still not there, but we’re putting ourselves out there to trial things [and] that leads to review”
  35. 35. Staff reflections“[I found] tension between wanting to promote the benefits of Web 2.0 in terms of innovative and flexible modes of engaging the students and I think sometimes when it comes to assessment too much innovation and flexibility can cause real problems or it certainly did in our students [...] Students really want maximum clarity. They really want to be told how many entries they’ve got to do, when they’ve got to do them and this kind of thing.”
  36. 36. 4. Producing and sharing resources• Watch this space: http://web2assessment.blogspot.com/• Bookmarks: www.citeulike.org/tag/assessment20
  37. 37. Publications and activitiesGray, K., Thompson, C., Clerehan, R., Sheard, J., & Hamilton, M. (2008). Web 2.0 authorship: Issues of referencing and citation for academic integrity. The Internet and Higher Education, 11(2), 112-118.Gray, K., Thompson, C., Sheard, J., Clerehan, R., & Hamilton, M. (2010). Students as web 2.0 authors: Implications for assessment design and conduct. Australasian Journal of Educational Technology, 26(1), 105-122.Webinar: www.transformingassessment.com/events_26_may_2010.phpWorkshops 2010-11 @ HERDSA, ATN Assessment, ASCILITE, Global Learn Asia PacificAJET Special Issue on Assessing Students’ Web 2.0 Activities in Higher Education: http://www.ascilite.org.au/ajet/about/special-issues/assess- students-web2-2011.html
  38. 38. AcknowledgementsProject Advisory Group• Matthew Allen, Bill Anderson, Greg Battye, Robyn Benson, Tracey Bretag, Jenny Buckworth, Denise Chalmers, Geoffrey Crisp, Leitha Delves, Bobby Elliott, Jacqui Ewart, Glenn Finger, Tom Franklin, Merrilyn Goos, Scott Grant, Ashley Holmes, Christopher Hughes, David Jones, Marj Kibby, Adrian Kirkwood, Mark Lee, Catherine McLoughlin, Beverley Oliver, Kaz Ross, Alison Ruth, Royce Sadler, Mary Simpson, Arthur Winzenried, Katina Zammit, Lynette Zeeng.Project Reference Group• Michael Abulencia, Robyn Benson, John Benwell, Marsha Berry, Marilys Guillemin, Laura Harris, Deborah Jones, Gregor Kennedy, Shaun Khoo, George Kotsanas, Lauren O’Dwyer, Jason Patten, Emma Read, Julianne Reid, Gordon Sanson, Cristina Varsavsky.Project Pilot-testing Group• Matthew Absolom, Anne Davies, Cathy Farrell, Scott Grant, Terry Hallahan, Michael Henderson, John Hurst, Ramon Lobato, Warren McKeown, Michael Nott, Kerry Pantzopoulos, Michele Ruyters, Michael Smith, Sandra Smith, Robyn Spence-Brown, Elizabeth Stewart, John Terrell, Jenny Weight, Lynette ZeengALTC Support for this project has been provided by the Australian Learning and Teaching Council Ltd. (www.altc.edu.au), an initiative of the Australian Government Department of Education, Employment and Workplace Relations. The views expressed in this presentation do not necessarily reflect the views of the Australian Learning and Teaching Council, or the views of individual contributors apart from the project team.

×