The OLDS MOOC EvaluationPlan and an Experimentin Recognising Learning usingBadgesSlides by Simon Cross and Rebecca GalleyInstitute of Educational Technology, The Open UniversityPresented by Simon Cross as part of:Cross, S. and Mor, Y. (2013) OLDS MOOC: design,evaluation and badges, eLearning Community Event:MOOCs at the Open University: past, present and future,13 March 2013, OU, Milton KeynesFor more on the event visit: http://cloudworks.ac.uk/cloud/view/8043
Presentation overview1. OLDS MOOC Evaluation Plan (themes, structure, questions)2. OLDS MOOC Badge Strategy (what are badges, what are they for, and how could we use them)3. Use and user comments (examples of twitter posts that comment on role of badges + Bonus slide on expectations)
September 2012 Questions on Previous Experience Registration of MOOCs Questionnaire Prior understanding and knowledge OLDS n=35 n=1400+ MOOCMOOC ‘EvaluationPanel’ recruitedbefore start for Expectations and Evaluation Strategylongitudinal study Motivationsand to ensure we Questionnaire: oneare likely to interview week prior to startsome people at endwho ‘dropped out’ n=60+ n=15 MOOC STARTS Weekly manual Weekly capture and recording of use archive of all tweets statistics from made across the Cloudworks and channels Bibsonomy Recording of data Feedback form relating to badge submissions approvals n=25 at present n=80+ and rising MOOC ENDS MOOC Experience Quesitonnaire - open to all Opens March Telephone interviews with Panel and others thought to be interesting cases Opens March April 2013
Expectations and Motivation in MOOCsA review of pre-course survey results anduser posts will provide insight in to whatpeople expected of the MOOC and whythey were engaging with it. What variety of expectation is there? How does this compare to the reality of the MOOC experience? Who and when could we use this type of MOOC structure/format? How is content and structure informed by the aims and intentions of the designers?
Use of Badges and approach to rewardNine badges have been developed inconjunction with a badging strategy. Amethod of creating and awarding badgesadded to Cloudworks. What role did the badges have in motivating participants and preventing withdrawal? What can we learn about using badges to support learners and promote learning?
… September 2012 1. What are badges? 2. What are they for? 3. How could we use them?
… 1. What is a badge? My working definition: “A badge is visual public symbol that communicates to others a particular quality, achievement or affiliation possessed by the owner. It is one of a variety of ways that such recognition can take a material form.”
… 2. What are they for? The Role of the badge for receiver and issuer… As a motivator As an engager As a meaning maker As a signifier of learning objectives As a valuer As a symbol of identity As a means of association As an empower As an entrencher http://www.olds.ac.uk/blog/moocbadgingandthelearningarc
… 3. How can we use them? Additional Cloudworks technical development undertaken: •Badge unload functionality •Define approval criteria •Apply for badge •Approval (two processes) •Display on personal profile •Aligned to Mozilla badging
… 3. How can we use them? Additional Cloudworks technical development
OLDS MOOC Badges Achievement Endeavour and Progress Practice http://www.olds.ac.uk/blog/olds-moocbadgingstrategy
OLDS MOOC Participant Profile From a sample of registrations http://www.olds.ac.uk/blog/moocprofiles
Study planning [Am] interested in gaining badges, and working with colleagues in a local study group. [My personal aims]: 1. completing one week of the MOOC and applying for a badge (maybe not getting the badge but at least applying for one week:) I like the Badges, works as motivation for me. [Will] chose to get [the] Endeavour [ones] due [to] time constraints. Maybe Achievement.
A bit of fun ... even a surprise @[B]: Earned my Week 1Badge from #oldsmooc ! Unexpectedly pleased to have a reward for efforts! What fun and motivates … Wooh [I] too have got a badge from #oldsmooc_w1 how #awsome Just applied for my #oldsmooc week 1 badge. Not that it matters but .... if I get it I will print it out (in colour) and pin it on my lapel First #oldsmooc badge...what fun, feel like a Girl Guide again Thursday motivation - got my 3 weeks #oldsmooc badge” @[T] Well done and snap! Delighted to get my 3 weeks badge too - likin those badges more than expected!
Confidence, collecting, something to show Hooray for my first @openbadges! ... and badges def. do work. Im bagsing as many as I can to show that Im learning loads… I love #openbadges. See my #oldsmooc badges [x]Guiding reflective learning Impressed #oldsmooc badge linked to evidence of reflective blogging #openbadges > me too, I got a badge :-) @[R] I love the #oldsmooc badges http://t... - keep up the reporting on your learning, love it
Reason to ask for peer review Facilitator: OER developer badge is crowdverified, you need to get your peers to approve your application. dont be shy to ask! Have applied for OER developer badge, pls verify http://t... Hi Folks Just applied 4 my #oldsmooc collaborator badge http://t.co/[x] Could you do the honours?
Supporting sense of communityand checking progress Just applied for Resource Gatherer Badge...anyone else? thanks to @[D]s reminder I have got another #oldsmooc badge http://t.co/[X]... I got another #oldsmooc badge!! #oldsmooc_wk4 looks puzzling though …See how we go :) Yay! #cloudworks #badges now link to "Who has been awarded this badge?” … Just need to figure out my #mozilla backpack now :)
‘Own Path’ participants I havent qualified for a badge yet as Im pursuing a somewhat singular path through #oldsmooc -- Im OK with that “Oh - and a final word: dont let badge envy ruin your day :)” “I’m starting to really enjoy OLDSMOOC, I’ve accepted that I’m no longer on the approved route but wandering about reading, thinking about stuff and chatting with like-minded people is great learning anyway. Having said that, I got my Week 1 badge today!”
Concluding remarks • Promoting deeper and more expansive learning? • Promoting richer learning experience? • Alternatives to summative and formative assessment? • Appeal to broader range of study motivations / practice? • Help students better value their study? • Help students better communicate their achievements?
Concluding remarks Design considerations • Overjustification • Reflective practice / Affirmation • Status / Reputation • Overlearning • Goal setting / Identity • Community endorsement • Whenever • Fun and enjoyment • Collecting …
What do you want to get out of it?Immersion in an online social environment As a bonus, here is a summaryBrushing up on skills or knowledge of just some of the reasonsLearning new skills, knowledge, participation skills participants gave for wanting toInternational working with many people from many different countriesLocal working with colleagues do the MOOC. More to follow inStructured format for thinking about and designing learning the full project evaluationBeing challengedTaking time out to reflect on practiceNew networksTrying a MOOC / a (c)MOOC / new technologiesAchieving objectives Just 8 of 39 mentioned completionCompleting the courseCompleting the subject of the course - a learning projectExploring or gaining exposure to new resources / design tools / teaching tools or technologyExperiencing a MOOC and seeing how it worksEvaluating the potential of the MOOCDocumenting experience to publish, for own reflection or for studentsResearch case study or for research dataTeam work skills required / seeing how team working is made to workPeer discussion of issues around the subjects with peersPeer feedback on own learning projects and ideasCollaboration with others on developing a courseSharing and aggregating toolsComplementing formal study (post-graduate / graduate courses)Gain badgesGroup working with local colleaguesLearning how to exploit technology for teachingOverview of fieldExtending existing learnt knowledge in area previously studiedCutting edge ideas and research
This presentation was prepared on the eve of the final weekof the OLDS MOOC. A full evaluation will take place betweenMarch and May 2013 so watch out for more detailedanalysis and review at:OLDS MOOC Blog: http://www.olds.ac.uk/blogAnd personal reflection at:Simon’s Blog: http://latestendeavour.wordpress.com/Please use an appropriate reference or acknowledgement whenciting this presentation.For further information please contact:email@example.com@firstname.lastname@example.org