• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Moocs, learning design and assessment 12/12/12

Moocs, learning design and assessment 12/12/12



Presentation detailing research by Sarah Lambert (University of Wollongong, Australia) into the different types of Moocs, including variables, learning design and assessment. Cases are presented with ...

Presentation detailing research by Sarah Lambert (University of Wollongong, Australia) into the different types of Moocs, including variables, learning design and assessment. Cases are presented with personal comments on the experience of the learner. Presented at a staff forum on 12/12/12 - extra comments added to the presentation to reflect the information shared by participants on their experience of different Moocs.



Total Views
Views on SlideShare
Embed Views



2 Embeds 136

http://uowblogs.com 135
http://www.linkedin.com 1



Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Moocs, learning design and assessment 12/12/12 Moocs, learning design and assessment 12/12/12 Presentation Transcript

    • MOOCS and Open LearningVariables, Cases, Learning Design: Dec 2012 Research and analysis: Sarah Lambert Presented to colleagues 11/12/12
    • Open online courses• Structured subjects or courses with – A schedule and topics, and usually a start/end date – Open resources eg iTunesU, YouTube, open academic journals, open textbooks – Open forums and blogs • Some also have open aggregated webspaces that bring together material created by participants eg course-wide blog with rss feeds from participants’ blogs/tweets • Some also have private aggregated webspaces that bring together material created by participants and shared across cohort – Open assignments and final exams/assessments • Often with a peer-review component – Facilitation by teachers and/or educational researchers with support from volunteers including support to use the web components, feedback on posts and assessments
    • Moocs • A Typical Massive Online Open Courses (MOOCs) – Often offered by consortiums of wealthy universities – Not for credit (especially to their enrolled students) • Used as an optional, free, extra resource to formal learning • Can be used by separate tutoring, learning assistance orgs – Have a start and end date – Tens of thousands enroll • Curious about the topic, curious about Moocs – Minority engage, fewer complete/master topic – Media frenzy, tranformative/disruptive • “biggest change to education since Gutenberg” (Massachusetts Bay Community College president John ODonnell in reference to edX)
    • Smoocs• A typical Small Online Open Courses (SMOOCs) – Hundreds enroll, 20-30% engage, 5-10% complete assessments – Have a start and end date – Smaller and singular providers – Perhaps a greater emphasis on curriculum, incremental assessment, certification and completion • Equivalence for some professional development/in- service type courses
    • The Roman World (La Trobe - iTunes U)
    • iTunesU courses• More a lecture series than a course• No assessment or facilitation• However now being used to sequence other learning materials eg handouts• Getting close to a resource-based MOOC• A good iTunes U collection + quiz series/final exam can be relatively easily turned into a Mooc eg La Trobe’s recent iTunesU offering on Ancient Greece attracted 45K subscribers (Australian Education supplement, 8/8/12)
    • Open University Australia
    • Open University Australia• Not free, but “open” in that: – No pre-requisites – Fully Online• Access to course materials and assessment is after paying the course fee c$2500 (postgrad public health unit Nov2012) – Includes access to generic academic skills courses• Nov 2012: considering hosting Moocs for other universities
    • OERU• Aims to deliver the first for-credit course in the world• Not-for-profit consortium of universities – Contributing small number of subjects towards a Bachelor of General Studies – Resources free, assessment for credit at cost• UOW a founding partner• First subject has just gone live – Regional relations in Asia and the Pacific (AST1000) from USQ – http://wikieducator.org/AST1000
    • Variables in Open online courses• Length of course: 5-7/11-12/24 weeks• Level: school, uni-prep, uni-transition, 1st year – 4th year• Timing: co-incides with F2F/campus course vs arbitrary start-date• Blended cohort: separate online cohort vs blend of online and formal/campus cohorts• Syncronous virtual events: 0/>4/fortnightly/weekly/biweekly• Media resources: all new media vs re-use existing inc from other providers vs mix of both• Assessment for certification: n/a vs single vs mix/multiple• Formative assessment: form/posts/quiz/mix• Who owns it: Proprietory platforms and IP in resources vs Creative Commons, re-usable by anybody content• Distinctive features: these vary in each case eg cohesive website, open-textbooks, ePortfolio for all students, virtual graduation/certification events
    • Open learning casesTESTING THE VARIABLES
    • POLSC101 (Saylor.org)
    • POLSC101 (Saylor.org)• Length of course: 12 weeks• Level: 1st year• Timing: roughly same time as spring/autumn sessions• Blended cohort: separate online cohort• Syncronous virtual events: 0/>4/fortnightly/weekly/biweekly• Media resources: mostly re-use existing media from other providers• Assessment for certification: final exam• Formative assessment: form/posts plus assignments that are self- assessed by comparing to answer-sheet/sample answer• Distinctive features: – ePorfolio – cohesive website/elearning platform shows complete subject offerings which can be taken in any sequence to create individual learning ePortfolio
    • Intro to Mathematical Thinking(Coursera/Standford)
    • Intro to Mathematical Thinking• Length of course: 7 weeks• Level: uni-transition, 1st year• Timing: intentionally co-incides with F2F/campus course• Blended cohort: proactively encouraging teachers and their cohorts of campus learners to join to created a blended cohort, aiming to create blended study groups• Synchronous virtual events: nil• Media resources: mix of new and re-purposed• Assessment for certification: progress quizzes (auto marked “Problem sets”), final exam (peer reviewed)• Formative assessment: forms/study groups/, weekly assignments (worked problems, feedback by comparing answers with students in forums), practice peer review activity (to get used to the technical/website interface),• Distinctive features: – 2 Education PhD students as core facilitators, pre and post course evaluation part of their research – Pre-course survey identified c900 volunteer “teaching assistants” – open-textbook specially created for this course – Quizzes embedded in lecture videos – attempts to create mixed study groups of online/remote students plus campus-based students doing formal maths study ie peer learning • they are hoping the campus tutors will help the campus students who will in turn help the online students who have no formal virtual coach/tutor
    • Intro to Mathematical Thinking• Sarah’s experience“I started this one but dropped out halfway. Video lectures weregreat. The activities were comprehensive and provided goodopportunity for deep learning. But I was overwhelmed by the massof activity and impersonal nature of the forums. The forums weredesperately ‘on-topic’ and dry. Even the “Maths for musicians” one Ijoined. No personal connection with other learners – maybe it wouldhave helped to motivate me to stick with it. . The Professor wasnever in the forums that I saw (just weekly emails), no sense of hispresence in the website.In the end, I drowned in Greek. I got busy at home and travelled onthe weekends (instead of doing some maths.) Subject matter wentfrom “gee, I can do this” to “erg I have not got a clue”. I fell behind,and needed hours to catch up. Which I didn’t have. There were nocost or work implications for dropping it. So when the going gottough, Maths was the first thing to go.
    • Intro to Mathematical Thinking• Sarah’s experience An alarming post-script. Despite not doing the Final Exam (which I thought was compulsory), and getting a final mark of 21.3%...I was awarded a certificate “Statement of Accomplishment”!
    • (John Hopkins Bloomberg School of Public Health via Coursera)Community Change in Public Health• Length of course: 5 weeks• Level: uni-transition, 1st year• Timing: did not seem to co-incide with a semester• Blended cohort: none that I could see• Synchronous virtual events: nil• Media resources: some new, mostly re-purposed• Assessment for certification: 2 progress quizzes, final assignment(peer reviewed)• Formative assessment: fewer, well structured forums• Distinctive features: – Really well scaffolded major assignment, with 2 rich case- studies, 6 questions per case, and 2 small essays • Peer-review guided by very good rubric and marking notes • Took 2 hours to do, and 30-45minutes to mark • Each student who submitted was asked to review 4 assignments
    • Community Change in Public Health• Sarah’s experience“Totally different experience to the Maths course. Tightly alignedwith experience in my private life. This was to see if I would like moreformal study in the area in which I currently volunteer.The video lectures were great and I could apply my experience tothem, creating my own examples of applications of the theories Iwas learning. I never even bothered to use the forums.The workload was less, but engaged me more. 2 quizzes were easy,and boosted confidence. The final assignment was brilliant learningdesign, allowed me to review and apply all the theoretical models to2 new cases. Reading others’ work further extended the learning,questioning my answers in the light of other different ones.I got 91%, reduced to 73% for the lateness of my quiz attempts. Can’twait to get my certificate. I’ll be putting this on my CV if I decide toapply for formal study in the area.”
    • Anybody done one like this?• Notes: At the session there were a number of colleagues who had enrolled and/or completed a Coursera Mooc – One did a course in Astrology • Very active teacher, regular emails even after the course is over, inc offering to sign completion signatures if attending same conference • 6 hours video lectures per week, hard to digest, did not work on mobile phone • Moderators keen for answers not to be shared too early in the forums, would often delete posts • Overall positive experience however
    • Anybody done one like this? – Another colleague did a course in Environmental issues • Very large cohort from all over the globe • Forums overwhelming • At one point the forums were “hi-jacked” by environmental lobbiests promoting a particular couse – In another example of a programming Mooc, a special interest group of Python programmers were enlisted as volunteer teaching assistants in the online forums – New Stanford-based Mooc provider was noted by a colleague http://class.stanford.edu • Separate to Coursera and Venture Lab
    • PH100 Intro to Physics (Udacity) Anytime (asynchronous)Realtime / live(synchronous)
    • PH100 Intro to Physics (Udacity) Tags let youOpen filter/findforums particular topics Badges identify people/roles
    • PH100 Intro to Physics (Udacity)• Length of course: 7 weeks of content, but you can take whatever time you like to do it• Level: uni-transition, 1st year• Timing: anytime• Blended cohort: not promoted as far as I can see• Synchronous virtual events: weekly• Media resources: mostly new, high-def video• Assessment for certification: single/final exam• Formative assessment: forums/lecture quiz/problems• Distinctive features: – problem-based learning philosophy – great web platform puts real time and anytime learning tools together in one place, – links students and CVs to employees to help students get jobs (IT/computing science fields at this stage)
    • Anybody done one like this?• Notes: Nobody at the 11/12/12 staff forum had done a Udacity Mooc
    • Smooc for new uni teachers (JISC, HEA, Oxford Brookes) Course hub is a blogOptionalMoodleforums Welcome audio (Soundcloud)
    • Smooc for new uni teachers (JISC, HEA, Oxford Brookes)• Length of course: 5 weeks• Level: probably equivalent to 1st year Education course• Timing: not related to F2F course• Blended cohort: not promoted as far as I can see• Synchronous virtual events: weekly• Media resources: Mix of new and re-used• Assessment for certification: – 150 register for the Moodle site – 60 logged in and participated – 28 created their own blog to document their progress and had this linked/aggregated to the course blog – 12 completed the assessment (project)• Formative assessment: forums/lecture /problems• Distinctive features: – Students are new university teachers – Short time frame
    • POT2012 Program for Online Teaching Final assessments sharedVirtualcertificatepresentationnight
    • POT2012 Program for Online Teaching• Length of course: 2 x 12 week session ie 1 year• Level: probably equivalent to 2nd-3rd year Education course• Timing: not related to F2F course• Blended cohort: no• Synchronous virtual events: weekly• Media resources: Mix of new and re-used• Assessment for certification: – Not sure how many enrolled – 60 logged in and participated – 16 completed the assessment (project presentation)• Formative assessment: forums/lectures /problems• Distinctive features: – Students are university teachers – Long time frame, depth of learning
    • Anybody done one like this?• Notes: Nobody at the 11/12/12 staff forum had done a smaller-scale Mooc or SMooc
    • Designing a New Learning Environment(Stanford, Venture Lab)• Cedir Learning Designer Lesley Knapp is enrolled• Interview at http://blogs.uow.edu.au/openuow/• Much smaller than Coursera (c1000 enrolled)• Groupwork is central to the subject design (and the Venture Lab learning platform) – Develop a new learning environment • Following early task that describes a learning problem
    • OERU pilot Regional Relations in Asia and the Pacific (USQ)• Length of course: 1 session• Level: 1st year• Timing: same as campus cohort• Blended cohort: not sure• Synchronous virtual events: 0• Media resources: New video signposts to be created• Assessment for certification: – 3 essays and reflective journal• Formative assessment: students find/gather/share and analyse their own materials• Distinctive features: – Students find resources (‘discovery pedagogy” rather than digesting materials found for them
    • OERU pilot Mathematical Journey (Empire State College, UNY)• Length of course: 1 session• Level: 1st year/Bridging• Timing: same as campus cohort tbc• Blended cohort: tbc• Synchronous virtual events: Skype tutorials re final project• Media resources: Some new plus written resources, podcasts and other media (all OERs) - indexed for easier access• Assessment for certification: – Learning journal and ePortfolio assessed• Formative assessment: • Pre-test leads to learning plan (inc project) • Interaction with peers via ePortfolio throughout course• Distinctive features: – Individual learning plans and projects
    • International development in open learningCRACKING FOR CREDITASSESSMENT
    • Blended: Moocs + local colleges• “EdXs popular introductory computer science course from MIT will provide the backbone of a class at the community college -- a key gateway to degree programs -- with supplemental teaching and help from community college faculty on the ground.• This is where the rubber meets the road for transforming higher education. Community colleges are beset by waitlists (400,000 in California alone) and bottlenecks in important introductory courses, as well as low success rates.• If scaled-up MIT-quality teaching can help with solve those problems, MOOCS could be truly revolutionary. Massachusetts Bay Community College president John ODonnell calls edX an invention comparable to Gutenbergs printing press.”• “New frontier for scaling up online classes: credit”, The Australian, Higher Education, Nov 20, 2012 (sourced online 10/12/12) Re-use of materials between consortium partners for local assessment part of the OERU model since its conception
    • Assessment, cheating, identity• “Multiple-choice online quizzes are simple enough, but on more open-ended assignments, MOOC students now are mostly grading each others papers. When they have questions, theyre mostly asking fellow students.• "Crowd-sourced assessment" raises obvious questions. MOOC leaders are exploring artificial intelligence solutions but admit many arent fully baked. …• Cheatings another problem that suddenly matters with credit at stake. EdX is working with a testing company to arrange for proctored exams in centres around the world. Coursera says it will be easier for far-away students to let them wave an ID card and take a test in front of a webcam, proctored from afar. MOOCs wont offer those things for free. But they could cost much less than, well, the full string quartet.”• “New frontier for scaling up online classes: credit” , The Australian, Higher Education, Nov 20, 2012 (sourced online 10/12/12)
    • External review, accreditation• “Last week, Coursera, which offers classes from 34 universities, announced the American Council on Education would begin evaluating a handful of Coursera courses and could recommend other universities accept them for credit (individual colleges ultimately decide what credits to accept). Antioch University, Excelsior College and the University of Texas system are already planning to award credit for some MOOCs. …• Molly Broad, president of the American Council on Education, said MOOCs are promising, but her group will send faculty out to "kick the tires" and research whether online courses enrolling 150,000 can really be credit-worthy. Theyll talk to both students who complete and those who drop out (at edX, 80 to 95 percent who sign up dont finish the work).• A likely outcome is more blended models like the Massachusetts experiment, where MOOCs provide the backbone and resources local institutions cant offer, but local institutions still handle the one-on-one and award the credit.• Such models could be "the best of both worlds," said Coursera co-founder Daphne Koller. Versions are already in places as varied as San Jose State and the National University of Mongolia.”• “New frontier for scaling up online classes: credit “, The Australian, Higher Education, Nov 20, 2012 (sourced online 10/12/12)
    • Degree: a passport stamped by many• Date: Sunday 12 August 2012 Publication: The Age Title: Free courses from worlds top unis a swipe away in online revolution• Imagine a university degree that is like a passport: a subject from Swinburne stamped alongside another from Sydney University, with courses from overseas colleges such as Stanford or Harvard thrown in. You could earn your degree without travelling further than your laptop, and far more cheaply than on campus.• Coursera co-founder Andrew Ng, a Stanford professor, told The Sunday Age that although the venture has committed to not becoming a university, already some colleges are offering course credits for Coursera certificates. …What we have seen also is there are a few universities that are incorporating some of our material into their own classrooms, he says.
    • What are we doing to move towards open-learning?IDENTIFY AND FIND RESOURCESFOR PILOTS We have a shortlist of subjects that would work well in the open, and are talking to Faculties about workload, timing, development. Looking at pilot and ongoing resourcing, what roles needed, eg student media developers from CACS302 Creative Arts interns.
    • Want to know more? Follow our blogPublic blog http://blogs.uow.edu.au/openuow