This document summarizes a meeting of the 3rd Working Group on MOOCs at Work. The agenda included updates on the MOOCs at Work ecosystem and initiatives, presentations on evaluation frameworks for MOOCs and learner experiences with MOOCs, and a discussion on designing MOOCs. The working group's objectives were to explore MOOCs as a potential mainstream employee learning option and evaluate their suitability and how workplace learners might use them. Presenters discussed different ways MOOCs could be incorporated into learning strategies and evaluated.

1. Learning Cafe Call
MOOCs
@Work
27 July 2013
3rd Working Group
Meeting
MOOCs (Massively Open
Online Courses) can be a
mainstream employee
learning option. It offers cost
effective learning with the
benefits far outweighing the
challenges. L&D/HR need to
be proactive in exploring and
including MOOCs in learning
strategies.
1

2. MOOCs at Work Framework

3. Agenda
• MOOCS at Work Ecosystem Update
• Jeevan Joshi, Learning Café – 5 mins
• Impact challenges Platform
• John Forrest, Impact Challenges -5 mins
• An Evaluation Framework for MOOCs
• – Michelle Ockers, Coca Cola Amatil – 15 mins
• Discussion – Our experiences as MOOCs Learner
• Lisa Henderson, QBE - 15 mins
• Discussion – Designing a MOOC
• Jeevan Joshi - Learning Café – 10 mins
• Q&A and Call to Action
• 10 mins
3

4. MOOCs at Work Update
• .
• Organisational Learning
• Being explored for refresher course or
keep up to date.
• Business Model
• Bill orders Florida education officials
to study and set rules that would allow
students who have yet to enrol in
college to earn transfer credits by
taking MOOCs.
• The adoption of nontraditional forms
of education hinges on accessibility,
flexibility, and cost—not quality. Clay
Shirkey
4
• Initiative
New Members
• Qantas
• Alison Woodage
• Kirsty Smith
• MTC Training
• P Isaac
• Website being recreated.
• POC platform almost ready
• Need blogs on MOOCs
• MOOCS@Work – Big component of
LearningCafe UnConference 20 Feb
2014

5. New Khan Academy Interface
• Pretest
• Skill Levels
• Gamification
• Knowledgemaps
5

6. AN EVALUATION
FRAMEWORK FOR MOOCS
Michelle Ockers input for WG3
27 Aug 2013
6

7. Why evaluate?
• Outcome - to verify that aims and
intentions have been met
• Process – to understand the
process that achieved the
outcomes
MOOC Quality Project
http://mooc.efquel.org
“perspectives from top quality
experts on the evolution,
development and quality
assurance of MOOC-based
education”
Education-focused; offer some
perspectives on MOOC quality
assurance relevant to workplace
learning.

8. What are our aims and intentions?
• Working Group - ‘Explore’ the
Learning Café Call, understand
possible future scenarios, and
equip ourselves for it.
• Organisational L&D Practitioners -
Propose that this should be same
as any other mainstream learning
option
e.g. to positively impact performance and
business results, whilst improving
employee engagement
Propose at this stage WG
should focus on the Learning
Café call.
Learning Café Call - MOOCs
(Massively Open Online Courses)
can be a mainstream employee
learning option. It offers cost
effective learning with the benefits
far outweighing the challenges.
L&D/HR need to be proactive in
exploring and including MOOCs in
learning strategies.

9. Process – Learner use of MOOCs?
9
• Differing objectives for undertaking a MOOC & different styles of MOOC
• Goal setting – learner led vs. predefined learning outcomes
• Completion not necessarily a goal (approx. 10% completion rate)
• Some dip in and out; take bits they find engaging (low financial and
emotional commitment)
• “Drive-by” learners – take a look, see if they like anything and move on
• Vehicle for learning via discovery and experience through interaction and
connection (rather than content transmission) – can generate unexpected,
although highly inconsistent results
• Discussion points
• How do we want workplace learners to use MOOCs if they are to be a
mainstream employee learning option?
• At this stage do we just need to understand how workplace learners will
use MOOCs with different levels of direction?
“One of the central premises of MOOCs is that students need to apply their
own measurements of quality and success.” Dave Cormier, MOOC Quality
Project post (Week 3

10. Process - How might MOOCs be used in
workplace learning strategies?
• As whole courses either:
• Taken independently
• Taken with others within organisation
• With support and guidance from L&D (e.g. structured study groups)
• Integrated into internally developed learning programs
• As extended and/or refresher learning
• As performance support
• As informal / social learning – self-directed learning, with network
building
• As a source of content either to be incorporated into a learning program
• Other?
• Discussion point – for what uses does the WG we want to evaluate
MOOCS?

11. MOOC evaluation available online
Site Scope What do students rate/review?
coursetalk.org 13 providers,
>3,200 courses
1-5 stars (no set criteria ) &
commentary.
Note – scope makes this a useful
course directory
grademycourse.com Coursera, edX,
Udacity
Difficulty (easy, med, hard)
Workload – hours
Rating – A+ to F (nor set criteria)
topfreeclasses.com 1-5 stars (no set criteria) &
commentary

12. What do we want to evaluate?
Are MOOCs ready for workplace learning?
• Specific MOOCs with the aims of:
• Identifying good quality content for workplace learning (be it courses or
performance support)
• Identifying MOOC styles that best meet workplace learning needs
(70/20/10)
Are workplace learners ready for MOOCs?
• Workplace learner behaviours and outcomes using MOOCs:
• In self-directed learning situation in different styles of MOOC
• In situations where L&D guide and support the learner
Propose that to achieve the WG objectives we need to be more
deliberate in how we try to use MOOCs.
12

13. DESIGNING MOOCS
Jeevan Joshi
Learning Cafe

14. MOOCs to Learner Approaches
14
DIY Facilitated Organised
Laissez
faire
Learners
search for
MOOCs on
internet
Complete
Reporting in LMS
Learners go
to a portal set
up by L&D
Complete
Learners go to a
portal or
recommendations
pushed by LMS
Share with
internal
community
Complete
Share with
internal
community
Learners gets
personalised
recommendation
& supported by
L&D
Self Report Self Report Self Report L&D Report

15. 15

16. edX
16

17. 17

18. 18

19. 19

20. MOOCS LEARNER
EXPERIENCE
Lisa Henderson
QBE

21. MOOC learner
If you have doe MOOCs
• 1. How did you find the use of media (if any)? Any
technical issues? Any quality (design) issues?
• 2. What stood out about the presenter within the visual
media (if any)? Were they a pro or green? Did this impact
your learning experience or motivation?
• 3. How did you find the content density (includes media,
quiz, assessment)...too much to ingest or just right?

22. MOOC learner
• 4. How was your motivation, can you recall what left you
flat vs excited you?
• 5. Did your MOOC use badges to help with motivation?
Did it work?
• 6. Was your MOOC easy to manage with work / life
commitments?
• 7. Did you participate in the discussion boards? If so,
what did you use them for eg to get technical help, to
assist in problem solving, to ask a content question?
• 8. Did you have multiple assessment attempts? How did
this impact your motivation or stress levels?

23. Call to Action
23
Follow MOOCs at work
• LinkedIn Groups
• Follow on Twitter - @moocsatwork
• Sign up at
Identify your area of interest
Cost
Usually free – Certificate - $70
Commercial MOOCS - $60

24. MOOCs at Work
• MOOCs Update
• MOOCs visibility is growing
• Media
• Learner/ consumer -
• Either you know about
MOOCs or your don’t.
Academic world knows
• Course are generally Uni
based MOOCs or short paid
courses on Technology/Start
Ups
• Other topics and models are
emerging
24
Web site – moocsatwork.com
@moocsatwork
LinkedIn Open Group
Moocs for Workplace
Learning