Dr. Andreina Bloom Parisi-Amon, Teaching and Learning Team, Coursera. Learners First & Empowering Instructors: Coursera’s Approach to the Creation of the Transformative Learning Experiences

1. Coursera’s Approach to the Creation of
Transformative Learning Experiences
Learners First &
Empowered Instructors
Andreina Bloom Parisi-Amon, PhD
andreina@coursera.org

2. Four years ago, a Stanford classroom of
400...
became a global classroom of
100,000
Today, we’re a global community of
22,000,000+

3. We envision a world where anyone,
anywhere can transform their life
by accessing the world’s
best learning experience.

4. We envision a world where anyone,
anywhere can transform their life
by accessing the world’s
best learning experience.

6. 40% of active learners come from emerging economies
5M
2.5M
3.5M
3.5M
0.5M
1.5M
0.5M
1M
Global learning community

7. Age
18-24
25-34
35-44
Employment
employed
fulltime
Self- employed
unemployed
Global learner demographics
Bachelor’s
degree
Graduate
degree
Education
No degree
full time lives →
volunteer learners

8. ● 91% of millennials change jobs in < 3 years...
...and new jobs often require new skills
● 65% of tomorrow’s jobs don’t exist
● 58% of employers worldwide believe that new
college grads are inadequately prepared for
work
A few statistics …

9. Coursera learners see real impact
In 2015, researchers at the University of Pennsylvania, University of Washington, and Coursera published
the first longitudinal study of open online learning outcomes. The results, published in the Harvard Business
Review, show that these learners are seeing tangible, positive results.
52%
of learners said that career
was their key motivation
28%
of learners said that academic
advancement was their key motivation

10. Coursera learners see real impact
In 2015, researchers at the University of Pennsylvania, University of Washington, and Coursera published
the first longitudinal study of open online learning outcomes. The results, published in the Harvard Business
Review, show that these learners are seeing tangible, positive results.
52%
of learners said that career
was their key motivation
28%
of learners said that academic
advancement was their key motivation
87%
reported
career
benefits
88%
reported
academic
benefits

11. Coursera learners see real impact
In 2015, researchers at the University of Pennsylvania, University of Washington, and Coursera published
the first longitudinal study of open online learning outcomes. The results, published in the Harvard Business
Review, show that these learners are seeing tangible, positive results.
52%
of learners said that career
was their key motivation
28%
of learners said that academic
advancement was their key motivation
87%
reported
career
benefits
88%
reported
academic
benefits35%
of them found
a new job or
started their
own business
18%
of them got
credits or had
academic
prerequisites
waived

12. 80%
of OECD,
non-student,
academic-focused
learners saw
benefits
32%
of OECD
career-focused
learners saw
tangible benefits
Non-OECD learners see greater impact
36%
of non-OECD
career-focused
learners saw
tangible benefits
87%
of non-OECD,
non-student,
academic-focused
learners saw
benefits

13. We envision a world where anyone,
anywhere can transform their life
by accessing the world’s
best learning experience.

14. World-class partners across the globe
150+ partner
institutions
1000+ world-class
instructors
Top universities in six
continents:
● 60+ N. America
● 30+ Europe
● 15+ Asia

15. Helping to
create
learning
experiences
that enable
learners
to achieve
their goals 15
Teaching & Learning Team

16. 1. Synthesize data from literature and learner
engagement into general preferred practices

17. 2. Provide data and recommendations specific
to individual courses

18. 2500+
Beta testers
300+
Courses tested
in 2016
3-8
Courses tested
per week
Beta Testing

19. Our Beta Testers
Dedicated - Motivated - Lifelong learners
---

20. “This is an introduction
course, but from what I have
seen, it was a bit too easy.”
“I felt a little "cheated"
upon finding the 4th
week consisted of only a
quiz. Suggest that you
move half of week 3 into
week 4.” “I'm glad I didn't spend
money on this. As it is, I want
the past hour of my life back.”
“This course is hard,
but I will continue to
study.”
“Wonderful course as a part
of the specialization, but
not even near that good as
a separate course.”
Our learners are honest, direct and also give us good suggestions on how to improve, because they care about Coursera.

21. Analyse, summarize, prioritize.

22. 3. Make it possible to run experiments to
collect novel data: Content A/B testing
Progression Rate
Progression Rate

23. Where
they need
to be
Practice &
demonstration
opportunities
Appropriate
timing & materials
Meeting
learners where
they are
Learners First

24. Where
they need
to be
Meeting
learners where
they are
Content Strategy
Learning Objectives
Application & Assessment
Accessible Instructional Materials

25. Where
they need
to be
Meeting
learners where
they are
Content Strategy
Learning Objectives
Application & Assessment
Accessible Instructional Materials
Iteration

26. Content Strategy

27. ★ Business
★ Technology: Computer Science
★ Data Science
★ Personal & Professional Development
○ industry and content-area agnostic skills
■ Cognitive & Learning Skills
■ Personal Growth Skills
■ Career Success Skills
■ Interpersonal & Communication Skills
Content
Strategy

28. What’s your learner’s story?
Content
Strategy
Learner
need
Your
expertise
★

29. Learner Story:
Python for Data Analysis
Ashley is a senior business analyst with 2-4 years of job experience.
She’s responsible for business reporting and revenue modeling, and
primarily uses SQL and Excel to model business insights. She’d like to
learn Python to automate tasks and eventually move into more
advanced predictive modeling. Her aspiration is to master advanced
data analysis techniques and apply for a data scientist role at a
company with a strong analytics focus.
Let your learner be the guide

30. Learning Objectives

31. Learning
Objectives
★ What will our learners be like after their experience?
★ What are the desired results?
★ What should learners ...
○ feel about the subject?
○ know?
○ be able to do?
Actionable Specific Criteria for Success
Performance-Based Objectives

32. Learning Objectives:
Python for Data Analysis
After successfully completing this specialization, Ashley will be able to:
★ apply for data scientist or analyst roles that require python for data analysis
★ be comfortable programming in Python, given a data set and problem to investigate
★ implement, execute, and visualize analyses in common python packages for data
analysis, e.g. pandas, scikit learn, numpy, scipy, matplotlib, seaborn
★ communicate data results to others via Jupyter notebooks (fka iPython)
★ showcase her skills through a portfolio of data analysis projects from the
specialization

35. Application
and
Assessment

36. Learners want to master difficult material
Application
&
Assessment

37. Need scaffolded assessment opportunities to:
★ Recall their new knowledge
★ Practice applying their new skills
★ Assess their abilities (and deficiencies!)
★ Reflect on their learning
Learners want to master difficult material
Application
&
Assessment

38. Quizzes Peer Review Programming
Assignment
Application
&
Assessment
Evaluate learning at the end of an
instructional unit by comparing it
against some standard or benchmark
Monitor learner learning to provide
ongoing feedback that can be used by
to improve their mastery
Formative Summative

39. More frequent in-video quizzes correlate
with better retention
Application
&
Assessment

40. As a direct result of BT feedback,
a programming course made significant
changes including:
“In the first module, instructors talk about
polymorphism, inheritance and
abstraction. For someone who has no prior
programming experience, these are
definitely not suitable topics to start with.”
1. Created 7programming assignments
2. Added content and scaffolding materials for beginners
3. Clarified instructions, and more.

41. Accessible
Instructional Materials

42. Content Availability
Auto-cohorts resulted in high completion and content availability
Accessible
Instructional
Materials

43. Week 1
Week 1 Week 2
Accessible
Instructional
Materials Which is the winner,
in terms of learner completion of all graded assessments?

44. 44
★ Practice, practice, practice!
★ Engaging slide design: a cohesive look and feel
★ Instructors engaging with the camera
★ Mixture of video styles
Accessible
Instructional
Materials
Using video wisely

45. Content Interesting-ness
Being ‘interesting’ retains learners
Accessible
Instructional
Materials

46. ★ Be animated and passionate
★ Speak naturally and look at
the camera
★ Have good production
values
How to be interesting
Accessible
Instructional
Materials

47. Accessible
Instructional
Materials
Speaking more quickly is associated
with better retention

48. Create focused lectures
★ Length: 4-7 minutes
★ Be concise: focus on the important info
★ Chunk content into relevant building blocks
Accessible
Instructional
Materials

49. Discussion Prompts
Readings &
Resources
Slideshow
Not just videos...
Accessible
Instructional
Materials

50. A tech course team Doubled their total
lecture time average
as a result of beta testing
from 7 minutes per module
to 15 minutes per module
“Too many readings in pdf
format, too little video lectures.”

51. Accessible
Instructional
Materials

52. Accessible
Instructional
Materials

53. Iteration

54. Content Flags
Iteration
Allow teams to see what
learners flag as problematic

55. Allow teams to see what
learners have done
Iteration
Grades & Stats

56. Allow teams to see what
learners think
Iteration
Course Ratings

57. Content A/B testing
Allows teams to conduct controlled experiments on your
courses to determine pedagogical best-practices.
★ Quizzes VS Peer reviews ?
★ Graded VS Ungraded ?
★ More content VS Less content ?
★ Talking head VS Animation ?
Iteration

59. Thanks!
Any questions?

60. 60