1
Learning Analytics
on MOOCs
T.C. Pong
Senior Advisor to the Executive Vice-President & Provost
Professor of Computer Science & Engineering
Hong Kong University of Science & Technology
Trends and Convergence
High quality content and
access to digital resources
Platforms that support
interaction and collaboration
Mobile & wireless ubiquity
Greater understanding
of learning and learner
Focus on outcomes and
mastery
Emphasis on active and
collaborative learning
Pedagogy Technology
Blended
Learning
Flipped Courses
MOOCs
Traditional
Lecture
Style
Teaching
Criticism of Lecture Style Teaching
The criticisms of lecture style teaching can be
summarized by a quote attributed to Mark Twain:
•College is a place where a professor’s lecture notes go
straight to the student’s lecture notes, without passing
through the brains of either.
• How to make teaching and learning more effective?
• Chinese proverb (Xun Zi 荀子 ):
Tell me and I will forget (闻之不若见之) ,
Show me and I will remember (见之不若知之) ,
Involve me and I will understand (知之不若行之) .
Active Learning
“Improved Learning in a Large-Enrollment Physics Class,”
Louis Deslauriers, Ellen Schelew and Carl Wieman, Science (2011).
Massive Open Online Courses
A massive open online course (MOOC) is a
type of online course aimed at large-scale
participation and open access via the Internet.
• In addition to online video lectures, learners
are involved actively in the learning process
• MOOCs go beyond just offering courses and
content. Learning analytics allow us to
understand how students learn and how
teachers can improve their teaching.
MOOC Platforms
Major MOOC platforms:
– Coursera and edX in the US
– FutureLearn in the UK
– France Université Numérique (France
Digital University) in France
– Iversity in Germany
– Open2Study in Australia
– JMOOC in Japan
– KMOOC in Korea
– XuetangX & CNMOOC in China
– …..
Massive Open Online Program
(MOOP)
7
Last October, Coursera partnered with BNY Mellon to
provide US$250,000 for the production of a Finance
Specialization:
• Learn the finance skills that every business
organization needs
• Foundational Topics
– Finance I: Accounting & Principles of Corporate Finance
– Finance II: Capital Markets & Financial Institutions
• Advanced Topics
– Focus on Capital Markets
– Focus on Valuation & Investing
Coursera – BNY Mellon
Fundamentals of Finance Specialization
Google and Coursera are looking for a forward-
thinking, innovative university to create and launch a
series of Software Engineering Specializations
• Google will be offering $300,000 to Coursera university
partners to create the specialization designed to be
accessible and affordable to students around the world
• Google will work with the winning Universities to create a
number of creative problem-solving projects for the courses
• Google will also offer guest lectures from working engineers
to explain the connections between theoretical and applied
computer science.
Coursera – Google
Software Development Specializations
Coursera Specifications
Request for Proposals
Coursera issued Request for Proposals (RFPs) in topics of high
demand among learners:
• Full Stack Web Development
• Professional Sales
• Product Management
• Social Media Marketing
• People Management
• Business Strategy
• Data for Managers
• Introduction to Analytics (Business Analytics)
• Data Analysis Fundamentals
• Video Game Design
• iOS Application Development and Design
HKUST
edX Professional Education
11
US President Obama Announces edX’s
Free Teacher Professional Development
12
Massive Open Online Degree
(MOOD)
13
MOOC Degree
14
edX – Arizona State University
Global Freshman Academy
15
Global Freshman Academy provides students the
opportunity to complete their entire university freshman year
coursework, risk free, and pay for the credit earned after
passing the course.
16
The first MBA on Coursera
from the University of Illinois
The first MBA on Coursera
from the University of Illinois
Launching for admissions in 2015, the iMBA consists of a set of
Specializations in different areas of business expertise.
• Learners can take one course, one Specialization, or go on to take a set of
Specializations in order to earn an MBA degree from the University of Illinois.
Each Specialization is a “stackable credential”.
• The iMBA curriculum will be freely available for everyone, with the option to
enroll in the paid Specialization and earn a Verified Certificate at $79 per
modules and $553 per specialization.
• Learners can choose to augment the Specializations with additional, high-
engagement interactions with peers and faculty at the University of Illinois
to earn University of Illinois credit.
• Learners who complete six Specializations along with their high engagement
extensions will be eligible to apply for admission and earn a degree from the
University of Illinois’ College of Business.
• This fully online MBA will cost less than a third of the cost of MBAs from
similar institutions with a targeted cost of around $20,000
Course Syllabus
Improving Business Finances and Operations:
• Managerial Accounting, Corporate Finance, Process Management
Strategic Management:
• Business Strategy, Marketing Management, Global Strategy
Leadership and Management:
• Executive Leadership, Designing and Managing Organizations, Fostering Creative
Thinking, Strategic Human Capital
Business Tools for Successful Execution:
• Business Statistics, data analysis and decision making, Firm-level Economics,
Financial Accounting
Business Environments and Corporate Responsibility:
• Country Level Economics, Business Ethics and Corporate responsibility, Global
Business Horizons
Entrepreneurship and Innovation:
• Investments, Entrepreneurship and Corporate renewal, Advance Marketing
Strategies
MOOC completion rates
HKUST’s MOOC Experience
Over 200,000 learners have registered and
over 10,000 have earned certificates from
HKUST MOOCs.
Blended Learning and MOOC:
Introduction to Computing with Java
• Offered as a flipped course in Spring 2014
– The course is divided into two sections of 45 and 65
students.
– Students watch online lecture videos and participate in
online quizzes before class.
– Classes start with Q&A and then redo some of the online
quizzes.
– Students then participate in group discussions and
activities.
Student groups No. of students Total views Average views
Midterm score >= 90 25 1465 1.33
Midterm score >= 80 54 2975 1.25
Midterm score >= 50 92 4846 1.20
Midterm score < 50 15 607 0.92
Student Performance vs Video
Views
Total number of videos = 44
Average video hit counts per student per video
Student groups No. of students Total views Average views
Final score >= 90 21 1276 1.38
Final score >= 80 44 2502 1.29
Final score >= 50 82 4231 1.17
Final score < 50 25 1138 1.03
Student Performance vs Video
Views
Total number of videos = 44
Average video hit counts per student per video
Blended Learning and MOOC:
Introduction to Computing with Java
• Offered as a flipped course in Spring 2014
– The course is divided into two sections of 45 and 65
students.
– Students watch online lecture videos and participate in
online quizzes before class.
– Classes start with Q&A and then redo some of the online
quizzes.
– Students then participate in group discussions and
activities.
• Offered as a MOOC on edX started in June 2014
MOOC on edX:
Introduction to Computing with Java
Introduction to Computing
with Java on edX
26
Introduction to Computing with Java:
Demographic Distribution
27
Introduction to Computing with Java:
Activities on edX platform
28
Introduction to Computing with Java:
Activities on edX platform
29
Introduction to Computing with Java:
Activities on edX platform
30
Introduction to Computing with Java:
Comments from edX discussion forum
31
Survey results:
Overall learning experience
32
Survey results
Top Students from COMP102x
Top Students with almost perfect scores
Learning Analytics on MOOCs
VisMOOC: A visual analytics tool for MOOC
developed by Prof. Huamin Qu’s research group
35
VisMOOC Demo
http://vis.cse.ust.hk/vismooc/
36
Learning Analysis on
clickstream patterns
37
Learning Analysis on
clickstream patterns
38
39
Learning Analysis on clickstream
pattern using stacked graphs
40
Learning Analysis on
Discussion Forum
41
Learning Analysis on
Discussion Forum
42
Learning Analysis on
Video Temporal Hotness
43
Learning Analysis on
Video Temporal Hotness
44
Learning analytics example:
Science of Gastronomy
Learning analytics example:
Square apple problem
45
46
Chemists Online
for Secondary School Students
Chemists Online is a collaborative project
between universities and secondary schools (or
high schools) in Hong Kong that aims at
• nurturing scientific literacy and questioning
skills of students
• providing them with exposure to university
education through blended learning
Chemists Online
for Secondary School Students
Students participated in the
lectures given by university
professors live and online.
Students participating live in a
university lecture hall.
Students participating online in a
classroom in their school.
Number of
students
% of
registered
% of
active
Registered
students
4,113 100% N/A
Active students 2,865 70% 100%
Students received
certificates
2,127 52% 74%
No of Participating Schools from Hong Kong: 143
and Macau: 1
Chemist Online Award Scheme
(Spring 2014 cohort)
Wharton-QS Stars Awards 2014
Reimagine Education
50
A Joint E-learning/MOOC Platform for
Hong Kong’s Tertiary Education Sector
51
MOOC as a Catalyst for
Enhancing Learning Experience
Using our own MOOCs
• A basis for developing blended / flipped credit-bearing
courses
– A blended learning initiative was launched in 2013 to pilot courses
in different disciplines and levels.
• Using Learning analytics to improve students’ learning
experience
– Faculty from different disciplines work together to analyze the
MOOC data and develop approaches for improving students’
learning experience.
• Outreaching to prospective undergraduate/graduate
students
53
Thank you!

Learning analytics on MOOCs

  • 1.
    1 Learning Analytics on MOOCs T.C.Pong Senior Advisor to the Executive Vice-President & Provost Professor of Computer Science & Engineering Hong Kong University of Science & Technology
  • 2.
    Trends and Convergence Highquality content and access to digital resources Platforms that support interaction and collaboration Mobile & wireless ubiquity Greater understanding of learning and learner Focus on outcomes and mastery Emphasis on active and collaborative learning Pedagogy Technology Blended Learning Flipped Courses MOOCs Traditional Lecture Style Teaching
  • 3.
    Criticism of LectureStyle Teaching The criticisms of lecture style teaching can be summarized by a quote attributed to Mark Twain: •College is a place where a professor’s lecture notes go straight to the student’s lecture notes, without passing through the brains of either. • How to make teaching and learning more effective? • Chinese proverb (Xun Zi 荀子 ): Tell me and I will forget (闻之不若见之) , Show me and I will remember (见之不若知之) , Involve me and I will understand (知之不若行之) .
  • 4.
    Active Learning “Improved Learningin a Large-Enrollment Physics Class,” Louis Deslauriers, Ellen Schelew and Carl Wieman, Science (2011).
  • 5.
    Massive Open OnlineCourses A massive open online course (MOOC) is a type of online course aimed at large-scale participation and open access via the Internet. • In addition to online video lectures, learners are involved actively in the learning process • MOOCs go beyond just offering courses and content. Learning analytics allow us to understand how students learn and how teachers can improve their teaching.
  • 6.
    MOOC Platforms Major MOOCplatforms: – Coursera and edX in the US – FutureLearn in the UK – France Université Numérique (France Digital University) in France – Iversity in Germany – Open2Study in Australia – JMOOC in Japan – KMOOC in Korea – XuetangX & CNMOOC in China – …..
  • 7.
    Massive Open OnlineProgram (MOOP) 7
  • 8.
    Last October, Courserapartnered with BNY Mellon to provide US$250,000 for the production of a Finance Specialization: • Learn the finance skills that every business organization needs • Foundational Topics – Finance I: Accounting & Principles of Corporate Finance – Finance II: Capital Markets & Financial Institutions • Advanced Topics – Focus on Capital Markets – Focus on Valuation & Investing Coursera – BNY Mellon Fundamentals of Finance Specialization
  • 9.
    Google and Courseraare looking for a forward- thinking, innovative university to create and launch a series of Software Engineering Specializations • Google will be offering $300,000 to Coursera university partners to create the specialization designed to be accessible and affordable to students around the world • Google will work with the winning Universities to create a number of creative problem-solving projects for the courses • Google will also offer guest lectures from working engineers to explain the connections between theoretical and applied computer science. Coursera – Google Software Development Specializations
  • 10.
    Coursera Specifications Request forProposals Coursera issued Request for Proposals (RFPs) in topics of high demand among learners: • Full Stack Web Development • Professional Sales • Product Management • Social Media Marketing • People Management • Business Strategy • Data for Managers • Introduction to Analytics (Business Analytics) • Data Analysis Fundamentals • Video Game Design • iOS Application Development and Design HKUST
  • 11.
  • 12.
    US President ObamaAnnounces edX’s Free Teacher Professional Development 12
  • 13.
    Massive Open OnlineDegree (MOOD) 13
  • 14.
  • 15.
    edX – ArizonaState University Global Freshman Academy 15 Global Freshman Academy provides students the opportunity to complete their entire university freshman year coursework, risk free, and pay for the credit earned after passing the course.
  • 16.
    16 The first MBAon Coursera from the University of Illinois
  • 17.
    The first MBAon Coursera from the University of Illinois Launching for admissions in 2015, the iMBA consists of a set of Specializations in different areas of business expertise. • Learners can take one course, one Specialization, or go on to take a set of Specializations in order to earn an MBA degree from the University of Illinois. Each Specialization is a “stackable credential”. • The iMBA curriculum will be freely available for everyone, with the option to enroll in the paid Specialization and earn a Verified Certificate at $79 per modules and $553 per specialization. • Learners can choose to augment the Specializations with additional, high- engagement interactions with peers and faculty at the University of Illinois to earn University of Illinois credit. • Learners who complete six Specializations along with their high engagement extensions will be eligible to apply for admission and earn a degree from the University of Illinois’ College of Business. • This fully online MBA will cost less than a third of the cost of MBAs from similar institutions with a targeted cost of around $20,000
  • 18.
    Course Syllabus Improving BusinessFinances and Operations: • Managerial Accounting, Corporate Finance, Process Management Strategic Management: • Business Strategy, Marketing Management, Global Strategy Leadership and Management: • Executive Leadership, Designing and Managing Organizations, Fostering Creative Thinking, Strategic Human Capital Business Tools for Successful Execution: • Business Statistics, data analysis and decision making, Firm-level Economics, Financial Accounting Business Environments and Corporate Responsibility: • Country Level Economics, Business Ethics and Corporate responsibility, Global Business Horizons Entrepreneurship and Innovation: • Investments, Entrepreneurship and Corporate renewal, Advance Marketing Strategies
  • 19.
    MOOC completion rates HKUST’sMOOC Experience Over 200,000 learners have registered and over 10,000 have earned certificates from HKUST MOOCs.
  • 20.
    Blended Learning andMOOC: Introduction to Computing with Java • Offered as a flipped course in Spring 2014 – The course is divided into two sections of 45 and 65 students. – Students watch online lecture videos and participate in online quizzes before class. – Classes start with Q&A and then redo some of the online quizzes. – Students then participate in group discussions and activities.
  • 21.
    Student groups No.of students Total views Average views Midterm score >= 90 25 1465 1.33 Midterm score >= 80 54 2975 1.25 Midterm score >= 50 92 4846 1.20 Midterm score < 50 15 607 0.92 Student Performance vs Video Views Total number of videos = 44 Average video hit counts per student per video
  • 22.
    Student groups No.of students Total views Average views Final score >= 90 21 1276 1.38 Final score >= 80 44 2502 1.29 Final score >= 50 82 4231 1.17 Final score < 50 25 1138 1.03 Student Performance vs Video Views Total number of videos = 44 Average video hit counts per student per video
  • 23.
    Blended Learning andMOOC: Introduction to Computing with Java • Offered as a flipped course in Spring 2014 – The course is divided into two sections of 45 and 65 students. – Students watch online lecture videos and participate in online quizzes before class. – Classes start with Q&A and then redo some of the online quizzes. – Students then participate in group discussions and activities. • Offered as a MOOC on edX started in June 2014
  • 24.
    MOOC on edX: Introductionto Computing with Java
  • 25.
  • 26.
    26 Introduction to Computingwith Java: Demographic Distribution
  • 27.
    27 Introduction to Computingwith Java: Activities on edX platform
  • 28.
    28 Introduction to Computingwith Java: Activities on edX platform
  • 29.
    29 Introduction to Computingwith Java: Activities on edX platform
  • 30.
    30 Introduction to Computingwith Java: Comments from edX discussion forum
  • 31.
  • 32.
  • 33.
    Top Students fromCOMP102x Top Students with almost perfect scores
  • 34.
    Learning Analytics onMOOCs VisMOOC: A visual analytics tool for MOOC developed by Prof. Huamin Qu’s research group
  • 35.
  • 36.
  • 37.
  • 38.
  • 39.
    39 Learning Analysis onclickstream pattern using stacked graphs
  • 40.
  • 41.
  • 42.
  • 43.
  • 44.
  • 45.
  • 46.
  • 47.
    Chemists Online for SecondarySchool Students Chemists Online is a collaborative project between universities and secondary schools (or high schools) in Hong Kong that aims at • nurturing scientific literacy and questioning skills of students • providing them with exposure to university education through blended learning
  • 48.
    Chemists Online for SecondarySchool Students Students participated in the lectures given by university professors live and online. Students participating live in a university lecture hall. Students participating online in a classroom in their school.
  • 49.
    Number of students % of registered %of active Registered students 4,113 100% N/A Active students 2,865 70% 100% Students received certificates 2,127 52% 74% No of Participating Schools from Hong Kong: 143 and Macau: 1 Chemist Online Award Scheme (Spring 2014 cohort)
  • 50.
    Wharton-QS Stars Awards2014 Reimagine Education 50
  • 51.
    A Joint E-learning/MOOCPlatform for Hong Kong’s Tertiary Education Sector 51
  • 52.
    MOOC as aCatalyst for Enhancing Learning Experience Using our own MOOCs • A basis for developing blended / flipped credit-bearing courses – A blended learning initiative was launched in 2013 to pilot courses in different disciplines and levels. • Using Learning analytics to improve students’ learning experience – Faculty from different disciplines work together to analyze the MOOC data and develop approaches for improving students’ learning experience. • Outreaching to prospective undergraduate/graduate students
  • 53.