Hatch, Match and Dispatch:
Examining the relationship between student intent,
expectations, behaviours and outcomes in six...
The Dream Team
Bodong Chen (@bodongchen)
Stian Håklev (@houshuang)
William Heikoop
Laurie Harrison
Hedieh Najafi
Carol Rol...
About Open UToronto MOOC Initiative
• University of Toronto is committed to exploring new ways of
teaching and sharing kno...
Problemshift Inc.
A small, privately held Canadian company that
specializes in the research, design and
development of ana...
Research Objectives
In the context of 6 University of Toronto MOOCs,
we examine:
Student intentions, goals and motivation ...
Research Design
Multiple case study of six MOOCs:
• Aboriginal Worldviews and Education
• Introduction to Psychology
• Lea...
Research Design
Multiple case study of six MOOCs:
• Aboriginal Worldviews and Education
• Introduction to Psychology
• Lea...
Data Sources
1. Entry Survey (demographic info, motivation,
expected engagement)
2. Student behaviour data (clickstream da...
Data Sources
1. Entry Survey (demographic info, motivation,
expected engagement)
2. Student behavior data (clickstream dat...
Data Analysis
Part I: Entry survey (n=70,418 for all MOOCs)
Principal component analysis (PCA) of surveys
to
reduce comple...
Data Analysis Part Ia:
Entry Surveys

| Open UToronto

| Problemshift Inc.
Data Analysis Part Ib: Principal
Components Analysis of Survey Data

| Open UToronto

| Problemshift Inc.
Data Analysis Part Ic: PCA continued

| Open UToronto

| Problemshift Inc.
Data Analysis Part Ic: PCA continued

| Open UToronto

| Problemshift Inc.
Data Analysis Part Ic: PCA continued
BE

be

Be

“Enjoyment”

bE

“Betterment”

| Open UToronto

| Problemshift Inc.
Data Analysis Part IIa: Clickstream Data
Info from Coursera

This is what clickstream
data looks like

| Open UToronto

| ...
Data Analysis Part IIb: Clickstream Data
Wrangling Workflow
1. Raw Clickstream
Hadoop & Hive

2. Cleaned
Clickstream

Pyth...
Data Analysis Part IIc: Clickstream Analysis
(Sequential Pattern Mining)
…we extract
frequently occurring
sequences
Using ...
Results (so far)
We compare
different
categories,
similar to
approach
taken by
Sabourin,
Mott &
Lester, 2013

| Open UToro...
Next Steps
•
•
•
•

Examine outcome measures
Repeat with other MOOCs
Refine feature identification
Better i-support and s-...
For more information visit
open.utoronto.ca
www.problemshift.com
Upcoming SlideShare
Loading in...5
×

Hatch, Match and Dispatch MRI13 Presentation

1,174

Published on

Early results from frequent sequence analysis of frequent sequence mining from Coursera clickstream files.

Published in: Education, Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,174
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
8
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Hatch, Match and Dispatch MRI13 Presentation

  1. 1. Hatch, Match and Dispatch: Examining the relationship between student intent, expectations, behaviours and outcomes in six Coursera MOOCs at the University of Toronto
  2. 2. The Dream Team Bodong Chen (@bodongchen) Stian Håklev (@houshuang) William Heikoop Laurie Harrison Hedieh Najafi Carol Rolheiser Chris Teplovs (@cteplovs) | Open UToronto | Problemshift Inc.
  3. 3. About Open UToronto MOOC Initiative • University of Toronto is committed to exploring new ways of teaching and sharing knowledge • Supporting research within the MOOC arena is a key component of the Open Utoronto initiative. • This project is part of a broader series of research studies that aim to investigate: ◦ ◦ ◦ student demographics and learning goals pedagogical approaches and design factors patterns of engagement and learning among participants | Open UToronto
  4. 4. Problemshift Inc. A small, privately held Canadian company that specializes in the research, design and development of analytics-driven technology. | Problemshift Inc.
  5. 5. Research Objectives In the context of 6 University of Toronto MOOCs, we examine: Student intentions, goals and motivation (Hatch) and their relationship (Match) to student behavior and student performance based on formative and summative assessment (Dispatch) | Open UToronto | Problemshift Inc.
  6. 6. Research Design Multiple case study of six MOOCs: • Aboriginal Worldviews and Education • Introduction to Psychology • Learn to Program: The Fundamentals • Learn to Program: Crafting Quality Code • The Social Context of Mental Health and Illness • Statistics: Making Sense of Data | Open UToronto | Problemshift Inc.
  7. 7. Research Design Multiple case study of six MOOCs: • Aboriginal Worldviews and Education • Introduction to Psychology • Learn to Program: The Fundamentals • Learn to Program: Crafting Quality Code • The Social Context of Mental Health and Illness • Statistics: Making Sense of Data | Open UToronto | Problemshift Inc.
  8. 8. Data Sources 1. Entry Survey (demographic info, motivation, expected engagement) 2. Student behaviour data (clickstream data) 3. Outcome data (quiz scores, final grades) 4. Exit survey 5. Instructor interviews | Open UToronto | Problemshift Inc.
  9. 9. Data Sources 1. Entry Survey (demographic info, motivation, expected engagement) 2. Student behavior data (clickstream data) 3. Outcome data (quiz scores, final grades) 4. Exit survey 5. Instructor interviews | Open UToronto | Problemshift Inc.
  10. 10. Data Analysis Part I: Entry survey (n=70,418 for all MOOCs) Principal component analysis (PCA) of surveys to reduce complexity of categorization. Two axes: “betterment” and “enjoyment” Part II: Clickstream data (n=16,900,926 events for Introductory Psychology only) Sequential pattern mining Part III: Comparing frequent patterns from Part II based on classification from Part I | Open UToronto | Problemshift Inc.
  11. 11. Data Analysis Part Ia: Entry Surveys | Open UToronto | Problemshift Inc.
  12. 12. Data Analysis Part Ib: Principal Components Analysis of Survey Data | Open UToronto | Problemshift Inc.
  13. 13. Data Analysis Part Ic: PCA continued | Open UToronto | Problemshift Inc.
  14. 14. Data Analysis Part Ic: PCA continued | Open UToronto | Problemshift Inc.
  15. 15. Data Analysis Part Ic: PCA continued BE be Be “Enjoyment” bE “Betterment” | Open UToronto | Problemshift Inc.
  16. 16. Data Analysis Part IIa: Clickstream Data Info from Coursera This is what clickstream data looks like | Open UToronto | Problemshift Inc.
  17. 17. Data Analysis Part IIb: Clickstream Data Wrangling Workflow 1. Raw Clickstream Hadoop & Hive 2. Cleaned Clickstream Python 4,804,167 transactions 58,691 sequences | Open UToronto 3. “Buckets” | Problemshift Inc.
  18. 18. Data Analysis Part IIc: Clickstream Analysis (Sequential Pattern Mining) …we extract frequently occurring sequences Using the ‘arulesSequences’ package in R, which implements Zaki (2001)… M. J. Zaki. (2001). SPADE: An Efficient Algorithm for Mining Frequent Sequences. Machine Learning Journal, 42, 31–60. | Open UToronto | Problemshift Inc.
  19. 19. Results (so far) We compare different categories, similar to approach taken by Sabourin, Mott & Lester, 2013 | Open UToronto Comparing the prevalence of frequently occurring sequences across the four partitions (BE, Be, bE, and be), we see that “be” shows more “return to outline” behaviours and less engagement with forums. | Problemshift Inc.
  20. 20. Next Steps • • • • Examine outcome measures Repeat with other MOOCs Refine feature identification Better i-support and s-support calculations | Open UToronto | Problemshift Inc.
  21. 21. For more information visit open.utoronto.ca www.problemshift.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×