SlideShare a Scribd company logo
1 of 25
Using Learning Analytics to Illuminate Student 
Learning Pathways in an Online Fraction Game 
Taylor Martin, Nicole Forsgren Velasquez 
Active Learning Lab, Huntsman School of Business 
Utah State University
The Opportunity 
• The new microscope 
• Rich and growing streams of digital 
learning data 
• Better measures of learning and teaching
Teaching Fractions
http://games.cs.washington.edu/Refraction/
Visualizing Game States
Learning Gains 
• Results: Students improve (pre to post) 
after playing game 
• But… 
– Visualizations suggest different strategies 
– What about personalized learning? 
• To investigate different strategies, we use 
cluster analysis
Cluster Analysis 
• Variables 
– Number of unique board states 
– Total number of board states 
– Average time per board state 
– Number of moves until initial 1/3 board state 
– Success on game level 
• Results: 5 clusters (fussing strategies) 
– Duncan’s Multiple Range Test used to 
interpret
Cluster 1: Minimal 
• Clustering variables 
– Number of unique board states: Low 
– Total number of board states: Low 
– Average time per board state: Very High 
– # moves until initial 1/3 board state: Very High 
– Success on game level: Low
Minimal
Cluster 2: Haphazard 
• Clustering variables 
– Number of unique board states: Medium 
– Total number of board states: Very High 
– Average time per board state: Low 
– # moves until initial 1/3 board state: Very High 
– Success on game level: Low
Haphazard
Cluster 3: Explorer 
• Clustering variables 
– Number of unique board states: High 
– Total number of board states: Medium 
– Average time per board state: High 
– # moves until initial 1/3 board state: High 
– Success on game level: Medium
Explorer
Cluster 4: Strategic Explorer 
• Contrast to Haphazard 
• Clustering variables 
– Number of unique board states: Very High 
– Total number of board states: High 
– Average time per board state: Very Low 
– # moves until initial 1/3 board state: Medium 
– Success on game level: High
Strategic Explorer
Cluster 5: Careful 
• Contrast to Minimal 
• Clustering variables 
– Number of unique board states: Low 
– Total number of board states: Very Low 
– Average time per board state: Medium 
– # moves until initial 1/3 board state: Low 
– Success on game level: Very High
Careful
Learning Gains: Transfer 
• Posttest transfer score not associated with strategy 
• Strategy used is related to learning 
• If prior knowledge is medium or better: 
– Explorer strategy learned the most 
– All high-fussing strategies (strategic explorers, 
explorers, haphazard) were good 
• If prior knowledge is low: 
– Minimal strategy was better than Haphazard 
– High fussing is counterproductive
Initial Conclusions 
• Fussing at a medium level productive 
• Careful (non fussing) strategies can be 
productive, particularly with low prior 
knowledge 
• Students with low prior knowledge may 
benefit from directed activities or hints
Next Steps 
• Towards Adaptivity 
– What degree of fussing? 
– When? 
– For whom? 
• Process Analytics 
– Identify exploration sequences
Thank You! 
• activelearninglab.org 
• taylor.martin@usu.edu 
• nicolefv@usu.edu

More Related Content

More from Nicole Forsgren

More from Nicole Forsgren (20)

DevEx Essentials
DevEx EssentialsDevEx Essentials
DevEx Essentials
 
Vmware2021 why even devop nicolefv
Vmware2021 why even devop nicolefvVmware2021 why even devop nicolefv
Vmware2021 why even devop nicolefv
 
The Data Behind DevOps: What Does it Take to be a High Performer? Jenkins Wor...
The Data Behind DevOps: What Does it Take to be a High Performer? Jenkins Wor...The Data Behind DevOps: What Does it Take to be a High Performer? Jenkins Wor...
The Data Behind DevOps: What Does it Take to be a High Performer? Jenkins Wor...
 
Software as Craft
Software as CraftSoftware as Craft
Software as Craft
 
Secrets and surprises of high performance: What the data says
Secrets and surprises of high performance: What the data saysSecrets and surprises of high performance: What the data says
Secrets and surprises of high performance: What the data says
 
Tools Won't Fix Your Broken DevOps
Tools Won't Fix Your Broken DevOpsTools Won't Fix Your Broken DevOps
Tools Won't Fix Your Broken DevOps
 
If you don't know where you're going it doesn't matter how fast you get there
If you don't know where you're going it doesn't matter how fast you get thereIf you don't know where you're going it doesn't matter how fast you get there
If you don't know where you're going it doesn't matter how fast you get there
 
The Key to High Performance - What the Data Says
The Key to High Performance - What the Data SaysThe Key to High Performance - What the Data Says
The Key to High Performance - What the Data Says
 
The Data Behind DevOps: Becoming a High Performer
The Data Behind DevOps: Becoming a High PerformerThe Data Behind DevOps: Becoming a High Performer
The Data Behind DevOps: Becoming a High Performer
 
Are We There Yet? Signposts On Your Journey to Awesome
Are We There Yet? Signposts On Your Journey to AwesomeAre We There Yet? Signposts On Your Journey to Awesome
Are We There Yet? Signposts On Your Journey to Awesome
 
How Metrics Make Your DevOps Awesome
How Metrics Make Your DevOps AwesomeHow Metrics Make Your DevOps Awesome
How Metrics Make Your DevOps Awesome
 
DOES 2016 Sciencing the Crap Out of DevOps
DOES 2016 Sciencing the Crap Out of DevOpsDOES 2016 Sciencing the Crap Out of DevOps
DOES 2016 Sciencing the Crap Out of DevOps
 
The Data on DevOps: Making the Case for Awesome
The Data on DevOps: Making the Case for AwesomeThe Data on DevOps: Making the Case for Awesome
The Data on DevOps: Making the Case for Awesome
 
Sciencing the Crap Out of DevOps
Sciencing the Crap Out of DevOpsSciencing the Crap Out of DevOps
Sciencing the Crap Out of DevOps
 
2016 State of DevOps
2016 State of DevOps2016 State of DevOps
2016 State of DevOps
 
2016 metrics-as-culture
2016 metrics-as-culture2016 metrics-as-culture
2016 metrics-as-culture
 
Continuous Delivery + DevOps = Awesome
Continuous Delivery + DevOps = AwesomeContinuous Delivery + DevOps = Awesome
Continuous Delivery + DevOps = Awesome
 
DevOps: The Key to IT Performance
DevOps: The Key to IT PerformanceDevOps: The Key to IT Performance
DevOps: The Key to IT Performance
 
Nf final chef-lisa-metrics-2015-ss
Nf final chef-lisa-metrics-2015-ssNf final chef-lisa-metrics-2015-ss
Nf final chef-lisa-metrics-2015-ss
 
DevOps: Next (ChefConf 2015 keynote)
DevOps: Next (ChefConf 2015 keynote)DevOps: Next (ChefConf 2015 keynote)
DevOps: Next (ChefConf 2015 keynote)
 

Recently uploaded

Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
gajnagarg
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
ZurliaSoop
 
Gartner's Data Analytics Maturity Model.pptx
Gartner's Data Analytics Maturity Model.pptxGartner's Data Analytics Maturity Model.pptx
Gartner's Data Analytics Maturity Model.pptx
chadhar227
 
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
wsppdmt
 
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi ArabiaIn Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
ahmedjiabur940
 
Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We ...Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We ...
gajnagarg
 
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
nirzagarg
 
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
Health
 
Lecture_2_Deep_Learning_Overview-newone1
Lecture_2_Deep_Learning_Overview-newone1Lecture_2_Deep_Learning_Overview-newone1
Lecture_2_Deep_Learning_Overview-newone1
ranjankumarbehera14
 
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
HyderabadDolls
 
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
gajnagarg
 

Recently uploaded (20)

Aspirational Block Program Block Syaldey District - Almora
Aspirational Block Program Block Syaldey District - AlmoraAspirational Block Program Block Syaldey District - Almora
Aspirational Block Program Block Syaldey District - Almora
 
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
 
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
 
Gartner's Data Analytics Maturity Model.pptx
Gartner's Data Analytics Maturity Model.pptxGartner's Data Analytics Maturity Model.pptx
Gartner's Data Analytics Maturity Model.pptx
 
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
 
Digital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham WareDigital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham Ware
 
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
 
RESEARCH-FINAL-DEFENSE-PPT-TEMPLATE.pptx
RESEARCH-FINAL-DEFENSE-PPT-TEMPLATE.pptxRESEARCH-FINAL-DEFENSE-PPT-TEMPLATE.pptx
RESEARCH-FINAL-DEFENSE-PPT-TEMPLATE.pptx
 
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi ArabiaIn Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
 
20240412-SmartCityIndex-2024-Full-Report.pdf
20240412-SmartCityIndex-2024-Full-Report.pdf20240412-SmartCityIndex-2024-Full-Report.pdf
20240412-SmartCityIndex-2024-Full-Report.pdf
 
Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We ...Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We ...
 
Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now
Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book nowVadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now
Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now
 
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
 
Statistics notes ,it includes mean to index numbers
Statistics notes ,it includes mean to index numbersStatistics notes ,it includes mean to index numbers
Statistics notes ,it includes mean to index numbers
 
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
 
Lecture_2_Deep_Learning_Overview-newone1
Lecture_2_Deep_Learning_Overview-newone1Lecture_2_Deep_Learning_Overview-newone1
Lecture_2_Deep_Learning_Overview-newone1
 
Kings of Saudi Arabia, information about them
Kings of Saudi Arabia, information about themKings of Saudi Arabia, information about them
Kings of Saudi Arabia, information about them
 
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
 
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
 

Using Learning Analytics to Illuminate Student Learning Pathways in an Online Fraction Game

  • 1. Using Learning Analytics to Illuminate Student Learning Pathways in an Online Fraction Game Taylor Martin, Nicole Forsgren Velasquez Active Learning Lab, Huntsman School of Business Utah State University
  • 2.
  • 3. The Opportunity • The new microscope • Rich and growing streams of digital learning data • Better measures of learning and teaching
  • 6.
  • 7.
  • 9.
  • 10. Learning Gains • Results: Students improve (pre to post) after playing game • But… – Visualizations suggest different strategies – What about personalized learning? • To investigate different strategies, we use cluster analysis
  • 11. Cluster Analysis • Variables – Number of unique board states – Total number of board states – Average time per board state – Number of moves until initial 1/3 board state – Success on game level • Results: 5 clusters (fussing strategies) – Duncan’s Multiple Range Test used to interpret
  • 12. Cluster 1: Minimal • Clustering variables – Number of unique board states: Low – Total number of board states: Low – Average time per board state: Very High – # moves until initial 1/3 board state: Very High – Success on game level: Low
  • 14. Cluster 2: Haphazard • Clustering variables – Number of unique board states: Medium – Total number of board states: Very High – Average time per board state: Low – # moves until initial 1/3 board state: Very High – Success on game level: Low
  • 16. Cluster 3: Explorer • Clustering variables – Number of unique board states: High – Total number of board states: Medium – Average time per board state: High – # moves until initial 1/3 board state: High – Success on game level: Medium
  • 18. Cluster 4: Strategic Explorer • Contrast to Haphazard • Clustering variables – Number of unique board states: Very High – Total number of board states: High – Average time per board state: Very Low – # moves until initial 1/3 board state: Medium – Success on game level: High
  • 20. Cluster 5: Careful • Contrast to Minimal • Clustering variables – Number of unique board states: Low – Total number of board states: Very Low – Average time per board state: Medium – # moves until initial 1/3 board state: Low – Success on game level: Very High
  • 22. Learning Gains: Transfer • Posttest transfer score not associated with strategy • Strategy used is related to learning • If prior knowledge is medium or better: – Explorer strategy learned the most – All high-fussing strategies (strategic explorers, explorers, haphazard) were good • If prior knowledge is low: – Minimal strategy was better than Haphazard – High fussing is counterproductive
  • 23. Initial Conclusions • Fussing at a medium level productive • Careful (non fussing) strategies can be productive, particularly with low prior knowledge • Students with low prior knowledge may benefit from directed activities or hints
  • 24. Next Steps • Towards Adaptivity – What degree of fussing? – When? – For whom? • Process Analytics – Identify exploration sequences
  • 25. Thank You! • activelearninglab.org • taylor.martin@usu.edu • nicolefv@usu.edu

Editor's Notes

  1. If the US population demonstrated the level of illiteracy in reading and writing that we have in mathematics (let alone science), the uproar would be deafening. How many times have you been on a plane or at a party explaining that you do math, science, engineering etc education and had a high functioning adult say to you, “oh, I hate math, was never any good at it.”???
  2. Refraction based on the concept of splitting. So, in this level of refraction, we’ll split the whole laser into halves to power the spaceships. Here the student first tries a 1/3 splitter and you see the ships don’t have enough power. Then the student uses the 1/2 splitter and everyone is happy. So far, the research shows this is the best approach for developing core ideas like a fraction is one number and more and less than w/fractions and preventing misconceptions when we get to more difficult problems (e.g., answer over 1) and expanding the number system (e.g., extending to decimals, percents etc).... But, there’s not a lot of research - that’s what we’re doing. Creating games based these concepts and researching how kids learn in each concept, which are better, maybe even what order they’re best taught in... global structure for comparing proportions - allows students to make general judgments of relative magnitude like deciding whether a container is more than half full. numerical structure of splitting and doubling that produces exact quantified results. Understanding rational number is coordinating the two. Refraction is an online fraction game (& we won the Disney Learning Challenge @ SIGGRAPH last summer!) • Users arrange and split laser beams to achieve their objectives (freeing spaceships). •These tasks are created so that users are solving fraction equivalence, operations, and comparison problems, while being engaged by the environment so that they do not give up on even difficult challenges.
  3. AKA BOARD STATES