Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Harnessing big data and videogames to 
deliver personalized cognitive training
Sponsors 
Welcome!
Harnessing big data and videogames to 
deliver personalized cognitive training 
Chaired by: Zack Lynch, 
Executive Directo...
Harnessing big data and videogames to 
deliver personalized cognitive training 
Dr. Joe Hardy, 
VP of R&D at Lumos Labs
Insights and Personalization in 
Cognitive Training with Big Data 
LUMOS LABS, INC. 
Joe Hardy, MBA, PhD 
VP of R&D, Lumos...
LUMOS LABS, INC. 
Agenda 
1. The Lumosity data set 
2. Insights I: Lifestyle factors and cognitive performance 
3. Insight...
LUMOS LABS, INC. 
The Lumosity Dataset: 
Cognitive Performance 
Over 60 million users 
Over 2 billion gameplays 
Cognitive...
The Lumosity Dataset: 
Date of Birth 
Gender 
Education Level 
Profession 
Health and lifestyle 
Real world cognition 
Soc...
LUMOS LABS, INC. 
Sternberg, Ballard, Hardy, Katz, Doraiswamy & Scanlon 
(2013). Frontiers in Human Neuroscience. 
B 
C 
F...
LUMOS LABS, INC. 
Insights II: 
Job types and cognitive performance 
N = 85,028
LUMOS LABS, INC. 
Personalization 
1. In-game adaptivity 
- Up/Down difficulty mechanisms 
- Moment-to-moment personalizat...
Personalization should optimize for 
LUMOS LABS, INC. 
transfer to untrained tasks 
Improvement in untrained cognitive tas...
Harnessing big data and videogames to 
deliver personalized cognitive training 
Mark Baxter, 
Head of Products (FitBrains)...
Harnessing videogames and big data to deliver 
personalized cognitive training. 
October 28, 2014 
© 2014 Rosetta Stone. N...
About me. 
• Mark Baxter 
Co-founder & Head of Product at Fit Brains / Rosetta Stone 
• Background in Psychology, Videogam...
About Fit Brains/Rosetta Stone. 
Fit Brains provides fun and engaging brain training activities for all ages, 
from childr...
Fit Brains Training System. 
Fit Brains is designed to offer COMPLEX, NOVEL, and DIVERSE cognitive 
stimulation to help bu...
The Value of “Big Data”. 
Big data allows us to gain insight into subtle patterns and correlations 
within large data sets...
Videogames and Cognitive Training. 
Increased levels of interest/arousal can contribute to higher levels of 
brain functio...
Combining Big Data and Videogames. 
The successful combination of videogames and big data allows us to 
offer brain traini...
Accessibility to Wider Audiences. 
The ubiquitous video game mantra: “easy to learn, hard to master” is a 
good fit with c...
Example Data: Improvement Potential. 
These charts show that the potential for user improvement DECREASES 
with Memory Gam...
Targeting User Needs Effectively. 
Adaptive brain training systems can automatically adjust to each user, 
similar to how ...
Increasing Participation and Long-term Engagement. 
"Big data" allows us to more effectively smooth out difficulty progres...
Example Data: Difficulty Progression – Beginner Level. 
By examining the scoring data from each of the games across differ...
Q & A 
© 2014 Rosetta Stone. No reproduction of any contents without explicit written permission from Rosetta Stone, Inc. ...
Harnessing big data and videogames to 
deliver personalized cognitive training 
Itmar Lesuisse, 
CEO of Peak 
© 2014 Roset...
28 
ENJOY 
BECOMING 
BETTER 
EVERY DAY
Big data and cognitive training 
The science of motivation, adaptation 
and habit forming design 
29
30
Capture Insight Action 
31
Capture Insight Action 
32
time 
33 
sleep activity
Capture Insight Action 
34
Peak insights 
35 
WE ARE CREATING A DASHBOARD FOR YOUR BRAIN
36
Capture Insight Action 
37
Fostering motivation with personalized 
challenges and goals 
38
Tackling weaknesses with 
high-intensity workouts
Better working habits 
40
SLeep smart 
41
42
Capture Insight Action 
43
Let’s work together 
44 
scientists and 
researchers 
trackers and 
sensors 
design 
partners 
SCIENCE@PEAK.NET
Harnessing big data and videogames to 
deliver personalized cognitive training 
Aki Nikolaidis, 
NSF Fellow at the Univers...
Cognitive Enhancement and 
Neuroplasticity 
Aki Nikolaidis- University of Illinois 
Champaign Urbana 
Twitter- @AkiNikolai...
Empowerment Technology
Empowerment Technology 
• Democracy- control over governance 
Photos: www.brainpickings.org/ 
http://cdn.patch.com/
Empowerment Technology 
• Democracy- control over governance 
• Internet & Social Media- empowering the 
marginalized 
Pho...
Cognitive Enhancement 
• Empowerment Technology 
• Potentially large societal benefit
Making it Work 
• How?
Making it Work 
• How? 
• We need answers to two questions: 
– How does brain plasticity lead to learning? 
– How does lea...
How Does Brain Plasticity Lead to 
Learning? 
• Reorganization of brain’s 
computations during 
learning 
www.nicabm.com
How Does Brain Plasticity Lead to 
Learning? 
• Reorganization of brain’s 
computations during 
learning 
• Complexity is ...
How Does Learning Lead to Transfer? 
• Failures to replicate 
• What about learning leads to transfer?
How Does Learning Lead to Transfer? 
• Failures to replicate 
• What about learning leads to transfer? 
• Ways to change t...
How Does Learning Lead to Transfer? 
• Failures to replicate 
• What about learning leads to transfer? 
• Ways to change t...
Space Fortress 
Vo et al 2011; PLOS One
Space Fortress 
Nikolaidis et al. 2014, Frontiers in Human 
Neuroscience 
• Does training 
induced plasticity 
drive trans...
Space Fortress 
Nikolaidis et al. 2014, Frontiers in Human 
Neuroscience 
• Does training 
induced plasticity 
drive trans...
Training Strategy & Network Plasticity 
• Two Strategies- 
– Variable Priority 
– Fixed Priority 
• Two Networks 
– Execut...
Moving Forward 
• Double Dissociation 
– Executive Plasticity Predicts Variable Priority 
Learning 
– Subcortical Plastici...
Where do we go from here? 
• We are on the right track, but 
– Need to solidify findings 
• How can we increase brain plas...
Acknowledgements 
• Art Kramer 
• LBC Lab 
– Michelle Voss 
– Hyunkyu Lee 
– Loan Vo 
– Drew Goatz 
• Funding 
– NSF, ONR,...
Thank you 
– Google+ & LinkedIn: Aki Nikolaidis 
– Twitter @AkiNikolaidis 
– Email- gnikola2@illinois.edu
 To learn more, visit SharpBrains.com
Upcoming SlideShare
Loading in …5
×

Har­ness­ing big data and videogames to deliver per­son­al­ized cog­ni­tive training

7,184 views

Published on

(Session held at the 2014 SharpBrains Virtual Summit; October 28-30th, 2014)

10:00–11:30am. Har­ness­ing big data and videogames to deliver per­son­al­ized cog­ni­tive training
- Dr. Joe Hardy, ‎VP of R&D at Lumos Labs
- Mark Bax­ter, Head of Prod­uct (Fit­Brains) at Rosetta Stone
- Ita­mar Lesu­isse, CEO of Peak
- Aki Niko­laidis, NSF Fel­low at the Uni­ver­sity of Illi­nois Cham­paign Urbana
- Chair: Zack Lynch, Exec­u­tive Direc­tor of the Neu­rotech­nol­ogy Indus­try Organization

Learn more here:
http://sharpbrains.com/summit-2014/agenda/

Published in: Data & Analytics

Har­ness­ing big data and videogames to deliver per­son­al­ized cog­ni­tive training

  1. 1. Harnessing big data and videogames to deliver personalized cognitive training
  2. 2. Sponsors Welcome!
  3. 3. Harnessing big data and videogames to deliver personalized cognitive training Chaired by: Zack Lynch, Executive Director of the Neurotechnology Industry Organization Dr. Joe Hardy, VP of R&D at Lumos Labs Mark Baxter, Head of Products (FitBrains) at Rosetta Stone Itmar Lsuisse, CEO of Peak Aki Nikolaidis,NSF Fellow at the University of Illinois Champaign Urbana
  4. 4. Harnessing big data and videogames to deliver personalized cognitive training Dr. Joe Hardy, VP of R&D at Lumos Labs
  5. 5. Insights and Personalization in Cognitive Training with Big Data LUMOS LABS, INC. Joe Hardy, MBA, PhD VP of R&D, Lumosity
  6. 6. LUMOS LABS, INC. Agenda 1. The Lumosity data set 2. Insights I: Lifestyle factors and cognitive performance 3. Insights II: Job types and cognitive performance 4. Personalization of training
  7. 7. LUMOS LABS, INC. The Lumosity Dataset: Cognitive Performance Over 60 million users Over 2 billion gameplays Cognitive training games 40+ exercises Brain Performance Test An online, repeatable assessment battery
  8. 8. The Lumosity Dataset: Date of Birth Gender Education Level Profession Health and lifestyle Real world cognition Social behavior Personality LUMOS LABS, INC. Profile Surveys Location Reported city and state IP-based estimate Location-related covariates Additional data
  9. 9. LUMOS LABS, INC. Sternberg, Ballard, Hardy, Katz, Doraiswamy & Scanlon (2013). Frontiers in Human Neuroscience. B C FIGURE 2 | (A) Exercises used in the analysis of the health and lifestyle survey. (B) The effect of reported sleep on game performance. (C) The effect of reported alcohol intake on game performance (controlling for age, gender, and level of education). quadratic effects of alcohol for all three tasks. Low to moder-ate alcohol intake was associated with better performance in all as alcohol intake increased from there. The presence of nega-tivequadraticeffectsfor both predictorsindicated that theeffects Insights I: Lifestyle factors and Cognitive performance N= 162,462 N= 161,717 N= 127,048
  10. 10. LUMOS LABS, INC. Insights II: Job types and cognitive performance N = 85,028
  11. 11. LUMOS LABS, INC. Personalization 1. In-game adaptivity - Up/Down difficulty mechanisms - Moment-to-moment personalization 2. Domain preferences - e.g., interest in speed, attention, memory 3. Game ratings 4. Balance over time
  12. 12. Personalization should optimize for LUMOS LABS, INC. transfer to untrained tasks Improvement in untrained cognitive tasks as a function …hours trained on Lumosity of… …improvement on Lumosity N= 32, 792 Sternberg, Hardy, Ballard & Scanlon (2013). Measuring changes in cognitive performance with a reliable online assessment battery. Society for Neurosciences Annual Meeting.
  13. 13. Harnessing big data and videogames to deliver personalized cognitive training Mark Baxter, Head of Products (FitBrains) at Rosetta Stone © 2014 Rosetta Stone. No reproduction of any contents without explicit written permission from Rosetta Stone, Inc.
  14. 14. Harnessing videogames and big data to deliver personalized cognitive training. October 28, 2014 © 2014 Rosetta Stone. No reproduction of any contents without explicit written permission from Rosetta Stone, Inc.
  15. 15. About me. • Mark Baxter Co-founder & Head of Product at Fit Brains / Rosetta Stone • Background in Psychology, Videogame Production & Game/Interaction Design • 16+ years in Digital Media and Videogames • Diverse experience in: – AAA Console Games – Multiplayer Games – Serious/Educational Games – Social Games – Mobile Games © 2014 Rosetta Stone. No reproduction of any contents without explicit written permission from Rosetta Stone, Inc. 15
  16. 16. About Fit Brains/Rosetta Stone. Fit Brains provides fun and engaging brain training activities for all ages, from children to older adults, that can be played anytime, anywhere. • Fit Brains was founded in 2007 – Acquired by Rosetta Stone at the beginning of 2014 as a part of Rosetta’s expanded learning initiative that includes Language Learning, Brain Fitness & Literacy for all ages. • Combine Fun + Brain Training through our unique DNA: Neuropsychology (Dr. Paul Nussbaum) & Videogames (Mark Baxter) © 2014 Rosetta Stone. No reproduction of any contents without explicit written permission from Rosetta Stone, Inc. 16
  17. 17. Fit Brains Training System. Fit Brains is designed to offer COMPLEX, NOVEL, and DIVERSE cognitive stimulation to help build new connections within the brain. • Targets 6 major brain areas: Memory, Concentration, Problem Solving, Visuospatial, Processing Speed & Language – divided into 30+ cognitive sub-measures (e.g. free recall vs. delayed recall) • 60+ brain activities to provide a wide diversity of mental stimulation – multiple levels of skill per activity to ensure maximum stimulation – utilizes an adaptive learning system to scale challenges to each user • Train anywhere to increase accessibility and regularity of training – Fit Brains can be used on nearly any modern smart device, including PC/Mac, iOS, Android, Surface etc. • Has more than 11M users with over 1B points of training data © 2014 Rosetta Stone. No reproduction of any contents without explicit written permission from Rosetta Stone, Inc. 17
  18. 18. The Value of “Big Data”. Big data allows us to gain insight into subtle patterns and correlations within large data sets that we wouldn’t otherwise be able to observe. – EXAMPLE: understanding which kinds of brain activity decline the fastest within different age brackets, which in turn can allow us to fine-tune the recommended areas of stimulation for each demographic. • Also allows us to fine-tune the user experience to help reduce potential barriers to participation • Allows us to more accurately provide relevant levels of cognitive stimulation to specific user populations divided by: – age – gender – education – medical condition – additional cognitive or lifestyle factors © 2014 Rosetta Stone. No reproduction of any contents without explicit written permission from Rosetta Stone, Inc. 18
  19. 19. Videogames and Cognitive Training. Increased levels of interest/arousal can contribute to higher levels of brain function for associated activities. When we enjoy what we are doing, we have more potential to function at a higher level. • Video games are designed to be entertaining • Good video games generally share common traits: – They are visceral – The present layered challenges – They reward the user for progress towards and achievement of core goals • By carefully weaving these elements into cognitive training, we are able to enhance user motivation, which makes it easier for users to: – Engage more frequently – For longer sessions – To stick with a program for years, rather than just months © 2014 Rosetta Stone. No reproduction of any contents without explicit written permission from Rosetta Stone, Inc. 19
  20. 20. Combining Big Data and Videogames. The successful combination of videogames and big data allows us to offer brain training products that are: 1. More accessible to wider audiences. 2. More effective at targeting individual user needs. 1. More effective at increasing regular participation and long-term retention. © 2014 Rosetta Stone. No reproduction of any contents without explicit written permission from Rosetta Stone, Inc. 20
  21. 21. Accessibility to Wider Audiences. The ubiquitous video game mantra: “easy to learn, hard to master” is a good fit with cognitive training, which needs to be accessible to new users, and also scale to meet the deeper needs of long-term users. • Data analysis helps us to fine-tune the difficulty progression of each cognitive activity to ensure they are all relevant to the widest possible audiences – Improved Accessibility: A broader range of users with varying levels of skill are able enter into brain training programs without finding it overly difficult – Broader Challenge Spectrum: Provide a wide spectrum of activities that span from beginner skill levels to advanced skill levels – Increased Challenge Longevity: Provide maximum stimulation for those who have been training for a long time © 2014 Rosetta Stone. No reproduction of any contents without explicit written permission from Rosetta Stone, Inc. 21
  22. 22. Example Data: Improvement Potential. These charts show that the potential for user improvement DECREASES with Memory Games (as age increases), while the potential for improvement INCREASES with Problem Solving Games. © 2014 Rosetta Stone. No reproduction of any contents without explicit written permission from Rosetta Stone, Inc. 22 Problem Solving Games Memory Games
  23. 23. Targeting User Needs Effectively. Adaptive brain training systems can automatically adjust to each user, similar to how Netflix or Amazon provides relevant personalized recommendations through demographic analysis. • Personalization through “adaptive scaling”: – Each cognitive activity begins with simple tasks, and as the user’s skill level increases, it automatically scales the difficulty level to match the user – Each cognitive activity has 3 skill levels (beginner, intermediate & advanced) • Every user is provided with activities that match their current ability levels, which in turn maximizes the effectiveness of their brain activity • This adaptation ensures a user is usually doing the hardest, most stimulating activity they are capable of in a particular brain area • By automatically adjusting the activities presented to each user, no two training programs are identical © 2014 Rosetta Stone. No reproduction of any contents without explicit written permission from Rosetta Stone, Inc. 23
  24. 24. Increasing Participation and Long-term Engagement. "Big data" allows us to more effectively smooth out difficulty progression curves, so that users progress through challenges in small, gradual steps, rather than big jumps that can be frustrating and cause them to disengage and drop out of training. • Well-tuned activities should feel “just right”: – Not too easy, which leads to boredom – Not too challenging, which leads to frustration By finding the sweet spot of each user, and providing an entertaining and enjoyable experience, we can help motivate them to return regularly and stick with the program for longer periods of time. © 2014 Rosetta Stone. No reproduction of any contents without explicit written permission from Rosetta Stone, Inc. 24
  25. 25. Example Data: Difficulty Progression – Beginner Level. By examining the scoring data from each of the games across different demographics, we can smooth out the progression so that each problem is a consistent increment of difficulty over the next (inconsistent progression increments are correlated with increased user falloff). © 2014 Rosetta Stone. No reproduction of any contents without explicit written permission from Rosetta Stone, Inc. 25 Initial Difficulty Progression Improved Difficulty Progression Age Brackets Score Score # of Plays # of Plays
  26. 26. Q & A © 2014 Rosetta Stone. No reproduction of any contents without explicit written permission from Rosetta Stone, Inc. 26 Any Questions? Contact Info: mbaxter@rosettastone.com
  27. 27. Harnessing big data and videogames to deliver personalized cognitive training Itmar Lesuisse, CEO of Peak © 2014 Rosetta Stone. No reproduction of any contents without explicit written permission from Rosetta Stone, Inc.
  28. 28. 28 ENJOY BECOMING BETTER EVERY DAY
  29. 29. Big data and cognitive training The science of motivation, adaptation and habit forming design 29
  30. 30. 30
  31. 31. Capture Insight Action 31
  32. 32. Capture Insight Action 32
  33. 33. time 33 sleep activity
  34. 34. Capture Insight Action 34
  35. 35. Peak insights 35 WE ARE CREATING A DASHBOARD FOR YOUR BRAIN
  36. 36. 36
  37. 37. Capture Insight Action 37
  38. 38. Fostering motivation with personalized challenges and goals 38
  39. 39. Tackling weaknesses with high-intensity workouts
  40. 40. Better working habits 40
  41. 41. SLeep smart 41
  42. 42. 42
  43. 43. Capture Insight Action 43
  44. 44. Let’s work together 44 scientists and researchers trackers and sensors design partners SCIENCE@PEAK.NET
  45. 45. Harnessing big data and videogames to deliver personalized cognitive training Aki Nikolaidis, NSF Fellow at the University of Illinois Champaign Urbana © 2014 Rosetta Stone. No reproduction of any contents without explicit written permission from Rosetta Stone, Inc.
  46. 46. Cognitive Enhancement and Neuroplasticity Aki Nikolaidis- University of Illinois Champaign Urbana Twitter- @AkiNikolaidis
  47. 47. Empowerment Technology
  48. 48. Empowerment Technology • Democracy- control over governance Photos: www.brainpickings.org/ http://cdn.patch.com/
  49. 49. Empowerment Technology • Democracy- control over governance • Internet & Social Media- empowering the marginalized Photos: www.brainpickings.org/ http://cdn.patch.com/
  50. 50. Cognitive Enhancement • Empowerment Technology • Potentially large societal benefit
  51. 51. Making it Work • How?
  52. 52. Making it Work • How? • We need answers to two questions: – How does brain plasticity lead to learning? – How does learning lead to transfer?
  53. 53. How Does Brain Plasticity Lead to Learning? • Reorganization of brain’s computations during learning www.nicabm.com
  54. 54. How Does Brain Plasticity Lead to Learning? • Reorganization of brain’s computations during learning • Complexity is essential • Need to embrace complexity to answer our questions www.nicabm.com
  55. 55. How Does Learning Lead to Transfer? • Failures to replicate • What about learning leads to transfer?
  56. 56. How Does Learning Lead to Transfer? • Failures to replicate • What about learning leads to transfer? • Ways to change training? – Feedback delivery – Training schedules – Adaptive training
  57. 57. How Does Learning Lead to Transfer? • Failures to replicate • What about learning leads to transfer? • Ways to change training? – Feedback delivery – Training schedules – Adaptive training • Brain plasticity should lead to transfer
  58. 58. Space Fortress Vo et al 2011; PLOS One
  59. 59. Space Fortress Nikolaidis et al. 2014, Frontiers in Human Neuroscience • Does training induced plasticity drive transfer?
  60. 60. Space Fortress Nikolaidis et al. 2014, Frontiers in Human Neuroscience • Does training induced plasticity drive transfer? • Yes-plasticity in working memory areas predicted working memory improvement.
  61. 61. Training Strategy & Network Plasticity • Two Strategies- – Variable Priority – Fixed Priority • Two Networks – Executive Control – Subcortical Nikolaidis et al, (In Preparation); Hong et al. 2013 PLOS One
  62. 62. Moving Forward • Double Dissociation – Executive Plasticity Predicts Variable Priority Learning – Subcortical Plasticity Predicts Fixed Priority Learning • Training strategy has large influence on network plasticity • Design training strategies to target certain networks or transfer constructs
  63. 63. Where do we go from here? • We are on the right track, but – Need to solidify findings • How can we increase brain plasticity in relevant networks? – Training Strategies – Exercise – Non-invasive Brain Stimulation?
  64. 64. Acknowledgements • Art Kramer • LBC Lab – Michelle Voss – Hyunkyu Lee – Loan Vo – Drew Goatz • Funding – NSF, ONR, CSAI Award
  65. 65. Thank you – Google+ & LinkedIn: Aki Nikolaidis – Twitter @AkiNikolaidis – Email- gnikola2@illinois.edu
  66. 66.  To learn more, visit SharpBrains.com

×