Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
Nyc data scienceacademy_shinyparti
Next
Download to read offline and view in fullscreen.

3

Share

Download to read offline

Nyc open data presentation by Knewton Data Scientist, Chaitu Ekanadham,

Download to read offline

On March 18 we were lucky to host Chaitu Ekanadham, a data scientist from Knewton. Knewton is an education technology company that uses adaptive learning techniques. It uses data about past learning to aid students in future learning experiences.

For example, in traditional “text book” learning, students move through the content in a linear fashion, one chapter and then the next. While this is a tried and true method of teaching, it isn’t the best solution for every student in every situation. Some students may need only a review of some particular content, while others may need to spend a great deal of time on the same content. For that reason, it makes sense to direct students to content dynamically–show them only the content that would be most helpful for them at that time.

When Knewton begins working with a publisher, they start by organizing the material in the textbook into a Knewton knowledge graph. This is used to represent the ways that content is related to each other conceptually. In this way students’ progression can be evaluated automatically. This gets the right content to the right student at the right time.

So how does one even begin to apply models to this sort of information? Knewton primarily utilizes the data to create models describing the learners, and models describing the educational content.

Modeling Learners

The model of the learner is individualized to each student. After a student has completed some amount of graded content, the system has some idea of their capability. Knewton can then model the likelihood that a student will correctly answer some future question. Quiz questions can then be presented that are neither too hard nor too easy.

Modeling Content

When recommending new content for a student, the system avoids any content that is predicted to have a likelihood of either close to 0 (indicating the student will most likely not learn or retain this information) or 1 (which indicates that the student is familiar with the concepts and does not need any further instruction).

All material is given a score for how difficult it is based on the number of students who historically found it difficult. This value is used in the above model of likelihood a student will get it correct.

In addition, the content is judged for how it engages the students. If students generally move through the content with few breaks it is considered engaging – while content that is consistently associated with long breaks, is considered not engaging. Response time is summed up for all students to give a value for a typical response time for some given content.

How These Models are Used

Knewton uses this data in several different business purposes
1. Generate recommendations for students–what is the best content for that student to consume right now?
2. Analytics–are students on track? What do they need to do in order to pass the next assessment of their skills?
3. Content insights for the cre

Nyc open data presentation by Knewton Data Scientist, Chaitu Ekanadham,

  1. 1. NYC Open Data, March 18, 2015 Chaitanya Ekanadham Representing learning experiences
  2. 2. personalize learning experiences for students around the world PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. Knewton’s mission content inventory student goals interaction data recommendations analytics content insights
  3. 3. provide high quality adaptive education to everyone in the world PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2014 KNEWTON, INC. Knewton’s mission
  4. 4. leverage past learning experiences to improve future learning experiences PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. data science team’s mission
  5. 5. when a learner is presented some content and acquires knowledge as a result PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2014 KNEWTON, INC. what’s a “learning experience”? learner s content alice sharo n .. .. . video lecture on limits derivatives definition integrals quiz limits exercise related rates word problem bob past future
  6. 6. leverage past learning experiences to improve future learning experiences PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
  7. 7. past learning experiences models learner representation content representation
  8. 8. models past learning experiences learner representation proficiency learning speed time investment preferred mode content representation similarities difficulty length effectiveness
  9. 9. leverage past learning experiences to improve future learning experiences PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
  10. 10. models learner representation content representation hypothetical future learning experience score
  11. 11. path dependent effects difficult to quantify do not have final grades leverage past learning experiences to improve future learning experiences PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. unique challenges
  12. 12. content representation
  13. 13. PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. video lecture on limits limits problem set derivatives definition common derivatives differentiation exercises integrals as limits of Riemann sums fundamental theorem of calculus summation notation reference not scalable!
  14. 14. PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. video lecture on limits limits problem set derivatives definition common derivatives differentiation exercises integrals as limits of Riemann sums fundamental theorem of calculus summation notation reference
  15. 15. PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. video lecture on limits limits problem set derivatives definition common derivatives differentiation exercises integrals as limits of Riemann sums fundamental theorem of calculus summation notation reference instruction assessment both
  16. 16. PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
  17. 17. measuring learner ability
  18. 18. models learner representation content representation hypothetical future learning experience score
  19. 19. PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. learner proficiency item difficulty item discrimination
  20. 20. PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. learner s assessment content alice sharon ... .. . limits exercise integrals quiz limits challenge problem related rates word problem bob differentiation problem
  21. 21. PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. learner s assessment content alice sharon ... ... limits exercise differentiation problem integrals quiz limits challenge problem related rates word problem bob
  22. 22. PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. learner s alice sharon .. . bob assessment content limits exercise differentiation problem integrals quiz limits challenge problem related rates word problem...
  23. 23. PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. learner s alice sharon .. . bob assessment content limits exercise differentiation problem integrals quiz limits challenge problem related rates word problem...
  24. 24. PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. ? ? likelihood prior posterior ? ?
  25. 25. PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. likelihood prior ? ? ? ? ? ?? ? ? t=3 ? ? t=2 posterior ? ? t=1
  26. 26. PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. differentiation related rates word problem 3D shape volume
  27. 27. PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. assessing multiple concepts compensatory likelihood: knowing 1 concept is good enough differentiation related rates word problem 3D shape volume
  28. 28. PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. assessing multiple concepts non-compensatory likelihood: have to know both concepts differentiation related rates word problem 3D shape volume
  29. 29. learner timing patterns
  30. 30. PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. time
  31. 31. timesession break response times
  32. 32. histogram of log-taus with highlighting! 48K students 9.6K modules 1M interactions
  33. 33. histogram of log-taus with highlighting! 48K students 9.6K modules 1M interactions
  34. 34. histogram of log-taus with highlighting! session breaks response times ? 48K students 9.6K modules 1M interactions
  35. 35. log time probability response times session breaks
  36. 36. normalized response time normalizedquittingrate
  37. 37. normalized response time normalizedquittingrate high engagement
  38. 38. normalized response time normalizedquittingrate low engagement high engagement
  39. 39. normalized response time normalizedquittingrate non-sticky low engagement high engagement
  40. 40. normalized response time normalizedquittingrate non-sticky boredom? low engagement high engagement
  41. 41. normalized response time normalizedquittingrate non-sticky boredom? low engagement high engagement slow
  42. 42. normalized response time normalizedquittingrate non-sticky boredom? low engagement high engagement slow frustration? deep thought?
  43. 43. PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC. exciting challenges ● offline learning ● learner affinity for pedagogical strategies ● automating content graphing ● student agency ● controlled experiments ● “adaptivity-ready” content design
  44. 44. chaitanya ekanadham managing data scientist Knewton, Inc. chaitu@knewton.com Twitter: @knewton thank you.
  45. 45. appendix
  • fskycn

    Sep. 12, 2017
  • choeungjin

    Apr. 25, 2015
  • hustwj

    Apr. 24, 2015

On March 18 we were lucky to host Chaitu Ekanadham, a data scientist from Knewton. Knewton is an education technology company that uses adaptive learning techniques. It uses data about past learning to aid students in future learning experiences. For example, in traditional “text book” learning, students move through the content in a linear fashion, one chapter and then the next. While this is a tried and true method of teaching, it isn’t the best solution for every student in every situation. Some students may need only a review of some particular content, while others may need to spend a great deal of time on the same content. For that reason, it makes sense to direct students to content dynamically–show them only the content that would be most helpful for them at that time. When Knewton begins working with a publisher, they start by organizing the material in the textbook into a Knewton knowledge graph. This is used to represent the ways that content is related to each other conceptually. In this way students’ progression can be evaluated automatically. This gets the right content to the right student at the right time. So how does one even begin to apply models to this sort of information? Knewton primarily utilizes the data to create models describing the learners, and models describing the educational content. Modeling Learners The model of the learner is individualized to each student. After a student has completed some amount of graded content, the system has some idea of their capability. Knewton can then model the likelihood that a student will correctly answer some future question. Quiz questions can then be presented that are neither too hard nor too easy. Modeling Content When recommending new content for a student, the system avoids any content that is predicted to have a likelihood of either close to 0 (indicating the student will most likely not learn or retain this information) or 1 (which indicates that the student is familiar with the concepts and does not need any further instruction). All material is given a score for how difficult it is based on the number of students who historically found it difficult. This value is used in the above model of likelihood a student will get it correct. In addition, the content is judged for how it engages the students. If students generally move through the content with few breaks it is considered engaging – while content that is consistently associated with long breaks, is considered not engaging. Response time is summed up for all students to give a value for a typical response time for some given content. How These Models are Used Knewton uses this data in several different business purposes 1. Generate recommendations for students–what is the best content for that student to consume right now? 2. Analytics–are students on track? What do they need to do in order to pass the next assessment of their skills? 3. Content insights for the cre

Views

Total views

2,278

On Slideshare

0

From embeds

0

Number of embeds

742

Actions

Downloads

39

Shares

0

Comments

0

Likes

3

×