Learning and Behavioral Analytics From concept to reality

1,120 views
951 views

Published on

How can learning analytics be taken from its design to its deployment in an educational institution? What are the issues, limitations, strategies? This presentation includes a descirption of Learning Analytics, examples, how to tackle systemic deployment and suggestions on how to build institutional capacity.

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,120
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
134
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Learning and Behavioral Analytics From concept to reality

  1. 1. Will Scullin Flickr Learning and Behavioral Analytics From concept to reality Dr Abelardo Pardo (@abelardopardo) Associate Head of Learning and Teaching School of Electrical and Information Engineering slideshare.net/abelardo_pardo Taylor’s University Academic Leaders Retreat 4 March 2014
  2. 2. jbgeronimi flickr.com “How to measure the effectiveness of fye activities and ensure we r on d right track?” Yesterday by CT “No data, no talk” Yesterday by DVC Pradeep Nair Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 2
  3. 3. Unhindered by Talent flickr.com Many colleges and universities have demonstrated that analytics can help significantly advance an institution in such strategic areas as resource allocation, student success, and finance (Bichsel, 2012) Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 3
  4. 4. cygnus921 Flickr What is LA? Current Projects Systemic Deployment Abelardo Pardo Learning and Behavioral Analytics. From concept to reality Building Institutional Capacity 4
  5. 5. cygnus921 Flickr What is LA? Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 5
  6. 6. NYTimes 16/02/12 Is she pregnant? Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 6
  7. 7. Daily Mail UK 20/03/12 TV is watching you Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 7
  8. 8. WSJ 19/07/12 How do you read? Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 8
  9. 9. NY Times 24/11/12 Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 9
  10. 10. Humans are Subjective Mind Sights: Original Visual Illusions, Ambiguities, and Other Anomalies, With a Commentary on the Play of Mind in Perception and Art, Roger N. Shepard, 1990 Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 10
  11. 11. Learning Analytics Measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs. (US Dept. of Education) Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 11
  12. 12. Kurafire flickr.com Feedback as a dialogue Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 12
  13. 13. Discipline in its early steps, but with large potential. Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 13
  14. 14. (Manyika et al., 2013) Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 14
  15. 15. http://www.learninganalytics.net/?p=131 (last visited Sep 2013) (Siemens & Long, 2011) Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 15
  16. 16. www.solaresearch.org Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 16
  17. 17. Werner Kunz Flickr.com Predict Refine Act The five steps of analytics Report Collect (Campbell, De Blois, Oblinger 2007, Academic Analytics, EDUCAUSE) Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 17
  18. 18. Thomas Leth-Olsen flickr.com Knowledge Discovery in Databases Technology enhanced learning Social network analysis Business Intelligence Abelardo Pardo Learning and Behavioral Analytics. From concept to reality Educational modelling 18
  19. 19. zzpza flickr.com • Statistical prediction • Clustering and profiling • Relationship mining • Link prediction in networks • Text analysis Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 19
  20. 20. Secretlondon123 flickr.com 1 2 3 4 Abelardo Pardo Context Questions Data + algorithms Level of intervention Learning and Behavioral Analytics. From concept to reality 20
  21. 21. measure Joe flickr.com 1 2 3 4 Abelardo Pardo Context: Student retention Questions: Why do they leave? Can it be avoided? Data + algorithms: ? Level of intervention: ? Learning and Behavioral Analytics. From concept to reality 21
  22. 22. Shaylor flickr.com 1 2 3 4 Context: Student success Questions: How did they learn? Data + algorithms: ? Level of intervention: ? Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 22
  23. 23. cygnus921 Flickr What is LA? Abelardo Pardo Current Projects Learning and Behavioral Analytics. From concept to reality 23
  24. 24. Joe Fruchey flickr.com MYTH: It is a highly sophisticated technique It can be deployed gradually Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 24
  25. 25. tudedude flickr.com MYTH: LA is a tool to fix a problem. LA is a process to build/refine a model Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 25
  26. 26. HikingArtist flickr.com MYTH: Once we have a model, we apply it to all scenarios Models are derived from the scenarios Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 26
  27. 27. DullHunk flickr.com Reduce attrition. Phone 10 students this semester. Which students? When? Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 27
  28. 28. colecamp flickr.com A/B Testing: Space usage after change in course/curriculum Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 28
  29. 29. (Macfadyen & Dawson, 2010) Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 29
  30. 30. (Romero, Ventura & García, 2008) Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 30
  31. 31. www.itap.purdue.edu/studio/signals (Last accessed Oct. 2013) Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 31
  32. 32. Signals Nottsexminer • Run by the instructor • Points so far (Gradebook) • Time on task (LMS) • Past performance (SIS) • Instructor chooses intervention Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 32
  33. 33. Rio Salado (AZ, USA) • Enrolment figures • LMS variables • Thirty factors considered • Separate course models • Naïve Bayes classification method • Model is applied to classify students http://campustechnology.com/Articles/2011/12/14/Monitoring-the-PACE-of-Student-Learning-Analytics-at-Rio-Salado-College.aspx?p=1 Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 33
  34. 34. www.snappvis.org (Last accessed Oct. 2013) Bookmarklet in your browser Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 34
  35. 35. Learning Glass (Leony et al., 2012) Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 35
  36. 36. Data uses • Multiple levels of intervention • Variety of data sources • Commonalities vs differences • Require multidisciplinary work (Bichsel, 2012) Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 36
  37. 37. Instructors Student Enrolment Abelardo Pardo Curriculum development Learning and Behavioral Analytics. From concept to reality 37 Zilverbat flickr.com CPSC
  38. 38. cygnus921 Flickr What is LA? Current Projects Systemic Deployment Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 38
  39. 39. (Greller and Drachler, 2012) Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 39
  40. 40. paral_lax flickr.com Optimize Predict What if scenarios Understand Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 40
  41. 41. puthoOr Photography flickr.com LMS interactions, quizzes, response times, sessions, hints requested, questionnaires, interviews, requests, demographics, high school grades, etc. (Bienkowski, M., Feng, M., & Means, B., 2012) Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 41
  42. 42. (Campbel, DeBlois, Oblinger, 2007) Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 42
  43. 43. deepwarren Flickr Observe in and out of the classroom Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 43
  44. 44. www.smh.com.au/technology/technology-news/universities-monitor-online-activity-20130413-2hsj2.html (last accessed Sep. 2013) 44 Learning and Behavioral Analytics. From concept to reality Abelardo Pardo
  45. 45. www.uts.edu.au/research-and-teaching/our-research/advanced-analytics-institute 45 Learning and Behavioral Analytics. From concept to reality Abelardo Pardo
  46. 46. www.crlt.umich.edu/slam Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 46
  47. 47. (Powell & MacNeil, 2012) Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 47 Salfalko flickr.com Data Wrangler: someone comfortable handling statistics, manipulating data for visualisation, and capable of engaging with academics about the student experience and course design. It is their job to experiment with different tools to interpret, visualise and share information with academics as a basis for gaining actionable insights.
  48. 48. (Buckingham Shum, 2013) Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 48
  49. 49. cygnus921 Flickr What is LA? Current Projects Systemic Deployment Abelardo Pardo Learning and Behavioral Analytics. From concept to reality Building Institutional Capacity 49
  50. 50. Impact Levels gregoriosz • Micro: at risk students, insight to learners, recommendations • Meso: clear understanding of objectives, increase productivity, help leaders • Macro: institutional transformation (Buckingham Shum, 2012) Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 50
  51. 51. Phase 1 Phase 2 Derpunk flickr.com Static Reports Dynamic Analysis and Intervention Optimization Phase 3 (Norris & Baer, 2013) Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 51
  52. 52. Allie Caufield flickr.com Student Success and Retention Practices Workflows Processes Policies Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 52
  53. 53. juhansonin flickr.com 1) Develop culture of using data to make decisions Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 53
  54. 54. Rosellsgirl flickr.com 2) IT dept. are key to initiate connection Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 54
  55. 55. Not so much flickr.com 3) Multidimensional from the start Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 55
  56. 56. Abraham C flickr.com 4) Gradual deployment Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 56
  57. 57. Fhemerick flickr.com 5) Interfaces that provide value to stakeholders Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 57
  58. 58. Doug Woods flickr.com 6) Refinement as intrinsic component of the process Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 58
  59. 59. Laverrue flickr.com 7) Privacy and Ethics Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 59
  60. 60. Pennstatenews flickr.com 8) Keep the student at the center Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 60
  61. 61. Will Scullin Flickr Learning and Behavioral Analytics From concept to reality Dr Abelardo Pardo (@abelardopardo) Associate Head of Learning and Teaching School of Electrical and Information Engineering slideshare.net/abelardo_pardo Taylor’s University Academic Leaders Retreat 4 March 2014
  62. 62. References Arnold, K. E. (2010). Signals: Applying Academic Analytics. EDUCAUSE Quarterly, 33(1), 10. Bichsel, J. (2012). Analytics in Higher Education Benefits, Barriers, Progress, and Recommendations. EDUCAUSE Center for Applied Research Bienkowski, M., Feng, M., & Means, B. (2012). Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics. US Department of Education. Buckingham Shum, S. (2012). Learning analytics. UNESCO policy brief. Retrieved from http://iite.unesco.org/pics/publications/en/files/3214711.pdfhttp:// iite.unesco.org/pics/publications/en/files/3214711.pdf, March 2014 Buckingham Shum, S. (2013). Building Analytics Capability @open.edu Retrieved from http://www.slideshare.net/sbs/building-analytics-capacity-openeduhttp://www.slideshare.net/sbs/buildinganalytics-capacity-openedu, March 2014 Campbell, J. P., DeBlois, P. B., & Oblinger, D. G. (2007). Academic Analytics. Educause Review (Vol. 42). EDUCAUSE White Paper. Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 62
  63. 63. References 2 Dawson, S. (2010). “Seeing” the learning community: An exploration of the development of a resource for monitoring online student networking. British Journal of Educational Technology, 41(5), 736–752. Ferguson, R. (2012). The State of Learning Analytics in 2012: A Review and Future Challenges a review and future challenges. Technical Report Knowledge Media Institute KMI–12–01). The Open University UK. Greller, W., & Drachsler, H. (2012). Translating learning into numbers: A generic framework for learning analytics. Educational Technology & Society, 15(3), 42–57. Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54(2), 588–599 Manyika, J., Chui, M., Groves, P., Farrell, D., Kuiken, S. Van, & Doshi, E. A. (2013). Open data: Unlocking innovation and performance with liquid information McKinsey Global Institute (p. 116). Norris, D. M., & Baer, L. L. (2013). Building Organizational Capacity for Analytics. Educause. Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 63
  64. 64. References and 3 Powell, S., & MacNeil, S. (2012). Analytics Series Institutional Readiness for Analytics (Vol. 1, pp. 1–11). JISC Center for Educational Technology & Interoperability Standards, Analytics Series, 1(8). Romero, C., Ventura, S., & Garcia, E. (2008). Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51(1), 368–384. Siemens, G., & Long, P. (2011). Penetrating the Fog: Analytics in Learning and Education. Educause Review, 48(5), Siemens, G. (2013). Learning Analytics: The Emergence of a Discipline. American Behavioral Scientist, 57(10), 1380–1400. Siemens, G., Dawson, S., & Lynch, G. (2013). Improving the Quality and Productivity of the Higher Education Sector. Policy and Strategy for Systems-Level Deployment of Learning Analytics. Technical Report, Office for Learning and Teaching, Australian Government Abelardo Pardo Learning and Behavioral Analytics. From concept to reality 64

×