Certifying the Hard Stuff (and the role of learning analytics)


Published on

MOOC Research Conference 2013, Arlington TX Panel on Certifying MOOCs

Published in: Education, Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Certifying the Hard Stuff (and the role of learning analytics)

  1. 1. MOOC Research Conference 2013, Arlington TX Panel on Certifying MOOCs moocresearch.com Certifying the hard stuff (and where learning analytics might help) Simon Buckingham Shum Knowledge Media Institute, The Open University, UK simon.buckinghamshum.net learningemergence.net @sbskmi 1
  2. 2. The final educational C… may be about to go Open content + curriculum certification community
  3. 3. Assessment regimes drive certification “What you assess is what you get; if you don’t test it, you won’t get it. To improve student performance we must recognize that essential intellectual abilities are falling through the cracks of conventional testing.” Lauren Resnick 3
  4. 4. Certification Assumes we know what quality looks like That we know how to assess it, reliably That certification is a brand (but based on scarcity) 4
  5. 5. The problem of Massive How do we avoid just counting what’s easiest to count? 5
  6. 6. Big Data and Analytics to the rescue! Finally, we can let the data speak for itself Learning analytics and eassessment make it possible to objectively evaluate performance at scale 6
  7. 7. Data and Analytics are infused with human values Finally, we can let the data speak for itself Learning analytics and eassessment make it possible to objectively evaluate performance at scale 7
  8. 8. Learning technology is not neutral Any technology embodies epistemological, pedagogical and assessment assumptions “Our learning analytics are our pedagogy” epistemology assessment pedagogy Knight S, Buckingham Shum S and Littleton K. (2013) Epistemology, Pedagogy, Assessment and Learning Analytics. Proc. 3rd International Conference on Learning Analytics & Knowledge. Leuven, BE: ACM, 75-84 Open Access Eprint: http://oro.open.ac.uk/36635 8
  9. 9. The challenge of authenticity: of learner, and of learning Are you really doing the work? Did you really write this? Plagiarism is not a new problem >>> “authentic assessment” 9
  10. 10. Authentic Assessment strategies resist plagiarism, promote deep learning and build real world competencies   Oral Interviews Teacher asks student questions about personal background, activities, readings, and other interests. cf. Jim Groom’s keynote   Story or Text Retelling Student retells main ideas or selected details of text experienced through listening or reading.   Writing Samples Student generates narrative, expository, persuasive, or reference paper with clear reference to their own experience and views.   Projects/Exhibitions Student works with other students as a team to create a project that often involves multimedia production, oral and written presentations, and a display.   Experiments/Demonstrations Student documents a series of experiments, illustrates a procedure, performs the necessary steps to complete a task, and documents the results of the actions.   Constructed-Response Items Student responds in writing to open-ended questions.   Teacher Observations Teacher observes and documents the students attention and interaction in class, response to instructional materials, and cooperative work with other students.   Portfolios A focused collection of student work to show progress over time. 10
  11. 11. Learning dispositions “In the growth mindset, people believe that their talents and abilities can be developed through passion, education, and persistence … It’s about a commitment to … taking informed risks … surrounding yourself with people who will challenge you to grow” Carol Dweck See the MRI mindset intervention projects Interview with Carol Dweck: http://interviewscoertvisser.blogspot.co.uk/2007/11/interview-with-carol-dweck_4897.html 11
  12. 12. Learning dispositions “We’re looking at the profiles of what it means to be effective in the 21st century. […] Resilience will be the defining concept. When challenged and bent, you learn and bounce back stronger.” “Dispositions are now at least as important as Knowledge and Skills. …They cannot be taught. They can only be cultivated.” John Seely Brown US Dept. of Educ. http://reimaginingeducation.org conference (May 28, 2013) Dispositions clip: http://www.c-spanvideo.org/clip/4457327 Whole talk: http://www.c-spanvideo.org/program/SecD 12
  13. 13. Learning dispositions “It’s more than knowledge and skills. For the innovation economy, dispositions come into play: readiness to collaborate; attention to multiple perspectives; initiative; persistence; curiosity. The purpose of learning in the 21st century is not to recite inert knowledge but to transform it. It’s time to change the subject.” Larry Rosenstock LearningREimagined project: http://learning-reimagined.com Larry Rosenstock: http://audioboo.fm/boos/1669375-50-seconds-of-larry-rosenstock-ceo-of-hightechhigh-on-how-he-would-re-imagine-learning 13
  14. 14. How Learning Analytics may help 14
  15. 15. Dispositional Learning Analytics Workshop http://learningemergence.net/events/lasi-dla-wkshp 15
  16. 16. 1st International Workshop on Discourse-Centric Learning Analytics solaresearch.org/events/lak/lak13/dcla13 analytics that look beneath the surface, and quantify linguistic proxies for ‘deeper learning’ http://www.glennsasscer.com/wordpress/wp-content/uploads/2011/10/iceberg.jpg Beyond number / size / frequency of posts; ‘hottest thread’
  17. 17. Social Network Analysis to gain insight into peer-peer and peer-mentor dynamics (SNAPP tool) http://www.slideshare.net/aneeshabakharia/snapp-20minute-presentation Bakharia A and Dawson S. (2011) SNAPP: a bird's-eye view of temporal participant interaction. Proceedings of the 1st International Conference on Learning Analytics and Knowledge. Banff, Alberta, Canada: ACM, 168-173. 17
  18. 18. Epistemic Gaming Shaffer, D. W., Hatfield, D., Svarovsky, G. N., Nash, P., Nulty, A., Bagley, E., Franke, K., Rupp, A. A., Mislevy, R. (2009). Epistemic Network Analysis: A prototype for 21st Century assessment of learning. The International Journal of Learning and Media. 1(2), 33-53. 18
  19. 19. Badging framework Open Learning Design Studio MOOC — Badging Framework: http://www.olds.ac.uk/blog/moocbadgingandthelearningarc 19