Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Fighting level 3:
From the LA framework to LA practice on
the micro-level
Hendrik Drachsler, @hdrachsler
LASI Spain, JULY ...
3
WhoAmI
Full Professor of
Educational Technologies and
Learning Analytics
School of Computer Science
Goethe University Fr...
@HDrachsler, #
_NL, Zeist, Netherlands
Slide 3 / 29 June 2014
1. LA Framework
in a nutshell
2. Fighting
level 3
Lecture st...
4
Greller, W. & Drachsler, H. (2012). Turning Learning into Numbers. Toward a Generic Framework for
Learning Analytics. Jo...
Let‘s not forget, LA is for teachers and
learners
Learner’s and teachers are most
of the located on the micro-level
with t...
@HDrachsler, #
_NL, Zeist, Netherlands
Slide 7 / 29 June 2014
1. LA Framework
in a nutshell
2. Fighting
level 3
Lecture st...
Sophistican model
Siemens, G., Dawson, S., & Lynch, G. (2014). Improving the Quality and Productivity of
the Higher Educat...
Maturing, Roll-Out, Uptake?
VERY URGENT for uptake of LA
• Ethics & Privacy (Dragan keynote)
• Stakeholder needs & Compete...
@HDrachsler, #
_NL, Zeist, Netherlands
Slide 10 / 29 June 2014
1. LA Framework
in a nutshell
2. Fighting
level 3
Lecture s...
STOTA Dashboards
1. Verbert, et al., (2013) Learning analytics
dashboard applications. American
Behavioral Scientist 57(10...
Educational concepts mentioned in LA dashboard
papers
Jivet, I., Scheffel, M., Drachsler, H., and Specht, M. (2017). Aware...
Competences mentioned in LA dashboard papers
Jivet, I., Scheffel, M., Drachsler, H., and Specht, M. (2017). Awareness is n...
Relation between educational concepts and
competencies
Jivet, I., Scheffel, M., Drachsler, H., and Specht, M. (2017). Awareness is not enough. Pitfalls of
learning analytics das...
• Design of LA Dashboard are technology driven (only 26 out
of 95 papers use edu. concepts)
• Mostly to support awareness ...
@HDrachsler, #
_NL, Zeist, Netherlands
Slide 17 / 29 June 2014
1. LA Framework
in a nutshell
2. Fighting
level 3
Lecture s...
Learning Analytics supported
Learning Design
Three main opportunities:
1. Using LA indicators for on demand measures
of th...
Opportunities and challenges for using LA in LD
Schmitz, M., Van Limbeek, E., Greller, W., Sloep, P., Drachsler., H. (2017...
Opportunities and challenges for using LA in LD
Schmitz, M., Van Limbeek, E., Greller, W., Sloep, P., Drachsler., H. (2017...
@HDrachsler, #
_NL, Zeist, Netherlands
Slide 21 / 29 June 2014
1. LA Framework
in a nutshell
2. Fighting
level 3
Lecture s...
Learning Analytics Readiness Instrument (LARI)
Oster, M., Lonn, S., D. Pistilli, M., and Brown., M., G. (2016). The learni...
Evaluation of the effect of LA
• If you want to measure uptake
of LA you need an instrument to
explore progress and adopti...
We were looking for something quick
and robust like ...
... the SUS scale.
But than dedicated
for LA.
Scheffel, M., Drachsler, H.,
Stoyanov, S., and Specht, M.
(2014). Quality Indicators
for Learning Analytics.
Educational T...
Evaluation Framework for Learning Analytics (EFLA)
Scheffel, M., Drachsler, H., Kreijns, K., de Kraker, J., Specht, M. (20...
Evaluation Framework for Learning Analytics (EFLA)
Quick evaluation of LA
tools in micro-settings.
We are looking for
part...
@HDrachsler, #
_NL, Zeist, Netherlands
Slide 28 / 29 June 2014
1. LA Framework
in a nutshell
2. Fighting
level 3
Lecture s...
The problem for uptake of LA
and mastering level 3 is ...
Educational
Science
Computer
Science
Learning
Analytics
MIDDLE S...
The solution, LA as a real middle space
Educational
Science
Computer
Science
Learning
Analytics
MIDDLE SPACE
31
@hdrachsler
drachsler@dipf.de | hendrik.drachsler@ou.nl
linkedin.com/in/hendrikdrachsler
Many thanks for the invitation...
Fighting level 3: From the LA framework to LA practice on the micro-level
Upcoming SlideShare
Loading in …5
×

Fighting level 3: From the LA framework to LA practice on the micro-level

760 views

Published on

This presentation explores shortcomings of learning analytics for the wide adoption in educational organisations. It is NOT about ethics and privacy rather than focuses on shortcomings of learning analytics for teachers and students in the classroom (micro-level). We investigated if and to what extend learning analytics dashboards are addressing educational concepts. Map opportunities and challenges for the use of Learning Analytics dashboards for the design of courses, and present an evaluation instrument for the effects of Learning Analytics called EFLA. EFLA can be used to measure the effects of LA tools at the teacher and student side. It is a robust but light (8 items) measurement to quickly investigate the level of adoption of learning analytics in a course (micro-level). The presentation concludes that Learning Analytics is still to much a computer science dicipline that does not fulfill the often claimed position of the middle space between educational and computer science research.

Published in: Education
  • Be the first to comment

Fighting level 3: From the LA framework to LA practice on the micro-level

  1. 1. Fighting level 3: From the LA framework to LA practice on the micro-level Hendrik Drachsler, @hdrachsler LASI Spain, JULY 4-5, 2017 CC-by: https://www.flickr.com/photos/juandelpozo/6110435588/
  2. 2. 3 WhoAmI Full Professor of Educational Technologies and Learning Analytics School of Computer Science Goethe University Frankfurt German Institute for international Educational Research (DIPF) Frankfurt am Main, Germany Welten Institute Research Centre for Learning, Teaching and Technology Open University, Heerlen, The Netherlands
  3. 3. @HDrachsler, # _NL, Zeist, Netherlands Slide 3 / 29 June 2014 1. LA Framework in a nutshell 2. Fighting level 3 Lecture structure 5. How to evaluate LA on the work floor? 3. Why are our dashboards not used? 4. How to connect LA and LD?
  4. 4. 4 Greller, W. & Drachsler, H. (2012). Turning Learning into Numbers. Toward a Generic Framework for Learning Analytics. Journal of Educational Technology & Society. http://ifets.info/journals/15_3/4.pdf Learning Analytics Framework
  5. 5. Let‘s not forget, LA is for teachers and learners Learner’s and teachers are most of the located on the micro-level with their needs and skills
  6. 6. @HDrachsler, # _NL, Zeist, Netherlands Slide 7 / 29 June 2014 1. LA Framework in a nutshell 2. Fighting level 3 Lecture structure 5. How to evaluate LA on the work floor? 3. Why are our dashboards not used? 4. How to connect LA and LD?
  7. 7. Sophistican model Siemens, G., Dawson, S., & Lynch, G. (2014). Improving the Quality and Productivity of the Higher Education Sector – Policy and Strategy for Systems-Level Deployment of Learning Analytics. Canberra, Australia: Office of Learning and Teaching, Australian Government. Retrieved from http://solaresearch.org/Policy_Strategy_Analytics.pdf LA Sophistication Model 1 2 3 4 5 Who of you arrived at level 3 with their organisation? How long you are already in level 1 and 2?
  8. 8. Maturing, Roll-Out, Uptake? VERY URGENT for uptake of LA • Ethics & Privacy (Dragan keynote) • Stakeholder needs & Competences (Hendrik) • LA supported learning design (Hendrik) LESS URGENT for uptake of LA (but lots of fun) • Infrastructure • Data (Katrien keynote) • Dashboards (Katrien keynote)
  9. 9. @HDrachsler, # _NL, Zeist, Netherlands Slide 10 / 29 June 2014 1. LA Framework in a nutshell 2. Fighting level 3 Lecture structure 5. How to evaluate LA on the work floor? 3. Why are our dashboards not used? 4. How to connect LA and LD?
  10. 10. STOTA Dashboards 1. Verbert, et al., (2013) Learning analytics dashboard applications. American Behavioral Scientist 57(10), 1500 2. Verbert, et al. (2014) Learning dashboards: an overview and future research opportunities. Personal and Ubiquitous Computing 18(6), 1499 3. Yoo, Y., Lee, H., Jo, I.H., Park, Y.: Educational dashboards for smart learning: Review of case studies. In: Emerging Issues in Smart Learning, pp. 145. Springer (2015) 4. Schwendimann, et al., Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies (2016) 5. Bodily, R., Verbert, K.: Trends and issues in student-facing learning analytics reporting systems research. In: Proc. of LAK'17. pp. 309{318. ACM (2017) For the question of uptake we have been investigating 26 out of 95 papers that have grounding in learning sciences: According to which educational concepts are learning analytics dashboards designed?
  11. 11. Educational concepts mentioned in LA dashboard papers Jivet, I., Scheffel, M., Drachsler, H., and Specht, M. (2017). Awareness is not enough. Pitfalls of learning analytics dashboards in the educational practise. 12th European Conference on Technology-Enhanced Learning. Tallinn, Estonia, 12-15 September 2017. Best paper candidate. 18 10 1 7 5 12
  12. 12. Competences mentioned in LA dashboard papers Jivet, I., Scheffel, M., Drachsler, H., and Specht, M. (2017). Awareness is not enough. Pitfalls of learning analytics dashboards in the educational practise. 12th European Conference on Technology-Enhanced Learning. Tallinn, Estonia, 12-15 September 2017. Best paper candidate. 34 8 12 5 13
  13. 13. Relation between educational concepts and competencies
  14. 14. Jivet, I., Scheffel, M., Drachsler, H., and Specht, M. (2017). Awareness is not enough. Pitfalls of learning analytics dashboards in the educational practise. 12th European Conference on Technology-Enhanced Learning. Tallinn, Estonia, 12-15 September 2017. Best paper candidate. Reference frames mentioned in LA dashboard papers 24 16 10
  15. 15. • Design of LA Dashboard are technology driven (only 26 out of 95 papers use edu. concepts) • Mostly to support awareness and reflection • According to McAlpine & Weston, refection is a mechanism to facilitate learning & teaching not an goal in itself. • More than half of the dashboards rely on SRL but none of the tool really support the cycle: fore-thought -> planning -> monitoring -> self-evaluation. • Social framing is more common than achievement framing According to which educational concepts are learning analytics dashboards designed? Jivet, I., Scheffel, M., Drachsler, H., and Specht, M. (2017). Awareness is not enough. Pitfalls of learning analytics dashboards in the educational practise. 12th European Conference on Technology-Enhanced Learning. Tallinn, Estonia, 12-15 September 2017. Best paper candidate.
  16. 16. @HDrachsler, # _NL, Zeist, Netherlands Slide 17 / 29 June 2014 1. LA Framework in a nutshell 2. Fighting level 3 Lecture structure 5. How to evaluate LA on the work floor? 3. Why are our dashboards not used? 4. How to connect LA and LD?
  17. 17. Learning Analytics supported Learning Design Three main opportunities: 1. Using LA indicators for on demand measures of the learning design 2. Intervening during the runtime of a course due to LA indicators 3. Increasing students learning outcomes and interventions through earlier interventions with LA indicators
  18. 18. Opportunities and challenges for using LA in LD Schmitz, M., Van Limbeek, E., Greller, W., Sloep, P., Drachsler., H. (2017). Opportunities and challenges in using Learning Analytics in Learning Design. 12th European Conference on Technology-Enhanced Learning. Tallinn, Estonia, 12-15 September 2017.
  19. 19. Opportunities and challenges for using LA in LD Schmitz, M., Van Limbeek, E., Greller, W., Sloep, P., Drachsler., H. (2017). Opportunities and challenges in using Learning Analytics in Learning Design. 12th European Conference on Technology-Enhanced Learning. Tallinn, Estonia, 12-15 September 2017.
  20. 20. @HDrachsler, # _NL, Zeist, Netherlands Slide 21 / 29 June 2014 1. LA Framework in a nutshell 2. Fighting level 3 Lecture structure 5. How to evaluate LA on the work floor? 3. Why are our dashboards not used? 4. How to connect LA and LD?
  21. 21. Learning Analytics Readiness Instrument (LARI) Oster, M., Lonn, S., D. Pistilli, M., and Brown., M., G. (2016). The learning analytics readiness instrument. LAK '16. ACM, DOI: https://doi.org/10.1145/2883851.2883925A
  22. 22. Evaluation of the effect of LA • If you want to measure uptake of LA you need an instrument to explore progress and adoption • It needs to be a light way instrument as students and teachers don’t fill overwhelming long surveys again and again.
  23. 23. We were looking for something quick and robust like ... ... the SUS scale. But than dedicated for LA.
  24. 24. Scheffel, M., Drachsler, H., Stoyanov, S., and Specht, M. (2014). Quality Indicators for Learning Analytics. Educational Technology & Society, 17 (4):117–132. Scheffel, M., Drachlser, H., Specht, M. (2015). Developing an evaluation framework of quality indicators for learning analytics. LAK ’15, NY, USA. ACM. Scheffel, M., Drachsler, H., de Kraker, J., Kreijns, K., Slootmaker, A., Specht, M. (2017). Widget, widget on the wall, am I performing well at all? IEEE Transactions on Learning Technologies, 10 (1):42–52. Evaluation Framework for Learning Analytics (EFLA)
  25. 25. Evaluation Framework for Learning Analytics (EFLA) Scheffel, M., Drachsler, H., Kreijns, K., de Kraker, J., Specht, M. (2017). Widget, widget as you lead, I am performing well indeed!: Using results from an exploratory offline study to inform an empirical online study about a learning analytics widget in a collaborative learning environment, LAK ’17, NY, USA. ACM. Scheffel, M., Drachsler, H., Toisoul, C., Ternier, S., Specht, M. (2017). The Proof of the Pudding: Examining Validity and Reliability of the Evaluation Framework for Learning Analytics. 12th European Conference on Technology-Enhanced Learning. Tallinn, Estonia, 12-15 September 2017.
  26. 26. Evaluation Framework for Learning Analytics (EFLA) Quick evaluation of LA tools in micro-settings. We are looking for partners to try EFLA and further develop it. A great outcome would be N numbers of LA case studies that applied EFLA and share their experiences. Can we work towards an EFLA score like the SUS score?http://www.laceproject.eu/evaluation- framework-for-la/
  27. 27. @HDrachsler, # _NL, Zeist, Netherlands Slide 28 / 29 June 2014 1. LA Framework in a nutshell 2. Fighting level 3 Lecture structure 5. How to evaluate LA on the work floor? 3. Why are our dashboards not used? 4. How to connect LA and LD? CONCLUSIONS for Uptake of LA LA is about learning! LA is about learning!
  28. 28. The problem for uptake of LA and mastering level 3 is ... Educational Science Computer Science Learning Analytics MIDDLE SPACE
  29. 29. The solution, LA as a real middle space Educational Science Computer Science Learning Analytics MIDDLE SPACE
  30. 30. 31 @hdrachsler drachsler@dipf.de | hendrik.drachsler@ou.nl linkedin.com/in/hendrikdrachsler Many thanks for the invitation! Questions now or later Looking for PhD or PostDoc positions in computer science on Learning Analytics? Get in contact!

×