Workshop on Educational and Social Sciences, 19-21 Jul, at the National University of Singapore. Tutorial.

1. Learning analytics
policy and adoption strategies
Dragan Gašević
@dgasevic
July 19, 2017
WESST, NUS
Singapore

2. Educational Landscape Today
Growing need for education
Active learning
Education defunding

3. Feedback loops between
students and instructors
are missing/weak!

4. LEARNING ANALYTICS

5. Learning
environment
Educators
Learners
Student
Information
Systems

6. Blogs
Videos/slides
Mobile
Search
Educators
Learners
Networks
Student
Information
Systems
Learning
environment

7. Blogs
Mobile
Search
Networks
Educators
Learners
Student
Information
Systems
Learning
environment
Videos/slides

8. EXAMPLES

9. Student retention
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
Year 1 Year 2 Year 3 Year 4
Course Signals
No Course Signals
Arnold, K. E., & Pistilli, M. D. (2012, April). Course Signals at Purdue: Using learning analytics to increase student success. In Proceedings of the 2nd International
Conference on Learning Analytics and Knowledge (pp. 267-270).

10. Learning analytics is not only
about deficit models
Kennedy, G.E. (2014). Personal Analytics: Getting off the deficit path. Keynote presentation at Australian Learning Analytics Summer Institute 2014. Sydney, NSW
Australia.

11. Tanes, Z., Arnold, K. E., King, A. S., & Remnet, M. A. (2011). Using Signals for appropriate feedback: Perceptions and practices. Computers & Education, 57(4),
2414-2422.
Can teaching be improved?

12. Wright, M. C., McKay, T., Hershock, C., Miller, K., & Tritz, J. (2014). Better Than Expected: Using Learning Analytics to Promote Student Success in Gateway Science.
Change: The Magazine of Higher Learning, 46(1), 28-34.

13. http://ontasklearning.org

14. Analytics-based feedback
Pardo, A., Jovanovic, J. Dawson, S., Gasevic, D., Miriahi. N., (2017). Using Learning Analytics to Scale the Provision of Personalised Feedback (submitted).

15. Analytics-based feedback
Pardo, A., Jovanovic, J. Dawson, S., Gasevic, D., Miriahi. N., (2017). Using Learning Analytics to Scale the Provision of Personalised Feedback (submitted).

16. Analytics-based feedback
Pardo, A., Jovanovic, J. Dawson, S., Gasevic, D., Miriahi. N., (2017). Using Learning Analytics to Scale the Provision of Personalised Feedback (submitted).

17. Analytics-based feedback
Pardo, A., Jovanovic, J. Dawson, S., Gasevic, D., Miriahi. N., (2017). Using Learning Analytics to Scale the Provision of Personalised Feedback (submitted).

18. INSTITUTIONAL ADOPTION: CURRENT STATE

19. Current state – Oz and Europe
http://sheilaproject.eu/http://he-analytics.com

20. Very few institution-wide
examples of adoption

21. Adoption challenge
Skills for data-informed
decision making
Tsai, Y. S., & Gasevic, D. (2017). Learning analytics in higher education – challenges and policies: a review of eight learning analytics policies. In Proceedings of the
Seventh International Learning Analytics & Knowledge Conference (pp. 233-242).

22. Adoption challenge
Leadership for strategic
implementation & monitoring
Tsai, Y. S., & Gasevic, D. (2017). Learning analytics in higher education – challenges and policies: a review of eight learning analytics policies. In Proceedings of the
Seventh International Learning Analytics & Knowledge Conference (pp. 233-242).

23. Adoption challenge
Equal engagement with
different stakeholders
Tsai, Y. S., & Gasevic, D. (2017). Learning analytics in higher education – challenges and policies: a review of eight learning analytics policies. In Proceedings of the
Seventh International Learning Analytics & Knowledge Conference (pp. 233-242).

24. Adoption challenge
Policies for learning analytics practice
Tsai, Y. S., & Gasevic, D. (2017). Learning analytics in higher education – challenges and policies: a review of eight learning analytics policies. In Proceedings of the
Seventh International Learning Analytics & Knowledge Conference (pp. 233-242).

25. What’s necessary to move forward?

26. DIRECTIONS

27. Data – Model – Transformation
Barton, D., & Court, D. (Oct 2012). Making Advanced Analytics Work for You. Harvard Business Review, 79-83, https://hbr.org/2012/10/making-advanced-
analytics-work-for-you/ar/1

28. Data – Model – Transformation
Creative data sourcing
Gašević, D., Dawson, S., Pardo, A. (2016). How do we start? State and Directions of Learning Analytics Adoption. Oslo, Norway: International Council for Open and
Distance Education. http://bit.ly/icde_la_16

29. Student evaluation of teaching are not
a holy grail of learning analytics
Uttl, B., White, C. A., & Gonzalez, D. W. (2016). Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are
not related. Studies in Educational Evaluation, 54, 22–42, https://doi.org/10.1016/j.stueduc.2016.08.007.

30. Challenges of self-reported data
Winne, P. H., & Jamieson-Noel, D. (2002). Exploring students’ calibration of self reports about study tactics and achievement. Contemporary Educational
Psychology, 27(4), 551–572. https://doi.org/10.1016/S0361-476X(02)00006-1

31. Analytics to measure engagement
Joksimovic, S., Poquet, O., Kovanovic, V., Dowell, D., Mills, C., Gasevic, D., Dawson, S., Graesser, A. C., Brooks , C. (2017). How do we measure learning at scale? A
systematic review of the literature. Submitted to Review of Educational Research

32. Awareness of limitations and
challenging assumptions
Kovanović, V., Gašević, D., Dawson, S., Joksimović, S., Baker, R. (2015). Does Time-on-task Estimation Matter? Implications on Validity of Learning Analytics
Findings. Journal of Learning Analytics, 2(3), 81-110.

33. Privacy and ethics should be considered

34. The DELICATE checklist
Drachsler, H., & Greller, W. (2016, April). Privacy and analytics: it's a DELICATE issue a checklist for trusted learning analytics. In Proceedings of the Sixth
International Conference on Learning Analytics & Knowledge (pp. 89-98). ACM.

35. Data – Model – Transformation
Question-driven, not data-driven
Gašević, D., Dawson, S., Pardo, A. (2016). How do we start? State and Directions of Learning Analytics Adoption. Oslo, Norway: International Council for Open and
Distance Education. http://bit.ly/icde_la_16

37. Field of research and practice
Gašević, D., Kovanović, V., & Joksimović, S. (2017). Piecing the Learning Analytics Puzzle: A Consolidated Model of a Field of Research and Practice. Learning:
Research and Practice, 3(2), 63-78. doi:10.1080/23735082.2017.1286142

38. Learning analytics is about learning
Gašević, D., Dawson, S., Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64-71.

39. One size fits all does not work in
learning analytics

40. Gašević, D., Dawson, S., Rogers, T., Gašević, D. (2016). Learning analytics should not promote one size fits all: The effects of course-specific technology use in
predicting academic success. The Internet and Higher Education, 28, 68–84.
Learning context
Instructional conditions shape
learning analytics results

41. Learner agency
Jovanović, J., Gašević, D., Dawson, S., Pardo, A., & Mirriahi, N. (2017). Learning analytics to unveil learning strategies in a flipped classroom. The Internet and Higher
Education, 33, 74-85.
More time online does not always mean
better learning

42. Data – Model – Transformation
Gašević, D., Dawson, S., Pardo, A. (2016). How do we start? State and Directions of Learning Analytics Adoption. Oslo, Norway: International Council for Open and
Distance Education. http://bit.ly/icde_la_16

43. Systemic Adoption Model
Colvin, C., et al. (2015). Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement. Sydney: Australian Office
for Learning and Teaching.

44. Solution-focused Model
Colvin, C., et al. (2015). Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement. Sydney: Australian Office
for Learning and Teaching.

45. Process-focused Model
Colvin, C., et al. (2015). Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement. Sydney: Australian Office
for Learning and Teaching.

46. Data – Model – Transformation
Inclusive approaches to adoption
Gašević, D., Dawson, S., Pardo, A. (2016). How do we start? State and Directions of Learning Analytics Adoption. Oslo, Norway: International Council for Open and
Distance Education. http://bit.ly/icde_la_16

47. Inclusive adoption process

48. Inclusive adoption process
http://sheilaproject.eu/

49. Students expect the use of their data
provided ethics and privacy is considered
http://sheilaproject.eu/

50. Students and teaching staff
don’t always share the same perspectives
http://sheilaproject.eu/

51. Inclusive adoption process
Macfadyen, L., Dawson, S., Pardo, A., Gašević, D., (2014). The learning analytics imperative and the sociotechnical challenge: Policy for complex systems. Research
& Practice in Assessment, 9(Winter 2014), 17-28.

52. SHEILA policy making framework
Action – Challenges – Policy – Instruments
http://sheilaproject.eu/

53. The University of Edinburgh (2017). Learning Analytics, http://www.ed.ac.uk/information-services/learning-technology/learning-analytics

54. Learning analytics principles
Data incompleteness, bias perpetuation,
avoidance of deficit models, facilitation of training,
not used for performance assessment
The University of Edinburgh (2017). Learning Analytics Policy, http://www.ed.ac.uk/academic-services/projects/learning-analytics-policy

55. Learning analytics purposes
Quality, equity, personalized feedback, coping with scale,
student experience, skills, and efficiency
The University of Edinburgh (2017). Learning Analytics Policy, http://www.ed.ac.uk/academic-services/projects/learning-analytics-policy

56. Data – Model – Transformation
Inclusive approaches to adoption
Analytics tools for non-statistics experts
Gašević, D., Dawson, S., Pardo, A. (2016). How do we start? State and Directions of Learning Analytics Adoption. Oslo, Norway: International Council for Open and
Distance Education. http://bit.ly/icde_la_16

57. Visualizations can be harmful
Corrin, L., & de Barba, P. (2014). Exploring students’ interpretation of feedback delivered through learning analytics dashboards. In Proceedings of the ascilite 2014
conference (pp. 629-633). ascilite.

58. FINAL REMARKS

59. Rhetoric of simplistic technological fixes
is unproductive

60. Embracing complexity of
educational systems

61. Capacity development
Multidisciplinary teams in
institutions critical

62. Development of analytics culture
Manyika, J. et al. (2011). Big Data: The Next Frontier for Innovation, Competition, and Productivity. McKinsey Global Institute, http://goo.gl/Lue3qs

63. DISCUSSION

64. Step 1: Map political context
n Is there government/teacher/student/administrator
pressure to deploy LA in your organisation?
n Or pressure not to deploy it?
http://sheilaproject.eu/

65. Step 2: Identify key stakeholders
n Who / which people inside and outside your organisation
would be / are the key stakeholders for the use of LA?
n How informed consent will be obtained?
n Who will benefit from the use of LA?
http://sheilaproject.eu/

66. Step 3: Identify desired behaviour changes
n What are your reasons for adopting LA?
n What benefits is it expected to bring / have brought?
http://sheilaproject.eu/

67. Step 4: Develop engagement strategy
n How well prepared do you think your organisation is to deploy LA?
n What needs to be done to achieve the desired changes identified
previously?
¡ Areas to consider are ethical/legal, technical, pedagogical, and cultural.
http://sheilaproject.eu/

68. Step 5: Analyse internal capacity to effect
changes
n Does your organisation has the capacity to implement the
planned strategy?
n Do you need to acquire any skill-sets, funding, or technological
systems?
http://sheilaproject.eu/

69. Step 6: Establish monitoring and learning
frameworks
n How will you track and evaluate the progress?
n How will this process inform the next step of
implementation?
n Will there be any success criteria?
http://sheilaproject.eu/

70. Towards systemic adoption of
learning analytics in higher education
Dragan Gašević
@dgasevic
July 12, 2017
The Hong Kong
Polytechnic University