Presentation at the European Distance Education and E-Learning Network (EDEN) Conference, Genoa, Italy, 17-20 June 2018. Authors: Paul Prinsloo, Sharon Slade and Mohammad Khalil

1. Stuck in the middle?
Making sense of the impact of micro, meso and
macro institutional, structural and organisational
factors on implementing learning analytics
Presentation at the European Distance Education and E-Learning Network (EDEN)
Conference, Genoa, Italy, 17-20 June 2018
Imagecredit:https://www.flickr.com/photos/furiousgeorge81/95137658
Paul Prinsloo
University of South Africa (Unisa)
@14prinsp
Sharon Slade
Open University (UK)
@SharonSlade
Mohammad Khalil
Delft University of
Technology, Netherlands
@TUMohdKhalil

2. ACKNOWLEDGEMENTS
We do not own the copyright of any of the images in this
presentation. We therefore acknowledge the original
copyright and licensing regime of every image used.
This presentation (excluding the images) is licensed
under a Creative Commons Attribution 4.0 International
License.

3. Overview of the presentation
• In the beginning…
• Why do we collect what data for what purpose?
• Data as broken and often, stuck in the middle
• Sticky factors: macro, meso and micro levels
• Getting (un)stuck
• Two key takeaways
• Contact details of authors

4. Source credit: https://er.educause.edu/articles/2011/9/penetrating-the-fog-analytics-in-learning-and-education
In the beginning…

5. Source credit: https://er.educause.edu/articles/2011/9/penetrating-the-fog-analytics-in-learning-and-education
Object of analysis
• Institutional
• Regional/national
• International
Who benefits
• Administrators/funders/marketing
• Funders/administrators
• National governments/educational
authorities
Academic analytics
Learning analytics
Object of analysis
• Course level - discourse analysis
• Departmental level – predictive
modeling/ patterns of success
Who benefits
• Learners
• Faculty

6. Source credit: Gašević, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about
learning. TechTrends, 59(1), 64-71.

7. We know, take into account and we
measure: age, gender, race, street address
and zip code, occupation, pre-enrolment
educational data, registration data,
engagement data, academic data, library
data, financial aid data, behavioural data,
location data, who-are-in-their-networks-
data, their chances of failing, dropping out,
stopping out…
Image credit: https://pixabay.com/en/side-profile-black-male-student-1440176/
And we use this data to…
Image credit: https://pixabay.com/en/girl-library-education-student-1721436/

8. Source credit: http://timoelliott.com/blog/2013/02/gartnerbi-emea-2013-part-1-analytics-moves-to-the-core.html

9. We know, take into account and we measure: age,
gender, race, street address and zip code,
occupation, pre-enrolment educational data,
registration data, engagement data, academic data,
library data, financial aid data, behavioural data,
location data, who-are-in-their-networks-data,
their chances of failing, dropping out, stopping
out…
Image credit: https://pixabay.com/en/side-profile-black-male-student-1440176/
We forget these data are
broken…
and often, stuck…
Image credit: https://pixabay.com/en/girl-library-education-student-1721436/

10. Our data is
broken and we
often work
with proxies
and zombie
categories
Our understanding of
learning is incomplete
and provisional
Agency
(institutional
and student) is
constrained
Imagecredit:https://www.publicdomainpictures.net/en/view-
image.php?image=172020&picture=brown-knot
1 3
2

11. Citation: Pink, S., Ruckenstein, M., Willim, R., & Duque, M. (2018). Broken data: Conceptualising data in an
emerging world. Big Data & Society, 5(1), 2053951717753228.
1

12. Image credit: https://pixabay.com/en/zombie-horror-creepy-night-531116/
Zombie categories
“categories from the past that we continue to use even
though they have outlived their usefulness and even
though they mask a different reality”
(Plummer, 2011, p. 195)

13. Image credit: https://pixabay.com/en/side-profile-black-male-student-1440176/
Image credit: https://pixabay.com/en/girl-library-education-student-1721436/
Macro-societal factors, e.g. economic, political, social,
technological, environmental and legal factors.
Institutional/lecturer/student inactions, inefficiencies, or
lack of control impacting and shaping students’ behaviour,
chances of failing, dropping out, stopping out…
2 3

14. Operational budgets/
resource allocation
Identified student
needs
Institutional
understanding
Student
understanding
Institutional locus of
control
Student locus of
control
Insufficient & broken
data
Students are excluded
as partners in learning
analytics
Stuck in the middle…

15. Subotzky, G., & Prinsloo, P. (2011). Turning the tide: A socio-critical model and framework for improving student success in open
distance learning at the University of South Africa. Distance Education, 32(2), 177-193.

16. Sticky factors: Macro level
• Institutional understanding and responses shaped by
changes at national/regulatory/ funding levels
• National imperatives and understanding of the function of
higher education
• Institutional senior management’s understanding (or lack
of) the interdependencies
• Beliefs about data/evidence
• Lack of political will (and resources) to engage students in
the meaning of their data, on what is collected, when it is
collected and what other data may provide both the
institution and learners with a more comprehensive view
of students’ habitus and capital in the “student walk”

17. Sticky factors: Meso level
Students have a clear responsibility to contribute to
their own student walk, but there is less attention
directed to other partners in that walk, namely faculty,
course teams, support staff, and the institutions
themselves.

18. Sticky factors: Micro level
• The nature, scope and (mis)understanding of
student data, the proxies we use and the the role
and noise of zombie categories
• Our understanding and expectations of causal links
and correlations
• What students, course teams, faculty and the
institution can do is often constrained and
dependent on what the other role players do in a
particular context

19. Important pointers for getting
(un)stuck:
1. The student and institution as situated agents –
constraint agency/loci of control
2. The student walk – learning as the result of multiple,
often mutually constitutive and intersecting variables
at the intersections of macro, meso and micro levels
3. The possibility and scope of agency depends on
student/institutional habitus, capital and the state of
the field. On the student side – the role of inter- and
intra-personal connections/relationships. On the
institution’s side – academic, social, operational

20. Source credit: http://oro.open.ac.uk/48173/
“The evidence shows that the use of learning
analytics to improve and to innovate learning and
teaching in Europe is still in its infancy. The high
expectations have not yet been realised.”
Where is the evidence?

21. Key Takeaway 1
To only consider student demographic, historical
and current academic data combined with their
behavioural data underestimates and may
misrepresent the complexity of learning and the
real potential of learning analytics.

22. Unsticking learning analytics depends on acknowledging
what students do, can do, and want to do is connected to,
and often dependent on what we do, and have the
resources to do, in a particular context
Key Takeaway 2
Image credit: https://pixabay.com/en/bubble-gum-shoes-glue-dirt-438404/

23. Paul Prinsloo
(Prof)
Research Professor in Open
Distance Learning (ODL)
College of Economic and
Management Sciences,
University of South Africa
(Unisa), South Africa
prinsp@unisa.ac.za
Skype: paul.prinsloo59
Personal blog:
http://opendistanceteachinga
ndlearning.wordpress.com
Twitter profile: @14prinsp
Sharon Slade
(Dr)
Senior Lecturer l Student
Experience Manager
Faculty of Business and
Law, Open University
Business and Law
Schools,
Open University, A1,
Michael Young Building,
Walton Hall, Milton
Keynes,
MK6 6AA, United
Kingdom
sharon.slade@open.ac.za
Twitter profile:
@SharonSlade
Mohammad Khalil
(Dr)
Centre for Education and
Learning, Delft University of
Technology, Netherlands
M.F.D.Khalil@tudelft.nl
Twitter profile:
@TUMohdKhalil