1) Learning analytics seeks new insights from educational data by measuring, collecting, analyzing and reporting data about learners and learning environments to optimize learning. 2) There are three eras of social science research: collecting simple data on important questions; getting the most from little data; and today's "big data" deluge allowing new questions. 3) Educational data can be analyzed through psychometrics, educational data mining, and learning analytics, typically focusing on assessment, learning over time, and wider contexts respectively.

1. Learning Analytics
seeking new insights from educational data
Andrew Deacon
Centre For Innovation in Learning and Teaching
University of Cape Town
Teaching and Learning with Technology workshop, CPUT, 2014

2. Outline
• What is changing with ‘big data’
• Three eras of social science research
• Three ways educational data is analyzed
• Changing roles of analytics with more data

3. Age of Big Data
Source: The Economist

4. Learning Analytics
… is the measurement, collection, analysis
and reporting of data about learners and
their contexts, for purposes of
understanding and optimising learning
and the environments in which it occurs.
https://tekri.athabascau.ca/analytics

5. ASKING NEW QUESTIONS
Three eras of social research

6. Three eras of social research
1. Age of Quételet
collect data on simple & important questions
2. Classical period
get the most information from a little data
3. Present day big data
deluge of data and questions

7. [1] UCT Student Experience Survey
• Understand students’
overall experience
• Data to effect change,
improve decisions and
policies, affirm good
practices & quality
assure
• Good practice

9. [2] Are streams being disadvantaged?
Within Degree Type:
• Differences in mean
final mark are
significant
• Across years,
differences in
means are similar
• Differences in 2013
are not unusual
Change in mode
of delivery

10. [3] UCT and social media
Prominent links to:
– Facebook
– Flickr
– LinkedIn
– Twitter

11. Twitter: UCT chatter
• Looked at 6 months of data
April – Sept 2011
• Selected tweets with a UCT hashtag or text
#UCT, #Ikeys, University of Cape Town, …
• Attributes
tweet amplification, app used, location
• Dataset
Just over 5,000 tweets

12. Twitter: apps & locations
27%
36%
20%
17%
1 2 3 4
Smartphone
geo-location
Cell phones
Blackberry

13. Twitter: tweeter relationships
Frequent tweeters:
1. Drama student
(162)
2. UCT Radio
(132)
3. Science student
(84)

14. Twitter: viral #UCT
Varsity Cup
final
Helicopter
crash
6 months of tweets

15. Flickr: helicopter crash at UCT
Ian Barbour - http://www.flickr.com/people/barbourians/

16. Twitter: helicopter crash at UCT
2 hours
after the
event
• Peak of 140 tweets
in 5 minutes
• Media organisations
tweets get re-tweeted
• Crash or hard-landing?

17. Facebook: all friend relationships
Paul Butler http://www.facebook.com/notes/facebook-engineering/visualizing-friendships/469716398919

18. 1st year course
combinations
at UCT Health
Sciences
Engineering
Humanities
Science
Commerce

19. DIFFERENT ANALYTICAL TOOLS
Three approaches to educational data

20. Three approaches to educational data
1. Psychometrics
placing measures on a scale (e.g., in
assessment)
2. Educational Data Mining
focus on learning over time (e.g., in schools)
3. Learning Analytics
typically wider contexts (e.g., in universities)

21. [3] Developing learning analytics

22. Students’ use of Vula in a course
Site visits
Chat room
activity
Sectioning
of students
Polling of
students
Content
accessed
Submission of
assignments
Submission of
assignments

23. Purdue University's Course Signals
• Early warning signs
provides intervention to
students who may not
be performing well
• Marks from course
• Time on tasks
• Past performance Source:
http://www.itap.purdue.edu/learning/tools/signals

24. Advisors – U Michigan
• Advisors are key element
• Data from LMS
– Measures to compare students
(LMS performance and LMS usage)
– Classifications
(<55% red and >85% green)
– Visualizations of student performance
• Engagement with advisors
– Dashboard

25. Measures to compare students
• LMS Gradebook and Assignments
– Student score as percentage of total
– Class mean score as percentage of total
• LMS Presence as proxy for ‘effort’
– Weekly total
– Cumulative total

26. Classifications of cohort
Comparisons are intra-class
Performance Change Presence Rank Action
>= 85% Encourage
75% to 85% < 15% Explore
>= 15% < 25% Explore
>= 15% >= 25% Encourage
65% to 75% < 15% < 25% Engage
< 15% >= 25% Explore
>= 15% Explore
55% to 65% >= 10% Explore
< 10% Engage
< 55% Engage

27. Advisor support
• Shorten time to intervene
– Weekly update
– Contact ‘red’ students
– Useful to prepare for consultation
• Contextualizing student performance
– Longitude trends (course and degree)
– Identify students who don’t need support

28. Example of tools: RapidMiner

29. Sociogram of a discussion forum
Dawson (2010)

30. Words used by Lecturers vs Students
Used more by
Students
Used more by
Lecturers/tutors
‘Weiten’ –
textbook
author
Marks;
thanks;
test;
Tut;
guys
Week;
pages

31. MOOC Completion Rates
http://www.katyjordan.com/MOOCproject.html

33. CHANGING ROLES OF ANALYTICS
The future with more data

34. Concerns about Big Data thinking
• Does Big Data…
– change the definition of knowledge
– increase objectivity and accuracy
– analysis improves with more data
– make the context less critical
– availability means using the data is ethical
– reduce digital divides
See (Boyd & Crawford 2012)

35. Effective visualisations
The success of a visualization is
based on deep knowledge and
care about the substance, and the
quality, relevance and integrity of
the content.
Tufte (1981)

36. Correlation and causation
• Correlation does not imply causation
– Covariation is a necessary but not a sufficient
condition for causality
– Correlation is not causation
(but could be a hint)

37. Future scenarios
• Analytics in educational research:
– More data means asking new questions
– Interpreting data in a student’s context
– Open up discussions and possibilities
– New ethical considerations
• Visualisations and analytics tools:
– Good open source software is available
– Encourage people to engage with learning analytics

38. Software references
• Gephi – network analysis, data collection
• NodeXL – network analysis, data collection
• TAGS – Twitter data collection (Google Drive)
• Word cloud – R package (wordcloud)
• RapidMiner – Data mining, predictive analytics
• Excel – spreadsheet, charts
• R – statistical analysis, graphs

39. Literature references
• Boyd, D., Crawford, K. (2012) Critical Questions for Big Data, Information,
Communication & Society, 15:5, 662-679
• Dawson, S. (2010) ‘Seeing’ the learning community: An exploration of the
development of a resource for monitoring online student networking.
British Journal of Educational Technology, 41(5), 736-752.
• Deacon, A., Paskeviciusat, M. (2011) Visualising activity in learning
networks using open data and educational analytics. Southern African
Association for Institutional Research Forum, Cape Town.
• Berland, M., Baker, R.S., Blikstein, P. (in press) Educational data mining and
learning analytics: Applications to constructionist research. To appear in
Technology, Knowledge, and Learning.
• Hansen, D., Shneiderman, B., Smith, M.A. (2011) Analyzing Social Media
Networks with NodeXL: Insights from a Connected World, Morgan
Kaufmann Publishers, San Francisco, CA.
• Tufte, E. (1981) The visual display of quantitative information. Cheshire,
Conn.: Graphics Press.

40. South African references