Professor Mark Brown gave a presentation on learning analytics research. He discussed the scope of learning analytics, what has been learned from previous projects, and future areas of research. Some key findings from past projects included increased student engagement and few opting out when given analytics feedback. Future research plans included analyzing social media discussions of MOOCs and exploring new methodologies.

1. Professor Mark Brown
Director, National Institute for Digital Learning
Beijing, China
Thursday 9th June 2016
Learning Analytics:
A Messy Research Construct

2. Outline…
1. What is the scope?
2. What have we learnt?
3. What are we going to do?
Learning Analytics:
A Messy Research Construct

3. “Learning
analytics
should be in
the service of
big ideas,
not as a big idea
in itself” (adapted from Barnett, 2011).
Key point…

4. 1. What is the scope?

5. 1. What is the scope?

6. 1. What is the scope?

7. What do
we mean by
learning analytics?
How can
we distinguish the
different dimensions?
1. What is the scope?

8. “According to the 1st International Conference on
Learning Analytics and Knowledge, “learning analytics
is the measurement, collection, analysis and reporting
of data about learners and their contexts, for purposes
of understanding and optimising learning and the
environments in which it occurs.”
Long, P. & Siemens, G. (2011). Penetrating the fog: analytics in learning and
education. EDUCAUSE Review, 46 (5), 31-40
1. What is the scope?

9. “Academic analytics, in contrast, is the application of
business intelligence in education and emphasizes
analytics at institutional, regional, and international
levels.”
Long, P. & Siemens, G. (2011). Penetrating the fog: analytics in learning and
education. EDUCAUSE Review, 46 (5), 31-40
“According to the 1st International Conference on
Learning Analytics and Knowledge, “learning analytics
is the measurement, collection, analysis and reporting
of data about learners and their contexts, for purposes
of understanding and optimising learning and the
environments in which it occurs.”
1. What is the scope?

10. Is this distinction still helpful and valid?
1. What is the scope?

11. • Policy-makers
• Managers
• Heads of Schools
• Programme Leaders
• Instructors
• Learners
1. What is the scope?
- What?
- When?
- How?
- Why?

12. • Policy-makers
• Managers
• Heads of Schools
• Programme Leaders
• Instructors
• Learners
• All of the above
- What?
- When?
- How?
- Why?
1. What is the scope?

13. “A prudent question is one-half of wisdom”
Francis Bacon
http://www.slideshare.net/mbrownz
Questions…

14. 2. What have we learnt?

15. - Collection
- Interpretation
Instructors…
2. What have we learnt?

16. - Action
- Evaluation
- Collection
- Interpretation
Instructors…
2. What have we learnt?

17. Design | Self-regulation | Validity
Learners…
2. What have we learnt?

18. 2. What have we learnt?

19. PredictEd…
A pilot learning analytics project using
LMS data on learner activity to provide
direct feedback to students and
lecturers during Semester.
2. What have we learnt?

20. - Moodle ActivityPredictEd
2. What have we learnt?

21. No significant difference in the entry profiles of
participants vs. non-participants overall
Participant Profile
2. What have we learnt?

22. Subject Description Non-Participant Participant
BE101 Introduction to Cell Biology and Biochemistry 58.89 62.05
CA103 Computer Systems 70.28 71.34
CA168 Digital World 63.81 65.26
ES125 Social&Personal Dev with Communication Skills 67.00 66.46
HR101 Psychology in Organisations 59.43 63.32
LG101 Introduction to Law 53.33 54.85
LG116 Introduction to Politics 45.68 44.85
LG127 Business Law 60.57 61.82
MS136 Mathematics for Economics and Business 60.78 69.35
SS103 Physiology for Health Sciences 55.27 57.03
Overall Dff in all modules 58.36 61.22
Average scores for participants are higher in 8 of the 10
modules analysed.
Academic Performance
2. What have we learnt?

23. 2. What have we learnt?
Survey findings...
Question Group 1 (more
detailed email)
Group 2
% of respondents who opted out of PredictED
during the course of the semester 4.5% 4.5%
% who changed their LMS usage as a result
of the weekly emails
43.3% 28.9%
% who would take part again/are offered and
are taking part again
72.2%
(45.6%/ 26.6% )
76.6%
(46% /30.6% )

24. 2. What have we learnt?

25. 2. What have we learnt?

26. 2. What have we learnt?

27. 2. What have we learnt?

28. Performance takes place in the Network
2. What have we learnt?

29. “Not everything that can be counted
counts, not everything that counts can
be counted.”
Albert Einstein
2. What have we learnt?

30. “A prudent question is one-half of wisdom”
Francis Bacon
http://www.slideshare.net/mbrownz
Questions…

31. 3. What are we going to do?

32. • How have MOOCs been portrayed in the
traditional newspaper media in Ireland?
• What trends are apparent over time in the
portrayal of MOOCs in the Irish media?
• How is the MOOC story being told through
social media?
Research questions…
3. What are we going to do?

33. • Who is telling the MOOC story?
• How are they telling the MOOC story?
• What are we being told about MOOCs?
• What is missing from the MOOC story?
• Whose voice is not being heard?
Deeper questions…
3. What are we going to do?

34. The story
about MOOCs is
not really about MOOCs
“A story is never just a story – it is a statement of belief
and of morality” (Pinar, 2014, p.12).
3. What are we going to do?

35. • Education in change
• Contradictory messages
• Declining level of interest
• Maturing and shifting focus
3. What are we going to do?

36. • Paucity of country specific information
• Need to understand MOOC discourses
at local level
• How have MOOCs been portrayed in
both traditional newspapers and social
media?
Gap in the literature…
3. What are we going to do?

37. Total of 74 publications – by 1st July 2015
Discourse
Analysis
3. What are we going to do?

38. 70%
30%
Type of Reporting
Descriptive
Interpretative
3. What are we going to do?

39. 4%
19%
77%
Stance Adopted
Negative
Neutral
Positive
3. What are we going to do?

40. Elite
73%
Non Elite
21%
NA
6%
Institutional Status
3. What are we going to do?

41. 0 5 10 15 20 25 30
Fear of Missing Out
Institutional Branding
Increase Student Recruitment
Extend Access
Reduce Costs
Generate Revenue
Unbundling of Services
Innovation
Curriculum Renewal
Industry Training
Supporting Student Readiness
Promote Research
Not Stated
Major Drivers
3. What are we going to do?

42. 3. What are we going to do?

43. 3. What are we going to do?
What is Twitter Saying about MOOCs?

44. 3. What are we going to do?
• GNIP API Sep – Dec 2015 -> find #MOOC
• GZIP -> JSON- > MongoDB – > R
• 32,309 tweets (17,910 original)
• Descriptive analytics, content analytics,
network analytics (Chae, 2015; Lynn et. al.,
2015)
Pilot study…

45. 3. What are we going to do?
Word Frequency Word Frequency Word Frequency
mooc 21,199 education 1,126 onlinelearn 501
course 2,733 join 1,113 science 492
learn 2,686 start 1,055 marketing 478
online 2,442 register 954 Data 471
elearn 2,079 onlinecourse 736 Life 447
free 1,911 highered 667
coursera 1,410 digital 562
edxonlin
e
1,332 university 556
edtech 1,259 open 520
business 1,165 students 518
Top 25 Words in the MOOC Dataset

46. 3. What are we going to do?
Hashtags

47. 3. What are we going to do?
Num Topic
33 Promotion of MOOC on “Cognitive Technology and its growing importance for
business” by David Schatsky, course instructor and senior mananger Deloitte LLP.
72 Promotion of Deloitte’s MOOC on 3D printing.
31 Successful Societies (by Princeton University) promoting MOOC on ‘How can
Governments Improve Citizen Services and Cabinet Office Coordination.’
54 Successful Societies promoting MOOC on ‘Making Government work in Hard Places’.
40 Successful Societies promoting MOOC on ‘How Leaders Overcome Governance
Challenges’.
53 Successful Societies promoting MOOC on ‘Writing Science of Delivery Case Studies’
58 NutritionMOOCs promoting MOOC on ‘Nutrition and Health: Micronutrients and
Malnutrition’.
182 NutritionMOOCs promoting 2nd part of MOOC on Nutrition and Health from
Wageningen University.
30 Promotion of Coursera’s MOOC on ‘Training TESOL Certificate Part 1: Teach English’.
43 Retweet of Quizalizeapp’s tweet ‘How to say Merry Christmas in 77 Languages’.
Peak Tweets

48. Sentiment Analysis
Exemplar Tweet Sentiment
Score
@thesiswhisperer I think the #MOOC is providing wonderful supportive
pillow of trust & honesty- glad I'm taking part- thank u #survivephd15
6
STUNNING #mathed animations from .@robertghrist in his calculus
#MOOC. Beautiful and effective. Kudos. http://t.co/GgdN2KHFZf
4
Just discovered a great free #Social Innovation online course, on this
cool learning platform - #iVersity #MOOC ~ http://t.co/kGPzmqONFq
4
#ememitalia Teixeira: focusing on dropout as a problem to criticize
#MOOC education is a conceptual mistake
-4
CloudComputingApplications - definitely the worst @coursera #MOOC
I've ever taken. Irrelevant videos & useless tuts #unenrolled
-3
3. What are we going to do?

49. 3. What are we going to do?
Actions….
1. Complete full analysis of 2015 MOOC tweets
- Conference paper
- Write up and publish in IRRODL
- Develop a conceptual framework
- Develop a paper discussing methodologies
2. DCU to rerun BNU’s MOOC key words and
grounded truth data for cross platform analysis
3. DCU will attempt to analyse Sina Weibo key terms
for 2015 data

50. 3. What are we going to do?
Potential follow ups…
1. Extract Chinese Language posts on Twitter
2. Investigate hypotheses such as do more Chinese
students complain about MOOCs on Twitter?
3. Investigate the use of Twitter and/or Sina Weibo
by educators at MOOC related conferences
4. Lots of other possibilities

51. “It is theory that
decides what we
can observe…”
Albert Einstein
Summary

52. Liverpool Hut, Mt Aspiring

54. ReconceptualizingDeschooling
ReschoolingReproducing
• Competencies
• Entrepreneurship
• Technology as progress
• Increased market competition
• Just society
• Lifelong learning
• Pillars of learning
• Education for change
• Democratic
• Opening access
• Micro credentials
• New learning pathways
• Sifting agent
• Human capital
• Cultural heritage
• Education as commodity
Learning Society
Knowledge Economy
Mark Brown, 2016
TWO CO-EXISTING WORLDVIEWS
• Openness
• Digital learning
• Anytime, anywhere learning
E-learning •
Online learning •
Technology-enhanced learning •

55. Summary
Final remarks…

56. Final remarks…
• Share common interest in MOOCs
Summary

57. Final remarks…
• Share common interest in MOOCs
• Share common interest in Social Media
Summary

58. Final remarks…
• Share common interest in MOOCs
• Share common interest in Social Media
• Share common interest in research on the
use of Educational Technology
Summary

59. http://www.laceproject.eu
Summary

60. Go raibh
maith
agaibh!

61. Professor Mark Brown
Director, National Institute for Digital Learning
www.dcu.ie/nidlmark.brown@dcu.ie
@mbrownz www.slideshare.net/mbrownz