Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Professor Mark Brown
Director, National Institute for Digital Learning
Beijing, China
Thursday 9th June 2016
Learning Anal...
Outline…
1.  What is the scope?
2. What have we learnt?
3. What are we going to do?
Learning Analytics:
A Messy Research C...
“Learning
analytics
should be in
the service of
big ideas,
not as a big idea
in itself” (adapted from Barnett, 2011).
Key ...
1. What is the scope?
1. What is the scope?
1. What is the scope?
What do
we mean by
learning analytics?
How can
we distinguish the
different dimensions?
1. What is the scope?
“According to the 1st International Conference on
Learning Analytics and Knowledge, “learning analytics
is the measurement...
“Academic analytics, in contrast, is the application of
business intelligence in education and emphasizes
analytics at ins...
Is this distinction still helpful and valid?
1. What is the scope?
• Policy-makers
• Managers
• Heads of Schools
• Programme Leaders
• Instructors
• Learners
1. What is the scope?
- What?
-...
• Policy-makers
• Managers
• Heads of Schools
• Programme Leaders
• Instructors
• Learners
• All of the above
- What?
- Wh...
“A prudent question is one-half of wisdom”
Francis Bacon
http://www.slideshare.net/mbrownz
Questions…
2. What have we learnt?
- Collection
- Interpretation
Instructors…
2. What have we learnt?
- Action
- Evaluation
- Collection
- Interpretation
Instructors…
2. What have we learnt?
Design | Self-regulation | Validity
Learners…
2. What have we learnt?
2. What have we learnt?
PredictEd…
A pilot learning analytics project using
LMS data on learner activity to provide
direct feedback to students an...
- Moodle ActivityPredictEd
2. What have we learnt?
No significant difference in the entry profiles of
participants vs. non-participants overall
Participant Profile
2. What h...
Subject Description Non-Participant Participant
BE101 Introduction	to	Cell	Biology	and	Biochemistry 58.89 62.05
CA103 Comp...
2. What have we learnt?
Survey findings...
Question Group 1 (more
detailed email)
Group 2
% of respondents who opted out o...
2. What have we learnt?
2. What have we learnt?
2. What have we learnt?
2. What have we learnt?
Performance takes place in the Network
2. What have we learnt?
“Not everything that can be counted
counts, not everything that counts can
be counted.”
Albert Einstein
2. What have we le...
“A prudent question is one-half of wisdom”
Francis Bacon
http://www.slideshare.net/mbrownz
Questions…
3. What are we going to do?
• How have MOOCs been portrayed in the
traditional newspaper media in Ireland?
• What trends are apparent over time in the...
• Who is telling the MOOC story?
• How are they telling the MOOC story?
• What are we being told about MOOCs?
• What is mi...
The story
about MOOCs is
not really about MOOCs
“A story is never just a story – it is a statement of belief
and of morali...
• Education in change
• Contradictory messages
• Declining level of interest
• Maturing and shifting focus
3. What are we ...
• Paucity of country specific information
• Need to understand MOOC discourses
at local level
• How have MOOCs been portra...
Total of 74 publications – by 1st July 2015
Discourse
Analysis
3. What are we going to do?
70%
30%
Type of Reporting
Descriptive
Interpretative
3. What are we going to do?
4%
19%
77%
Stance Adopted
Negative
Neutral
Positive
3. What are we going to do?
Elite
73%
Non Elite
21%
NA
6%
Institutional Status
3. What are we going to do?
0 5 10 15 20 25 30
Fear of Missing Out
Institutional Branding
Increase Student Recruitment
Extend Access
Reduce Costs
Gene...
3. What are we going to do?
3. What are we going to do?
What is Twitter Saying about MOOCs?
3. What are we going to do?
•  GNIP API Sep – Dec 2015 -> find #MOOC
•  GZIP -> JSON- > MongoDB – > R
•  32,309 tweets (17...
3. What are we going to do?
Word	 Frequency	 Word	 Frequency	 Word	 Frequency	
mooc	 21,199	 education	 1,126	 onlinelearn...
3. What are we going to do?
Hashtags
3. What are we going to do?
Num	 Topic	
33	 Promotion of MOOC on “Cognitive Technology and its growing importance for
busi...
Sentiment Analysis
Exemplar Tweet	 Sentiment
Score	
@thesiswhisperer I think the #MOOC is providing wonderful supportive
p...
3. What are we going to do?
Actions….
1.  Complete full analysis of 2015 MOOC tweets
-  Conference paper
-  Write up and p...
3. What are we going to do?
Potential follow ups…
1.  Extract Chinese Language posts on Twitter
2. Investigate hypotheses ...
“It is theory that
decides what we
can observe…”
Albert Einstein
Summary
Liverpool Hut, Mt Aspiring
ReconceptualizingDeschooling
ReschoolingReproducing
• Competencies
• Entrepreneurship
• Technology as progress
• Increased...
Summary
Final remarks…
Final remarks…
• Share common interest in MOOCs
Summary
Final remarks…
• Share common interest in MOOCs
• Share common interest in Social Media
Summary
Final remarks…
• Share common interest in MOOCs
• Share common interest in Social Media
• Share common interest in researc...
http://www.laceproject.eu
Summary
Go raibh
maith
agaibh!
Professor Mark Brown
Director, National Institute for Digital Learning
www.dcu.ie/nidlmark.brown@dcu.ie
@mbrownz www.slide...
Learning Analytics: A Messy Research Construct
Upcoming SlideShare
Loading in …5
×

Learning Analytics: A Messy Research Construct

404 views

Published on

Invited presentation at Beijing Normal University, 9th June, 2016.

Published in: Education
  • Be the first to comment

Learning Analytics: A Messy Research Construct

  1. 1. Professor Mark Brown Director, National Institute for Digital Learning Beijing, China Thursday 9th June 2016 Learning Analytics: A Messy Research Construct
  2. 2. Outline… 1.  What is the scope? 2. What have we learnt? 3. What are we going to do? Learning Analytics: A Messy Research Construct
  3. 3. “Learning analytics should be in the service of big ideas, not as a big idea in itself” (adapted from Barnett, 2011). Key point…
  4. 4. 1. What is the scope?
  5. 5. 1. What is the scope?
  6. 6. 1. What is the scope?
  7. 7. What do we mean by learning analytics? How can we distinguish the different dimensions? 1. What is the scope?
  8. 8. “According to the 1st International Conference on Learning Analytics and Knowledge, “learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs.” Long, P. & Siemens, G. (2011). Penetrating the fog: analytics in learning and education. EDUCAUSE Review, 46 (5), 31-40 1. What is the scope?
  9. 9. “Academic analytics, in contrast, is the application of business intelligence in education and emphasizes analytics at institutional, regional, and international levels.” Long, P. & Siemens, G. (2011). Penetrating the fog: analytics in learning and education. EDUCAUSE Review, 46 (5), 31-40 “According to the 1st International Conference on Learning Analytics and Knowledge, “learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs.” 1. What is the scope?
  10. 10. Is this distinction still helpful and valid? 1. What is the scope?
  11. 11. • Policy-makers • Managers • Heads of Schools • Programme Leaders • Instructors • Learners 1. What is the scope? - What? - When? - How? - Why?
  12. 12. • Policy-makers • Managers • Heads of Schools • Programme Leaders • Instructors • Learners • All of the above - What? - When? - How? - Why? 1. What is the scope?
  13. 13. “A prudent question is one-half of wisdom” Francis Bacon http://www.slideshare.net/mbrownz Questions…
  14. 14. 2. What have we learnt?
  15. 15. - Collection - Interpretation Instructors… 2. What have we learnt?
  16. 16. - Action - Evaluation - Collection - Interpretation Instructors… 2. What have we learnt?
  17. 17. Design | Self-regulation | Validity Learners… 2. What have we learnt?
  18. 18. 2. What have we learnt?
  19. 19. PredictEd… A pilot learning analytics project using LMS data on learner activity to provide direct feedback to students and lecturers during Semester. 2. What have we learnt?
  20. 20. - Moodle ActivityPredictEd 2. What have we learnt?
  21. 21. No significant difference in the entry profiles of participants vs. non-participants overall Participant Profile 2. What have we learnt?
  22. 22. Subject Description Non-Participant Participant BE101 Introduction to Cell Biology and Biochemistry 58.89 62.05 CA103 Computer Systems 70.28 71.34 CA168 Digital World 63.81 65.26 ES125 Social&Personal Dev with Communication Skills 67.00 66.46 HR101 Psychology in Organisations 59.43 63.32 LG101 Introduction to Law 53.33 54.85 LG116 Introduction to Politics 45.68 44.85 LG127 Business Law 60.57 61.82 MS136 Mathematics for Economics and Business 60.78 69.35 SS103 Physiology for Health Sciences 55.27 57.03 Overall Dff in all modules 58.36 61.22 Average scores for participants are higher in 8 of the 10 modules analysed. Academic Performance 2. What have we learnt?
  23. 23. 2. What have we learnt? Survey findings... Question Group 1 (more detailed email) Group 2 % of respondents who opted out of PredictED during the course of the semester 4.5% 4.5% % who changed their LMS usage as a result of the weekly emails 43.3% 28.9% % who would take part again/are offered and are taking part again 72.2% (45.6%/ 26.6% ) 76.6% (46% /30.6% )
  24. 24. 2. What have we learnt?
  25. 25. 2. What have we learnt?
  26. 26. 2. What have we learnt?
  27. 27. 2. What have we learnt?
  28. 28. Performance takes place in the Network 2. What have we learnt?
  29. 29. “Not everything that can be counted counts, not everything that counts can be counted.” Albert Einstein 2. What have we learnt?
  30. 30. “A prudent question is one-half of wisdom” Francis Bacon http://www.slideshare.net/mbrownz Questions…
  31. 31. 3. What are we going to do?
  32. 32. • How have MOOCs been portrayed in the traditional newspaper media in Ireland? • What trends are apparent over time in the portrayal of MOOCs in the Irish media? • How is the MOOC story being told through social media? Research questions… 3. What are we going to do?
  33. 33. • Who is telling the MOOC story? • How are they telling the MOOC story? • What are we being told about MOOCs? • What is missing from the MOOC story? • Whose voice is not being heard? Deeper questions… 3. What are we going to do?
  34. 34. The story about MOOCs is not really about MOOCs “A story is never just a story – it is a statement of belief and of morality” (Pinar, 2014, p.12). 3. What are we going to do?
  35. 35. • Education in change • Contradictory messages • Declining level of interest • Maturing and shifting focus 3. What are we going to do?
  36. 36. • Paucity of country specific information • Need to understand MOOC discourses at local level • How have MOOCs been portrayed in both traditional newspapers and social media? Gap in the literature… 3. What are we going to do?
  37. 37. Total of 74 publications – by 1st July 2015 Discourse Analysis 3. What are we going to do?
  38. 38. 70% 30% Type of Reporting Descriptive Interpretative 3. What are we going to do?
  39. 39. 4% 19% 77% Stance Adopted Negative Neutral Positive 3. What are we going to do?
  40. 40. Elite 73% Non Elite 21% NA 6% Institutional Status 3. What are we going to do?
  41. 41. 0 5 10 15 20 25 30 Fear of Missing Out Institutional Branding Increase Student Recruitment Extend Access Reduce Costs Generate Revenue Unbundling of Services Innovation Curriculum Renewal Industry Training Supporting Student Readiness Promote Research Not Stated Major Drivers 3. What are we going to do?
  42. 42. 3. What are we going to do?
  43. 43. 3. What are we going to do? What is Twitter Saying about MOOCs?
  44. 44. 3. What are we going to do? •  GNIP API Sep – Dec 2015 -> find #MOOC •  GZIP -> JSON- > MongoDB – > R •  32,309 tweets (17,910 original) •  Descriptive analytics, content analytics, network analytics (Chae, 2015; Lynn et. al., 2015) Pilot study…
  45. 45. 3. What are we going to do? Word Frequency Word Frequency Word Frequency mooc 21,199 education 1,126 onlinelearn 501 course 2,733 join 1,113 science 492 learn 2,686 start 1,055 marketing 478 online 2,442 register 954 Data 471 elearn 2,079 onlinecourse 736 Life 447 free 1,911 highered 667 coursera 1,410 digital 562 edxonlin e 1,332 university 556 edtech 1,259 open 520 business 1,165 students 518 Top 25 Words in the MOOC Dataset
  46. 46. 3. What are we going to do? Hashtags
  47. 47. 3. What are we going to do? Num Topic 33 Promotion of MOOC on “Cognitive Technology and its growing importance for business” by David Schatsky, course instructor and senior mananger Deloitte LLP. 72 Promotion of Deloitte’s MOOC on 3D printing. 31 Successful Societies (by Princeton University) promoting MOOC on ‘How can Governments Improve Citizen Services and Cabinet Office Coordination.’ 54 Successful Societies promoting MOOC on ‘Making Government work in Hard Places’. 40 Successful Societies promoting MOOC on ‘How Leaders Overcome Governance Challenges’. 53 Successful Societies promoting MOOC on ‘Writing Science of Delivery Case Studies’ 58 NutritionMOOCs promoting MOOC on ‘Nutrition and Health: Micronutrients and Malnutrition’. 182 NutritionMOOCs promoting 2nd part of MOOC on Nutrition and Health from Wageningen University. 30 Promotion of Coursera’s MOOC on ‘Training TESOL Certificate Part 1: Teach English’. 43 Retweet of Quizalizeapp’s tweet ‘How to say Merry Christmas in 77 Languages’. Peak Tweets
  48. 48. Sentiment Analysis Exemplar Tweet Sentiment Score @thesiswhisperer I think the #MOOC is providing wonderful supportive pillow of trust & honesty- glad I'm taking part- thank u #survivephd15 6 STUNNING #mathed animations from .@robertghrist in his calculus #MOOC. Beautiful and effective. Kudos. http://t.co/GgdN2KHFZf 4 Just discovered a great free #Social Innovation online course, on this cool learning platform - #iVersity #MOOC ~ http://t.co/kGPzmqONFq 4 #ememitalia Teixeira: focusing on dropout as a problem to criticize #MOOC education is a conceptual mistake -4 CloudComputingApplications - definitely the worst @coursera #MOOC I've ever taken. Irrelevant videos & useless tuts #unenrolled -3 3. What are we going to do?
  49. 49. 3. What are we going to do? Actions…. 1.  Complete full analysis of 2015 MOOC tweets -  Conference paper -  Write up and publish in IRRODL -  Develop a conceptual framework -  Develop a paper discussing methodologies 2. DCU to rerun BNU’s MOOC key words and grounded truth data for cross platform analysis 3. DCU will attempt to analyse Sina Weibo key terms for 2015 data
  50. 50. 3. What are we going to do? Potential follow ups… 1.  Extract Chinese Language posts on Twitter 2. Investigate hypotheses such as do more Chinese students complain about MOOCs on Twitter? 3.  Investigate the use of Twitter and/or Sina Weibo by educators at MOOC related conferences 4.  Lots of other possibilities
  51. 51. “It is theory that decides what we can observe…” Albert Einstein Summary
  52. 52. Liverpool Hut, Mt Aspiring
  53. 53. ReconceptualizingDeschooling ReschoolingReproducing • Competencies • Entrepreneurship • Technology as progress • Increased market competition • Just society • Lifelong learning • Pillars of learning • Education for change • Democratic • Opening access • Micro credentials • New learning pathways • Sifting agent • Human capital • Cultural heritage • Education as commodity Learning Society Knowledge Economy Mark Brown, 2016 TWO CO-EXISTING WORLDVIEWS • Openness • Digital learning • Anytime, anywhere learning E-learning • Online learning • Technology-enhanced learning •
  54. 54. Summary Final remarks…
  55. 55. Final remarks… • Share common interest in MOOCs Summary
  56. 56. Final remarks… • Share common interest in MOOCs • Share common interest in Social Media Summary
  57. 57. Final remarks… • Share common interest in MOOCs • Share common interest in Social Media • Share common interest in research on the use of Educational Technology Summary
  58. 58. http://www.laceproject.eu Summary
  59. 59. Go raibh maith agaibh!
  60. 60. Professor Mark Brown Director, National Institute for Digital Learning www.dcu.ie/nidlmark.brown@dcu.ie @mbrownz www.slideshare.net/mbrownz

×