Data Wranglers:
Human data interpreters
to close the feedback loop
Doug Clow
Institute of Educational Technology
The Open ...
The Open University
• Largest university in the UK
• 200,000 students
• 7,000 tutors
• 1,000 academic staff
• Supported op...
At scale, each year
• 600 courses
• > 200,000 students
• > 1 million assignments
• > 1 billion views of OU/BBC coproductio...
data wrangling
“Learning is a complex social activity” (Siemens, 2012)
• Lots of data
• Lots of tools
• Humans to
make sense
Photo CC (BY...
Data Wrangling
• Institutional capacity
building
• 8 Wranglers
• Faculty link
• Regular reports
Data
• Delivery data
• Completion,
pass rates,
demographics
Pelophylax perezi Photo CC (BY-SA) Paulo Brandão on Flickr: ht...
Photo CC (BY) Preneur d‟Image on Flickr: http://www.flickr.com/photos/surbykids/9368298544/
Report Process
• Briefing disc...
Photo (cc)-BY-NC nataliej on Flickr: http://www.flickr.com/photos/nataliejohnson/2122722198/
• Scrutiny in draft
• Standar...
Responsive „quick reports‟
Photo (CC)-BY Mark Dumont on Flickr: http://www.flickr.com/photos/wcdumonts/9256493975/
Doing e...
reaction
users data
users data
users data
examples
0
100
200
300
400
500
600
700
800
0 1 2 3 4 5 6 7 8 9 10 11 12 13
Forum Wiki Pages Quiz
Unique visits per week to VLE comp...
Course use of Elluminate
Students using
Elluminate
None 18%
Informal 27%
General student support 35%
Specified activities ...
conclusions
Photo CC (BY-NC-SA) quas on Flickr http://www.flickr.com/photos/quas/1703493/
Evaluation:
• Data quality
• Data quantity
•...
Photo CC (BY-NC) Alexey Kljatov on Flickr http://www.flickr.com/photos/chaoticmind75/9516811453/
It is only through the de...
• Gill Kirkup, Mary Thorpe, Alison Ashby & her team
– esp. Vicky Marsh and Jim Peard
• Oliver Millard
• Evaghn DeSouza
• M...
Doug Clow
@dougclow
dougclow.org
doug.clow@open.ac.uk
This work is licensed under a Creative Commons Attribution 3.0 Unpor...
cc licensed ( BY ) flickr photo by David Goehring: http://flickr.com/photos/carbonnyc/33413040/
Doing extras
• 60 points vs 30 points
Photo (CC)-BY Steve Snodgrass on Flickr: http://www.flickr.com/photos/stevensnodgras...
cc licensed ( BY ) flickr photo by zzpza: http://www.flickr.com/photos/zzpza/3269784239/
Workshops
• Integrated/joint Lear...
(CC) Eoin Gardner on Flickr http://www.flickr.com/photos/18091975@N00/5310045271/
The impersonal is political
• Data used ...
Data
• Delivery data
• Completion,
pass rates,
demographics
Photo CC (BY-NC) ChaoticMind75 on Flickr: http://www.flickr.co...
Data Wranglers: Human data interpreters to close the feedback loop
Data Wranglers: Human data interpreters to close the feedback loop
Upcoming SlideShare
Loading in …5
×

Data Wranglers: Human data interpreters to close the feedback loop

613 views
494 views

Published on

Presentation at LAK14, the fourth international conference on Learning Analytics and Knowledge, Indianapolis, March 2014.

Published in: Education, Sports, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
613
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
8
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Data Wranglers: Human data interpreters to close the feedback loop

  1. 1. Data Wranglers: Human data interpreters to close the feedback loop Doug Clow Institute of Educational Technology The Open University, UK
  2. 2. The Open University • Largest university in the UK • 200,000 students • 7,000 tutors • 1,000 academic staff • Supported open learning Milton Keynes
  3. 3. At scale, each year • 600 courses • > 200,000 students • > 1 million assignments • > 1 billion views of OU/BBC coproductions • > 3 million Moodle transactions per day Photo (cc) Marieke IJsendoorn-Kuijpers http://www.flickr.com/photos/mape_s/333862026//
  4. 4. data wrangling
  5. 5. “Learning is a complex social activity” (Siemens, 2012) • Lots of data • Lots of tools • Humans to make sense Photo CC (BY) Tobias Lindman on Flickr: http://www.flickr.com/photos/cowb0y2000/6243008944/
  6. 6. Data Wrangling • Institutional capacity building • 8 Wranglers • Faculty link • Regular reports
  7. 7. Data • Delivery data • Completion, pass rates, demographics Pelophylax perezi Photo CC (BY-SA) Paulo Brandão on Flickr: http://www.flickr.com/photos/paulobrandao/3567716506/ • Student feedback • Activity data (Moodle)
  8. 8. Photo CC (BY) Preneur d‟Image on Flickr: http://www.flickr.com/photos/surbykids/9368298544/ Report Process • Briefing discussion • Data analysis and report writing • Draft discussion • Final report
  9. 9. Photo (cc)-BY-NC nataliej on Flickr: http://www.flickr.com/photos/nataliejohnson/2122722198/ • Scrutiny in draft • Standard document template • Executive summary • Recommendations Quality assurance
  10. 10. Responsive „quick reports‟ Photo (CC)-BY Mark Dumont on Flickr: http://www.flickr.com/photos/wcdumonts/9256493975/ Doing extras
  11. 11. reaction
  12. 12. users data
  13. 13. users data
  14. 14. users data
  15. 15. examples
  16. 16. 0 100 200 300 400 500 600 700 800 0 1 2 3 4 5 6 7 8 9 10 11 12 13 Forum Wiki Pages Quiz Unique visits per week to VLE components for one course
  17. 17. Course use of Elluminate Students using Elluminate None 18% Informal 27% General student support 35% Specified activities not assessed 49% Specified activities referenced in assessment 95% Usage of Elluminate broken down by course use of Elluminate for one Faculty between 2011- 2012.
  18. 18. conclusions
  19. 19. Photo CC (BY-NC-SA) quas on Flickr http://www.flickr.com/photos/quas/1703493/ Evaluation: • Data quality • Data quantity • Unevenness • High cost • Slow • Bottom up
  20. 20. Photo CC (BY-NC) Alexey Kljatov on Flickr http://www.flickr.com/photos/chaoticmind75/9516811453/ It is only through the detailed process of engagement and dialogue between analysts, stakeholders and the data that insight and organisational change are developed.
  21. 21. • Gill Kirkup, Mary Thorpe, Alison Ashby & her team – esp. Vicky Marsh and Jim Peard • Oliver Millard • Evaghn DeSouza • My fellow Data Wranglers Thanks to …
  22. 22. Doug Clow @dougclow dougclow.org doug.clow@open.ac.uk This work is licensed under a Creative Commons Attribution 3.0 Unported License
  23. 23. cc licensed ( BY ) flickr photo by David Goehring: http://flickr.com/photos/carbonnyc/33413040/
  24. 24. Doing extras • 60 points vs 30 points Photo (CC)-BY Steve Snodgrass on Flickr: http://www.flickr.com/photos/stevensnodgrass/4034636727/
  25. 25. cc licensed ( BY ) flickr photo by zzpza: http://www.flickr.com/photos/zzpza/3269784239/ Workshops • Integrated/joint Learning Design/Data Wrangling • Module production workshops –Intensive all-week workshop –Many presentations (LTS, other relevant modules) –IET: Data Wrangling and Learning Design
  26. 26. (CC) Eoin Gardner on Flickr http://www.flickr.com/photos/18091975@N00/5310045271/ The impersonal is political • Data used as weaponry in ongoing battles
  27. 27. Data • Delivery data • Completion, pass rates, demographics Photo CC (BY-NC) ChaoticMind75 on Flickr: http://www.flickr.com/photos/chaoticmind75/8629790711/ • Student feedback • Activity data (Moodle)

×