This document summarizes a workshop on using data to inform blended learning design. The workshop covered learning design principles and techniques, introduced a new guide on freely available tools and techniques, and discussed how institutions currently approach learning design and use data. Participants worked through examples of relating learning analytics data to evaluating and improving learning design. The workshop aimed to help participants apply data more purposefully to answer questions about students' experience and progress through a course.

1. Data-informed blended learning design:March 2018
Pilot workshop
Image CC BY-NC-ND 2.0 iandolphin

2. Overview
» Morning – learning design
» 13.00 (approx) - Lunch
» Afternoon – using data to improve
learning design
» 16.00 - Close
Outline of the day
Image CC BY-SA 2.0 Aspire-edu

3. New guide
» Based on interviews & case studies
» Aimed at FE and HE
» Freely available tools & techniques
» ji.sc/design-learn-assess
Appreciative inquiry approach
Image CC BY-NC-ND Jisc

4. What is learning design?
» The design of learning as a purposeful
activity with a planned outcome
» Course, modules, lessons, activities
A spectrum …

5. What is learning analytics?
» The application of big data
techniques such as machine based
learning and data mining to help
learners and institutions meet their
goals
–improve retention
–improve achievement
–reduce differential outcomes
Jisc definition
Image CC BY-SA 2.0 Aspire-edu

6. Shift happens
'Curriculum design in higher education is
not a formal activity and there is little
support, formal or informal, provided in
most higher education institutions to
help academics become better at
designing learning activities, modules
and courses.'
Nicol, D. (2012). Transformational Change in Teaching and Learning Recasting the Educational Discourse: evaluation of the Viewpoints
project. Jisc. http://wiki.ulster.ac.uk/download/attachments/23200594/Viewpoints_Evaluation_Report.pdf
Image CC BY-SA 2.0 Aspire-edu

8. Your approach to learning design
What is the approach to
learning design in your
organisation at the moment?
Go to www.menti.com
and use the code
78 82 52
Image CC BY-SA 2.0 Aspire-edu

9. Learning design: workshop audience

10. Your use of data
What data do you
currently use to inform
teaching practice?
Go to www.menti.com
and use the code
89 93 79
Image CC BY-SA 2.0 Aspire-edu

11. Use of data: workshop audience

12. Time for aTrojan horse
» Support for learning design is often ad hoc
rather than strategic
» Various types of learning analytics but
little focus on curriculum as yet
» Buzzword bingo! Digital strategy; Blended
learning; Learning analytics; Dashboards;
Big data!
» Need to relate data to day-to-day
learning and teaching practice
Data-informed learning design
Image CC BY-NC-ND 2.0 emmapatsie

13. » Tried and tested
» Scaleable for all types of learning
» Storyboard approach
» Based on sound research
» Find out more and download the
resources:
http://blogs.ucl.ac.uk/abc-ld/
ABC learning design
Easy as ABC
Image CC BY-SA 2.0 Aspire-edu

14. » Clarity of purpose – tweet your module
» Sis types of learning – what’s your mix
» Storyboard the design
» Select the activities
» What will you assess?
» What has changed?
» Find out more:
http://blogs.ucl.ac.uk/abc-ld/
The ABC learning design approach
ABC step by step
Image CC BY-SA 2.0 Aspire-edu

15. Lunch break
Image CC BY-NC 2.0 motivational pasta

16. Now that you have a design
what might impact how it
works in practice ?
Design and fit
Image CC BY-SA 2.0 Aspire-edu

17. Now that you have a design what
might impact how it works in
practice ?
Design and fit
Image CC BY-SA 2.0 Aspire-edu
» Teaching
» Student capabilities
» Student behaviour
» Course level factors
» Assessment
» External factors
What data can help you tell how things
are working?

18. Typical data sets: Digifest audience

19. Typical data sets: workshop audience

20. Why don’t you make full
use of available data?
And so ...
Image CC BY-NC 2.0 motivational pasta

21. » What do you really want to know?
» Fit with academic practice
» Link to threshold concepts
» Link to self-study activities
» Is there a ‘critical path’ through the
learning?
» Answering specific questions not looking
for patterns
What data is most useful?
Designing in data ‘hooks’
Image CC BY-NC-ND 2.0 ian dolphin

22. Activity
Using the example you created
» What questions should we be asking about the effectiveness of the learning
design?
» What data should we collect to help answer these questions?
» Can you amend the design to help this process?

23. » Digital in the blend to give a data footprint
» In-flight vs post hoc interventions
» Can we identify a critical path?
» Appropriate level of granularity?
» From engagement to attainment
Data & design considerations
Art or science?
Image CC BY 2.0 kent schimke

24. Using data to explore learning design
»Open University study into
impact of learning design
»Learning design data
»Student feedback
»Student records
»VLE logs
»140-300 modules, taken by
over 100,000 students.
10/09/2015 Moving from electronic management of assessment challenges to solutions

25. Average time spent per week in VLE

26. Constructivist
Learning Design
Assessment
Learning Design
Productive
Learning Design
Socio-construct.
Learning Design
Student
Satisfaction
Student
retention
150+ modules
VLE Engagement
Week 1 Week 2 Week30
+
Rienties, B., Toetenel, L., (2016). The impact of learning design on student behaviour, satisfaction and performance: a cross-institutional comparison across
151 modules. Computers in Human Behavior, 60 (2016), 333-341
Nguyen, Q., Rienties, B., Toetenel, L., Ferguson, R., Whitelock, D. (2017). Examining the designs of computer-based assessment and its impact on student
engagement, satisfaction, and pass rates. Computers in Human Behavior. DOI: 10.1016/j.chb.2017.03.028.
Communication

27. Toetenel, L., Rienties, B. (2016) Learning Design – creative design to visualise learning activities. Open Learning.

28. Improving the student learning experience

29. Recommendations
» Individual tutors/Module leads
» Course/Programme leads
» Dept/Faculty/Institution
» Staff developers
» Any other roles …
Ideas to be taken forward by ...
Image CC BY-SA 2.0 jiscinfonet

30. Next steps
» What do you most want to know about
your learners?
» What’s the most important data source
that you don’t currently have?
» What kinds of support do you need?
How can Jisc help
Image CC BY-SA 2.0 Aspire-edu

31. » Designing learning and assessment in a digital age ji.sc/design-learn-assess
» Jisc learning analytics project
https://www.jisc.ac.uk/rd/projects/effective-learning-analytics
» Student experience blog https://elearning.jiscinvolve.org
» Learning design network https://sites.google.com/site/learningdesignsig/home
Key resources
Further information

32. Contact us
Sam Ahern
s.ahern@ucl.ac.uk
Gill Ferrell
gill@aspire-edu.org
Patrick Lynch
p.lynch@hull.ac.uk
Nataša Perović
n.perovic@ucl.ac.uk
CliveYoung
c.p.l.young@ucl.ac.uk

33. jisc.ac.uk
Except where otherwise noted, this work
is licensed under CC-BY-NC-ND
Talk to Jisc…
Contact
Sarah Knight
Head of change
Student Experience
sarah.knight@jisc.ac.uk