@cbkandiko visiting our ABC learning gains project today @LearningGains. Exciting :-)

1. Affective-Behaviour-
Cognition (ABC)
Learning Gains
Project
Open University 15-02-2016
https://twitter.com/LearningGains

2. ABC approach

3. Project management
• Project team:
• Three universities bring together their expertise to the ABC project
• Project management:
• Project team meets monthly over skype
• Everyone has access to shared folders
• Within universities meetings are held every week
• 2nd face-to-face meeting of all partners (Sep)

4. Open University Team
• Strength
• Learning analytics
• Quantitative data analysis
• Assessment
• Online learning and learning design
• Cross-sector thinking and working
• Responsibilities
• Overall leading the project
• Leading the Phase 1 of the project
(secondary data analysis)
• Phase 1 ethics applications
• Developing models of learning gains
Dr Bart Rienties
Dr Jekaterina Rogaten
Prof Denise Whitelock
Dr Simon Cross
Prof Allison Littlejohn

5. Open University
1. Learning Analytics
Largest VLE datasets of affect, behaviour and
cognition of 200K students, with 2 large scale
implementations of predictive modelling
2. Learning Design
Extensive modelling of learning designs (OULDI)
3. Student Experience on a Module
Extensive datasets of students’ experiences with
blended and online learning

6. Oxford Brooks University Team
Prof Rhona Sharpe
Dr Ian Scott
Strength
Expertise in learners’ experience
Mixed-methods approach
Measures of learning and engagement
Learning analytics
Responsibilities
Co-leading Phase 2 of the project
In depth-interviews and diary data collection
Testing validity of self report measures of learning gains
Triangulation

7. Oxford Brookes University
1. Academic Performance Tracking Tool
Presents staff with data from student information
system, NSS and module evaluations in form of
dashboards used for module and programme
reviews.
2. Grade Point Average
Provides students with information about their
progress. Provides staff with a more nuanced
indicator of attainment.
3. Survey of Student Engagement
Developed own scales for monitoring student
engagement with Assessment Compact, Academic
Advising and Graduate Attributes.

8. University of Surrey Team
Responsibilities
Co-leading Phase 2 of the project
Testing the validity of the self-reported measures of learning
gains (+OB)
Testing the validity of learning analytics modelling (+OU)
Dissemination: website and Twitter
Prof Ian Kinchin Prof Steven Warburton Dr Simon Lygo-Baker
Strength
University pedagogy and concept mapping
Educational technologies
Expertise in innovative teaching approaches development

9. University of Surrey
• Business Intelligence - Progression Analysis project has built / is building:
• Live data on students who are at risk, to act as an early warning indicator so that active support
measures can be put in place;
• Visualisation of data via dashboards for academics to put in place appropriate actions;
• Extension into WP & Outreach domain with specific data feeds, modelling, and a WP progression
dashboard.
• Institutional success:
• Ability to makes links between what gains our students make in correlation with an institution which
has high levels of satisfaction and has enhanced learning and teaching across all Faculty.

10. WP5 – social media
presence: Wordpress
blog and Twitter feed

11. Learning gain literature review – in progress
1) Self-reported measures of learning gain
2) Average Normalized gain (post-pre)/(Nq-pre)
3) Percentage change (Post-Pre)/Pre
4) Raw change (Post-Pre)
• Literature review shows that the concept of learning gain primarily used to examine
the effect of any particular educational ‘intervention’
• Web of Science core collection
• The time frame of the search 1995 - 2015.
• The key word search included: “learning gain*”, higher education, college, graduate, not school,
not child.
• In total 188 studies were identified.
• Review criteria:
1) Does the study assesses learning gain?
2) What learning gain was examined i.e., behavioural, affective, cognitive?
3) How was it measured?

12. Structure of the
ABC project

13. Timeline

14. Current progress
• Phase 1
• Ethics approval
• Ethics are now obtained for the
Phase 1 at the OU
• Secondary data analysis
• Collected (OU)
• Demographics data
• Academic performance data
• Module information data
• In progress (OU)
• VLE data (e.g., discussion
participation, number of visits,
length of visits)
• Students satisfaction data
• Secondary data analysis
• Data cleaning and descriptive
statistics

15. Current progress: Example Arts AA100
• The arts past and present
• 30 week, 60 credit, Level 1 module
• Introduces to university-level study in the arts across a range of subject areas - art history, classical studies, English,
history, philosophy, music and religious studies.
• Assignment 01 (10%)
• Assignment 02 (10%);
• Assignment 03 (15%);
• Assignment 04 (20%);
• Assignment 05 (15%);
• Assignment 06 (20%);
• Assignment 07 (10%)
• 25% - BA (Honours) History, 19% - BA (Honours) Humanities, 16.2% - BA (Honours) English Language and
Literature, 12.5% - BA (Honours) English Literature
• Design: 75% Assimilative, 20% Assessment, 4% Productive

16. Participants
3262 students from years 2013/14 and 2014/15
2147 Females (age M=33.2, SD=12.8) & 1115 Males (age M=35.5, SD=14.2)
Motivational orientation:
36.8% - employment and personal development, 13.6% - employment, 23.9% - personal development
Occupational status:
43.9% in full-time employment, 24.2% part-time employment, 13.4% not in paid work, 10.9% unemployed
93.2% White; 19% reported study disability
50% have A level or equivalent, 30.9% have below A level, 14.3% have HE qualification
81.4% were only taking AA100 module

17. Current progress: Example Arts AA100

18. Current progress: Example Arts AA100

19. Current progress: Example Arts AA100

20. Next step: Multilevel Modelling
Level 1
Level 2
Level 3
ABC1
Student1
ABC3 ABC1 ABC2 ABC3 ABC1 ABC2 ABC3ABC2
Student2 Student3
Course1 Course2
ABC1 ABC2 ABC3
Student4
ABC1 ABC2 ABC3
Student5
Course3

21. Dissemination
• Inside government - Measuring Learning Gain in Higher Education (May)
• Presentation on ABC project
• EARLI conference (August)
• Presentation of the 3-level model for OU students sample
• SRHE conference (December)
• Presentation of the 4-level model for OU+OB+US students samples
• Stakeholder dissemination event (Sep)
• [BART TO ADD DETAILS]
• Literature review (April)
• Meta-analysis to be submitted to Educational Research Review or Review of
Educational Research.

22. Affective-
Behaviour-
Cognition (ABC)
Learning Gains
Project
Open University 15-02-2016
https://twitter.com/LearningGains