Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Longitudinal analysis of students’ learning gains in Higher Education across two UK institutions

116 views

Published on

Although learning gain as a concept is relatively easy to define, its measurement is potentially problematic.
Building on initial work presented at SRHE2016, in this follow-up study amongst two “traditional” universities
we sought to replicate the feasibility of using assessment grades as a measure of learning gain. Our
multi-level growth analyses of 3,537 students across 2 * 20 degree programmes indicated on average
students showed improvement in standardised grades, although this was only significant for one university.
Furthermore, the variance explained differed between the levels, whereby University 2 had more variance at
the departmental level and within students than University 1, while at the University 1 variance was mainly
nested between students. This has important implications for TEF when assessing learning gains at an
institutional level, as aggregate learning gains estimates can result in misleading estimates of students’
learning gains.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Longitudinal analysis of students’ learning gains in Higher Education across two UK institutions

  1. 1. Longitudinal analysis of students’ learning gains in Higher Education across two UK institutions RHONA SHARPE JEKATERINA ROGATEN Marius Jugariu Ceri Hitchings Ian Scott Bart Rienties Ian Kinchin Simon Lygo-Baker
  2. 2. Defining and measuring learning gains  “LA is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs” (LAK 2011)  Learning Gains is a growth or change in knowledge, skills and abilities over time that can be linked to the desired learning outcomes or goals of the course
  3. 3.  Pre-post standardised testing is resource intensive and becomes even more so if one wants to estimate learning gains across various disciplines and number of universities  Advantages:  Assessment data readily available  Widely recognized as appropriate measure of learning  Relatively free from self-reported biases  Allows a direct comparison of research finding with the results of other studies Can assessment data be a measure of learning gain?
  4. 4. Three-level Growth Curve Model Level 1 Level 2 Level 3 G1 Student1 G3 G1 G2 G3 G1 G2 G3G2 Student2 Student3 Qualification 1 Qualification 2 G1 G2 G3 Student4 G1 G2 G3 Student5 Qualification 3
  5. 5. Present Study  University 1  1,990 undergraduate students  University 2  1,547 undergraduate students  20 degree programmes within each university  DV – average grade yearly grade University 1 University 2 Year M SD M SD 1 60.65 7.62 63.75 12.66 2 61.31 6.81 65.64 12.74 3 63.32 6.63 64.12 14.02
  6. 6. Data Analysis: Descriptive Plots University 1 University 2
  7. 7. Data Analysis: Model comparison University 1 Regression S.E. 2-levels S.E. 3-levels S.E. Intercept B0 60.252 0.114 60.2 0.16 60.337 0.687 Slope B1 0.429** 0.027 0.422** 0.024 0.365** 0.111 Deviance 74016.17 68227.62 67983.61 X2 change 5788.548** 244.012** University 2 Intercept B0 64.322 0.325 64.225 0.323 63.626 1.419 Slope B1 0.184 0.251 0.22 0.211 -0.131 0.723 Deviance 33145.79 32600.89 32255.87 X2 change 544.898** 345.028**
  8. 8. Variance partitioning University 1 University 2 Variance at Department level 13.1% 22% Variance between students 59.8% 22% Variance within students (between years) 27.1% 56%
  9. 9. Summary of findings  Although both universities overall showed positive gains, substantial differences were present in variance at departmental level.  Aggregate learning gains estimates can result in misleading estimates of students’ learning gains on a discipline or degree level.  Multilevel modelling is a more accurate method in comparison with simple linear models when estimating students’ learning gains.
  10. 10. Possible implications  Support for subject level TEF  Guidance on where to focus interventions and resources  Visualisation could promote data informed learning design decisions Questions still to answer:  Does grade trajectory reflect students’ learning gains?  Can we make a meaningful comparison between universities?  What impact grade trajectory has on students? Do they need to know their own trajectory and how it compares to others?
  11. 11. Longitudinal analysis of students’ learning gains in Higher Education across two UK institutions RHONA SHARPE JEKATERINA ROGATEN Marius Jugariu Ceri Hitchings Ian Scott Bart Rienties Ian Kinchin Simon Lygo-Baker https://twitter.com/LearningGains https://abclearninggains.com/

×