Download to read offline
![What students think they gain?
I think I am more openly critical
(in the positive sense)
Day to day when I have my book I have very different
approach from recording my notes for example
[in my new job], there will reports and
planning to be drawn and I think that this will
be an aspect of my job where I can say yes
the OU study and discipline I’ve received
from the OU has actually contributed to that.
I observe things better, work into deeper and
work on the whole picture rather than narrow.
I think more logically and more ‘why did that
happen, why did that happen’, there is more
questioning, instead of just to accept things.
I am much better at time management, I am much more
organised now and planning things in advance.
now I say, ‘you know what, I can do that in future’.
I feel more confident and I am happier
because I am doing something I have always
wanted to be doing and something that
interests me
I think I can go confidently to
speak what I learned. But
even to a job that isn’t directly
related to this subject area. I
could talk about my
experiences, my time
management, team working,
computer skills as I feel much
more confident, I can say,
‘actually I have done this’.
Which was one of the
reasons I wanted to a degree.](https://image.slidesharecdn.com/bigdata-bartrjekaterinar-openuniversity-180326095115/75/Are-assessment-scores-good-proxies-of-estimating-learning-gains-3-2048.jpg)
![Do grades matter?
How well do your grades represent your progress?
probably in the same way that many other people when
they look at their own assignment results and exam results
…. I feel that I am doing fairly well but I’d always like to
improve myself to my results.
I get quite upset when I get around 70s
… because I am putting so much effort I want my grades to reflect it.
They usually go up. But it is Marginal. 5 marks across all the TMAs
that’s the variance, it just varies very slightly
Even if it is 1-2 marks I say what did I do differently and
I go back to tutor to see what did I do differently. What
happened, what caused it?
Well there are questions with the text books, exercises. So
if I get correct answer, I know I am doing fine. When I say
correct answer that’s not the end product that’s the whole
answer check through it
“I suppose you could say… the skills you learn, like group work, presenting and being able to talk to people…
I would say the main way that you think about [achievement], it’s just the grade because… that’s what is going
on your CV… and affect what job you get. … I’d say the skills you learn as well as becoming an all-rounded person
are quite important as well”.](https://image.slidesharecdn.com/bigdata-bartrjekaterinar-openuniversity-180326095115/75/Are-assessment-scores-good-proxies-of-estimating-learning-gains-4-2048.jpg)
Uploaded byHigher Education Funding Council for England
Are assessment scores good proxies of estimating learning gains?
This study examined learning gains among humanities and science students using assessment scores from a large sample of over 17,000 students. The researchers found that science students showed more variance in initial achievement and learning gains compared to social science students. Socio-demographic factors like gender, ethnicity and prior education explained more variance in learning gains for science students. The study also compared learning gains across universities, finding substantial differences at the departmental level, suggesting that aggregate estimates can be misleading. Multilevel modeling provided a more accurate method for estimating learning gains compared to simple linear models.
More Related Content
Similar to Are assessment scores good proxies of estimating learning gains?
More from Higher Education Funding Council for England
Are assessment scores good proxies of estimating learning gains?
- 1. Are assessment scoresgood proxies of estimating learning gains: a large-scale study amongst humanities and science students Dr. Jekaterina Rogaten Prof Bart Rienties https://twitter.com/LearningGains https://abclearninggains.com/
- 2. ABC project tomeasure learning gains? Affect Behaviour Cognition Academic performance VLE Satisfaction
- 3. What students thinkthey gain? I think I am more openly critical (in the positive sense) Day to day when I have my book I have very different approach from recording my notes for example [in my new job], there will reports and planning to be drawn and I think that this will be an aspect of my job where I can say yes the OU study and discipline I’ve received from the OU has actually contributed to that. I observe things better, work into deeper and work on the whole picture rather than narrow. I think more logically and more ‘why did that happen, why did that happen’, there is more questioning, instead of just to accept things. I am much better at time management, I am much more organised now and planning things in advance. now I say, ‘you know what, I can do that in future’. I feel more confident and I am happier because I am doing something I have always wanted to be doing and something that interests me I think I can go confidently to speak what I learned. But even to a job that isn’t directly related to this subject area. I could talk about my experiences, my time management, team working, computer skills as I feel much more confident, I can say, ‘actually I have done this’. Which was one of the reasons I wanted to a degree.
- 4. Do grades matter? Howwell do your grades represent your progress? probably in the same way that many other people when they look at their own assignment results and exam results …. I feel that I am doing fairly well but I’d always like to improve myself to my results. I get quite upset when I get around 70s … because I am putting so much effort I want my grades to reflect it. They usually go up. But it is Marginal. 5 marks across all the TMAs that’s the variance, it just varies very slightly Even if it is 1-2 marks I say what did I do differently and I go back to tutor to see what did I do differently. What happened, what caused it? Well there are questions with the text books, exercises. So if I get correct answer, I know I am doing fine. When I say correct answer that’s not the end product that’s the whole answer check through it “I suppose you could say… the skills you learn, like group work, presenting and being able to talk to people… I would say the main way that you think about [achievement], it’s just the grade because… that’s what is going on your CV… and affect what job you get. … I’d say the skills you learn as well as becoming an all-rounded person are quite important as well”.
- 5. Do students makelearning gains, and what is the power of LA to predict learning gains? Using assessment results for estimating learning gains has number of advantages: Assessment data readily available Widely recognized as appropriate measure of learning Relatively free from self-reported biases Allows a direct comparison of research finding with the results of other studies Rogaten, J., Rienties, B, Whitelock, D. (2016). Assessing learning gains, TEA Conference, Tallinn, Estonia
- 6. Estimating learning trajectories Level1 Level 2 Level 3 Grade1 Student1 Grade3 Grade1Grade2Grade3Grade1Grade2Grade3Grade2 Student2 Student3 Course1 Course2 Grade1Grade2Grade3 Student4 Grade1Grade2Grade3 Student5 Course3
- 7. Study 1 resultsslides removed
- 8. How do STEMstudents compare to Social Science students Participants 5,791 Science students of whom 58.2% were females and 41.8% were males with average age of M = 29.8, SD = 9.6. 11,909 Social Science students of whom 72% were females and 28% were males with average age of M = 30.6, SD = 9.9 Measures Tutor Marked Assessments (TMA) Across 111 modules
- 9. Descriptive statistics: SocialScience Social Science Science
- 10. How do STEMstudents compare to Social Science students Social Science Science Variance in students’ initial achievements 6% 33% variance in students’ learning gains 19% 26% VPC module 4% 25% VPC student 56% 51% VPC TMAs 40% 22%
- 11. Effect of socio-demographic factors VariableSocial Science Beta Model1 Beta Model2 Beta Mode 3 Gender 0.64* Unknown -2.4* Other -6.68** Mixed -3.27** Asian -4.66** Black -7.99** HE qualification 0.29 Lower than A levels -3.11** No formal qualification -6.93** PG qualification 2.62** Variance explained in learning gains 0.1% 3% 3.1% Variable Science Beta Model 1 Beta Model 2 Beta Model 3 Gender 0.29 Unknown 1.76 Other -4.09 Mixed -0.59 Asian -7.31** Black -13.07** HE qualification 2.64** Lower than A levels -4.59** No formal qualification -9.48** PG qualification 8.19** Variance explained in learning gains 0.3% 2.2% 3%
- 12.
- 13. Social Science Science Alevels or equivalent HE Qualification Lower than A levels No formal qualificatio PG qualification
- 14. University 1 1,990 undergraduate students University 2 1,547 undergraduate students 20 degree programmes within each university DV – average grade yearly grade University 1 University 2 Year M SD M SD 1 60.65 7.62 63.75 12.66 2 61.31 6.81 65.64 12.74 3 63.32 6.63 64.12 14.02 Can we use this approach to look across different universities?
- 15. Study 3 resultsslides removed
- 16. Summary of findings Although both universities overall showed positive gains, substantial differences were present in variance at departmental level. Aggregate learning gains estimates can result in misleading estimates of students’ learning gains on a discipline or degree level. Multilevel modelling is a more accurate method in comparison with simple linear models when estimating students’ learning gains.
- 17. Possible implications Supportfor subject level TEF Guidance on where to focus interventions and resources Visualisation could promote data informed learning design decisions Questions still to answer: Does grade trajectory reflect students’ learning gains? Can we make a meaningful comparison between universities? What impact grade trajectory has on students? Do they need to know their own trajectory and how it compares to others?
- 18. Dr. Jekaterina Rogaten ProfBart Rienties https://twitter.com/LearningGains https://abclearninggains.com/