Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How do students understand and interpret learning gains? Perspectives from in-depth interviews of UK distance learners.

273 views

Published on

Universities are increasingly looking at Learning Gains as a means for the effective measurement of student
progress and the contribution they make to their students’ learning and development. However, there remains
much to understand about the validity and reliability of these measures used. This paper seeks to probe
the relationship between how students understand and interpret the learning gains they experience and the
proxy measures of learning gain such as assessment marks universities use. The findings are based on an
analysis of nineteen semi-structured interviews of UK distance learners. The paper will present key findings
and discuss their significance in respect to how to reconcile students’ own experience of gain and proxy
measures of gain and the assumptions on which learning gain measures are predicated.

Simon Cross, Jekaterina Rogaten, Duygu Bektik, Denise Whitelock, Bart Rienties, The Open University, UK

Published in: Education
  • Be the first to comment

  • Be the first to like this

How do students understand and interpret learning gains? Perspectives from in-depth interviews of UK distance learners.

  1. 1. Simon Cross, Jekaterina Rogaten, Duygu Bektik, Denise Whitelock, Bart Rienties Institute of Educational Technology, The Open University SRHE Conference / 7 December 2017 How do students understand and interpret learning gains? Perspectives from in-depth interviews of UK distance learners
  2. 2. PRESENTATION OVERVIEW Using data from interviews undertaken in May 2017 at the Open University UK, this presentation will probe the relationship between how students understand and interpret the learning gains they experience and the meaning and significance they give to metrics in use for measuring learning gain. ABC Learning Gains Project
  3. 3. PRESENTATION OVERVIEW Using data from interviews undertaken in May 2017 at the Open University UK, this presentation will probe the relationship between how students understand and interpret the learning gains they experience and the meaning and significance they give to metrics in use for measuring learning gain. ABC Learning Gains Project
  4. 4. 07/12/2017 Learning Gains As TEF develops we will incorporate new common metrics on engagement … and learning gain, once they are sufficiently robust. Government Green Paper Policymakers are now called on to provide evidence to inform questions such as: How do students’ knowledge, skills and work-readiness change and improve through their experience of higher education?... Traditional measures for evaluating student performance remain essential …but they do not provide all of the evidence needed to address these questions. And this evidence is more important than ever… HEFCE Average gain score needs to be interpreted with caution and there remain many outstanding issues that need investigating. Pascarella et al. 2011.
  5. 5. 07/12/2017 1 How do students understand and interpret learning gains they experience? 2 How do students understand and interpret measures of learning gain currently in use? 3 To what extent can these two be reconciled? Research questions How do students understand and interpret learning gains?
  6. 6. 07/12/2017 Semi-structured interviews with nineteen part-time distance learners resident in the UK lasting 30-60 minutes • Student perception of gain and progress • Cognitive, behavioural and affective change • Relationship between grades and progress • Study expectations • Work relevancy Methodology and participants How do students understand and interpret learning gains?
  7. 7. 07/12/2017 First stage Interim analysis based on review of interviewer notes, second listen to interview and coding of interview transcripts. Methodology and participants How do students understand and interpret learning gains? Student 8 Female High Lowest STEM Student 19 Female Low Medium Arts/Soc.Sci Student 5 Female Low Highest Edn/Lang Student 10 Male High Medium Arts/Soc.Sci Student 12 Female Low Lowest STEM Student 18 Female High Highest STEM Attainment grouping as measured by assessment marks Progress grouping as measured by assessment marks Gender Faculty Sources of quotations used in presentation
  8. 8. 07/12/2017 Emerging themes How do students understand and interpret learning gains? OWNERSHIP OF LEARNING RELATIVE VALUE OF GRADES WORK-LIFE • LEVEL OF STUDYLEVEL OF STUDY
  9. 9. 07/12/2017 Level of Study ‘Student 8’ How do students understand and interpret learning gains? [Initially] it was very much like study for study purposes [but] about a year and half into it, my mind-set changed, it was like I enjoy what I’m doing and it’s giving me something tangible. This year was the first year when … it reflected back on my day job. The outcome was a decision to increase study intensity… I [feel] that I have the knowledge, it’s the time that I was missing… I don’t think for me it’s grade itself is… it’s actually the knowledge. [Getting the grade] is what’s needed this year for me not to give up so that I had an opportunity to do the exam and face the next step. Looking back Current module I have decided that I will care about my grades next year and try to get the highest scores I physically can just to see [how well I can do] if I don’t have that additional 60 credits. Looking ahead
  10. 10. 07/12/2017 Relative Value of Grade Explaining lower grades How do students understand and interpret learning gains? There were things in the first two modules that I already knew [but] when I come to this year [then] I would say probably 95% of what I’ve learned … I’ve never heard of. So, I think this has been a real turning point for me.  Reassessment and reprioritisation  Focus on elements relevant to own interests  Expended the oxygen of prior learning  Expectation that modules will become tougher and grades harder to achieve  Module does not directly build on last  Comparatively less effective learning design  Greater study workload [This year] was my turning point… Your TMA (continuous tutor marked assessment) is not everything. It’s not, it’s supposed to be what you actually physically know yourself inside… I think that’s really important.
  11. 11. 07/12/2017 Owning the process How do students understand and interpret learning gains? I sort of challenged [the TMA score], not ‘challenged’ it really, but asked more questions. [I’ve learned] don’t be afraid to.. I thought I possibly would have done better, you know maybe 5 or 10 marks here and there so I have challenged. [The tutor response] helped me… I like to read it, dissect it [and] this year the feedback I’ve got has included things that I possibly might have missed. Despite the drop in marks, I still think my understanding has deepened and now … I feel more engaged in [my learning] actually, not just ticking off what I need to do Initiating discussion for feed-forward Counter-intuitive deeper engagement
  12. 12. 07/12/2017 Work-Life Reference (Points) How do students understand and interpret learning gains? The way I think, the way that I possibly act at times, my life feels different now … I can talk confidently to people. It’s just when you learn something you [then] become aware of, for instance, either news, work itself, everyday life; which again kind of changes your perspective and then it allows you to properly build your own confidence because you understand things Becoming confident Becoming aware I notice that my approach to things outside of the academic context is different … I judge [gain] on the assignment side and the out-of-academic context side …as well.
  13. 13. 07/12/2017How do students understand and interpret learning gains? OWNERSHIP OF LEARNING RELATIVE VALUE OF GRADES WORK-LIFE • LEVEL OF STUDYLEVEL OF STUDY Emerging relationships
  14. 14. 07/12/2017 Reconciling perceptions of learning gain? • Many factors impact on the quality of assessment that students choose to submit. • Students’ perception of ‘learning gain’ change as they progress. Level 2 seems to be a key turning point. • Students and assessors can interpret and give meaning to the same grade in different ways • Whilst assessors set the assessment criteria, it is the student who decides its relevancy to their learning. • Becoming an independent learner means taking ownership of the meaning and use-for-learning of the assessment process • Students (co-)determine the significance of a grade through a process of contextualisation across study, social and work lives. How do students understand and interpret learning gains?
  15. 15. 07/12/2017 Concluding remarks Significance and importance of qualitative research Short and medium term goals An A+B+C Approach How do students understand and interpret learning gains? Email: simon.j.cross@open.ac.uk

×