Professor Christina Hughes
Sunil Maher
The University of Warwick
LEGACY - Learning and Employability
Gain Assessment CommunitY
• HEFCE Pilot Programme
• 3 Years
– Oct 2015 – Sept 2018
• 4 Work Packages
– Measurement of
Learning Gain
– Employability
• Higher Education
• 18 Russell Group
Universities
• Birmingham, Bristol,
Durham, Cambridge, Exeter,
Imperial, Kings, Leeds,
Liverpool, Manchester,
Newcastle, Nottingham,
Queen Mary, Sheffield,
Southampton, UCL,
Warwick, York
• Identify (new) alternative Learning
Gain Assessment Methods
• Evidence Based Recommendations
• Measure ‘Distance Travelled’
• Methodological Comparability
– Longitudinal
– Cross-sectional Measures
Project Aims
LEARNING
GAIN
CORE
DIMENSIONS
QUALITATIVE
/QUANTITIVE
ASSESSMENT
IMPACTRESEARCH
KNOWLEDGE
SECTOR
DIALOGUE
Objectives
DEVELOP
• PROCESSES
• TOOLS
ENGAGE
• TARGET GROUPS
• SURVEYS, QUESTIONNAIRES & INTERVIEWS
COLLATE
• ON-LINE METHODS
ANALYSE
• DATA MERGING, ANALYSIS & FINDINGS
Approach
Year 1 2015/16 Year 2 2016/17 Year 3 2017/18
Timeline
Measurement of Learning Gain WP1
R2 Strengths WP2
Career Adaptabilities WP3
International Experience & Employability WP4
• Identify Core Dimensions of Learning Gain
– Across different disciplines
• Develop Measurement Instrument
– Reliable, validated and usable tool
• Produce New Scientific Knowledge
– Longitudinal Development in HE & Disciplines
• Test Explanatory Model
– Personal & Environmental Factors
Measurement of Learning Gain(1)
Work Package 1
Year 1
2015/16
Year 2
2016/17
Year 3
2017/18
Lit. Review
Analysis 2
Analysis 1
Findings
Data
Merge
Interviews
Student
Engagement
Tool Develop. &
Test
Measurement of Learning Gain(2)
Other outcome measures
Degree outcome
Graduation
Classification
Grades
Scores
GPAs
Employment
Potentially
Controls
Prior learning
A-levels
UCAS points
Reasoning ability
Self-concept
Academic
Global
Demographics
Gender
Age
Household
Parents
Employment
Study
Institution
Subject
Cognitive
component
Critical thinking
Analytical thinking
Cognitive abilities
Synthesising
Analysing
Evaluating
Problem solving
Meta-cognitive
component
Self-regulation
Life-long learning attitude
and motivation
Learning to learn
Need for cognition
(information seeking)
Affective
component
Attitudes towards own
discipline and
learning/studying
Motivation
Engagement
Professional and
academic interest
Socio-communicative
component
Levels of belonging in
social learning networks
Social embeddedness
Communication skills
Openness dimension epistemological stance; view of intelligence; open-mindedness
Research dimension curiosity; interest in research; interest in knowledge; attitude to sharing ideas
Moral dimension moral reasoning
Longitudinal measurement
Vermunt, Vignoles, Ilie; 2016,
R2 Strengths(1)
Work Package 2
R2
Instrument• Strength
Development
• Confidence
Building
Final
Year UG
• Career Role
Evaluation
• Personal
Strengths
• 1:1 Feedback
Metrics
• Self Awareness
• Outcome Focussed
Metrics
• Distance Travelled
Year 1
2015/16
Year 2
2016/17
R2 Strengths(2)
Analysis
1:1
Feedback
Interviews
Student
Engagement
Survey
Development
R2 Profile
Creation
• CAAI-UK – Career
Readiness Measures
• Identify Barriers
• Develop Supportive
Action Plan
• Career Support
Services Provision
Career AdaptAbilities(1)
Work Package 3
Confidence Control
CuriosityConcern
Year 1
2015/16
Year 2
2016/17
Year 3
2017/18
Career Adaptabilities(2)
Scoping
Analysis
Barriers
Findings
Data
Merge
Action Plan
Student
Engagement
Tool Develop. &
Test
• Builds on 4 UK Universitas 21 partners
• Self-perceived Employability Gains
– Study Abroad
– Work Abroad
– On Campus
• Impact of Employability Focussed Interventions
• Development of HEI Resource Bank
– Employability of Learning Gain
– International Experiences
International Experience &
Employability(1)
On
Campus
Study
Abroad
Work
Abroad
International Experience &
Employability(2)
Learning
Gain
Outcomes
METRICS
DISCIPLINARY
VARIANCES
METHODOLOGIES
MEASUREMENT
INSTRUMENTS
CONSIDERATIONS
FOR TEF
SOCIO-
ECONOMIC
VARIABLES
TOOLS
RESOURCE
BANK
HYPER
COMPLEXITIES ARTICULATE
LEARNING
TOOLS
Find out more...
University of Warwick:
Programme Professor Christina Hughes
Lead: Christina.Hughes@warwick.ac.uk
Project Sunil Maher
Manager: Sunil.Maher.1@warwick.ac.uk
Research Dr. Heike Behle
Fellow: Heike.Behle@warwick.ac.uk
WP Leaders:
Cambridge: Professor Jan Vermunt jdhv@cam.ac.uk
Professor Anna Vignoles av404@cam.ac.uk &
Dr. Sonia Ilie isi22@cam.ac.uk
Birmingham: Eluned Jones E.G.Jones@bham.ac.uk
Nottingham: Dr. Nalayini Thambar Nalayini.Thambar@nottingham.ac.uk
Warwick: Anne Wilson A.E.Wilson@warwick.ac.uk

Sunil Maher

  • 1.
    Professor Christina Hughes SunilMaher The University of Warwick
  • 2.
    LEGACY - Learningand Employability Gain Assessment CommunitY • HEFCE Pilot Programme • 3 Years – Oct 2015 – Sept 2018 • 4 Work Packages – Measurement of Learning Gain – Employability • Higher Education • 18 Russell Group Universities • Birmingham, Bristol, Durham, Cambridge, Exeter, Imperial, Kings, Leeds, Liverpool, Manchester, Newcastle, Nottingham, Queen Mary, Sheffield, Southampton, UCL, Warwick, York
  • 3.
    • Identify (new)alternative Learning Gain Assessment Methods • Evidence Based Recommendations • Measure ‘Distance Travelled’ • Methodological Comparability – Longitudinal – Cross-sectional Measures Project Aims
  • 4.
  • 5.
    DEVELOP • PROCESSES • TOOLS ENGAGE •TARGET GROUPS • SURVEYS, QUESTIONNAIRES & INTERVIEWS COLLATE • ON-LINE METHODS ANALYSE • DATA MERGING, ANALYSIS & FINDINGS Approach
  • 6.
    Year 1 2015/16Year 2 2016/17 Year 3 2017/18 Timeline Measurement of Learning Gain WP1 R2 Strengths WP2 Career Adaptabilities WP3 International Experience & Employability WP4
  • 7.
    • Identify CoreDimensions of Learning Gain – Across different disciplines • Develop Measurement Instrument – Reliable, validated and usable tool • Produce New Scientific Knowledge – Longitudinal Development in HE & Disciplines • Test Explanatory Model – Personal & Environmental Factors Measurement of Learning Gain(1) Work Package 1
  • 8.
    Year 1 2015/16 Year 2 2016/17 Year3 2017/18 Lit. Review Analysis 2 Analysis 1 Findings Data Merge Interviews Student Engagement Tool Develop. & Test Measurement of Learning Gain(2) Other outcome measures Degree outcome Graduation Classification Grades Scores GPAs Employment Potentially Controls Prior learning A-levels UCAS points Reasoning ability Self-concept Academic Global Demographics Gender Age Household Parents Employment Study Institution Subject Cognitive component Critical thinking Analytical thinking Cognitive abilities Synthesising Analysing Evaluating Problem solving Meta-cognitive component Self-regulation Life-long learning attitude and motivation Learning to learn Need for cognition (information seeking) Affective component Attitudes towards own discipline and learning/studying Motivation Engagement Professional and academic interest Socio-communicative component Levels of belonging in social learning networks Social embeddedness Communication skills Openness dimension epistemological stance; view of intelligence; open-mindedness Research dimension curiosity; interest in research; interest in knowledge; attitude to sharing ideas Moral dimension moral reasoning Longitudinal measurement Vermunt, Vignoles, Ilie; 2016,
  • 9.
    R2 Strengths(1) Work Package2 R2 Instrument• Strength Development • Confidence Building Final Year UG • Career Role Evaluation • Personal Strengths • 1:1 Feedback Metrics • Self Awareness • Outcome Focussed Metrics • Distance Travelled
  • 10.
    Year 1 2015/16 Year 2 2016/17 R2Strengths(2) Analysis 1:1 Feedback Interviews Student Engagement Survey Development R2 Profile Creation
  • 11.
    • CAAI-UK –Career Readiness Measures • Identify Barriers • Develop Supportive Action Plan • Career Support Services Provision Career AdaptAbilities(1) Work Package 3 Confidence Control CuriosityConcern
  • 12.
    Year 1 2015/16 Year 2 2016/17 Year3 2017/18 Career Adaptabilities(2) Scoping Analysis Barriers Findings Data Merge Action Plan Student Engagement Tool Develop. & Test
  • 13.
    • Builds on4 UK Universitas 21 partners • Self-perceived Employability Gains – Study Abroad – Work Abroad – On Campus • Impact of Employability Focussed Interventions • Development of HEI Resource Bank – Employability of Learning Gain – International Experiences International Experience & Employability(1)
  • 14.
  • 15.
  • 16.
    Find out more... Universityof Warwick: Programme Professor Christina Hughes Lead: Christina.Hughes@warwick.ac.uk Project Sunil Maher Manager: Sunil.Maher.1@warwick.ac.uk Research Dr. Heike Behle Fellow: Heike.Behle@warwick.ac.uk WP Leaders: Cambridge: Professor Jan Vermunt jdhv@cam.ac.uk Professor Anna Vignoles av404@cam.ac.uk & Dr. Sonia Ilie isi22@cam.ac.uk Birmingham: Eluned Jones E.G.Jones@bham.ac.uk Nottingham: Dr. Nalayini Thambar Nalayini.Thambar@nottingham.ac.uk Warwick: Anne Wilson A.E.Wilson@warwick.ac.uk