Image: world.edu

Student engagement in teaching,
learning and assessment

@mikehamlyn
Outline
Individual engagement
retention and success
a sense of belonging
Benefits of engagement
for the individual
and society
Engagement Outcomes
Module and award level
Public informatiion
League Tables
What can we do with the data
portfolio performance
how to get “smart”
how to improve outcomes from better engagement
The future
How will we carry on?
Is this engagement?
How do you know?

Does this reflect reality?

What will engaged students look like in the future?
Individual Engagement
• Student Retention and Success Project
• 1. At the heart of student retention and success is a strong sense of
belonging in HE for students. This is most effectively nurtured
through mainstream activities that all students participate in.
• 2. The academic sphere is the most important site for nurturing
belonging.
• 3. Specific interventions cannot be recommended over and above
each other. Rather the institution, department and programme
should all nurture a culture of belonging.
• 4. Student belonging is an outcome of: supportive peer relations;
meaningful interaction between staff and students; developing
knowledge, confidence and identity as successful HE learners; and
an HE experience which is relevant to interests and future goals.
Benefits of engagement
Engaged students get better outcomes

Celebrating individual success
Maximising individual rewards
Maximising contribution to society
Market and non-market benefits
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/25410
1/bis-13-1268-benefits-of-higher-education-participation-the-quadrants.pdf
How do we do this in a region of low aspiration?
Engagement Outcomes
Engagement Outcomes
• Students act
increasingly as engaged
consumers
– Access price
comparisons
– Access to performance
comparisons
– Demand better
outcomes

• University needs to
optimise
– Module and award level
outcomes
– Student survey results
– Public information
– League Tables
An engaged student reads….. An engaged student
contributes to…

Inputs

Outcomes

Spend per student
Staff student ratios
Entry standards
Research ratings
Cost of living
Spend on services
Faculty spend

Number of “good”
degrees
National Student
Survey results
Retention rates
Employability
Student Input to Information
•

National Student Survey
–
–
–
–

•
•
•

Measure of final year students
Questions on satisfaction with course
Satisfaction with teaching and learning
Satisfaction with assessment and feedback

Degree results
Retention
Employability

Used in 3 areas:
– League tables/KIS (external)
– Portfolio performance review (internal)
– Award annual monitoring (internal)
League Tables – a mirror.
Are we trying to improving
position or performance?
• Clearly, we can try to play the game of moving
our league table position
• What we really want to do is improve our
performance in each of the key areas to make
sure there is a sustainable and genuine change
Guardian criteria

Suggested Action

Entry standards

Review all current standard offers to pitch
ourselves properly against competitors

Student/staff ratio

Reviewed more thoroughly the data we submit
to HESA
Developing better models of SSR to identify
where investment is most needed

Spend per student

Reviewed more thoroughly the data we submit
to HESA
Reviewed classification of spend
Increased recent spend on libraries and IT will
have an impact
Guardian criteria

Action

Value added

Increased number of “good” degrees awarded.
Reviewing all level 6 modules with low pass rates
and average marks.
Identifying through portfolio review awards with
consistently poor progression and attainment
L&T conference on attainment
BME student performance

NSS teaching, assessment
and feedback and overall
satisfaction

Faculty action plans, and award level plans
Increased student engagement with survey
Seven principles of feedback
Online assessment and feedback project
Review through portfolio performance tool

employment

Staffordshire Graduate – improving our students’
chances of success
On-campus graduate internships
Student Input to Information

Used in 3 areas:
– League tables/KIS (external)
– Portfolio performance review (internal)
– Award annual monitoring (internal)
Portfolio performance review
• Measures:

• Uses

– Market
– Academic outcomes inc
good degrees
– Student satisfaction
– Employability
SNC/AB
B 2nd
stage
enrol
per
offer
0.5

0.22
0.12
0.42

total
apps/2
nd retenti retenti
% good % good
stage on
on
progres progres degree degree
enrol 2011 2012 s 2011 s 2012 s 2011 s 2012
5

9.50
16.00
3.40

90.0

90.0

90.0

90.0

56.0

100.0
86.4
77.8

100.0
80.0
100.0

100.0
72.7
77.8

66.7
25.0
50.0

50.0
66.7
25.0

NSS
2012
total all total all total all total all
NSS
assess
levels levels levels levels dlhe
dlhe
2012 and
NSS
enrol enrol enrol enrol 2010- 2011- teachin feedba 2012
2010 2011 2012 2013 11
12
g
ck
overall

60.0

100.0
93.3
100.0

– Comparison against
targets
– Comparison between
awards
– Annual monitoring
– Portfolio decisions
NSS
2013
score 2 assess
and
NSS
score 1 market quality led
NSS 2013 feedba 2013 led (enter
teaching ck
overall target below)

80

10
27
9

10
28
7

8
21
9

9
14
9

85

86

70

85

86

72

85

80.00

100.0
50.0
0.0

80.0
0.0
50.0

78.0
97.0
78.0

65.0
94.0
65.0

75.0
100.0
75.0

89.0
83.0
89.0

74.0
85.0
74.0

90.0
87.0
90.0

80.87
60.71
94.68

100.69
81.83
79.92
Student Input to Information

Used in 3 areas:
– League tables/KIS (external)
– Portfolio performance review (internal)
– Award annual monitoring (internal)
Annual Monitoring
• Key part of quality process
– Reflection on award performance
– Reflection on student engagement and outcomes
•
•
•
•
•

Statistics – progression, achievement, retention
Response to external examiner comments
Reflection on employer input
Staffordshire Graduate
Reports are made available to students, to demonstrate our
engagement

• Critically important for partner input, and to
recognise joint responsibility for improvements
How it fits together
Student surveys

Student satisfaction

Action Plans

Student
engagement and
outcomes

League tables

Annual Monitoring

Course improvements

reputation

Market demand
Improved engagement leads to
improved outcomes
• improved outcomes for individual students
– Grades and classifications
– Employability –getting into the right jobs
– Satisfaction
– Satisfying aspirations

• improved overall outcomes
– Institutional success and reputation
– Changing the aspirations of our communities
How institutional data and
analysis can help?
• Identifying performance metrics at module
level
• Identifying performance metrics for awards
• Developing a culture where this analysis
becomes embedded
– Being prepared to deal with the issue!
TLA practice
Relevant TL and assessment practice

Responsive curricula – employer and
technology led

Management Intervention
Questioning low average marks, poor
attainment rates
Managing the performance of the portfolio

Increased student engagement in course
design and monitoring

Increased emphasis on EEE (staff and students)

Staff engaged in reflective practice

Improving quality of teaching

Continuous staff development

How could we improve engagement
and outcomes for institution?

Uses of technology
Electronic assessment and feedback
Online communities

The future…..
How we teach in 2013?

Student portal

How to learn in 2020?

Mobile technologies
Improving individual engagement
Using “big data”?
Learning Analytics –
can work if all systems joined up
A way to encourage or check on student engagement?
Every log in
Every use of VLE – pages read, scores on formative tests
Use of library – frequency, downloads, loans
Attendance at class or online
A way to audit teaching as well…….
Or is this another example of technological solutionism?

LEADERSHIP AND LEARNING ANALYTICS, http://www.educause.edu/library/resources/leadership-and-learning-analytics
“To Save Everything, Click Here: Technology, Solutionism, and the Urge to Fix Problems that Don’t Exist” Evgeney Morozov, pb Allen Lane
2103, ISBN-10: 1846145481”
How will we engage students in learning when:
Half of what they learn in first year is out of date by the time they graduate?
They will have maybe 10 different jobs in their career?
The jobs they will do don’t even exist yet using technology that isn't invented yet,
to solve problems we don’t know are problems yet.
100 Billion queries a month on Google – who did we ask before?
90% of the data in the world was created in the last 2 years
There are more students in the top 5% in China than all the students in the UK
http://vimeo.com/58839986

@mikehamlyn

blogs.staffs.ac.uk/mgh1/

International conference

  • 1.
    Image: world.edu Student engagementin teaching, learning and assessment @mikehamlyn
  • 2.
    Outline Individual engagement retention andsuccess a sense of belonging Benefits of engagement for the individual and society Engagement Outcomes Module and award level Public informatiion League Tables What can we do with the data portfolio performance how to get “smart” how to improve outcomes from better engagement The future How will we carry on?
  • 3.
    Is this engagement? Howdo you know? Does this reflect reality? What will engaged students look like in the future?
  • 4.
    Individual Engagement • StudentRetention and Success Project • 1. At the heart of student retention and success is a strong sense of belonging in HE for students. This is most effectively nurtured through mainstream activities that all students participate in. • 2. The academic sphere is the most important site for nurturing belonging. • 3. Specific interventions cannot be recommended over and above each other. Rather the institution, department and programme should all nurture a culture of belonging. • 4. Student belonging is an outcome of: supportive peer relations; meaningful interaction between staff and students; developing knowledge, confidence and identity as successful HE learners; and an HE experience which is relevant to interests and future goals.
  • 6.
    Benefits of engagement Engagedstudents get better outcomes Celebrating individual success Maximising individual rewards Maximising contribution to society Market and non-market benefits
  • 7.
  • 8.
    How do wedo this in a region of low aspiration?
  • 9.
  • 10.
    Engagement Outcomes • Studentsact increasingly as engaged consumers – Access price comparisons – Access to performance comparisons – Demand better outcomes • University needs to optimise – Module and award level outcomes – Student survey results – Public information – League Tables
  • 11.
    An engaged studentreads….. An engaged student contributes to… Inputs Outcomes Spend per student Staff student ratios Entry standards Research ratings Cost of living Spend on services Faculty spend Number of “good” degrees National Student Survey results Retention rates Employability
  • 12.
    Student Input toInformation • National Student Survey – – – – • • • Measure of final year students Questions on satisfaction with course Satisfaction with teaching and learning Satisfaction with assessment and feedback Degree results Retention Employability Used in 3 areas: – League tables/KIS (external) – Portfolio performance review (internal) – Award annual monitoring (internal)
  • 13.
  • 22.
    Are we tryingto improving position or performance? • Clearly, we can try to play the game of moving our league table position • What we really want to do is improve our performance in each of the key areas to make sure there is a sustainable and genuine change
  • 23.
    Guardian criteria Suggested Action Entrystandards Review all current standard offers to pitch ourselves properly against competitors Student/staff ratio Reviewed more thoroughly the data we submit to HESA Developing better models of SSR to identify where investment is most needed Spend per student Reviewed more thoroughly the data we submit to HESA Reviewed classification of spend Increased recent spend on libraries and IT will have an impact
  • 24.
    Guardian criteria Action Value added Increasednumber of “good” degrees awarded. Reviewing all level 6 modules with low pass rates and average marks. Identifying through portfolio review awards with consistently poor progression and attainment L&T conference on attainment BME student performance NSS teaching, assessment and feedback and overall satisfaction Faculty action plans, and award level plans Increased student engagement with survey Seven principles of feedback Online assessment and feedback project Review through portfolio performance tool employment Staffordshire Graduate – improving our students’ chances of success On-campus graduate internships
  • 25.
    Student Input toInformation Used in 3 areas: – League tables/KIS (external) – Portfolio performance review (internal) – Award annual monitoring (internal)
  • 26.
    Portfolio performance review •Measures: • Uses – Market – Academic outcomes inc good degrees – Student satisfaction – Employability SNC/AB B 2nd stage enrol per offer 0.5 0.22 0.12 0.42 total apps/2 nd retenti retenti % good % good stage on on progres progres degree degree enrol 2011 2012 s 2011 s 2012 s 2011 s 2012 5 9.50 16.00 3.40 90.0 90.0 90.0 90.0 56.0 100.0 86.4 77.8 100.0 80.0 100.0 100.0 72.7 77.8 66.7 25.0 50.0 50.0 66.7 25.0 NSS 2012 total all total all total all total all NSS assess levels levels levels levels dlhe dlhe 2012 and NSS enrol enrol enrol enrol 2010- 2011- teachin feedba 2012 2010 2011 2012 2013 11 12 g ck overall 60.0 100.0 93.3 100.0 – Comparison against targets – Comparison between awards – Annual monitoring – Portfolio decisions NSS 2013 score 2 assess and NSS score 1 market quality led NSS 2013 feedba 2013 led (enter teaching ck overall target below) 80 10 27 9 10 28 7 8 21 9 9 14 9 85 86 70 85 86 72 85 80.00 100.0 50.0 0.0 80.0 0.0 50.0 78.0 97.0 78.0 65.0 94.0 65.0 75.0 100.0 75.0 89.0 83.0 89.0 74.0 85.0 74.0 90.0 87.0 90.0 80.87 60.71 94.68 100.69 81.83 79.92
  • 27.
    Student Input toInformation Used in 3 areas: – League tables/KIS (external) – Portfolio performance review (internal) – Award annual monitoring (internal)
  • 28.
    Annual Monitoring • Keypart of quality process – Reflection on award performance – Reflection on student engagement and outcomes • • • • • Statistics – progression, achievement, retention Response to external examiner comments Reflection on employer input Staffordshire Graduate Reports are made available to students, to demonstrate our engagement • Critically important for partner input, and to recognise joint responsibility for improvements
  • 29.
    How it fitstogether Student surveys Student satisfaction Action Plans Student engagement and outcomes League tables Annual Monitoring Course improvements reputation Market demand
  • 30.
    Improved engagement leadsto improved outcomes • improved outcomes for individual students – Grades and classifications – Employability –getting into the right jobs – Satisfaction – Satisfying aspirations • improved overall outcomes – Institutional success and reputation – Changing the aspirations of our communities
  • 31.
    How institutional dataand analysis can help? • Identifying performance metrics at module level • Identifying performance metrics for awards • Developing a culture where this analysis becomes embedded – Being prepared to deal with the issue!
  • 32.
    TLA practice Relevant TLand assessment practice Responsive curricula – employer and technology led Management Intervention Questioning low average marks, poor attainment rates Managing the performance of the portfolio Increased student engagement in course design and monitoring Increased emphasis on EEE (staff and students) Staff engaged in reflective practice Improving quality of teaching Continuous staff development How could we improve engagement and outcomes for institution? Uses of technology Electronic assessment and feedback Online communities The future….. How we teach in 2013? Student portal How to learn in 2020? Mobile technologies
  • 33.
    Improving individual engagement Using“big data”? Learning Analytics – can work if all systems joined up A way to encourage or check on student engagement? Every log in Every use of VLE – pages read, scores on formative tests Use of library – frequency, downloads, loans Attendance at class or online A way to audit teaching as well……. Or is this another example of technological solutionism? LEADERSHIP AND LEARNING ANALYTICS, http://www.educause.edu/library/resources/leadership-and-learning-analytics “To Save Everything, Click Here: Technology, Solutionism, and the Urge to Fix Problems that Don’t Exist” Evgeney Morozov, pb Allen Lane 2103, ISBN-10: 1846145481”
  • 34.
    How will weengage students in learning when: Half of what they learn in first year is out of date by the time they graduate? They will have maybe 10 different jobs in their career? The jobs they will do don’t even exist yet using technology that isn't invented yet, to solve problems we don’t know are problems yet. 100 Billion queries a month on Google – who did we ask before? 90% of the data in the world was created in the last 2 years There are more students in the top 5% in China than all the students in the UK
  • 35.