Being known and knowing stuff:
Linking feedback and RIT
@solentlearning
@tansyjtweets
Tansy Jessop, SLTI
26 June 2017
Today’s session
• About TESTA
• Your experience of
feedback
• Why it doesn’t work
• Ways to make it
work
• Principles of
effective feedback
• RIT
• what it is
• why we need it
• It’s already happening!
• how we can do it more
A modular programme
The Methodology
Programme
Team
Meeting
Assessment
Experience
Questionnaire
(AEQ)
TESTA
Programme
Audit
Student
Focus Groups
The million $ question: does TESTA
work?
60%
65%
70%
75%
80%
85%
90%
95%
Q5 Q6 Q7 Q8 Q9 OS
AVERAGENSSSCORES
COMPARISON OF 73 PROGS IN 14 UNIVERSITIES WITH SECTOR SCORES
NSS 2015 SCORES TESTA SCORES
It was heavy, tons of marking for
the tutor. It was such hard work.
It was criminal.
Media Course Leader
I’m really bad at reading
feedback. I’ll look at the mark
and then be like ‘well stuff it, I
can’t do anything about it’
Student, TESTA focus group
TESTA reveals knotty problems
Think back and make jottings on….
• ….feedback you received which had a
damaging effect on you.
• ….feedback which spurred you on to great
heights.
• Two minute chat with a partner, sharing some
of your experience.
So why doesn’t feedback work?
1) Modular structures impede feedback
2) Students don’t feel known and understood
3) Lecturers evaluate too much; students too little
4) Feedback is not growth-oriented
5) Grades and competition matter more than progress
1. What students say about modular
impediments
The feedback is generally focused on the module.
It’s difficult because your assignments are so
detached from the next one you do for that
subject. They don’t relate to each other.
Because it’s at the end of the module, it doesn’t
feed into our future work.
Does IKEA 101 work for feedback?
2. I don’t feel known
Because they have to mark so many that our essay
becomes lost in the sea that they have to mark.
It was like ‘Who’s Holly?’ It’s that relationship
where you’re just a student.
Here they say ‘Oh yes, I don’t know who you are.
Got too many to remember, don’t really care, I’ll
mark you on your assignment’.
…and being assessed is harrowing
It’s always the negatives you remember… we
hardly ever pick out the really positive points
because once you’ve seen the negative, the
negatives can outweigh the positives.
I feel physically sick handing in an assignment. I
can’t sleep for days before because I panic that
it’s not right and it’s so pathetic.
with an emotional impact
..so some lecturers just pacify us
They just pacify really. I went for help and they
just told me what I wanted to hear, not what I
needed to know.
Its very positive like nobody ever says ‘no you’ve
done that completely wrong’. It's always 'You've
done that very well‘. Well why have a got a low
grade then? It doesn’t really help you from
there.
…with rare exceptions
I’m baffled.
Students love my
feedback but
they are a voice
in the
wilderness…
Pause
1. What does the feedback
you give look like?
2. What do you think of
these examples of
conversational feedback?
3. How honest is your
feedback? What keeps you
from being honest?
3. Lecturers evaluate too much,
students too little
I: Do you have any peer feedback?
R2: No. Because they'll just copy it.
1: Do you have any feedback from peers?
S: We just have questions at the end.
4. Feedback is not growth-oriented
It told you some of the problems but it doesn’t tell
you how you can manage to fix that. It was,
“Well, this is the problem.” I was like, “How do I fix
it?” They said, “Well, some people are just not
good at writing.”
(TESTA Focus group data)
Here are some ways
in which you can
improve…
Some people are
just not good at
writing…
5. Grades and competition matter more
than progress
1. Breaking down silos
• Sequencing tasks and varieties across modules
• Developing coversheets in cycles of reflection
across modules
• Diagnostic feedback on one task addressed in
next one
• Giving feedback to feedforward to next task
• Quick exam feedback, or two-stage exams
https://www.youtube.com/watch?v=ZVFwQzlVFy0
2. Getting students involved
• Students asking for feedback on areas of an
assessment at hand in
• Students peer reviewing posters, presentations,
lab reports.
• Self assessment
• Students using rubrics to mark
• Students doing synthesis tasks with feedback
3. Being known in feedback
• Peer processes
• Make time to explore the purpose of feedback
• Share your experiences – be vulnerable
• Use questioning rather than ‘telling’
• Audio and screencast feedback
• Personalise, surprise, challenge – have a
conversation
Principles of feedback
1. Feedback needs to be prompt, detailed, specific,
developmental (Gibbs 2004)
2. Dialogue not monologue (Nicol 2010)
3. Needs to close the loop not “dangle the data
better? (Sadler 1989)
4. Sustainable feedback (Boud 2000; Carless 2011)
RIT the heartbeat of higher education?
What is RIT?
Go to www.menti.com and use the code 14 22 31
Type in three phrases capturing your view of RIT
What is RIT?
Four perspectives of RIT
RIT fusion plus:
the virtuous teaching-research circle
Research informs
teaching
Action research
Scholarship of
teaching
Evidence-informed
practice
Teaching informs
research
Preparation drives
research
Student fire new
questions and angles
Research
assumptions refined
Example 1: ‘Research-tutored’
https://vimeo.com/214664726
Example 2: Research-based
https://vimeo.com/214664656
2 x Solent students
Posters in Parliament
17 x students at BCUR
Watch this space:
Sheffield April 2017
Why it matters: extraordinary impact
claims
• Self-confidence
• Independence in learning
• Increasing epistemological sophistication
• Entry into discipline research cultures
• Collegial relations with academics
• Improved grades
• Enhanced metacognition
• Increased engagement
• Employability skills
(Levy and Petrulis, 2012, p.88)
How can we do more RIT
Go to www.menti.com and use the code 66 80 08
How can we do more RIT?
References
Boud, D. and Molloy, E (2013) Rethinking models of feedback for learning: the challenge of design Assessment & Evaluation in Higher
Education, 38:6, 698-712
Boud, D. and Molloy, E (2013) Feedback in Higher and Professional Education. Understanding it and doing it better. Abingdon. Routledge.
Dweck, C. (2006) Mindset: How can fulfil your potential. New York. Random House.
Gibbs, G. & Simpson, C. (2004) Conditions under which assessment supports students' learning. Learning and Teaching in Higher Education.
1(1): 3-31.
Hattie, J. (2007) The Power of Feedback. Review of Educational Research. 77(1) 81-112.
Hughes, G. (2014) Ipsative Assessment. Basingstoke. Palgrave MacMillan.
Healey, M. and A. Jenkins, 2009. Developing undergraduate research and inquiry. York: Higher Education Academy.
Jessop, T. (2017) Inspiring transformation through TESTA’s programme approach. Scaling up Assessment for Learning in Higher Education..
Jessop, T. And Tomas, C. (2016) The implications of programme assessment patterns for student learning. Assessment and Evaluation in HE.
Jessop, T. and Maleckar, B. (2014). The Influence of disciplinary assessment patterns on student learning: a comparative study. Studies in Higher
Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large-scale study of students’ learning in response
to different assessment patterns. Assessment and Evaluation in Higher Education. 39(1) 73-88.
Levy, P. and Petrulis, R. (2012) How do first year students experience inquiry and research, and what are the implications for the practice of
inquiry-based learning? Studies in Higher Education, 37:1, 85-101.
Nicol, D. (2010) From monologue to dialogue: improving written feedback processes in mass higher education. Assessment & Evaluation in
Higher Education, 35: 5, 501 – 517.
Nicol, D. and McFarlane-Dick D. (2006) Formative Assessment and Self-Regulated Learning: A Model and Seven Principles of Good Feedback
Practice. Studies in Higher Education. 31(2): 199-218.
Sadler, D.R. (1989) Formative assessment and the design of instructional systems. Instructional Science, 18, 119-144.
TESTA (2009-16) Transforming the Experience of Students through Assessment (www.testa.ac.uk)

Being known and knowing stuff: linking feedback and RIT

  • 1.
    Being known andknowing stuff: Linking feedback and RIT @solentlearning @tansyjtweets Tansy Jessop, SLTI 26 June 2017
  • 2.
    Today’s session • AboutTESTA • Your experience of feedback • Why it doesn’t work • Ways to make it work • Principles of effective feedback • RIT • what it is • why we need it • It’s already happening! • how we can do it more
  • 4.
  • 5.
  • 6.
    The million $question: does TESTA work? 60% 65% 70% 75% 80% 85% 90% 95% Q5 Q6 Q7 Q8 Q9 OS AVERAGENSSSCORES COMPARISON OF 73 PROGS IN 14 UNIVERSITIES WITH SECTOR SCORES NSS 2015 SCORES TESTA SCORES
  • 7.
    It was heavy,tons of marking for the tutor. It was such hard work. It was criminal. Media Course Leader I’m really bad at reading feedback. I’ll look at the mark and then be like ‘well stuff it, I can’t do anything about it’ Student, TESTA focus group TESTA reveals knotty problems
  • 8.
    Think back andmake jottings on…. • ….feedback you received which had a damaging effect on you. • ….feedback which spurred you on to great heights. • Two minute chat with a partner, sharing some of your experience.
  • 9.
    So why doesn’tfeedback work? 1) Modular structures impede feedback 2) Students don’t feel known and understood 3) Lecturers evaluate too much; students too little 4) Feedback is not growth-oriented 5) Grades and competition matter more than progress
  • 10.
    1. What studentssay about modular impediments The feedback is generally focused on the module. It’s difficult because your assignments are so detached from the next one you do for that subject. They don’t relate to each other. Because it’s at the end of the module, it doesn’t feed into our future work.
  • 11.
    Does IKEA 101work for feedback?
  • 12.
    2. I don’tfeel known Because they have to mark so many that our essay becomes lost in the sea that they have to mark. It was like ‘Who’s Holly?’ It’s that relationship where you’re just a student. Here they say ‘Oh yes, I don’t know who you are. Got too many to remember, don’t really care, I’ll mark you on your assignment’.
  • 13.
    …and being assessedis harrowing It’s always the negatives you remember… we hardly ever pick out the really positive points because once you’ve seen the negative, the negatives can outweigh the positives. I feel physically sick handing in an assignment. I can’t sleep for days before because I panic that it’s not right and it’s so pathetic.
  • 14.
  • 15.
    ..so some lecturersjust pacify us They just pacify really. I went for help and they just told me what I wanted to hear, not what I needed to know. Its very positive like nobody ever says ‘no you’ve done that completely wrong’. It's always 'You've done that very well‘. Well why have a got a low grade then? It doesn’t really help you from there.
  • 16.
    …with rare exceptions I’mbaffled. Students love my feedback but they are a voice in the wilderness…
  • 17.
    Pause 1. What doesthe feedback you give look like? 2. What do you think of these examples of conversational feedback? 3. How honest is your feedback? What keeps you from being honest?
  • 18.
    3. Lecturers evaluatetoo much, students too little I: Do you have any peer feedback? R2: No. Because they'll just copy it. 1: Do you have any feedback from peers? S: We just have questions at the end.
  • 19.
    4. Feedback isnot growth-oriented
  • 20.
    It told yousome of the problems but it doesn’t tell you how you can manage to fix that. It was, “Well, this is the problem.” I was like, “How do I fix it?” They said, “Well, some people are just not good at writing.” (TESTA Focus group data)
  • 21.
    Here are someways in which you can improve… Some people are just not good at writing…
  • 22.
    5. Grades andcompetition matter more than progress
  • 23.
    1. Breaking downsilos • Sequencing tasks and varieties across modules • Developing coversheets in cycles of reflection across modules • Diagnostic feedback on one task addressed in next one • Giving feedback to feedforward to next task • Quick exam feedback, or two-stage exams https://www.youtube.com/watch?v=ZVFwQzlVFy0
  • 24.
    2. Getting studentsinvolved • Students asking for feedback on areas of an assessment at hand in • Students peer reviewing posters, presentations, lab reports. • Self assessment • Students using rubrics to mark • Students doing synthesis tasks with feedback
  • 25.
    3. Being knownin feedback • Peer processes • Make time to explore the purpose of feedback • Share your experiences – be vulnerable • Use questioning rather than ‘telling’ • Audio and screencast feedback • Personalise, surprise, challenge – have a conversation
  • 26.
    Principles of feedback 1.Feedback needs to be prompt, detailed, specific, developmental (Gibbs 2004) 2. Dialogue not monologue (Nicol 2010) 3. Needs to close the loop not “dangle the data better? (Sadler 1989) 4. Sustainable feedback (Boud 2000; Carless 2011)
  • 27.
    RIT the heartbeatof higher education?
  • 28.
    What is RIT? Goto www.menti.com and use the code 14 22 31 Type in three phrases capturing your view of RIT
  • 29.
  • 30.
  • 31.
    RIT fusion plus: thevirtuous teaching-research circle Research informs teaching Action research Scholarship of teaching Evidence-informed practice Teaching informs research Preparation drives research Student fire new questions and angles Research assumptions refined
  • 32.
  • 33.
  • 34.
    2 x Solentstudents Posters in Parliament 17 x students at BCUR Watch this space: Sheffield April 2017
  • 35.
    Why it matters:extraordinary impact claims • Self-confidence • Independence in learning • Increasing epistemological sophistication • Entry into discipline research cultures • Collegial relations with academics • Improved grades • Enhanced metacognition • Increased engagement • Employability skills (Levy and Petrulis, 2012, p.88)
  • 37.
    How can wedo more RIT Go to www.menti.com and use the code 66 80 08
  • 38.
    How can wedo more RIT?
  • 39.
    References Boud, D. andMolloy, E (2013) Rethinking models of feedback for learning: the challenge of design Assessment & Evaluation in Higher Education, 38:6, 698-712 Boud, D. and Molloy, E (2013) Feedback in Higher and Professional Education. Understanding it and doing it better. Abingdon. Routledge. Dweck, C. (2006) Mindset: How can fulfil your potential. New York. Random House. Gibbs, G. & Simpson, C. (2004) Conditions under which assessment supports students' learning. Learning and Teaching in Higher Education. 1(1): 3-31. Hattie, J. (2007) The Power of Feedback. Review of Educational Research. 77(1) 81-112. Hughes, G. (2014) Ipsative Assessment. Basingstoke. Palgrave MacMillan. Healey, M. and A. Jenkins, 2009. Developing undergraduate research and inquiry. York: Higher Education Academy. Jessop, T. (2017) Inspiring transformation through TESTA’s programme approach. Scaling up Assessment for Learning in Higher Education.. Jessop, T. And Tomas, C. (2016) The implications of programme assessment patterns for student learning. Assessment and Evaluation in HE. Jessop, T. and Maleckar, B. (2014). The Influence of disciplinary assessment patterns on student learning: a comparative study. Studies in Higher Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large-scale study of students’ learning in response to different assessment patterns. Assessment and Evaluation in Higher Education. 39(1) 73-88. Levy, P. and Petrulis, R. (2012) How do first year students experience inquiry and research, and what are the implications for the practice of inquiry-based learning? Studies in Higher Education, 37:1, 85-101. Nicol, D. (2010) From monologue to dialogue: improving written feedback processes in mass higher education. Assessment & Evaluation in Higher Education, 35: 5, 501 – 517. Nicol, D. and McFarlane-Dick D. (2006) Formative Assessment and Self-Regulated Learning: A Model and Seven Principles of Good Feedback Practice. Studies in Higher Education. 31(2): 199-218. Sadler, D.R. (1989) Formative assessment and the design of instructional systems. Instructional Science, 18, 119-144. TESTA (2009-16) Transforming the Experience of Students through Assessment (www.testa.ac.uk)

Editor's Notes

  • #2 Tansy
  • #5 We are all looking at our own modules and it doesn’t add up. Very little knowledge of the whole beast. TESTA helps people see the whole picture, particularly from a student perspective, and helps academics design in more connections, less measurement, and deeper learning. Breaks down walls and silos. Team approach.
  • #8 Feedback: all that effort, but what is the effect? Margaret Price. Feedback more important to teachers than students… But lots of projects and programmes do….
  • #11 disposable
  • #13 Impoverished dialogue
  • #18 Orienting students to the purpose of feedback, winners and losers, ipsative assessment, questioning rather than telling, self-evaluation, into the secret garden
  • #28 Is my teaching still alive? Are my students breathing? Curiosity, new knowledge, choice, authentic learning. If it is then why so patchily practised It is a vital approach for all students: one assertion; three arguments; one case study
  • #29 Exploring using Healey’s model Tansy do you have a better image? This is all blurry!
  • #33 And we have evidence in shedloads
  • #37 It seems to be borne out in the Teaching Excellence Framework – if you believe the TEF measures teaching excellence! GPA grade point average – outputs, impact and environment combined.