SlideShare a Scribd company logo

Calibrating Assessment Literacy Through Benchmarking Tasks

Slides that partner with the paper Simon Knight, Andrea Leigh, Yvonne C. Davila, Leigh J. Martin, Daniel W. Krix, Assessment and Evaluation in Higher Education https://doi.org/10.1080/02602938.2019.1570483 In calibration tasks students assess exemplar texts using criteria against which their own work will be assessed. Typically these tasks are used in the context of training for peer assessment. Little research has been conducted on the benefits of calibration tasks, such as benchmarking, as learning opportunities in their own right. This paper examines a dataset from a long-running benchmarking task (~500 students per semester, for four semesters). We investigate the relationship of benchmarking performance to other student outcomes, including ability to self-assess accurately. We show that students who complete the benchmarking perform better, that there is a relationship between benchmarking performance and self-assessment performance, and that students appreciate the support for learning that benchmarking tasks provide. We discuss implications for teaching and learning flagging the potential of calibration tasks as an under-explored tool.

1 of 26
Download to read offline
Calibrating Assessment
Literacy Through
Benchmarking
Simon Knight, Andy Leigh, Yvonne
Davila, Leigh Martin
@sjgknight
2
Feedback at scale?
How do we give
feedback at scale?
• Assess infrequently
• Make assessments that are easy to give feedback on
(e.g. quizzes)
Teaching & Learning Forum 2018 3
How do we give
feedback at scale?
• Assess infrequently
• Make assessments that are easy to give feedback on
(quizzes, etc.)
• Use peer and self-assessment
• Ensure teams of tutors (and students) can give quality
feedback
• Provide opportunities for whole-cohort practice
Teaching & Learning Forum 2018 4
How do we give
feedback at scale?
Benchmarking tasks require students to give feedback on
previously marked exemplars, typically of varying quality.
Teaching & Learning Forum 2018 5
Benchmarking
How do we give
feedback at scale?
Benchmarking tasks require students to give feedback on
previously marked exemplars, typically of varying quality.
Why?
1. Students engage with criteria & their use
2. Students critically assess exemplars
3. Students & academics see how well students apply the
criteria (how they’re calibrated) – feedback opportunity
Teaching & Learning Forum 2018 6
Benchmarking
Ad

Recommended

COVID-19 and Disruption in Management and Education Academics: Bibliometric M...
COVID-19 and Disruption in Management and Education Academics: Bibliometric M...COVID-19 and Disruption in Management and Education Academics: Bibliometric M...
COVID-19 and Disruption in Management and Education Academics: Bibliometric M...faisalpiliang1
 
2015-Reg-Brochure1
2015-Reg-Brochure12015-Reg-Brochure1
2015-Reg-Brochure1aspiegel17
 
702 cognitive theory of multimedia
702 cognitive theory of multimedia702 cognitive theory of multimedia
702 cognitive theory of multimediaNikita Dukes
 
“School’s Out, But Class’s On”, The Largest Online Education in the World Tod...
“School’s Out, But Class’s On”, The Largest Online Education in the World Tod...“School’s Out, But Class’s On”, The Largest Online Education in the World Tod...
“School’s Out, But Class’s On”, The Largest Online Education in the World Tod...eraser Juan José Calderón
 
Vallejo Together 2020: Education Through Distance Learning
Vallejo Together 2020: Education Through Distance LearningVallejo Together 2020: Education Through Distance Learning
Vallejo Together 2020: Education Through Distance LearningMichael Barbour
 
CIDER 2020 - Sense of irony or perfect timing: Examining the research support...
CIDER 2020 - Sense of irony or perfect timing: Examining the research support...CIDER 2020 - Sense of irony or perfect timing: Examining the research support...
CIDER 2020 - Sense of irony or perfect timing: Examining the research support...Michael Barbour
 
The rights to inclusive quality physical education post covid-19
The rights to inclusive quality physical education post covid-19The rights to inclusive quality physical education post covid-19
The rights to inclusive quality physical education post covid-19Reema Alpana
 

More Related Content

What's hot

Current Trends in Education Programs
Current Trends in Education  ProgramsCurrent Trends in Education  Programs
Current Trends in Education ProgramsCollege/University
 
SITE 2018 - Virtual Schools in the US: Case Studies of Policy, Performance an...
SITE 2018 - Virtual Schools in the US: Case Studies of Policy, Performance an...SITE 2018 - Virtual Schools in the US: Case Studies of Policy, Performance an...
SITE 2018 - Virtual Schools in the US: Case Studies of Policy, Performance an...Michael Barbour
 
League of Women Voters Lansing Area (2021) - What Does the Research Tell Us? ...
League of Women Voters Lansing Area (2021) - What Does the Research Tell Us? ...League of Women Voters Lansing Area (2021) - What Does the Research Tell Us? ...
League of Women Voters Lansing Area (2021) - What Does the Research Tell Us? ...Michael Barbour
 
Virtual Learning Policy Consideration (iNacol)
Virtual Learning Policy Consideration (iNacol)Virtual Learning Policy Consideration (iNacol)
Virtual Learning Policy Consideration (iNacol)Blacketor Consultants, LLC
 
Team 5 (narrated ppt)
Team 5 (narrated ppt)Team 5 (narrated ppt)
Team 5 (narrated ppt)acefellowspro
 
EduForum 2020 Panel - Adapting Student Learning/Delivery From a Distance Amid...
EduForum 2020 Panel - Adapting Student Learning/Delivery From a Distance Amid...EduForum 2020 Panel - Adapting Student Learning/Delivery From a Distance Amid...
EduForum 2020 Panel - Adapting Student Learning/Delivery From a Distance Amid...Michael Barbour
 
Attitude of Secondary School Students towards Online Education during Covid 1...
Attitude of Secondary School Students towards Online Education during Covid 1...Attitude of Secondary School Students towards Online Education during Covid 1...
Attitude of Secondary School Students towards Online Education during Covid 1...ijtsrd
 
Key Findings from Focus Groups with College Students
Key Findings from Focus Groups with College StudentsKey Findings from Focus Groups with College Students
Key Findings from Focus Groups with College StudentsRobert Kelly
 
SITE 2018 - Examining the Validity of the iNACOL Standards: Improving K-12 On...
SITE 2018 - Examining the Validity of the iNACOL Standards: Improving K-12 On...SITE 2018 - Examining the Validity of the iNACOL Standards: Improving K-12 On...
SITE 2018 - Examining the Validity of the iNACOL Standards: Improving K-12 On...Michael Barbour
 
Alternative Learning Delivery Modalities (ALDM) of Secondary Social Studies T...
Alternative Learning Delivery Modalities (ALDM) of Secondary Social Studies T...Alternative Learning Delivery Modalities (ALDM) of Secondary Social Studies T...
Alternative Learning Delivery Modalities (ALDM) of Secondary Social Studies T...AJHSSR Journal
 
DLAC 2019 - Eight Key Trends In K-12 Digital Learning
DLAC 2019 - Eight Key Trends In K-12 Digital LearningDLAC 2019 - Eight Key Trends In K-12 Digital Learning
DLAC 2019 - Eight Key Trends In K-12 Digital LearningMichael Barbour
 
SD DL Academy 2017 - Best Practices in Blended and Distance Learning in Canada
SD DL Academy 2017 - Best Practices in Blended and Distance Learning in CanadaSD DL Academy 2017 - Best Practices in Blended and Distance Learning in Canada
SD DL Academy 2017 - Best Practices in Blended and Distance Learning in CanadaMichael Barbour
 
Umea University Symposium on Distance Learning 2021 - Pandemic Pedagogy, Remo...
Umea University Symposium on Distance Learning 2021 - Pandemic Pedagogy, Remo...Umea University Symposium on Distance Learning 2021 - Pandemic Pedagogy, Remo...
Umea University Symposium on Distance Learning 2021 - Pandemic Pedagogy, Remo...Michael Barbour
 
AERA 2017 - Examining the International Association for K–12 Online Learning ...
AERA 2017 - Examining the International Association for K–12 Online Learning ...AERA 2017 - Examining the International Association for K–12 Online Learning ...
AERA 2017 - Examining the International Association for K–12 Online Learning ...Michael Barbour
 
Towards a New End: New Pedagogies for Deep Learning
Towards a New End: New Pedagogies for Deep LearningTowards a New End: New Pedagogies for Deep Learning
Towards a New End: New Pedagogies for Deep Learningeraser Juan José Calderón
 
Solano County Virtual Fair 2020 - K-12 Schooling and the 2020-21 School Year
Solano County Virtual Fair 2020 - K-12 Schoolingand the 2020-21 School YearSolano County Virtual Fair 2020 - K-12 Schoolingand the 2020-21 School Year
Solano County Virtual Fair 2020 - K-12 Schooling and the 2020-21 School YearMichael Barbour
 
DLAC 2020 - The Union Experience: Examining Contract Language Related to E-Le...
DLAC 2020 - The Union Experience: Examining Contract Language Related to E-Le...DLAC 2020 - The Union Experience: Examining Contract Language Related to E-Le...
DLAC 2020 - The Union Experience: Examining Contract Language Related to E-Le...Michael Barbour
 
Systematic Review And Environmental Scan On Digital Learning At Minority Serv...
Systematic Review And Environmental Scan On Digital Learning At Minority Serv...Systematic Review And Environmental Scan On Digital Learning At Minority Serv...
Systematic Review And Environmental Scan On Digital Learning At Minority Serv...Tanya Joosten
 
DLAC 2019 - State Of The Nation: K-12 e-Learning In Canada
DLAC 2019 - State Of The Nation: K-12 e-Learning In CanadaDLAC 2019 - State Of The Nation: K-12 e-Learning In Canada
DLAC 2019 - State Of The Nation: K-12 e-Learning In CanadaMichael Barbour
 
Common core vers 1
Common core vers 1Common core vers 1
Common core vers 1asberg10
 

What's hot (20)

Current Trends in Education Programs
Current Trends in Education  ProgramsCurrent Trends in Education  Programs
Current Trends in Education Programs
 
SITE 2018 - Virtual Schools in the US: Case Studies of Policy, Performance an...
SITE 2018 - Virtual Schools in the US: Case Studies of Policy, Performance an...SITE 2018 - Virtual Schools in the US: Case Studies of Policy, Performance an...
SITE 2018 - Virtual Schools in the US: Case Studies of Policy, Performance an...
 
League of Women Voters Lansing Area (2021) - What Does the Research Tell Us? ...
League of Women Voters Lansing Area (2021) - What Does the Research Tell Us? ...League of Women Voters Lansing Area (2021) - What Does the Research Tell Us? ...
League of Women Voters Lansing Area (2021) - What Does the Research Tell Us? ...
 
Virtual Learning Policy Consideration (iNacol)
Virtual Learning Policy Consideration (iNacol)Virtual Learning Policy Consideration (iNacol)
Virtual Learning Policy Consideration (iNacol)
 
Team 5 (narrated ppt)
Team 5 (narrated ppt)Team 5 (narrated ppt)
Team 5 (narrated ppt)
 
EduForum 2020 Panel - Adapting Student Learning/Delivery From a Distance Amid...
EduForum 2020 Panel - Adapting Student Learning/Delivery From a Distance Amid...EduForum 2020 Panel - Adapting Student Learning/Delivery From a Distance Amid...
EduForum 2020 Panel - Adapting Student Learning/Delivery From a Distance Amid...
 
Attitude of Secondary School Students towards Online Education during Covid 1...
Attitude of Secondary School Students towards Online Education during Covid 1...Attitude of Secondary School Students towards Online Education during Covid 1...
Attitude of Secondary School Students towards Online Education during Covid 1...
 
Key Findings from Focus Groups with College Students
Key Findings from Focus Groups with College StudentsKey Findings from Focus Groups with College Students
Key Findings from Focus Groups with College Students
 
SITE 2018 - Examining the Validity of the iNACOL Standards: Improving K-12 On...
SITE 2018 - Examining the Validity of the iNACOL Standards: Improving K-12 On...SITE 2018 - Examining the Validity of the iNACOL Standards: Improving K-12 On...
SITE 2018 - Examining the Validity of the iNACOL Standards: Improving K-12 On...
 
Alternative Learning Delivery Modalities (ALDM) of Secondary Social Studies T...
Alternative Learning Delivery Modalities (ALDM) of Secondary Social Studies T...Alternative Learning Delivery Modalities (ALDM) of Secondary Social Studies T...
Alternative Learning Delivery Modalities (ALDM) of Secondary Social Studies T...
 
DLAC 2019 - Eight Key Trends In K-12 Digital Learning
DLAC 2019 - Eight Key Trends In K-12 Digital LearningDLAC 2019 - Eight Key Trends In K-12 Digital Learning
DLAC 2019 - Eight Key Trends In K-12 Digital Learning
 
SD DL Academy 2017 - Best Practices in Blended and Distance Learning in Canada
SD DL Academy 2017 - Best Practices in Blended and Distance Learning in CanadaSD DL Academy 2017 - Best Practices in Blended and Distance Learning in Canada
SD DL Academy 2017 - Best Practices in Blended and Distance Learning in Canada
 
Umea University Symposium on Distance Learning 2021 - Pandemic Pedagogy, Remo...
Umea University Symposium on Distance Learning 2021 - Pandemic Pedagogy, Remo...Umea University Symposium on Distance Learning 2021 - Pandemic Pedagogy, Remo...
Umea University Symposium on Distance Learning 2021 - Pandemic Pedagogy, Remo...
 
AERA 2017 - Examining the International Association for K–12 Online Learning ...
AERA 2017 - Examining the International Association for K–12 Online Learning ...AERA 2017 - Examining the International Association for K–12 Online Learning ...
AERA 2017 - Examining the International Association for K–12 Online Learning ...
 
Towards a New End: New Pedagogies for Deep Learning
Towards a New End: New Pedagogies for Deep LearningTowards a New End: New Pedagogies for Deep Learning
Towards a New End: New Pedagogies for Deep Learning
 
Solano County Virtual Fair 2020 - K-12 Schooling and the 2020-21 School Year
Solano County Virtual Fair 2020 - K-12 Schoolingand the 2020-21 School YearSolano County Virtual Fair 2020 - K-12 Schoolingand the 2020-21 School Year
Solano County Virtual Fair 2020 - K-12 Schooling and the 2020-21 School Year
 
DLAC 2020 - The Union Experience: Examining Contract Language Related to E-Le...
DLAC 2020 - The Union Experience: Examining Contract Language Related to E-Le...DLAC 2020 - The Union Experience: Examining Contract Language Related to E-Le...
DLAC 2020 - The Union Experience: Examining Contract Language Related to E-Le...
 
Systematic Review And Environmental Scan On Digital Learning At Minority Serv...
Systematic Review And Environmental Scan On Digital Learning At Minority Serv...Systematic Review And Environmental Scan On Digital Learning At Minority Serv...
Systematic Review And Environmental Scan On Digital Learning At Minority Serv...
 
DLAC 2019 - State Of The Nation: K-12 e-Learning In Canada
DLAC 2019 - State Of The Nation: K-12 e-Learning In CanadaDLAC 2019 - State Of The Nation: K-12 e-Learning In Canada
DLAC 2019 - State Of The Nation: K-12 e-Learning In Canada
 
Common core vers 1
Common core vers 1Common core vers 1
Common core vers 1
 

Similar to Calibrating Assessment Literacy Through Benchmarking Tasks

Ai Fl Presentation
Ai Fl PresentationAi Fl Presentation
Ai Fl Presentationstninplc
 
Assessment: Achieving improved efficiency, effectiveness, educational integri...
Assessment:Achieving improved efficiency, effectiveness, educational integri...Assessment:Achieving improved efficiency, effectiveness, educational integri...
Assessment: Achieving improved efficiency, effectiveness, educational integri...Diana Quinn
 
Improving student learning through programme assessment
Improving student learning through programme assessmentImproving student learning through programme assessment
Improving student learning through programme assessmentTansy Jessop
 
Assessment System And Model Is Based On The Following...
Assessment System And Model Is Based On The Following...Assessment System And Model Is Based On The Following...
Assessment System And Model Is Based On The Following...Caitlin Wilson
 
2.26.15-CEIT-Assessment-Rubric-Development-PowerPoint.pdf
2.26.15-CEIT-Assessment-Rubric-Development-PowerPoint.pdf2.26.15-CEIT-Assessment-Rubric-Development-PowerPoint.pdf
2.26.15-CEIT-Assessment-Rubric-Development-PowerPoint.pdfIvyAnnLeizelQuintas
 
TESTA to FASTECH (November 2011)
 TESTA to FASTECH (November 2011) TESTA to FASTECH (November 2011)
TESTA to FASTECH (November 2011)TESTA winch
 
TESTA, Durham University (December 2013)
TESTA, Durham University (December 2013)TESTA, Durham University (December 2013)
TESTA, Durham University (December 2013)TESTA winch
 
13. using rubrics in student assessment
13. using rubrics in student assessment13. using rubrics in student assessment
13. using rubrics in student assessmentCate Atehortua
 
The future of feedback: Evaluation of information and digital literacy teachi...
The future of feedback: Evaluation of information and digital literacy teachi...The future of feedback: Evaluation of information and digital literacy teachi...
The future of feedback: Evaluation of information and digital literacy teachi...IL Group (CILIP Information Literacy Group)
 
Online writing feedback: A national study exploring the service and learning ...
Online writing feedback: A national study exploring the service and learning ...Online writing feedback: A national study exploring the service and learning ...
Online writing feedback: A national study exploring the service and learning ...Studiosity.com
 
Write Sample-Spring 2016 Success Counseling Report
Write Sample-Spring 2016 Success Counseling ReportWrite Sample-Spring 2016 Success Counseling Report
Write Sample-Spring 2016 Success Counseling ReportNathan Menacher
 
Assessment cpd
Assessment cpdAssessment cpd
Assessment cpdMrsMcGinty
 
TESTA to FASTECH Presentation
TESTA to FASTECH PresentationTESTA to FASTECH Presentation
TESTA to FASTECH PresentationTansy_Jessop
 
An evidence based model
An evidence based modelAn evidence based model
An evidence based modelJisc
 
Portsmouth BAM Knowledge and Learning SIG
Portsmouth BAM Knowledge and Learning SIGPortsmouth BAM Knowledge and Learning SIG
Portsmouth BAM Knowledge and Learning SIGTansy Jessop
 

Similar to Calibrating Assessment Literacy Through Benchmarking Tasks (20)

Ai Fl Presentation
Ai Fl PresentationAi Fl Presentation
Ai Fl Presentation
 
Intro to rubrics
Intro to rubricsIntro to rubrics
Intro to rubrics
 
Intro to rubrics
Intro to rubricsIntro to rubrics
Intro to rubrics
 
Assessment: Achieving improved efficiency, effectiveness, educational integri...
Assessment:Achieving improved efficiency, effectiveness, educational integri...Assessment:Achieving improved efficiency, effectiveness, educational integri...
Assessment: Achieving improved efficiency, effectiveness, educational integri...
 
Improving student learning through programme assessment
Improving student learning through programme assessmentImproving student learning through programme assessment
Improving student learning through programme assessment
 
Assessment System And Model Is Based On The Following...
Assessment System And Model Is Based On The Following...Assessment System And Model Is Based On The Following...
Assessment System And Model Is Based On The Following...
 
2.26.15-CEIT-Assessment-Rubric-Development-PowerPoint.pdf
2.26.15-CEIT-Assessment-Rubric-Development-PowerPoint.pdf2.26.15-CEIT-Assessment-Rubric-Development-PowerPoint.pdf
2.26.15-CEIT-Assessment-Rubric-Development-PowerPoint.pdf
 
TESTA to FASTECH (November 2011)
 TESTA to FASTECH (November 2011) TESTA to FASTECH (November 2011)
TESTA to FASTECH (November 2011)
 
REA: Performance Appraisal & Workplace Performance
REA: Performance Appraisal & Workplace PerformanceREA: Performance Appraisal & Workplace Performance
REA: Performance Appraisal & Workplace Performance
 
TESTA, Durham University (December 2013)
TESTA, Durham University (December 2013)TESTA, Durham University (December 2013)
TESTA, Durham University (December 2013)
 
13. using rubrics in student assessment
13. using rubrics in student assessment13. using rubrics in student assessment
13. using rubrics in student assessment
 
The future of feedback: Evaluation of information and digital literacy teachi...
The future of feedback: Evaluation of information and digital literacy teachi...The future of feedback: Evaluation of information and digital literacy teachi...
The future of feedback: Evaluation of information and digital literacy teachi...
 
Online writing feedback: A national study exploring the service and learning ...
Online writing feedback: A national study exploring the service and learning ...Online writing feedback: A national study exploring the service and learning ...
Online writing feedback: A national study exploring the service and learning ...
 
Write Sample-Spring 2016 Success Counseling Report
Write Sample-Spring 2016 Success Counseling ReportWrite Sample-Spring 2016 Success Counseling Report
Write Sample-Spring 2016 Success Counseling Report
 
Assessment cpd
Assessment cpdAssessment cpd
Assessment cpd
 
App pgr workshop3
App pgr workshop3App pgr workshop3
App pgr workshop3
 
TESTA to FASTECH Presentation
TESTA to FASTECH PresentationTESTA to FASTECH Presentation
TESTA to FASTECH Presentation
 
An evidence based model
An evidence based modelAn evidence based model
An evidence based model
 
Portsmouth BAM Knowledge and Learning SIG
Portsmouth BAM Knowledge and Learning SIGPortsmouth BAM Knowledge and Learning SIG
Portsmouth BAM Knowledge and Learning SIG
 
Experience of using formative assessment and students perception of formative...
Experience of using formative assessment and students perception of formative...Experience of using formative assessment and students perception of formative...
Experience of using formative assessment and students perception of formative...
 

More from Simon Knight

CiteLearn: An academic tool for learning to cite sources
CiteLearn: An academic tool for learning to cite sourcesCiteLearn: An academic tool for learning to cite sources
CiteLearn: An academic tool for learning to cite sourcesSimon Knight
 
Aligning Learning Analytics with Classroom Practices & Needs
Aligning Learning Analytics with Classroom Practices & NeedsAligning Learning Analytics with Classroom Practices & Needs
Aligning Learning Analytics with Classroom Practices & NeedsSimon Knight
 
Who to believe: How epistemic cognition can inform science communication (key...
Who to believe: How epistemic cognition can inform science communication (key...Who to believe: How epistemic cognition can inform science communication (key...
Who to believe: How epistemic cognition can inform science communication (key...Simon Knight
 
Writing Analytics for Epistemic Features of Student Writing #icls2016 talk
Writing Analytics for Epistemic Features of Student Writing #icls2016 talkWriting Analytics for Epistemic Features of Student Writing #icls2016 talk
Writing Analytics for Epistemic Features of Student Writing #icls2016 talkSimon Knight
 
Wikimedia UK volunteer structure proposals
Wikimedia UK volunteer structure proposalsWikimedia UK volunteer structure proposals
Wikimedia UK volunteer structure proposalsSimon Knight
 
Developing a multiple-document-processing performance assessment for epistem...
 Developing a multiple-document-processing performance assessment for epistem... Developing a multiple-document-processing performance assessment for epistem...
Developing a multiple-document-processing performance assessment for epistem...Simon Knight
 
Epistemic networks for Epistemic Commitments
Epistemic networks for Epistemic CommitmentsEpistemic networks for Epistemic Commitments
Epistemic networks for Epistemic CommitmentsSimon Knight
 
Learning analytics for learning and becoming in practice - #ICLS2014 workshop
Learning analytics for learning and becoming in practice - #ICLS2014 workshopLearning analytics for learning and becoming in practice - #ICLS2014 workshop
Learning analytics for learning and becoming in practice - #ICLS2014 workshopSimon Knight
 
Wikimedia Strategy - making it impactful, measuring impact, and thinking abou...
Wikimedia Strategy - making it impactful, measuring impact, and thinking abou...Wikimedia Strategy - making it impactful, measuring impact, and thinking abou...
Wikimedia Strategy - making it impactful, measuring impact, and thinking abou...Simon Knight
 
Learning and analytics – where do the two meet? #HEABigData summit day
Learning and analytics – where do the two meet? #HEABigData summit dayLearning and analytics – where do the two meet? #HEABigData summit day
Learning and analytics – where do the two meet? #HEABigData summit daySimon Knight
 
Finding Knowledge: Assessing Knowledge in the Age of Search
Finding Knowledge: Assessing Knowledge in the Age of SearchFinding Knowledge: Assessing Knowledge in the Age of Search
Finding Knowledge: Assessing Knowledge in the Age of SearchSimon Knight
 
Analysing Learning Through Mediawiki
Analysing Learning Through MediawikiAnalysing Learning Through Mediawiki
Analysing Learning Through MediawikiSimon Knight
 
Mediawiki for oer and learning analytics - understanding learning resources a...
Mediawiki for oer and learning analytics - understanding learning resources a...Mediawiki for oer and learning analytics - understanding learning resources a...
Mediawiki for oer and learning analytics - understanding learning resources a...Simon Knight
 
LAK13: Epistemology, Pedagogy, Assessment and Learning Analytics
LAK13: Epistemology, Pedagogy, Assessment and Learning AnalyticsLAK13: Epistemology, Pedagogy, Assessment and Learning Analytics
LAK13: Epistemology, Pedagogy, Assessment and Learning AnalyticsSimon Knight
 
Dcla13 discourse, computation and context – sociocultural dcla
Dcla13 discourse, computation and context – sociocultural dclaDcla13 discourse, computation and context – sociocultural dcla
Dcla13 discourse, computation and context – sociocultural dclaSimon Knight
 
SoLAR Storm talk: epistemology, pedagogy, assessment
SoLAR Storm talk: epistemology, pedagogy, assessmentSoLAR Storm talk: epistemology, pedagogy, assessment
SoLAR Storm talk: epistemology, pedagogy, assessmentSimon Knight
 

More from Simon Knight (16)

CiteLearn: An academic tool for learning to cite sources
CiteLearn: An academic tool for learning to cite sourcesCiteLearn: An academic tool for learning to cite sources
CiteLearn: An academic tool for learning to cite sources
 
Aligning Learning Analytics with Classroom Practices & Needs
Aligning Learning Analytics with Classroom Practices & NeedsAligning Learning Analytics with Classroom Practices & Needs
Aligning Learning Analytics with Classroom Practices & Needs
 
Who to believe: How epistemic cognition can inform science communication (key...
Who to believe: How epistemic cognition can inform science communication (key...Who to believe: How epistemic cognition can inform science communication (key...
Who to believe: How epistemic cognition can inform science communication (key...
 
Writing Analytics for Epistemic Features of Student Writing #icls2016 talk
Writing Analytics for Epistemic Features of Student Writing #icls2016 talkWriting Analytics for Epistemic Features of Student Writing #icls2016 talk
Writing Analytics for Epistemic Features of Student Writing #icls2016 talk
 
Wikimedia UK volunteer structure proposals
Wikimedia UK volunteer structure proposalsWikimedia UK volunteer structure proposals
Wikimedia UK volunteer structure proposals
 
Developing a multiple-document-processing performance assessment for epistem...
 Developing a multiple-document-processing performance assessment for epistem... Developing a multiple-document-processing performance assessment for epistem...
Developing a multiple-document-processing performance assessment for epistem...
 
Epistemic networks for Epistemic Commitments
Epistemic networks for Epistemic CommitmentsEpistemic networks for Epistemic Commitments
Epistemic networks for Epistemic Commitments
 
Learning analytics for learning and becoming in practice - #ICLS2014 workshop
Learning analytics for learning and becoming in practice - #ICLS2014 workshopLearning analytics for learning and becoming in practice - #ICLS2014 workshop
Learning analytics for learning and becoming in practice - #ICLS2014 workshop
 
Wikimedia Strategy - making it impactful, measuring impact, and thinking abou...
Wikimedia Strategy - making it impactful, measuring impact, and thinking abou...Wikimedia Strategy - making it impactful, measuring impact, and thinking abou...
Wikimedia Strategy - making it impactful, measuring impact, and thinking abou...
 
Learning and analytics – where do the two meet? #HEABigData summit day
Learning and analytics – where do the two meet? #HEABigData summit dayLearning and analytics – where do the two meet? #HEABigData summit day
Learning and analytics – where do the two meet? #HEABigData summit day
 
Finding Knowledge: Assessing Knowledge in the Age of Search
Finding Knowledge: Assessing Knowledge in the Age of SearchFinding Knowledge: Assessing Knowledge in the Age of Search
Finding Knowledge: Assessing Knowledge in the Age of Search
 
Analysing Learning Through Mediawiki
Analysing Learning Through MediawikiAnalysing Learning Through Mediawiki
Analysing Learning Through Mediawiki
 
Mediawiki for oer and learning analytics - understanding learning resources a...
Mediawiki for oer and learning analytics - understanding learning resources a...Mediawiki for oer and learning analytics - understanding learning resources a...
Mediawiki for oer and learning analytics - understanding learning resources a...
 
LAK13: Epistemology, Pedagogy, Assessment and Learning Analytics
LAK13: Epistemology, Pedagogy, Assessment and Learning AnalyticsLAK13: Epistemology, Pedagogy, Assessment and Learning Analytics
LAK13: Epistemology, Pedagogy, Assessment and Learning Analytics
 
Dcla13 discourse, computation and context – sociocultural dcla
Dcla13 discourse, computation and context – sociocultural dclaDcla13 discourse, computation and context – sociocultural dcla
Dcla13 discourse, computation and context – sociocultural dcla
 
SoLAR Storm talk: epistemology, pedagogy, assessment
SoLAR Storm talk: epistemology, pedagogy, assessmentSoLAR Storm talk: epistemology, pedagogy, assessment
SoLAR Storm talk: epistemology, pedagogy, assessment
 

Recently uploaded

A LABORATORY MANUAL FOR ORGANIC CHEMISTRY.pdf
A LABORATORY MANUAL FOR ORGANIC CHEMISTRY.pdfA LABORATORY MANUAL FOR ORGANIC CHEMISTRY.pdf
A LABORATORY MANUAL FOR ORGANIC CHEMISTRY.pdfDr.M.Geethavani
 
The Ministry of Utmost Happiness by Arundhati Roy
The Ministry of Utmost Happiness by Arundhati RoyThe Ministry of Utmost Happiness by Arundhati Roy
The Ministry of Utmost Happiness by Arundhati RoyTrushali Dodiya
 
Overview of Databases and Data Modelling-1.pdf
Overview of Databases and Data Modelling-1.pdfOverview of Databases and Data Modelling-1.pdf
Overview of Databases and Data Modelling-1.pdfChristalin Nelson
 
UniSC Moreton Bay Library self-guided tour
UniSC Moreton Bay Library self-guided tourUniSC Moreton Bay Library self-guided tour
UniSC Moreton Bay Library self-guided tourUSC_Library
 
Plant Genetic Resources, Germplasm, gene pool - Copy.pptx
Plant Genetic Resources, Germplasm, gene pool - Copy.pptxPlant Genetic Resources, Germplasm, gene pool - Copy.pptx
Plant Genetic Resources, Germplasm, gene pool - Copy.pptxAKSHAYMAGAR17
 
ACTIVIDAD DE CLASE No 1 sopa de letras.docx
ACTIVIDAD DE CLASE No 1 sopa de letras.docxACTIVIDAD DE CLASE No 1 sopa de letras.docx
ACTIVIDAD DE CLASE No 1 sopa de letras.docxMaria Lucia Céspedes
 
Ideotype concept and climate resilient crop varieties for future- Wheat, Rice...
Ideotype concept and climate resilient crop varieties for future- Wheat, Rice...Ideotype concept and climate resilient crop varieties for future- Wheat, Rice...
Ideotype concept and climate resilient crop varieties for future- Wheat, Rice...AKSHAYMAGAR17
 
DISCOURSE: TEXT AS CONNECTED DISCOURSE
DISCOURSE:   TEXT AS CONNECTED DISCOURSEDISCOURSE:   TEXT AS CONNECTED DISCOURSE
DISCOURSE: TEXT AS CONNECTED DISCOURSEMYDA ANGELICA SUAN
 
2.20.24 Asian Americans and the Myth of the Model Minority.pptx
2.20.24 Asian Americans and the Myth of the Model Minority.pptx2.20.24 Asian Americans and the Myth of the Model Minority.pptx
2.20.24 Asian Americans and the Myth of the Model Minority.pptxMaryPotorti1
 
Overview of Databases and Data Modelling-2.pdf
Overview of Databases and Data Modelling-2.pdfOverview of Databases and Data Modelling-2.pdf
Overview of Databases and Data Modelling-2.pdfChristalin Nelson
 
Routes of Drug Administrations PPT..pptx
Routes of Drug Administrations PPT..pptxRoutes of Drug Administrations PPT..pptx
Routes of Drug Administrations PPT..pptxRenuka N Sunagad
 
Grades 7 to 8 Anti- OSAEC and CSAEM session.pptx
Grades 7 to 8 Anti- OSAEC and CSAEM session.pptxGrades 7 to 8 Anti- OSAEC and CSAEM session.pptx
Grades 7 to 8 Anti- OSAEC and CSAEM session.pptxGladysValencia13
 
UniSC Fraser Coast library self-guided tour
UniSC Fraser Coast library self-guided tourUniSC Fraser Coast library self-guided tour
UniSC Fraser Coast library self-guided tourUSC_Library
 
UniSC Sunshine Coast library self-guided tour
UniSC Sunshine Coast library self-guided tourUniSC Sunshine Coast library self-guided tour
UniSC Sunshine Coast library self-guided tourUSC_Library
 
BÀI TẬP BỔ TRỢ 4 KĨ NĂNG TIẾNG ANH LỚP 8 - HK2 - GLOBAL SUCCESS - NĂM HỌC 202...
BÀI TẬP BỔ TRỢ 4 KĨ NĂNG TIẾNG ANH LỚP 8 - HK2 - GLOBAL SUCCESS - NĂM HỌC 202...BÀI TẬP BỔ TRỢ 4 KĨ NĂNG TIẾNG ANH LỚP 8 - HK2 - GLOBAL SUCCESS - NĂM HỌC 202...
BÀI TẬP BỔ TRỢ 4 KĨ NĂNG TIẾNG ANH LỚP 8 - HK2 - GLOBAL SUCCESS - NĂM HỌC 202...Nguyen Thanh Tu Collection
 
2.20.24 The March on Washington for Jobs and Freedom.pptx
2.20.24 The March on Washington for Jobs and Freedom.pptx2.20.24 The March on Washington for Jobs and Freedom.pptx
2.20.24 The March on Washington for Jobs and Freedom.pptxMaryPotorti1
 
11 CI SINIF SINAQLARI - 1-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 1-2023-Aynura-Hamidova.pdf11 CI SINIF SINAQLARI - 1-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 1-2023-Aynura-Hamidova.pdfAynouraHamidova
 
Food Web SlideShare for Ecology Notes Quiz in Canvas
Food Web SlideShare for Ecology Notes Quiz in CanvasFood Web SlideShare for Ecology Notes Quiz in Canvas
Food Web SlideShare for Ecology Notes Quiz in CanvasAlexandraSwartzwelde
 

Recently uploaded (20)

A LABORATORY MANUAL FOR ORGANIC CHEMISTRY.pdf
A LABORATORY MANUAL FOR ORGANIC CHEMISTRY.pdfA LABORATORY MANUAL FOR ORGANIC CHEMISTRY.pdf
A LABORATORY MANUAL FOR ORGANIC CHEMISTRY.pdf
 
The Ministry of Utmost Happiness by Arundhati Roy
The Ministry of Utmost Happiness by Arundhati RoyThe Ministry of Utmost Happiness by Arundhati Roy
The Ministry of Utmost Happiness by Arundhati Roy
 
Overview of Databases and Data Modelling-1.pdf
Overview of Databases and Data Modelling-1.pdfOverview of Databases and Data Modelling-1.pdf
Overview of Databases and Data Modelling-1.pdf
 
Lipids as Biopolymer
Lipids as Biopolymer Lipids as Biopolymer
Lipids as Biopolymer
 
UniSC Moreton Bay Library self-guided tour
UniSC Moreton Bay Library self-guided tourUniSC Moreton Bay Library self-guided tour
UniSC Moreton Bay Library self-guided tour
 
Plant Genetic Resources, Germplasm, gene pool - Copy.pptx
Plant Genetic Resources, Germplasm, gene pool - Copy.pptxPlant Genetic Resources, Germplasm, gene pool - Copy.pptx
Plant Genetic Resources, Germplasm, gene pool - Copy.pptx
 
first section physiology laboratory.pptx
first section physiology laboratory.pptxfirst section physiology laboratory.pptx
first section physiology laboratory.pptx
 
ACTIVIDAD DE CLASE No 1 sopa de letras.docx
ACTIVIDAD DE CLASE No 1 sopa de letras.docxACTIVIDAD DE CLASE No 1 sopa de letras.docx
ACTIVIDAD DE CLASE No 1 sopa de letras.docx
 
Ideotype concept and climate resilient crop varieties for future- Wheat, Rice...
Ideotype concept and climate resilient crop varieties for future- Wheat, Rice...Ideotype concept and climate resilient crop varieties for future- Wheat, Rice...
Ideotype concept and climate resilient crop varieties for future- Wheat, Rice...
 
DISCOURSE: TEXT AS CONNECTED DISCOURSE
DISCOURSE:   TEXT AS CONNECTED DISCOURSEDISCOURSE:   TEXT AS CONNECTED DISCOURSE
DISCOURSE: TEXT AS CONNECTED DISCOURSE
 
2.20.24 Asian Americans and the Myth of the Model Minority.pptx
2.20.24 Asian Americans and the Myth of the Model Minority.pptx2.20.24 Asian Americans and the Myth of the Model Minority.pptx
2.20.24 Asian Americans and the Myth of the Model Minority.pptx
 
Overview of Databases and Data Modelling-2.pdf
Overview of Databases and Data Modelling-2.pdfOverview of Databases and Data Modelling-2.pdf
Overview of Databases and Data Modelling-2.pdf
 
Routes of Drug Administrations PPT..pptx
Routes of Drug Administrations PPT..pptxRoutes of Drug Administrations PPT..pptx
Routes of Drug Administrations PPT..pptx
 
Grades 7 to 8 Anti- OSAEC and CSAEM session.pptx
Grades 7 to 8 Anti- OSAEC and CSAEM session.pptxGrades 7 to 8 Anti- OSAEC and CSAEM session.pptx
Grades 7 to 8 Anti- OSAEC and CSAEM session.pptx
 
UniSC Fraser Coast library self-guided tour
UniSC Fraser Coast library self-guided tourUniSC Fraser Coast library self-guided tour
UniSC Fraser Coast library self-guided tour
 
UniSC Sunshine Coast library self-guided tour
UniSC Sunshine Coast library self-guided tourUniSC Sunshine Coast library self-guided tour
UniSC Sunshine Coast library self-guided tour
 
BÀI TẬP BỔ TRỢ 4 KĨ NĂNG TIẾNG ANH LỚP 8 - HK2 - GLOBAL SUCCESS - NĂM HỌC 202...
BÀI TẬP BỔ TRỢ 4 KĨ NĂNG TIẾNG ANH LỚP 8 - HK2 - GLOBAL SUCCESS - NĂM HỌC 202...BÀI TẬP BỔ TRỢ 4 KĨ NĂNG TIẾNG ANH LỚP 8 - HK2 - GLOBAL SUCCESS - NĂM HỌC 202...
BÀI TẬP BỔ TRỢ 4 KĨ NĂNG TIẾNG ANH LỚP 8 - HK2 - GLOBAL SUCCESS - NĂM HỌC 202...
 
2.20.24 The March on Washington for Jobs and Freedom.pptx
2.20.24 The March on Washington for Jobs and Freedom.pptx2.20.24 The March on Washington for Jobs and Freedom.pptx
2.20.24 The March on Washington for Jobs and Freedom.pptx
 
11 CI SINIF SINAQLARI - 1-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 1-2023-Aynura-Hamidova.pdf11 CI SINIF SINAQLARI - 1-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 1-2023-Aynura-Hamidova.pdf
 
Food Web SlideShare for Ecology Notes Quiz in Canvas
Food Web SlideShare for Ecology Notes Quiz in CanvasFood Web SlideShare for Ecology Notes Quiz in Canvas
Food Web SlideShare for Ecology Notes Quiz in Canvas
 

Calibrating Assessment Literacy Through Benchmarking Tasks

  • 1. Calibrating Assessment Literacy Through Benchmarking Simon Knight, Andy Leigh, Yvonne Davila, Leigh Martin @sjgknight
  • 3. How do we give feedback at scale? • Assess infrequently • Make assessments that are easy to give feedback on (e.g. quizzes) Teaching & Learning Forum 2018 3
  • 4. How do we give feedback at scale? • Assess infrequently • Make assessments that are easy to give feedback on (quizzes, etc.) • Use peer and self-assessment • Ensure teams of tutors (and students) can give quality feedback • Provide opportunities for whole-cohort practice Teaching & Learning Forum 2018 4
  • 5. How do we give feedback at scale? Benchmarking tasks require students to give feedback on previously marked exemplars, typically of varying quality. Teaching & Learning Forum 2018 5 Benchmarking
  • 6. How do we give feedback at scale? Benchmarking tasks require students to give feedback on previously marked exemplars, typically of varying quality. Why? 1. Students engage with criteria & their use 2. Students critically assess exemplars 3. Students & academics see how well students apply the criteria (how they’re calibrated) – feedback opportunity Teaching & Learning Forum 2018 6 Benchmarking
  • 8. How do we measure impact? Measuring impact of teaching innovation is hard Typically lots of semester-semester changes (including the students!) Impact often measured via ‘happy sheets’ and few enthusiastic learners Teaching & Learning Forum 2018 8
  • 9. How do we measure impact? ~500 students in first year Biocomplexity Since 2012 they have done a benchmarking task via SPARKPlus + self-assessment Analysed 2012-15 data Teaching & Learning Forum 2018 9 Our context
  • 10. Q1: How accurate are students in their self- assessments, and what is the relationship of this to their grade? Teaching & Learning Forum 2018 10 Histogram comparing the distribution of staff marks to student self-assessments Students over-estimate their grades;
  • 11. Q1: How accurate are students in their self- assessments, and what is the relationship of this to their grade? Teaching & Learning Forum 2018 11 Histogram comparing the distribution of staff marks to student self-assessments Students over-estimate their grades; those who overestimate do worse Strong correlation; i.e. such that a relationship between over- estimating, and having a lower mark (and vice-versa); r(2012) = .68, p < .0001.
  • 12. Q2: Do students who complete the benchmarking perform better in their assessment than those who do not? Teaching & Learning Forum 2018 12 t(137.88) = 5.41, p < .0001. d = 0.62 (medium effect) (ignoring students who dropped out). Students who didn’t do the task also varied more in their criterion-level marks Students who do not complete the benchmarking perform worse (SD = 9.28, N = 1979) Did the task M = 74.14 (SD = 12.16, N = 129) Did not do the task M = 68.24
  • 13. Teaching & Learning Forum 2018 13 Q3: Is accuracy on the benchmarking predictive of final mark? No evidence of link between benchmarking accuracy and final mark
  • 14. Teaching & Learning Forum 2018 14 Comparing distances (i.e. the mark they gave themselves, subtracted from their actual mark) a medium effect (d = .62) t(133.84) = 3.00, p = .0032. Q4: Are students who complete the benchmarking significantly more accurate in their self- assessment Students who benchmark, are better self-assessors (SD = 16.13, N = 1979) Did the task M = 1.15 (SD = 27.47, N = 129) Did not do the task M = 8.13
  • 15. Teaching & Learning Forum 2018 15 small significant relationship between the benchmarking distance scores and student self- assessment distances, r(1887) = .10, p < .0001. Q5: Is accuracy on the benchmarking related to self-assessment accuracy? And, students who are more accurate at benchmarking are more accurate self-assessors
  • 16. Teaching & Learning Forum 2018 16 2012 & 2013 430 students (~45% of the cohort) did a feedback survey. The feedback from these cohorts was generally positive (>75% agree or strongly agree on all qs) The SPARK benchmarking process (week 4) helped me to engage early with the report assessment criteria The report assessment criteria helped me to understand what was expected in my report I followed the assessment criteria closely when writing my report I understood how each assessment criterion contributed to a particular Graduate Attribute Self-assessing my report helped me to critically evaluated my own academic performance in this task I have a better understanding of why scientific writing skills are important for a scientific career Overall I was satisfied with the report-writing learning process Q6: What are student perceptions of feedback structures to support their assignment completion? Students think the task is valuable
  • 17. For example… Teaching & Learning Forum 2018 17 Benchmarking helped me to understand what level of writing was expected for each grade. The feedback and re-submission really helped me to better my writing and to understand how I could improve That it forced me to be familiar with the marking criteria BEFORE writing the assignment. Usually I look at the criteria after writing the assignment and seeing whether it met the criteria, but with this method I made sure to incorporate the points whilst writing. Having previous reports to look at and gain understanding how to write and what the markers are looking for.
  • 19. How do we give feedback at scale? • Is quality of student’s written comments in benchmarking related to their other learning outcomes? • If students learn from giving feedback, how do we build that capacity? • How do we support students to understand the feedback they receive, and to make sure they get consistently good feedback from tutors? Teaching & Learning Forum 2018 19
  • 20. How do we give feedback at scale? • Is quality of student’s written comments in benchmarking related to their other learning outcomes? (ongoing analysis) • If students learn from giving feedback, how do we build that capacity? • How do we support students to understand the feedback they receive, and to make sure they get consistently good feedback from tutors? • Created a tutor & student feedback guide Teaching & Learning Forum 2018 20
  • 21. Your turn… Teaching & Learning Forum 2018 21
  • 23. 23Feedback on “References” criteria from Example B 1-5 “It has got 6 references which is good number of credible references.” In this report, the person did demonstrate a well knowledge about referencing “Harvard style referencing was well used. Next time include volumes/editions/page number to specify what section of the book/journal information was obtained. The quality of paraphrasing is really only of a credit standard - it’s evident you’ve used some secondary sources, but the in-text referencing style needs a little work; to improve, have a look at the rubric given, and familiarise yourself with the resources provided on “in-text” referencing.” The citation was well presented and done correctly however the reference list was poorly set out. Out of the four resources on the reference list only one contained authors. The others were scientific journals and books and therefore needed authors on this list. The second resource lacks volume and page numbers whilst the third lacks a sub heading reference that the second has. Lack of consistency is found throughout this reference list and needs more work. Lastly only four references is not enough to validate the argument. Very little citation of these references are found in the discussion and therefore isn't linking the work of the valid resources to the reasons within the experiment. Great improvement needed. “Referencing was okay but infrequent and some references were ancient.” Rank these pieces of feedback in order of “most useful” (1) to “least useful” (5)
  • 24. Your turn… Teaching & Learning Forum 2018 24 https://tinyurl.com/BenchmarkingGuide
  • 25. Thank you Simon Knight, Andy Leigh, Yvonne Davila, Leigh Martin @sjgknight Thanks to Dan Krix and Alex Thompson for their work on the benchmarking project, and to other academics and students who have contributed to the benchmarking development. Thanks to Shirley Alexander for VCLT funding in support of this project Draft paper available on request https://tinyurl.com/BenchmarkingGuide
  • 26. 2012-15 data analysed Data from 2012-15 of this innovation was analysed to investigate the relationship between accuracy of student-assessments and learning outcomes, and to understand the features of quality feedback in these tasks. Analysis indicates that: students who complete the benchmarking task perform better that students who are more accurate self-assessors perform better That students who are more accurate in the benchmarking task are also more accurate in the self-assessment task The students are overwhelmingly positive about the task, and are able to articulate its key intended learning outcomes Teaching & Learning Forum 2018 26

Editor's Notes

  1. One approach – assess infrequently…the worst of the approaches. Easy auto-grading quizzes can be quite a good method to introduce practice opportunities, but limited scope
  2. So, peer and self assessment is interesting, and good evidence it leads to positive outcomes
  3. students who overestimate their mark (i.e., have negative distances) are more likely to have lower overall marks, while students who underestimate their mark significantly (i.e. have positive distances), are more likely to have higher overall marks
  4. students who overestimate their mark (i.e., have negative distances) are more likely to have lower overall marks, while students who underestimate their mark significantly (i.e. have positive distances), are more likely to have higher overall marks
  5. There was a significant difference in the overall marks of students such that those who completed the benchmarking task scored higher (M = 74.14, SD = 9.28, N = 1979), compared to those who did not (M = 68.24, SD = 12.16, N = 129); t(137.88) = 5.41, p < .0001. d = 0.62. In addition, students who completed the benchmarking task had significantly lower mark variability among criteria (computed by calculating the standard deviation of their marks across the criteria) (M = 5.54, SD = 3.28, N = 1972), than those who did not (M = 6.73, SD = 5.59, N = 129); t(133.82) = 2.41, p = .01734, d = 0.20. That is, students who did not complete the benchmarking task performed significantly poorer overall, and achieved less consistent marks across the criteria, implying a poorer ability to calibrate against these criteria. d (or Cohen’s d) is an effect size measure representing the difference between the two group means divided by the average of their standard deviations, thus a d of 1 represents that the two groups differ by 1 SD, .5 by half an SD, etc., with .2 considered small, .5 medium, and .8 large
  6. Next phase is to look at the written comments
  7. That is, (in)accuracy in the benchmarking task and (in)accuracy in self-assessment are related, such that those who were more inaccurate in the benchmarking were also more inaccurate in their self-assessment.