Successfully reported this slideshow.
Your SlideShare is downloading. ×

The future of feedback: Evaluation of information and digital literacy teaching in higher education - Siobhan Dunlop

Ad

The future of feedback
Evaluation of information and digital literacy
teaching in higher education
Siobhan Dunlop (they/th...

Ad

Background to the project
Part of a Postgraduate Certificate in
Academic Practice.
Work in a team that delivers digital sk...

Ad

What was the project?
Evaluation of information and digital literacy teaching in higher education
Evaluative practice in h...

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Check these out next

1 of 14 Ad
1 of 14 Ad

The future of feedback: Evaluation of information and digital literacy teaching in higher education - Siobhan Dunlop

Download to read offline

Presented at LILAC 2022

Presented at LILAC 2022

Advertisement
Advertisement

More Related Content

More from IL Group (CILIP Information Literacy Group) (20)

Advertisement

The future of feedback: Evaluation of information and digital literacy teaching in higher education - Siobhan Dunlop

  1. 1. The future of feedback Evaluation of information and digital literacy teaching in higher education Siobhan Dunlop (they/them) University of York
  2. 2. Background to the project Part of a Postgraduate Certificate in Academic Practice. Work in a team that delivers digital skills training in University of York's Library, Archives and Learning Services. Need to think beyond our current Google Form.
  3. 3. What was the project? Evaluation of information and digital literacy teaching in higher education Evaluative practice in higher education (Sanders, 2011) - not the evaluation of HE Information and digital literacy Embedded within modules or non-embedded and outside of academic courses Synchronous teaching and asynchronous teaching interventions
  4. 4. Why do the review? Examine current landscape of evaluating information and digital literacy teaching. Build on previous literature reviews by Schilling and Applegate (2012) and Erlinger (2018). Aimed to draw conclusions about how to choose methods for individual evaluative practice.
  5. 5. Methodology Searched 4 databases Screening criteria: ➔ Published post-2000 ➔ Teaching intervention - not measuring students' skills ➔ Definable methodology for evaluation Categorised full-text articles
  6. 6. Review stats Teaching intervention type Count Percentag e embedded synchronous 31 20.39% embedded asynchronous 20 13.16% non-embedded synchronous 14 9.21% non-embedded asynchronous 18 11.84% embedded drop in 1 0.66% non-specific/mixed 68 44.74% Evaluation Count Percentage single 68 44.74% multiple 84 55.26% Evaluation method Count session-specific survey 67 end of module survey 15 focus group 9 single self-assessment of skills 2 pre- and post- self-assessment of skills 15 single skills test 10 pre- and post- skills test 36 assessment results/grades 17 analysis of student work 33 student reflective writing 13 student observation 3 interviews/1:1 meetings 6 electronic resource stats 9 in-class formative assessment/feedback 9 other 24 Frequency of teaching intervention types Single or multiple evaluation methods Frequency of evaluation methods
  7. 7. Key limitations of evaluation methodologies (1) Experimental design ➔ Lack of control group Impact of voluntary attendance ➔ Don't know why students choose to come ➔ Students may come to one or many sessions - hard to evaluate as a series Collecting subjective views of students ➔ Using only surveys makes it hard to compare feedback with data on students' learning
  8. 8. Key limitations of evaluation methodologies (2) Survey issues ➔ Low response rates ➔ Unclear terminology in survey - e.g. what does "helpful" mean to each student? Time and student recruitment ➔ Getting students to take part ➔ Time for focus groups and interviews ➔ Time for keeping skills/self-assessment questions up to date
  9. 9. Review conclusions Consider "contextualised" evaluation (Nygaard and Belluigi, 2011). Use mixed methods that complement each other. Issues of longitudinal data and ongoing evaluation as technology and platforms change. Need further research into evaluation as day-to-day practice, rather than for specific research projects, where time and experimental design differ.
  10. 10. Impact on our work Re-evaluation of survey questions. Use of in-session formative assessment/self-assessment. Collaborate with academics to get more meaningful evaluation of our embedded teaching.
  11. 11. Obligatory pandemic reflection Pivoted to Google Forms from a paper form - response rates declined. Attendees more likely to leave session early. Should we evaluate online and face-to-face teaching in the same ways, or different?
  12. 12. What is the future (of feedback)? ➔ Could we move away from the survey? Do we want to? ➔ Does there need to be a wider move in higher education away from the survey? ➔ What is the purpose of feedback going forward? Can that change?
  13. 13. References Erlinger, A. (2018). Outcomes assessment in undergraduate information literacy instruction: a systematic review. College and research libraries, 79(4), 442-479. Nygaard, C. and Belluigi, D.Z. (2011). A proposed methodology for contextualised evaluation in higher education. Assessment and Evaluation in Higher Education, 36(6), 657-671. Sanders, M. (2011). Setting the scene: the four domains of evaluative practice in higher education. In Sanders, M., Trowler, P. and Bamber, V. (Eds.), Reconceptualising evaluation in higher education: the practice turn. Berkshire: McGraw Hill, pp. 1-17. Schilling, K. and Applegate, R. (2012). Best methods for evaluating educational impact: a comparison of the efficacy of commonly used measures of library instruction. Journal of the Medical Library Association, 100(4), 258-269.
  14. 14. Siobhan Dunlop Teaching and Learning Advisor University of York Email: siobhan.dunlop@york.ac.uk

Editor's Notes

  • Part of a Postgraduate Certificate in Academic Practice.
    team approach to feedback
    pandemic
    Difficulty of getting ethics approval to actually get feedback…

  • Define terms 'evaluation', types of teaching
    Aims of evaluation
  • Previous lit reviews
    Interest in methods we might use going forward
  • Search - ERIC, British Education Index, Scopus, Web of Science
    Post 2000, relates to teaching invention, not measuring students' skills, has definable methodology for evaluating a teaching intervention
    Categorisation - types of teaching intervention,, details about evaluation methodologiies and interventions - then afterwards, sorted into single and multiple methods and methods themselves categorised
  • Contextualised evaluation as per Nygaard and Belluigi
    Quantitative methods benefit from control groups, which isn't practical for day to day teaching evaluation
    Skills tests lack holistic detail
    Keep test questions up to date and longitudinal data - need to include generic questions?
    Mixed methods - e.g. combining surveys and self assessments with something measuring student performance like skills tests or course grades. Or combine surveys and reflective writing with focus groups or interviews.

  • We use the same qs - good for longitudinal data, but not contextualised - combine with in session formative/self assessment of skills?
    Collaborative more with academics in terms of embedded teaching and doing more than end of module survey feedback

  • Survey response rates - we used to have a paper one, now we use Google Forms and we have a much lower response rate
    Do we need another method too? What else could we do?

  • Always going to need feedback, both for improving teaching and quantifying/qualifying the usefulness of information and digital literacy teaching (budget etc)
    Moving away from the survey? Can we? Does it need to be part of a larger move in HE away from end of module surveys?
    Or should we move away from data and move towards more formative feedback that is specifically focused on improving teaching? Can we?

×