Measuring the impact of information literacy instruction: A starting point for moving forward


Published on

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Engage further with academic community
  • Using results:
    Benchmarking activities, Improving services, Justifying resources used for services, Campaigning for increased funding.
  • Seeking information on modules
    Two parts: 1. Evaluating information skills delivery to date. 2. Identifying what would be useful in the future
    Survey Monkey: 11 questions; Mainly quantitative; MCQs; Measurement/ranking questions
    Level of module – undergraduate/taught postgraduate
    If there has been Library instruction
    Part 1:
    Kind of instruction
    Impact of instruction on students’ abilities
    Part 2:
    Should all students have library instruction
    When should library instruction happen?
    Which services should be developed
    Free Question
  • Approx. 700 module coordinators
    Only possible to target academic staff
    Not all module coordinators are academic staff and not all academic staff are module coordinators
    Targeting email – approx 1,900
    College Liaison Librarian contacts
    Three weeks
  • Tours good for using better & broader range resources of improved ability at finding information
    Significant proportion feel tours are not good for citing and plagiarism awareness
    Disagreement regarding students being better at everything
    Feel Taught postgrads get more out of tour in all aspects
    Significant difference to UGs is that no disagreement in all aspects except plagiarism
  • Similar to tours at good:
    Lectures good for better & broader range of resources & improved ability at finding information
    Less disagree that students more aware of citing & plagiarism
    Disagreement regarding students being better at everything
  • Significantly better in all aspects
    Most noticeable is that very few disagree with any
    Stage 2 significantly more positive
  • Bespoke sessions – mostly T.PGs (EndNote, PBL, special collections)
    Specifically tailored – much better at achieving all categories
  • Watermelon
    Strawberry on white background
  • Strawberry on white background
  • Choosing vinyl
  • In line with our new strategy
    Online options
    Constructive comments & suggestions on current and future services
    Complimenting services to date
    Comments on the survey
    5 respondents made comments inline with our new strategy
    Programme Approach
    Reusable Learning object and Online tools
    6 respondents made comments about proposed future instruction
    12 respondents made comments about current services and changes
  • Benefit of current services. Fear of loosing these services
    Over half (28) complimented current strategy
    Campaigning’ for traditional service
    Acknowledging individuals
    Recognising value of information literacy to students
  • Remains difficult to identify ‘real’ impact
  • Shared understanding between library & academics in:
    Benefit of instruction
    Programme approach
    Ensuring all students get same opportunity
    Difference in methodology
  • Fear of loosing these services
  • Fear of loosing these services
  • Fear of loosing these services
  • Helping little hands
  • Teaching
  • Measuring the impact of information literacy instruction: A starting point for moving forward

    1. 1. Measuring the impact of information literacy instruction: A starting point for moving forward Lorna Dodd User Services Manager, UCD Library Twitter:@LornaDodd
    2. 2. Why? …snapshot of impact to date & best path forward
    3. 3. Looking back
    4. 4. Large University with no single strategy for the development of Information Literacy Deliver a heavy load of information skills, driven by individuals Sometimes we evaluate but we rarely measure. with no clear indication in meeting learning outcomes Looking back
    5. 5. Looking Forward New Library Strategic Plan 2010-2014 aims to “monitor, measure and evaluate the Library’s teaching and learning strategy and activities.” (UCD Library, 2010) As a result there has been a realignment of staffing structure. We have shifted from module to programme approach And from a responsive to consultative role A key goal is to measure our impact on student learning
    6. 6. Focus on process & outputs Limitations of traditional approach in measuring success Important to use results of any study which measures the impact of Library Service Focus on Library use rather than instruction Common tools include surveys, behavioural observation and pre/post tests Literature Review
    7. 7. Challenges Data protection rules Measuring long-term impact It takes time to measure impact of Library Instruction
    8. 8. Promote new service Identify what has been successful Gather feedback from those not engaged Identify useful methods going forward Measuring impact retrospectively Perceived impact is subjective External factors influence skills development Opt for the familiar
    9. 9. Survey all module coordinators
    10. 10. Promotion
    11. 11. Survey Results
    12. 12. Yes 43% No 35% Don’t know 22% Has the Library ever delivered instruction in your module?
    13. 13. 35% Lecture 33% Workshop 25% Tour 7% Other What kind of instruction did your students have?
    14. 14. Impact of Library Instruction
    15. 15. “Students have always received instruction to date and so difficult to answer the comparison questions below”
    16. 16. Should ALL students have Library Instruction? Yes 86% Don’t Know 9% No 5%
    17. 17. Should ALL students have Library Instruction? Yes 96% Don’t Know 3% No 2% Those who had Library Instruction
    18. 18. Should ALL students have Library Instruction? Yes 77% N0 14% Don’t Know 10% Those who did NOT have Library Instruction
    19. 19. Should Library Instruction happen in every year?
    20. 20. “Instruction in Year one should be built on in subsequent years to enhance skills by year”
    21. 21. Preferred Services
    22. 22. 120 Subject specific workshops 62 Library Tours by peer mentors 58 Generic workshops 50 Online tutorials 48 Online instructional videos 45 Tutorials First choice for undergraduates
    23. 23. 66 Generic workshops 52 Tutorials 50 Online instructional videos 45 Online tutorials 42 Subject specific workshops 36 Library tours with peer mentors Second choice for undergraduates
    24. 24. First choice for taught postgraduates 154 Subject specific workshops 62 Library Tours by peer mentors 57 Generic workshops 53 Online instructional videos 51 Online tutorials
    25. 25. Influence of instruction on preference?
    26. 26. Influence of instruction on preference?
    27. 27. Influence of instruction on preference?
    28. 28. “…create materials that are reusable and for any student, regardless of level. Just because a student moves up a level doesn't mean they've acquired all of the learning they should have” “…really good online video materials, available on demand is the way to go”.
    29. 29. Qualitative Feedback 20% of respondents
    30. 30. Themes In line with strategy Online Options Current & Future Services Comments on service Comments on survey
    31. 31. “…same 'introductory' talks by librarian using lecture slots of various modules for at least three times (the third time he skipped as he found it ridiculous) “Students just skip the guest lecture.”
    32. 32. “…requires personal contact with a friendly face early on. “we really love our librarians; thank you”
    33. 33. “…it makes a noticeable difference in the quality of assignment students produce. The Library Tutorial is typically rated as one of the most important lectures they have in that Stage 1 module.”
    34. 34. Implications for the Future
    35. 35. Impact of Instruction Definite benefit in Library instruction Stronger evidence in discovery and use of resources
    36. 36. Active learning tailored to specific subject needs Tours perceived as ineffective Preferred delivery of instruction
    37. 37. Asked to identify how effective instruction was at improving students’ information literacy skills. When they request instruction, do they usually consider this? Questions
    38. 38. More Questions Did respondents choose workshop environment over online version due to familiarity? Then why did those who had no instruction choose workshop? If delivering workshops on this scale is unsustainable, What can we do ensure effectiveness? What work do we need to do in terms of marketing?
    39. 39. We have a shared understanding… …how do we move forward? Conclusion
    40. 40. Blagden, P. (2005) ‘The LIRG/SCONUL Measuring Impact Initiative: Overview of phase 1 impact projects’, Library & Information Research (LIR) vol. 29 (91) Dodd, L. & Kendlin, V. (2010) ‘Damned if we do and Damned if we don’t: How to address sustainability in the delivery of information literacy components in UCD’: Librarians' Information Literacy Annual Conference (LILAC), March 29-31, Limerick, Ireland. Markless, S & Streatfield D. (2006) ‘Gathering and applying evidence of the impact of UK university libraries on student learning and research: A facilitated action research approach’, Journal of Information Management vol. 26 pp. 3-15 Poll, R & Payne, P. (2006) ‘Impact measures for libraries and information services’, Library Hi Tech vol. 24 (4) pp. 547-562 Schilling, K. & Applegate, R. (2012) ‘Best methods for evaluating education impact: a comparison of the efficacy of commonly used measures in library instruction’, Journal of the Medical Library Association, vol. 100 (4) pp. 258-269 Stone, G. et al (2011) ‘Does Library use affect student attainment? A preliminary report on the library impact data project’ Liber Quarterly vo. 21(1) pp. 5-22 UCD Library (2010) ‘UCD Library Strategic Plan 2010-2014’ Lorna Dodd: ANLTC 2013 References
    41. 41. Off the starting blocks why can't i breathe whenever i think about you? Looking Into the Past - Custom House To go to Literature review Nissan Qashqai Challenge London Playing chess Surveying for Central Station, 1900 Shouting kid Results of elections Four stages of the moon Meteor crater Writing Used books Quality Barnstorm Apples & Oranges - They Don't Compare Lorna Dodd: ANLTC 2013 Images
    42. 42. Watermelon Orange Strawberry on white background white-background-image28694065 One gooseberry fruits isolated isolated-image20607952 Construction Tunnel Online Conversation Backyard fresh Abandoned amphitheatre Lego love Student working Future Punch teach me Question proposal Light Bulb walk ahead and cross the sky Lorna Dodd: ANLTC 2013 Images