Measuring the impact ofinformation literacy instruction:A starting point for movingforwardLorna DoddUser Services Manager,...
Why?…snapshot of impact to date & bestpath forward
Looking back
Large University with no single strategy forthe development of InformationLiteracyDeliver a heavy load of information skil...
Looking ForwardNew Library Strategic Plan 2010-2014 aims to“monitor, measure and evaluate the Library’s teaching and learn...
Focus on process &outputsLimitations of traditionalapproach in measuringsuccessImportant to use results ofany study which ...
ChallengesData protection rulesMeasuring long-term impactIt takes time to measure impact of LibraryInstruction
Promote new serviceIdentify what has been successfulGather feedback from those notengagedIdentify useful methods goingforw...
Survey all module coordinators
Promotion
SurveyResults
Yes 43%No 35%Don’t know22%Has the Library ever delivered instruction in your module?
35%Lecture33%Workshop25%Tour7%OtherWhat kind of instruction did your students have?
Impact of Library Instruction
“Students have alwaysreceived instruction todate and so difficult to answer thecomparison questions below”
Should ALL students have Library Instruction?Yes86% Don’tKnow9%No5%
Should ALL students have Library Instruction?Yes96% Don’tKnow3%No2%Those who had Library Instruction
Should ALL students have Library Instruction?Yes77% N014%Don’tKnow10%Those who did NOT have Library Instruction
Should Library Instruction happen in every year?
“Instruction in Yearone should bebuilt on insubsequent years toenhanceskills byyear”
Preferred Services
120Subjectspecificworkshops62LibraryTours bypeermentors58Genericworkshops50Onlinetutorials48Onlineinstructionalvideos45Tut...
66Genericworkshops52Tutorials50Onlineinstructionalvideos45Onlinetutorials42Subjectspecificworkshops36Librarytours withpeer...
First choice for taught postgraduates154Subjectspecificworkshops62LibraryTours bypeermentors57Genericworkshops53Onlineinst...
Influence of instructionon preference?
Influence of instructionon preference?
Influence of instructionon preference?
“…create materials that are reusable and forany student, regardless of level.Just because a student moves up a leveldoesnt...
Qualitative Feedback 20% of respondents
ThemesIn linewithstrategyOnlineOptionsCurrent &FutureServicesCommentson serviceCommentson survey
“…same introductory talksby librarian using lecture slots ofvarious modulesfor at least three times(the third time he skip...
“…requires personalcontact with a friendlyface early on.“we really love our librarians; thank you”
“…it makes a noticeable difference in thequality of assignment students produce. TheLibrary Tutorial is typically rated as...
Implications forthe Future
Impact ofInstructionDefinite benefitin LibraryinstructionStrongerevidence indiscovery anduse of resources
Active learningtailored to specificsubject needsTours perceived asineffectivePreferred deliveryof instruction
Asked to identify howeffective instruction wasat improving students’information literacyskills.When they requestinstructio...
More QuestionsDid respondents choose workshopenvironment over online version due tofamiliarity?Then why did those who had ...
We have a shared understanding……how do we move forward?ConclusionLorna.dodd@ucd.ie@LornaDodd
Blagden, P. (2005) ‘The LIRG/SCONUL Measuring Impact Initiative: Overview of phase 1impact projects’, Library & Informatio...
Off the starting blocks http://www.flickr.com/photos/nickuzma/2570878552/why cant i breathe whenever i think about you? ht...
Watermelon http://www.dreamstime.com/stock-photography-watermelon-image22521832Orange http://www.dreamstime.com/royalty-fr...
Measuring the Impact of Information Literacy Instruction: A Starting Point for Moving Forward
Measuring the Impact of Information Literacy Instruction: A Starting Point for Moving Forward
Measuring the Impact of Information Literacy Instruction: A Starting Point for Moving Forward
Measuring the Impact of Information Literacy Instruction: A Starting Point for Moving Forward
Upcoming SlideShare
Loading in …5
×

Measuring the Impact of Information Literacy Instruction: A Starting Point for Moving Forward

620 views

Published on

Presentation made by Lorna Dodd, User Services Manager, University College Dublin Library, at ANLTC Seminar "Library Impact and Assessment", held on Tuesday, 7th May 2013 at Trinity College Dublin Library.

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
620
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Engage further with academic community
  • Using results: Benchmarking activities, Improving services, Justifying resources used for services, Campaigning for increased funding.
  • Seeking information on modules Two parts: 1. Evaluating information skills delivery to date. 2. Identifying what would be useful in the future Survey Monkey: 11 questions; Mainly quantitative; MCQs; Measurement/ranking questions Introduction: Level of module – undergraduate/taught postgraduate If there has been Library instruction Part 1: Kind of instruction Impact of instruction on students’ abilities Part 2: Should all students have library instruction When should library instruction happen? Which services should be developed Free Question
  • Approx. 700 module coordinators Only possible to target academic staff Not all module coordinators are academic staff and not all academic staff are module coordinators Targeting email – approx 1,900 College Liaison Librarian contacts Three weeks
  • Tours good for using better & broader range resources of improved ability at finding information Significant proportion feel tours are not good for citing and plagiarism awareness Disagreement regarding students being better at everything Feel Taught postgrads get more out of tour in all aspects Significant difference to UGs is that no disagreement in all aspects except plagiarism
  • Similar to tours at good: Lectures good for better & broader range of resources & improved ability at finding information Less disagree that students more aware of citing & plagiarism Disagreement regarding students being better at everything
  • Significantly better in all aspects Most noticeable is that very few disagree with any Stage 2 significantly more positive
  • Bespoke sessions – mostly T.PGs (EndNote, PBL, special collections) Specifically tailored – much better at achieving all categories
  • Watermelon http://www.dreamstime.com/stock-photography-watermelon-image22521832 Orange http://www.dreamstime.com/royalty-free-stock-photo-orange-image23480475 Strawberry on white background http://www.dreamstime.com/royalty-free-stock-photo-one-strawberry-white-background-image28694065
  • Strawberry on white background http://www.dreamstime.com/royalty-free-stock-photo-one-strawberry-white-background-image28694065
  • Choosing vinyl http://www.flickr.com/photos/sblackley/1128813156/sizes/o/in/photostream/
  • In line with our new strategy Online options Constructive comments & suggestions on current and future services Complimenting services to date Comments on the survey 5 respondents made comments inline with our new strategy Programme Approach Reusable Learning object and Online tools 6 respondents made comments about proposed future instruction 12 respondents made comments about current services and changes
  • Benefit of current services. Fear of loosing these services Over half (28) complimented current strategy Campaigning’ for traditional service Acknowledging individuals Recognising value of information literacy to students
  • Remains difficult to identify ‘real’ impact
  • http://www.flickr.com/photos/davidyuweb/4446734924/
  • Shared understanding between library & academics in: Benefit of instruction Programme approach Ensuring all students get same opportunity Difference in methodology
  • Fear of loosing these services
  • Fear of loosing these services
  • Fear of loosing these services
  • Measuring the Impact of Information Literacy Instruction: A Starting Point for Moving Forward

    1. 1. Measuring the impact ofinformation literacy instruction:A starting point for movingforwardLorna DoddUser Services Manager, UCD LibraryLorna.dodd@ucd.ieTwitter:@LornaDodd
    2. 2. Why?…snapshot of impact to date & bestpath forward
    3. 3. Looking back
    4. 4. Large University with no single strategy forthe development of InformationLiteracyDeliver a heavy load of information skills,driven by individualsSometimes we evaluate but we rarely measure.with no clear indication in meeting learningoutcomesLooking back
    5. 5. Looking ForwardNew Library Strategic Plan 2010-2014 aims to“monitor, measure and evaluate the Library’s teaching and learningstrategy and activities.” (UCD Library, 2010)As a result there has been a realignmentof staffing structure.We have shifted from module to programmeapproachAnd from a responsive to consultative roleA key goal is to measure our impact on studentlearning
    6. 6. Focus on process &outputsLimitations of traditionalapproach in measuringsuccessImportant to use results ofany study which measuresthe impact of LibraryServiceFocus on Library use ratherthan instructionCommon tools includesurveys, behaviouralobservation and pre/posttestsLiterature Review
    7. 7. ChallengesData protection rulesMeasuring long-term impactIt takes time to measure impact of LibraryInstruction
    8. 8. Promote new serviceIdentify what has been successfulGather feedback from those notengagedIdentify useful methods goingforwardMeasuring impact retrospectivelyPerceived impact is subjectiveExternal factors influence skillsdevelopmentOpt for the familiar
    9. 9. Survey all module coordinators
    10. 10. Promotion
    11. 11. SurveyResults
    12. 12. Yes 43%No 35%Don’t know22%Has the Library ever delivered instruction in your module?
    13. 13. 35%Lecture33%Workshop25%Tour7%OtherWhat kind of instruction did your students have?
    14. 14. Impact of Library Instruction
    15. 15. “Students have alwaysreceived instruction todate and so difficult to answer thecomparison questions below”
    16. 16. Should ALL students have Library Instruction?Yes86% Don’tKnow9%No5%
    17. 17. Should ALL students have Library Instruction?Yes96% Don’tKnow3%No2%Those who had Library Instruction
    18. 18. Should ALL students have Library Instruction?Yes77% N014%Don’tKnow10%Those who did NOT have Library Instruction
    19. 19. Should Library Instruction happen in every year?
    20. 20. “Instruction in Yearone should bebuilt on insubsequent years toenhanceskills byyear”
    21. 21. Preferred Services
    22. 22. 120Subjectspecificworkshops62LibraryTours bypeermentors58Genericworkshops50Onlinetutorials48Onlineinstructionalvideos45TutorialsFirst choice for undergraduates
    23. 23. 66Genericworkshops52Tutorials50Onlineinstructionalvideos45Onlinetutorials42Subjectspecificworkshops36Librarytours withpeermentorsSecond choice for undergraduates
    24. 24. First choice for taught postgraduates154Subjectspecificworkshops62LibraryTours bypeermentors57Genericworkshops53Onlineinstructionalvideos51Onlinetutorials
    25. 25. Influence of instructionon preference?
    26. 26. Influence of instructionon preference?
    27. 27. Influence of instructionon preference?
    28. 28. “…create materials that are reusable and forany student, regardless of level.Just because a student moves up a leveldoesnt mean theyve acquired all of thelearning they should have”“…really good online videomaterials, availableon demand is the wayto go”.
    29. 29. Qualitative Feedback 20% of respondents
    30. 30. ThemesIn linewithstrategyOnlineOptionsCurrent &FutureServicesCommentson serviceCommentson survey
    31. 31. “…same introductory talksby librarian using lecture slots ofvarious modulesfor at least three times(the third time he skippedas he found itridiculous)“Students just skip theguest lecture.”
    32. 32. “…requires personalcontact with a friendlyface early on.“we really love our librarians; thank you”
    33. 33. “…it makes a noticeable difference in thequality of assignment students produce. TheLibrary Tutorial is typically rated as one of themost important lectures they have in thatStage 1 module.”
    34. 34. Implications forthe Future
    35. 35. Impact ofInstructionDefinite benefitin LibraryinstructionStrongerevidence indiscovery anduse of resources
    36. 36. Active learningtailored to specificsubject needsTours perceived asineffectivePreferred deliveryof instruction
    37. 37. Asked to identify howeffective instruction wasat improving students’information literacyskills.When they requestinstruction, do theyusually consider this?Questions
    38. 38. More QuestionsDid respondents choose workshopenvironment over online version due tofamiliarity?Then why did those who had noinstruction choose workshop?If delivering workshops on this scale isunsustainable,What can we do ensure effectiveness?What work do we need to do in termsof marketing?
    39. 39. We have a shared understanding……how do we move forward?ConclusionLorna.dodd@ucd.ie@LornaDodd
    40. 40. Blagden, P. (2005) ‘The LIRG/SCONUL Measuring Impact Initiative: Overview of phase 1impact projects’, Library & Information Research (LIR) vol. 29 (91)Dodd, L. & Kendlin, V. (2010) ‘Damned if we do and Damned if we don’t: How to addresssustainability in the delivery of information literacy components in UCD’: LibrariansInformation Literacy Annual Conference (LILAC), March 29-31, Limerick, Ireland.Markless, S & Streatfield D. (2006) ‘Gathering and applying evidence of the impact of UKuniversity libraries on student learning and research: A facilitated action researchapproach’, Journal of Information Management vol. 26 pp. 3-15Poll, R & Payne, P. (2006) ‘Impact measures for libraries and information services’,Library Hi Tech vol. 24 (4) pp. 547-562Schilling, K. & Applegate, R. (2012) ‘Best methods for evaluating education impact: acomparison of the efficacy of commonly used measures in library instruction’, Journalof the Medical Library Association, vol. 100 (4) pp. 258-269Stone, G. et al (2011) ‘Does Library use affect student attainment? A preliminary reporton the library impact data project’ Liber Quarterly vo. 21(1) pp. 5-22UCD Library (2010) ‘UCD Library Strategic Plan 2010-2014’http://www.ucd.ie/library/news_publicity/showcase_strategy/Lorna Dodd: ANLTC 2013References
    41. 41. Off the starting blocks http://www.flickr.com/photos/nickuzma/2570878552/why cant i breathe whenever i think about you? http://www.flickr.com/photos/breannajosephine/5098047755/Looking Into the Past - Custom House http://www.flickr.com/photos/coreytempleton/5893927056/To go to http://www.flickr.com/photos/rubensmj/5399226001/Literature review http://www.flickr.com/photos/carowallis1/4297460415/Nissan Qashqai Challenge London http://www.flickr.com/photos/parmiter/2521273890/Playing chess http://www.flickr.com/photos/bburt33/5930699148/Surveying for Central Station, 1900 http://www.flickr.com/photos/state-records-nsw/4010230662Shouting kid http://www.flickr.com/photos/jamessarmiento/4353936770/Results of elections http://www.flickr.com/photos/ntr23/4650249185/Four stages of the moon http://www.flickr.com/photos/leehaywood/6607585765/Meteor crater http://www.flickr.com/photos/homer4k/113472442/Writing http://www.flickr.com/photos/jjpacres/3293117576/Used books http://www.flickr.com/photos/babblingdweeb/23816128/Quality Barnstorm http://commons.wikimedia.org/wiki/File:Quality_Barnstar_v2.0.pngApples & Oranges - They Dont Compare http://www.flickr.com/photos/thebusybrain/2492945625/Lorna Dodd: ANLTC 2013Images
    42. 42. Watermelon http://www.dreamstime.com/stock-photography-watermelon-image22521832Orange http://www.dreamstime.com/royalty-free-stock-photo-orange-image23480475Strawberry on white background http://www.dreamstime.com/royalty-free-stock-photo-one-strawberry-white-background-image28694065One gooseberry fruits isolated http://www.dreamstime.com/stock-photography-one-gooseberry-fruits-isolated-image20607952Construction Tunnel http://www.flickr.com/photos/joshuarothhaas/2250657031 Onlinehttp://www.flickr.com/photos/orkomedix/5398470372/Conversation http://www.flickr.com/photos/stijnnieuwendijk/3098445189/Backyard fresh http://www.flickr.com/photos/jim_belford/8544552591/Abandoned amphitheatre http://www.flickr.com/photos/pixarman/5456444943/Lego love http://www.flickr.com/photos/evaysucamara/5419530024/Student working http://www.flickr.com/photos/92399131@N07/8399835572/Future http://www.flickr.com/photos/maniya/4020026753/Punch http://www.flickr.com/photos/timcaynes/148110505/teach me http://www.flickr.com/photos/giovannamatarazzo/182108786Question proposal http://www.flickr.com/photos/eleaf/2536358399/Light Bulb http://www.flickr.com/photos/polifemus/561986434/walk ahead and cross the sky http://www.flickr.com/photos/ldawei/4920681478/Lorna Dodd: ANLTC 2013Images

    ×