Measuring the impact of information literacy instruction: A starting point for moving forward
1. Measuring the impact of
information literacy instruction:
A starting point for moving
forward
Lorna Dodd
User Services Manager, UCD Library
Lorna.dodd@ucd.ie
Twitter:@LornaDodd
4. Large University with no single strategy for
the development of Information
Literacy
Deliver a heavy load of information skills,
driven by individuals
Sometimes we evaluate but we rarely measure.
with no clear indication in meeting learning
outcomes
Looking back
5. Looking Forward
New Library Strategic Plan 2010-2014 aims to
“monitor, measure and evaluate the Library’s teaching and learning
strategy and activities.” (UCD Library, 2010)
As a result there has been a realignment
of staffing structure.
We have shifted from module to programme
approach
And from a responsive to consultative role
A key goal is to measure our impact on student
learning
6. Focus on process &
outputs
Limitations of traditional
approach in measuring
success
Important to use results of
any study which measures
the impact of Library
Service
Focus on Library use rather
than instruction
Common tools include
surveys, behavioural
observation and pre/post
tests
Literature Review
8. Promote new service
Identify what has been successful
Gather feedback from those not
engaged
Identify useful methods going
forward
Measuring impact retrospectively
Perceived impact is subjective
External factors influence skills
development
Opt for the familiar
32. “…create materials that are reusable and for
any student, regardless of level.
Just because a student moves up a level
doesn't mean they've acquired all of the
learning they should have”
“…really good online video
materials, available
on demand is the way
to go”.
35. “…same 'introductory' talks
by librarian using lecture slots of
various modules
for at least three times
(the third time he skipped
as he found it
ridiculous)
“Students just skip the
guest lecture.”
37. “…it makes a noticeable difference in the
quality of assignment students produce. The
Library Tutorial is typically rated as one of the
most important lectures they have in that
Stage 1 module.”
40. Active learning
tailored to specific
subject needs
Tours perceived as
ineffective
Preferred delivery
of instruction
41. Asked to identify how
effective instruction was
at improving students’
information literacy
skills.
When they request
instruction, do they
usually consider this?
Questions
42. More Questions
Did respondents choose workshop
environment over online version due to
familiarity?
Then why did those who had no
instruction choose workshop?
If delivering workshops on this scale is
unsustainable,
What can we do ensure effectiveness?
What work do we need to do in terms
of marketing?
43. We have a shared understanding…
…how do we move forward?
Conclusion
44. Blagden, P. (2005) ‘The LIRG/SCONUL Measuring Impact Initiative: Overview of phase 1
impact projects’, Library & Information Research (LIR) vol. 29 (91)
Dodd, L. & Kendlin, V. (2010) ‘Damned if we do and Damned if we don’t: How to address
sustainability in the delivery of information literacy components in UCD’: Librarians'
Information Literacy Annual Conference (LILAC), March 29-31, Limerick, Ireland.
Markless, S & Streatfield D. (2006) ‘Gathering and applying evidence of the impact of UK
university libraries on student learning and research: A facilitated action research
approach’, Journal of Information Management vol. 26 pp. 3-15
Poll, R & Payne, P. (2006) ‘Impact measures for libraries and information services’,
Library Hi Tech vol. 24 (4) pp. 547-562
Schilling, K. & Applegate, R. (2012) ‘Best methods for evaluating education impact: a
comparison of the efficacy of commonly used measures in library instruction’, Journal
of the Medical Library Association, vol. 100 (4) pp. 258-269
Stone, G. et al (2011) ‘Does Library use affect student attainment? A preliminary report
on the library impact data project’ Liber Quarterly vo. 21(1) pp. 5-22
UCD Library (2010) ‘UCD Library Strategic Plan 2010-2014’
http://www.ucd.ie/library/news_publicity/showcase_strategy/
Lorna Dodd: ANLTC 2013
References
45. Off the starting blocks http://www.flickr.com/photos/nickuzma/2570878552/
why can't i breathe whenever i think about you? http://www.flickr.com/photos/breannajosephine/5098047755/
Looking Into the Past - Custom House http://www.flickr.com/photos/coreytempleton/5893927056/
To go to http://www.flickr.com/photos/rubensmj/5399226001/
Literature review http://www.flickr.com/photos/carowallis1/4297460415/
Nissan Qashqai Challenge London http://www.flickr.com/photos/parmiter/2521273890/
Playing chess http://www.flickr.com/photos/bburt33/5930699148/
Surveying for Central Station, 1900 http://www.flickr.com/photos/state-records-nsw/4010230662
Shouting kid http://www.flickr.com/photos/jamessarmiento/4353936770/
Results of elections http://www.flickr.com/photos/ntr23/4650249185/
Four stages of the moon http://www.flickr.com/photos/leehaywood/6607585765/
Meteor crater http://www.flickr.com/photos/homer4k/113472442/
Writing http://www.flickr.com/photos/jjpacres/3293117576/
Used books http://www.flickr.com/photos/babblingdweeb/23816128/
Quality Barnstorm http://commons.wikimedia.org/wiki/File:Quality_Barnstar_v2.0.png
Apples & Oranges - They Don't Compare http://www.flickr.com/photos/thebusybrain/2492945625/
Lorna Dodd: ANLTC 2013
Images
46. Watermelon http://www.dreamstime.com/stock-photography-watermelon-image22521832
Orange http://www.dreamstime.com/royalty-free-stock-photo-orange-image23480475
Strawberry on white background http://www.dreamstime.com/royalty-free-stock-photo-one-strawberry-
white-background-image28694065
One gooseberry fruits isolated http://www.dreamstime.com/stock-photography-one-gooseberry-fruits-
isolated-image20607952
Construction Tunnel http://www.flickr.com/photos/joshuarothhaas/2250657031 Online
http://www.flickr.com/photos/orkomedix/5398470372/
Conversation http://www.flickr.com/photos/stijnnieuwendijk/3098445189/
Backyard fresh http://www.flickr.com/photos/jim_belford/8544552591/
Abandoned amphitheatre http://www.flickr.com/photos/pixarman/5456444943/
Lego love http://www.flickr.com/photos/evaysucamara/5419530024/
Student working http://www.flickr.com/photos/92399131@N07/8399835572/
Future http://www.flickr.com/photos/maniya/4020026753/
Punch http://www.flickr.com/photos/timcaynes/148110505/
teach me http://www.flickr.com/photos/giovannamatarazzo/182108786
Question proposal http://www.flickr.com/photos/eleaf/2536358399/
Light Bulb http://www.flickr.com/photos/polifemus/561986434/
walk ahead and cross the sky http://www.flickr.com/photos/ldawei/4920681478/
Lorna Dodd: ANLTC 2013
Images
Editor's Notes
Engage further with academic community
Using results:
Benchmarking activities, Improving services, Justifying resources used for services, Campaigning for increased funding.
Seeking information on modules
Two parts: 1. Evaluating information skills delivery to date. 2. Identifying what would be useful in the future
Survey Monkey: 11 questions; Mainly quantitative; MCQs; Measurement/ranking questions
Introduction:
Level of module – undergraduate/taught postgraduate
If there has been Library instruction
Part 1:
Kind of instruction
Impact of instruction on students’ abilities
Part 2:
Should all students have library instruction
When should library instruction happen?
Which services should be developed
Free Question
Approx. 700 module coordinators
Only possible to target academic staff
Not all module coordinators are academic staff and not all academic staff are module coordinators
Targeting email – approx 1,900
College Liaison Librarian contacts
Three weeks
Tours good for using better & broader range resources of improved ability at finding information
Significant proportion feel tours are not good for citing and plagiarism awareness
Disagreement regarding students being better at everything
Feel Taught postgrads get more out of tour in all aspects
Significant difference to UGs is that no disagreement in all aspects except plagiarism
Similar to tours at good:
Lectures good for better & broader range of resources & improved ability at finding information
Less disagree that students more aware of citing & plagiarism
Disagreement regarding students being better at everything
Significantly better in all aspects
Most noticeable is that very few disagree with any
Stage 2 significantly more positive
Bespoke sessions – mostly T.PGs (EndNote, PBL, special collections)
Specifically tailored – much better at achieving all categories
Watermelon http://www.dreamstime.com/stock-photography-watermelon-image22521832
Orange http://www.dreamstime.com/royalty-free-stock-photo-orange-image23480475
Strawberry on white background http://www.dreamstime.com/royalty-free-stock-photo-one-strawberry-white-background-image28694065
Strawberry on white background http://www.dreamstime.com/royalty-free-stock-photo-one-strawberry-white-background-image28694065
In line with our new strategy
Online options
Constructive comments & suggestions on current and future services
Complimenting services to date
Comments on the survey
5 respondents made comments inline with our new strategy
Programme Approach
Reusable Learning object and Online tools
6 respondents made comments about proposed future instruction
12 respondents made comments about current services and changes
Benefit of current services. Fear of loosing these services
Over half (28) complimented current strategy
Campaigning’ for traditional service
Acknowledging individuals
Recognising value of information literacy to students
Shared understanding between library & academics in:
Benefit of instruction
Programme approach
Ensuring all students get same opportunity
Difference in methodology
Fear of loosing these services
Fear of loosing these services
Fear of loosing these services
Helping little hands http://www.flickr.com/photos/barnabywasson/301538814/