Dodd - The information literacy impact factor: how to measure value

The Information Literacy Impact
factor: How to Measure Value

Lorna Dodd
College Liaison Librarian
UCD Library | Dublin 4.
email: Lorna.dodd@ucd.ie
Tel: +353 (0)1 716 7074
www.ucd.ie/library

Outline

 Why conduct a survey of module coordinators?
 Review of the Literature
 Rationale
 How the survey was conducted
 Survey results
 Implications for the future
 Conclusions

Lorna Dodd: LILAC 2013

Why?


Why?
Looking Back
&
Looking Forward


Looking Back

 Large University
 No single strategy for the development of
Information Literacy
 Heavy load of information skills delivery
 Driven by individuals
 Sometimes evaluated but rarely measured
 No clear indication in meeting learning outcomes
(Dodd, L. & Kendlin, V., 2010)


Looking Forward…

New Library Strategic Plan 2010-2014

“monitor, measure and evaluate the Library’s teaching and
learning strategy and activities.”
(UCD Library, 2010)


Looking forward

 Realignment of staffing structure
 From approx 17 Liaison Librarians to 6 College
Liaison Librarians

 Move from module to programme approach

 Move from responsive to consultation

 Measure impact on student learning


Looking Forward…


Literature Review (1)
 A lot available discussing why it is important to measure
impact
 Much of what has been written focuses on Library use rather
than instruction.
(Stone 2011; Poll & Payne 2006; Schilling & Applegate 2012).

 Limitations of traditional organisational approaches at
measuring success and meeting targets.
 Most libraries focus on process and output indicators rather
than measuring impact.
 Measuring impact on student learning requires systematic
evaluation of training
(Markless & Streatfield 2006; Schilling & Applegate 2012))


 Several problems arise when measuring impact.
 Data protection rules
 Differing value of services for specific groups
 Difficulty in measuring long-term impact
 Time consuming nature of measuring impact of Library
instruction.
 Important to use results of any study conducted to measure
the impact of Library services
 Benchmarking activities
 Improving services
 Justifying resources used for services
 Campaigning for increased funding.
 Common/Successful ways to measure impact of Library
instruction
 Surveys; Pre and post-tests; Self-assessments; Behavioural observations.

(Poll & Payne 2006)


2006 LIR/SCONUL measuring impact initiative
Developed impact process
Identified key performance indicators
Used to evaluate the effectiveness of interventions
supporting teaching and learning
Research provided information regarding emerging
issues throughout the process.
Results included:
 Improved relationships with academic community
 A raised profile for Library staff
 Better understanding of how the Library can support
academic programmes.

(Markless & Streatfield 2006; Blagden, 2005)

2012 CONUL Information Literacy Survey
Gathered feedback on the impact of information literacy
instruction
 Found library instruction had a positive impact on
students’ skill development
 Information Literacy often assessed
Targeted respondents
Academics already fully engaged
First year students only
Small sample from each institution

(CONUL Advisory Committee on Information Literacy, 2013)


Rationale

We needed to: Problems:
Make academics aware of new Measuring impact
approach retrospectively
Create a strategy to get Perceived impact is
academics to embrace new
subjective
approach
External factors influence
Identify what has been
successful to date development of information
skills
Collect feedback from those
not actively engaged with the More likely to opt for
Library options that are familiar
Identify what methods would
be useful going forward


Survey
 Seeking information on modules

 Two parts:
1. Evaluating information skills delivery to date
2. Identifying what would be useful in the future

 Survey Monkey
 11 questions
 Mainly quantitative
 Multiple choice questions
 Measurement/ranking questions


Survey
Introduction:
Level of module – undergraduate/taught postgraduate
If there has been Library instruction
Part 1:
Kind of instruction
Impact of instruction on students’ abilities
Part 2:
Should all students have library instruction
When should library instruction happen?
Which services should be developed

Free Question

Promotion

 Approx. 700 module coordinators
 Only possible to target academic staff
 Not all module coordinators are academic staff
and not all academic staff are module
coordinators
 Targeting email – approx 1,900
 College Liaison Librarian contacts
 Three weeks


Survey Results

 Type of instruction
 Impact of instruction
 When and who?
 Preferred services for the future
 Qualitative feedback


Type of Library
Instruction

Impact of Library
Instruction


Mostly Taught Postgraduate:
 EndNote
 Special Collections
 PBL
 One-on-One Consultation


“Students have always received
instruction to date and so difficult to
answer the comparison questions below”


Should ALL students have Library Instruction?


Should Library Instruction happen in every year?


Which Stage?


“Instruction in Year one should be built
on in subsequent years to enhance skills
by year”


Preferred Services for
the Future

Influence of instruction had on preference?

Instruction First Choice Second Choice Last Choice
had
Tour Subject Generic workshop Tour
Specific Tutorial
Workshops Online Video
Lecture Subject Generic Workshop Tour/Online
Specific Tutorial Video
Workshops
Workshop Subject Generic Workshop Tour
Specific Tutorial
Workshops
No Subject Subject Specific Tour
Instruction Specific Workshops
Workshops


“One of the best approaches would be to create
materials that are reusable and for any student,
regardless of level. Just because a student moves up a
level doesn't mean they've acquired all of the learning
they should have”

“…really good online video materials, from complete
intro to specific and advanced, and available on
demand is the way to go”.

“Tutorials (tutors would be trained by Library
staff)”

Qualitative Feedback

Comments Themes

 20% (53) of those who completed the survey
made comments

 In line with our new strategy
 Online options
 Constructive comments & suggestions on current
and future services
 Complimenting services to date
 Comments on the survey


Comments

 5 respondents made comments inline with our
new strategy
 Programme Approach
 Reusable Learning object and Online tools

 6 respondents made comments about proposed
future instruction


Comments
 12 respondents made comments about current
services and changes

 Problems with current service:

“…same 'introductory' talks by librarian using lecture slots of
various modules for at least three times (the third time he
skipped as he found it ridiculous)

“Students just skip the guest lecture.”


Comments

 Benefits of current service:

“…requires personal contact with a friendly face early on.

“The current reduction in library services is a real
retrograde step especially with the explosion in available
information sources”


Comments

Over half (28) complimented current strategy

“we really love our librarians; thank you”

‘Campaigning’ for traditional service
Acknowledging individuals
Recognising value of information literacy to students


“…it makes a noticeable difference
in the quality of assignment
students produce. The Library
Tutorial is typically rated as one of
the most important lectures they
have in that Stage 1 module.”


Implications for the
Future


Impact of Instruction

 Definite benefit in Library instruction
 Active learning environment is preferred
 Stronger evidence in the discovery and use
of resources
 Remains difficult to identify the ‘real’
impact


Timing of Instruction

 Most module coordinators feel all students
should have Library instruction
 Instruction for students at every stage is desirable
 Taught Postgraduates
 Beginning and middle of Semester 1

 Undergraduates
 Stage 1
 Stage 3


Preferred Services
 Active learning sessions tailored to subject needs most
preferred
 Tours perceived as ineffective

Questions:
 Asked to identify how effective instruction was at improving
students’ information literacy skills.
 When they request instruction, do they usually consider this?
 Or do they feel students need to know about the library rather
than what they need in order to transition from 2nd level, develop
researching skills etc.


Questions

Did respondents choose workshop environment over
online version because they are more familiar?
If so, then why did the majority of those who had no
instruction choose workshop?
If delivering workshops on this scale is unsustainable,
what can we do ensure effectiveness?
What work do we need to do in terms of marketing?


Conclusion

Shared understanding between Library & Academic in:
 Benefit of Instruction
 Taking a programme approach
 Ensuring all students get an equal opportunity (core
modules)

Difference in:
 Most appropriate method of delivery


References
 Blagden, P. (2005) ‘The LIRG/SCONUL Measuring Impact Initiative: Overview of
phase 1 impact projects’, Library & Information Research (LIR) vol. 29 (91)
 Dodd, L. & Kendlin, V. (2010) ‘Damned if we do and Damned if we don’t: How to
address sustainability in the delivery of information literacy components in UCD’:
Librarians' Information Literacy Annual Conference (LILAC), March 29-31, Limerick,
Ireland.
 Markless, S & Streatfield D. (2006) ‘Gathering and applying evidence of the impact of
UK university libraries on student learning and research: A facilitated action research
approach’, Journal of Information Management vol. 26 pp. 3-15
 Poll, R & Payne, P. (2006) ‘Impact measures for libraries and information services’,
Library Hi Tech vol. 24 (4) pp. 547-562
 Schilling, K. & Applegate, R. (2012) ‘Best methods for evaluating education impact: a
comparison of the efficacy of commonly used measures in library instruction’, Journal
of the Medical Library Association, vol. 100 (4) pp. 258-269
 Stone, G. et al (2011) ‘Does Library use affect student attainment? A preliminary
report on the library impact data project’ Liber Quarterly vo. 21(1) pp. 5-22
 UCD Library (2010) ‘UCD Library Strategic Plan 2010-2014’
http://www.ucd.ie/library/news_publicity/showcase_strategy /


Dodd - The information literacy impact factor: how to measure value

More Related Content

What's hot

Viewers also liked

Similar to Dodd - The information literacy impact factor: how to measure value

More from IL Group (CILIP Information Literacy Group)

Dodd - The information literacy impact factor: how to measure value

Editor's Notes