What does that mean? Well we’re going to talk about how we used an evidence based method to review our spend on resources and identify 10% of our journal resources to cancel to based on budget limitations.
WE knew that we needed this information urgently and that it had to be a transparent, accountable and evidence based system if we were to convince schools of the need for the review.
http://www.flickr.com/photos/usnationalarchives/4266495658/sizes/l/ I’m going to be looking at three things: an outline of the evidence based approach that we’ve taken Some of the ways data was collected and used in the past and the changes that prompted the newer approach What we did to effect the move to an evidence based practice
Asking questions, finding information, using information in a practical fashion Evidence tempered by user feedback, applied towards decisions Make it clear that the idea of evidence-based librarianship was only applied retrospectively as a framework to make similar projects in the future more results-based (?) Also that a confluence of various events – the budgetary reviews mentioned by Eoin, the review exercises prompted by the review, and the roll-out of the MIS system to staff – meant that each event was seen in a slightly different context. It was only in the course of doing reviews that we realised there was not only evidence available to shape the content of the reviews but also a systematic way to extract the evidence. This in turn led staff to consider other forms of available evidence.
Addresses areas of information overload and delay – focused towards making the information actionable
Quantitative Data Collected as part of normal business functions: MIS Reporting, Vendor Statistics, Website Usage Qualitative Data Collected for other projects: Survey Data Other Evidentiary Data: Library Gates, ILL requests, etc.
In order to more effectively use the data (in the ways that Valerie will describe) a change in culture was necessary Data was previously collected and used in such a way that it didn’t provide the best support for the evidence-based approach Assumption that staff responsible for managing and extracting data are best judges of potential use of data…right people not seeing right data at the right time…
Clarify what increasing workloads means (system librarian responsible for running all reports…) New institutional emphasis on having teams work and think together…
In order to effect these changes, this is what we did…
Initially followed a distributed deployment model where trained staff conferred with Systems Librarian Trained staff subsequently trained additional staff Knowledge Management: staff mobility meant that front-line staff could use previous experience in other functional areas to create reports Team building & Knowledge Transfer
Reports generated on scheduled basis and locally controlled within Library units Myriad staff involvement Detailed data manipulation and presentation done outside of Talis Decisions Excel expertise Reports specially catered to internal customers and external customers
Limitations in so far as not taken up by all trained staff (training not entirely successful…) Can disparate data sets be linked: ex. decrease in number of patrons linked to decrease in number of print journals subscriptions? Valerie is now going to describe a finite project within Business & Law that demonstrates this process of asking a question, finding evidence to answer the question, and applying the information towards demonstrable result.
Subject Support (front of house) LLs providing help and training on using resources, finding books, etc etc Science/Technology/Med./ Arch./Eng. Humanities/Soc.Sci/ Business/ Law Recently Restructured Opportunity to centralise data gathering Review services over entire unit usage statistics more freely available via Book usage data (Talis Decisions and training prog) Fund data (Talis Decisions and training programme)
Then came a reason to put all this data to good use…………………………money got tight!
Budget deficit in University & nationally Spiralling costs of subscriptions IReL-funded, discipline-specific resources Usage stats driving renewal Lack of profile of existing resources/usage “ Luxury Resources V Essential Research Tools”
Slide speaks for itself – only comment to make is 10% cut required across school or equivalent across the colleges* - some freedom to make local adjustments at higher, college level once we came in on budget on the amount allocated for cancellation
1 How to identify titles and resources for cancellation -what drivers? -how to maintain standards in collection Previous difficulties in getting lists of journal titles due to multitude of electronic sources Pulled out list of journals in hard copy by fund Needed to identify which ones were duplicated in electronic format and on what platform Huge amount of titles-funded by library budget and IReL 2 Managing perception and involving academics in cancellation decisions -Unhappy with existing gaps in collections of journals -Duplication in formats-double payment -Memories of previous “journal crises -Maintaining core titles and preserving research strengths-a worry Thematic Research Areas/Interdisciplinary minded as much as possible 3 Promoting current resources-print& E Dovetail this unexpectedly with access as explained by Caleb to TD data on book usage Use the crisis as a chance to profile existing strengths based on usage stats-access to book usage stats easier, numbers of items at particular Deweys etc – an opportunity to support School teaching/research areas, interdiscipliary thematic research etc by providing data on books used, never borrowed, location etc plus other support material eg spec colls, oppubs etc 4 Role of liaisons as “resource managers” -usage data not hugely used due to separate nature of Systems and RS units in Library from AS (ie circulation desk not part of As function in biggest library) Management of resources: liaision trained on how to use, funding done by IRM-not seeing themselves as active participants in resource mgt Previous difficulties in getting lists of journal titles due to multitude of electronic sources LLs traditionally worked alone and did not perceive themselves as part of wider team of LLs-job descriptions varied, earlier and later hires had different emphasis, retirement culling the existing cohort and chnaging dynamics Management data v anecdotal evidence
Focused on Irel funded titles To remove duplication Focused on other titles (non IReL) To provide 10% cut LL team lead by Head and Deputy Identification of electronic AND print titles via MIS Reports – subs by fund Ejournal finder - locate eversions of journals Administrative Support from Library Assistants Checking of titles List of proposed cancellations Excel calculations – 10% requirement
Carefully worded communications with consistent message regardless of team member communicating it Clear listings of titles -Excel sheets -easy to read and clearly formatted Individual approach to School heads to endorse need for feedback which they did Attendance at school meetings by Unti Head, deputy and liaison-united front, library policy to outline approach and ask for feedback Feedback collated and responded to We did what we said we would do Provision of other data to inform title selection Impact Factor rankings for each title* Business: Equis Accreditation - rankings of titles and resources
10% cut or equivalent once the amount identified, some schools benefitted as the cuts were made across the Colleges and so every school did not necessarily have to lose 10%! Biz and law had “slack” in their system and benefitted from overal
This provided us with understanding of kind of data available to us Closer working relationships within & across teams Staff in different units (IRM; Systems) Staff in same units but offsite Open Dialogue Point of need access to key data LLs had full access to key data via shared docs on Shared Drive Gradual combining of print and e usage data to provide full picture to Schools -low levels of usage – responsibility of school and Full picture of resource usage to Schools ALL subs reviewed and significant areas of under-utliisation identified Such an exhaustive review would not have been done Review of ALL ongoing subs via Talis Decisiosn listing meant that a thorough weed and review of collections far more doable
This provided us with understanding of kind of data available to us WHAT DOES THIS SLIDE MEAN? Need to weed back last point Rename – Evid Baseld Librarianship for School/College Benefit quality reviewsetcc To what end?? To show Value for money Active support Areas for improvement
Use it or lose it : evidence based librarianship and resource management in recessionary times. Authors: Caleb Derven, Eoin McCarney, Valerie Kendlin
Use it or Lose it: Evidence BasedLibrarianship and ResourceManagement in Recessionary TimesCaleb Derven, Valerie Kendlin &Eoin McCarney UCD Library Leabharlann UCD
Eoin McCarney Acquisitions,Information Periodicals,Resources Resource DescriptionManagementValerie Kendlin Information skills,Library School Liaison,Academic Collection DevelopmentServicesCaleb Derven LMS Support,Library IT Application Support,Services E-resource support
Overview• What is Evidence Based Librarianship?• Setting the scene• Gathering the data• The evidence based approach• A change in culture• What we did & what we learned
What is Evidence Based Librarianship? “encourages more rigorous forms over less rigorous forms of evidence when making decisions.” (Booth 2003) “EBL employs the best available evidence based upon library science research to arrive at sound decisions about solving practical problems in librarianship.” (Eldredge 2000) “The best available evidence, moderated by user needs and preferences, is applied to improve the quality of professional judgements" (Booth 2001) “ It involves asking questions, finding information to answer them (or conducting one’s own research) and applying that knowledge to our practice.” (Kouogiannakis & Crumley 2002)
Setting the Scene• 12% cut in overall Library budget• Root and branch review of resource spend• 10% cut in subscriptions required• Reduce number of copies, review ratios• Review content of high use collections
Gathering the Data• School Fund, Title, Cost, Site, Supplier - Subs shared across multiple funds - Subs not cancelled correctly - Variations in currency• Manual adjustment required: IReL etc
Evidence Based Approach• Accurate data required urgently• Transparent & accountable system vital• MIS available via TALIS Decisions• Talis Decisions rolled out to staff• Evidence based approach fitted well
An Evidence-Based Approach:A Decision Making Framework• Context to formulate questions• Practitioner-based research to answer the questions• Practical applications for answers to questions• Evaluate effectiveness of action
What An Evidence-Based ApproachOffers• Promotes use of research in making decisions• Addresses areas of information overload and delay• As a research-based practice, allies librarianship to other disciplines• As an outcome-based practice, quantifies the benefits of library services
Evidence Available in UCD Library• Quantitative Data: MIS Reporting, Vendor Statistics, Website Usage• Qualitative Data: Survey Data, Focus Groups• Other Evidentiary Data: Library Gates, ILL requests, etc.
A Change In Culture• How Data was Collected & Used – Data collected and used by limited staff – Ad-hoc/ Infrequent requests, deadline-driven – Limited to specific areas of the library – Staff best-situated to assess data not always consulted – Infrastructure constraints
A Change In Culture – Catalysts forChange• What prompted the evidence-based approach – Regular budgetary reviews prompted by economic crisis – Increasing workloads – Streamlining delivery, reducing duplication – Organizational/ structural changes within the Library
What We Did - Training• Rolled-out MIS reporting tool (Talis Decisions) to key staff• Arranged on-site, 1 day training from vendor• Hands-on, lab-based training attended by Deputy Heads & Senior Library Assistants
What We Did – Post-Training• Initially staff conferred with Systems Librarian• Trained staff subsequently assisted their colleagues• Knowledge Management• Team building & Knowledge Transfer
What We Did – Changes in Collection& Use• Reports scheduled basis and locally controlled within Library units• Myriad staff involvement• Excel expertise• Reports specially catered to internal customers and external customers
How the process could be improved inthe future• Limitations of this approach• Identify other potential sources of data• Link disparate data sets• Formalise research-practitioner model
MIS data and the Business and Law Library Service
Business and Law Librarians andMIS:• Library Academic Services=Front of House – Includes Business and Law • Information skills • School Liaison • Collection Development• Library realignment • centralise data gathering • review services over entire unit
The Current Crisis:• Budget deficit-University & nationally• Spiralling costs of subscriptions• IReL -discipline-specific resources – Usage stats driving renewal• Lack of profile of existing resources• “Luxury v Essential Resources
This Meant…..• Current level of subs unsustainable!• 10% cut required across Schools• Limited time to cancel• Transparent identification of titles for cancellation had to be devised
The Question for Us Was….. NOT • WHETHER WE WOULD CANCEL • BUT • WHAT WOULD ASSIST THE SCHOOL IN HELPING US MAKE CANCELLATION DECSISIONS?
Library Academic Services-Issues• How to identify resources• Managing academic perception• Promoting existing resources• Liaison Librarians-team members
Our Approach1. Identify method of cancellation – Remove duplication (IReL) – Remaining titles to provide 10% cut2. Listing of all titles via – MIS Reports listing journals by fund – SFX to identify e-versions
Our Approach (2)3. Administrative Support – Checking of titles – Excel calculations – List of proposed cancellations4. Team Leader -Process Driver
Selling it to the Academics1. Consistent message2. Subscriptions listings-clear3. Approach to School heads4. Attendance at school meetings5. Feedback responded to6. Provision of other data 1.Impact factor rankings 2.Business: Equis Accreditation
Result • 10% reduction in subscriptions achieved – Within budget – On time – Creation of EBL framework for future projects – While also taking opportunity to highlight existing resource “gems” in the Library
Benefits of Improved Access to MISData for Academic Services• Up-skilling team• Closer working relationships• School-Library Dialogue Improved• Point of need access to key data• Appreciation of MIS data usage• “COLLECTION” = print & electronic
Summary• Recession prompted in depth review of all spend• Challenging but invaluable exercise• Key skills acquired – MIS, data manipulation• Knowledge transferred to broader staff base• Evidence based system developed• Transparent, accountable model for use with schools
The Future..• Quality Reviews• Annual reports by school• Accreditation by professional bodies• Metrics for purchase models (Ratio of books to students)• Produce Information Resource Policies• Track developments