Downloaded 21 times
Uploaded byJoe Matthews
Library Outcomes: The Holy Grail
The document discusses measuring the value and impact of libraries. It provides examples of how libraries can define and measure outcomes related to student success, learning, retention, and career outcomes. It also discusses measuring financial returns on investment and demonstrating impacts on areas like faculty research productivity, institutional reputation, and community benefits. The document emphasizes the importance of measuring outcomes that align with institutional goals and of communicating assessment results through stories and statistics to demonstrate the library's value and success.
More Related Content
Similar to Library Outcomes: The Holy Grail
Library Outcomes: The Holy Grail
- 1. Library Outcomes: The HolyGrail Joe Matthews August 2012
- 3. Broadcast Model We do They utilize
- 8. In the informationage, it is not what you know, but what you can find; genius will be measured by the speed at which you can find things. Anonymous
- 17. 1. Measure success 2.Measurement of Value 3. Innovate 4. Communicate
- 18.
- 21.
- 22. Logic Model INPUTS OUTPUTS OUTCOMES What we What we do What results invest in Who we reach are achieved? If - Then
- 23. Logic Model forLibraries Community So that … or organization benefits in So that … Individual some way benefits in some way due to the Customers use of its Benefits So that … use library’s collections & or physical & services Impacts virtual Library collections & Outcomes So that … converts the services inputs to useful collections & Outputs Library receives a services budget to support the Process goals of the community Inputs
- 24. Outcomes - Impact Learning Action Conditions Awareness Behavior Performance Knowledge Practice Attitudes Decision- Social Skills making Opinions Policies Economic Aspirations Social Action Motivations Environmental
- 25. Perspectives on Value Impacts Direct Personal Organizational Use Indirect Financial Benefits Option – Preservation of option for future use by me Nonuse Existence – Perceived value and significance to the community Legacy – Value of preservation for future generations
- 26.
- 27.
- 28. Define, develop, andmeasure outcomes that contribute to institutional effectiveness ACRL Standards for Libraries in Higher Education
- 29. If the physicalproximity of print collections had a demonstrable impact on researcher productivity, no university would hesitate to allocate prime real estate to library stacks.
- 30. How to DemonstrateImpact in … Student Faculty • Enrollment • Research productivity • Achievement • Grants • Learning • Teaching • Retention & graduation • Experiences • Career success Institutional Reputation
- 31.
- 32.
- 34. University of Wollongong •Data into the Library Cube • The Library Cube provides the information needed to support continuous improvement in three areas: collection development; academic relationships; and marketing. • The Library has seen a positive correlation between borrowing activity and academic performance
- 35. University of Minnesota Gym Bags and Mortarboards Use Campus Recreational Facilities At least 25 times, first-year retention increased 1% & 5-year graduation rates increased 2%
- 37.
- 39. How scalable islibrary instruction?
- 40. Student Retention &Graduation Persistence Attrition Retention Graduation Rates Completion
- 41. Australasian Survey ofStudent Engagement (AUSSE)
- 42.
- 43. Library Retention Studies Universityof Minnesota Libraries – 77% of undergrads made use of the libraries, 85% of grad students made use of the libraries – Students who used the library at least once were 1.54 times more likely to re-enroll
- 44. This is important! •Helps tie-in with institutional goals and objectives • Helps to integrate the library with other departments & faculties • May open increased funding opportunities • Helps better serve the needs of our users, students
- 45. Student Career Success Grad School Exams Alumni Surveys
- 47. Ithaka Studies • Libraryservices not understood • Library services not valued • The Library is disappearing
- 48.
- 49. University of Illinois,Urbana-Champaign Connected citations to resources in the library’s collection to successful grant proposals, and the income the grants generated ROI = 4.38:1 Other studies – 0.27:1 to 15.54:1
- 50. • Comprehensive assessment of the library • ROI of the journal collection & readership • ROI for support of teaching & learning • ROI of digitized special collections • ROI of eBooks • Value of library commons
- 51. ROI instruments andcalculations do not work for academic libraries, and present naive and misinterpreted assessments of our roles and impacts at our institutions and across higher education. James Neal
- 53. Be cautious aboutcause-and-effect relationships
- 54. Problems with LibraryAssessment • Some studies correlate library use and retention, but no causative links • Strength of correlations is weak – at best • Problem with almost all existing library research are the small sample size • Only a handful of research has been done and a lot of it is old
- 55.
- 56. Layers of Data Other University-wide Data Student Surveys, Faculty Surveys, Alumni Surveys, … Performance Data Cumulative GPA, ACT score, CLA score, …. Demographic Data College, Level, Major, Gender, Ethnicity, Age, … Library Data Circulation, Logins, Downloads, Reference, Instruction, …
- 58. Privacy Use student IDsto match records from one data set to another and then Strip the student ID number from the combined record Work with Office of Institutional Research & the campus Institutional Review Board
- 63.
- 64. Social Benefits • Basicreading literacy • Local history & genealogy • Summer reading “gap” • Health & well-being • Business/career • Social cohesion • Information literacy • General information • Library as place • Welcoming newcomers
- 66. Tracking Value The EngagedLibrary: Stories of Community Building •Prove that public libraries build social capital •Identify & connect the library’s assets to the community •Assess & strengthen the library’s connections with and use of community assets •Produce a toolkit for other libraries to adopt •Mapping tools to perform an inventory services, identify areas for improvement and highlight library’s contribution to the community’s wider social, educational, cultural and economic goals.
- 68. Measurement OfValue
- 69. Low value Moderate value Little value High value Cost Use
- 70.
- 72.
- 73. Nature of Informationis Changing Information Information was …. is …. Scare, controlled All around us Expensive Cheap or free Shaped by elites Shaped by consumers One-way, mass Designed for sharing, consumption participation & feedback Slow moving Immediate External to our worlds Embedded to our worlds
- 74. Liberate Knowledge Atoms to bits
- 75. Online Participation Creators Collectors Conversationalists Critics Joiners Spectators Inactives 0 10 20 30 40 50 60 70 80
- 76. Expertise & influenceshifts to networks Share the stage with amateur experts
- 77. All products andservices will be Where? Social networks What? Social content How? Social publishing Social gaming Social shopping Social search What for? Social impact Social currency
- 85. Over 303 Million resources Over 52,000 corrections in one day
- 86.
- 88. National Library ofthe Netherlands
- 90.
- 92. Where is thevalue?
- 93. Groups – CanadianHistory 389 members Ontario 3,677 members
- 94.
- 95. 65 images –most relevant Niagara-on-the-Lake Public Library
- 96.
- 97. Notice all thehyperlinks
- 98.
- 101.
- 104. Nature of Libraryis Changing Library Library was …. is …. Lending Sharing Distribution Contribution Informing Conversing Artifacts Knowledge Access Participation
- 105.
- 106.
- 108. Stories + Stats= Success
- 110. Measure Value Communicate Innovate
- 111.
- 112. Joe@JoeMatthews.Org www.joematthews.org Joe Matthews Library Consultant
Editor's Notes
- #3 Libraries (Museums) are similar to the 20th century broadcast model – also called a logic modelIt’s a group of people who have, over time, figured out how to work together to provide to provide a product or serviceThis model is true for businesses, government, and non-profitsResources go it and outcomes come outYour resources include budget, staff, buildings, collections, stuff, trust, reputation, brandSomething happens inside (it’s a mystery) – sometimes called a processAnd out the other end come beneficial outcomesHopefully what you get, especially for a mission driven organization, is something meaningful, valuable and precious
- #4 The broadcast model says you hire really smart people, some real experts, and they do the things that need doing.The results are made available to the passive and grateful customer.The broadcast model was really good (for awhile) and gave us radio, television, cable TV, newspapers, magazines, and so forth.
- #5 Libraries have changed more in the past two decades than in the prior two centuries. Technology is the major driver . . .We need to recognize that all this change has only begun, and that change is irreversible.
- #6 It is the physicality of the information medium that made possible the rationing, gate keeping, and in the case of for-profit scientific journals, profit gouging that are the key characteristics of the paper-based information age
- #7 The Internet has changed everything
- #9 Broadband access has reached 75 % of US householdsIn Canada much higher %Canadians spend more time online than do Americans
- #10 Mobile devices are proliferatingMore cell phones than land linesMultimedia and online communication for the bulk of what young people are reading today
- #11 eBook popularityBenefits - ConvenienceWeightCost of e-book titlesDisadvantagesBook formats Digital Rights ManagementSlow e-ink displayPage navigationConsumer privacy and e-book permanence
- #12 Google BooksDigitization
- #13 Streaming is responsible for more than half of all Internet traffic
- #14 Free is real competition
- #15 Trumps everything!Ease of use is an important differentiatorGood enough – satisficing (Herb Simon)Just-in-time
- #16 Eli Neiburgerlocalness loses most of its value when anybody can retrieve something from anywhere in milliseconds. The value of the local copy is further diminished since there is little difference between transmission and duplication. This means that collections are being disrupted as we move from atoms to bits. The Internet has lowered the cost of transmitting information to negligible levels. Unfortunately, most libraries still operate under the assumption that their collections are critical
- #17 Given the Tsunami of change, how will successful libraries differentiate themselves?
- #18 4 ThemesOf Successful Libraries
- #19 Theme of foundational importance – how does the library define and measure success?How will we know when we get there?
- #20 Successful libraries strive to help their parent organization achieve its goals and objectives
- #21 It is not how much a libraryis used that matters,rather how does the library impact or benefit the customer? In other words, outcome measures are keyUse as a planning toolUse as an evaluation toolUse as a reporting tool
- #22 Key differentiatorOutcomes – how library services contribute to their parent organization – what is important to the organizationUsers definition of “value” is a moving target. Measures that matter to your stakeholdersMeasures that are communicated using the stakeholders languageDemonstrate how the library makes a difference
- #23 Framework for describing the relationships between investments, activities, and results.Popular in non-profit sector & increasingly in librariesBased on “logical links” cause & effectOrr’s Input, Process, Output, Outcomes ModelA common approach for integrating planning, implementation, evaluation, and reporting. Stephen Covey – “start with the end in mind”Refocus from the activity to the results(impact)
- #24 Inputs or Resources: budget, staff, collections, facilities, & technologyProcess or Productivity measures – time or cost to perform an activityOutputs or Use measures – counts of activitiesOutcomes, Benefits or Impacts – Immediate (short-term), Mid-term, and long-termLogic model is different for each segment of the community being served (and why they are coming to the library)
- #25 Learning at the individual level – short term resultsAction – Medium term resultsConditions – Ultimate impacts
- #27 information encounters must be contextualized, to activate and extend prior understanding
- #29 Review possible academic library outcomes first
- #30 A large local inventory was a hallmark of academic reputationWe no longer live in that world.
- #31 Megan’s categories
- #32 One study – Gary Reynolds. The Impact of Facilities on Recruitment & Retention of Students. New Directions for Institutional Research, 135, Fall 2007.– showed that the library was the 2nd or 3rd most important reason why a university was selectedPartnering with Student Affairs – campus tour offices – Yet, a more recent study (with a small sample size of 297) suggested that 2/3rds of the respondents said that library was “no factor” in their decision to attend a specific university and another 29% said the library was only a “minor factor” in their decision. Lombard JAL, July 2012.Connecting with parentsGoogle backtoschool_XS.jpgusd261.com
- #33 Student learning takes place in many places – classroom, library, faculty offices, dorm rooms, interacting with friends, ANDGoogle 123RF Portrait of happy young people sitting in pub, drinking beer, looking at camera, smiling.
- #34 UK Library Impact Data Project
- #35 Library Cube – relational database
- #36 Regression Analysis - led to $59 million expansion
- #39 Hope College
- #41 Many confusing terms to describe the same thingRetention rates range from 60 to 80%Do you know the retention rate for your university?
- #42 2008 AUSSE data – more you use the library the LESS likely to consider leavingDeparture intention“How libraries and librarians can support student engagement”
- #43 Early warning system!
- #45 Retention
- #47 What is the impact of the library on the faculty?Flickr AKMA Seabury Faculty
- #48 Ithaka plus Cluff & Murrah 1987 JALFlickr Ben Heine We All Disappear Someday
- #49 Flickr {e u g e n e } Financial Growth
- #50 Formula ? Convoluted –“Funding does not regenerate funding, but reputation does”
- #51 Carol Tenopir Univ of Tennessee
- #56 Library needs to collaborate with others on campus to COMBINE dataOffice of Institutional Research
- #60 Outcomes for Public Libraries
- #61 Stratford PL OntarioAustralian project - Libraries/Building/CommunitiesDeveloping social capitalProviding a welcoming environmentCreating a pride of placeAttracting users from all walks of lifeReaching out to the communityAppreciation of cultural differencesBuilding bridges to governmentEncouraging collaboration across the community
- #62 Hamilton PL OntarioLibraries/Building/CommunitiesOvercoming the digital divideMaking technology accessibleExploiting technology to benefit the communityCreating informed communitiesCommunity informationGovernment informationProviding a gateway to the world of information
- #63 Pickering PL OntarioLibraries/Building/CommunitiesConvenient and comfortable places of learningDeveloping information skillsStimulating ideas and discussionSupporting vulnerable learnersSupporting students
- #64 Public Library AssetsBonding social capitalBridging social capitalLinking social capital
- #66 Urban Libraries Council. Making Cities Stronger: Public Library Contributions to Local Economic Development, 2007.Building a stronger local economyImproving early literacy & school readinessBuilding workforce participationSupporting small businessThe power of place
- #67 Urban Libraries Council 2006 Chicago PLrepresent a step towards a more ethnography based approach to measuring the value of public libraries and digital services.
- #68 ROI in Public Libraries – 4:1 to 6:1What's the Return on ROI? The Benefits and Challenges of Calculating Your Library's Return on Investment. Library Leadership & Management, 25 (1), 2011, 1-14
- #69 Second ThemeUsing performance measures – identify services that no longer represent a good value propositionNot the same thing as efficiencyCritical thinking is about:Openness, flexibility, adaptability• Articulating the goal• Clarifying assumptions• Questioning status quo• Facts• Focusing on the future
- #70 Back of envelope analysisOutsourcing – lowers costs for activities that contribute little valueEliminate or reduce reference? Eliminate reference collections? Eliminate reference desks? Majority of reference questions are driven by structural barriers that libraries impose on their customers – Poor Web site design, confusing buildings and signage, use of professional jargon Eliminate all service desks?Simplify. Automate.
- #71 QualityHerb Simon “satisficing”What is the value of a “good” cataloging record?
- #72 Listen to your customersLean – Efficiency3 RulesSimplifyAutomateEliminate
- #73 Third Theme – Spring – Fresh, new & innovative servicesCreate value in the lives of the library’s customers
- #75 Google BooksDigitization
- #76 Creators – blog, Web site, upload video, upload audio, upload presentations, write stories or articlesCollectors – RSS feeds, online voting, add tagsConversationalists – update social media site, Post on TwitterCritics – post ratings/reviews, add comments, contribute to online forums, add/edit a wikiJoiners – Maintain a profile on a social networking site,Visit social networking siteSpectators – Read blogs, listen to podcasts, watch video, read online forums, read customer ratings/reviews, read Tweets
- #77 Quality of library resources & services – no longer highly valuedWisdom of the massesOnline peer information exchange and networking
- #78 The network is re-configuring organizational boundaries everywhereThe net has significantly reduced the cost of establishing and managing interactions with external parties
- #79 Tagging
- #80 API for library catalogs – add tags
- #81 Reviews – Amazon
- #82 Amazon If you liked, others bought ….
- #83 Crowd sourcingLego, Nike, Starbucks, P&G,
- #84 Howard Rheingold’s Smart Mobs (2002) –Collective action without central authority, crowdsourcing, the rise of cheap mobile platforms, global knowledge commonsLawrence Lessig’sThe Future of Ideas (2002) – groups form easily to discuss any issueTim O’Reilly’s article “What is Web 2.0?” (2005) talks about some of the implications of the Long tail, users add value (network effect), perpetual beta, cooperate (don’t control)Dan Tapscott and Anthony Williams Wikinomics (2006) discusses the benefits of mass collaborationClay Shirky’sCognitive Surplus – talks about the available free time people have to contribute – over 1 trillion hours world wide each year that can be harnessed for social good
- #85 All of these sites get 1 MILLION hours / year of community effort
- #86 National Library of Australia – correcting OCR errors, add comments, add tags
- #87 Transcribing
- #89 Translation & transcription
- #90 Ways you can use crowds – involvement!Correction & transcription
- #91 Historical photos – add names and tags – when, where, who, why (what else is happening at that time) – State Lib of New South WalesComplementary collectionsClassification
- #92 London Re-Cut – co-created remixing of filmHistoric moving image collections plus still photographsCreate their own films
- #98 http://en.wikisource.org/wiki/Laura_Secord:_A_Study_in_Canadian_Patriotism
- #99 Google Books
- #100 Crowd funding
- #101 Conducted a brief survey – what can we add that would be of value3 hours later – lots of great suggestions – easy to implement
- #102 Opt in email notices, newsletters, customization
- #103 For users (access) and with users (transparency)Seed community processes by providing content and toolsLet go – use tools that already exist
- #104 Successful libraries regularly and deeply engage with their customers to ensure they are attuned with the customer’s needs
- #106 Partner – with your community members and community organizations
- #107 Fourth Theme - SummerCommunication of value to a wide variety of stakeholdersBe intentional – plan your message
- #108 What measures of library success will resonate in your organizational setting?Be visible with your funding decision makers – build personal connectionsIn your environment, how is value defined, measured & communicated?
- #109 Tell your story in numbers and stories Your message – “We have contributed towards YOUR goals by ….”Story Bridge, Brisbane, Australia
- #110 Use lots of color and excitement to convey both written and spoken stories of the value of the libraryA framework such as the Balanced Scored or the Social ROI Triple Bottom Line – Social, Economic, Environment
- #111 Successful Libraries – 4 themes