Library Outcomes: The Holy Grail


Published on

An exploration of what differentiates successful versus less-than-successful libraries.

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Libraries (Museums) are similar to the 20th century broadcast model – also called a logic modelIt’s a group of people who have, over time, figured out how to work together to provide to provide a product or serviceThis model is true for businesses, government, and non-profitsResources go it and outcomes come outYour resources include budget, staff, buildings, collections, stuff, trust, reputation, brandSomething happens inside (it’s a mystery) – sometimes called a processAnd out the other end come beneficial outcomesHopefully what you get, especially for a mission driven organization, is something meaningful, valuable and precious
  • The broadcast model says you hire really smart people, some real experts, and they do the things that need doing.The results are made available to the passive and grateful customer.The broadcast model was really good (for awhile) and gave us radio, television, cable TV, newspapers, magazines, and so forth.
  • Libraries have changed more in the past two decades than in the prior two centuries. Technology is the major driver . . .We need to recognize that all this change has only begun, and that change is irreversible.
  • It is the physicality of the information medium that made possible the rationing, gate keeping, and in the case of for-profit scientific journals, profit gouging that are the key characteristics of the paper-based information age
  • The Internet has changed everything
  • Broadband access has reached 75 % of US householdsIn Canada much higher %Canadians spend more time online than do Americans
  • Mobile devices are proliferatingMore cell phones than land linesMultimedia and online communication for the bulk of what young people are reading today
  • eBook popularityBenefits - ConvenienceWeightCost of e-book titlesDisadvantagesBook formats Digital Rights ManagementSlow e-ink displayPage navigationConsumer privacy and e-book permanence
  • Google BooksDigitization
  • Streaming is responsible for more than half of all Internet traffic
  • Free is real competition
  • Trumps everything!Ease of use is an important differentiatorGood enough – satisficing (Herb Simon)Just-in-time
  • Eli Neiburgerlocalness loses most of its value when anybody can retrieve something from anywhere in milliseconds. The value of the local copy is further diminished since there is little difference between transmission and duplication. This means that collections are being disrupted as we move from atoms to bits. The Internet has lowered the cost of transmitting information to negligible levels. Unfortunately, most libraries still operate under the assumption that their collections are critical
  • Given the Tsunami of change, how will successful libraries differentiate themselves?
  • 4 ThemesOf Successful Libraries
  • Theme of foundational importance – how does the library define and measure success?How will we know when we get there?
  • Successful libraries strive to help their parent organization achieve its goals and objectives
  • It is not how much a libraryis used that matters,rather how does the library impact or benefit the customer? In other words, outcome measures are keyUse as a planning toolUse as an evaluation toolUse as a reporting tool
  • Key differentiatorOutcomes – how library services contribute to their parent organization – what is important to the organizationUsers definition of “value” is a moving target. Measures that matter to your stakeholdersMeasures that are communicated using the stakeholders languageDemonstrate how the library makes a difference
  • Framework for describing the relationships between investments, activities, and results.Popular in non-profit sector & increasingly in librariesBased on “logical links” cause & effectOrr’s Input, Process, Output, Outcomes ModelA common approach for integrating planning, implementation, evaluation, and reporting. Stephen Covey – “start with the end in mind”Refocus from the activity to the results(impact)
  • Inputs or Resources: budget, staff, collections, facilities, & technologyProcess or Productivity measures – time or cost to perform an activityOutputs or Use measures – counts of activitiesOutcomes, Benefits or Impacts – Immediate (short-term), Mid-term, and long-termLogic model is different for each segment of the community being served (and why they are coming to the library)
  • Learning at the individual level – short term resultsAction – Medium term resultsConditions – Ultimate impacts
  • information encounters must be contextualized, to activate and extend prior understanding
  • Review possible academic library outcomes first
  • A large local inventory was a hallmark of academic reputationWe no longer live in that world.
  • Megan’s categories
  • One study – Gary Reynolds. The Impact of Facilities on Recruitment & Retention of Students. New Directions for Institutional Research, 135, Fall 2007.– showed that the library was the 2nd or 3rd most important reason why a university was selectedPartnering with Student Affairs – campus tour offices – Yet, a more recent study (with a small sample size of 297) suggested that 2/3rds of the respondents said that library was “no factor” in their decision to attend a specific university and another 29% said the library was only a “minor factor” in their decision. Lombard JAL, July 2012.Connecting with parentsGoogle
  • Student learning takes place in many places – classroom, library, faculty offices, dorm rooms, interacting with friends, ANDGoogle 123RF Portrait of happy young people sitting in pub, drinking beer, looking at camera, smiling.
  • UK Library Impact Data Project
  • Library Cube – relational database
  • Regression Analysis - led to $59 million expansion
  • Hope College
  • Many confusing terms to describe the same thingRetention rates range from 60 to 80%Do you know the retention rate for your university?
  • 2008 AUSSE data – more you use the library the LESS likely to consider leavingDeparture intention“How libraries and librarians can support student engagement”
  • Early warning system!
  • Retention
  • What is the impact of the library on the faculty?Flickr AKMA Seabury Faculty
  • Ithaka plus Cluff & Murrah 1987 JALFlickr Ben Heine We All Disappear Someday
  • Flickr {e u g e n e } Financial Growth
  • Formula ? Convoluted –“Funding does not regenerate funding, but reputation does”
  • Carol Tenopir Univ of Tennessee
  • Library needs to collaborate with others on campus to COMBINE dataOffice of Institutional Research
  • Outcomes for Public Libraries
  • Stratford PL OntarioAustralian project - Libraries/Building/CommunitiesDeveloping social capitalProviding a welcoming environmentCreating a pride of placeAttracting users from all walks of lifeReaching out to the communityAppreciation of cultural differencesBuilding bridges to governmentEncouraging collaboration across the community
  • Hamilton PL OntarioLibraries/Building/CommunitiesOvercoming the digital divideMaking technology accessibleExploiting technology to benefit the communityCreating informed communitiesCommunity informationGovernment informationProviding a gateway to the world of information
  • Pickering PL OntarioLibraries/Building/CommunitiesConvenient and comfortable places of learningDeveloping information skillsStimulating ideas and discussionSupporting vulnerable learnersSupporting students
  • Public Library AssetsBonding social capitalBridging social capitalLinking social capital
  • Urban Libraries Council. Making Cities Stronger: Public Library Contributions to Local Economic Development, 2007.Building a stronger local economyImproving early literacy & school readinessBuilding workforce participationSupporting small businessThe power of place
  • Urban Libraries Council 2006 Chicago PLrepresent a step towards a more ethnography based approach to measuring the value of public libraries and digital services.
  • ROI in Public Libraries – 4:1 to 6:1What's the Return on ROI? The Benefits and Challenges of Calculating Your Library's Return on Investment. Library Leadership & Management, 25 (1), 2011, 1-14
  • Second ThemeUsing performance measures – identify services that no longer represent a good value propositionNot the same thing as efficiencyCritical thinking is about:Openness, flexibility, adaptability• Articulating the goal• Clarifying assumptions• Questioning status quo• Facts• Focusing on the future
  • Back of envelope analysisOutsourcing – lowers costs for activities that contribute little valueEliminate or reduce reference? Eliminate reference collections? Eliminate reference desks? Majority of reference questions are driven by structural barriers that libraries impose on their customers – Poor Web site design, confusing buildings and signage, use of professional jargon Eliminate all service desks?Simplify. Automate.
  • QualityHerb Simon “satisficing”What is the value of a “good” cataloging record?
  • Listen to your customersLean – Efficiency3 RulesSimplifyAutomateEliminate
  • Third Theme – Spring – Fresh, new & innovative servicesCreate value in the lives of the library’s customers
  • Google BooksDigitization
  • Creators – blog, Web site, upload video, upload audio, upload presentations, write stories or articlesCollectors – RSS feeds, online voting, add tagsConversationalists – update social media site, Post on TwitterCritics – post ratings/reviews, add comments, contribute to online forums, add/edit a wikiJoiners – Maintain a profile on a social networking site,Visit social networking siteSpectators – Read blogs, listen to podcasts, watch video, read online forums, read customer ratings/reviews, read Tweets
  • Quality of library resources & services – no longer highly valuedWisdom of the massesOnline peer information exchange and networking
  • The network is re-configuring organizational boundaries everywhereThe net has significantly reduced the cost of establishing and managing interactions with external parties
  • Tagging
  • API for library catalogs – add tags
  • Reviews – Amazon
  • Amazon If you liked, others bought ….
  • Crowd sourcingLego, Nike, Starbucks, P&G,
  • Howard Rheingold’s Smart Mobs (2002) –Collective action without central authority, crowdsourcing, the rise of cheap mobile platforms, global knowledge commonsLawrence Lessig’sThe Future of Ideas (2002) – groups form easily to discuss any issueTim O’Reilly’s article “What is Web 2.0?” (2005) talks about some of the implications of the Long tail, users add value (network effect), perpetual beta, cooperate (don’t control)Dan Tapscott and Anthony Williams Wikinomics (2006) discusses the benefits of mass collaborationClay Shirky’sCognitive Surplus – talks about the available free time people have to contribute – over 1 trillion hours world wide each year that can be harnessed for social good
  • All of these sites get 1 MILLION hours / year of community effort
  • National Library of Australia – correcting OCR errors, add comments, add tags
  • Transcribing
  • Translation & transcription
  • Ways you can use crowds – involvement!Correction & transcription
  • Historical photos – add names and tags – when, where, who, why (what else is happening at that time) – State Lib of New South WalesComplementary collectionsClassification
  • London Re-Cut – co-created remixing of filmHistoric moving image collections plus still photographsCreate their own films
  • Google Books
  • Crowd funding
  • Conducted a brief survey – what can we add that would be of value3 hours later – lots of great suggestions – easy to implement
  • Opt in email notices, newsletters, customization
  • For users (access) and with users (transparency)Seed community processes by providing content and toolsLet go – use tools that already exist
  • Successful libraries regularly and deeply engage with their customers to ensure they are attuned with the customer’s needs
  • Partner – with your community members and community organizations
  • Fourth Theme - SummerCommunication of value to a wide variety of stakeholdersBe intentional – plan your message
  • What measures of library success will resonate in your organizational setting?Be visible with your funding decision makers – build personal connectionsIn your environment, how is value defined, measured & communicated?
  • Tell your story in numbers and stories Your message – “We have contributed towards YOUR goals by ….”Story Bridge, Brisbane, Australia
  • Use lots of color and excitement to convey both written and spoken stories of the value of the libraryA framework such as the Balanced Scored or the Social ROI Triple Bottom Line – Social, Economic, Environment
  • Successful Libraries – 4 themes
  • Library Outcomes: The Holy Grail

    1. 1. Library Outcomes:The Holy Grail Joe Matthews August 2012
    2. 2. Broadcast ModelWe do They utilize
    3. 3. In the information age, it is not what you know,but what you can find; genius will be measuredby the speed at which you can find things. Anonymous
    4. 4. 1. Measure success2. Measurement of Value3. Innovate4. Communicate
    5. 5. Define &Measure Success
    6. 6. Measurementof Value
    7. 7. Logic ModelINPUTS OUTPUTS OUTCOMESWhat we What we do What resultsinvest in Who we reach are achieved? If - Then
    8. 8. Logic Model for Libraries Community So that … or organization benefits in So that … Individual some way benefits in some way due to the Customers use of its Benefits So that … use library’s collections & or physical & services Impacts virtual Library collections & Outcomes So that … converts the services inputs to useful collections & Outputs Library receives a services budget to support the Process goals of the community Inputs
    9. 9. Outcomes - ImpactLearning Action ConditionsAwareness Behavior PerformanceKnowledge PracticeAttitudes Decision- SocialSkills makingOpinions Policies EconomicAspirations Social ActionMotivations Environmental
    10. 10. Perspectives on Value Impacts Direct Personal Organizational Use Indirect FinancialBenefits Option – Preservation of option for future use by me Nonuse Existence – Perceived value and significance to the community Legacy – Value of preservation for future generations
    11. 11. Personal
    12. 12. Organizational
    13. 13. Define, develop, and measure outcomes that contribute toinstitutional effectiveness ACRL Standards for Libraries in Higher Education
    14. 14. If the physical proximity ofprint collections had a demonstrable impact on researcher productivity, no university would hesitate to allocate prime real estate to library stacks.
    15. 15. How to Demonstrate Impact in …Student Faculty• Enrollment • Research productivity• Achievement • Grants• Learning • Teaching• Retention & graduation• Experiences• Career success Institutional Reputation
    16. 16. Student Enrollment
    17. 17. Student Learning
    18. 18. University of Wollongong• Data into the Library Cube• The Library Cube provides the information needed to support continuous improvement in three areas: collection development; academic relationships; and marketing.• The Library has seen a positive correlation between borrowing activity and academic performance
    19. 19. University of Minnesota Gym Bags and Mortarboards Use Campus Recreational Facilities At least 25 times, first-year retention increased 1% & 5-year graduation rates increased 2%
    20. 20. University of Minnesota
    21. 21. How scalable is library instruction?
    22. 22. Student Retention & Graduation Persistence Attrition RetentionGraduation Rates Completion
    23. 23. Australasian Survey of Student Engagement (AUSSE)
    24. 24. University of Huddersfield
    25. 25. Library Retention StudiesUniversity of Minnesota Libraries – 77% of undergrads made use of the libraries, 85% of grad students made use of the libraries – Students who used the library at least once were 1.54 times more likely to re-enroll
    26. 26. This is important!• Helps tie-in with institutional goals and objectives• Helps to integrate the library with other departments & faculties• May open increased funding opportunities• Helps better serve the needs of our users, students
    27. 27. Student Career Success Grad School ExamsAlumni Surveys
    28. 28. Ithaka Studies• Library services not understood• Library services not valued• The Library is disappearing
    29. 29. Financial
    30. 30. University of Illinois, Urbana-Champaign Connected citations to resources in the library’s collection to successful grant proposals, and the income the grants generated ROI = 4.38:1 Other studies – 0.27:1 to 15.54:1
    31. 31. • Comprehensive assessment of the library• ROI of the journal collection & readership• ROI for support of teaching & learning• ROI of digitized special collections• ROI of eBooks• Value of library commons
    32. 32. ROI instruments and calculationsdo not work for academic libraries,and present naive and misinterpretedassessments of our roles and impacts at our institutions and across higher education. James Neal
    33. 33. Be cautious about cause-and-effect relationships
    34. 34. Problems with Library Assessment • Some studies correlate library use and retention, but no causative links • Strength of correlations is weak – at best • Problem with almost all existing library research are the small sample size • Only a handful of research has been done and a lot of it is old
    35. 35. Collaboration
    36. 36. Layers of Data Other University-wide Data Student Surveys, Faculty Surveys, Alumni Surveys, … Performance Data Cumulative GPA, ACT score, CLA score, …. Demographic Data College, Level, Major, Gender, Ethnicity, Age, … Library DataCirculation, Logins, Downloads, Reference, Instruction, …
    37. 37. PrivacyUse student IDs to match records fromone data set to another and thenStrip the student ID number from thecombined recordWork with Office of Institutional Research &the campus Institutional Review Board
    38. 38. (Rooney-Browne, 2009b).
    39. 39. Social Benefits• Basic reading literacy • Local history & genealogy• Summer reading “gap” • Health & well-being• Business/career • Social cohesion• Information literacy • General information• Library as place • Welcoming newcomers
    40. 40. Tracking ValueThe Engaged Library:Stories of Community Building•Prove that public libraries build social capital•Identify & connect the library’s assets to thecommunity•Assess & strengthen the library’s connections withand use of community assets•Produce a toolkit for other libraries to adopt•Mapping tools to perform an inventory services,identify areas for improvement and highlight library’scontribution to the community’s wider social,educational, cultural and economic goals.
    41. 41. Measurement Of Value
    42. 42.   Low value Moderate value    Little value High valueCost   Use
    43. 43. Quality
    44. 44. Innovation
    45. 45. Nature of Information is Changing Information Information was …. is …. Scare, controlled All around us Expensive Cheap or free Shaped by elites Shaped by consumers One-way, mass Designed for sharing, consumption participation & feedback Slow moving Immediate External to our worlds Embedded to our worlds
    46. 46. Liberate Knowledge Atoms to bits
    47. 47. Online Participation Creators CollectorsConversationalists Critics Joiners Spectators Inactives 0 10 20 30 40 50 60 70 80
    48. 48. Expertise & influence shifts to networksShare thestage with amateur experts
    49. 49. All products and services will beWhere? Social networksWhat? Social contentHow? Social publishing Social gaming Social shopping Social searchWhat for? Social impact Social currency
    50. 50. Over 303 Million resourcesOver 52,000corrections inone day
    51. 51. Over 980,000dishes transcribed
    52. 52. National Library of the Netherlands
    53. 53. Contextualization
    54. 54. Where is the value?
    55. 55. Groups – Canadian History 389 members Ontario 3,677 members
    56. 56. 843 results
    57. 57. 65 images – most relevantNiagara-on-the-Lake PublicLibrary
    58. 58. 494 results
    59. 59. Notice all the hyperlinks
    60. 60. 18,100 results
    61. 61. User Profile
    62. 62. Nature of Library is Changing Library Library was …. is …. Lending SharingDistribution Contribution Informing Conversing Artifacts Knowledge Access Participation
    63. 63. BuildingSharedValuefor aSharedFuture
    64. 64. Communication
    65. 65. Stories + Stats = Success
    66. 66. Measure Value CommunicateInnovate
    67. 67. Thanks
    68. 68. Joe Matthews Library Consultant