The value of academic libraries


Published on

The presentation slides for a half-day workshop that reviews the methods to identify the value of the academic library for students, faculty and the college or university itself.

Published in: Education, Business
1 Like
  • Brian -
    Of course it really depends on the overall goals and objectives of your university - is it retention, is it boosting graduate rates, is it ...
    Once you have a clear view of these goals then you can decide on what is important that the library should focus.
    Regards, Joe
    Are you sure you want to  Yes  No
    Your message goes here
  • now that is the BIG question. i'm actually compiling my notes from ACL and your talks, and will meet with our library staff and director to discuss the thought-provoking ideas you brought up. so...first i am working on gathering the gist of what you presented, then we can answer the application question. Any advice?
    Are you sure you want to  Yes  No
    Your message goes here
  • Hi Brian -

    Thanks - glad you find the presentation to be of value. How will you apply some of the content in your library?
    Are you sure you want to  Yes  No
    Your message goes here
  • Very good and thought-provoking presentation.
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • You can download one and buy the other
  • Claire Creaser and Valerie Spezi. Working Together. June 2012. UK: Loughborough University.Libraries are busy measuring activity and not measuring value
  • Richard Orr
  • Focus on outcomes not process
  • Not all impacts are positiveNot all impacts are intendedNot all impacts are immediateDifficulty in separating library impact from other influences
  • The How Questions
  • Allan Pratt and Ellen Altman. Live by the Numbers, Die by the Numbers
  • 2/3rds of managers responsible for library budgets have no idea of how to evaluate or value the library
  • How good is the library? Quality and capabilitiesWhat good does the library do? Benefits, Impacts - ValueHow well is the library managed? Efficiency - benchmarkingHow can we move from bad to good?
  • New Zealand Libraries, March 1990
  • What are some of the Tools for Assessment - Evaluation
  • Direct method – the financial savings method is used widely to monetize the direct use benefits of public libraries.As such, it undercounts benefits that have had an estimated value assigned to it Not so in academic libraries
  • Focus groups – open ended – standard, guided, exploratoryEthnography – work study, photo essays, mapping dairies, etc.University of Rochester –they “know” their customers
  • Survey - random sample best, set questions (no follow up), statistical analysis, representative sample, response ratesExplore - How was it?, What do you do?, What do you want?, What did you do?
  • TriangulationFlickeAGrinberg
  • Correlation does not mean cause-and-effect
  • Deborah Goodall & David Pattern . Academic library non/low use and undergraduate student achievement. Library Management, 32 (3), 2011, 159-170.p. 161
  • DefinitionsA nounExchange for or equivalenceMonetary or material worthUsefulness, utilityPrinciple, standard, or qualityToll, cost or priceDarkness or lightness of color A verbEstimate the worth of something (appraise)Regard highly (esteem)Assign a value to something Other definitions depending on the fieldRegardless of context, defining value is a complex issue with its own philosophical discipline; axiology (Cram, 1999, p.11). Axiology, or Value Theory defines three different dimensions of value; extrinsic value, systematic value and intrinsic value (Hartman, 1969, p. 114). Thus, there are a number of different value types, including personal value, aesthetic value, religious value, spiritual value; and ethical value. Bequest value – willingness to pay for the endowment of the good or service for future generationsGoogle images -
  • Adam SmithValue-in-exchangeValue-in-use (utility theory)Google image adam‑
  • Value is a moving targetWhat was valuable vs.What is valuable vs.What will be valuable
  • Eleanor Jo Rodger. Value & Vision. American Libraries, November 2002, 50-52.Ideas of value have changedMoving target, constantly needs assessmentUsefulness, quality, availability, imageHistoric is not valuableDoesn't’t correspond to staff ideas of importanceIs not about our professional valuesDoing wrong things well does not create value
  • Mobile technology is the needle, and social networks are the thread (materials being woven)
  • Libraries are so screwed Value of the local collections is being diminished
  • Douglas Badenoch et al
  • Robert Taylor
  • a result Discovery happens elsewhere
  • Hope College Library
  • A large local inventory was a hallmark of academic reputationWe no longer live in that world.
  • Photo Flickr.com401K College
  • Flickr Trinity College Library
  • Two different people receive different value when using the same item
  • Similar to Robert Orr’s Input-Process-Output-Outcomes modelImpact of any one characteristic of the University environment is clearly, at best, indirectInput are the entering student characteristicsOutput are the graduating student characteristicsSimilar to Orr’s Input, Process, Output, Process model
  • Richard Shavelson. Measuring College Learning Responsibility: Accountability in a New Era. Stanford, CA: Stanford University Press, 2010, pg 13
  • Libraries building communities – 75% of users and nonusers agreed that the library is a good place for facilitating social interaction
  • Personal Perspectives
  • TefkoSaracevic and Paul Kantor
  • Gates Foundation
  • Organizational Perspective
  • NSSE results
  • Pascarella, Seifert, and Blaich. How Effective are the NSSE Benchmarks in Predicting Important Educational Outcomes? Change, January 2010.NSSE data is reliable yet there are still concerns about use of the data
  • N=380,000 NSSE dataGeorge Kuh and Robert Gonyea. The Role of the Academic Library in Promoting Student Engagement in Leaning. College & Research Libraries, July 2003, 256-282.
  • The Halo EffectGary Pike – The Constant Error of the Halo – For freshmen – halo error accounts for half of the explained varianceFor seniors – one quarter to one half of the explained varianceSelf-reported gains in student achievement and college experiencesPhil Rosenzweig – The Halo Effect – Jim Collins Good to Great
  • Richard Arum and JosipaRoksa. Academically Adrift: Limited Learning on College Campuses. Chicago: University of Chicago Press, 2011.Almost half (45%) showed no improvement – using CLA before and after data – after 2 years35% showed no improvement over 4 yearsLiberal arts majors do better than other majorsTo gauge summative performance authentically, the CLA presents realistic problems that require students to analyze complex materials and determine the relevance to the task and credibility.  Students' written responses to the tasks are evaluated to assess their abilities to think critically, reason analytically, solve problems and communicate clearly and cogently.  Scores are aggregated to the institutional level to provide a signal to the institution about how their students as a whole are performing. Whether CLA actually measures what it says it measures is another matter
  • Pascarella et al. How Robust Are the Findings of Academically Adrift? Change, May-June 2011CAAP - Collegiate Assessment of Academic Proficiency
  • Megan Oakleaf – RAILS Project
  • Wong Chan and Chu. JAL, July 2006More recently,Reinsfelder. Citation Analysis as a Tool to Measure the Impact of Individual Research Consultants. C&RL, May 2012, 263-277. Found that as number of sources improved, the grade improved yet other research found either no correlations or negative correlations.
  • Kuh & Gonyea. The role of the academic library in promoting student engagement in learning.College & Research Libraries, 64 (7), July 2003, 256-82.300,000 student respondents – NSSE data
  • Sept 2010
  • Megan’s categories – all her suggestions rely on indirect measures or surrogates
  • One study – Gary Reynolds. The Impact of Facilities on Recruitment & Retention of Students. New Directions for Institutional Research, 135, Fall 2007.– showed that the library was the 2nd or 3rd most important reason why a university was selectedPartnering with Student Affairs – campus tour offices – Connecting with parentsMore recently, Lombard The Role of the Academic Library in College Choice. The Journal of Academic Librarianship, July 2012, 237-41.62% lib. no factor, & another 29% the lib. Was only a minor factor. 9% either an important factor or deciding factorGoogle
  • ACRL Value Report possible surrogate measuresChoice of college (from the students perspective) has a big impact on how long it takes to graduate with a BA, onbeing accepted into a high quality grad schoolAs well as amount of income over your lifetime. Private vs public
  • Google
  • ACRL Value Report possible surrogate measures
  • Acculturation – learning how to function (and succeed) in their new environmentTechnology & bureaucracy are the biggest problems
  • So assessment of student learning is difficult So too is the assessment of the library’s contribution to student learning
  • Google 123RF Portrait of happy young people sitting in pub, drinking beer, looking at camera, smiling.
  • Standardized test – Collegiate Assessment of Academic Proficiency (CAAP)Measure of Academic Proficiency and Progress (MAPP)the Collegiate Leaning Assessment (CLA) test has three components:Make an argumentCritique an argumentPerformance task – prepare a briefing reportEssays are scored using a rubricThe institution is the primary unit of analysis
  • Derek Rodriquez – PhD uses rubrics to assess the impact of student capstone projects
  • De Jager 2002, Zhong & Alexander 2007, Julien & Boon 2004, Wong & Webb 2011Jim Self 1987 no correlation between use of reserve collections and gradesLibrary-related interactions – Dickensen 2006Behaviors – Poll 2003, Poll & Payner 2006
  • Good newsUK Library Impact Data Project
  • UK Library Impact Data Project
  • Book borrowing by students at Huddersfield University in the UK
  • University of WollongongData into the Library Cube – R squared = .91The Library Cube provides the information needed to support continuous improvement in three areas: collection development; academic relationships; and marketing.The Library has seen a positive correlation between borrowing activity and academic performance
  • Univ of Wollongong Aus School of CommerceAs use of library increases, grades go up
  • Reported in College & Research Libraries
  • Small study but interesting results
  • Regression Analysis -
  • led to $59 million expansion
  • Hong King Baptist University – Wong and Cmor CR&L Sept 2011
  • Melissa Bowles-Terry. Library Instruction and Academic Success: A Mixed-Methods Assessment of a Library Instruction Program. Evidence Based Library and Information Practice, 7(1), 2012.
  • 6,300 students in the sample ICT skills 2007
  • Many confusing terms to describe the same thingRetention rates range from 60 to 80%Do you know the retention rate for your university?
  • ACRL Value Report surrogate measures
  • In the 2005 book “College Student Retention” edited by Alan Seidman, Linda Hagedornhas a great chapter that explains the differences between various student retention concepts
  • Defined by US National Center for Education Statistics  institutionsretain; individualspersistPersistence: from entry to graduation? From first to second year?Completion: from entry to grad? What about student goals upon entering?Graduation rates: time period? What about transfers? Can only really belong to one institution – so transfer school X captures the student in their graduation rates; beginning school Y calls the student a non-persister or a dropoutAttrition: leaving the system? Leaving the institution?So just a few more things to think about when you are exploring the student retention literature.
  • Retention and drop out rates are NOT dichotomous – students might leave during one time period, but then come backStopout; Drop out; TransferVoluntary / Involuntary – how is this (IS this?) accounted for in student retention stats?Non-persistence is not always a bad thing. Retention theorist Alexander Astin believes student GOALS coming in, and INTENTIONS are most significant. (e.g. did that student only intend to pick up one language course at the community college? Did that student travelling far from home only intend to spend one year at the institution for the experience before transferring to a more affordable institution?
  • On-campus undergraduate students
  • On-campus undergraduate students
  • Carroll et al – graduate distant education student Retention Model
  • Develop a personal connection with the university – especially in the first yearGlasgow Caledonia Univ – highest retention and fastest progression rates among students who have high use of eResources Crawford et al 2004Social integration, academic integrationTemple University – Reasons to drop outLost financial aid/change in financial situation reason for dropping outAlternative textbook project – eBooks
  • Association for the Study of Higher Ed (ASHE) report
  • 2008 AUSSE data – more you use the library more likely to NOT consider leavingDeparture intention“How libraries and librarians can support student engagement”
  • Curtin University – sample of 4,461 students66% had NOT borrowed an item were more likely to withdrawThose that HAD accessed eResources were more likely to remain in schoolStudents with low SES DO use computer workstations in the library
  • Early warning system! In the UK Compare Current vs Dropout
  • Engagement is particularly important for the first-year student
  • Why – data is easily accessibleJane E Hiscock, 1986Long Island UniversityLibrary Expenditures increased retentionHamrick, Schuh, and Shelley, 2004Iowa State University Predicting graduation rates based on resource allocation and other institutional characteristics.Found library expenditures (not counting materials) significantly related to graduation ratesCAUSAL RELATIONSHIP: very difficult to prove!! Focus instead on showing statistically significant differences.Mezick, 2007Long Island University (Brookville, NY)Analyze relationships to both library expenditures and number of professional library staff to student persistence. Strongest relationships:Library expenditures (overall) and retention at undergraduate institutionsLibrary professional staff and retention at doctoral institutions
  • Lloyd A. Kramer & Martha B. Kramer, 1968California State Polytechnic CollegeStatistically significant correlation between library use (in terms of book borrowing) and persistence from 1st to 2nd year<CLICK>Patricia B. Knapp, 1966Monteith College (Wayne State University, MI)Students involved in library skills program showed lower attrition, higher grades
  • Large body of work that supports the idea that campus jobs generally promote persistence – Astin, TerenziniStanley Wilder, 1990; Darla Rushing & Deborah Poole, 2002Louisiana State University & Loyola University, New OrleansRelationship between employment in the library and student completionWill Weston San Diego State Univ 2010
  • The problem however is that almost all studies make the library the focus of the investigation and not student retention (and student success)
  • NSSE - Students tend to over-report their experiences and their skills
  • ACRL Value Report surrogate measures
  • What is the impact of the library on the faculty?Flickr AKMA Seabury Faculty
  • Creaser & Spezi. Working Together: Evolving value for academic libraries. June 2012.
  • Tenopir and Valentine 2012Back files are important
  • Time spent reading represents an exchange value – about 25 days per yearCarol Tenopir ALA Midwinter 2012
  • Ithaka plus Cluff & Murrah 1987 JALFlickr Ben Heine We All Disappear Someday
  • ACRL Value Report surrogate measures
  • Creaser & Spezi. Working Together: Evolving value for academic libraries. June 2012.For researchers - Help with lit searching – huge opportunity
  • Jason Priem – Univ of North Carolina – Chapel Hill Altmetrics– use social media
  • US News & World ReportStudents from better schools get into better graduate schoolsWeiner JAL Jan 2009 Total library expenditure is related to US News Peer Assessment ScoresBudgets down from 3.7% to 2.5% over 10 years
  • Work together to …
  • AustraliaLibrary provides access to information resources that are:
  • Flickr {e u g e n e } Financial Growth
  • Is the library “at the table” with funding decision makersOr is it “on the menu”?
  • What do you think? What is your definition of ROI?
  • In short, ROI is a tool to help with decision-making in either planning or evaluation of services. Most often, it is a measure of the dollar amount of an investment compared to the estimated value in dollars and cents of the outcomes, outputs, or uses of the results. The most effective ROI analyses are, however, best described as stories that weave together qualitative and quantitative assessments of the value of the library and its services.
  • Organizational level – ROI studies often called Library Valuation studiesIn financial circles, ROI often referred to as payback period, net present value, internal rate of return
  • In planning, estimate of measurable outcomes expected to be achieved as a result of investment of resourcesIn evaluating performance, measurement of outcomes achieved as a result of investment of resources
  • CBAConsumer surplus – cost of the vacation = X ringgits, value of the vacation experience = pricelessCost of Time and Effort – “Is it worth my time?”Contingent Valuation – more later
  • Typically, analysis of competing projects- capital expenditure projectsROI is sometimes called ROCE – Return on Capital Employed
  • In library studies, the ratio of benefits compared to costs is usedIn reality, it should be (value of benefits minus cost) divided by costs
  • Don King et al 2004 Contingent valuation ROI = 2.9.1:1
  • Formula ? Convoluted –“Funding does not regenerate funding, but reputation does”TenopirPlus a study in Germany
  • The calculations
  • StoptheMadness:TheInsanityofROIandtheNeedforNewQualitativeMeasuresofAcademicLibrarySuccess.James G. NealACRL National Conference, March 30-April 2, 2011 Philadelphia, Pennsylvania.
  • For-profit university – grown to over 100,000 students – all classes are taught onlineLibrary responsible for linking to online resources PLUSeBookstore (demands at least a 35% reduction in price for all eTextbooks and eliminates all shipping costs)APU ePress creates eTextbooks for high-demand classesOverall library achieves at least a 15:1 ROI
  • Visit the project’s Web site assessment of the library – Bryant University – Don King report Syracuse University – Bruce Kingma (surveys) Several NY universities – Megan Oakleaf – Web-based surveysROI of the journal collection & readership – selected US and UK universities – surveys, critical incident, open-ended Drexel UniversityROI for support of teaching & learning – Univ of Tennessee – survey, focus group, writing rubricROI of digitized special collections – Univ of TennesseeROI of eBooks – COUNTER stats, logbooks, surveys – Univ of Illinois, UCValue of library commons – Univ of Tennessee – surveys of undergrad and grad studentsProject bibliography
  • Access to journal articlesPurpose of reading by faculty – 60% research, 17% teaching, 10% proposals, reportsPurpose of reading by students – 56% course assignment or paper, 20% required reading, 20% class reportFaculty do not get books from the library – 75% from other sourcesReference ROI = 1.50:1Also examined the costs and benefits of providing instruction and access to equipment, photocopiers, AV equipmentSurvey instruments included in the reportStudents do not rely on the library to read books – only 20% do so
  • Faculty & student surveysBruce KingmaLIBValue Project ALA Midwinter 2012
  • Annually, faculty spend 216 hours (27 8-hour days) reading articles, 148 hours reading books, and 84 hours reading other publications
  • What are the values? Using contingent valuation survey and Google analytics
  • eBooks offer value to the library in both a monetary way through documented “usefulness” to patrons – YMMD – Your mileage may differ - use is locally influencedeBooks are less expensive to own, circulate, maintain and preserve than print bookseBooks offer value to patrons, who “esteem” them due to:Accessibility and availability – 24/7, anywherePortabilitySearch and navigation capabilitiesRoom for improvement
  • Survey of students and others who enterUniv of Tennessee library
  • ROI for all of the State of Michigan’s Universities is quite highState funding of $1.5 Billion yield a benefit of $41 BillionYet, even this high ROI has done little to avoid the budget reductions in higher ed
  • How and what you say has a real impact!
  • Impact – use VERBS – Learning, collaborating, assisting, empowering
  • What measures of library success will resonate in your organizational setting?Be visible with your funding decision makers – build personal connectionsIn your environment, how is value defined, measured & communicated?
  • Pain gapGain gap - outcomes
  • The Value Proposition must fill two requirements:Actionable by you and your organizationCredible and compelling to target audienceExamples – It’s the most technologically advanced and robust system on the marketWe improve communication and moraleWe offer training classes in a wide variety of areasMy product was rated the best-in-class by leading authorities
  • Alignment Organization’s mission and goalsWhat is a valuable library? One that contributes to reaching the goals and objectives of the organization.
  • Partner with others
  • Remember that ROI is only one piece of the puzzleFunding for libraries is a reflection of public or campus supportSupport (in the form of your budget) reflects the perceived value of the library to each family, student, faculty member –In short, to your communityHowever, the value is judged in the context of today’s economy and today’s society
  • Tell your story in numbers and stories Your message – “We have contributed towards YOUR goals by ….”Harbor Bridge, Sydney, Australia
  • Use lots of color and excitement to convey both written and spoken stories of the value of the library
  • Had enough bullet points?
  • 85% of US population HS grads; 50+% some college1980 data AA = 6%; College degree = 19% no degree – 75%2010 data AA = 8%, College degree – 19%,
  • For insurance purposes - LC
  • The value of academic libraries

    1. 1. How to Identify Your Impact:The Value of LibrariesJoe MatthewsJune 10, 2013
    2. 2. Megan Oakleaf
    3. 3. Outline• Performance Measures• Value• Value of Information• Value of Information Services
    4. 4. Outline• Value of a Library– Personal value• Direct measures• Indirect measures– Organizational value– Financial value
    5. 5. Few libraries exist in a vacuum,accountable only to themselves. There isthus always a larger context foraccessing library quality, that is, what andhow well does the library contribute toachieving the overall goals of the parentconstituencies?Sarah Pritchard
    6. 6. There is no systematic evidencecollected which shows the valueof academic librariesfor teaching and research staff.Claire Creaser and Valerie Spezi
    7. 7. Performance MeasuresInput ProcessOutput OutcomesOutcomesLibraryServicesIndividual SocietyEfficiency EffectivenessCostEffectivenessImpactVALUECost benefitResourcesCapability Use Beneficial effects
    8. 8. Start with the end in mind:work backwardsRefocus from the activityto the impact
    9. 9. Impact
    10. 10. Library ControlHow much? How many? How economical? How prompt?Magnitude% of change lastyear% of overall changeCostMagnitudeChangeResources usedUnits processedCycle timesTurnaround timeAnticipatory
    11. 11. Library & Customers DecideHow valuable? How reliable? How accurate?Effort expendedCostBenefits obtainedDependabilityAccessAccuracyCompletenessComprehensivenessCurrency
    12. 12. Customers DecideHow well? How courteous? How responsive? How satisfied?AccuracyPromptnessCourtesyExpertiseAttentiveWelcomingAnticipatoryHelpfulEmpatheticExpectations metMaterials obtainedPersonal interactionEase of useEnvironmentComfortWillingness toreturn
    13. 13. Live by the numbers, ….
    14. 14. ChallengesLack of consensus about what should bemeasured and howLack of understanding of performancemeasurement and metricsOrganizational structural issuesLack of precision in measuring performance,andalignment issuesDetermining the “bottom line” is too far awayMajority of stakeholders are too far awayLibrary staff find it difficult to see the “big”
    15. 15. And the survey said ….
    16. 16. Lack of a Connection• Budget and outputs (and outcomes) areseparated• No “bottom line” measure for libraries• Decision-making process is bigger thanthe library• Library has neither champions nor foes• Library benefits are not widely self-evident
    17. 17. Orr’s Fundamental Questions• How good is the library?• What good does the library do?• How well is the library managed?
    18. 18. We should be a bit wary of the “little library”…For when it is good, it is very, very goodand when it is bad,it’s a “pretty good library for a town thissize.”Eleanor Jo Rodger
    19. 19. Levels of Assessment …• Individual student• Course• Departmental/Program• College or University
    20. 20. Types of Measures• Direct– Provide tangible, visible andself-explanatory evidence ofwhat students have & have notlearned• Indirect– Capture students’ perceptions oftheir knowledge & skills;supplement direct measures;sometimes called surrogates
    21. 21. Qualitative Tools• Focus groups – openended• Biography• Phenomenology –capture the “Aha!”moment• Grounded theory• Ethnography• Case study
    22. 22. Qualitative Assessment• Provides in-depth understanding of userresponses and interactions• Represents part of a long-term strategy offormative evaluative
    23. 23. Quantitative Tools• Surveys• Transaction logs• Statistics from systems• Observations (count)
    24. 24. Quantitative Assessment• Analyses to determine library impacts onacademic performance, retention rates• Describe retention rates and GPAs indefined populations over semesters andusers• Compare users & non-users of libraryservices while adjusting for academicpreparation and background differences• Conduct quasi-experimental designsemploying multivariate analysis ofcovariance & hierarchical regression
    25. 25. Useful Assessment
    26. 26. Be cautious about cause-and-effect relationships
    27. 27. The Issue• Is it: Use library resources/services andyou will get better grades.• Or: I want to do well and so I work hard toachieve better grades - and one way I dothat is to use library resources/services.
    28. 28. “Not surprisingly, librarians are keen to showthat the use of expensive, scholarlymaterialspositively correlates with higher grades,although they cannot prove that this is so.”Deborah Goodall & David Pattern
    29. 29. “There is growing pressure on all academiclibrary managers to be more accountable forhow they use limited resources and toachieve institutional outcomes perceivedas important by college and universitystakeholders….”Elizabeth Mezick
    30. 30. Value of Information• Expect value-in-use• Library’s collection reflects a “potentialvalue”• Collection also reflects a “future value”• Value of local collection is declining
    31. 31. Valuable is not about our professionalvalues;in the paradigm of the value of publiclibraries,we are the producers,not the consumers of services.Our sense of what is valuablereally doesn’t mattermuch at all unless itmatches that our our customers.
    32. 32. Fundamental ChangesLibraries have changed more in the past twodecades than in the prior two centuries.Technologyis the major driver . . .We need to recognize that all this changehas only begun, and that change isirreversible.
    33. 33. Increasingly it is importantto remember that librariesprovide few unique services.
    34. 34. Information is woven into ourlives
    35. 35. Quality of InformationThis fast food approach to information consumption driveslibrarians crazy. “Our information is healthier and tastesbettertoo” they shout. But nobody listens. We’re too busyGoogling.”Peter Morville
    36. 36. Key Characteristics ofInformationUncertainty KnowledgeAmbiguity IndeterminacyRedundancy System dependencySharing TimelinessCompression PresentationStability Multiple life cyclesLeakability Substitutability
    37. 37. Criteria for Judging ValueCustomer Criteria Value Added by the ServiceEase of use Browsing, formatting, mediation service,orientation service, ordering, physicalaccessibilityNoise reduction Access (item identification, subjectdescription, subject summary), linkage,precision, selectivityQuality Accuracy, comprehensiveness, currency,reliability, validityAdaptability Closeness to problem, flexibility,simplicity, stimulatoryTime savings Response speedCost savings Cost savings
    38. 38. Collections are disruptedAtoms to bits
    39. 39. Nature of Information isChangingScare, controlledExpensiveShaped by elitesOne-way, massconsumptionSlow movingExternal to ourworldsAll around usCheap or freeShaped by consumersDesigned for sharing,participation &feedbackImmediateEmbedded in ourworldsInformationwas ….Informationis ….
    40. 40. Value of the Academic Library
    41. 41. Ifthe physical proximity ofprint collections had a demonstrableimpact on researcher productivity,no university would hesitate toallocate prime real estateto library stacks.
    42. 42. Traditional Value PropositionWithout a greatlibrary, there canbe no greatuniversity.David Kinly,President ofThe University of Illiniois1929
    43. 43. Universities Provide• Private goods &services– Courses exchanged fortuition– Research completed forfunding• The value propositionThe value to an individual or anorganization determines
    44. 44. Academic Libraries Provide• Public goods andservicesPrint and online resources areshared by all, usually withoutthe exchange of payment• Value propositionThe collective value of all usersmust be estimated to determineif a good or service should be
    45. 45. Value is determined by the userand theuseof information
    46. 46. Astin’s IEO ModelInput OutputEnvironmentProgramsInstitutionalcharacteristicsLibraryFellow studentsFacultyPlace of residence
    47. 47. Student Learning OutcomesModelIntelligenceGeneral ReasoningBroad AbilitiesKnowledge, Understanding, andReasoningAbstract,ProcessOrientedConcrete,Content-Oriented
    48. 48. Define, develop, and measureoutcomesthat contribute toinstitutional effectivenessACRL Standards for Libraries in Higher Education
    49. 49. ChallengeIt is not how much a libraryis used that matters,rather how does the libraryimpact or benefit thecustomer?
    50. 50. Perspectives on ValueBenefitsUseNonuseDirectIndirectOption – Preservation of option forfuture use by meExistence – Perceived value andsignificanceto the communityLegacy – Value of preservation forfuture generationsPersonalOrganizationalFinancialImpacts
    51. 51. Personal
    53. 53. Generic Learning OutcomesKnowledge & Understanding SkillsAttitudes Enjoyment, Inspiration, CreativityActivity, Behavior, Progression
    54. 54. Organizational
    55. 55. • Full-time students• Live on campus• Interact more with faculty• Study more• Collaborate with theirpeersStudent Learning is Affected by…
    56. 56. NSSE• 5 benchmarks of effective educational practice– Level of academic challenge– Active & collaborative learning– Student-Faculty interaction– Enriching educational experiences– Supportive campus environment• Student self-reported gains in intellectual &personal development• No overlap between self-reported data &standardized objective tests
    57. 57. NSSE – Use of the AcademicLibrary• 50% never used the library• Use of libraries at small, academicallychallenging liberal arts colleges arecorrelated with other purposeful activities• Library use less intensive at largeruniversities• Students who work harder use library
    58. 58. Assessing Student Achievement• Direct measures– Capstone experience– Use of a portfolio– Standardized exam (CollegiateLearning Assessment)• Indirect measures
    59. 59. • Gains in studentperformance arequite low• Individual learningis characterizedby persistence• Notable variationwithin and acrossinstitutionsAssessment of Higher Ed
    60. 60. Wabash National Study• Different instrument – CAAP• 2,212 students• Nearly identical results to AcademicallyAdrift• 44 percent no gains in the first year• 33 percent no gains in 4 years• Students only study about 15 hours perweek
    61. 61. • Opinion surveys• Skills testing• Observed behaviors
    62. 62. Bibliographic Instruction• Improvement in basic library skills is themeans and not the end• Yet the means is the focus for evaluationefforts• Evaluation efforts focus on– Opinion surveys– Skills improvement– Pre-test & post-test knowledge• Not the impact on student achievement
    63. 63. Library experiences do not seem todirectlycontribute to gains in informationliteracy,to what students gain overall in college,orto student satisfaction.Kuh & Gonyea
    64. 64. “One way to demonstrate the library’scontribution is to assess whetherstudents’experiences with the librarydirectly or indirectlycontribute to desired outcomes of college.”George D. Kuh & Robert M. Gonyea
    65. 65. How to Demonstrate Impact in…• Student enrollment• Student experiences• Student learning• Student grades (GPA) &achievement• Student retention & graduation• Student career success• Faculty productivity• Institutional reputation• The environment
    66. 66. Student Enrollment
    67. 67. • Recruitment of prospectivestudents• Matriculation of admitted students• Recommendation of currentstudentsStudent Enrollment
    68. 68. StudentLearning
    69. 69. • GPA• Professional/educational testscores• Learning assessments• Faculty judgmentsStudent Learning
    70. 70. Meta-analysis• Entering student characteristics– SES– High school GPA– ACT/SAT scores• Environment - Psychosocial and studyskill factors– Academic goals, skills and self-confidence– Social support & engagement (acculturation)
    71. 71. Student Learning Occurs …• in the classroom• in the laboratory• with peers• in the student union• in the dorm• in the library (for some)• online• and
    72. 72. Direct Measures• Capstone Experience• Portfolios• Standardized Test• Locally-developedtest
    73. 73. Capstone Projects
    74. 74. Indirect Measures• Surveys• Retention, graduation rates• Grades• Acceptance rates intograduate programs• Job placement rates• Exit interviews• Alumni surveys
    75. 75. StudentGrades
    76. 76. Non/Low Use
    77. 77. Usage of Electronic ResourcesWAM = Weighted Average Marks (Grades)
    78. 78. Other Studies• Hong Kong Baptist University• Georgia State University
    79. 79. Hope College
    80. 80. University of MinnesotaGym Bags and MortarboardsUse Campus Recreational FacilitiesAt least 25 times, first-year retention increased1%&5-year graduation ratesincreased 2%
    81. 81. University of Minnesota Library• 5,368 first-year non-transfer students• Use of library was associated with a .23increase in students GPA• More use of the library, GPA also goes up
    82. 82. University of Minnesota Library
    83. 83. Library Instruction and GPA• Surveys of student opinions & habits• Assessing student work for specific skills• Analysis of grade point average• Mixed results
    84. 84. Library Instruction and GPAHong King Baptist University• 45 sample groups – N=31 to 1,223, studymajors• Pairs of data• One-fourth (11) had a positive relationship• Results:– 1 or 2 workshops – little impact on GPA– 3 or 4 workshops – ½ show a positive impact– 5 workshops (1 sample group) – 100% had ahigher GPA
    85. 85. Library Instruction and GPAUniversity of Wyoming Libraries• Analysis of 4,489 transcripts• Slight positive relationship between upper-level library instruction courses and GPA –0.075 GPA difference – that’s less than1/10th of 1 percent
    86. 86. • Research statement – 44%• Evaluate Web site – objectivity –52%– authority – 65%• Presentation to persuade – 12%
    87. 87. How scalable is library instruction?
    88. 88. Student Retention & GraduationAttritionRetentionPersistenceCompletionGraduationRates
    89. 89. • Fall-to-fall retention• Graduation ratesStudent Retention
    90. 90. Retention Concepts• Institutional retention– Enrolling & graduating from the sameinstitution• Program retention– Enrolling & graduating with the samemajor/department/school• System retention– Students who leave one university yetcontinue and complete post-secondarystudies elsewhere
    91. 91. Measures of Retention• Persistence (Continuation rate)– From first to second year? Entry tograduation?• Completion rate– From entry to graduation (Student goals?)• Graduation rates– Are transfers included? Time period?• Attrition– Leaving university? Leaving higher ed?
    92. 92. Measures of Retention• Stopout– Leave university with the intention (andaction) of returning later to complete aprogram• Dropout– Leave university with intention (and action) ofNOT returning• Transfer– Change institutions yet persist in highereducation– May change type of institution
    93. 93. Why Students Leave?• Students’ decision to leave University is influencedby many personal factors– Financial reasons– Family responsibilities– Lack of academic ability– Poor fit, etc.• Foundational Theories from Education / Psychology:– Tinto’s “Model of Student Integration”– Bean’s “Model of Student Attrition”
    94. 94. Tinto’sModel of Student IntegrationPre-entryAttributesGoals /CommitmentsInstitutionalExperiencesIntegration Goals /CommitmentsOutcomeSESSkills &AbilitiesQualityofEducationIntentionsInstitutionalCommitmentsAcademic SystemPerformanceEngagementExtra-CurricularPeer groupSocial SystemAcademicIntegrationSocialIntegrationIntentionsGoalInstitutionalMatchStayorLeave
    95. 95. Bean’sModel of Student AttritionGradesCoursesEducational GoalsMajor & Job CertaintyOpportunity to TransferFamily ApprovalOrganizationalVariablesPersonalVariablesEnvironmentalVariablesLoyalty AttitudesCertaintyPracticalValueIntentDropout
    96. 96. Student Retention & Graduation• Important because … rankings,revenues, educational achievement,emotional well-being• Many reasons for drop-outs are notunder the control of the university• Engagement is the key
    97. 97. Indicators• Student goalattainment• Course retention• Subsequent coursework• Fall-to-fall persistence• Time to degree• Degree completion• Grad schoolenrollment• Transfer rate & success• Employer assessment• Academic value add• Student satisfaction• Professional growth• Student involvement• Citizenship &engagement
    98. 98. Australasian Survey of StudentEngagement (AUSSE)
    99. 99. Curtin University
    100. 100. University of Huddersfield
    101. 101. “A high rate of attritionis indicative of a failureon the part of an institutionto achieve its purpose.”Elizabeth Mezick
    102. 102. Student Engagement“many studentsdon’t develop apersonal connectionwith their institution.And when theydon’t, they leave”Gonzales 2010 NSSE &CSEQ
    103. 103. Library Retention Studies• Statistically significant relationships betweenlibrary expenditures, or staffing levels andstudent retention E.g. Hiscock, 1986Hamrick, Schuh, & Shelley, 2004Mezick, 2007
    104. 104. Graduation Rates & LibraryExpenditures• Used IPEDS data on institutionalcharacteristics & resource allocations• Library expenditures was stronglycorrelated with graduation rates – 1.77percent increase in graduation rates• Greatest payoff is attributable to enhancedlibrary expenditures (+0.92) and instruction(+0.80) while increased non-librarycontributions were quite modest (+0.27)Hamrick, Schuh & Shelley
    105. 105. Library Retention Studies• Relationships between library use(collections) and student retention– Student who borrowed books = more likely topersist• E.g. Kramer & Kramer, 1968• Impact of instruction– Students involved in library skills programsshowed lower attrition rates• E.g. Knapp, 1966
    106. 106. Library Retention StudiesUniversity of Minnesota– 77% of undergrads made use of thelibraries, 85% of grad students made use ofthe libraries– Students who used the library at least oncewere 1.54 times more likely to re-enroll
    107. 107. Library Retention Studies• Some library involvement in first yearexperience programs; specific programs for“at risk” groups– NOT proven to have significant effect• E.g. Hollis, 2001Colton, et al, 2002Aguilar & Keating, 2009Love, 2009
    108. 108. Library Retention Studies• Relationship between library employment& retention– Higher completion rate among library studentworkers• E.g. Wilder, 1990Rushing & Poole, 2002
    109. 109. “If strong linkages between librariesand student retention can be made,then the perceived value of the librarymay indeed rise.”Steven Bell
    110. 110. • Some groups, some majors & seniorsengage in more library-related activities• Academic support expenditures tend tocorrelate with increased engagement• Institutional academic challengecorrelates with library use
    111. 111. Student Career SuccessGrad School ExamsAlumni Surveys
    112. 112. • Job placement rates• First-year job salaries• Professional/graduate schoolacceptance• Internship success• Marketable skillsStudent Success
    113. 113. • Integration of library resources andservices into course syllabi, Websites,lectures, labs, reserve readings, etc.• Faculty/librarian collaborations;cooperative curriculum, assignment, orassessment designFaculty Teaching
    114. 114. Perceived Benefits for Teaching• Savings– Of own time– Of own money– Of other resources• Improvements– Teaching– Course-related materials– Student performance
    115. 115. Impact on Faculty• Library is the source for most journalarticles (individual subscriptions are waydown)• If library subscriptions were unavailable –productivity would decrease 17%• Library is not the source of book readings• 42% of reading material is library provided
    116. 116. Time• Academics spend a lot of time reading• Article reading inspires new thinking,improved results, changed focus• Award-winning academics read more• Academics who publish more use morelibrary resources
    117. 117. Ithaka Studies• Library services notunderstood• Library services not valued• The Library isdisappearing
    118. 118. • Number of publications, number ofpatents, value of technology transfer• Tenure/promotion judgmentsFaculty Research ProductivityFaculty Grants• Number of grant proposals (funded orunfunded)• Value of grants funded
    119. 119. Assessment of Research• Payback model – form of ROI• Research impact• Research utilization ladder• Lavis decision-making impact model• Weiss logic model• HTA organization assessmentframework• Societal impact framework• Research assessment exercise• Becker medical library model
    120. 120. For Most Impact Models• Indicators of research output• Indicators of knowledge transfer• Indicators of implementation• Indicators of community benefit
    121. 121. Faculty Productivity
    122. 122. • Faculty recruitment• Institutional rankings• Community engagementInstitutional Reputation & Prestige
    123. 123. Institutional Reputation• Changes in reputational rankings affectsstudent & faculty recruitment• University budget allocations to librarieshave decreasedSince the library absorbs a very small percentage of auniversity budget, the contribution of the library isdisproportionately high relative to its cost to the institution.Sharon Weiner
    124. 124. University & the Library Can• Attract outstanding faculty• Retain outstanding faculty• Foster innovative research• Align library activities withuniversity goals
    125. 125. • Indispensable for their research• Maintain a high-level overview of their field• Value for money is good• Library not available, costs would increase40%• Take 31% longer to locate same information
    126. 126. Financial
    127. 127. At the table orOn the menu?
    128. 128. What is ROI?
    129. 129. ROI Applications• Projects• Services• Organization
    130. 130. ROI in Library Contexts• Demonstrating the value of libraries• Evaluating existing services, collections,etc.• Making the case for additional services orresources• Recruiting support for a program orinitiative
    131. 131. ROI Terminology• Cost-Benefit Analysis (CBA)– Compares cost or purchase price withestimated value of variables that are difficult tomeasure• Consumer Surplus– Value that consumers place on theconsumption of a good or service in excess ofwhat they paid for it• Cost of Time and Effort– Measures time and effort expended by users• Contingent Valuation
    132. 132. Cost/Benefit Methodologies• Maximize the benefits for given costs• Minimize the costs for a given level ofbenefits• Maximize the ratio of benefits over costs• Maximize the net benefits (present valueof benefits minus the present value ofcosts)• Maximize the internal rate of return
    133. 133. Ratio of Benefits to CostsValue of benefitsdivided byCostsROI = Benefit – CostCost
    134. 134. University of Pittsburgh ROI• If the library’s journal collection (print &electronic) were not available, facultywould use 250,000 hours and $2.1 millionto find alternative sources for the articles• It would cost the university 4.38 timesthe cost of the current library journalcollections for the same amount ofinformation gathering to be carried out
    135. 135. University of Illinois, Urbana-ChampaignConnected citations to resourcesinthe library’s collection tosuccessfulgrant proposals, and the incomethegrants generated
    136. 136. ROI may provide us with calculations thatseek to document a financial relationshipbetween action and benefit, but too oftenin the library community these studies arepoorly constructed, ineffectivelyexecuted, and naïvely communicated.And in the final analysis, do not respondto the legitimate questions being raisedby our administrators and funders, and donot advance the academic library as acritical factor in institutional success.James Neal
    137. 137. • Comprehensive assessment of the library• ROI of the journal collection & readership• ROI for support of teaching & learning• ROI of digitized special collections• ROI of eBooks• Value of library commons• Bibliography
    138. 138. Bryant University – FacultyAccessArticle ROI = 3.2:1Book ROI = 3.5:1Other publications ROI = 3.2:1
    139. 139. Syracuse University – ROI4.49:1Faculty StudentsEconomicIn Person $13.6 $23.1Remote 19.0 14.5TOTAL $32.6 $37.6EnvironmentalRemote access $1.6 $3.7Read not-printed 0.1 0.7Social ? ?TOTAL $34.3 $42.0(Millions of Dollars)
    140. 140. Journal Collections & Reading
    141. 141. ROI for Support of Teaching &LearningPerceived Benefits –• Savings …– Of own time– Of own money– Of other resources – printing, copier• Improvements …– Teaching– Course-related materials– Student performance
    142. 142. ROI of Digitized Special CollectionsUser• What is the value to a user in terms oftime and money spent?Prestige• What is the prestige to the institutionfor high visibility digital specialcollection?Development• What value accrues to the
    143. 143. ROI of Digitized Special CollectionsEnvironmental• What is the value of the environmentalsavings from limited physical access tounique and often fragile material?Scholars• What value accrues from the role of specialcollections in attracting graduate students?Collections• What is the value of digital collection inattracting additional special collections?
    144. 144. ROI of eBooks
    145. 145. The Commons
    146. 146. ROI = 26:1
    147. 147. Communicating Value
    148. 148. Past FutureFocusStructureChangeOutreachDecisionmakingMeasures ofSuccessBuilding &maintainingcollectionsEngaging students &facultyInternal silos University prioritiesIncremental TransformationalTop downPeriodicSharedDeliberativeTraditional Impact
    149. 149. Library Value• How integral it is to the community• How well it supports learning andteaching• How well it supports research
    150. 150. What yourlibrarydoes wellWhat yourcustomersvalueValue Proposition
    151. 151. • An offer, not a demand• Not what you value• Only valuable perspective is thecustomer’s• Valuable in a competitive environmentThe Value Proposition
    152. 152. OrThepromise that a library makes to itscustomers about what they canexpectto receive in return for their time,their effort, their loyalty, andespecially their dollars.
    153. 153. The library needs to …Focus on customersand whatthey want and needas well ashow they want and need it.
    154. 154. Stories + Stats =
    155. 155. Is the value in the glass, the wineor the savoring?
    156. 156. The financial crisis is looking even worse,but you will pleased to know that thedirector reports that the library performancewent up a half a point on the“library goodness scale” last week.Michael Buckland
    157. 157. Joe@JoeMatthews.Orgwww.joematthews.orgJoe MatthewsLibrary Consultant
    158. 158. What Are the ResultsFour YearCollegeFull-timeFour YearCollegePart-timeTwo Year CollegeStopout/TransferCollege degree – 19%AA degree – 8%No college degree –60%No collegeGraduate degree –11%
    159. 159. Valuing the CollectionDewey SubclassNumber of Titles2010-2011 AvgList Total Value ($)001 - Knowledge $76.71 $0.00002 - The book $62.45 $0.00003 - Systems $129.77 $0.00004 - Data processing. Computer science $89.82 $0.00005 - Computer programming, programs, data $69.14 $0.00006 - Special computer methods $83.60 $0.00010 - Bibliography $73.65 $0.00011 - Bibliographies $69.46 $0.00012 - Bibliographies of individuals $0.00 $0.00013 - Of works by specific classes of authors $0.00 $0.00014 - Of anonymous and pseudonymous works $55.95 $0.00015 - Of works from specific places $184.99 $0.00016 - Of works on specific subjects $134.87 $0.00017 - General subject catalogs $0.00 $0.00018 - Catalogs arranged by author, date, etc. $0.00 $0.00019 - Dictionary catalogs $0.00 $0.00020 - Library and information sciences $56.06 $0.00021 - Library relationships $62.60 $0.00022 - Administration of the physical plant $65.50 $0.00023 - Personnel administration $56.00 $0.00025 - Library operations $77.02 $0.00026 - Libraries for specific subjects $89.99 $0.00027 - General libraries $66.25 $0.00