Collection evaluation techniques for academic libraries


Published on

Sally Halper, Lead Content Specialist - Business & Management, British Library. An excellent introduction to some really good practical qualitative and quantitative tools including White's brief tests. A bibliography of further readings is also provided.

1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Welcome, and introduce myself. Explain would work just as well for Special Libraries.
  • One of the considerations when planning your evaluation is the PURPOSE and the likely IMPACT on the collection. Often, evaluations are done to support business cases to maintain expenditure or to manage the impact of budget cuts on the collection. More positively, you may be testing your collection to see how well it compares with other similar libraries to plan future collection development and to be able to promote your collection’s strengths. Or you may be part of a collaborative collecting arrangement already for one part of your collection, and want to see if a collaborative arrangement might work for another area. In the budget cutting scenario, if you’ve been asked to make a one-off saving that will only affect one financial year, that’s one thing. But if what you’re being asked to do is actually scale back acquisitions for several years to come, you might have to do more detailed work looking at the impact of that, and more investigation of alternative sources for your users and whether they can be relied on to still have that collection in a few years’ time.
  • I’m not going to talk about all of these, but I will say something about most of them and focus on the one in red, brief tests of collection strength.
  • All the methods I’ve come across basically involve drawing up a list of titles, against which you compare your library’s collection. The methodology can be simple – where you use a list that’s already been defined such as a publisher’s catalogue or the holdings of another library who you’ve chosen to benchmark against – to more complex methods such as the Brief Test Methodology developed in the mid 1990s by Howard White, a librarian at . I’m going to tell you a bit more about White’s Brief Test methodology in a minute, but in case you’re wondering why I’ve suggested you ask your research-active Faculty to suggest titles, that’s a political move – they may be good advocates for your cause, as well as a good source of specialist subject knowledge.
  • Before you start reading this slide – a word of explanation. The 40-title list is NOT a sample in the usual sense, that is it is not used to estimate within known margins of error, what percentage of the bibliography of a subject some library holds (i.e. its coverage). It’s more like an ability test – if you think of yourself setting the questions for a maths or spelling test which your collection has to sit, you’re setting 10 easy questions (Level 1), 10 that are a bit tougher (Level 2), 10 that are more demanding still (Level 3) and 10 that are quite hard (Level 4) you’ll get the idea. If you want to measure the fraction of a subject literature that your library holds, then you’ll need to use a much longer checklist. Conspectus – ask if they’ve heard of it
  • Watch what your users do and watch their expression – for instance, my management researchers’ eyes light up at our journal display and they cannot resist fondling the journals when they are shown them on their first visit.
  • Say ‘I will show you BNB Online on the next slide’… Current Serials Received: Free to search, on BL website. You can browse by title or Search it. The search is a bit basic, but it is a quick way to see if we have a lending copy (DSC). Personally I would always search the main Integrated Catalogue as well, because not all of our serials are on Current Serials Received, and our Holdings are improving as we take in more low-use serials from Universities as part of the UK Research Reserve.
  • Exclusions include: Official Publications, Looseleafs, Straight reprints of an earlier edition where the publisher and format is unchanged, and export editions published in the UK but which are not available for sale in this country. Electronic resources include SOME grey literature published on the web which we have deliberately added to our collections.
  • Slightly adapted White’s Brief Test methodology, because - being the national collection - I was pretty confident we’d ‘pass’ the first 3 levels, so I wanted to spend more time checking our holdings of the more advanced, research-level texts.
  • Usage data – bit unsatisfactory, really…
  • 3 rd point: LBS sees itself as an international BS so Gill Dwyer compares her stock against that of the leading American and European Business Schools. Maybe there’s a subject of professional group you could work with e.g. BBSLG, MLG…
  • Howard White’s book is OOP but costs about £27 second hand on Amazon, or you can consult it in the British Library reading rooms in London.
  • Collection evaluation techniques for academic libraries

    1. 1. Collection evaluationtechniques for academiclibrariesSally HalperLead Content Specialist – Business & ManagementSocial Sciences Collections & ResearchThe British
    2. 2. This presentation covers1. Collection evaluation (sometimes called collection mapping)2. Some techniques and tools3. Example4. Further considerations5. Further reading6. Questions and answers 2
    3. 3. 1. Collection evaluation Evaluate: a) The collection b) Its use The purpose influences your choice of method and how much detail you go into.For instance: Do you need to justify spend/budgets for next year? For future years? As a benchmark that guides collection development for the next 5 years or more?? 3
    4. 4. 2. Some techniques and tools Collection based Use or user basedQuantitative Collection size/growth Interlibrary loan statistics Materials budget size/growth Circulation stats Collection size standards and In-house use statistics formulas Document delivery statistics Shelf availability stats/requestsQualitative List checking User opinion surveys Verification studies Citation analysis User observation Direct collection checking Collection mapping (assigning Focus groups Conspectus levels) Brief tests of collection strengthSource: Johnson 2009 p. 270 4
    5. 5. 2. Some techniques and tools a) How good is your collection? White’s Brief Test methodology “The two most important determinants of quality in librarycollections are money, which is everyone’s favourite measure,and love, which is unquantifiable... this brief test registers theeffect of their combined power” White (1995) p.3 A ‘quality’ list e.g. The ABS Journal Quality Guide A simple list – either one you draw up from scratch (ask yourresearchers to suggest titles) or a benchmarke.g. Testing BL’s holdings for the subject of Leadership against aspecial library for management, Roffey Park Institute Library 5
    6. 6. White’s Brief Test methodology(see White, H. D. (1995) Brief Tests of Collection Strength)1. Decide on the level of granularity of your test Level Conspectus definition e.g. Economics as a whole, or just Econometrics, or just a topic within 1 Minimal Econometrics?2. Compile a list of 40 titles*: 2 Basic 10 at each of Conspectus Level 1, 2, 3 and 4. 3 Instructional You decide the Level of a title partly on the 4 Research basis of your expertise and partly on the basis of how many other libraries hold it. For 5 Comprehensive academic libraries, the best guide may be the number of COPAC libraries who hold it, so Must have at least 50% of titles you’re comparing like with like. to ‘pass’ a level *You can use more titles e.g. 20 at each level. Aim is to find the highest level passed3. Test your holdings against the list Bit like ‘the long tail’ – do highest level items circulate the least? 6
    7. 7. b) Evaluating use1. Quantitative: Usage statistics: circulation stats, inter-library loan stats, requests, sample surveys of open access/reference stock use (users tick or sign to show use)2. Qualitative: User opinion surveys, User observation, Focus groups Consult Faculty, PHD students etc3. Is there a copy in another library that’s accessible - and that your users would go and use? What are the access barriers? Cost to borrow? Will it still be available in 5 years’ time?4. Work out a rough cost of not owning it e.g. ILL charge x no. users x no. uses a year = £ x5. Factor in the cost of delay (up to 6 weeks if the item is out on loan) and if you’re feeling frisky, the possible consequences e.g. loss of research funding bid, loss of only extant copy 7
    8. 8. Tools include BNB: on subscription or via BLcatalogue subset BL catalogues, including CurrentSerials Received 60,000 titles (BLDSC and STM collections) Ulrich’s Periodicals Directory  SUNCAT Index of Conference Proceedings  Publishers’ catalogues and websites Blackwells’ Collection Manager  (Other) University reading lists Nielsen Bookdata  Benchmark library catalogues COPAC (includes BL records)  …and fellow subject librarians Worldcat, Amazon 8
    9. 9. BNB Online - on Free to search as subset of BLIntegrated Catalogue at – you need tolog in with your reader pass number Focuses on mainstreammonographs available throughnormal book buying channels.See exclusions policy at Records for UK online electronicresources began to appear in theBNB from the third week of February2003 9
    10. 10. 3. Example: Seminal texts in Management & Business Studies (MBS)First, checked we had 10 titles at each of Level 1 (minimal, I used ‘ProjectManagement for Dummies’ type texts), 2 (basic e.g. Penguin Dictionary ofBusiness) and Level 3 (instructional, used London Universities’ readinglists to identify current core texts for UG courses).Then, compiled test list from 2 well-known bibliographies of ‘classic’ managementResearch – 80 research-level monos (40 UK and 40 US) from 1950s-2000s: 10
    11. 11. 3. Example: Seminal texts in Management &Business Studies (MBS)Found the collections are Level 4 (88% titles held at Level 4)But there were 9 titles we did not hold at all (2 UK, 7 published in the USA), which wewent on to acquireUsage ‘data’Very hard to get from BL systems – DSC no longer ‘wand’ each loan and ABRS datais only available since 2004.Evidence (of sorts) from looking at wear and tear on the books (about half of the 80titles show signs of heavy use, Peter Senge’s The Fifth Discipline had to be sent forrepair)However, evidence from focus groups (c80 management researchers in 2007)suggests most buy these books if they’re still in print, if they cannot get them fromtheir own University (or other) Library. Acquisitive lot!So maybe use of BL copies is mainly by students, or people without UniversityLibraries e.g. practitioners, managers in NGOs? 11
    12. 12. 4. Further considerations Researcher behaviour is changing: ‘satisficing’ or unofficialmethods of getting literature are more prevalent and library use isdeclining Does it matter who is using your collection? Are some users seenas ‘better’ than others? Benchmarking clubs: choosing who to compare your collectionagainst, doing collection evaluation as a joint activity Collaborative collection development or management e.g. UKRRfor low-use serials, special collections BUT can you guarantee preservation and access in the long term? 12
    13. 13. 5. Further reading White, H. D. (1995) Brief Tests of Collection Strength. London:Greenwood Press. ISBN 978-0313297533 Twiss, T.M. A validation of brief tests of collection strength.Collection Management 25 (3) 2001 pp. 23-37 Beals, J. B. Assessing library collections using the brief testmethodology. Electronic Journal of Academic & SpecialLibrarianship 7 (3) 2006. Available online at Chapter 9 Collection Analysis: Evaluation and Assessment. InJohnson, P. (2009) Fundamentals of Collection Management (2nded.) Chicago: ALA. ISBN 978-0838909720 13
    14. 14. 6. Any questions?For help with enquiries: 020 7412 7676 10am-8pm Monday -Thursday 10am-5pm Friday & Saturday The British Library, 96 Euston Road, London NW1 2DB Thank you 14