The document provides an analysis of electronic usage statistics in libraries, highlighting trends in user engagement with various digital services from 2007 to 2008. It focuses on the methodological challenges of data collection and interpretation, specifically in the context of measuring usage parameters like homepage visits, database sessions, and article accesses. The findings aim to stimulate further research and discussion on the effectiveness of electronic information services in university settings.

1. 2009 eUsage Stats & Analysis LibMeter - Library eMetrics Workshop ZBW-HH Part 3-4 Nutzungsanalyse elektronischer Informationsdienstleistungen für das praktische Bibliotheks-Management Veranstalter: Berufsverband Information Bibliothek e. V. (BIB) Landesgruppe Hamburg 6. November 2009 Referent: Dr. Peter Ahrens Freier Referent Assistenz: Tanja Haberkorn Beta 0.8 2009-11-01

2. Nutzungsanalyse elektronischer Informationsdienstleistungen für das praktische Bibliotheks-Management Grundlagen Standortbestimmung & Perspektiven Vorstellung Programm Überblick 1. Grundlagen und Begriffe 2. DBS & BIX Vorstelllung praktischer Teil – BIX Übungen 3. Nationale eUsage Jahres-Statistiken 4. COUNTER, OpenURL & u.a. Monatsstatistiken 5. Advanced Topics 6. Diskussion & Feedback

3. LibMeter Seminar 3. WebLibStats x

4. New Thinking required for Virtual Library Services World

5. Rating, Profiling & Charting of eUsage - One Institution over time Ggf. noch erläutern Source: Statistics data published by BIX 2006-2009 ; Meta-Statistical calculations by Author Cave: very small population N=15-18

6. Who is your Online Customer? Many ways to count Users Multiple Definitions for User Base Affiliated Users Primary users (Students, Staff) Guests (registered and others) Special User Groups (faculties, institutes etc.) Active Users Recurring Users (e.g. last 12 months) Power Users (frequent uses) Identification methods Individual Registration (OPAC for Borrowing, ILL, Copies) IP-based Log-In (+/- Proxy; -> Institutional IP-Domains) Other Authentication via single-sign-on, SHIBBOLETH

7. DBS Starts reporting on Electronic Usage in 2007 DBS Variable Analysis (VA) is available since 2004 DBS-VA contains annual data back to 1999 DBS reports on 6 electronic „OUTPUT“-parameters (=use or usage) since 2007

8. Use of Electronic Resources & Online Services in DBS (2007) General Services Visits of Library-Homepage (Zählpixel) = BIX Value OPAC Searches  BIX, Visits of OPAC (Zählpixel) Content Delivery Full Text Downloads of locally licensed articles (COUNTER) Database Search Sessions (DBIS+/-COUNTER) Fulltext Views of locally managed Electronic Documents Context – Naviagtion-Service Links to licensed eJournals (Homepages) (EZB=A-Z List)

9. CAVEAT – Pros and Cons of Statistical Analysis of DBS eUsage Data Issues that may in part seriously limit value of data for Meta-Statistics Still limited Data quality control <-> Improved for 2008 (+ Median is most robust against this) Low percentage valid values <-> Much higher for BIX-matched sample of 41 UBs (see below) Lack of Standardization of some measurements <-> User external measurements and standards where available DBS-#179 OPAC searches should not be evaluated Instead BIX OPAC visits are taken !!

10. Careful new Methodology Approaches Combined DBS & BIX analysis on Univ Libs, i.e. well defined and fairly constant group Focus on Median and indexed Change Take only valid data pairs from both DBS& BIX both years of comparison ->Reduce effect of varying samples Meta-Indicators Represent the average Change as median of individual changes (more indicative)

11. Disclaimer Please do not regard the following Meta-Analysis as ultimate validated hard fact results. This is a first pilot study attempting to reveal more empirical evidence from published „soft data“ Motivation: Disclose potential trends (best available evidence) Stimulate further studies & methodological work Experiment more with visualisation techniques Stimulate discussion, further Analysis & Research

12. Comparing one Group: DE Univ Libs (N= 41) with DBS & BIX over 2007 and 2008

13. Compare within one Group Median Index Method Dutch Philosophy: Compare against the Median / middle and not the best! Slide Presented at Dt. Bibliothekartag Erfurt, 2009; Slide: & Summary data Courtesy of Henk Voorbij

14. Availability of Electronic Usage Data is Still Limited (DBS 2007) Compiled from Data downloaded from: http://www.bibliotheksstatistik.de/eingabe/dynrep/index.php [2009-04-15]

15. Extraction of Data from DBS & BIX Calculation of Medians & Changes Source: Statistics data published by DBS & BIX 2009 ; Meta-Statistical calculations by Author

16. Result Overview – 7x eUsage in DBS & BIX 2007/2008 Source: Statistics data published by DBS & BIX 2008 & 2009 ; Meta-Statistical calculations by Author

17. Interpreting Level of Uses per User Number of Service Events per User per Year Difficult: What are very high numbers for University libraries? Scope ? (50+); High (5+) Logarithmic Scale approach like Richter Scale One level „up“ = raw values * 10 Do not overlook young or niche services With 10.000 users 0,5 per Year means average 50 service events absolute (interpret !) Think about it using „Long Tail“ approach

18. VarB: Comparing and Visualizing Annual Stats Changes between Library Types Author‘s own Meta-Statistics Research Experimental Framework (Excerpt) Source: Author‘s Calculation Framework; anonymized data 3,6% 10,7% 10,5% Median Change Variablitiy 2,4% 0,46 2,4% 10,3% 0,39 10,1% Average Change of Median 1 18% 0,33 +13% 5 11% 0,35 +10% Indicator K 1% 0,67 +2% -18% 0,64 -29% Indicator J -2% 0,46 -2% 3 16% 0,36 +14% Indicator I 4 10% 0,53 +11% 4 15% 0,50 +19% Indicator H Dimension III 2 13% 0,38 +11% 1 23% 0,42 +25% Indicator G 3 11% 0,40 +9% -4% 0,44 -5% Indicator F -2% 0,54 -2% 7% 0,32 +6% Indicator E Dimension II -3% 0,09 -1% 5% 0,07 +1% Indicator D 6% 0,62 +7% 10% 0,60 +15% Indicator C -1% 0,59 -2% 11% 0,40 +11% Indicator B 5 7% 0,60 +9% 11% 0,39 +11% Indicator A Dimension I Dyna-mic-Rank Trend Indi- cator cf. Def. Var Coeff [VC] 2009 Trend- Par. A Median Chng.% [MÄ-A] Dyna-mic-Rank Trend Indi- cator cf. Def. Var Coeff [VC] 2009 Trend- Par. A Median Chng % [MÄ-A] Library Indicator Medians Change from Year n to Year n+1 Dimension Dynamics Lib Group Y Dynamics Lib Group X

19. Interpreting Annual Changes Annual Change Rates Median of indexed Values most robust What is strong growth /Decline Add Color scale Growth class means doubling Growth % (0-2,5%), 5 %, 10 %, 20 %, 40 %, > Breakout Important side question: How many years (months) after introduction Absolute Numbers at what importance level

20. German University Libraries in BIX & DBS – Change 2008 over 07 Source: Statistics data published by BIX 2008 & 2009 ; Meta-Statistical calculations by Author Actions per User per Year 2008 MedianGrowth 2007 to 2008 eUsage Indicator 65,1 432,6 € 7,26 129,3 70,4 9,10 24,5 23,8 1% 1% 2% 4% 9% 17% 24% 40% Homepage Visits per User Library Funds per User in € Journal Accesses per User Comb (Virtual Use per User) OPAC Visits per User* Database Sessions per User Article Views per User Repository Views per User

21. German Universities eService Growth Rating (Medians of paired Indexvalues) Source: Statistics data published by DBS & BIX 2009 ; Meta-Statistical calculations by Author

22. German Universities eService Growth Potential vs. Volume (Level) Source: Statistics data published by DBS & BIX 2009 ; Meta-Statistical calculations by Author

23. Charting Usage of Homepage (Visits) with Magic Quadrant Source: Statistics data published by DBS 2009 ; Meta-Statistical calculations by Author

24. Charting Usage of OPAC (Visits) with Magic Quadrant Source: Statistics data published by DBS & BIX 2009 ; Meta-Statistical calculations by Author

25. Comparing eService indicators internationally over time

26. eService Reporting on Universities* National Library Stats Initiatives *= Not including Universities of Applied Sciences (Polytechs) **= Each official BIX year equals the preivous reporting year of DBS ***= Austria adopted DBS fully in 2008 ****= ZHAW – Zürcher Hochschule für angewandte Wissenschaften Sources: Annual Library statistics of respective countries ; Meta-Statistical calculations by Author 2.000 € No No 170 € (?) No 170 € No Libr. Pays for stat 50% 100% 100% 75% 90% Ca. 35% Ca. 90% % Lib incl. in Stat 50 % 16 2008* 1 (2) BIX A*** 10% 11 2007 6 DBS A*** 100 % Ca. 70 % >95 % Ca. 90 % Ca. 25 % % data submit 2007/8 6 ? ? Priv CH**** 12(14) 2003 3 BFS CH 14 2003 ? 2 UKB NL Ca. 80 2008** 1 (2) BIX DE Ca. 160 2007 6 DBS DE # of Organiz. particip. First Year of repor-ting # of Para-me-ters Stats Body Coun-try

27. Data available to public or members Sources: Annual Library statistics of respective countries ; Meta-Statistical calculations by Author Yes No No Yes No Yes No Libr. Pays for stat No (Yes) Yes BIX A*** No (No) Yes DBS A*** No Yes No Yes Yes Publish indi-vidual values No No Yes (Yes) (No) Publish sum-mary data No Priv CH**** No BFS CH (Yes) UKB NL No BIX DE No DBS DE Publish trends Stats Body Country

28. Swiss Univ. Libraries 2003-2007 Databases, eJournals, Homepage Earliest Logging internationally NB.: Small pupulation (N=12), data completeness 60-80 % Positive (positive & negative) produces erratic AVERAGE curves More robust MEDIAN growth !!! Broad Growth Classification: Primary Data published at: http://www.bfs.admin.ch/ ; All calculations by the Author (unpublished) Very high Service Level, stable +/- 0 56 Homepage 3 High service level, still growth 10 % 30 eJournals 2 High level & Growth = „Tiger-Service“ 25 % 24 Databases 1 Interpretation of figures Av. Growth Rates Level 2007 Events per User/Year eService

29. Conventional & electronic Use Dutch Universities - 2004-2007 Using Science Direct & Web of Knowledge (ISI) core indicators Continuous multi Year Growth 2004 until 2007 Range ca. +12-25 % Adding both Average: +18 % Comp with DE 2008: Database: +17% Articles: +24 % Slide Presented at Dt. Bibliothekartag Erfurt, 2009; Slide: & Summary data Courtesy of Henk Voorbij, KB/UKB

30. Go to Screens List Kaffee Pause

31. LibMeter Seminar 4. COUNTER OpenURL x

32. Which Parties are involved in Generating Service and Stats ? One Party-Scenario (Examples: OPAC hits, Homepage) Service Provider = Library += Raw Statistics Provider = Webserver/ IT Department (+/- allied special statistics service provider) Two Party-Scenario (Example: COUNTER stats) Lib. + Content Provider = Online Publisher / Service Host Three Party Scenario (Examples EZB; OpenURL) Lib. + Content + Web Service provider - Library Computing Service usually Server with powerful integrated specific statistic tools Multi Party-Scenario: (Example BIX Virtual Library Use) Lib. + Content + Web Service + Stats Provider

33. Integrating 3rd Party Services Statistics despite Competition True Library Cooperation Free service offer by consortium or national library community services e.g. EZB Regensburg, Library Computing Centers (BVB, GBV, hbz, hebis, KOBV, SWB) Service partner hosts and operates service in close cooperation with libraries Service partner collects logging data, develops reports & provides usage statistics at various levels as requested by libraries „ Cooptition“ Scenario Free service offer by commercial 3rd Party (e.g. Google Scholar, PubMed) 3rd Party wants to offer localized value-added linking services to users affiliated with scientific institutions Does not want to provide usage statistics to the library Community „forces“ 3rd party to adopt OpenURL standard for the combined service OpenURL-resolver (Linkserver run by library or consortium) generates statistics for library

34. Journal & Database Vendors Collections (COUNTER)

35. COUNTER provides „standardized“ Usage Stats

36. Example: Usage of Subscribed Journals Via COUNTER & SUSHI & ScholarlyStats

37. COUNTER Statistics Benefits & Limits Benefits Working towards Standardization of Online Content Usage Counting Accepted international (commercial Providers) Code of Conduct Available for hundreds of publishers & Database providers Available for most e-Journals Basis for Commercial Discussions with Publishers and Hosts External: With providers Internal: With faculty Limits Commercial Content only OpenAccess, PubMed, Google Scholar missing Expertise & work needed to handle the hundreds and thousands of spreadsheets Needs Tools to handle Data XML-Harvester (SUSHI) Warehouse: ScholarlyStats Standardization Problems No ISO Standard ! Focus on Publishers Content, not Library e-Services Can you trust your Providers ?

38. Standardzed Library Services (e.g. OpenURL)

39. Linking with OpenURL Resolver Principle & URL-Syntax http://sfx.aaa.edu/menu? genre=article&issn=1234-5678 &volume=12&issue=3&spage=1&epage=8&date=1998 &aulast=Smith&aufirst=Paul

40. OpenURL Dynamically Links to the „Appropriate Copies“

41. OpenURL Statistics available ad hoc with Breakdowns - SFX Admin Center Ermittlung der Link-Nutzung in der Datenbank CAPlus für Feb.-Apr. 06 zusammengefaßt monatlich Eingeschränkt nur auf die Gruppe der (echten) universitären Nutzer Ausgabe differenziert nach Dokumentenzahl zu denen elektronischer Volltext geboten werden konnte (incl. Nationallizenzen) versus kein elektronischer Volltext (nur andere SFX-Dienste der Bibliothek) Anmerkung: Dieser Statistik-Typ erlaubt das einfache Abschätzen und Monitoring von Volltext-Abdeckungsgraden für verschiedene Datenbanken, Fachbereiche oder Anwenderungruppen.

42. OpenURL Statistics Benefits & Limits Benefits One global ISO standard across all publishers and content (including free & Open Access) Very broadly used, strong growth Built-In easy Report Generators Service and Context focussed (not just content) Multiple Facets breakdown possible Sources, Targets, Services … Variable Periods Sub-Groups of Users (IP-based or other) distinguishes licensing types (local, consortial, national, free) Limits Not all libraries have OpenURL resolver yet (about 95 % of University Libraries do) Not official DBS-parameter (yet?) Extra Cost of OpenRUL Resolver Service Not all Links go through OpenURL (same applies for other link types) Some sources do not send source-ID -> remain unidentified

43. Innovative and Experimental Examples of Statistics of eServiceUsage

44. Homepage Views @ one University – Annual, seasonal trends & „Events“ Data & Slide to right, Courtesy of UB Dortmund, Graph on left, Authors calcuatlion Cave 2005/10

45. Example Consortial COUNTER Statistics Beck-Online 3 Years Charts Presented at Österr. Bibliothekartag Graz, 2009; Courtesy of Joachim Kreische, ULB Düsseldorf

46. Using COUNTER Inforamtion (Cost per Download + other Criteria B. Mittermaier, Zeitschriftenmanagement an einer wissenschaftlichen Spezialbibliothek, Bibliothekartag Erfurt, 2009-06-03

47. How many years does eService growth last ? OpenURL AMS , MPG, ULB D SFX Growth Compare slides with many years – How long does Ggf. noch ausführen Logging Data courtesy of Nol VERHAGEN & Marc van den BERG, UvA, All Analysis by Author

48. OpenURL Linking as Indicator for differenetial eSource Usage trends Logging Data courtesy of Nol VERHAGEN & Marc van den BERG, UvA, All Analysis by Author

49. Example of experimental Usage-Meta-Report Demonsttrating Shift/trends in Usage of Data-bases via relative OpenURL Source volumes Compiled by Author from published monthly log report summary data published online quarterly by Max-Planck Society

50. Usage of eJournal Linking (EZB) vs. OpenURL Services (SFX)

51. Changes in Use of Linking Services Look at 12-monthly developments Bei den 3 Werten „Kreische“ Handelt es sich wohl doch um SFX TARGET informationen

52. Different Library Linking Services at same Library Data taken from University of Düsseldorf Normalize all absolute counts (see chart) to „Primary Users“ (DBS: 2008 ca. 17.600) Determine key indicators Primary Data published at: http://www.bfs.admin.ch/ ; All calculations by the Author (unpublished) 2006 2004 2001 Service intro-duction Year Low level, very high growth, High Potential + 25 % 3,4 OpenURL Linking (SFX) 3 Medium level, medium growth -> Successful + 10 % 6,3 Database Links (DBIS) 2 High Level, stable – established and mature +/- 0 10,5 eJournal Links (EZB) 1 Interpretation of figures Av. Growth Rates Level 2008 Events p. U/Y eService

53. OpenURL Linking Service Types The „Long Tail“ Story 2007-2009

54. OpenURL-Linking by service Types relative Developments over time

55. Diskussionsfragen Wer schaut sich was von DBS/BIX an ? Wer nutzt was von COUNTER, wie und wozu ? Was kann man alles mit Monatsstatistiken anstellen ? Welche Web-Dienste lasssen sich noch &quot;monitoren&quot; ? Welche Usage Statistiken bieten Konsortien an ? Wie ?