Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

2009 E Usage Stats LibMeter Zbw Hh Workshop


Published on

NatLibStats, dbs, bix, local: Counter OpenURL
Part 3-4 of 5; Beta Version 0.8 of Transparencies
about Management Workshop on Usage analysis of electronic library services
At ZBW-Hamburg 2009-11-06
(C) LibMeter , 2009, Peter Ahrens
all rights reserved

Published in: Technology, Education
  • Be the first to comment

2009 E Usage Stats LibMeter Zbw Hh Workshop

  1. 1. 2009 eUsage Stats & Analysis LibMeter - Library eMetrics Workshop ZBW-HH Part 3-4 Nutzungsanalyse elektronischer Informationsdienstleistungen für das praktische Bibliotheks-Management Veranstalter: Berufsverband Information Bibliothek e. V. (BIB) Landesgruppe Hamburg 6. November 2009 Referent: Dr. Peter Ahrens Freier Referent Assistenz: Tanja Haberkorn Beta 0.8 2009-11-01
  2. 2. Nutzungsanalyse elektronischer Informationsdienstleistungen für das praktische Bibliotheks-Management Grundlagen Standortbestimmung & Perspektiven <ul><li>Vorstellung Programm Überblick </li></ul><ul><li>1. Grundlagen und Begriffe </li></ul><ul><li>2. DBS & BIX Vorstelllung </li></ul><ul><li>praktischer Teil – BIX Übungen </li></ul><ul><li>3. Nationale eUsage Jahres-Statistiken </li></ul><ul><li>4. COUNTER, OpenURL & u.a. Monatsstatistiken </li></ul><ul><li>5. Advanced Topics </li></ul><ul><li>6. Diskussion & Feedback </li></ul>
  3. 3. LibMeter Seminar <ul><li>3. WebLibStats </li></ul>x
  4. 4. New Thinking required for Virtual Library Services World
  5. 5. Rating, Profiling & Charting of eUsage - One Institution over time Ggf. noch erläutern Source: Statistics data published by BIX 2006-2009 ; Meta-Statistical calculations by Author Cave: very small population N=15-18
  6. 6. Who is your Online Customer? Many ways to count Users <ul><li>Multiple Definitions for User Base </li></ul><ul><ul><li>Affiliated Users </li></ul></ul><ul><ul><ul><li>Primary users (Students, Staff) </li></ul></ul></ul><ul><ul><ul><li>Guests (registered and others) </li></ul></ul></ul><ul><ul><ul><li>Special User Groups (faculties, institutes etc.) </li></ul></ul></ul><ul><ul><li>Active Users </li></ul></ul><ul><ul><ul><li>Recurring Users (e.g. last 12 months) </li></ul></ul></ul><ul><ul><ul><li>Power Users (frequent uses) </li></ul></ul></ul><ul><li>Identification methods </li></ul><ul><ul><li>Individual Registration (OPAC for Borrowing, ILL, Copies) </li></ul></ul><ul><ul><li>IP-based Log-In (+/- Proxy; -> Institutional IP-Domains) </li></ul></ul><ul><ul><li>Other Authentication via single-sign-on, SHIBBOLETH </li></ul></ul>
  7. 7. DBS Starts reporting on Electronic Usage in 2007 <ul><li>DBS Variable Analysis (VA) is available since 2004 </li></ul><ul><li>DBS-VA contains annual data back to 1999 </li></ul><ul><li>DBS reports on 6 electronic „OUTPUT“-parameters (=use or usage) since 2007 </li></ul>
  8. 8. Use of Electronic Resources & Online Services in DBS (2007) <ul><li>General Services </li></ul><ul><ul><li>Visits of Library-Homepage (Zählpixel) = BIX Value </li></ul></ul><ul><ul><li>OPAC Searches  BIX, Visits of OPAC (Zählpixel) </li></ul></ul><ul><li>Content Delivery </li></ul><ul><ul><li>Full Text Downloads of locally licensed articles (COUNTER) </li></ul></ul><ul><ul><li>Database Search Sessions (DBIS+/-COUNTER) </li></ul></ul><ul><ul><li>Fulltext Views of locally managed Electronic Documents </li></ul></ul><ul><li>Context – Naviagtion-Service </li></ul><ul><ul><li>Links to licensed eJournals (Homepages) (EZB=A-Z List) </li></ul></ul>
  9. 9. CAVEAT – Pros and Cons of Statistical Analysis of DBS eUsage Data <ul><li>Issues that may in part seriously limit value of data for Meta-Statistics </li></ul><ul><ul><li>Still limited Data quality control <-> Improved for 2008 (+ Median is most robust against this) </li></ul></ul><ul><ul><li>Low percentage valid values <-> Much higher for BIX-matched sample of 41 UBs (see below) </li></ul></ul><ul><ul><li>Lack of Standardization of some measurements <-> User external measurements and standards where available </li></ul></ul><ul><ul><ul><li>DBS-#179 OPAC searches should not be evaluated </li></ul></ul></ul><ul><ul><ul><li>Instead BIX OPAC visits are taken !! </li></ul></ul></ul>
  10. 10. Careful new Methodology Approaches <ul><li>Combined DBS & BIX analysis on Univ Libs, i.e. well defined and fairly constant group </li></ul><ul><li>Focus on Median and indexed Change </li></ul><ul><li>Take only valid data pairs from both DBS& BIX both years of comparison ->Reduce effect of varying samples Meta-Indicators </li></ul><ul><li>Represent the average Change as median of individual changes (more indicative) </li></ul>
  11. 11. Disclaimer <ul><li>Please do not regard the following Meta-Analysis as ultimate validated hard fact results. </li></ul><ul><li>This is a first pilot study attempting to reveal more empirical evidence from published „soft data“ </li></ul><ul><li>Motivation: </li></ul><ul><li>Disclose potential trends (best available evidence) </li></ul><ul><li>Stimulate further studies & methodological work </li></ul><ul><li>Experiment more with visualisation techniques </li></ul><ul><li>Stimulate discussion, further Analysis & Research </li></ul>
  12. 12. Comparing one Group: DE Univ Libs (N= 41) with DBS & BIX over 2007 and 2008
  13. 13. Compare within one Group Median Index Method <ul><li>Dutch Philosophy: Compare against the Median / middle and not the best! </li></ul>Slide Presented at Dt. Bibliothekartag Erfurt, 2009; Slide: & Summary data Courtesy of Henk Voorbij
  14. 14. Availability of Electronic Usage Data is Still Limited (DBS 2007) Compiled from Data downloaded from: [2009-04-15]
  15. 15. Extraction of Data from DBS & BIX Calculation of Medians & Changes Source: Statistics data published by DBS & BIX 2009 ; Meta-Statistical calculations by Author
  16. 16. Result Overview – 7x eUsage in DBS & BIX 2007/2008 Source: Statistics data published by DBS & BIX 2008 & 2009 ; Meta-Statistical calculations by Author
  17. 17. Interpreting Level of Uses per User <ul><li>Number of Service Events per User per Year </li></ul><ul><ul><li>Difficult: What are very high numbers for University libraries? Scope ? (50+); High (5+) </li></ul></ul><ul><ul><li>Logarithmic Scale approach </li></ul></ul><ul><ul><ul><li>like Richter Scale </li></ul></ul></ul><ul><ul><ul><li>One level „up“ = raw values * 10 </li></ul></ul></ul><ul><ul><li>Do not overlook young or niche services </li></ul></ul><ul><ul><ul><li>With 10.000 users 0,5 per Year means average 50 service events absolute (interpret !) </li></ul></ul></ul><ul><ul><ul><li>Think about it using „Long Tail“ approach </li></ul></ul></ul>
  18. 18. VarB: Comparing and Visualizing Annual Stats Changes between Library Types <ul><li>Author‘s own Meta-Statistics Research </li></ul><ul><li>Experimental Framework (Excerpt) </li></ul>Source: Author‘s Calculation Framework; anonymized data   3,6%       10,7%   10,5%     Median Change Variablitiy   2,4% 0,46 2,4%   10,3% 0,39 10,1%     Average Change of Median   1 18% 0,33 +13% 5 11% 0,35 +10%   Indicator K   1% 0,67 +2%   -18% 0,64 -29% Indicator J   -2% 0,46 -2% 3 16% 0,36 +14% Indicator I 4 10% 0,53 +11% 4 15% 0,50 +19%     Indicator H Dimension III 2 13% 0,38 +11% 1 23% 0,42 +25%     Indicator G 3 11% 0,40 +9%   -4% 0,44 -5% Indicator F   -2% 0,54 -2%   7% 0,32 +6% Indicator E Dimension II   -3% 0,09 -1%   5% 0,07 +1%   Indicator D   6% 0,62 +7%   10% 0,60 +15% Indicator C   -1% 0,59 -2%   11% 0,40 +11%   Indicator B 5 7% 0,60 +9%   11% 0,39 +11%   Indicator A Dimension I Dyna-mic-Rank Trend Indi- cator cf. Def. Var Coeff [VC] 2009 Trend- Par. A Median Chng.% [MÄ-A] Dyna-mic-Rank Trend Indi- cator cf. Def. Var Coeff [VC] 2009 Trend- Par. A Median Chng % [MÄ-A] Library Indicator Medians Change from Year n to Year n+1 Dimension   Dynamics Lib Group Y   Dynamics Lib Group X  
  19. 19. Interpreting Annual Changes <ul><li>Annual Change Rates </li></ul><ul><ul><li>Median of indexed Values most robust </li></ul></ul><ul><ul><li>What is strong growth /Decline </li></ul></ul><ul><ul><ul><li>Add Color scale </li></ul></ul></ul><ul><ul><li>Growth class means doubling Growth % </li></ul></ul><ul><ul><li>(0-2,5%), 5 %, 10 %, 20 %, 40 %, > Breakout </li></ul></ul><ul><ul><li>Important side question: </li></ul></ul><ul><ul><ul><li>How many years (months) after introduction </li></ul></ul></ul><ul><ul><ul><li>Absolute Numbers at what importance level </li></ul></ul></ul>
  20. 20. German University Libraries in BIX & DBS – Change 2008 over 07 Source: Statistics data published by BIX 2008 & 2009 ; Meta-Statistical calculations by Author Actions per User per Year 2008 MedianGrowth 2007 to 2008 eUsage Indicator 65,1 432,6 € 7,26 129,3 70,4 9,10 24,5 23,8 1% 1% 2% 4% 9% 17% 24% 40% Homepage Visits per User Library Funds per User in € Journal Accesses per User Comb (Virtual Use per User) OPAC Visits per User* Database Sessions per User Article Views per User Repository Views per User
  21. 21. German Universities eService Growth Rating (Medians of paired Indexvalues) Source: Statistics data published by DBS & BIX 2009 ; Meta-Statistical calculations by Author
  22. 22. German Universities eService Growth Potential vs. Volume (Level) Source: Statistics data published by DBS & BIX 2009 ; Meta-Statistical calculations by Author
  23. 23. Charting Usage of Homepage (Visits) with Magic Quadrant Source: Statistics data published by DBS 2009 ; Meta-Statistical calculations by Author
  24. 24. Charting Usage of OPAC (Visits) with Magic Quadrant Source: Statistics data published by DBS & BIX 2009 ; Meta-Statistical calculations by Author
  25. 25. Comparing eService indicators internationally over time
  26. 26. eService Reporting on Universities* National Library Stats Initiatives *= Not including Universities of Applied Sciences (Polytechs) **= Each official BIX year equals the preivous reporting year of DBS ***= Austria adopted DBS fully in 2008 ****= ZHAW – Zürcher Hochschule für angewandte Wissenschaften Sources: Annual Library statistics of respective countries ; Meta-Statistical calculations by Author 2.000 € No No 170 € (?) No 170 € No Libr. Pays for stat 50% 100% 100% 75% 90% Ca. 35% Ca. 90% % Lib incl. in Stat 50 % 16 2008* 1 (2) BIX A*** 10% 11 2007 6 DBS A*** 100 % Ca. 70 % >95 % Ca. 90 % Ca. 25 % % data submit 2007/8 6 ? ? Priv CH**** 12(14) 2003 3 BFS CH 14 2003 ? 2 UKB NL Ca. 80 2008** 1 (2) BIX DE Ca. 160 2007 6 DBS DE # of Organiz. particip. First Year of repor-ting # of Para-me-ters Stats Body Coun-try
  27. 27. Data available to public or members Sources: Annual Library statistics of respective countries ; Meta-Statistical calculations by Author Yes No No Yes No Yes No Libr. Pays for stat No (Yes) Yes BIX A*** No (No) Yes DBS A*** No Yes No Yes Yes Publish indi-vidual values No No Yes (Yes) (No) Publish sum-mary data No Priv CH**** No BFS CH (Yes) UKB NL No BIX DE No DBS DE Publish trends Stats Body Country
  28. 28. Swiss Univ. Libraries 2003-2007 Databases, eJournals, Homepage <ul><li>Earliest Logging internationally </li></ul><ul><li>NB.: Small pupulation (N=12), data completeness 60-80 % </li></ul><ul><li>Positive (positive & negative) produces erratic AVERAGE curves </li></ul><ul><li>More robust MEDIAN growth !!! </li></ul><ul><li>Broad Growth Classification: </li></ul>Primary Data published at: ; All calculations by the Author (unpublished) Very high Service Level, stable +/- 0 56 Homepage 3 High service level, still growth 10 % 30 eJournals 2 High level & Growth = „Tiger-Service“ 25 % 24 Databases 1 Interpretation of figures Av. Growth Rates Level 2007 Events per User/Year eService
  29. 29. Conventional & electronic Use Dutch Universities - 2004-2007 <ul><li>Using Science Direct & Web of Knowledge (ISI) core indicators </li></ul><ul><li>Continuous multi Year Growth 2004 until 2007 </li></ul><ul><ul><li>Range ca. +12-25 % </li></ul></ul><ul><li>Adding both </li></ul><ul><ul><li>Average: +18 % </li></ul></ul><ul><li>Comp with DE 2008: </li></ul><ul><ul><li>Database: +17% </li></ul></ul><ul><ul><li>Articles: +24 % </li></ul></ul>Slide Presented at Dt. Bibliothekartag Erfurt, 2009; Slide: & Summary data Courtesy of Henk Voorbij, KB/UKB
  30. 30. Go to Screens List <ul><ul><li>Kaffee </li></ul></ul><ul><ul><li>Pause </li></ul></ul>
  31. 31. LibMeter Seminar <ul><li>4. COUNTER OpenURL </li></ul>x
  32. 32. Which Parties are involved in Generating Service and Stats ? <ul><li>One Party-Scenario (Examples: OPAC hits, Homepage) </li></ul><ul><ul><li>Service Provider = Library </li></ul></ul><ul><ul><li>+= Raw Statistics Provider = Webserver/ IT Department </li></ul></ul><ul><ul><li>(+/- allied special statistics service provider) </li></ul></ul><ul><li>Two Party-Scenario (Example: COUNTER stats) </li></ul><ul><ul><li>Lib. + Content Provider = Online Publisher / Service Host </li></ul></ul><ul><li>Three Party Scenario (Examples EZB; OpenURL) </li></ul><ul><ul><li>Lib. + Content + Web Service provider - Library Computing Service </li></ul></ul><ul><ul><ul><li>usually Server with powerful integrated specific statistic tools </li></ul></ul></ul><ul><li>Multi Party-Scenario: (Example BIX Virtual Library Use) </li></ul><ul><ul><li>Lib. + Content + Web Service + Stats Provider </li></ul></ul>
  33. 33. Integrating 3rd Party Services Statistics despite Competition <ul><li>True Library Cooperation </li></ul><ul><li>Free service offer by consortium or national library community services </li></ul><ul><ul><li>e.g. EZB Regensburg, </li></ul></ul><ul><ul><li>Library Computing Centers (BVB, GBV, hbz, hebis, KOBV, SWB) </li></ul></ul><ul><li>Service partner hosts and operates service in close cooperation with libraries </li></ul><ul><li>Service partner collects logging data, develops reports & provides usage statistics at various levels as requested by libraries </li></ul><ul><li>„ Cooptition“ Scenario </li></ul><ul><li>Free service offer by commercial 3rd Party (e.g. Google Scholar, PubMed) </li></ul><ul><li>3rd Party wants to offer localized value-added linking services to users affiliated with scientific institutions </li></ul><ul><li>Does not want to provide usage statistics to the library </li></ul><ul><li>Community „forces“ 3rd party to adopt OpenURL standard for the combined service </li></ul><ul><li>OpenURL-resolver (Linkserver run by library or consortium) generates statistics for library </li></ul>
  34. 34. Journal & Database Vendors Collections (COUNTER)
  35. 35. COUNTER provides „standardized“ Usage Stats
  36. 36. Example: Usage of Subscribed Journals Via COUNTER & SUSHI & ScholarlyStats
  37. 37. COUNTER Statistics Benefits & Limits <ul><li>Benefits </li></ul><ul><li>Working towards Standardization of Online Content Usage Counting </li></ul><ul><li>Accepted international (commercial Providers) Code of Conduct </li></ul><ul><li>Available for hundreds of publishers & Database providers </li></ul><ul><ul><li>Available for most e-Journals </li></ul></ul><ul><li>Basis for Commercial Discussions with Publishers and Hosts </li></ul><ul><ul><li>External: With providers </li></ul></ul><ul><ul><li>Internal: With faculty </li></ul></ul><ul><li>Limits </li></ul><ul><li>Commercial Content only </li></ul><ul><ul><li>OpenAccess, PubMed, Google Scholar missing </li></ul></ul><ul><li>Expertise & work needed to handle the hundreds and thousands of spreadsheets </li></ul><ul><li>Needs Tools to handle Data </li></ul><ul><ul><li>XML-Harvester (SUSHI) </li></ul></ul><ul><ul><li>Warehouse: ScholarlyStats </li></ul></ul><ul><li>Standardization Problems </li></ul><ul><ul><li>No ISO Standard ! </li></ul></ul><ul><li>Focus on Publishers Content, not Library e-Services </li></ul><ul><li>Can you trust your Providers ? </li></ul>
  38. 38. Standardzed Library Services (e.g. OpenURL)
  39. 39. Linking with OpenURL Resolver Principle & URL-Syntax genre=article&issn=1234-5678 &volume=12&issue=3&spage=1&epage=8&date=1998 &aulast=Smith&aufirst=Paul
  40. 40. OpenURL Dynamically Links to the „Appropriate Copies“
  41. 41. OpenURL Statistics available ad hoc with Breakdowns - SFX Admin Center <ul><li>Ermittlung der Link-Nutzung in der Datenbank CAPlus für Feb.-Apr. 06 </li></ul><ul><li>zusammengefaßt monatlich </li></ul><ul><li>Eingeschränkt nur auf die Gruppe der (echten) universitären Nutzer </li></ul><ul><li>Ausgabe differenziert nach Dokumentenzahl zu denen elektronischer Volltext geboten werden konnte (incl. Nationallizenzen) versus kein elektronischer Volltext (nur andere SFX-Dienste der Bibliothek) </li></ul><ul><li>Anmerkung: Dieser Statistik-Typ erlaubt das einfache Abschätzen und Monitoring von Volltext-Abdeckungsgraden für verschiedene Datenbanken, Fachbereiche oder Anwenderungruppen. </li></ul>
  42. 42. OpenURL Statistics Benefits & Limits <ul><li>Benefits </li></ul><ul><li>One global ISO standard across all publishers and content (including free & Open Access) </li></ul><ul><li>Very broadly used, strong growth </li></ul><ul><li>Built-In easy Report Generators </li></ul><ul><li>Service and Context focussed (not just content) </li></ul><ul><li>Multiple Facets breakdown possible </li></ul><ul><ul><li>Sources, Targets, Services … </li></ul></ul><ul><ul><li>Variable Periods </li></ul></ul><ul><ul><li>Sub-Groups of Users (IP-based or other) </li></ul></ul><ul><li>distinguishes licensing types (local, consortial, national, free) </li></ul><ul><li>Limits </li></ul><ul><li>Not all libraries have OpenURL resolver yet (about 95 % of University Libraries do) </li></ul><ul><li>Not official DBS-parameter (yet?) </li></ul><ul><li>Extra Cost of OpenRUL Resolver Service </li></ul><ul><li>Not all Links go through OpenURL (same applies for other link types) </li></ul><ul><li>Some sources do not send source-ID -> remain unidentified </li></ul>
  43. 43. Innovative and Experimental Examples of Statistics of eServiceUsage
  44. 44. Homepage Views @ one University – Annual, seasonal trends & „Events“ Data & Slide to right, Courtesy of UB Dortmund, Graph on left, Authors calcuatlion Cave 2005/10
  45. 45. Example Consortial COUNTER Statistics Beck-Online 3 Years Charts Presented at Österr. Bibliothekartag Graz, 2009; Courtesy of Joachim Kreische, ULB Düsseldorf
  46. 46. Using COUNTER Inforamtion (Cost per Download + other Criteria B. Mittermaier, Zeitschriftenmanagement an einer wissenschaftlichen Spezialbibliothek, Bibliothekartag Erfurt, 2009-06-03
  47. 47. How many years does eService growth last ? <ul><li>OpenURL AMS , MPG, ULB D SFX Growth </li></ul><ul><li>Compare slides with many years – </li></ul><ul><li>How long does </li></ul>Ggf. noch ausführen Logging Data courtesy of Nol VERHAGEN & Marc van den BERG, UvA, All Analysis by Author
  48. 48. OpenURL Linking as Indicator for differenetial eSource Usage trends Logging Data courtesy of Nol VERHAGEN & Marc van den BERG, UvA, All Analysis by Author
  49. 49. Example of experimental Usage-Meta-Report Demonsttrating Shift/trends in Usage of Data-bases via relative OpenURL Source volumes Compiled by Author from published monthly log report summary data published online quarterly by Max-Planck Society
  50. 50. Usage of eJournal Linking (EZB) vs. OpenURL Services (SFX)
  51. 51. Changes in Use of Linking Services Look at 12-monthly developments Bei den 3 Werten „Kreische“ Handelt es sich wohl doch um SFX TARGET informationen
  52. 52. Different Library Linking Services at same Library <ul><li>Data taken from University of Düsseldorf </li></ul><ul><li>Normalize all absolute counts (see chart) to „Primary Users“ (DBS: 2008 ca. 17.600) </li></ul><ul><li>Determine key indicators </li></ul>Primary Data published at: ; All calculations by the Author (unpublished) 2006 2004 2001 Service intro-duction Year Low level, very high growth, High Potential + 25 % 3,4 OpenURL Linking (SFX) 3 Medium level, medium growth -> Successful + 10 % 6,3 Database Links (DBIS) 2 High Level, stable – established and mature +/- 0 10,5 eJournal Links (EZB) 1 Interpretation of figures Av. Growth Rates Level 2008 Events p. U/Y eService
  53. 53. OpenURL Linking Service Types The „Long Tail“ Story 2007-2009
  54. 54. OpenURL-Linking by service Types relative Developments over time
  55. 55. Diskussionsfragen <ul><ul><li>Wer schaut sich was von DBS/BIX an ? </li></ul></ul><ul><ul><li>Wer nutzt was von COUNTER, wie und wozu ? </li></ul></ul><ul><ul><li>Was kann man alles mit Monatsstatistiken anstellen ? </li></ul></ul><ul><ul><li>Welche Web-Dienste lasssen sich noch &quot;monitoren&quot; ? </li></ul></ul><ul><ul><li>Welche Usage Statistiken bieten Konsortien an ? Wie ? </li></ul></ul>