Striving for Excellence with Electronic Library Statistics Indicators for Performance Evaluation Introduction - Basics Institut für Informationswissenschaft, University of Applied Sciences, Cologne (FH-Köln IIW) Seminar on Library Evaluation (S. FÜHLES-UBACH) Peter Ahrens Guest Lecture V 1.00a – 2009-05-18 Cologne, 7th May 2009

LibMeter Seminar The LibMeter Introduction Seminar is continued with the following Slideshare Presentations: Current Practice eUsage Reporting It also relates to: LibMeter Case Study #1 (University of Düsseldorf) x

The LibMeter Introduction Seminar

is continued with the following

Slideshare Presentations:

Current Practice eUsage Reporting

It also relates to:

LibMeter Case Study #1

(University of Düsseldorf)

Roadmap of this Presentation Introduction - Basics I. Introduction - Basics A Quick Futuristic Web Tour Usage Stats for the Rest of Us (No Maths please!) Library Role, Services & Usage II. Introduction – Current Practise Where to get Usage Figures from ? How to interpret usage Figures ? LibMeter Case Studies III. Perspectives

I. Introduction - Basics

A Quick Futuristic Web Tour

Usage Stats for the Rest of Us (No Maths please!)

Library Role, Services & Usage

II. Introduction – Current Practise

Where to get Usage Figures from ?

How to interpret usage Figures ?

LibMeter Case Studies

III. Perspectives

Roadmap of this Presentation LibMeter Introduction Basics I. Introduction A Quick Futuristic Web Tour Usage Stats for the Rest of Us (No Maths please!) Library Role, Services & Usage II. Practise Where to get Usage Figures from ? How to interpret usage Figures ? LibMeter Case Studies III. Perspectives

I. Introduction

A Quick Futuristic Web Tour

Usage Stats for the Rest of Us (No Maths please!)

Library Role, Services & Usage

II. Practise

Where to get Usage Figures from ?

How to interpret usage Figures ?

LibMeter Case Studies

III. Perspectives

LibMeter Seminar Introduction x

Introduction

Questions to Start Are you familiar with ? EZB DBIS DBS BIX DOI/CrossRef COUNTER OpenURL

EZB

DBIS

DBS

BIX

DOI/CrossRef

COUNTER

OpenURL

New Kind of Science & Library Stats on Web 2.0

Example for Webometrics: Ranking of World Universities http://www.webometrics.info/top100_continent.asp?cont=europe [2009-04-25]

What „Google Insight“ can tell about German Libraries and Archives ? (I) http://www.google.com/insights/search/# q=Anna%20amalia&cmpt=q [2009-05-18]

What „Google Insight“ tells about German Libraries and Archives II http://www.google.com/insights/search/#q=anna%20amalia%2Cstadtarchiv%20k%C3%B6ln&cmpt=q (2009-04-24)

Usage Stats Theory Part I For the Rest of Us (No Maths please!)

Basic Terms I Log Record of Events Count = Value = Figure Numerical Quantity Data Parameter Variable (numerical) value; Here: Output ! Indicator Something giving information Statistics = „Stats“ Collection of numerical data Analysis and Interpretation of Statistical Data

Log

Record of Events

Count = Value = Figure

Numerical Quantity Data

Parameter

Variable (numerical) value; Here: Output !

Indicator

Something giving information

Statistics = „Stats“

Collection of numerical data

Analysis and Interpretation of Statistical Data

Basic Terms II Logging Storing and preserving a File-record with detailed technical Information about all occuring service events (Web) Usage (figures or statistics) Quantitative parameters/values derived from filtering and processing Log-Files (aggregating, relating, normalizing) Usage monitoring = analysing (web) usage figures over time (longitudinal) or between services (transversal) Usability Degree of suitability of (web) service for purpose

Logging

Storing and preserving a File-record with detailed technical Information about all occuring service events (Web)

Usage (figures or statistics)

Quantitative parameters/values derived from filtering and processing Log-Files (aggregating, relating, normalizing)

Usage monitoring = analysing (web) usage figures over time (longitudinal) or between services (transversal)

Usability

Degree of suitability of (web) service for purpose

Basic Terms III Performance Degree of Effectiveness (e.g. of work or a service) Ranking Position in Relation to others Working out Ranking Order Rating classification of something on a scale according to how much or how little of a quality it possesses Benchmarking Comparing against a standard against which something can be measured

Performance

Degree of Effectiveness (e.g. of work or a service)

Ranking

Position in Relation to others

Working out Ranking Order

Rating

classification of something on a scale according to how much or how little of a quality it possesses

Benchmarking

Comparing against a standard against which something can be measured

Reporting Frequencies & Effect on Potential Uses and Users It takes Time for Results 2 periods for first value 3 periods for first pair 4 periods for confirmed trend 5 periods for change in trend Annual Reporting Good picture after 5 yrs Longterm Dev. & Strategy Monthly reporting Picture after a quarter Planning on Department or workgroup level Daily reporting Picture after a week Individual actions

It takes Time for Results

2 periods for first value

3 periods for first pair

4 periods for confirmed trend

5 periods for change in trend

Annual Reporting

Good picture after 5 yrs

Longterm Dev. & Strategy

Monthly reporting

Picture after a quarter

Planning on Department or workgroup level

Daily reporting

Picture after a week

Individual actions

Usage Stats Theory Part II Analysis & Interpretation (Sorry some Maths required!)

Digesting Usage Figures - I Determining Relevant Derivatives Help! Too many too confusing too large numbers ... Various Aspects of „Usage“ Baseline Figures Normalized and Standardized Figures Change-Rates Previous Periods (+/- Comparison with other Services) (+/- External Comparison) Ranking, Rating

Help! Too many too confusing too large numbers ...

Various Aspects of „Usage“

Baseline Figures

Normalized and Standardized Figures

Change-Rates

Previous Periods

(+/- Comparison with other Services)

(+/- External Comparison)

Ranking, Rating

Basic Terms IV Usage Counts and Derivatives Value Example: Downloads (D) Normalized Value or Ratio (per external basis) Example: Downloads per User (DpU) First Derivative (by time=rate) – per period Example: Downloads per User per Year (DpUpY) Second Derivative –> Trend Example: Growth/Decline=Change (CoDpUpY) Third Derivative - > Trend Change Example: Acceleation/Slowdown (CoCoDpUpY)

Value

Example: Downloads (D)

Normalized Value or Ratio (per external basis)

Example: Downloads per User (DpU)

First Derivative (by time=rate) – per period

Example: Downloads per User per Year (DpUpY)

Second Derivative –> Trend

Example: Growth/Decline=Change (CoDpUpY)

Third Derivative - > Trend Change

Example: Acceleation/Slowdown (CoCoDpUpY)

Product Introduction Phases My Own Idealized Schema Phase I – Breakout Accelleration from zero base Pattern A/B/C 0>+ / 0>++ / 0>+++ Low counts, quickly rising growth rate, very positive trend Timeline: 0 - 33 % Phase II – Growth (exponent.) +>+++ / ++>+++>++ / +++>0> --- Timeline: 34 – 66 % Phase III – Maturation Slowing Growth, to Plateau +++ / ++>0 / - >0 Timeline: 67 – 100 % I II III

Phase I – Breakout

Accelleration from zero base

Pattern A/B/C 0>+ / 0>++ / 0>+++

Low counts, quickly rising growth rate, very positive trend

Timeline: 0 - 33 %

Phase II – Growth (exponent.)

+>+++ / ++>+++>++ / +++>0> ---

Timeline: 34 – 66 %

Phase III – Maturation

Slowing Growth, to Plateau

+++ / ++>0 / - >0

Timeline: 67 – 100 %

From Library Service Introduction to Product Life Cycle Approach Consider a Library Service a Standard Product Apply Theories from Econometrics to „Webometrics“ See Overlaps Substitution Cycles

Consider

a Library Service a Standard Product

Apply

Theories from Econometrics to „Webometrics“

See

Overlaps

Substitution Cycles

Questions asked: How is the Library doing with eServices? How high is a specific usage ? Absolute / standardized / ranked / rated Where is Library/are Users heading ? Externally: Ranking compared to Peer-Institutions Other Countries Internally Count development: Growth/Decline Increased Speed or Slowdown Reversal of current Trend Comparison with potential alternative/competing Services

How high is a specific usage ?

Absolute / standardized / ranked / rated

Where is Library/are Users heading ?

Externally: Ranking compared

to Peer-Institutions

Other Countries

Internally

Count development: Growth/Decline

Increased Speed or Slowdown

Reversal of current Trend

Comparison with potential alternative/competing Services

Which Library Decicisons are supported by Usage Figures ? CONTENT Acquisitions / Subscriptions Renewals Cancelations Newly identified content SERVICES Web Customer Communication Web Services, Interfaces & Features Computer System Investments Human Resource Allocation

CONTENT

Acquisitions / Subscriptions

Renewals

Cancelations

Newly identified content

SERVICES

Web Customer Communication

Web Services, Interfaces & Features

Computer System Investments

Human Resource Allocation

LibMeter Seminar Current Practise eUsage Reporting x

Current Practise

eUsage Reporting

(Electronic) Library Role, Services & Usage

How Needs, Services, Usage & Community Value are related ? C O M M U N I T Y with N E E D S E R V I C E S C O P E U T I L I T Y USABILITY ACCESSIBILITY VISIBILITY AWARENESS USAGE ( = ACCEPTANCE) VALUE

New Thinking required for Virtual Library Services World

Who is your Customer ? Many ways to count Users Multiple Definitions for User Base Affiliated Users Primary users (Students, Staff) Guests (registered and others) Special User Groups (faculties, institutes etc.) Active Users Recurring Users (e.g. last 12 months) Power Users (frequent uses) Identification methods Individual Registration (OPAC for Borrowing, ILL, Copies) IP-based Log-In (+/- Proxy; -> Institutional IP-Domains) Other Authentication via single-sign-on, SHIBBOLETH

Multiple Definitions for User Base

Affiliated Users

Primary users (Students, Staff)

Guests (registered and others)

Special User Groups (faculties, institutes etc.)

Active Users

Recurring Users (e.g. last 12 months)

Power Users (frequent uses)

Identification methods

Individual Registration (OPAC for Borrowing, ILL, Copies)

IP-based Log-In (+/- Proxy; -> Institutional IP-Domains)

Other Authentication via single-sign-on, SHIBBOLETH

Selected Readings to broaden and to deepen your Understanding BIX - Der Bibliotheksindex 2008. B.I.T.online Sonderheft. 2008 Juni ;1-56. Blecic DD, Fiscella JB, Wiberley Jr SE . Measurement of Use of Electronic Resources: Advances in Use Statistics and Innovations in Resource Functionality. College and Research Libraries. 2007 ;68(1):26-44. Ceynowa VK, Coners A . Balanced Scorecard für wissensch. Bibliotheken. Vittorio Klostermann; 2002. Gallagher J, Bauer K, Dollar DM . Evidence-based librarianship: Utilizing data from all available sources to make judicious print cancellation decisions. Lib. Collections, Acq. & Tech. Serv.. 2005 ;29(2):169-179. George Boston, Whang M . E-Resources Usage Data: Apples to Oranges and Fixing Holes [Internet]. Atlanta: 2008. [zitiert 2009 Apr 22] Available_from:http://209.85.129.132/search?q=cache:mj5wfYDT2BoJ:https://smartech.gatech.edu/bitstream/1853/20874/13/Boston_Whang_ERL_final.ppt+scholarly-stats+sfx&cd=4&hl=de&ct=clnk&gl=de&client=firefox-a Hutzler E . Bibliotheken gestalten Zukunft : Kooperative Wege zur digitalen Bibliothek. Dr. Friedrich Geißelmann zum 65. Geburtstag. Universitätsverlag Göttingen; 2008. Kreische J . Die Messung von Vernetzung. Nutzungsstatistiken mit SFX [Internet]. Münster: 2006. Available from: http://docserv.uni-duesseldorf.de/servlets/DocumentServlet?id=9728 7. Kreische J . Zwischen Ranking und Qualitätsmanagement: BIX WB [Internet]. Mannheim: 2008. Available from: http://www.opus-bayern.de/bib-info/volltexte/2008/583/pdf/BIX%20Mannheim%20(Word%202003).pdf Poll R, te Boekhorst P , Measuring quality, Saur, München, 2007. Lossau N . digital services in academic libraries. In: Digital Convergence - Libraries of the Future: Libraries of the Future‎. Springer; 2008. Shepherd P . The feasibility of developing and implementing journal usage factors: a research project sponsored by UKSG. Serials. 2007;20(2):117-123.

BIX - Der Bibliotheksindex 2008. B.I.T.online Sonderheft. 2008 Juni ;1-56.

Blecic DD, Fiscella JB, Wiberley Jr SE . Measurement of Use of Electronic Resources: Advances in Use Statistics and Innovations in Resource Functionality. College and Research Libraries. 2007 ;68(1):26-44.

Ceynowa VK, Coners A . Balanced Scorecard für wissensch. Bibliotheken. Vittorio Klostermann; 2002.

Gallagher J, Bauer K, Dollar DM . Evidence-based librarianship: Utilizing data from all available sources to make judicious print cancellation decisions. Lib. Collections, Acq. & Tech. Serv.. 2005 ;29(2):169-179.

George Boston, Whang M . E-Resources Usage Data: Apples to Oranges and Fixing Holes [Internet]. Atlanta: 2008. [zitiert 2009 Apr 22]

Available_from:http://209.85.129.132/search?q=cache:mj5wfYDT2BoJ:https://smartech.gatech.edu/bitstream/1853/20874/13/Boston_Whang_ERL_final.ppt+scholarly-stats+sfx&cd=4&hl=de&ct=clnk&gl=de&client=firefox-a

Hutzler E . Bibliotheken gestalten Zukunft : Kooperative Wege zur digitalen Bibliothek. Dr. Friedrich Geißelmann zum 65. Geburtstag. Universitätsverlag Göttingen; 2008.

Kreische J . Die Messung von Vernetzung. Nutzungsstatistiken mit SFX [Internet]. Münster: 2006.

Available from: http://docserv.uni-duesseldorf.de/servlets/DocumentServlet?id=9728 7.

Kreische J . Zwischen Ranking und Qualitätsmanagement: BIX WB [Internet]. Mannheim: 2008.

Available from: http://www.opus-bayern.de/bib-info/volltexte/2008/583/pdf/BIX%20Mannheim%20(Word%202003).pdf

Poll R, te Boekhorst P , Measuring quality, Saur, München, 2007.

Lossau N . digital services in academic libraries. In: Digital Convergence - Libraries of the Future: Libraries of the Future‎. Springer; 2008.

Shepherd P . The feasibility of developing and implementing journal usage factors: a research project sponsored by UKSG. Serials. 2007;20(2):117-123.

Special Thanks for critical & constructive discussions to: Joachim KREISCHE, ULB Düsseldorf Simone FÜHLES-UBACH, FH Köln Eric MULDER, Den Haag Marion MÜLLER , ULB Düsseldorf Jessica BUSCHMANN, Dortmund Nol VERHAGEN, UvA, Amsterdam Sylvia THIELE , UB Dortmund Britta RIEBSCHLÄGER , ULB Düsseldorf Peter KOSTÄDT, UB Köln

Joachim KREISCHE, ULB Düsseldorf

Simone FÜHLES-UBACH, FH Köln

Eric MULDER, Den Haag

Marion MÜLLER , ULB Düsseldorf

Jessica BUSCHMANN, Dortmund

Nol VERHAGEN, UvA, Amsterdam

Sylvia THIELE , UB Dortmund

Britta RIEBSCHLÄGER , ULB Düsseldorf

Peter KOSTÄDT, UB Köln

About me – Key Fields of Interests Libraries Early 1970s – Founding highschool library Early 1980s - Converting Card Catalog to electronic (Mainframe & punchcards) Late 1980s creating bibliographic database for 5.000+ article reprints (Mac) Since 1990 consulting scholarly libraries on retrieval systems, End user friendyness Hospital Libraries Life Sciences & Statistics Ph.D. thesis on pattern recognition in brain Several peer-reviewed publicatios Presentations Publishing & Software Editing & Managing highschool journal for 5 years Heading Cycling Campaign Newsletter for 3 years Knowledge Finder – Biomedical Search Engine Distribution & Localization for Europe 12 years e- Ressources Elsevier / Kluver / Ex Libris Science Direct,Embase.com, Ovid, SFX, MetaLib Knowledge Management 1990s Scientific Knowledge Management Software Development & Distribution Late 1990s e-library transformation consulting, Projects, trainings Mid 2000s Library e-services Development of Cooperative strategies Market Analysis & Business Planning Information Technology in the Sciences Late 1970s first own programming on handheld computers, then Mainframes, then Apple II, then Mac Late 1980s German Academic Software Price for co-developping a relational bibliographicpProgram - PARiS 1990s Fuzzy logic, natural language & relevance ranked output retrieval Late 1990s – Online e-Journal and integrated search platforms Mid 2000s – OpenURL, Context sensitive linking, Metasearch, ERM, Recommenders, Library 2.0

Publishing & Software

Editing & Managing highschool journal for 5 years

Heading Cycling Campaign Newsletter for 3 years

Knowledge Finder – Biomedical Search Engine

Distribution & Localization for Europe

12 years e- Ressources

Elsevier / Kluver / Ex Libris

Science Direct,Embase.com, Ovid, SFX, MetaLib

Knowledge Management

1990s Scientific Knowledge Management Software

Development & Distribution

Late 1990s e-library transformation

consulting, Projects, trainings

Mid 2000s Library e-services

Development of Cooperative strategies

Market Analysis & Business Planning

LibMeter Seminar The LibMeter Introduction Seminar is continued with the following Slideshare Presentations: Current Practice eUsage Reporting It also relates to: LibMeter Case Study #1 (University of Düsseldorf) x

The LibMeter Introduction Seminar

is continued with the following

Slideshare Presentations:

Current Practice eUsage Reporting

It also relates to:

LibMeter Case Study #1

(University of Düsseldorf)

LibMeter Seminar Personal Announcement: It is intended to continue this series slowly with further modules amongst others on: Detecting, following and predicting Usage-Trends Measuring Impact of e-Service Marketing Events International Library Comparisons Comparing Academic & Library Excellence x

Personal Announcement:

It is intended to continue this series slowly

with further modules amongst others on:

Detecting, following and predicting Usage-Trends

Measuring Impact of e-Service Marketing Events

International Library Comparisons

Comparing Academic & Library Excellence

LibMeter Seminar The End ! Thank You Peter Ahrens libmeter-at-gmail.com x

The End !

Thank You

Peter Ahrens

libmeter-at-gmail.com