1. Upping Our Game:
Leading on
Transformational
Analytics &
Getting Off the Hits
Train
Unlocking New Value from Content
Stephen Abram, MLS
Stephen.abram@gmail.com
stephenslighhouse.com

2. Are you on the ‗HITS‘ train?

3. DATA

4. QUALITATIVE INFORMATION
and
QUANTITATIVE DATA

5. STATISTICS
and
MEASUREMENTS

6. What do we do
when our
buyers are
asking for data
that does not
align with their
goals?

7. Have Journal Prices Really Increased Much in the Digital Age?
(Scholarly Kitchen blog) http://bit.ly/11b3hP2

8. Excellent Metaphor
 ―What if the only measurement of energy costs you
followed was the price of oil, while everyone was shifting
to cheaper and more efficient alternatives? And what if
you completely ignored the fact that everything around
you was using more and more power — your lights, your
phone, your car, your heat, your media center? You might
come to believe that energy is getting more
expensive, when actually, it‘s price is rising relatively
slowly while your usage is what is skyrocketing.
 The same thing might be happening with print journal
prices and digital journal licenses…

9. Good Questions
 What if prices of the predominant journal
form have actually been falling?
 What if we‘ve been measuring the wrong
things, or measuring insufficiently?
 And what if the growth in expenses are not
the result of price increases but a result of
the growth in science?‖

10. The Real Digital Story
 Print subscription prices are a misleading and
inaccurate method for tracking library serials
spending
 ―. . . libraries‘ spending on periodicals has increased
three-fold while their collections have tripled in size‖
 ―Spending three times as much to get three times as
much tells a very different story from the ―price
increases‖ story. . . .‖
 Published article output has grown 3.5% to 4% per
year since 1990
 Growth in research spending has been increasing by
3-4% per year
 In the US, spending on scientific research has more
than doubled since 1990 (from $150.2 billion to
$400.5 billion in 2010, in current dollars)

11. Numbers versus ROI
 ―In the midst of all this growth, prices have
risen modestly. Gantz notes that while the
economy in the US from 1990 to 2010 grew
at a compounded rate of 66.8% due to
inflation, the effective price of an average
journal is only 9% higher over the same time
period. In the UK, prices have actually gone
down by 11% since 2004.‖
 ―Price increases have been caused by more
science, more papers, and more
journals, not by price increases in licenses.
In fact, per-journal prices seem to have
peaked around 2000, and steadily declined
from there, as shown by the black line in the
chart below.‖

13. And this is all means?
We‘re playing a fool‘s
game when we play the
raw statistics game.

15. Are you locked into library financial mindsets?

16. What about value and impact?

17. Or shall we stick with this?

18. Grocery Stores

19. Grocery Stores

20. Grocery Stores

21. Cookbooks, Chefs . . .

22. Cookbooks, Chefs . . .

23. Meals

29. What do we count and share?
 Titles
 Clicks
 Downloads
 Sessions
 Session length
 COUNTER, (Counting Online Usage of
Networked Electronic Resources)
 SUSHI, Standardized Usage Statistics
Harvesting Initiative
 etc.

30. Or should we measure?
 Was there improved customer satisfaction?
 Do librarians or types of end users have
different values and behaviours?
 Did learning happen?
 Was there an impact on research or
strategic outcomes?
 Did the patient live, improve, survive, thrive?
 ...

31. Algorithms
 Search differentiator
 Commercial algorithms versus those based
on big data
 Measuring end user success versus known
item retrieval…
 ―Romeo and Juliet‖
 Problems with the unmonitored trial
 Wrong tests
 Poor sampling
 Mindset issues

32. Sharing Learning and Research
 Usability versus User Experience
 End users versus librarians
 Known item retrieval (favourite test) versus
immersion research
 Lists versus Discovery
 Scrolling versus pagination
 Devices and browsers and agnosticism
 Satisfaction and change
 Individual research experience vs. impacts on
e-courses, LibGuides, training materials, etc.

33. Gale Analytics

34. Focus and Understand on the Whole Experience

35. Inside Lego™ Pieces
 Foresee satisfaction and demographic data
 Counter & Sushi data
 Database usage (unique
user, session, length of
session, hits, downloads, etc.)
 Google Analytics
 Search Samples
 ILS Data
 Geo-IP data

36. What We Know (US/Canada)
 27% of our users are under 18.
 59% are female.
 29% are college students.
 5% are professors and 6% are teachers.
 Daily, 35% of our users are there for the very first time!
 Only 29% found the databases via the library website.
 59% found what they were looking for on their first search.
 72% trusted our content more than Google.
 But, 81% still use Google.

37. Statistics, Measurements and
Analytics
• Counter & Sushi data are very weak metrics
that don‘t provide insights into the critical stuff
• Database usage (unique user, session, length
37 of session, hits, downloads, etc.)
• Web and Google Analytics (6,000+ websites)
• Foresee satisfaction and demographic data
• Search Samples (underemphasized at this
point.)
• Time of Year Analysis
• ILS Data (from clients &n partnerships)
• Geo-IP data, analytics and mapping.
• Impact studies and sampling.

38. Advocacy Work and GCL
Alignment
 Michigan Outcomes Study
 Impact of In Context portals
38

40. Academic Libraries:
Central Michigan University
Grand Valley State University
Public Libraries:
Clinton Macomb Public Library
Howell District Library
Kent District Library
Portage District Library

42. Who are our audiences?
• Librarians (several languages
management, reference, acquisitions, systems
, LMS, etc.)
• Institutional information technology and
systems professionals
• eLearning professionals and developers
• Web design professionals
• Library Management team & Chief Librarian
• City or University administration, Provosts
 Or End users?

43. Analytics

44. What do we need to know?
 How do library databases compare with other web
experiences and expectations?
 Who are our core virtual users?
 What are user expectations for satisfaction?
 How does library search compare to consumer
search like Google?
 How do people find and connect with library virtual
services?
 What should we ‗fix‘ as a first priority?
 Are end users being successful in their POV?
 Are they happy? Will they come back? Tell a friend?

45. Top-Level Benchmarks
Gale-Cengage Browse Survey
August 01, 2010 - August 31, 2010
90 90 90 89 90 90 90 90 90
88 87 87
85 84
78 77
71 75 76
73 74 74
71 72 72 72
Gale Library
70
68
70 69
65
62
Databases Compare 59 59
48
41
Very Well 48
37
30 30
to Other
30
33
30 30
Web Experiences
0

46. Digging Into Satisfaction
3 4 5 9 6 8 7 2 1
Element
High Univers/ Other, Other Middle ary
School College please Profess- Profess- School School
Student Student Librar’n specify Teacher ional or Student Student Overall
Responses: 3,043 2,920 1,570 709 576 576 488 477 148 10,507
29% 28% 15% 7% 5% 5% 5% 5% 1%
Content 70 77 79 Users will
69 77 74 67 68 40 73
Look and Feel
Navigation
64
65
74
71
74
69
Return and
64
60
72
69
70
66
61
60
63
63
37
45
68
66
Search 61 73 73 Recommend
60 71 67 59 59 35 66
Site
Performance
73 79 77
Once Hooked
72 77 76 68 72 52 75
Satisfaction 63 74 72 60 72 66 59 62 35 67
Likelihood to
72 82 85 71 82 78 66 69 36 76
Return
Primary
63 73 68 56 70 60 51 60 33 65
Resource
Recommend 63 78 83 66 80 72 63 61 36 71

47. Gale-Cengage Browse Survey
Priority Map
October 01, 2009 - October 31, 2009
STATUS QUO REQUIRED MAINTAIN OR IMPROVE
High
Site Performance
74, 0.4 Content
73, 1.1
Library
SCORE
72
Search
Look and Feel
68, 0.4
Needs to
Navigation
67, 0.9
Improve Search
67, 2.8
Low
62
0.2 Low MONITOR 1.2 TOP PRIORITY
2.2 High
Key:
Position of each bubble indicates its score and impact IMPACT
Size of each bubble also indicates the relative size of impact

48. Who uses e-Resources?
High School Student 29%
University/ College Student 28%
Librarian
The Core User
15%
Other, please specify 7%
For Library
Teacher 5%
E-Resources
Other Professional 5%
is Clear
Professor 5%
Middle School Student 5%
Elementary School Student 1%
0% 5% 10% 15% 20% 25% 30% 35

49. Gale-Cengage Browse Survey
Did you find what you were looking for?
August 01, 2009 - September 06, 2010
(N: 10486)
Don't know yet
20%
Database Users
No
Are Being Yes
20%
Successful 60%

50. Gale-Cengage Browse Survey
What type of difficulty, if any, did you encounter with the navigation process on this website?
August 01, 2009 - September 06, 2010
(N: 10486)
Had technical difficulties
(e.g. broken links, error Links/labels are difficult to
messages) understand
4% 4%
Too many links or
navigational choices
5%
Could not navigate back to
previous information
8%
There are
Would often feel lost, not
know where I was
Training and Did not have any difficulty
navigating the site
8% 52%
Other, pleaseCommunication
specify
9%
Opportunities
Links did not take me where
I expected
10%

51. Gale-Cengage Browse Survey
What is your gender?
August 01, 2009 - September 06, 2010
(N: 10484)
Prefer not to respond
8%
More Males Use
Male
33%
Library
Databases Female
59%
Than Usual

52. Gale-Cengage Browse Survey
What is your age?
August 01, 2009 - September 06, 2010
(N: 10486)
Prefer not to respond
7%
26-35
10%
There is Not 18 and under
30%
19-25
11% A Demographic
Swing in Online
36-50
19% Usage 51+
23%

53. Gale-Cengage Browse Survey
Do you trust the content on this website more than the content you find through web search
engines?
August 01, 2009 - September 06, 2010
(N: 10486)
No
9%
Library Users
Trust Library
Haven't thought about it
18%
Databases
More. Yes
73%

54. Gale-Cengage Browse Survey
Which best describes the purpose of your research today?
August 01, 2009 - September 06, 2010
(N: 10486)
Just browsing
Other, please specify 5%
6%
Personal
interest
10%
School is the
Sweet Spot –
Professional project
School assignment
17%
But Other Users 62%
Abound Too.

55. Gale-Cengage Browse Survey
Which best describes your role on this website today?
August 01, 2009 - September 06, 2010
(N: 10486)
High School Student 29%
University/ College
28%
Student
Library’s
Librarian 15%
Other, please specify 7%
Natural Allies
Teacher 5%
Are Big Users
Other Professional 5%
Professor 5%
& Potential
Middle School Student 5%
Partners.
Elementary School
1%
Student
0% 5% 10% 15% 20% 25% 30% 35%

56. Gale-Cengage Browse Survey
How did you find out about this website?
August 01, 2009 - September 06, 2010
(N: 10486)
Other, please specify Web search
5% engine
3%
Used in the past
7%
Wow! Only 29% of Librarian
Users Find 32%
E-Resources
Teacher
24%
Through Library
School or library website
Websites. 29%

57. Gale-Cengage Browse Survey
How frequently do you use this website?
August 01, 2009 - September 06, 2010
(N: 10486)
Every 6 months or less
7% Daily
7%
About once every 3 months
And 39% of Your
8%
Users Are in Our First time
36%
Databases for
About once a month
10%
week
For the Very
Several times a
Several times a month
14%
First Time! 18%

58. Gale Analysis: Mobile
March 26 – September 25, 2012
Mobile Device Users: Interest in Accessing Gale Mobile Site by Topic (Top 10)
Business (n=105) 67% 16% 17%
Law/Government (n=89) 64% 20% 16%
Arts/Humanities (n=125) 62% 22% 15%
Science/Technology (n=132) 60% 17% 23%
Politics/Current Events
(n=108)
60% 22% 18%
Cultural Studies (n=101) 60% 21% 19%
Literature (n=98) 60% 24% 15%
Biography/Genealogy (n=84) 60% 29% 12%
Medicine/Health (n=137) 59% 15% 26%
History (n=149) 58% 21% 21%
0% 20% 40% 60% 80% 100%
Yes No Not Sure
Gale Analysis: Mobile

59. Gale-Cengage Browse Survey
Recommend - Scores and Distributions
August 01, 2009 - September 06, 2010
(Avg. Score: 71)
Your Users
End Users
Will
Recommend
Your
Our
How likely are you to
recommend this site to
someone else? 12% 2%3%3% 5% 5% 7% 11%
Databases
14% 39%
Avg Score: 7.4
N*: 10486
0% 20% 40% 60% 80% 100%
* N values exclude
"Don't Know" 1 2 3 4 5 6 7 8 9 10

60. Gale-Cengage Browse Survey
Primary Resource - Scores and Distributions
August 01, 2009 - September 06, 2010
(Avg. Score: 65)
End Users
Respect
Our
How likely are you to
Databases
use this site as your
primary resource for
your research needs?
12% 3% 4% 4% 6% 7% 10% 14% As Primary
13% 27%
Avg Score: 6.9
N*: 10486
Sources
0% 20% 40% 60% 80% 100%
* N values exclude
"Don't Know" 1 2 3 4 5 6 7 8 9 10

61. Gale-Cengage Browse Survey
Likelihood to Return - Scores and Distributions
August 01, 2009 - September 06, 2010
(Avg. Score: 76)
End Users
Are
Likely to
Return
How likely are you to
return to this site?
9% 2% 2% 4% 4%
2% 7% 11% 14% 46%
Avg Score: 7.9
N*: 10486
0% 20% 40% 60% 80% 100%
* N values exclude
"Don't Know" 1 2 3 4 5 6 7 8 9 10

62. Gale-Cengage Browse Survey
Satisfaction - Scores and Distributions
August 01, 2009 - September 06, 2010
(Avg. Score: 67)
End Users
How does this site
compare to your idea of
Evaluate
an ideal website? 10% 3% 4% 4% 7% 8% 13% 18% 15% 19%
Avg Score: 6.8
N*: 10486 Our Services
as Meeting
How well does this site
meet your
expectations? 10% 2%3% 4% 6% 6% 11% 18%
Expectations 18% 22%
Avg Score: 7.1
N*: 10486
What is your overall
satisfaction with this
site? 9% 2% 3% 5%
3% 7% 11% 18% 19% 23%
Avg Score: 7.2
N*: 10486
0% 20% 40% 60% 80% 100%
* N values exclude
"Don't Know" 1 2 3 4 5 6 7 8 9 10

63. Gale-Cengage Browse Survey
Number of Survey Respondents
August 01, 2009 - September 06, 2010
1,800
The School
1,600
1,400
Cycle Drives
1,200
Many Usage
Scenarios
Number of Respondents
1,000
800
600
400
200
0
Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 Feb-10 Mar-10 Apr-10 May-10 Jun-10 Jul-10 Aug-10 Sep-10
N 203 1,153 1,611 1,391 760 814 1,081 1,138 1,059 691 168 170 192 55
Time Periods

64. There‘s Great News!
We had Room for
Improvement BUT
Library Databases Compete
Very Well with User
Expectations and
Needs Fulfillment.

65. EDUCATE
and Lead

66. Quick Poll
Should our industry Should our industry
A B
 Invest in the development and  Just deliver the raw statistics
promotion of a suite of end-user that customers are asking for
impact and value measurement and let them perform the
tools that actually communicate analyses independently?
the value in our products?

67. Until lions learn to write their own story,
the story will always be from the perspective
of the hunter not the hunted.

68. Stephen Abram, MLS, FSLA
Consultant, Dysart & Jones/Lighthouse Partners
Cel: 416-669-4855
stephen.abram@gmail.com
Stephen’s Lighthouse Blog
http://stephenslighthouse.com
Facebook, Pinterest, Tumblr: Stephen Abram
LinkedIn / Plaxo: Stephen Abram
Twitter: @sabram
SlideShare: StephenAbram1