Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Owning The Discovery Experience
For Your Patrons
CharlestonConference, 2014
Robert McDonald,Courtney Greene McDonald,
Esth...
Welcome
Robert McDonald
What We’ll Share
Kate Lawrence, Vice President of User Research, EBSCO Information Services
The Google Influence on Discov...
Kate Lawrence, Vice President of User Research, EBSCO Information Services
The Google Influence on Discovery
@bykatelawren...
User Research at EBSCO
More than just usability testing
Data
What is the story
behind the data?
Secondary Research
What qu...
College Students Study
Digging deep to understanding their digital lives
154 3
22 Students:
High
School
College
Graduate
S...
“It’s their oxygen.”
Google
The first half of the first
page of results is critical.
Page 1 matters most
“If it doesn’t ex...
Wikipedia
Because college students like to start with an overview
1
2
3
The overview in “layman’s language”
The table of c...
Skimming and scanning
What gets students interested in research
Motivation
“When it’s about
me, it’s just more
interesting and
worthwhile.”
Frie...
Students “make the turn” towards informavore
Motivation
The Library Website
Getting there via Google
“I don’t speak library-ese.”
A simplified, elegant library website
When Discovery Means Find
“Everybody, Everything”: Crafting a Systemwide Discovery Strategy
Courtney Greene McDonald, Head Discovery & Research Serv...
One University, Many Front Doors
Systemwide Discovery
IUCAT
Shared online catalog
Powered by Blacklight
OnCourse
Shared le...
Diverse User Base, Divergent User Paths
Goal: Consistency regardless of chosen entry point
Defining opportunities at the campus level
Discovery at Bloomington
Leverage EDS API to pull
results into website, a cruci...
Balancing consistency with choice
Discovery Systemwide
Leverage API to pull results
into campus views; EDS
Reading List fo...
Esther Onega, The University of Virginia
Discovery Case Study: The University of Virginia
It all started with an email
Why Discovery?
TO: [lb-pubserv] Primooooooo!
On Jun 5, 2013, at 11:26, "Burks, Todd (tcb2e)" ...
How we got here
First, the meeting
Identified
problems
Brainstorm
options
Analysis
Started
Task force
Process for Evaluation
• Reinitiated ExLibris (Primo) customer service
• Limited results to peer-reviewed
• Included abstr...
Evaluation Criteria
How we decide
7
Essential
Elements
Functionality
Speed, relevancy
API
Necessary for incorporation into...
Evaluation Criteria
How we decide
API
• Each service has a web API:
• The service is engaged by opening a URL containing
t...
Comparing Metadata
Metadata
Ferreiro, Alberto. "Simon Magus, Nicolas Of Antioch, And Muhammad."
Church History 72.1 (2003)...
Comparing Metadata
Metadata
Ferreiro, Alberto. "Simon Magus, Nicolas Of Antioch, And Muhammad."
Church History 72.1 (2003)...
Comparing Metadata
Metadata
Ferreiro, Alberto. "Simon Magus, Nicolas Of Antioch, And Muhammad."
Church History 72.1 (2003)...
Limiting Results to Our Holdings
• Results automatically
limited by integration
with Serials Solutions
holdings.
• Updatin...
Article Detail Page: Primo
Not part of the
subjects data
Includes subtitle
and title of
magazine feature
Article Detail Page: EBSCO [EDS]
Refine?
Article Detail Page: Summon
LC-style subject
headings broken into
separate terms (e.g.
“Moose – Portrayals”
becomes “Moose...
Library Staff Lunch & Test
• 25 worksheets completed
• 14 votes received
• Many cookies consumed
• No clear winner, but 1 ...
User Test Results
EBSCO EDS was
favored overall,
but only by
3 searches
Qualitative Results – EBSCO [EDS]
Positives Negatives
• Quantity of articles
• Facets
• Relevancy
• Specificity
• Non-Engl...
Qualitative Results – Primo
Positives Negatives
• Diversity and range of articles
• Journal titles, specificity,
focus
• H...
Qualitative Results – Summon
Positives Negatives
• Perceived selectivity
• International diversity
• Cross-disciplinary
• ...
Products, Prices & Contracts
ExLibris (Primo)
$ • Would insist on annual contract
• Set-up fees paid 3 years ago
EBSCO [ED...
Questions
Thank you
Upcoming SlideShare
Loading in …5
×

Owning the Discovery Experience for Your Patrons

1,267 views

Published on

This is the slide deck for the presentation that was given with Kate Lawrence (VP User Experience EBSCO), Courtney McDonald (Indiana University), and Esther Onega (University of Virginia) at the 2014 Charleston Conference on Thursday Nov 6, 2014.

Published in: Education
  • Be the first to comment

Owning the Discovery Experience for Your Patrons

  1. 1. Owning The Discovery Experience For Your Patrons CharlestonConference, 2014 Robert McDonald,Courtney Greene McDonald, Esther Onega & Kate Lawrence
  2. 2. Welcome Robert McDonald
  3. 3. What We’ll Share Kate Lawrence, Vice President of User Research, EBSCO Information Services The Google Influence on Discovery Esther Onega, Head, Charles L. Brown Science & Engineering Library, The University of Virginia Discovery Tools, A Case Study: The University of Virginia Courtney Greene McDonald, Head Discovery & Research Services, Indiana University Libraries “Everybody, Everything”: Crafting a Systemwide Discovery Strategy
  4. 4. Kate Lawrence, Vice President of User Research, EBSCO Information Services The Google Influence on Discovery @bykatelawrence klawrence@ebsco.com
  5. 5. User Research at EBSCO More than just usability testing Data What is the story behind the data? Secondary Research What questions have been asked and answered previously? Primary Research Matching the right method to the research topic
  6. 6. College Students Study Digging deep to understanding their digital lives 154 3 22 Students: High School College Graduate School Schools included MIT, UNLV, Rice, Georgetown, UMass Amherst, GWU, UCSF, and more
  7. 7. “It’s their oxygen.” Google The first half of the first page of results is critical. Page 1 matters most “If it doesn’t exist on Google, does it exist?” I trust it Google is their point of triage, how they discover the world. “My door to the world.” Wikipedia is often the first result. Go to Images, News etc from this page. Students are more likely to search again than to look at page 2+ of results.
  8. 8. Wikipedia Because college students like to start with an overview 1 2 3 The overview in “layman’s language” The table of contents – “preview” The references and external links at the bottom
  9. 9. Skimming and scanning
  10. 10. What gets students interested in research Motivation “When it’s about me, it’s just more interesting and worthwhile.” Friends Peers are the most significant influencer among college students. Future Will this help me pay off my debt? Get a job? Not have to move home with my parents after graduation? Subject “I fell in love with computational fluid dynamics. Google wasn’t good enough any longer.” RelevanceMentor “I want my advisor to be proud of me.”
  11. 11. Students “make the turn” towards informavore Motivation
  12. 12. The Library Website Getting there via Google “I don’t speak library-ese.”
  13. 13. A simplified, elegant library website When Discovery Means Find
  14. 14. “Everybody, Everything”: Crafting a Systemwide Discovery Strategy Courtney Greene McDonald, Head Discovery & Research Services, Indiana University Libraries @xocg crgreene@indiana.edu
  15. 15. One University, Many Front Doors Systemwide Discovery IUCAT Shared online catalog Powered by Blacklight OnCourse Shared learning management system Currently Sakai Moving to Canvas in 2016 Indiana University Libraries System IPFW
  16. 16. Diverse User Base, Divergent User Paths Goal: Consistency regardless of chosen entry point
  17. 17. Defining opportunities at the campus level Discovery at Bloomington Leverage EDS API to pull results into website, a crucial discovery platform EDS IntegrationResponsive Planned study of new library website, specifically search results User Input Increasing proportion of mobile users – designing for that need
  18. 18. Balancing consistency with choice Discovery Systemwide Leverage API to pull results into campus views; EDS Reading List for CMS EDS IntegrationFlexibility Planned study of catalog interface User Input Campus views: consistency with options; open source community collaboration
  19. 19. Esther Onega, The University of Virginia Discovery Case Study: The University of Virginia
  20. 20. It all started with an email Why Discovery? TO: [lb-pubserv] Primooooooo! On Jun 5, 2013, at 11:26, "Burks, Todd (tcb2e)" <tcb2e@eservices.virginia.edu> wrote: Does anybody really like Primo? To be perfectly frank, I don’t. I don’t show it to First-Year students. In fact, I actively discourage them from using it. They’re set up for failure if they use it, and I need them to succeed in order to show the value of our services. Primo doesn’t add any value to the sessions I teach. What about you? Todd P.S. I’d like to have the default VIRGO search NOT include the Primo column on the right. It makes my job more difficult to steer new students away from it. If there are users that use it all the time, they should be able to set that preference without us having to do it for them.
  21. 21. How we got here First, the meeting Identified problems Brainstorm options Analysis Started Task force
  22. 22. Process for Evaluation • Reinitiated ExLibris (Primo) customer service • Limited results to peer-reviewed • Included abstract metadata in search results • Contacted peer institutions • Arranged open vendor presentations of top WSDS products & established evaluation criteria: • EBSCO (EDS) • ProQuest (Summon) • OCLC (WorldCat Local) • Analyzed link performance • User testing
  23. 23. Evaluation Criteria How we decide 7 Essential Elements Functionality Speed, relevancy API Necessary for incorporation into our catalog, VIRGO Metadata Searchable fields, field consistency, faceting Quality Control Results reflect library holdings Coverage Compatibility of Library holdings with those of the discovery service Customer Service MUST be available, helpful, and responsive Relevancy Are librarians and users finding results to be useful ?
  24. 24. Evaluation Criteria How we decide API • Each service has a web API: • The service is engaged by opening a URL containing the specifics of a given request • The output from the URL is used to construct the contents of the requesting page • A separate URL query and response is required for: • Each page of search results • Each individual article details page • Each service has its own unique URLs for sending requests • Each service has its own unique XML reply format for receiving results.
  25. 25. Comparing Metadata Metadata Ferreiro, Alberto. "Simon Magus, Nicolas Of Antioch, And Muhammad." Church History 72.1 (2003): 53-70. Primo • No abstract • No subject headings • Incomplete author name • Junk fields There is a duplicate record for this article further down in the results did have full name and abstract, but due to an incorrect title could not connect through the link resolver.
  26. 26. Comparing Metadata Metadata Ferreiro, Alberto. "Simon Magus, Nicolas Of Antioch, And Muhammad." Church History 72.1 (2003): 53-70. EBSCO • Abstract • 4 subject headings • No junk fields No duplicate records and the link resolver connected properly.
  27. 27. Comparing Metadata Metadata Ferreiro, Alberto. "Simon Magus, Nicolas Of Antioch, And Muhammad." Church History 72.1 (2003): 53-70. Summon • Abstract • 2 subject headings • More and better identifiers • No junk fields No duplicate records and the link resolver connected properly.
  28. 28. Limiting Results to Our Holdings • Results automatically limited by integration with Serials Solutions holdings. • Updating Serials Solutions holdings automatically updates Summon results. Yes Yes Yes Summon EBSCO Primo • Manually export and upload of single Serials Solutions holdings file. • Updating Serials Solutions holdings requires updating EBSCO. • Manually export, combine, and upload 27 separate Serials Solutions holdings files. • Updating Serials Solutions holdings requires updating Primo.
  29. 29. Article Detail Page: Primo Not part of the subjects data Includes subtitle and title of magazine feature
  30. 30. Article Detail Page: EBSCO [EDS] Refine?
  31. 31. Article Detail Page: Summon LC-style subject headings broken into separate terms (e.g. “Moose – Portrayals” becomes “Moose” and “Portrayals”)
  32. 32. Library Staff Lunch & Test • 25 worksheets completed • 14 votes received • Many cookies consumed • No clear winner, but 1 clear loser (Primo)
  33. 33. User Test Results EBSCO EDS was favored overall, but only by 3 searches
  34. 34. Qualitative Results – EBSCO [EDS] Positives Negatives • Quantity of articles • Facets • Relevancy • Specificity • Non-English language articles • Speed “Seems to come up with much more specified results of these keywords, possibly quite useful for a researcher already well- into the research process and quite a bit familiar with the materials...but not sure how good it would be for a beginning researcher. “One of the first things I noticed about the articles in these results was that the titles seemed more interesting and engaging, which made me more inclined to explore the article further. The search results included a good variety policy evaluation, study analysis, and general pieces on medical tort reform.”
  35. 35. Qualitative Results – Primo Positives Negatives • Diversity and range of articles • Journal titles, specificity, focus • High quality • Speed • Dated materials • Duplicates • Medium quality “The first page of results covers good ground in terms of major works one would need to be familiar with when conducting research in this field, and displays a good diversity in terms of sources.” “Using Firefox: Yielded 1,327 article results that were not exactly relevant to the paleo diet. The word 'paleo' [paleo diet] was included in some but the results were more on chemical reactions and such and i was looking more for experiments and peer reviewed journals covering the topic. It was not really relevant to the paleo diet at all.”
  36. 36. Qualitative Results – Summon Positives Negatives • Perceived selectivity • International diversity • Cross-disciplinary • Recency • Non-English language materials • Questionable faceting “I liked the Summon results a lot. The first three results were directly relevant, and most on the first page of results were at least somewhat relevant. There were also about 600 results (as opposed to about 1200 in Existing Virgo and Ebsco) so it seems that Summon weeded out some of the not relevant results.” “First article was from 1998, which is too old to be still relevant. Do not know why this is at the top of the articles list. Quality of journals is slightly suspect, although less so than EBSCO.”
  37. 37. Products, Prices & Contracts ExLibris (Primo) $ • Would insist on annual contract • Set-up fees paid 3 years ago EBSCO [EDS] $ • 6-month free trial included • 1-year contract • No set-up fee • Comes with full EDS ProQuest [Summon] $$ • 1-, 2-, 3-year contract options • Significant set-up fee
  38. 38. Questions
  39. 39. Thank you

×