Discovery impact erl2014

1,010 views

Published on

Michael Levine-Clark, John McDonald, Jason Price

Published in: Education
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,010
On SlideShare
0
From Embeds
0
Number of Embeds
53
Actions
Shares
0
Downloads
11
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Discovery impact erl2014

  1. 1. Not always discovered: Phase two of a study of the effect of discovery systems on online (journal) usage ER&L March 18, 2014 Michael Levine-Clark, University of Denver John McDonald, University of Southern California Jason Price, SCELC Consortium http://bit.ly/discovery-impact-erl2014
  2. 2. “…a steep increase in full text downloads and link resolver click‐throughs suggests Summon had a dramatic impact on user behavior and the use of library collections during this time period.” The Impact ofWeb-scale Discovery on the Use of a Library Collection DougWay (2010) http://scholarworks.gvsu.edu/library_sp/9/
  3. 3. http://www.oclc.org/partnerships/econtent/solutions.en.html Vendor marketing
  4. 4. Does implementation of a discovery service impact usage of publisher- hosted journal content?
  5. 5. Publisher-hosted journals are only part of the picture eBooks, pBooks, aggregator journal content, etc. publisher journal content
  6. 6. What did we measure? • Whether there is an effect • NOT why that effect exists (that’s a future study!)
  7. 7. • “Society will need to shed some of its obsession for causality in exchange for simple correlations: not knowing why, but only what” • (Cukier & Mayer-Schonberger. 2013. Big data: A revolution that will transform how we live, work, and think.)
  8. 8. Data collection • List of libraries with discovery services > Searched on lib-web-cats • Surveyed Libraries > Discovery service Implemented > Implementation Date (month/year) > Search box location > Marketing effort • 149 Libraries Gave Approval > 33 libraries selected for this phase > 6 for each of the 4 major discovery services and a group of 9 libraries with no service
  9. 9. Dataset• 33 Libraries – 28 US, 2 CA, 1 each from UK, AUS, NZ – WorldCat book holdings > Average: 1,114,193 ; Range: ~300k to ~2.6mil • Implementation dates (Discovery Libraries): > 2010 (3), 2011 (19), 2012 (2) • 6 Publishers • 9,206 Journals • 163,545 Usable Observations
  10. 10. Methodology Compared COUNTER JR1 total full text article views for the 12 months before vs 12 months after implementation date June2010Start Implementation May2011 May2012 End Year 1 Year 2 Included implementation month in Year 1 to ensure that both periods included an entire academic year
  11. 11. Examine Data for Outliers
  12. 12. Observations by Library & Service
  13. 13. Observations by Publisher
  14. 14. Average Usage Change By Discovery & Publisher
  15. 15. Analyzing Usage Change: % vs Total Use 12 months before Use 12 months after % Change Total Change Journal A 500 600 20% 100 Journal B 5 15 200% 10 Which is the better measure? Is it the same for publisher- & journal-level data?
  16. 16. Reducing variation due to institution size Currently converting to change per FTE Values are shown as x 1,000 to bring the change metric back per journal-library combination to a minimum of 0.1 2013 JISC Discovery study took a similar approach
  17. 17. Average Usage Change By Discovery & Publisher Per Journal & Per 10,000 FTE
  18. 18. Full Model Including Discovery Service, Publisher, and Library Including Discovery Service, Publisher, and Library
  19. 19. Nested ANOVA Model [all three factors – preliminary results]
  20. 20. Does usage change vary across libraries? Institution (sorted by Mean Change)
  21. 21. Does usage change vary across libraries using the same service? Library 10-15 Library 16-21 Library 22-27 Library 28-33Library 1-9
  22. 22. Does usage change vary across publishers? Publisher (sorted by Mean Change) B AAA A A
  23. 23. Does usage change vary across discovery services? A BB C D
  24. 24. Publisher Does the effect of discovery service differ across publishers?
  25. 25. Results Can we detect differences between Discovery Services, Publishers, and/or Libraries and/or their interactions? • Library – Yes • Publisher – No • Discovery Service – Yes • Differential discovery service effect by publisher – Yes
  26. 26. Next Steps • Design & test for effects of: – Aggregator full text availability – Publisher Size – Journal Subject – Overall usage trends (Requires Disc Srvc ‘control’) – Configuration options in Discovery services • Expand pool of libraries • Perhaps explore WHY
  27. 27. Sharing Data • With participating libraries – Customized reports for each library • With participating publishers – Customized reports for each publisher – Presentations as requested • With discovery vendors – Presentations as requested • In publications and presentations – Maintaining anonymity of data
  28. 28. Past/Future Presentations • Ithaka Sustainable Scholarship Conference (October 2013) • Charleston Conference (November 2013) • ER&L/Library Journal Webinar (December 2013) • Shangai Jiao Tong Univ / Beijing Univ Forum (Jan 2014) • SCELC Colloquium (March 2014) • ER&L (March 2014) http://bit.ly/discovery-impact-erl2014 • UKSG (April 2014) • Presentations posted on slideshare : – http://visualcv.com/lpq4t1s michael.levine-clark@du.edu | johndmcd@usc.edu | jason@scelc.org

×