Discovery impact scelc colloquium 2014mar05

349 views
236 views

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
349
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Discovery impact scelc colloquium 2014mar05

  1. 1. Discovery  or  Displacement?   A  Major  Longitudinal  Study  of  the  Effect   of  Web-­‐Scale  Discovery  Services  on   Online  (Journal)  Usage   SCELC  Colloquium   March  5,  2014     Michael  Levine-­‐Clark,  University  of  Denver   John  McDonald,  University  of  Southern  California   Jason  Price,  SCELC  ConsorNum  
  2. 2. “…a steep increase in full text downloads and link resolver click‐throughs suggests Summon had a dramatic impact on user behavior and the use of library collections during this time period.” The Impact ofWeb-scale Discovery on the Use of a Library Collection Doug Way (2010) http://scholarworks.gvsu.edu/ library_sp/9/
  3. 3. h"p://www.oclc.org/partnerships/econtent/solu4ons.en.html   Vendor  marke5ng  
  4. 4. Does  implementa4on  of  a  discovery   service  impact  usage  of  publisher-­‐ hosted  journal  content?  
  5. 5. What  did  we  measure?   •  Whether  there  is  an   effect   •  NOT  why  that  effect   exists  (that’s  a  future   study!)  
  6. 6. •  “Society  will  need  to  shed  some  of  its  obsession   for  causality  in  exchange  for  simple  correla5ons:   not  knowing  why,  but  only  what”     •  Cukier  &  Mayer-­‐Schonberger.  2013.  Big  data:  A  revolu4on  that  will   transform  how  we  live,  work,  and  think.  
  7. 7. Data  collec5on   •  List  of  libraries  with  discovery  services   > Searched  on  lib-­‐web-­‐cats   •  Surveyed  Libraries   > Discovery  service  Implemented   > ImplementaNon  Date  (month/year)   > Search  box  locaNon   > MarkeNng  effort   •  149  Libraries  Gave  Approval   > 33  libraries  selected  for  this  phase   > 6  for  each  of  the  4  major  discovery  services  and  a   group  of  9  libraries  with  no  service  
  8. 8. Dataset  •  33  Libraries   – 28  US,  2  CA,  1  each  from  UK,  AUS,  NZ   – WorldCat  book  holdings   >  Average:  1,114,193  ;  Range:  ~300k  to  ~2.6mil   •  ImplementaNon  dates  (Discovery  Libraries):     >  2010  (3),  2011  (19),  2012  (2)   •  6  Publishers   •  9,206  Journals   •  163,545  Usable  ObservaNons    
  9. 9. Methodology   Compared  COUNTER  JR1  total  full  text  arNcle  views  for  the   12  months  before  vs  12  months  aeer  implementaNon  date                 June  2010  Start   ImplementaNon   May  2011   May  2012   End   Year  1   Year  2   Included  implementaNon  month  in  Year  1  to  ensure  that   both  periods  included  an  enNre  academic  year  
  10. 10. Examine  Data  for  Outliers  
  11. 11. Observa5ons  by  Library  &  Service  
  12. 12. Observa5ons  by  Publisher  
  13. 13. Average  Usage  Change  By  Discovery  &  Publisher  
  14. 14. Analyzing  Usage  Change:  %  vs  Total     Use  12   months   before   Use  12   months   aRer   %  Change   Total   Change   Journal  A   500   600   20%   100   Journal  B   5   15   200%   10   Which  is  the  beier  measure?     Is  it  the  same  for  publisher-­‐  &  journal-­‐level  data?      
  15. 15. Reducing  varia5on  due  to  ins5tu5on  size   Currently  converNng  to  change  per  FTE       Values  are  shown  as  x  1,000  to  bring  the  change   metric  back  per  journal-­‐library  combinaNon  to  a   minimum  of  0.1     2013  JISC  Discovery  study  took  a  similar  approach  
  16. 16. Average  Usage  Change  By  Discovery  &  Publisher   Per  Journal  &   Per  10,000  FTE  
  17. 17. Full  Model   Including  Discovery  Service,  Publisher,  and  Library   Including  Discovery  Service,  Publisher,  and  Library  
  18. 18. Does the effect of discovery service differ across libraries? Library  10-­‐15   Library  16-­‐21   Library  22-­‐27   Library  28-­‐33  Library  1-­‐9  
  19. 19. Nested  ANOVA  Model     [all  three  factors  –  preliminary  results]  
  20. 20. Does  usage  change  vary  across  libraries?     Institution (sorted by Mean Change)
  21. 21. Does  usage  change  vary  across  publishers?     Publisher (sorted by Mean Change)
  22. 22. Does  usage  change  vary  across  discovery  services?  
  23. 23. Publisher Does  the  effect  of  discovery  service  differ  across  publishers?    
  24. 24. Results   Can  we  detect  differences  between  Discovery   Services,  Publishers,  and/or  Libraries  and/or   their  interac4ons?     •  Library  –  Yes   •  Publisher  –  No   •  Discovery  Service  –  Yes   •  DifferenNal  discovery  service  effect  by   publisher  –  Yes  
  25. 25. Next  Steps   •  Design  &  test  for  effects  of:     –  Aggregator  full  text  availability   –  Publisher  Size   –  Journal  Subject   –  Overall  usage  trends  (Requires  Disc  Srvc  ‘control’)   –  ConfiguraNon  opNons  in  Discovery  services   •  Expand  pool  of  libraries   •  Perhaps  explore  WHY        
  26. 26. Sharing  Data   •  With  par5cipa5ng  libraries   –  Customized  reports  for  each  library   •  With  par5cipa5ng  publishers   –  Customized  reports  for  each  publisher   –  PresentaNons  as  requested   •  With  discovery  vendors   –  PresentaNons  as  requested   •  In  publica5ons  and  presenta5ons   –  Maintaining  anonymity  of  data      
  27. 27. Doing  “Resarch”,  SCELC  Style!   •  Why  SCELC?   •  SCELC  Funding   –  staNsNcs  consultant     –  research  &  wriNng  retreats   –  See  hip://bit.ly/1dNMDL3  for  more  detail   •  SCELC  libraries  encouraged  to  parNcipate  in  next  round   –  Survey:  hip://bit.ly/DSparNcipaNon        

×