Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Measuring electronic resource availability final version

256 views

Published on

  • Be the first to comment

  • Be the first to like this

Measuring electronic resource availability final version

  1. 1. Sanjeet MannMeasuring Electronic University of Redlands SCELC Research DayResource Availability March 5, 2013
  2. 2. Armacost Library infrastructure •ILLIAD interlibrary loan system •Full text targets (databases, ejournals) (79,757 unique titles) •Serials Solutions 360 Link •A&I databases (~77,000 titles indexed) •Innovative Interfaces catalog/proxy
  3. 3. Why do electronic resource errors matter? Costs Frustrated expectations Undermined confidence Complicated instruction
  4. 4. Research question "How often does full text linking work?"
  5. 5. Availabilitystudies Sample of items Available? Yes/No Error? Order encountered Probabilities Prioritize fixes
  6. 6. Development of the availability technique •Print material availability card catalog user surveys (Reviewed in Mansbridge 1986, Nisonger 2007) •Linear sequence (De Prospo 1973) •Branching model (Kantor 1976) •Applied to e-resources 500 articles from 50 high impact journals (Nisonger 2009)
  7. 7. OpenURL performance •OpenURL-based reasons for availability error (Wakimoto et al. 1998) •“Digging into the Data” on link resolver failure (Trainor and Price 2010) •NISO Initiatives: KBART, IOTA, PIE-J (Chandler et al. 2011, Glasser 2012, Kasprowski 2012)
  8. 8. Usability studies focusing on e-resources •Database link pages (Fry 2011, Ponsford et al. 2011b) •Resolver menus (O’Neill 2009, Imler & Eichelberger 2011, Ponsford et al. 2011a) •Discovery services (Williams & Foster 2011, Fagan et al. 2012) •Entire process
  9. 9. Methodology 400 citations 4 questions X 10 databases X 10 results[18:11] redlandsreference:what is your research topic? Arts & Humanities[18:11] meeboguest59808: RILMOral Motor Activity MLA Philosopher’s Index[18:11] redlandsreference:Is this for a CommunicativeDisorders class? Social Sciences America: History & Life EconLit Sociological Index Sciences Biological Abstracts ComDisDome,
  10. 10. Link testing http://goo.gl/606us
  11. 11. Error coding• What is an error?• Six error categories• Updated criteria
  12. 12. Armacost Library failure points: P vs. O
  13. 13. Error details 1: Proxy errors Domain missing from forward table Domain missing from SSL Certificate Timeouts trying to establish connection
  14. 14. Error details 2: Source errors Missing metadata Erroneous metadata (e.g. rft.date=0001-01-01)
  15. 15. Error details 3: Knowledge base errors Title not selected in knowledge base Title selected, but in poorly chosen collection Knowledge base holdings do not reflect access entitlement (embargo, back issues, etc.)
  16. 16. Error details 4: Link resolver error Confusion between two similar titles Unusual OpenURL syntax
  17. 17. Error details 5: Target errors Content not loaded (supplement, embargo) Records concatenated from full text and non-full- text databases Server downtime
  18. 18. Error details 6: ILLIAD errors Unicode metadata not displayed properly rft.title used for both book title and article title, affects chapters and dissertations
  19. 19. Results Error rate: 150/400 (38%) Overall availability: 250/400 (62%) Locally downloadable: 104/400 (26%)
  20. 20. SamplingNecessary sample size for a yes/no condition is determined by:To use this, you need:•Availability rate from a small pre-test•Choose acceptable % confidence (95%)•Choose acceptable margin of error (+/- 5%)Plug values into the formula…•p = 0.625 (250 / 400 successes)•1-p = 0.375 (150 / 400 errors)•C = 0.95 (95% confidence)•Zc = 1.96 (statistical textbook orhttp://www.measuringusability.com/pcalcz.php)•E = 0.05 (5% error)I could have just used 360 citations…
  21. 21. ConfidenceYour confidence in a study of a particular sample size is given by:I could have just used 360 citations…
  22. 22. Discussion
  23. 23. Discussion (continued)
  24. 24. Local Availability / Errors by Discipline
  25. 25. Solutions Edit proxy forward table
  26. 26. Solutions Upgrad e ILLIAD
  27. 27. Solutions Customiz e Serials Solutions
  28. 28. SolutionsSimplify result screen interface and terminology
  29. 29. Summary•400 citations obtained through likely keyword searchesof 10 A&I databases•62% availability / 38% error rate (98% confidence, +/- 5%)•26% downloadable full text•Responses include fixing proxy, kb holdings, interfaces,upgrading systems•Strengths: quant + qual data, very flexible(n=100 allows 85% confidence)•Weaknesses: Does not account for issues withinterfaces, searching or evaluation faced by actual users
  30. 30. Towards availability testing with live students • More barriers: o confusing interfaces o difficulty formulating searches and evaluating sources o login errors • How to test: o cognitive walkthrough + recorded task protocols o analysis informs information literacy and interface design • Deliverables: o availability % o branching model o usability report
  31. 31. QuestionsFurther reading: http://goo.gl/4fu47Data set: http://goo.gl/606us skatdesign.co m

×