IOTA OpenURL Quality @ 2011 UKSG Conference


Published on

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

IOTA OpenURL Quality @ 2011 UKSG Conference

  1. 1. NISO’s IOTA Working Group Improving OpenURLs Through Analytics UKSG Conference Harrogate, United Kingdom April 4 – 6, 2011   Rafal Kasprowski, Rice University
  2. 2. Agenda <ul><ul><li>In the Beginning: Full-text linking and Advent of OpenURL </li></ul></ul><ul><li>  </li></ul><ul><ul><li>IOTA: Created in response to OpenURL linking problems </li></ul></ul><ul><li>  </li></ul><ul><ul><li>IOTA’s analytical approach </li></ul></ul><ul><li>  </li></ul><ul><ul><li>Community-derived reports comparing quality of vendor OpenURLs </li></ul></ul><ul><li>  </li></ul><ul><ul><li>Concept of the OpenURL Quality Index </li></ul></ul><ul><ul><li>IOTA & KBART: relationship & joint initiative </li></ul></ul><ul><li>  </li></ul><ul><ul><li>Community involvement in IOTA: necessary for best outcomes </li></ul></ul>
  3. 3. Before OpenURL: Proprietary Linking <ul><li>  </li></ul><ul><ul><li>A&I database providers offered option for full-text linking (e.g., CSA, PubMed, etc.) . </li></ul></ul><ul><li>  </li></ul><ul><ul><li>Libraries manually activated linking to full-text providers they had subscriptions with. </li></ul></ul><ul><li>  </li></ul><ul><ul><li>A&I --> Full Text </li></ul></ul><ul><li>  </li></ul>
  4. 4. Proprietary Linking: Cons and Pro <ul><ul><li>Linking had to be activated manually by libraries for each full-text provider. </li></ul></ul><ul><li>  </li></ul><ul><ul><li>A&I providers offering this option were few. </li></ul></ul><ul><li>  </li></ul><ul><ul><li>Selection of full-text providers was limited. </li></ul></ul><ul><li>  </li></ul><ul><li>But... </li></ul><ul><ul><li>  Once set up, the static links to full texts were accurate. </li></ul></ul>
  5. 5. Advent of OpenURL <ul><ul><li>Objective: Deliver full texts unrestrained by proprietary silos. </li></ul></ul><ul><li>  </li></ul><ul><ul><li>Open standard generating link at time of request. </li></ul></ul><ul><li>  </li></ul><ul><ul><li>Library's holdings indicate provider of &quot;appropriate copy&quot;. </li></ul></ul><ul><li>  </li></ul><ul><ul><li>A-Z list (e.g., e-journal, e-books): </li></ul></ul><ul><ul><ul><li>Knowledge base (KB) with library's holdings. </li></ul></ul></ul><ul><ul><ul><li>Intermediary in linking. </li></ul></ul></ul><ul><li>  </li></ul><ul><ul><li>A&I (&quot;Source&quot;) --> A-Z list (&quot;KB&quot;) --> Full Text (&quot;Target&quot;) </li></ul></ul>
  6. 6. <ul><li>A, Bernand , et al. &quot; A versatile nanotechnology to connect individual nano-objects for the fabrication of hybrid single-electron devices. &quot; Nanotechnology 21 , no. 44 ( November 5, 2010 ): 445201 . Academic Search Complete , EBSCO host (accessed October 24, 2010). </li></ul>OpenURL Syntax and Resolver 21 &issue= 44 &date= 20101105 &spage= 445201 &title= Nanotechnology &atitle= A+ versatile+nanotechnology+to+connect+individual+nano-objects+for+the+ fabrication+of+hybrid+single-electron+devices. &aulast= A++Bernand Source Citation Target OpenURL (Source OpenURL structured similarly)
  7. 7. Pros & Cons of OpenURL <ul><li>Pros : </li></ul><ul><ul><li>KB/Resolver vendors took over most of the linking setup: Less work for libraries and providers. </li></ul></ul><ul><ul><li>Participation by A&I platforms and full-text providers exceeded proprietary linking: OpenURL scales better </li></ul></ul><ul><li>  </li></ul><ul><li>Cons : </li></ul><ul><ul><li>Dynamic linking less predictable than static linking: more difficult to pinpoint cause of link failures </li></ul></ul><ul><ul><li>OpenURL linking not improved significantly last 10 years. </li></ul></ul><ul><ul><li>No systematic method exists to benchmark OpenURLs. </li></ul></ul>
  8. 8. Problem Statement & Methodology <ul><li>&quot;72% of respondents to the online survey either agreed or strongly agreed that a significant problem for link resolvers is the generation of incomplete or inaccurate OpenURLs by databases (for example, A&I products).&quot; </li></ul><ul><li>Culling, James. 2007. Link Resolvers and the Serials Supply Chain: Final Project Report for UKSG, p.33. </li></ul><ul><li>  </li></ul><ul><li>  </li></ul><ul><li>  </li></ul><ul><li>Recently, researchers have indicated the need for metadata quality metrics , including: </li></ul><ul><ul><li>completeness; </li></ul></ul><ul><ul><li>accuracy; </li></ul></ul><ul><ul><li>conformance to expectations; </li></ul></ul><ul><ul><li>logical consistency and coherence. </li></ul></ul><ul><li>Bruce, Thomas R. and Hillmann, Diane I. 2004. The Continuum of Metadata Quality: Defining, Expressing, Exploiting. In Metadata in Practice . Ed. Diane I. Hillmann and Elaine L. Westbrooks. Chicago: American Library Association, pp. 238-256. </li></ul>
  9. 9. Année philologique OpenURL Study <ul><li>2008 Cornell study led by Adam Chandler* </li></ul><ul><ul><li>Problem: Too often links sent from Aph did not successfully resolve to requested resource. </li></ul></ul><ul><ul><li>Objective: Examine quality of OpenURLs offered to users by Aph in order to improve the linking. </li></ul></ul><ul><li>  </li></ul><ul><li>Aph Study investigated: </li></ul><ul><ul><li>Faulty citation metadata from source database. </li></ul></ul><ul><ul><li>Method to evaluate the OpenURLs. </li></ul></ul>*Chandler, Adam. 2009. Results of L’Année philologique online OpenURL Quality Investigation: Mellon Planning Grant Final Report.
  10. 10. Scoring System & Aph Study Outcomes <ul><li>Concept of scoring in Aph study (based on B. Hughes study)* </li></ul><ul><ul><li>establish a baseline for comparison; </li></ul></ul><ul><ul><li>results to be shared with data providers; </li></ul></ul><ul><ul><li>develop a best practice. </li></ul></ul><ul><li>  </li></ul><ul><li>Problem analysis in Aph study limited to: </li></ul><ul><ul><li>source link </li></ul></ul><ul><ul><li>presence/absence of citation metadata elements </li></ul></ul><ul><li>  </li></ul><ul><li>Results: </li></ul><ul><ul><li>OpenURL quality model: compares elements in Aph OpenURLs to those of other providers. </li></ul></ul><ul><ul><li>No scores, but model is first step towards scoring system. </li></ul></ul>*Hughes, Baden. 2004. Metadata Quality Evaluation: Experience from the Open Language Archives Community. In Digital Libraries: International Collaboration and Cross-Fertilization. Ed. Zhaoneng Chen et al. Berlin: Springer-Verlag, 2004, pp. 320-329.
  11. 11. Creation of IOTA <ul><li>NISO accepts proposal to take Aph Study to wider community </li></ul><ul><ul><li>Improving OpenURLs Through Analytics (IOTA): </li></ul></ul><ul><ul><ul><li>Formed in January 2010. </li></ul></ul></ul><ul><li>  </li></ul><ul><li>Basic Assumptions: </li></ul><ul><ul><li>Results are achieved through an analytical investigation of how OpenURL links work. </li></ul></ul><ul><ul><li>Practical: Not OpenURL standard is addressed, but links (OpenURLs) generated by standard. </li></ul></ul><ul><ul><li>Selective changes to OpenURLs will lead to significant improvement in linking success rate. </li></ul></ul><ul><ul><ul><li>Motto: &quot;small changes. big improvements&quot; </li></ul></ul></ul>
  12. 12. Desired Outcomes <ul><ul><li>Produce qualitative reports that will help OpenURL providers quickly compare their OpenURL quality to that of their peers. </li></ul></ul><ul><ul><li>Develop community-recognized index for measuring the quality of OpenURL links generated by content providers. </li></ul></ul><ul><li>   </li></ul><ul><ul><li>Method: </li></ul></ul><ul><ul><ul><li>fair; </li></ul></ul></ul><ul><ul><ul><li>transparent; </li></ul></ul></ul><ul><ul><ul><li>scalable across all OpenURLs and their providers. </li></ul></ul></ul>
  13. 13. Why are OpenURL reports important? <ul><li>Content providers can submit their OpenURL data: </li></ul><ul><ul><li>Compare their OpenURL data with other vendors; </li></ul></ul><ul><li>Institutions can submit their OpenURL data: </li></ul><ul><ul><li>See how OpenURLs from providers work and make local adjustments to their OpenURL setup </li></ul></ul><ul><li>Third parties can use IOTA’s OpenURL data: </li></ul><ul><ul><li>E.g. link resolver vendors, web-scale discovery system vendors can use reports to adjust their OpenURL linking; </li></ul></ul>
  14. 14. Running reports
  15. 15. Reports: Log file providers
  16. 16. Report: source (vendor or database)
  17. 17. Report: selecting source = vendor
  18. 18. Report: element, source = vendor
  19. 19. Report: pattern, source = vendor
  20. 20. Report: element, source = database
  21. 21. Report: pattern, source = database
  22. 22. Report: element and pattern frequency
  23. 23. Report: element & pattern frequency - selecting vendor
  24. 24. Report: element/pattern frequency: Choosing Metric
  25. 25. Report: element/pattern by vendor
  26. 26. Report: element & pattern frequency - selecting dbase
  27. 27. Report: element/pattern by database
  28. 28. Reporting System: current improvements <ul><li>Consolidating variant instances of databases and vendors if the same; </li></ul><ul><li>Separating article-like requests from book-like requests </li></ul><ul><ul><li>Either/Or situation: most resources do not offer both formats </li></ul></ul><ul><ul><li>Once separation is completed, users will be given corresponding options to select OpenURL data by format: ARTICLE or BOOK </li></ul></ul>
  29. 29. OpenURL Quality Index: initial version <ul><li>1. Core elements: </li></ul><ul><ul><li>Any element contained in IOTA's OpenURL reporting system; </li></ul></ul><ul><ul><li>10M OpenURLs already obtained from libraries content providers. </li></ul></ul><ul><li>  </li></ul><ul><li>2. Scoring system based on assumption: </li></ul><ul><ul><li>Correlation exists between </li></ul></ul><ul><ul><ul><li># of core elements (&quot;OpenURL completeness&quot;) & </li></ul></ul></ul><ul><ul><ul><li>ability of OpenURLs to link to specific content. </li></ul></ul></ul><ul><li>  </li></ul><ul><li>3.  Weighting assigned to core elements: </li></ul><ul><ul><li>Based on relative importance </li></ul></ul><ul><ul><ul><li>spage vs atitle </li></ul></ul></ul><ul><ul><ul><li>issn vs jtitle </li></ul></ul></ul><ul><ul><ul><li>doi/pmid vs date, etc. </li></ul></ul></ul>
  30. 30. OpenURL Quality Index: vendor rating
  31. 31. Work in Progress <ul><ul><li>Element weighting still in progress: </li></ul></ul><ul><ul><ul><li>E.g., importance of identifiers (doi, pmid) vs bibliographic data (issn, volume, spage). </li></ul></ul></ul><ul><li>  </li></ul><ul><ul><li>Currently, IOTA focuses on OpenURLs from citation sources only. OpenURL quality is also influenced by: </li></ul></ul><ul><ul><ul><li>knowledge base, </li></ul></ul></ul><ul><ul><ul><li>resolver, </li></ul></ul></ul><ul><ul><ul><li>full-text provider (target). </li></ul></ul></ul><ul><li>  </li></ul><ul><ul><li>High &quot;completeness&quot; score of OpenURLs not always indicative of &quot;success&quot; in linking to full texts </li></ul></ul><ul><ul><ul><li>Combination of multiple indexes along linking nodes may provide more complete picture. </li></ul></ul></ul>
  32. 32. IOTA & KBART: related through OpenURL <ul><li>IOTA node: </li></ul><ul><ul><li>analyzing data sent from OpenURL source to link resolver. </li></ul></ul><ul><li>KBART node: </li></ul><ul><ul><li>creating formatting best practices for data sent from content providers to knowledge base (and link resolver) vendors. </li></ul></ul>
  33. 33. KBART/IOTA joint initiative: underway <ul><li>KBART/IOTA node : </li></ul><ul><ul><li>Exploring together the third source of failures: </li></ul></ul><ul><ul><ul><li>link-to (or target) syntax and behavior which couples link resolvers to content providers </li></ul></ul></ul><ul><li>Collaboration begun in March 2011 is meant to address OpenURL quality in a broader context. </li></ul>
  34. 34. How can I get involved? <ul><li>If you are a content provider: </li></ul><ul><ul><li>Contribute data to IOTA </li></ul></ul><ul><ul><li>Review the IOTA data </li></ul></ul><ul><ul><li>This data is meant to help make improvements in your OpenURL linking. </li></ul></ul><ul><li>  </li></ul><ul><li>If you are a librarian: </li></ul><ul><ul><li>Contribute data to IOTA </li></ul></ul><ul><ul><li>Help spread the word to vendors </li></ul></ul>
  35. 35. IOTA web presence
  36. 36. Questions? <ul><li> </li></ul><ul><li> </li></ul><ul><li>@nisoiota on twitter </li></ul>Rafal Kasprowski Electronic Resources Librarian Rice University Fondren Library MS 44 Houston, TX 77005 USA [email_address]