Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
NISO’s IOTA Working Group Improving OpenURLs Through Analytics UKSG Conference Harrogate, United Kingdom April 4 – 6, 2011...
Agenda <ul><ul><li>In the Beginning: Full-text linking and Advent of OpenURL </li></ul></ul><ul><li>  </li></ul><ul><ul><l...
Before OpenURL: Proprietary Linking <ul><li>  </li></ul><ul><ul><li>A&I database providers offered option for full-text li...
Proprietary Linking: Cons and Pro <ul><ul><li>Linking had to be activated manually by libraries for each full-text provide...
Advent of OpenURL <ul><ul><li>Objective: Deliver full texts unrestrained by proprietary silos. </li></ul></ul><ul><li>  </...
<ul><li>A, Bernand , et al. &quot; A versatile nanotechnology to connect individual nano-objects for the fabrication of hy...
Pros & Cons of OpenURL <ul><li>Pros : </li></ul><ul><ul><li>KB/Resolver vendors took over most of the linking setup: Less ...
Problem Statement & Methodology <ul><li>&quot;72% of respondents to the online survey either agreed or strongly agreed tha...
Année philologique  OpenURL Study <ul><li>2008 Cornell study led by Adam Chandler* </li></ul><ul><ul><li>Problem: Too ofte...
Scoring System &  Aph  Study Outcomes <ul><li>Concept of scoring in  Aph  study (based on B. Hughes study)* </li></ul><ul>...
Creation of IOTA <ul><li>NISO accepts proposal to take  Aph Study  to wider community </li></ul><ul><ul><li>Improving Open...
Desired Outcomes <ul><ul><li>Produce qualitative reports that will help OpenURL providers quickly compare their OpenURL qu...
Why are OpenURL reports important? <ul><li>Content providers can submit their OpenURL data: </li></ul><ul><ul><li>Compare ...
Running reports
Reports: Log file providers
Report: source (vendor or database)
Report: selecting source = vendor
Report: element, source = vendor
Report: pattern, source = vendor
Report: element, source = database
Report: pattern, source = database
Report: element and pattern frequency
Report: element & pattern frequency - selecting vendor
Report: element/pattern frequency: Choosing Metric
Report: element/pattern by vendor
Report: element & pattern frequency - selecting dbase
Report: element/pattern by database
Reporting System: current improvements <ul><li>Consolidating variant instances of databases and vendors if the same; </li>...
OpenURL Quality Index: initial version <ul><li>1. Core elements: </li></ul><ul><ul><li>Any element contained in IOTA's Ope...
OpenURL Quality Index: vendor rating
Work in Progress <ul><ul><li>Element weighting still in progress: </li></ul></ul><ul><ul><ul><li>E.g., importance of ident...
IOTA & KBART: related through OpenURL <ul><li>IOTA node:  </li></ul><ul><ul><li>analyzing data sent from  OpenURL source  ...
KBART/IOTA joint initiative: underway <ul><li>KBART/IOTA node : </li></ul><ul><ul><li>Exploring together the third source ...
How can I get involved? <ul><li>If you are a content provider: </li></ul><ul><ul><li>Contribute data to IOTA </li></ul></u...
IOTA web presence
Questions? <ul><li>http://openurlquality.niso.org </li></ul><ul><li>http://www.niso.org/workrooms/openurlquality </li></ul...
Upcoming SlideShare
Loading in …5
×

IOTA OpenURL Quality @ 2011 UKSG Conference

1,209 views

Published on

  • Be the first to comment

IOTA OpenURL Quality @ 2011 UKSG Conference

  1. 1. NISO’s IOTA Working Group Improving OpenURLs Through Analytics UKSG Conference Harrogate, United Kingdom April 4 – 6, 2011   Rafal Kasprowski, Rice University
  2. 2. Agenda <ul><ul><li>In the Beginning: Full-text linking and Advent of OpenURL </li></ul></ul><ul><li>  </li></ul><ul><ul><li>IOTA: Created in response to OpenURL linking problems </li></ul></ul><ul><li>  </li></ul><ul><ul><li>IOTA’s analytical approach </li></ul></ul><ul><li>  </li></ul><ul><ul><li>Community-derived reports comparing quality of vendor OpenURLs </li></ul></ul><ul><li>  </li></ul><ul><ul><li>Concept of the OpenURL Quality Index </li></ul></ul><ul><ul><li>IOTA & KBART: relationship & joint initiative </li></ul></ul><ul><li>  </li></ul><ul><ul><li>Community involvement in IOTA: necessary for best outcomes </li></ul></ul>
  3. 3. Before OpenURL: Proprietary Linking <ul><li>  </li></ul><ul><ul><li>A&I database providers offered option for full-text linking (e.g., CSA, PubMed, etc.) . </li></ul></ul><ul><li>  </li></ul><ul><ul><li>Libraries manually activated linking to full-text providers they had subscriptions with. </li></ul></ul><ul><li>  </li></ul><ul><ul><li>A&I --> Full Text </li></ul></ul><ul><li>  </li></ul>
  4. 4. Proprietary Linking: Cons and Pro <ul><ul><li>Linking had to be activated manually by libraries for each full-text provider. </li></ul></ul><ul><li>  </li></ul><ul><ul><li>A&I providers offering this option were few. </li></ul></ul><ul><li>  </li></ul><ul><ul><li>Selection of full-text providers was limited. </li></ul></ul><ul><li>  </li></ul><ul><li>But... </li></ul><ul><ul><li>  Once set up, the static links to full texts were accurate. </li></ul></ul>
  5. 5. Advent of OpenURL <ul><ul><li>Objective: Deliver full texts unrestrained by proprietary silos. </li></ul></ul><ul><li>  </li></ul><ul><ul><li>Open standard generating link at time of request. </li></ul></ul><ul><li>  </li></ul><ul><ul><li>Library's holdings indicate provider of &quot;appropriate copy&quot;. </li></ul></ul><ul><li>  </li></ul><ul><ul><li>A-Z list (e.g., e-journal, e-books): </li></ul></ul><ul><ul><ul><li>Knowledge base (KB) with library's holdings. </li></ul></ul></ul><ul><ul><ul><li>Intermediary in linking. </li></ul></ul></ul><ul><li>  </li></ul><ul><ul><li>A&I (&quot;Source&quot;) --> A-Z list (&quot;KB&quot;) --> Full Text (&quot;Target&quot;) </li></ul></ul>
  6. 6. <ul><li>A, Bernand , et al. &quot; A versatile nanotechnology to connect individual nano-objects for the fabrication of hybrid single-electron devices. &quot; Nanotechnology 21 , no. 44 ( November 5, 2010 ): 445201 . Academic Search Complete , EBSCO host (accessed October 24, 2010). </li></ul>OpenURL Syntax and Resolver http://ps4ps6lm2r.search.serialssolutions.com/?issn=0957-4484&volume= 21 &issue= 44 &date= 20101105 &spage= 445201 &title= Nanotechnology &atitle= A+ versatile+nanotechnology+to+connect+individual+nano-objects+for+the+ fabrication+of+hybrid+single-electron+devices. &aulast= A++Bernand Source Citation Target OpenURL (Source OpenURL structured similarly)
  7. 7. Pros & Cons of OpenURL <ul><li>Pros : </li></ul><ul><ul><li>KB/Resolver vendors took over most of the linking setup: Less work for libraries and providers. </li></ul></ul><ul><ul><li>Participation by A&I platforms and full-text providers exceeded proprietary linking: OpenURL scales better </li></ul></ul><ul><li>  </li></ul><ul><li>Cons : </li></ul><ul><ul><li>Dynamic linking less predictable than static linking: more difficult to pinpoint cause of link failures </li></ul></ul><ul><ul><li>OpenURL linking not improved significantly last 10 years. </li></ul></ul><ul><ul><li>No systematic method exists to benchmark OpenURLs. </li></ul></ul>
  8. 8. Problem Statement & Methodology <ul><li>&quot;72% of respondents to the online survey either agreed or strongly agreed that a significant problem for link resolvers is the generation of incomplete or inaccurate OpenURLs by databases (for example, A&I products).&quot; </li></ul><ul><li>Culling, James. 2007. Link Resolvers and the Serials Supply Chain: Final Project Report for UKSG, p.33. http://www.uksg.org/sites/uksg.org/files/uksg_link_resolvers_final_report.pdf. </li></ul><ul><li>  </li></ul><ul><li>  </li></ul><ul><li>  </li></ul><ul><li>Recently, researchers have indicated the need for metadata quality metrics , including: </li></ul><ul><ul><li>completeness; </li></ul></ul><ul><ul><li>accuracy; </li></ul></ul><ul><ul><li>conformance to expectations; </li></ul></ul><ul><ul><li>logical consistency and coherence. </li></ul></ul><ul><li>Bruce, Thomas R. and Hillmann, Diane I. 2004. The Continuum of Metadata Quality: Defining, Expressing, Exploiting. In Metadata in Practice . Ed. Diane I. Hillmann and Elaine L. Westbrooks. Chicago: American Library Association, pp. 238-256. </li></ul>
  9. 9. Année philologique OpenURL Study <ul><li>2008 Cornell study led by Adam Chandler* </li></ul><ul><ul><li>Problem: Too often links sent from Aph did not successfully resolve to requested resource. </li></ul></ul><ul><ul><li>Objective: Examine quality of OpenURLs offered to users by Aph in order to improve the linking. </li></ul></ul><ul><li>  </li></ul><ul><li>Aph Study investigated: </li></ul><ul><ul><li>Faulty citation metadata from source database. </li></ul></ul><ul><ul><li>Method to evaluate the OpenURLs. </li></ul></ul>*Chandler, Adam. 2009. Results of L’Année philologique online OpenURL Quality Investigation: Mellon Planning Grant Final Report. http://metadata.library.cornell.edu/oq/files/200902%20lannee-mellonreport-openurlquality-final.pdf.
  10. 10. Scoring System & Aph Study Outcomes <ul><li>Concept of scoring in Aph study (based on B. Hughes study)* </li></ul><ul><ul><li>establish a baseline for comparison; </li></ul></ul><ul><ul><li>results to be shared with data providers; </li></ul></ul><ul><ul><li>develop a best practice. </li></ul></ul><ul><li>  </li></ul><ul><li>Problem analysis in Aph study limited to: </li></ul><ul><ul><li>source link </li></ul></ul><ul><ul><li>presence/absence of citation metadata elements </li></ul></ul><ul><li>  </li></ul><ul><li>Results: </li></ul><ul><ul><li>OpenURL quality model: compares elements in Aph OpenURLs to those of other providers. </li></ul></ul><ul><ul><li>No scores, but model is first step towards scoring system. </li></ul></ul>*Hughes, Baden. 2004. Metadata Quality Evaluation: Experience from the Open Language Archives Community. In Digital Libraries: International Collaboration and Cross-Fertilization. Ed. Zhaoneng Chen et al. Berlin: Springer-Verlag, 2004, pp. 320-329.
  11. 11. Creation of IOTA <ul><li>NISO accepts proposal to take Aph Study to wider community </li></ul><ul><ul><li>Improving OpenURLs Through Analytics (IOTA): </li></ul></ul><ul><ul><ul><li>Formed in January 2010. </li></ul></ul></ul><ul><li>  </li></ul><ul><li>Basic Assumptions: </li></ul><ul><ul><li>Results are achieved through an analytical investigation of how OpenURL links work. </li></ul></ul><ul><ul><li>Practical: Not OpenURL standard is addressed, but links (OpenURLs) generated by standard. </li></ul></ul><ul><ul><li>Selective changes to OpenURLs will lead to significant improvement in linking success rate. </li></ul></ul><ul><ul><ul><li>Motto: &quot;small changes. big improvements&quot; </li></ul></ul></ul>
  12. 12. Desired Outcomes <ul><ul><li>Produce qualitative reports that will help OpenURL providers quickly compare their OpenURL quality to that of their peers. </li></ul></ul><ul><ul><li>Develop community-recognized index for measuring the quality of OpenURL links generated by content providers. </li></ul></ul><ul><li>   </li></ul><ul><ul><li>Method: </li></ul></ul><ul><ul><ul><li>fair; </li></ul></ul></ul><ul><ul><ul><li>transparent; </li></ul></ul></ul><ul><ul><ul><li>scalable across all OpenURLs and their providers. </li></ul></ul></ul>
  13. 13. Why are OpenURL reports important? <ul><li>Content providers can submit their OpenURL data: </li></ul><ul><ul><li>Compare their OpenURL data with other vendors; </li></ul></ul><ul><li>Institutions can submit their OpenURL data: </li></ul><ul><ul><li>See how OpenURLs from providers work and make local adjustments to their OpenURL setup </li></ul></ul><ul><li>Third parties can use IOTA’s OpenURL data: </li></ul><ul><ul><li>E.g. link resolver vendors, web-scale discovery system vendors can use reports to adjust their OpenURL linking; </li></ul></ul>
  14. 14. Running reports
  15. 15. Reports: Log file providers
  16. 16. Report: source (vendor or database)
  17. 17. Report: selecting source = vendor
  18. 18. Report: element, source = vendor
  19. 19. Report: pattern, source = vendor
  20. 20. Report: element, source = database
  21. 21. Report: pattern, source = database
  22. 22. Report: element and pattern frequency
  23. 23. Report: element & pattern frequency - selecting vendor
  24. 24. Report: element/pattern frequency: Choosing Metric
  25. 25. Report: element/pattern by vendor
  26. 26. Report: element & pattern frequency - selecting dbase
  27. 27. Report: element/pattern by database
  28. 28. Reporting System: current improvements <ul><li>Consolidating variant instances of databases and vendors if the same; </li></ul><ul><li>Separating article-like requests from book-like requests </li></ul><ul><ul><li>Either/Or situation: most resources do not offer both formats </li></ul></ul><ul><ul><li>Once separation is completed, users will be given corresponding options to select OpenURL data by format: ARTICLE or BOOK </li></ul></ul>
  29. 29. OpenURL Quality Index: initial version <ul><li>1. Core elements: </li></ul><ul><ul><li>Any element contained in IOTA's OpenURL reporting system; </li></ul></ul><ul><ul><li>10M OpenURLs already obtained from libraries content providers. </li></ul></ul><ul><li>  </li></ul><ul><li>2. Scoring system based on assumption: </li></ul><ul><ul><li>Correlation exists between </li></ul></ul><ul><ul><ul><li># of core elements (&quot;OpenURL completeness&quot;) & </li></ul></ul></ul><ul><ul><ul><li>ability of OpenURLs to link to specific content. </li></ul></ul></ul><ul><li>  </li></ul><ul><li>3.  Weighting assigned to core elements: </li></ul><ul><ul><li>Based on relative importance </li></ul></ul><ul><ul><ul><li>spage vs atitle </li></ul></ul></ul><ul><ul><ul><li>issn vs jtitle </li></ul></ul></ul><ul><ul><ul><li>doi/pmid vs date, etc. </li></ul></ul></ul>
  30. 30. OpenURL Quality Index: vendor rating
  31. 31. Work in Progress <ul><ul><li>Element weighting still in progress: </li></ul></ul><ul><ul><ul><li>E.g., importance of identifiers (doi, pmid) vs bibliographic data (issn, volume, spage). </li></ul></ul></ul><ul><li>  </li></ul><ul><ul><li>Currently, IOTA focuses on OpenURLs from citation sources only. OpenURL quality is also influenced by: </li></ul></ul><ul><ul><ul><li>knowledge base, </li></ul></ul></ul><ul><ul><ul><li>resolver, </li></ul></ul></ul><ul><ul><ul><li>full-text provider (target). </li></ul></ul></ul><ul><li>  </li></ul><ul><ul><li>High &quot;completeness&quot; score of OpenURLs not always indicative of &quot;success&quot; in linking to full texts </li></ul></ul><ul><ul><ul><li>Combination of multiple indexes along linking nodes may provide more complete picture. </li></ul></ul></ul>
  32. 32. IOTA & KBART: related through OpenURL <ul><li>IOTA node: </li></ul><ul><ul><li>analyzing data sent from OpenURL source to link resolver. </li></ul></ul><ul><li>KBART node: </li></ul><ul><ul><li>creating formatting best practices for data sent from content providers to knowledge base (and link resolver) vendors. </li></ul></ul>
  33. 33. KBART/IOTA joint initiative: underway <ul><li>KBART/IOTA node : </li></ul><ul><ul><li>Exploring together the third source of failures: </li></ul></ul><ul><ul><ul><li>link-to (or target) syntax and behavior which couples link resolvers to content providers </li></ul></ul></ul><ul><li>Collaboration begun in March 2011 is meant to address OpenURL quality in a broader context. </li></ul>
  34. 34. How can I get involved? <ul><li>If you are a content provider: </li></ul><ul><ul><li>Contribute data to IOTA </li></ul></ul><ul><ul><li>Review the IOTA data </li></ul></ul><ul><ul><li>This data is meant to help make improvements in your OpenURL linking. </li></ul></ul><ul><li>  </li></ul><ul><li>If you are a librarian: </li></ul><ul><ul><li>Contribute data to IOTA </li></ul></ul><ul><ul><li>Help spread the word to vendors </li></ul></ul>
  35. 35. IOTA web presence
  36. 36. Questions? <ul><li>http://openurlquality.niso.org </li></ul><ul><li>http://www.niso.org/workrooms/openurlquality </li></ul><ul><li>@nisoiota on twitter </li></ul>Rafal Kasprowski Electronic Resources Librarian Rice University Fondren Library MS 44 Houston, TX 77005 USA [email_address]

×