• Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,344
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. NISO’s IOTA Initiative Measuring the Quality of OpenURL Links NASIG Annual Conference St. Louis, MO June 2 – 5, 2011   Rafal Kasprowski, Rice University
  • 2. Agenda
      • In the Beginning: Full-text linking and Advent of OpenURL
    •  
      • IOTA: Created in response to OpenURL linking problems
    •  
      • IOTA’s analytical approach
    •  
      • Community-derived reports comparing quality of vendor OpenURLs
    •  
      • Concept of the OpenURL Quality Index
      • IOTA & KBART: relationship & joint initiative
    •  
      • Community involvement in IOTA: necessary for best outcomes
  • 3. Before OpenURL: Proprietary Linking
    •  
      • Certain A&I database providers (e.g., CSA, PubMed) offered full-text linking option for a select number of content providers.
    •  
      • Libraries manually activated full-text linking with providers they had subscriptions with.
    •  
      • A&I --> Full Text
    •  
  • 4. Proprietary Linking: Pros and Cons
      • Linking had to be activated manually by libraries for each full-text provider.
    •  
      • A&I providers offering this option were few.
    •  
      • Selection of full-text providers was limited.
    •  
    • But...
      •   Once set up, the static links to full texts were accurate.
      • Problem source pinpointed easily: A&I --> Full Text
  • 5. Advent of OpenURL
      • Objective: Deliver full texts unrestrained by proprietary silos.
    •  
      • Open standard generating dynamic links at time of request.
    •  
      • A-Z list (e.g., e-journal, e-books):
        • Knowledge base (KB) with library's holdings.
        • Replaces librarian as intermediary in linking.
        • Indicates provider of "appropriate copy"
    •  
      • A&I ("Source") --> A-Z list ("KB") --> Full Text ("Target")
  • 6.
    • A, Bernand , et al. " A versatile nanotechnology to connect individual nano-objects for the fabrication of hybrid single-electron devices. " Nanotechnology 21 , no. 44 ( November 5, 2010 ): 445201 . Academic Search Complete , EBSCO host (accessed October 24, 2010).
    OpenURL: resolver, syntax, linking nodes http://ps4ps6lm2r.search.serialssolutions.com/?issn=0957-4484&volume= 21 &issue= 44 &date= 20101105 &spage= 445201 &title= Nanotechnology &atitle= A+ versatile+nanotechnology+to+connect+individual+nano-objects+for+the+ fabrication+of+hybrid+single-electron+devices. &aulast= A++Bernand Source Citation Target OpenURL (Source OpenURL structured similarly)
  • 7. Pros & Cons of OpenURL
    • Pros:
      • KB/Resolver vendors took over most of the linking setup: Less work for libraries and providers.
      • Participation by A&I platforms and full-text providers exceeded proprietary linking: OpenURL scales better.
    •  
    • Cons:
      • Dynamic linking less predictable than static linking: more difficult to pinpoint cause of link failures.
      • OpenURL linking not improved significantly last 10 years.
      • No systematic method exists to benchmark OpenURLs.
  • 8. Identifying source of problem…
    • "72% of respondents to the online survey either agreed or strongly agreed that a significant problem for link resolvers is the generation of incomplete or inaccurate OpenURLs by databases (for example, A&I products)."
    • Culling, James. 2007. Link Resolvers and the Serials Supply Chain: Final Project Report for UKSG, p.33. http://www.uksg.org/sites/uksg.org/files/uksg_link_resolvers_final_report.pdf.
    •  
    • Defining methodology for addressing problem…
    •  
    • Recently, researchers have indicated the need for metadata quality metrics , including:
      • completeness;
      • accuracy;
      • conformance to expectations;
      • logical consistency and coherence.
    • Bruce, Thomas R. and Hillmann, Diane I. 2004. The Continuum of Metadata Quality: Defining, Expressing, Exploiting. In Metadata in Practice . Ed. Diane I. Hillmann and Elaine L. Westbrooks. Chicago: American Library Association, pp. 238-256.
  • 9. Année philologique OpenURL Study
    • 2008 Cornell study led by Adam Chandler*
      • Problem: Too often links sent from Aph did not successfully resolve to requested resource.
      • Objective: Examine quality of OpenURLs offered to users by Aph in order to improve the linking.
    •  
    • Aph Study investigated:
      • Faulty citation metadata from source database.
      • Method to evaluate the OpenURLs.
    *Chandler, Adam. 2009. Results of L’Année philologique online OpenURL Quality Investigation: Mellon Planning Grant Final Report. http://metadata.library.cornell.edu/oq/files/200902%20lannee-mellonreport-openurlquality-final.pdf.
  • 10. Scoring System & Aph Study Outcomes
    • Concept of scoring in Aph study (based on B. Hughes study)*
      • establish a baseline for comparison;
      • results to be shared with data providers;
      • develop a best practice.
    •  
    • Problem analysis in Aph study limited to:
      • source link
      • presence/absence of citation metadata elements
    •  
    • Results:
      • OpenURL quality model: compares elements in Aph OpenURLs to those of other providers.
      • No scoring was achieved for Aph , but model is first step towards scoring system.
    *Hughes, Baden. 2004. Metadata Quality Evaluation: Experience from the Open Language Archives Community. In Digital Libraries: International Collaboration and Cross-Fertilization. Ed. Zhaoneng Chen et al. Berlin: Springer-Verlag, 2004, pp. 320-329.
  • 11. Creation of IOTA
    • NISO:
      • Accepts proposal to take Aph Study to wider community.
      • New OpenURL quality metrics initiative formed in Jan. 2011.
      • Branded as: I mproving O penURLs T hrough A nalytics.
    •  
    • Basic Assumptions:
      • Results are achieved through an analytical investigation of how OpenURL links work.
      • Practical Goal: Not the OpenURL standard is addressed, but the links (OpenURLs) generated by standard.
      • Selective changes to OpenURLs will lead to significant improvement in linking success rate.
        • Motto: "small changes. big improvements"
  • 12. IOTA Desired Outcomes
      • Produce qualitative reports that will help OpenURL providers quickly compare their OpenURL quality to that of their peers.
      • Develop community-recognized index for measuring the quality of OpenURL links generated by content providers.
    •   
      • Method:
        • fair;
        • transparent;
        • scalable across all OpenURLs and their providers.
  • 13. How is comparing OpenURLs useful?
    • Content providers generating OpenURLs can:
      • compare their OpenURLs with other providers;
      • make improvements to their OpenURLs.
    • Institutions can:
      • compare OpenURL providers;
      • make local adjustments to OpenURL setup.
    • Link resolver vendors can:
      • compare OpenURL providers;
      • change their OpenURL provider settings:
        • Link resolvers;
        • Web-scale discovery products.
  • 14. Report types
    • Source reports
      • Viewing how a particular (1) vendor or (2) database
        • A. uses OpenURL elements (element frequency)
        • B. formats OpenURL elements (pattern frequency)
    • Element / Pattern reports
      • Viewing how a particular (1) element or format
        • A. is used across vendors
        • B. is used across databases
  • 15. Running reports
  • 16. Reports: Log file providers
  • 17. Report: source (vendor or database)
  • 18. Report: source = vendor
  • 19. Report: element, source = vendor
  • 20. Report: pattern, source = vendor
  • 21. Report: source = database
  • 22. Report: element, source = database
  • 23. Report: pattern, source = database
  • 24. Report: element and pattern frequency
  • 25. Report: element & pattern frequency: Vendor option
  • 26. Report: element/pattern frequency: Choosing Metric
  • 27. Report: element/pattern by vendor
  • 28. Report: element & pattern frequency: Database option
  • 29. Report: element/pattern by database
  • 30. Reporting System: improvements underway
    • Consolidating variant instances of databases and vendors if the same;
    • Separating article-like requests from book-like requests
      • Either/Or situation: most resources do not offer both formats
      • Once separation is completed, users will be given corresponding options to select OpenURL data by format: ARTICLE or BOOK
    • These improvements will also benefit the accuracy of the OpenURL scoring system.
  • 31. OpenURL Quality Index: initial version
    • 1. Core elements:
      • Any element contained in IOTA's OpenURL reporting system;
      • 13M OpenURLs already obtained from libraries content providers.
    •  
    • 2. Scoring system based on assumption :
      • Correlation exists between
        • # of core elements ("OpenURL completeness") &
        • ability of OpenURLs to link to specific content.
    •  
    • 3.  Weighting assigned to core elements:
      • Based on relative importance
        • spage vs atitle
        • issn vs jtitle
        • doi/pmid vs date, etc.
  • 32. OpenURL Quality Index: vendor rating
  • 33. Work in Progress
      • Element weighting still in progress:
        • E.g., importance of identifiers (doi, pmid) vs bibliographic data (issn, volume, spage).
    •  
      • Currently, IOTA focuses on OpenURLs from citation sources only. OpenURL quality is also influenced by:
        • knowledge base,
        • resolver,
        • full-text provider (target).
    •  
      • High "completeness" score of OpenURLs not always indicative of "success" in linking to full texts
        • Combination of multiple indexes along linking nodes may provide more complete picture.
  • 34. IOTA & KBART: NISO working groups
    • IOTA
    • Deals with issues specific to OpenURL linking;
    • Seeks improvements in OpenURL elements used by:
      • OpenURL providers.
    • KBART
    • “ Knowledge Bases And Related Tools”
    • Deals with data issues at the KB level
    • Seeks improvements in data exchange practices between:
      • content providers (e.g. OpenURL providers);
      • product vendors (e.g. link resolver vendors);
      • subscription agents.
  • 35. IOTA & KBART: related through OpenURL
    • IOTA node:
      • analyzing data sent from OpenURL source to link resolver.
    • KBART node:
      • creating formatting best practices for data sent from content providers to knowledge base (and link resolver) vendors.
  • 36. KBART-IOTA joint initiative
    • KBART-IOTA node :
      • Exploring together the third source of failures:
        • link-to (or target) syntax and behavior which couples link resolvers to content providers
    • Collaboration begun in March 2011 is meant to address OpenURL quality in a broader context.
  • 37. How can I get involved?
    • If you are an OpenURL provider:
      • Contribute data to IOTA
      • Review the IOTA data
      • This data is meant to help you make improvements in your OpenURL linking
    •  
    • If you are a librarian:
      • Contribute data to IOTA
      • Spread the word to vendors about IOTA
      • OpenURL data contact:
      • Adam Chandler, alc28@cornell.edu
  • 38. IOTA Web Presence
  • 39. Questions?
    • http://openurlquality.niso.org
    • http://www.niso.org/workrooms/openurlquality
    • @nisoiota on twitter
    Rafal Kasprowski Electronic Resources Librarian Rice University, Fondren Library MS 44 Houston, TX 77005 [email_address] http://www.slideshare.net/rkaspro