IOTA @ NASIG 2011: Measuring the Quality of OpenURL LinksPresentation Transcript
NISO’s IOTA Initiative Measuring the Quality of OpenURL Links NASIG Annual Conference St. Louis, MO June 2 – 5, 2011 Rafal Kasprowski, Rice University
In the Beginning: Full-text linking and Advent of OpenURL
IOTA: Created in response to OpenURL linking problems
IOTA’s analytical approach
Community-derived reports comparing quality of vendor OpenURLs
Concept of the OpenURL Quality Index
IOTA & KBART: relationship & joint initiative
Community involvement in IOTA: necessary for best outcomes
Before OpenURL: Proprietary Linking
Certain A&I database providers (e.g., CSA, PubMed) offered full-text linking option for a select number of content providers.
Libraries manually activated full-text linking with providers they had subscriptions with.
A&I --> Full Text
Proprietary Linking: Pros and Cons
Linking had to be activated manually by libraries for each full-text provider.
A&I providers offering this option were few.
Selection of full-text providers was limited.
Once set up, the static links to full texts were accurate.
Problem source pinpointed easily: A&I --> Full Text
Advent of OpenURL
Objective: Deliver full texts unrestrained by proprietary silos.
Open standard generating dynamic links at time of request.
A-Z list (e.g., e-journal, e-books):
Knowledge base (KB) with library's holdings.
Replaces librarian as intermediary in linking.
Indicates provider of "appropriate copy"
A&I ("Source") --> A-Z list ("KB") --> Full Text ("Target")
A, Bernand , et al. " A versatile nanotechnology to connect individual nano-objects for the fabrication of hybrid single-electron devices. " Nanotechnology 21 , no. 44 ( November 5, 2010 ): 445201 . Academic Search Complete , EBSCO host (accessed October 24, 2010).
KB/Resolver vendors took over most of the linking setup: Less work for libraries and providers.
Participation by A&I platforms and full-text providers exceeded proprietary linking: OpenURL scales better.
Dynamic linking less predictable than static linking: more difficult to pinpoint cause of link failures.
OpenURL linking not improved significantly last 10 years.
No systematic method exists to benchmark OpenURLs.
Identifying source of problem…
"72% of respondents to the online survey either agreed or strongly agreed that a significant problem for link resolvers is the generation of incomplete or inaccurate OpenURLs by databases (for example, A&I products)."
Culling, James. 2007. Link Resolvers and the Serials Supply Chain: Final Project Report for UKSG, p.33. http://www.uksg.org/sites/uksg.org/files/uksg_link_resolvers_final_report.pdf.
Defining methodology for addressing problem…
Recently, researchers have indicated the need for metadata quality metrics , including:
conformance to expectations;
logical consistency and coherence.
Bruce, Thomas R. and Hillmann, Diane I. 2004. The Continuum of Metadata Quality: Defining, Expressing, Exploiting. In Metadata in Practice . Ed. Diane I. Hillmann and Elaine L. Westbrooks. Chicago: American Library Association, pp. 238-256.
Année philologique OpenURL Study
2008 Cornell study led by Adam Chandler*
Problem: Too often links sent from Aph did not successfully resolve to requested resource.
Objective: Examine quality of OpenURLs offered to users by Aph in order to improve the linking.
Aph Study investigated:
Faulty citation metadata from source database.
Method to evaluate the OpenURLs.
*Chandler, Adam. 2009. Results of L’Année philologique online OpenURL Quality Investigation: Mellon Planning Grant Final Report. http://metadata.library.cornell.edu/oq/files/200902%20lannee-mellonreport-openurlquality-final.pdf.
Scoring System & Aph Study Outcomes
Concept of scoring in Aph study (based on B. Hughes study)*
establish a baseline for comparison;
results to be shared with data providers;
develop a best practice.
Problem analysis in Aph study limited to:
presence/absence of citation metadata elements
OpenURL quality model: compares elements in Aph OpenURLs to those of other providers.
No scoring was achieved for Aph , but model is first step towards scoring system.
*Hughes, Baden. 2004. Metadata Quality Evaluation: Experience from the Open Language Archives Community. In Digital Libraries: International Collaboration and Cross-Fertilization. Ed. Zhaoneng Chen et al. Berlin: Springer-Verlag, 2004, pp. 320-329.
Creation of IOTA
Accepts proposal to take Aph Study to wider community.
New OpenURL quality metrics initiative formed in Jan. 2011.
Branded as: I mproving O penURLs T hrough A nalytics.
Results are achieved through an analytical investigation of how OpenURL links work.
Practical Goal: Not the OpenURL standard is addressed, but the links (OpenURLs) generated by standard.
Selective changes to OpenURLs will lead to significant improvement in linking success rate.
Motto: "small changes. big improvements"
IOTA Desired Outcomes
Produce qualitative reports that will help OpenURL providers quickly compare their OpenURL quality to that of their peers.
Develop community-recognized index for measuring the quality of OpenURL links generated by content providers.
scalable across all OpenURLs and their providers.
How is comparing OpenURLs useful?
Content providers generating OpenURLs can:
compare their OpenURLs with other providers;
make improvements to their OpenURLs.
compare OpenURL providers;
make local adjustments to OpenURL setup.
Link resolver vendors can:
compare OpenURL providers;
change their OpenURL provider settings:
Web-scale discovery products.
Viewing how a particular (1) vendor or (2) database
A. uses OpenURL elements (element frequency)
B. formats OpenURL elements (pattern frequency)
Element / Pattern reports
Viewing how a particular (1) element or format
A. is used across vendors
B. is used across databases
Reports: Log file providers
Report: source (vendor or database)
Report: source = vendor
Report: element, source = vendor
Report: pattern, source = vendor
Report: source = database
Report: element, source = database
Report: pattern, source = database
Report: element and pattern frequency
Report: element & pattern frequency: Vendor option