The document discusses the KBART working group, which aims to improve the effectiveness of OpenURL linking by addressing problems with holdings data accuracy, OpenURL implementation, and lack of knowledge about OpenURL. The group is developing best practice guidelines for data transfer between publishers, link resolvers, and libraries. This will help ensure patrons get accurate links to full text. The document outlines challenges in dealing with inaccurate data and getting all stakeholders to participate.

1. KNOWLEDGE BASES AND RELATED TOOLS: IMPROVING OPENURL EFFECTIVENESS Jason Price, PhD Claremont Colleges/SCELC KBART Working Group Member ER&L 2009 Conference UCLA X X KBART K ? ok ?

2. Today’s Outline OpenURL Overview Measure of success; Positives and negatives KBART: Reviewing Problems & Seeking Solutions KBART background, goals, membership Main problem areas & Solutions improve holdings data accuracy Improve application of OpenURL syntax from “sources” Improve knowledge of OpenURL & its importance & issues KBART Deliverables

3. OpenURL Overview The evolution of the OpenURL in reality: If links fail, patrons will turn to the tool that always works Three main problems with OpenURL today: Bad data ; Bad formatting ; Lack of knowledge; ‘ ‘

4. The Measure of Success Better access for patrons Fewer false positives: saying it’s available when its not Fewer false negatives: saying it’s not available when it is Best-case scenario: IF a patron is seeking an item, and her library offers access to it through exactly seven online resources, THEN the OpenURL resolver returns exactly seven accurate links to the full text AND the ‘best’ resources appear first

5. Why we do what we do… http://tinyurl.com/59txop

6. The OpenURL resolver window

7. Transport to the target database…

8. … containing the full text

9. Dan in Real Life… The Positives – it gets patrons to content they would not otherwise have found It’s a great leap forward in library services It’s fairly straightforward; it’s not incredibly complicated The Negatives – it doesn’t get patrons to content as effectively as it should Inaccurate data leads to bad and missing links Incorrect implementation doesn’t transfer metadata properly Lack of knowledge of its importance means: some vendors aren’t using it many of others aren’t investing in improved source implementation or more accurate & timely data transfer But first, a bit of history OpenURL in Real Life…

10. KBART: A History UKSG 2007 research report by James Culling, “Link Resolvers and the Serials Supply Chain” (at http://www.uksg.org/projects/linkfinal ) Provided ideas on improving usage and accuracy Recommended follow-up to address some specifics NISO partnership to broaden reach and include US audience

11. KBART: An Introduction K nowledge B ases A nd R elated T ools UKSG and NISO collaborative project Get better data for everyone – Those who provide data (publishers, aggregators) Those who process data (link resolvers, ERMs, etc.) Those who present data (libraries, consortia) All for THOSE WHO USE DATA – library patrons Ensuring timely transfer of accurate data to knowledgebases, ERMs, etc.

12. Who’s in KBART? Core working group chaired by Peter McCracken (Serials Solutions) and Charlie Rapple (TBI Communications; formerly Ingenta) Link resolver/ERM suppliers – Ex Libris, Serials Solutions Publishers – British Medical Journal Group, Taylor & Francis Subscription agents/aggregators – Credo, EBSCO, Swets Consortia – California Digital Library, SCELC Libraries – Claremont, Cornell, Edinburgh, Leicester, Princeton, Pacific Northwest Technical Lab Monitoring group More of these plus other related groups e.g. NASIG Anyone can join monitoring group sign up for updates: [email_address]

13. Problem Overview Knowledge bases Date coverage Title relations Licensing Data & transfer Supply chain Compliance accuracy format vol/issue vs date date granularity (day, month, season, year) title changes title mapping abbreviations ISSN/ISBN variations re-use of ISSN effect on licensing genericism/granularity misrepresentation package variations accuracy free content format ownership contacts/feedback mechanisms incentive informal structure unclear responsibilities duplication of effort file format format definitions; shoe-horning age of data accuracy frequency link syntax and granularity

14. KBART: Examining the problems “ OpenURL’s Negatives” Inaccurate holdings data leads to bad & missing links Incorrect implementation doesn’t transfer metadata properly Lack of knowledge means some vendors aren’t using it and the remainder aren’t improving it

15. Inaccurate Data – The problem Error Level False (+) including links to inaccessible content False (-) lacking links to accessible content Title Access not activated by publisher Accessible title not listed in KB/Catalog Date Range Part of access not activated by publisher OR Years of access over-represented in KB/Catalog Years of access under-represented in KB/Catalog

16. Inaccurate Data – Impact Listing of ≈ 120,000 articles needed correction (based on estimated ave. 6x/yr & 10 articles/issue) Error Level False ( + ) False ( - ) Title 290 journal yrs 1205 journal yrs Date Range 485 journal yrs

17. Inaccurate Data – Current responses REACTIVE – correcting data for individual articles that patrons report as inaccessible But what about the (large) majority that go unreported (esp. the false negatives that prove that “Google has lots of content ’not available through the library’”) PROACTIVE – before we get (or don’t get) complaints title by title or package by package extremely labor intensive An example

18. Proactive reconciliation of an ejournal package list General Process – library, consortium or KB vendor (Re-)Request updated access list from publisher Sample publisher list for accuracy Translate publisher list to match KB list Number of titles never matches Perform ISSN match with MS Access Watch for & integrate title changes, mergers, acquisitions and losses Watch for publisher-reuse of ISSNs/title combinations Identify date discrepancies manually (inconsistent formats) Decide when its ‘good enough’ and go live/distribute new list Lather, Rinse, Repeat Correcting Inaccurate Data – the hard way

19. Correcting Inaccurate Data – the hard way 226 titles = 16%

20. Inaccurate Data – The KBART Solutions Standardize transfer of data within and among supply chain participants Phase I - Best practices recommendations specifying: means of data transfer frequency of updates File structure Data elements – Mandatory and Optional e.g. Start and end date format & granularity

21. KBART: progressive data element recommendations Under consideration (Mandatory or optional?) Title level information Issue completeness (includes all articles?) Article completeness (Includes tables & figs?) Full text format (html vs pdf) Embargo period (granular specification) Moving wall (a la Nature/Palgrave) Genre Freely accessible content listed separately Ebook fields

22. If we build it, & they don’t come … How do we handle incorrect data? Grading? Policing? Shaming? Biggest and most difficult problem to solve Highlight to content providers how important completely accurate data is to their end users Consider the ‘false positive’: arrrgh, that’s frustrating… Consider the ‘false negative’: much, much worse: how would you ever know?

23. Incorrect Implementation – the problem article citation (SOURCE) query (base URL + metadata string) link resolver/ knowledge base target (cited) article publisher website database print collections gateways publisher/provider holdings data repository

24. A book chapter citation in a database: Cognitive psychology, new test design, and new test theory: An introduction. Snow, Richard E.; Lohman, David F.; In: Test theory for a new generation of tests . Frederiksen, Norman; Mislevy, Robert J.; Bejar, Isaac I.; Hillsdale, NJ, England: Lawrence Erlbaum Associates, Inc, 1993. pp. 1-17. [Chapter] Incorrect Implementation – an example

25. Incorrect Implementation – an example No self-respecting OpenURL talk… http://ry6af4uu9w.search.serialssolutions.com/?genre=bookitem&isbn=0805805931&issn=& atitle =Test+theory+for+a+new+generation+of+tests.&volume=&issue=&date=19930101& title =Cognitive+psychology%2c+new+test+design%2c+and+new+test+theory%3a+An+introduction.&aulast=Snow%2c+Richar&spage=1&pages=1-17&sid=XXXX:PsycINFO&pid=%3Cui%3E1992-98936-001%3C/ui%3E&%3Cdate%3E19930101%3C/date%3E&%3Cdb%3EPsycINFO%3C/db%3E

26. Incorrect Implementations – the Example made worse http://ry6af4uu9w.search.serialssolutions.com/?genre= article &isbn=0805805931&issn=& atitle =Test+theory+for+a+new+generation+of+tests.&volume=&issue=&date=19930101& title =Cognitive+psychology%2c+new+test+design%2c+and+new+test+theory%3a+An+introduction.&aulast=Snow%2c+Richar&spage=1&pages=1-17&sid=XXXX:PsycINFO&pid=%3Cui%3E1992-98936-001%3C/ui%3E&%3Cdate%3E19930101%3C/date%3E&%3Cdb%3EPsycINFO%3C/db%3E Genre: Openurl 0.1 vs 1.0

27. Study at Claremont 5 Results in each of 5 genres from each of 5 databases Journal Articles, Books, Chapters, Newspaper arts, [Dissertations] Measure success rate, cause of each failure Preliminary analysis shows: Journal articles have significantly lower failure rate Source URL formation as major cause of failure Relative consistency within a database/genre combination Incorrect Implementations – the impact

28. Solving the Problems: Lack of Knowledge Some content providers simply aren’t aware of what OpenURL does and why it benefits them Education & advocacy Follow recommendations of Culling/SIS report; provide useful information to those content providers How to implement correctly Offer contacts for those needing assistance The remainder may not recognize the value of and their role in improving OpenURL effectiveness

29. Solving the Problem: Lack of knowledge Help content providers determine what is working, and what isn’t Cornell project to focus on source OpenURLs Identify correct and incorrect implementations Give opportunity for vendors to grade selves Offer more & better examples of why open OpenURL matters Quiet challenge (ok, at least out loud) to ER community: produce and distribute studies of the effect of OpenUrl (& poor implementations) on usage There is one underway at Claremont, we need many more So many possibilities, so little time (shortsighted? –we’re treating the symptoms, not curing the disease)

30. Summary: KBART Deliverables Create a report that provides general guidance on problematic issues Data problems Incorrect implementation Limited knowledge Offer best practices guidelines for how to effectively transfer accurate data among parties Provide better understanding of supply chain

31. Challenges Figuring out how to deal with data accuracy questions Ensuring uptake among smaller or less-committed content providers Providing ongoing support for new participants

32. Thanks! http://www.uksg.org/kbart http://www.niso.org/workrooms/kbart Peter McCracken (NISO co-chair) [email_address] Co-founder & Director for Research, Serials Solutions Charlie Rapple (UKSG co-chair) [email_address] Head of Marketing Development, TBI Communications Jason Price (Working group member) [email_address] Head of Collections @ Claremont; SCELC ejournal package analyst