UKSG 2018 Breakout - What is all this fuss about? Is wrong metadata …

No notes for slide

Today, we want to do another step towards the understanding of the issues caused by bad quality metadata. Today, we want to bring the perspective of the end user. How bad metadata affect them despite the efforts of libraries to propose them the best service investing in content and sometimes in last generation discovery layers. We will highlight the main challenges presented by metadata, why this matters and the impact inadequate metadata is having, to finish with some recommendations for stakeholders.

The theme about why metadata is important and the issues around the quality of metadata are not new. All the actors of the supply chain have a part to play and are affected. Here I will summaries where the different actors are in order to give the context of our presentation.
Lets start with Libraries. Libraries are looking for end to end workflows from acquisition to discovery of electronic resources in order to automate tasks and release staff time. They want to ensure user satisfaction and value for money.
Purchasing consortia, they expect that electronic resources will be provided with “adequate” metadata by the content providers when purchased. However, experience shows that in many cases this is not the case.
Knowledge Base Plus, we support libraries to workaround failures in supply and accessibility. We are suppliers of curated metadata. Our work have help to proof that the supply of good metadata is possible. We have also helped libraries to distinguish between the problems on the supply from content providers and the delivery from system vendors.
Content providers: they provide metadata of variable quality and they do not always adhere to commonly accepted standards/recommendations.
System Vendors: the increasing complexity of library information systems has led to new challenges in the management of metadata. Although these systems promise to reduce the effort in managing discovery metadata, current experience shows a more mixed picture.
KBART, there are recommendations and support to help content providers to deliver ingestible and enriched metadata.

What we know up to now:
We know that Metadata issues are not new.
We know that there are recommendations on how to do better for example KBART that are not always follow.
We know where all the actors of the supply chain are, their positive contribution, and for some of them their struggle.
Based on our researches and our day to day experience, we have identified that these are the current challenges, and we will present them to you along side with why this matters and the impact they are having.
And now it is time to me to pass the word to Amy…

Metadata for print resources
Metadata for electronic resources

The metadata is the same – we still need authors, titles and dates of publication, for example, subject headings and summaries. How we store the metadata has changed, and this has allowed us to do new things with it – sharing it – with each other, with aggregators, with commercial companies who can then sell us products for jazzing up our catalogues and so on.

But there are 2 big changes that Magaly and I end up coming back to…

Scale - storing our metadata electronically has exponentially increased the size and scope of our collections so that we can no longer be the sort of librarian who intimately knows our collection and dutifully connects each user with the resource we think they need…now we have to be skilled navigators of huge sets of metadata to help users find what they want…and there are additional metadata requirements to accommodate the form and method of access of new resources…new formats, licensing and collection and article level metadata needs to be created, shared and maintained in addition to the print format and content metadata

Data - submitted to ongoing technological processing – we may have know the above for a while but we are only just beginning to realize that unlike metadata that sits on a card in a box, our libraries rely on metadata that is dynamic – it is constantly being put through processes that ensure its maintenance, occasionally its enhancement, and sometimes its successful migration or export to a new system.

And it is rarely librarians who are responsible for these processes…we can and do enhance and move our metadata but the trend is towards linking to metadata stored and looked after centrally so that the updates to the vast resources can be made quickly, and ideally once so our users are accessing correct information about our resources at any given time.

So, publishers and increasingly LMS’ are having to maintain and enhance our metadata…and as an industry this has happened largely independently of each other and quickly…which leads to an often perplexing experience as both librarian in and user of our services…so now we have new services like KB+ to help us but if like us you are a small team and are tired of yet new places to go to fix part but not all of your metadata and access problems, we can need convincing…

But rather than talk theoretically about all of this I think it is much more useful to describe some examples so that we have concrete reasons for making an effort to get our metadata right – and while I know your institution and publishing metadata is obviously perfect (!) I’m sure many of you will have come across similar poor and sometimes even mind bending discoverability cases…

Us @ Aber
how many people here catalogue or deal with bibliographic metadata every day?
But, how many of you are affected if users can not access your resources?
All of us, therefore, are affected by bibliographic metadata and although you may never need to know about RDA or FRBR or how to catalogue a bound-with, you do need to know that at the heart of any library services is some data about your resources and that data has to be…of good quality at its creation and protected through out its life cycle if we want our users to be able to complete the FISO functions of finding, identifying, selecting and obtaining our resources and if we want to use our metadata for accurate print and electronic collection management.

I’m starting with a print example to ease us in…

We all know about legacy metadata issues of old practices and migration loss leaving many of our records without enough metadata to facilitate discovery and identification or to match successfully in external systems like COPAC or Worldcat but we are sometimes adding to these issues with poorly described new material.

Here we can see that some contemporary publications are slipping through with much worse metadata than that we have traditionally had… [click through 1-6 from Cambridge edition, 7-8 are from the more recent work…]

At end

With the more recent publication you have nowhere to go if confused by the content (e.g. contemporary intro), no editor to cite in your essay or article, no publisher, edition or place of publication so that others can look up the information you have referenced. No sense of who edited this version and in what era…meaning is affected because it has been built, in this case, over centuries by different scholars, translators, and editors and their interpretations.

The fact that it has become much faster and easier to publish than it used to be it shouldn’t undermine the importance of what publishers can provide that is unique & useful – context and accountability not just content. And while I’m sure no publisher at UKSG would produce such a poorly described print volume, there are a lot of ebooks out there that mirror this cheap production of metadata…

- electronic packages are vast and obviously offer incredible value for money yet the individual resource within them can easily become inaccessible through a lack of good metadata…

We were concerned some Ex Libris CZ collections we are linking to in our Library Management System Alma to provide our users with access to our e-resources in our discovery layer Primo were lacking in useful metadata…and this ebook was one example…it had a title and a publisher, and was wrongly described as a journal…and that was it.

This is how it looks in Primo to our user and you can see that I ‘found’ it by searching the exact title…not a lot else would ‘find’ this resource and if you did come across it and wanted to be sure it was what you wanted, what could you use to ‘identify’ it? And without being sure who wrote it, what edition it was, or even if it was a book or a journal, could you ‘select’ it from other similar titles? Would you bother ‘obtaining’ it? If you wanted to do an ebook overlap test with a consortia university would this book be able to match with another record for the same content? Or would we have to assume it was a different resource because the metadata is so poor?

And yet, searching this exact title search also brought back this resource in our results list…author, edition info, physical description, contents and subject headings.

Now this resource itself may well have adequate or even good metadata but if that is not shared with libraries and other systems for finding resources, or if it is shared and then overwritten by subsequent system metadata processes, then the content becomes worthless.

This book, Aristotle’s Poetics, was added by a lecturer to a reading list for Semester 1 2017/8

Reading list link checks showed that it wasn’t available via either Alma or Primo.

We can find it on the platform Proquest’s Ebook Lib Central – where this is what it looks like:

Although this screenshot shows subject headings and a publisher and publication date there is no one responsible for editing – which would help users in Primo identify the work versus criticism, and particular editions.

But it doesn’t appear in Alma/Primo anyway – there are two similar titles in same collection which are about Aristotle’s poetics but are not the text itself or the item linked to originally:

As of writing this presentation well into semester 2, it still isn’t available although we have had a lot of discussion about the other 2 resources which are available here – both EL and ProQuest keep saying we do have access because they find one of these two texts…

Like the print, we have become aware of this item through our checks but unlike the print, we can’t fix it. And worse than that, we do not know the scale of how often this happens because ProQuest and Ex Libris can’t say with any certainty what we have access to at any given point in a cycle of removals and updates and we largely have to rely on users telling us that something isn’t working.

So this is an example of something that doesn’t have great metadata for discovery – then doesn’t appear at all, and its lack of metadata then makes solving the access problem harder because it is difficult to “identify and select” in the FISO terminology.

This is another example of a CZ record that we link to, this time with plenty of metadata, just not quite the right metadata…

This is our record for the Journal of volcanology and seismology from our NESLi2 SpringerLink Journals Option 1

Academic got in touch because it was hard to ‘find’ and ‘identify’ when the search result for Journal of volcanology and seismology comes back with something in Russian.
The CZ record for it that we link to does suggest it is an English language version of a Russian text, starting in 1984 published in the US and london., and its subject headings include a Soviet emphasis. There is another issn in the 776 field for alternative format information so this may have been for a different language version perhaps….
I couldn’t tell from looking at the resource quickly if the Russian emphasis was accurate or not so looked the journal up in Suncat
2 records – the first looked just like ours (very Russian) but with a different issn
The second had our issn but looked like it was describing our resource as we would have expected – i.e. English, very little Russian, no soviet headings…and it starts in 2007 and is published only in the us by Springer.
Resource looks like this – the issns refer to the print and e versions so at this point we’re thinking that it is part of a known issue where EL have used print records for online resources but only changing the issn
Double checked the publisher and although there is a Russian connection…we prefer the online second suncat record because it has the right issn but is also in English as our user expects and starts in 2007.
Explained this to EL a few months ago and have just been told that we are welcome to edit the record in the CZ. We may well do this but this would be a global change affecting every single Alma customer linking to the resource, and our changes could also be overwritten by other processes that EL put the metadata through.

At least with the Russian journal title issue we could unravel what may have happened…sometimes you get a real oddity that never gets resolved…

This title Career information and resources for Austria 2011 (Proquest Ebook Central – Academic Complete UKI edition) collection was noticed because it was deleted and added in Academic Complete UKI updates and then we couldn’t find it in Alma or in Primo - and we couldn’t find it in the LibCentral either because it turns out the title in the bib record and therefore discovery layer is Austria Career Guide.

We should have access to it in our collection as Austria Career Guide, but so far, do not.

Since the problem was identified we have been given access via our EBSCO Business Source Complete collection (which really confused us!) again with the title Austria Career Guide although the resource itself is called Career information and resources for Austria. And record says it is in French. The record does say it is a book but the user is taken to a journal search page and has to search ‘within this publication’ to find a contents list of the book.

It might be findable, just, but working out what it is isn’t easy and obtaining it is enough to put users off.

So, we were still trying to find out what had happened to our UKI collection version…

My colleague Iona Hopkins found that there had been a 2004 edition in the Academic Complete UKI at one point…[show screenshot]

This has also disappeared but, and I think we were randomly trying a few of the other titles that had been removed and added, but we found that if you search for “Career information and resources china” in Alma and in Primo…you get three results, one 2004, one 2006 and one for a Hong Kong edition.

And if you click on the 2004 edition it takes you to… the 2011 edition of Career Information and Resources for Austria, with the title Austria Career Guide on the platform. And as you can see we now are getting the 2011 Austria volume and there is some metadata – a publication date, and an author.

So, from an item being ‘missing’ we eventually work out that: 1 title in 2 collections, both collections using the wrong title – one providing it through the wrong year and country record, and one as a journal in French.

Getting commercial companies – in this case 2 quite well connected companies… EL and ProQuest (nevermind Ebsco)– to talk to each other and to us us in a three way conversation is incredibly difficult and one specific example can be as hard to talk about as a package of a hundred thousand ebooks – I understand that as a publisher/platform/system or aggregator one resources amongst hundreds of thousands just isn’t that important… As customers, however, we represent users who experience specific resources not ‘value for money packages’ and we represent our collection management colleagues who need accurate data about our resources in order to take informed decisions. As librarians we have to find a way to communicate this to our providers.

Which is a good moment to pass you back to Magaly…

One of the most demanding tasks for librarians is to understand the value for money of their collections. In a context of tight budgets, recurrent cuts, librarians need to be able to answer questions about the real extend of their collections - what they have access to, the usage and what is the best acquisition method to fulfil the end users requests. The way how the electronic resources are sourced doesn’t make librarians life easy.

One of the preferred methods of content providers to present their products are bundles. Agreements negotiated by purchasing consortia include collections of journals and other electronic resources. There is the believe that there is value for money when buying a collection instead of the individual subscriptions.
Every renewal season, content providers advertise their proposal with phrases that imply that more content is being offer (with new titles for example, or because the terms and conditions allow more access).
Customers (in this case librarians) understand that they are gaining value for money. All this is very good, it is fair to show a good product and that the terms and conditions are advantageous. However, we know that the bundles of items have a weak point : metadata. The reality is that, in many cases, the bundles are not delivered with the adequate metadata that will allow full access and discoverability by end user and, more importantly for our point, to allow librarians to compare usage and value.
It is known, that the resources contained within a bundle may vary, either at the end of a fixed-term agreement, or even during the lifetime of agreement. When metadata describing the content of the bundle is not available, it is impossible to even understand what’s changed. This can make comparisons of value over time impossible. Given the already-tense situation regarding the cost of publications, this has the potential to lead to cancellations.

Let’s see some examples. I have based these example in real queries received by the KB+ team.
A librarian had a subscription to a bundle using a consortia agreement with the content provider for access to an specific collection. The agreement specified that new titles should be included.
KB+ team has added these titles in the correspondent title list in November 2017 but up to last week the end users of the library who reported the issues continue without finding these new titles (here a couple of examples). This will affect usage, without accurate usage it is very difficult to calculate cost per use or having a complete understanding of the entitlements.
This is not an isolated case, we had received more similar queries affecting other collections.

I have another example of missing titles. It is a digitalised collection. In this case, a librarian said: “The collection contains over 450 titles, however the current target available in our link resolver only contains 45”
The access to this collection is free to libraries and end users but this doesn’t mean it is cost free. The success of this digitalisation projects is on their usage. Again, in a time when money is tight, funders are more strict to allocate funding digitalisation projects and they want reassurance that the chosen collections are useful. If only 10% of the collection is discoverable, it doesn’t look right, does it?
I have the hint that records with not identifiers haven’t been processed (this is not very good for an archive collection). Also for the titles that were displayed, not always were identified as part of the free collection.
This could be a problem of delivery of metadata or a problem on the processing of the metadata by the system vendors.

The following example is similar but I analyse it differently. In this case the titles seemed to have been deleted from the link resolver’s target.
Again in this occasion, a librarian said: …..
Her institution had a subscription to a collection through a multi year agreement and she noticed that some journals were deleted and she was asking for our support clarify the entitlements.
I realised that these journals were labelled as the new journals proposed by the content provider. I decided to see what was the situation for other subscribers of the same agreement. Were these journals discoverable at all?
So I choose a group of subscribers (10), to the same collection, and I started clicking…For one of them, it was impossible for me to search their holdings so they do not appear on the results.

Well here the results. I have scored them based on the percentage of titles that were discoverable. Libraries use different system vendors that are connected to their own knowledge bases and link resolvers, so as expected we have different results. Some of the institutions used the same vendor (Vendor B) (these are institutions 2,3,4,6, and 9) but they have different results. The results depend on the targets they have activated.
I have also added for information the cost of the individual institutional subscriptions, the cost of pay per view. The libraries belong to different sizes institutions (identified by the Jisc bands) as an indicators of resources available.
Only one institution had 100% score. The lowest score is 26 % from “institution 4” where we found only 5 journals out of 19. For “institution 6”, it wasn’t clear where their access were coming from so I was uncertain to score them the same way.
We do not know how much the results are influenced positively due to the librarians checking and correcting. Well, I know a bit, I know that Institution 1 expends time checking what their link resolver. They nearly got it. Only one missing. If they have improved their situation manually, it would indicate a hidden cost for libraries when buying bundles.
This is just an example and the problem is scalable. How many more titles are affected and nobody know about it? How much institutions are paying for no discoverable items? How many disappointed end user are?, For end users, simply these titles are not there.

At the end of the day all we do and we don’t do is a question of priorities.

This part will refer to the first slide we visit, the one about the story so far and where the different actors of the supply chain are.
Last year, the KBART Standing Committee run a survey to identify the obstacles of adoption of the KBART recommendations within content providers. When we asked them “why they haven't used the KBART Phase 2” they replied: no time/other priorities and, no demand from customer base
The part that stroke me the most was to learnt about the lack of customer requests.
Our assumption was that librarians needed help to argue the benefits of good metadata in an effective way. To support this area we worked with Sero Consulting to produce a paper addressed to librarians. The aim was to elaborated a short document with few bullet points describing arguments that librarians could use during their conversations with content providers to make the customer requests easier. To prepare this paper there were interviews to electronic resources librarians, senior librarians and other library staff.
In this case, the results were again surprising:
Libraries can often work around the failures in supply and accessibility through their discovery layers, either manually managing lists or with the help of services such as Knowledge Base Plus but libraries are missing opportunities to improve the situation at the source during purchasing and negotiation. Internal priorities are contradictory depending the role of the librarian.
In addition, no all librarians are confident on their knowledge of the metadata standards and recommendations that could help on the interaction with suppliers.
Finally System Vendors, we have seen in our examples that the performance of the systems provided by the vendors is a mixed picture. For all the examples we have mentioned the librarians contacted the relevant vendor and the KB+ team did it too. The responses were slow and not all the problems were solved. If corrections were carried out, they happened case by case and not as a revision of their data processing. They operate globally so their focus is not just on the UK market. As a result, when metadata issues are reported by an individual institution in the UK, system vendors may not perceive this to be a priority issue.

We are arriving to the end of our presentation. We want to finish with a positive note and some ideas. As we have seen, the impact of unsuitable metadata is bad. It can’t be never good to have something incomplete or wrong or too late. We thing that to address the situation several actions could be taken

Librarians should agreed their needs regarding metadata and use all their interaction points with suppliers (content providers and system vendors) to pass the message across. From Procurement process to conversations with suppliers reps.
This can have good results, I am aware of at least one case (regarding eBooks) where a major vendor proved unable or unwilling to comply with what the library consortium viewed as basic standards in metadata and was therefore excluded from the procurement.
Services as Knowledge Base Plus has helped to bring more information to the table and break barriers as the believe that accurate and timely metadata is not possible. It is! And this has been possible thanks to a committed team and the support of the community of users. It is a win-win situation, librarians will save time and at the same time it will increase the quality of the metadata they all use.
Libraries have the power as customers to influence vendors. Libraries can develop the dialogue around the provision and processing of metadata, both pre- and post-procurement. System vendors need clarity about pain points and to plan changes in their development processes and resourcing. The library community can use existing user groups and can move things forward making sure that current customer’s experiences reach prospective customers.
Content providers: Metadata is part of the product and it will influence the view of the customers. Reduction of usage could lead to cancellations.
A working metadata supply system can only be achieved if stakeholders step in and see themselves as owners of the problem. Clearly, each actor has somewhat different priorities, and makes a different balance of costs and benefits, but high-quality metadata benefits all stakeholders in the supply and use of library content.

UKSG 2018 Breakout - What is all this fuss about? Is wrong metadata really bad for libraries and their end users? - Bascones and Staniforth

UKSG: connecting the knowledge community

UKSG 2018 Breakout - What is all this fuss about? Is wrong metadata really bad for libraries and their end users? - Bascones and Staniforth