UKSG conference - Impact of library discovery technologies on usage of e-resources - Valerie Spezi
Upcoming SlideShare
Loading in...5
×
 

UKSG conference - Impact of library discovery technologies on usage of e-resources - Valerie Spezi

on

  • 108 views

This is a presentation given by Valerie Spezi (LISU, Loughborough University, UK) at the plenary session of the 2014 UKSG annual conference, which took place in Harrogate, 14-16 April 2014. ...

This is a presentation given by Valerie Spezi (LISU, Loughborough University, UK) at the plenary session of the 2014 UKSG annual conference, which took place in Harrogate, 14-16 April 2014.
It reports on the findings and recommendations of a 6-month project funded by UKSG, in collaboration with Jisc, which looked into the impact of library discovery technologies on usage of e-resources. A video of the presentation is also available on YouTube (https://www.youtube.com/watch?v=ZZS_wm0bjrE )
This presentation follows on an earlier presentation (also available on Slideshare) which reported on the early findings of the project back in November 2013.

Statistics

Views

Total Views
108
Views on SlideShare
108
Embed Views
0

Actions

Likes
0
Downloads
1
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Good morning everyone. I am Valérie Spezi from LISU, at Loughborough University. Today I am going to present a project that we’ve completed just before xmas, and which looked into the impact of library discovery technologies such as Web-scale resource discovery services.
  • The project was commissioned by UKSG, in collaboration with Jisc – it reflects the great interest we observe today in resource discovery technologies.This study is a relativelysmall-scalestudy and it is focused on the UK.There are very few usage data analyses available and this study aims to fill the gap, to some extent.There are two other -BIGGER - studies which were conducted at the same time as ours:One is the JSTOR study [JSTOR: Can Users Find Your Content Anymore?]The other one is the Denver study [Discovery or Displacement?: A Large Scale Longitudinal Study of the Effect of Discovery Systems on Online Journal Usage] Work on this project started in July 2013, with a reporting date in the Autumn, so quite a tight deadline. The report was released in December and is now available through the UKSG website.
  • As I said before the focus of this study is primarily on ResourceDiscovery Services (R D S).The research had four objectives.The first objective was to provide an evaluation of the impact of RDSs on the usage of academic resources. This included e-journals, e-books and databases.The second objective was to provide evidence to determine if there is a case for libraries to investin library discovery technologiesand a case for publisher and other stakeholders in the information supply chain to engage with library discovery technologiesThe third objective was to provide recommendations for stakeholdersto best support the discovery of academic resourcesAnd finally the fourth objective was to identify additional research, data and initiatives needed to support the discovery of academic resources.
  • I am going to present very briefly the methodology for this project because we don’t have much time today.We first conducted a surveyof UK higher education libraries – the objective of this questionnaire was to determine the current RDS landscape in the UK.The online survey was distributed UK HE library directors - 62 institutions completed the survey We then conducted a series of case studies with libraries and publishers. Participating publishers and libraries were required to provide some monthly aggregated usage data for the usage analysis. The data we requested were COUNTER reports for journals, e-books and databases – JR1, BR2 and DB1 or close equivalentWe also talked to each library and publisher to collect their views and perceptions on the impact of RDSs on the usage of academic resources.We had a good mix of publishers and content providers, ranging from small and specialist publisher to very big publishers. We then conducted interviews with stakeholders in the information supply chain. The objective was to understand the big picture.
  • So, the findings. Here is how I am going to present the findings.Firstly, I am going to present the UK RDS landscape, which was based on the survey findings.I am then going to present the findings for libraries, based both on the survey and the case studies. Then, I am going to present the findings for publishers and content aggregators, based on the case studies we’ve conducted.I’m not going to present the details of the usage analysis, but just the trends. If you are interested in seeing the detailed graphs, I invite you to look in the report.
  • So, the current RDS landscapein the UK.RDS is becoming a major element of the academic library landscape, with 77% of survey respondents having already implemented an RDS at their institution, and a further 11% in the process of doing so, at the time of the survey. What the graph shows as well is that early adopters of RDSsimplemented it back in 2007-08.The survey findings suggest that RDS implementation in HE libraries had probably reached its peak in the last 12 months. And you can see on the graph that there were as many implementationsdone in the last academic year (2012-13) as in the last three years (2009-2012)CLICKThe UK higher education marketis mainly covered by three products: Ebsco’s EDS, Ex-Libris’ Primo and Serials Solutions’ Summon. They are the products most frequently used, together accounting for over 76% of systems in use.
  • The participating libraries reported very high levels of satisfaction with their RDS. Their satisfaction with RDSs was primarilydown to the one stop shop experiencethey were now able to offer. Now, users could search across almost all collections from a single entry point linked to full text. Libraries want to offer their users a Google-like experience, because this is what users are familiar with. And this seems to be working! – the libraries we spoke to indicated that user feedbackis generally very positive. Undergraduate students are seen as the primary users and beneficiaries of library discovery technologies. They are the target population. It was recognised though that RDSs were not necessarily the best tool for postgraduate students and researchers.CLICK - From the usage study, we can say that:There is no straightforward answer but it appears that overall RDSs may influence positively content usage, most visibly for e-books. The impact varies by resource, and across libraries.Libraries’ perceptions of increased usagefollowing RDS implementation are actually borne out by the usage data.E-book usage appears to have accelerated in the case study libraries following RDS implementation, while e-journal usage has increased just a little or decreased in some instances.Database results were inconclusive.CLICK - Isolating the sole impact of RDS is a challenge; there is a lot of noise in the results because it is amulti-dimensional environment.Many other factors can affect usage, including:the link resolver, the options libraries select within the RDS, an increase in the volume of subscriptions, a growing appetite for electronic content, particularly e-books, the promotion of electronic content by libraries and academics, via reading lists for instance etc.In our usage study, we have tried to control for content growthby providing an analysis based on constant titlesBut,more data is needed for a meaningful analysis - 2-year post-implementation data is just not enough to pick up a trend and isolate other variables influencing usage.
  • So, I’ve talked about the high levels of satisfaction but some challenges associated with RDSs were also reported:Libraries are unable to see how well their resources match the RDS index, although they believe the match to be 50% or more. So, there is definitely a lack of clarity in coverage from RDS suppliersThere are gaps in the coverage of some collections, particularly Law. There were some concerns about vendor rivalries, and the lack of co-operation between some content owners or providers and some RDS suppliers. This was seen as unhelpful by libraries. Some libraries also expressed concerns that the content covered in the RDS was not always provided on a neutral basis. This is probably because libraries did not always understand how the relevancy ranking works.Interoperability between library systems was also a concern. It was not always easy to maintain different systems from different vendors as they were not always talking to each other adequately. Libraries were happy to move to a single suite of products from a same vendor, risking to be tied up into one particular ecosystem.
  • Now looking at publishers and content providers, what we can say is that visibility of content is a key motivation for publishers and content providers to engagewith RDS. In terms of content usage it is a very mixed picture for publishers and content providers. Publishers and content providers have no clear evidenceof the impact of RDS on the usage of their content. They don’t always know whether their usage has been affected by RDS – the traffic from RDS seems to remain very low and publishers and content providers cannot always tell from their analytics whether traffic to their site is mediated via an RDS.And indeed the analysis we did on publisher usage data shows that variations are actually on a very small scale.Some publishers are engaging and contributing without any reservations while others do have some reservations – there was an indication that small publishers may benefit more from RDSs than bigger publishers.Publishers are engaging with RDSs but they also have some reservations and concerns. CLICK - Those concerns are reported here:Some publishers had to work on their metadata to optimise discoverability of their content in the RDS.Some publishers insisted on the issue of the dilution of the publisher’s brand within the RDSPublishers have concerns that they are not being necessarily well served by RDS providers who are primarily concerned with their library customers.Some publishers were also concerned about the lack of clarity on how their data is used by the RDS. This touches upon how the relevancy ranking works within each RDS.
  • From what I have presented we’ve come up with 3 simple, high level conclusions:There is a lot of data out there but it is imperfect. There is a lot of noise in the data, which makes it difficult to isolate the sole effect of RDS on content usage.RDS are a fantastic resource discovery tool for library end-users but it is still early days and a lot more could be done to take full advantage of this technology.Engagement and collaboration between the different groups involved is key to success.
  • The recommendations I am going to present now aim to improve content discovery by end-users in an RDS environment. Based on the findings I’ve just presented, we’ve made recommendations for 4 groups:libraries, RDS suppliers, publishers and content providers and other stakeholders in the information supply chainThose recommendations revolve around 2 main ideas which are:transparency and clarity, and working relationshipsCLICK - I don’t think any of the recommendations I am going to present are particularly controversial but some of you may see things differently.
  • We believe the library community should work with bodies such as SCONUL, RLUK, UKSG, Jiscto consider ways of:Empowering libraries to drive service development within the RDS communityStrengthening the library voice to make sure that RDS suppliers and content providers are providing end-users with the best information discovery experience, and that issues such as transparency, neutrality and relevancy ranking are dealt with in a way that is acceptable to the library communityWe also believe libraries should work closely with RDS suppliers and content providers to gain a better mutual understanding of how minor changes in the RDS settings may affect directly usage of certain resources.CLICKNow looking at RDS suppliers, we recommend an open communication with libraries and content providers on how the RDS system works and how publisher content is surfaced (RDS settings, relevancy ranking).We believe that having in place some user-testing for publisherswould be helpful.Providing clearer information on what is indexed by the RDSANDsupporting the development of working relationships between competing RDS vendors with close links with the publishing sector would be beneficial to the library sector and ultimately the end-users.There are more recommendations for RDS suppliers and I invite you to have a look at the report.
  • For publishers, we recommend publishers and content providers engage and work more closely with both libraries and RDS suppliers to make sure content is surfaced properly. This would also be an opportunity for publishers and content providers to open up and build up communication channels with RDS suppliers.CLICKWe recommend UKSG and Jisc to follow closely the developments led by COUNTER 4. This may help with the collection and reliability of database usage statistics which proved challenging in our study.We recommend the inclusion of usage data from RDS suppliers and link resolvers in initiatives such as JUSP, but also the development of a COUNTER code of practice for RDS usage data.International bodies such as NISO, ODI and COUNTER should consider working together to establish industry standards and encourage RDS suppliers to take notice of those developments.Finally, we believe there is a need for further detailed usage research.
  • Well, this is a lot to take in and for a more detailed reading of the findings I invite you to look at the report, which provides detailed information to support the recommendations that I’ve just presented.Thank you!

UKSG conference - Impact of library discovery technologies on usage of e-resources - Valerie Spezi UKSG conference - Impact of library discovery technologies on usage of e-resources - Valerie Spezi Presentation Transcript

  • Evaluation of the impact of library discovery technologies on usage of academic content Valérie Spezi, LISU (Loughborough University, UK)
  • Why this study? • Commissioned by UKSG/Jisc in July 2013 Lots of interest in library discovery technologies Questions about whether libraries, publishers and other stakeholders should be engaging with those technologies • Small-scale study  A UK perspective  No previous usage data analyses at the time - fills in the gap  Complements the 2 other studies that are currently taking place • Report available on UKSG website (Dec 2013)
  • Objectives of the research  Evaluation of the impact of library discovery technologies on usage of academic resources  Provide evidence to determine if there is a case for  Investment in library discovery technologies by libraries  Engagement with library discovery technologies by publishers and other stakeholders in the information supply chain  Provide recommendations for stakeholders to best support the discovery of academic resources  Identify additional research, data, discussion and initiatives that will support the findings of the study
  • Methodology Phase 1: survey of UK HE libraries  Objective: determine the current RDS landscape  Online questionnaire to UK HE library directors – 62 respondents Phase 2: case studies of libraries and publishers  Objective: collect usage data + views and perceptions on the impact of library discovery technologies  8 publishers and content providers; 6 case study libraries; Data received from 6 libraries & 4 content providers - COUNTER JR1, BR2 and DB1 or close equivalent (2 years pre and post-RDS implementation) Phase 3: interviews with stakeholders  Objective: obtain a bigger picture on the perceived impact of library discovery technologies and an insight of where the sector is going
  • Findings 1. UK RDS landscape (survey findings) 2. Libraries – usage trends & experiences 3. Publishers and content aggregators – usage trends & perceptions
  • Journals – mixed picture, possibly some positive influence to varied extent E-books – positive correlation Database results were inconclusive LIBRARIES –usage data and experiences • Improved user experience through a single search interface linked to full-text – high level of satisfaction • One stop shop experience for users • Better use of subscriptions – no silos • Possibly a positive influence of RDS on content usage, most visibly for e-books Multi-dimensional environment ***** Difficult to isolate the sole effect of RDS ***** Multitude of other factors at play?
  • LIBRARIES –usage data and experiences Perceived challenges *********************** • Lack of clarity in coverage - RDS coverage of subscribed resources ‘believed’ to be 50% or more – gaps in some disciplines • Lack of cooperation between some vendors is a concern – not helpful according to libraries • Interoperability between library systems – ‘ecosystem’ • No routine analysis of the RDS usage data (yet) • RDS searching aimed at undergrads? • Starting point? • Can researchers benefit from RDS too?
  • PUBLISHERS –usage data and experiences Key motivation - improving discoverability & visibility of content • Publishers have no clear evidence of their usage is being affected by RDS • difficult to isolate traffic mediated by RDS • Still low traffic compared to search engines • Our usage study shows a very mixed picture for publishers • Some publishers may benefit more from RDS than bigger publishers Perceived challenges: • Metadata –compatibility and optimisation for improved discoverability • Dilution of the publisher’s brand within the RDS • Lack of feedback/communication from RDS suppliers • Lack of clarity and understanding of how data are used  Relevancy ranking Engagement - can publishers afford to wait and see where this is going?
  • Conclusions • There is a lot of data out there but it is imperfect • Fantastic tool for library end-users but more work needs to be done to take full advantage of RDS technology • Collaboration is key to success
  • What next? Recommendations
  • Recommendations Libraries  Library community working closely with bodies such as SCONUL, RLUK, UKSG, Jisc  Consider issue of interoperability between products from different vendors vs. moving into a particular vendor’s ecosystem  Engage in cross-sectorial talks to understand better how minor changes in the RDS settings can affect usage of certain resources RDS suppliers  Working towards an open communication with libraries and content owners/providers  Consider user-testing for publishers and content providers  Provide clearer information about what is indexed by the RDS  Support the development of working relationships between competing suppliers on the issue of disclosure and exchange of data for the benefit of end-users … etc.
  • Recommendations Publishers  Engage and work closely with libraries and RDS suppliers to optimise content discoverability  Voice the need for more communication and feedback from RDS suppliers Other stakeholders  Monitor developments led by COUNTER 4, particularly in the area of database usage  Inclusion of usage data from RDS suppliers and link resolvers in initiatives such as JUSP or KB+  Development of a COUNTER code of practice for RDS usage data  COUNTER, NISO, ODI to work together and establish industry standards  Support new research
  • THANK YOU! Report available on the UKSG website: http://www.uksg.org/researchstudy Email address: v.c.l.spezi@lboro.ac.uk Get in touch with us!