Charleston Conference 2017 - What's Past is Still Messing With Our Workflows
•Download as PPTX, PDF•
0 likes•117 views
Scholars Portal has been aggregating locally loaded ebooks for Ontario universities on an eBrary-backed platform since 2009— eons ago in the world of library technology! Over the last year, Scholars Portal Books has received a rewrite from the ground up, and this time the focus is on building a platform that anticipates the future of ebook access and usage. After extensive community consultation, we knew we wanted a front-end that was accessible beyond legal requirements, a back-end where metadata was harmonized using the BITS standard, and an admin module where local eresource staff could generate usage stats, MARCs, and KBART title lists on the fly. Finally, we needed all of it set up to move through a Trustworthy Digital Repository. No surprise, there have been many challenges along the way, and none of them are unique to consortia: How do we handle corrections to old PDFs? What do we do with six ISBNs? More broadly, how do we support local scholarship at scale, and how can we make space for the open educational resources increasingly being integrated into higher education? This session will look at the complex ebook landscape through a consortial lense — from licensing and entitlements management, to wrangling a dozen XML schemas and implementing ever-changing DRM restrictions, towards the ultimate goal of preserving Ontario university’s books content for the long term
Recommended
More Related Content
Similar to Charleston Conference 2017 - What's Past is Still Messing With Our Workflows
Similar to Charleston Conference 2017 - What's Past is Still Messing With Our Workflows (20)
More from Jacqueline Whyte Appleby
Recently uploaded
Recently uploaded (20)
Charleston Conference 2017 - What's Past is Still Messing With Our Workflows
- 1. What’s past is… still messing with our workflows Jacqueline Whyte Appleby Scholars Portal Ontario Council of University Libraries
- 2. What is the future of ebooks management?
- 3. Context - Ontario ● 21 universities ● All are public, all have a research mandate. ● Range in size from 1,300 to 83,000 students ● Their libraries work together through OCUL, the Ontario Council of University Libraries.
- 4. Context - Scholars Portal ● Scholars Portal builds & maintains digital services for Ontario university libraries. ● Mix of content & member services ● Locally loading journals since 2001, books since 2009 ● Journals platform has been TDR certified since 2013, Books is next
- 5. Context - Books platform ● Hosts about 250,000 commercial texts, 400,000+ OA or public domain texts ● PDF & XML-based texts ● Platform released in 2009, software it’s built on sunsetted in 2011 ● Platform redevelopment 2016-2018
- 7. 1. Files will be sent in standard packages & formats
- 8. 1. Files will be sent in standard packages & formats
- 9. 2. We’ll get MARCs for everything The dream: 1. Get 1000 PDFs 2. Get 1000 MARCs 3. Match 4. Load everything 5. Celebrate The reality: 1. Get some PDFs 2. Maybe get some MARCs? 3. Try to match 4. ¯_(ツ)_/¯ 5. Load what we can 6. Cry
- 10. 3. Everyone will buy everything
- 11. 4. DRM will be loose or non-existent
- 12. In sum ● Lack of standardization ● Third party miscommunication ● Licenses are a wild ride
- 13. Now what?
- 15. #BOOKSGOALS
- 16. Harmonized metadata ● BITS (Books Interchange Tag Suite) is the sister XML tag suite to JATS ● Allows for book-level and chapter-level metadata ● All publisher metadata is crosswalked to BITS, then ingested into MarkLogic
- 17. Accessible, accessible, accessible ● ACE: respond to Accessibility for Ontarians with Disabilities Act, reduce duplication, offer our students more. ● With a token, students can access scanned copies of books from their local collections ● Now: access the whole of their ebooks entitlements, request alternate formats on the fly.
- 19. Admin tool for all w/ hierarchical collections ● KBART on the fly ● MARC packages on the fly ● COUNTER 5 stats on the fly ...it’ll be pretty fly
- 21. DRM - the friendly version ● The friendly version is no DRM ● The second friendliest version replicates the experience non-DRM restricted content as closely as possible
- 24. Unsolved challenges ~ an interlude ~
- 25. Corrections ● There is no standard for how corrections to an already-loaded book are sent. ● If it’s the whole book - is it a duplicate? ● If it’s a page or chapter - how to integrate?
- 28. Adding local content ● ETD ● Other IR content ● Other local publications ● Workflows? ● Metadata? ● Entitlements?
- 29. OER ● The development of a provincial OER strategy is a hot topic ● First of all: preservation ● But what about copy-editing, remixing, peer-review within the system?
- 30. Web archiving ● Archive-It use is on the upswing ● How can we make a home for non-PDF content? ● How can institutions contribute their own collections?
- 31. What is the future of managing ebooks? Stewardship
- 32. Thanks for listening! Questions? jacqueline@scholarsportal.info The Scholars Portal Books team is: Bartek Kawula, Sadia Khwaja, Ivan Jankovic, Sunil Manikonda, Ravit David, Annie Thomas Selvarajan, Jacqueline Whyte Appleby With support from: Kate Davis, Amaz Taufique, Bikram Singh, Harpinder Singh, and Carlos McGregor Muro.
Editor's Notes
- Good morning, my name is Jacqueline Whyte Appleby and I’m the Scholarly Resources Librarian with Scholars Portal, the Ontario Council of University Libraries Despite the title, I only plan to spend a little while talking about what’s messed up…the real question I want to address now isn’t why everything is so messy but
- How do we anticipate we anticipate the future of ebooks? My organization is building an ebooks platform now, fully aware that the ebooks landscape will be vastly different in five years. How do we get ready for what’s ahead? That’s not just a technology question, it’s a licensing question, it’s a budget question, it’s a staff development question. For today I’m mostly going to treat it as a technology question.
- Scholars Portal is the service arm of OCUL Content: local hosting & discovery point for books, journals, microdata, geospatial data Member: ILL, chat reference, Somewhere in the middle: accessible texts repository, research data management support Most of them do not participate in Portico, We are the long term preservation strategy of many of our members. This means that almost every license that OCUL negotiates for journals and books has a local load clause, ariculates the rights we need for long term preservation.
- Publishers loaded include IEEE, Wiley, Taylor & Francis, Springer, Morgan & Claypool, Oxford, Cambridge, all Canadian university presses, many other UPs Theplatform was released in late 2009, and the back end built using software that was shortly thereafter purchased by a major library vendor and not developed any further after 2011. So this is olllld. And kind of a black box in a lot of ways. We began a redevelopment of the platform in the summer of 2016. It’s available to all of our library staff in beta right now, and its public beta is scheduled for release in January. Full release in April. This is a really exciting opportunity for us because
- We made some assumptions way back in 2009. I do want to talk briefly about these assumptions, because I think a lot of us are still carrying around these assumptions, maybe not explicitly, but buried in our workflows.
- When we get journals, they come to us as issues, with a whole bunch of PDFs, one for each article. They always come like this. The formats and structures in which books are packaged are much, much more diverse. These are some pretty standard formats. The blue represents a folder, pink and red represent different file types.
- But there are also ways that we might get the content. For stuff that crosses many years, different books might have different file types. So we might have really good XML mark up, but only for books from 2016 onward. Or we might get some books with chapters, some books as single PDFs. Might get TEI. Might just get an excel file. So what this means is that it’s impossible to write loaders that can account for all of these configurations. It needs to know where to find Front Matter, it needs to know if it should be looking to concatenate chapters. And automation has been a real challenge too because this will change from load to load — a new person will be packaging the files and will do it differently. It’s not to blame them— there’s no standard for how these should come.
- We assumed this one so hard, we purchased a platform is dependent on it. We cannot load a book without an associated MARC record. That’s the only place the software is capable of grabbing metadata from. But the workflow for sending PDFs and MARCs together is a dream and not a reality. The reality of course is that there are always going to be some records missing, or a delay in sending them. But the reality is also that often third parties are hired to create the records, so there’s a communication disconnect. And in these cases what happens is we get emails from librarians and faculty going, “my book just came out where is it!!!!” and we can’t do anything until we have a record.
- Think back to the magical time that was 2009 when the big deal was still generally accepted as the way to get content. So our entitlements module was set up with the understanding that folks either bought the Oxford ebooks collection, or they didn’t. Fast forward to 2017 and we have: schools buying incredibly granular packages schools dropping out of deals halfway through a year, which means breaking up those entitlements. We have mergers and acquisitions which mean metadata no longer distinguishes between two publishers, while backlist is still sold separately. We have package configurations that change over time as publishers build or cease certain subject areas. And we have some schools buying through third party vendors while other schools are buying directly through publisher. Same books, but possibly sold in different configurations, and also possibly with different licensing terms.
- Which leads me to my last assumption— we went many many years only signing deals without concurrent use restrictions. And then one day a concurrent use restricted license showed up, and our software was not designed to deal with that. In many cases users are not allowed to print or copy even a single page from high demand content. We wound up integrating Adobe Content Server into the system, which did not go over well. A major selling point of our platform is a consistent interface for many different ebook packages, and this broke this. It was a surprise and an annoyance to users who were used to always being able to access everything in the browser.
- So we made some assumptions, and we were wrong, because of these things… - And I don’t think any of these issues have really gone away, so they need to be at the front of our minds as we go forward.
- So: we have an amazing opportunity to do things differently, to learn from past trials & tribulations. We got funding to hire two new programmers full time for two years. It has been fantastic to have new staff not only because it’s more people working on everything but because they are not bogged down in the history of the project. They are able to step back and say, “why?” Our new books platform is running on MarkLogic, which is the software we’ve been using for Journals for years. It’s also what Healthcare.gov runs on. It’s also what the NSA uses, I hear. It can handle a lot of data.
- This is what the reading experience will be like on the new platform. All PDFs are rendered as HTMLs, which makes for a much more fluid reading experience. The bar on the left can be toggled closed.
- In broad terms, we want a better user experience - (like everyone). But we’re really trying to think broadly about who who a user might be it’s a user at an OCUL institution It’s a user at an OCUL institution with a visual or perceptive disability It’s a library staff member at an OCUL institution It’s a publisher It’s our own staff It’s anyone in the world interested in the growing open access and public domain content we load
- BITS is the sister XML tag suite to JATS, which we use for journals. The plan is that a certain point books and journals content can be more integrated. They currently sit on two different platforms. Having related metadata standards is one of the most important steps we can take in that direction. We don’t get any metadata in BITS, we need to write crosswalks for 100% of the content we receive. We didn’t need to used to do that because a MARC is a MARC, just ingest what you get. We’re now able to make use of much richer metadata...but it’s always go a unique DTD. Onix 2 is not the same from publisher to publisher. Long term we hope BITS will be more widely adopted. JATS is pretty prevalent now, it’s been deemed pretty useful.
- As you probably know, students with a disabilties can request that a print book be scanned for them but this takes time and once they have the file, where is kept? The idea with ACE was to centtralize that process - scan once, then make available to all schools that have a local copy. And host it on the books platform so that students with disabilities can search and discover other works they have access to. BUT it was its own separate portal. It’s now been fully integrated into the Books platform. Authenticated students with a registered disability will have access to the ACE collection but they will also be able to search and download any book they’re entitled to in the system – that’s 250,000 titles from most commercial publishers.. And they’ll be able to request an alternate format if a standard PDF doesn’t work for them. Internet Archive has a tool that generates alternate formats, and turnaround is a couple of days. Just to be clear - our schools do not sign licenses that do not give us the right to transform content for users with disabilities.
- Journals is a TDR, Books is not yet. So part of building the new platform has been figuring out the workflow for this. The workflow will be different from journals, the landscape has changes since we began preserving that content. We will likely be using Archivematica, a tool for creating Archival Information Packages, and we’ll store the final product in the Ontario Library Research Cloud, our distributed cloud storage network which has nodes at five universities across the province. Preservation is as much about policy as it is about technology, so we’re really confirming our rights with each renewal. We need the right to locally load, the right to perpetual access, and the right to transform content over time. We cannot preserve a collection without those three grants, and our schools do not sign licenses that do not allow for long term preservation.
- As I said, eresource staff are users too, and we want to build a library of MarkLogic queries that mean staff can at any time grab a current list of titles loaded, can get MARCs if they use them—and a list of books that don’t have an associated MARC! —, and can also get COUNTER 5 stats. It does not make sense for us to work in COUNTER 4 at this point, so some crosswalking will be necessary.
- Collections are nested and hierarchical, so we can break them down as far as we need to.The admin tool is live, but the buttons are a mock-up.
- As I said, we weren’t set up to monitor concurrent use in the browser, we now feel fairly confident that we can…but we don’t have good data on user behaviour because no one’s had an option to not use ADE. Are there people who do actually prefer that format
- So we’re going to offer both. If you just want to browse a bit, or read a single chapter, you can do that in the browser. If you want to really engage with the book, you can download it using Adobe Digital Editions. And if a year from now we can see that no one is bothering to check out the books, we’ll probably cease using it.
- Again just a mock up
- Those are things we feel pretty confident about putting into place, these are challenges we’re
- This is an extremely unsexy issue that we have got to figure out how to deal with, but even better would be some standards organization deal with it and demand everyone fall in line. Since we can get initial deliveries in a standard format, I’m not holding out a lot of hope but...corrections are an issue.
- Poor mark up. It looks bad and is confusing, and is also an accessibility issue. To publishers credit most are very receptive to feedback, and we expect 2018 and 2019 books to look better, but no one is going to go back and fix these. The work it would take to teach a computer to scan a book and recognize a chapter heading and then replace what we’ve got is ….a lot for a small issue, so it’s on the back burner.
- So that’s the stuff we’ve done in the last year, or we know we need to get done shortly. But we also know there are broader, bigger changes happening, We know that digital scholarship is pushing the boundaries of what a book is —that the monograph read cover to cover or a chapter at a time is no longer the most useful unit of self-expression or study. So the bigger question is: how can we be adaptable to what comes next?
- Right now we’re getting one offs - often local faculty publications or local conference proceedings. And we deal with all of that by email. Could this act as a more repository-like tool? We have the underlying preservation infrastructure in place. Who’s in charge of making sure metadata is good and entitlements are accurate?
- Long term hosting, but also allowing for in-site remixing? Displaying reuse clauses Notification of reuse Versioning Discovery? Situation where instructor at one school wants to use book in a certain configuration, another instructor at another school wants to use it in an alternate way. Can we host both versions in a way that is useful? Preservation on the module level? In site peer review?? Can we integrate with tools like PressBooks to allow for copy editing, remixing, peer-review within the system?
- ArchiveIt is a tool for capturing and preserving web data. There’s a lot of concern about the preservation of local history and municipal documents. Federal and provincial there are some mandates in place, but noting at the more local level. For instances where PDFs are available, there’s interest in creating metadata and hosting the PDFs locally. It’s a lot of work, but we’re well set up to host PDFs But what about web data? Can we flexible enough to be able to display archived versions of websites? We think this will be an important piece of the preservation puzzle, long term. But to be useful, they should be indexed and searchable.