The document discusses BiblioShare, a platform for distributing bibliographic data. It provides statistics on the usage of BiblioShare's ONIX 2.1 and 3.0 datasets in 2019, such as the number of new and updated records. It also shares the results of a survey conducted with BiblioShare users to understand their needs and pain points. Based on these findings, BiblioShare is working to rebuild their platform to improve efficiency and communication with users through features like receipt confirmations and validation reports.
6. Year over Year 2.1
• 2019
• Average New Records/day : 2942
• Average Updated : 140,612
• 2018
• Average New Records/day : 3,014
• Average Updated : 103,232
• 2017
• Average New Records/day: 1,836
• Average Updated : 101,556
Tech Forum 2020 #TechForum
7. Year over Year 3.0
• 2019
• Average New Records/day : 274
• Average Updated : 8000
• 2018
• Average New Records/day : 408
• Average Updated : 9290
• 2017
• Average New Records/day: 375
• Average Updated : 4278
Tech Forum 2020 #TechForum
8. 2019
– ONIX 2.1
– 923,756 new records were added in the year
– While 44,152,304 updates were applied in the year
– ONIX 3.0
– 84,792 new records
– While 2,490,456 updates were applied
Tech Forum 2020 #TechForum
9. BiblioShare 2019
61 new - 2.1 accounts added
15 new - 3.0 accounts added
Tech Forum 2020 #TechForum
10. BiblioShare 2019
39 - Publisher accounts
4 - Supplier accounts
33 - Web Service accounts
Tech Forum 2020 #TechForum
11. BiblioShare 2019
• 545 uploads from the 43 new provider
accounts for a total of ~4 gb
• avg = 6800 kb file size
• 18445 file uploads for a total of ~50 gb of
data
• avg = 50000 kb file size
Tech Forum 2020 #TechForum
16. BiblioShare 2019
• 560 thousand requests from the 33 new
webservice accounts looking for everything
from the full ONIX record to covers and
more.
Tech Forum 2020 #TechForum
17. BiblioShare 2019
Returns a list of all ISBNs held in BiblioShare regarding an
Asset including:
• Sample
• Excerpt
• TOC
• Guide
• Teacher's guide
• Back cover
• Author photo
Tech Forum 2020 #TechForum
18. BiblioShare 2019
• We added new capabilities to take in and
serve out:
• Back covers
• Teachers Guides
Tech Forum 2020 #TechForum
24. ONIX 2.1
• 132,404 Canadian ‘Authored’ Works (an
increase of 10k over last year)
ONIX 3.0
• Doubled to 32, 931 = (10% of total
BiblioShare 3.0 dataset)
Contributor Code = CA
25. Tale Two – Take Two
Rebuilding BiblioShare
Tech Forum 2020 #TechForum
26. Who answered the survey
Total participants: 49
• Publishers
• Distributors
• Data Service Providers
• Retailers
27. ONIX?
Do you create or use ONIX data for books?
88% (43/49) participants saw this question
84% - YES
Other methods for those who don’t (10%):
• Excel
• BNC Webform (so ONIX)
• MARC
• “Does not create, store, or distribute book metadata to aggregators”
28. ONIX Version
ONIX 2.1 reigns supreme
• Using ONIX 2.1 only - 57%
• Using ONIX 3.0 only - 7%
• Both - 37%
29. Workflow
Number 1 pain point is: inefficiency
• Duplication of efforts by staff members
• Fragmented ONIX flows (one for ebooks, one for print, etc.)
• Hard or cumbersome to make changes/updates.
32. BiblioShare
48% ALWAYS verify if their data has successfully distributed, 36% sometimes do
Main problems
• Lack of consistent communication from vendors/partners (some email, some don't)
• Lack of consistency in type of communication from vendors/partners (some send
confirmation emails, other only if there's an error)
• Do not have the staff resources to deal with verification follow-up
• Many are doing their own sleuthing by checking the websites they've distributed to
(or say they do)
33. BiblioShare
Communication
What information do you want provided back to you upon receipt of your data?
Top 3
1. Receipt confirmation – 28%
2. Certification & Report card – 19% each
3. In-depth report validation
34. At the same time ongoing interviews
We are talking in more detail with certain
stakeholders
• Library Wholesalers
• Retailers
• Solution providers
• Publishers
Tech Forum 2020 #TechForum
35. Next we sat down to try to capture
• All features that currently exist and try to set
priorities. Are some features no longer
needed? Are there new features that have
risen up?
Tech Forum 2020 #TechForum
36. Then what Architecture to use?
• Components:
– Decide on DB Solution / DB Design
– Decide on Coding Language Framework
– Decide on Solution/Project Structure
Tech Forum 2020 #TechForum @YourTwitterHandle
37. Then start modeling
• ONIX Incoming file manager
• ONIX Inbound processor
• Account and User management iteration 1
• ONIX Record Management iteration 1
• File Trackers iteration 1
• B1>B2 File copier
• Replication of the FTP directories /Live data processing
(consideration file cleanout routine add to the file
manager)
Tech Forum 2020 #TechForum @YourTwitterHandle
Hi Everyone and welcome to this years BiblioShare update for Tech Forum 2020. My name is Tim Middleton and I am one of the project managers at BookNet Canada . As mentioned, today I’m going to be talking about the BookNet data aggregation platform - BiblioShare. Before I get started I just wanted to say I hope everyone is doing well and taking care at this time of the Covid19 disruption.
This year we have two stories to tell about our BiblioShare platform. The first story is about the ongoing processing and delivery of data of BiblioShare. How it just keeps
The first story is about the latest greatest in BiblioShare specifically looking at how much data came in during 2019 and how much data being distributed
The first story is about our investment in rebuilding BiblioShare, from the ground up. This is not a task to approach lightly, nor have we. We have been spending many hours together as a team to discuss what can be improved, what are the essential features that we need, how can we build a faster, more flexible, more mature platform? How can we take a lot of the tasks that we do, that have emerged over the past 13 years of current BiblioShare productivity, leverage that knowledge and improve processes?
Clayton Christensen was an American academic and business consultant who developed the theory of "disruptive innovation", which was first introduced in his 1997 book The Innovator's Dilemma. Christensen passed away in January but the ideas he introduced are still very much alive. One of his ideas that I appreciate deeply is his ‘jobs-to-be-done’ insight. In a famous example of his Christensen was asked to figure out how a fast food chain could sell more milkshakes. The company had done all kinds of market research and had all kinds of data but ultimately none of that data helped them to sell more milkshakes and the main reason this data didn’t help the company was that they were missing a crucial insight. And that insight was that people were hiring the milkshake to do a job a for them. I won’t go into the details here but the main take away was that companies need to understand what job it is that people are hiring your product to do. So, what job can you hire BiblioShare to do?
Well one of the obvious jobs that BiblioShare can take care of is storing your data. BiblioShare can act as a huge Filing Cabinet – lets call it the Cabinet of Dr. Bibliographia. People can send their data to BiblioShare with the understanding that BiblioShare will store that record and have the data available when you, or someone else wanting the data are next looking for it.
BiblioShare is currently the filing cabinet for #over
3.4 million ONIX 2.1 records
And over 300,000 ONIX 3.0 records – and counting.
Looking at our #year over year growth
We can see that the filing cabinet is getting more and more use every day, if not for adding new files at least for updating the files that are already stored in the cabinet.
In 2019 the users of the BiblioShare filing cabinet #added
We can see that the filing cabinet is getting more and more use every day, if not for adding new files at least for updating the files that are already stored in the cabinet.
In 2019 the users of the BiblioShare filing cabinet #added
since April 01 2019 ONIX 2.1
923,756 new records
44,152,304 updates were applied
ONIX 3.0
84,792 new records
2,490,456 updates were applied
BiblioShare also #added
New companies that were looking for a good place to send and store their data. BiblioShare has had 61 new accounts added that want to send us their 2.1 data and 15 new accounts that want to send us 3.0 data. Currently BiblioShare has to take both 2.1 and 3.0 files since the industry has not cut over yet and there are still lots of companies out there that are looking to hire your 2.1 data from BiblioShare to do jobs at their firms.
The new companies that were added #breakdown
Into these three types of accounts in BiblioShare - 39 publisher accounts, 4 supplier accounts and 33 web service accounts.
Here I will just explain a bit about the differences of these three types of accounts.
The publisher accounts allows a publisher, regardless of their size or number of records they have to send into the great “Cabinet of Dr. Bibliographia”, to send us data that we then can say that that publisher "owns" in BiblioShare -- in particular Sales Rights.
A supplier is really just a different word for distributor. These accounts send data that can contain many different publishers ONIX data.
In theory if a supplier sends say something like measures that the publisher doesn’t provide then BiblioShare takes them from that supplier record, but in general suppliers/distrubotors are probably using a publisher file without addition. But ultimately BiblioShare does NOT update based on the Supplier file once the publisher owns the record -- we rely the publisher feed.
The one exception is the Supply Detail, which is owned by the Supplier.
This is an important consideration if, say, your files contains your US distributor's information and you want that in CataList. We can manually set ownership to the publisher on the entire record and "exclude" the supplier record if it's necessary. That's done on an ISBN level so it can be very surgical but it's a manual process where any new ISBN has to be added to the list. In general we hope that distribution belonging to the supplier is appropriate (and will carry their information like discount codes and the price they use and such).
The web service accounts are those types of accounts that are looking to hire your data to get jobs done.
Data is the new oil
Increased by over 500 thousand
#
170 – 240 thousand interior images
#
Almost double
We are also still trying to ramp
#up on Samples
Samples, Table of contents and reader guides all of which will soon be used in BNC CataList and Loanstars. We definitely have a ways to go here but there are more and more use cases emerging that require this data and so – well if you have it send it!
560 thousand requests from the 33 new webservice accounts
ISBNs for Assets: Returns a list of all ISBNs held in BiblioShare regarding an Asset including:
Sample
Excerpt
TOC
Guide
Teacher's guide
Back cover
Author photo
ISBNs for Assets: Returns a list of all ISBNs held in BiblioShare regarding an Asset including:
Sample
Excerpt
TOC
Guide
Teacher's guide
Back cover
Author photo
For a list of all of our webservices and how they are invoked you can check our Bibliographic Data Distribution documentation.
Our inaugural post was with Andrea Lacson who worked toward curating a list of books for diverse audiences.
If you or someone you know are using BiblioShare services in interesting ways we want to know. We have a set of 8 questions to…..
BiblioShare Webform is for small presses, unconventional publishers, museums, universities, self-published authors… in short, anyone who wants to provide clean, accurate metadata about their books to the supply chain through BiblioShare, but doesn’t have the resources to create complex ONIX files from scratch.
#Currently in Webform we have
Pricing
Annual pricing is based on the number of title records (ISBNs) your organization creates or maintains using the Webform. We offer a 25% discount for publishers who are also subscribed to CataList.
Annual Price
$50 $150 $175 $200 $375
Record Limit
10 50 100 200 500
We added 19 new accounts to Webform this year which has helped boost the total number of current Webform accounts to 56 for a total 2,913 titles created and sent to BiblioShare.
BiblioShares heart of hearts is really very Canadian. We are driven to help the discovery of Canadian content
And looking into the heart of BiblioShare with our Canadian flag glasses on
#we see
Publishers who have added CA when the author is Canadian. This is so helpful when we are asked about Canadian content.
We can and have created files that are used to populate sites that are only looking for Canadian titles. And we can add things like our lovely Canadian Flag in products that we create. You can see how in our ONIX 3.0 dataset there is an obvious slant toward Canadian publishers since you can assume most Canadian authored titles are coming from Canadian owned publishers. At least I assume that.
But is it enough? Canada is a big place, not every author identifies as Canadian so we can look into other places to uncover Canada
#one area we can look into is the new global classification system
As mentioned this year BiblioShare has two stories to tell. The second story is about our investment in rebuilding BiblioShare, from the ground up.
This is not a task to approach lightly, nor have we. We have been spending many hours together as a team to discuss what can be improved, what are the essential features that we need, how can we build a faster, more flexible, more mature platform? How can we take a lot of the tasks that we do, that have emerged over the past 13 years of current BiblioShare productivity, leverage that knowledge and improve processes?
Our first step was to survey our stakeholders in the industry to understand better how they currently interact with BiblioShare. The survey had
Special interest = those who use BiSh in ways that are slightly off the beaten path OR are of a type that we wish to recruit more of
Out of 30 participants:
ONIX Edit – 28%
Booksonix – 14%
COVERS - 28% (29)
Interior images - 17% (17)
Full book - 16% (16)