Establishing the Connection: Creating a Linked Data Version of the BNB

Establishing the Connection: Creating a Linked Data Version of the BNB Neil Wilson Head of Metadata Services

Changing Expectations Public Sector Metadata
The Web has accelerated development of a collaboration culture & fostered expectations that information & content should be as freely available as the Internet itself
Many wider benefit arguments have been advanced for public bodies to make their data freely available
2009 saw an increasing Government commitment to the principle of opening up public data for wider re-use.
The “Putting the Frontline First: Smarter Government” report required “ the majority of government-published information to be reusable, linked data by June 2011”

Developing an Open Metadata Strategy Choices and Challenges
When developing an open metadata strategy we wanted to:
Try and break away from library specific formats e.g. MARC and use more cross domain XML based standards e.g . DC, RDF etc
Develop the new formats with communities using the metadata
Get some form of attribution while also adopting a licensing model appropriate to the widest re-use of the metadata
Adopt a multi track approach addressing the needs of:
Traditional libraries
Researchers wanting to ‘data mine’ catalogues
& new linked data developers & users
… And deliver the above with decreasing resources

First Steps Toward An Open Metadata Strategy During 2010 We…
Developed a capability to supply metadata using RDF/XML standards used in the wider web community
Conducted trials with a range of new users including: the UK Intellectual Property Office & UNESCO
Developed a free Z39.50 MARC record download service for libraries to assist with derived cataloguing etc
Hosted a linked data workshop with 40 representatives from key international organisations

Current Status Since August 2010 We Have:
Created a new email enquiry point for BL metadata issues: [email_address]
Signed up nearly 400 organisations worldwide to the free MARC21 Z39.50 service
Worked with JISC, Talis & other linked data implementers on technical challenges, standards & licensing issues
Begun to offer sets of RDF/XML metadata under a Creative Commons 0 (CC0) license
Supplied multi-million record sets to organisations including: the Open Bibliography Project, the Open Library & Wikimedia Commons

Library Metadata & The Promise of Linked Data
Traditional library metadata uses a self contained, proprietary document based model
The Semantic Web uses a more dynamic data based model to establish relationships between data elements via links
By migrating from traditional models libraries could begin to:
Integrate their resources in the web, increasing visibility & reaching new users
Offer users a richer resource discovery experience
Transition from costly specialist technologies & suppliers & widen their choice of options
Open Standards
Dynamic/Reactive
Links to external resources
Micro Portal - Interacts with users & systems in response to queries
Offers options for further inquiry
Proprietary, library specific standards
Passive
Self contained
Linear text -‘Read’ by users as result of database query
Offers end result
‘ Semantic’ Metadata Properties Traditional Library Metadata Properties

Our Linked Data Journey… What to Offer?
Wanted to offer data allowing useful experimentation & advancing discussions from theory to practice
Why BNB?
General database of published output and not an institutional catalogue of unique items
Mass produced works on all subjects, many with internationally recognised identifiers e.g. ISBN
Reasonably uniform format across 60 years of publication
Significant amount of data – 3 million records in various languages

Our Linked Data Journey… What do we need to get there?
Wanted to undertake the work as an extension of existing activities and as an opportunity to develop expertise using:
Existing staff – librarians rather than IT experts
As many pre-existing tools or technologies as possible
Standard PC hardware for conversion
Library MARC21 data as a starting point
Established linked data resources to connect to
A proven platform that would enable us to concentrate on the data issues

Our Linked Data Journey… First stage: How To Migrate the Metadata?
From a flat catalogue card model to something more appropriate…
Preliminaries:
Staff training in linked data modelling concepts & increased familiarisation with RDF & XML concepts
Experience of working with: JISC Open Bibliography Project & Others
Feedback on initial MARC to XML conversion work
Incremental approach adopted
Open Data License
RDF/XML Format
Add External Links
Re-model
Create Linked Data

Our Linked Data Journey… Second stage: Selecting trusted resources to link to
To begin placing library data in a wider context & supplement or replace literal values in records
Looked for library sites:
Dewey Info
LCSH SKOS
VIAF
Plus more general sites:
GeoNames
Lexvo
RDF Book Mashup

Our Linked Data Journey… Third Stage: Matching and Generating Links
Three main approaches used:
Automatic Generation of URIs from elements in records e.g. DDC
Matching of text in records with linked data dumps e.g. personal names to VIAF & subjects to LCSH to identify URIs
Two stage crosswalk/matching process for some coded information e.g. MARC country & language codes for GeoNames

Our Linked Data Journey… MARC to RDF Conversion Workflow 1) Selection In-house utilities / MARC Report Exclusions (CIP; multiparts; serials)
2) Pre-processing
MARC Global
Normalise data values,
Remove trailing punctuation
Move/copy data values to
improve machine matching/transformation
3) Character set conversion
In-house utilities
Decomposed UTF-8 converted
to precomposed for conformance
with W3C recommendations
4) URI creation In-house utilities Create BL URIs in MARC fields) Harvest URIs from external sources 5) Data Transformation MARC Report & MARC 21/RDF XSLT Convert to RDF & Insert URI prefixes
MARC to RDF Conversion Consists of multiple
automated steps using
a range of tools

Our Linked Data Journey… MARC to RDF Conversion Workflow

Our Linked Data Journey… Which took us from here...

Our Linked Data Journey… Via here...

Our Linked Data Journey… To here...

bnb.data.bl.uk Preview Options
bnb.data.bl.uk/sparql
bnb.data.bl.uk/describe
bnb.data.bl.uk/search
.
Includes: BNB Books 2005-11
485,000 records
18,000,000 RDF Triples

bnb.data.bl.uk Sample ‘Labelled Concise Bound Description ’

Our Linked Data Journey… Journey’s End…Point?
Preview Details at:
http:// www.bl.uk/bibliographic/datafree.html
Roadmap for next steps includes:
Staged release over coming months for: books, serials, multi-parts etc
Aiming to update on a monthly basis once complete
Documentation & further refinement of data model
Looking at RDF triple dump option
What else might be offered?

Lessons Learned on the Journey General
It is a new way of thinking
Legacy data wasn’t designed for this purpose so starting can be problematic
There are many opinions…but few real certainties Everyone is learning & multiple solutions exist so you may be the best judge
Don’t reinvent the wheel...there are often tools or experience you can use. Start simple & develop in line with evolving staff expertise
Give careful thought to data modelling & sustainability issues e.g.
Where possible use cross domain standards e.g. ISO codes in data
Select relevant & stable targets when providing links if you are doing so

Lessons Learned on the Journey Data Issues
Reality check by offering samples for feedback to wider groups
Be prepared for some technical criticism in addition to positive feedback & try to continually improve in response
Conversion inevitably identifies hidden data issues…& creates new ones!
… But it’s often better to release an imperfect something than a perfect nothing!

Lessons Learned Along The Way Staff and Resource Issues
It can be a steep learning curve so:
Look for training opportunities to develop staff skills to support new open metadata standards
Cultivate a culture of enquiry & innovation among staff to widen perspectives on new possibilities
Look into collaborative pilot projects with peer organisations to share resources & expertise
See what tools are already out there that can save you development time or assist in checking data

Final Thoughts… For Others Contemplating a Similar Journey
It’s never going to be perfect first time
We expect to make mistakes
We aim to learn from them
We hope others will learn something too
… and that everyone benefits from the experience
So if anyone is thinking of undertaking a similar journey…..
Just do it!

Any Questions…? bnb.data.bl.uk/sparql bnb.data.bl.uk/describe bnb.data.bl.uk/search Images from

http://www.diglib.org	470
http://consulting.talis.com	432
http://talis-systems.com	196
http://translate.googleusercontent.com	3
http://webcache.googleusercontent.com	1

Establishing the Connection: Creating a Linked Data Version of the BNB

by nw13

Accessibility

Categories

Upload Details

Usage Rights

5 Embeds 1,102

Statistics