Leaving Black Boxes Behind
benefits and challenges
of running in-house developed
e-resource management and discovery systems
Evelyn Weiser
Annika Domin
Leipzig University Library
2016/06/08 ELAG 2016 | Copenhagen
Outline finc
amsl
index
administration
interface
1. project finc
open source discovery and commercial aggregated index
2. project amsl
open source electronic resource management
3. aggregated index
renewal or do-it-yourself?
4. aggregated index
building our own
5. synergy effects
index administration with amsl
Part 1:
Projects and Background
Part 2:
A Metadata Source’s
Lifecycle
Outline
Part 1:
Projects and Background
Part 2:
A Metadata Source’s
Lifecycle
Contract NegotiationLicensing Request Evaluation Technical Implementation Selection Processing and Update Catalogue
1. Licensing Request
2. Contract Negotiation
3. Evaluation
4. Technical Implementation
5. Selection
6. Processing and Update
7. Catalogue
Part 1:
Projects and Background
finc
amsl
index
administration
interface
2011 - 2013 : Project finc
development of resource discovery systems based on VuFind for 11
university libraries in Saxony
integration of heterogeneous data sources as well as Ex Libris’ Primo
Central index
first finc catalogue went live in March 2012
establishment of the finc user community in September 2015
finc
amsl
index
administration
interface
development of an electronic resource management system based on the
editor OntoWiki and linked data technologies
management of contracts, licences, holdings, contact details in RDF
integration of external bibliographic and organisation data
amsl went into live operation in 4th quarter of 2014
2013 contd : Project amsl
finc
amsl
index
administration
interface
licence for Primo Central was to expire mid-2015
evaluation of the index led to the question:
Is it possible, with moderate expense and with no qualitative or
quantitative restrictions, to build an aggregated index on our own?
2014 : Reconsidering the aggregated index
finc
amsl
index
administration
interface
motivation and goal:
greater transparency and more options for action
● sources
● quality
● indexing
● processes
● currentness
● coverage
2014 : Reconsidering the aggregated index
finc
amsl
index
administration
interface
Q3 / Q4 2014 evaluation of available metadata sources,
preliminary talks with providers
January 2015 decision to build our own aggregated index
Q1 / Q2 2015 technical realisation
April 2015 go live
2014 contd : Project finc AI - aggregated index
finc
amsl
index
administration
interface
2016 : Index administration using amsl
20 participating libraries
70 metadata sources
5000 collections
⇒ complex administration
⇒ amsl as management backend for the index
Part 2:
A Metadata Source’s Lifecycle
Licensing
Request
Documentation of potential metadata
sources in amsl from the very beginning.
Licensing Request
Metadata usage terms need to be included into the
licensing agreement.
Contract
Negotiation
Contract Negotiation
Taken from http://www.nationallizenzen.de/tools/al-musterlizenz
Licensing Request
A metadata test set needs to be evaluated both
bibliographically and technically.
Evaluation
Contract Negotiation EvaluationLicensing Request
The individual download and
normalization routines need to be
implemented.
Technical
Implementation
Contract Negotiation Evaluation Technical ImplementationLicensing Request
Contract Negotiation Evaluation Technical Implementation
metadata in
processing
format
download and
normalization
Technical
Implementation
XML/DC
XML/propr.
JSON/
JATS
XML/
JATS
OAI HTTP FTP DVD
Provider
JSON/
DC
CSV/
propr.
The individual download and
normalization routines need to be
implemented.
Licensing Request
Selection
Contract Negotiation Evaluation Technical Implementation Selection
Participating libraries can
select the collections they
want to display in their
catalogue frontends.
Licensing Request
The metadata is processed for the aggregated
index update.
Processing and
Update
Contract Negotiation Evaluation Technical Implementation Selection Processing and UpdateLicensing Request
Processing and
Update
Contract Negotiation Evaluation Technical Implementation
Provider
holdings
files
metadata in
processing
format
amsl
solr
metadata +
licensing
information
web catalogue
download and
normalization
enrichment with library
and licensing information
conversion to
solr format
Selection Processing and UpdateLicensing Request
The new metadata is available in
the web catalogue.
Catalogue
Contract Negotiation Evaluation Technical Implementation Selection Processing and Update CatalogueLicensing Request
Conclusion:
Benefits and Challenges
!
Challenges
increased staff requirement and
capacity building
project management and
cross-departmental cooperation
awareness-raising
for effort and expense
transition from project
to operational business
Benefits
vendor independence
software tailored to our needs
more options for action
active support of open standards
capacity building
!
Challenges
increased staff requirement and
capacity building
project management and
cross-departmental cooperation
awareness-raising
for effort and expense
transition from project
to operational business
Benefits
vendor independence
software tailored to our needs
more options for action
active support of open standards
capacity building
Any questions? Please feel free to contact us!
Leipzig University Library, Germany
http://ub.uni-leipzig.de
http://finc.info
http://amsl.technology
Code on Github:
Python:
https://github.com/ubleipzig/siskin
Go:
https://github.com/ubleipzig/span
OAI-Harvester:
https://github.com/ubleipzig/metha
Leander Seige
Head of IT
seige@ub.uni-leipzig.de
Evelyn Weiser
Head of e-resources
weiser@ub.uni-leipzig.de
Annika Domin
amsl ERMS
domin@ub.uni-leipzig.de

Leaving Blackboxes Behind - ELAG 2016

  • 1.
    Leaving Black BoxesBehind benefits and challenges of running in-house developed e-resource management and discovery systems Evelyn Weiser Annika Domin Leipzig University Library 2016/06/08 ELAG 2016 | Copenhagen
  • 2.
    Outline finc amsl index administration interface 1. projectfinc open source discovery and commercial aggregated index 2. project amsl open source electronic resource management 3. aggregated index renewal or do-it-yourself? 4. aggregated index building our own 5. synergy effects index administration with amsl Part 1: Projects and Background Part 2: A Metadata Source’s Lifecycle
  • 3.
    Outline Part 1: Projects andBackground Part 2: A Metadata Source’s Lifecycle Contract NegotiationLicensing Request Evaluation Technical Implementation Selection Processing and Update Catalogue 1. Licensing Request 2. Contract Negotiation 3. Evaluation 4. Technical Implementation 5. Selection 6. Processing and Update 7. Catalogue
  • 4.
  • 5.
    finc amsl index administration interface 2011 - 2013: Project finc development of resource discovery systems based on VuFind for 11 university libraries in Saxony integration of heterogeneous data sources as well as Ex Libris’ Primo Central index first finc catalogue went live in March 2012 establishment of the finc user community in September 2015
  • 6.
    finc amsl index administration interface development of anelectronic resource management system based on the editor OntoWiki and linked data technologies management of contracts, licences, holdings, contact details in RDF integration of external bibliographic and organisation data amsl went into live operation in 4th quarter of 2014 2013 contd : Project amsl
  • 7.
    finc amsl index administration interface licence for PrimoCentral was to expire mid-2015 evaluation of the index led to the question: Is it possible, with moderate expense and with no qualitative or quantitative restrictions, to build an aggregated index on our own? 2014 : Reconsidering the aggregated index
  • 8.
    finc amsl index administration interface motivation and goal: greatertransparency and more options for action ● sources ● quality ● indexing ● processes ● currentness ● coverage 2014 : Reconsidering the aggregated index
  • 9.
    finc amsl index administration interface Q3 / Q42014 evaluation of available metadata sources, preliminary talks with providers January 2015 decision to build our own aggregated index Q1 / Q2 2015 technical realisation April 2015 go live 2014 contd : Project finc AI - aggregated index
  • 10.
    finc amsl index administration interface 2016 : Indexadministration using amsl 20 participating libraries 70 metadata sources 5000 collections ⇒ complex administration ⇒ amsl as management backend for the index
  • 11.
    Part 2: A MetadataSource’s Lifecycle
  • 12.
    Licensing Request Documentation of potentialmetadata sources in amsl from the very beginning. Licensing Request
  • 13.
    Metadata usage termsneed to be included into the licensing agreement. Contract Negotiation Contract Negotiation Taken from http://www.nationallizenzen.de/tools/al-musterlizenz Licensing Request
  • 14.
    A metadata testset needs to be evaluated both bibliographically and technically. Evaluation Contract Negotiation EvaluationLicensing Request
  • 15.
    The individual downloadand normalization routines need to be implemented. Technical Implementation Contract Negotiation Evaluation Technical ImplementationLicensing Request
  • 16.
    Contract Negotiation EvaluationTechnical Implementation metadata in processing format download and normalization Technical Implementation XML/DC XML/propr. JSON/ JATS XML/ JATS OAI HTTP FTP DVD Provider JSON/ DC CSV/ propr. The individual download and normalization routines need to be implemented. Licensing Request
  • 17.
    Selection Contract Negotiation EvaluationTechnical Implementation Selection Participating libraries can select the collections they want to display in their catalogue frontends. Licensing Request
  • 18.
    The metadata isprocessed for the aggregated index update. Processing and Update Contract Negotiation Evaluation Technical Implementation Selection Processing and UpdateLicensing Request
  • 19.
    Processing and Update Contract NegotiationEvaluation Technical Implementation Provider holdings files metadata in processing format amsl solr metadata + licensing information web catalogue download and normalization enrichment with library and licensing information conversion to solr format Selection Processing and UpdateLicensing Request
  • 20.
    The new metadatais available in the web catalogue. Catalogue Contract Negotiation Evaluation Technical Implementation Selection Processing and Update CatalogueLicensing Request
  • 21.
  • 22.
    ! Challenges increased staff requirementand capacity building project management and cross-departmental cooperation awareness-raising for effort and expense transition from project to operational business Benefits vendor independence software tailored to our needs more options for action active support of open standards capacity building
  • 23.
    ! Challenges increased staff requirementand capacity building project management and cross-departmental cooperation awareness-raising for effort and expense transition from project to operational business Benefits vendor independence software tailored to our needs more options for action active support of open standards capacity building
  • 24.
    Any questions? Pleasefeel free to contact us! Leipzig University Library, Germany http://ub.uni-leipzig.de http://finc.info http://amsl.technology Code on Github: Python: https://github.com/ubleipzig/siskin Go: https://github.com/ubleipzig/span OAI-Harvester: https://github.com/ubleipzig/metha Leander Seige Head of IT seige@ub.uni-leipzig.de Evelyn Weiser Head of e-resources weiser@ub.uni-leipzig.de Annika Domin amsl ERMS domin@ub.uni-leipzig.de