JISC CNI Meeting, Edinburgh 2010

Supporting Technical Innovation in the UK: RepositoriesUK Paul Walk p.walk@ukoln.ac.uk UKOLN is supported by: www.ukoln.ac.uk A centre of expertise in digital information management

innovation support • UKOLN is now one of two JISC-funded Innovation Support Centres • this role is being worked out • UKOLN has a long-standing role supporting and helping to develop the JISC Information Environment • repositories • UKOLN has an increasing role in supporting developers in UK HE • RepositoriesUK is a JISC-funded UKOLN project 2

provenance.... • Intute IRS • nothing to do with taxes.... • Intute Institutional Repository Search • a managed aggregation underpinning a search interface for researchers • ePrints UK and the Resource Discovery Network 3

lessons • the aggregation has general potential value • a cache on the network • a search service is only one realisation of that potential value • separation of concerns was needed • a particular service (such as search) should not dictate the entire infrastructure • lessons from this project complemented some thinking I was doing elsewhere.... 4

familiar? machine interfaces API AP I I AP some aggregated data of broad interest and potential usefulness UI end-user 5

a pessimistic view.... end-user end-user end-user UI UI UI Future Future 3rd-party Future 3rd-party dev 3rd-party dev dev API AP I I AP some aggregated data of broad interest and potential usefulness = certainty UI = belief = speculation end-user 6

why is this? • funding follows services & happy users (& new features?) • funders like to see their investment showcased • infrastructure is mostly invisible - hard to ascertain impact from users • so, there is strong motivation to develop a user- facing service, and then concentrate resources on this 7

a better pattern? = certainty = belief end-user end-user = speculation UI UI 3rd-party focussed app app application pre-existing user- developed for facing service specific (OPAC, VLE, API requirement Facebook, (might be simply NetVibes....) for research and some aggregated data of broad interest and potential usefulness development) http://blog.paulwalk.net/2009/12/07/an-infrastructure-service-anti-pattern/ 8

RepUK • RepositoriesUK • a managed aggregation of repository metadata from UK HE institutions • un-normalised records • well-formed XML (no check for validity) • focussed on academic papers • goals: 1. support innovation 2. develop some business intelligence 3. develop infrastructure component for services 9

design principles • tiered service model (quasi SOA) • serving intermediaries • negotiated supply to consumers • built around an unnormalised cache of metadata • well-formed is good enough Local Service • Common just as well really.... Service Local Core Service Services API Common Service Local Service closely integrated loosely coupled 10

RepUK 2 XML XML XML Files Files WorldCat Google LCSH Identities & language & MIMAS identiﬁer JACS Names 3 SOLR SOLR Index Index Operational Metadata MySQL Registry Database (OpenDOAR) Export Export Process Export Process Process RDF Scheduler Database Harvester & Admin 4 XQuery HTML &XML RDFaXML Files 5 Files Repository XML Repository Repository Database 1 Document HTTP Server 11

progress • 750,000+ metadata records • ~140 repositories • 6 consuming projects so far.... 12

‘consumers’ to date • RIDIR • identiﬁers • Writeslike.us & FixRep • metadata & full-text • RKBExplorer & sameas • metadata to inform linked data • NaCTeM • full-text (text-mining) • Talis....? • hosting linked data 13

developer appreciation "We have found that the RepUK aggregated repository datasets are a very useful basis on which to build, and have used the data in a number of projects.... The ability to build on other services means that we can reuse what has been done, rather than replicating functionality, freeing more time to work on the key functionality of our own projects." 14

issues • state management is the real challenge! • deletions • changes • federation is consequently non-trivial • scale & inequality (one repository = half of all the records) • linking? • should the records in the aggregation ever be the target of a link? Or, should such links point to the source repository? • if we succeed with SEO, are we undermining source repositories? 15

new lessons • developers need infrastructure too! • ﬁnding the right place to intervene • funders need to ﬁnd ways to measure value which does not necessarily stem from direct end-user satisfaction • a leap of faith.... • doing what no one else wants to do, to paraphrase Prof. David Baker • creating the right environmental conditions to allow innovative services to emerge 16

http://blog.paulwalk.net	754
http://localhost	20
http://webcache.googleusercontent.com	1
http://www.paulwalk.net	1
http://mail.paulwalk.net	1

JISC CNI Meeting, Edinburgh 2010

by Paul Walk

Accessibility

Categories

Upload Details

Usage Rights

5 Embeds 777

Statistics

JISC CNI Meeting, Edinburgh 2010 Presentation Transcript