Presentation to Search Solutions conference, London, November 2013. Discusses use of open source technology including Solr and Blacklight to build a search engine with multiple content types, file formats and metadata standards from many collections

1. Dermot Frost
Digital Repository of Ireland
Trinity College Dublin

2. Mission
DRI is a trusted digital repository for Humanities
and Social Sciences Data
- linking and preserving the rich data held by Irish
institutions, with a central internet access point
- Our Cultural & Social Heritage

3. International Networks

4. App
App
Linked
Logainm
App
DRI Platform
Access
Preservation
Discovery
Federated Archives, Storage

5. Metadata

6. Formats

7. Global Good Data Practice
Digital Preservation
Data citation, Permanent IDs
Metrics, funding, allowable costs
Training
Sustained e-infrastructure
Copyright, IPR, licensing, data protection, embargoes
Open metadata, open access
Policy, Services, Systems → Practice

8. Kilkenny Design Workshops
Dr. Una Walker, NIVAL
Photographic archive
DRI Presentation

9. Irish Language
Dr. Seathrún Ó Tuairisg, NUIG
RTÉ RnG: 40 years broadcast history
Folklore gathering, béaloideas
DRI Presentation

10. Harry Clarke 1889 to 1931
Robin Adams, TCD Library
Stained Glass Windows
Business records, ephemera
DRI Presentation

11. Prof Chris Morash & Dr John Keating, NUIM
THE MEDIA ENVELOPE
SOURCE TYPE: All Sources
☛
01 January 1973 ☛
GENRE: ALL GENRES
☛
SEARCH FACILITY
Mo
n
Tu
e
W
ed
15:00
LANGUAGE: All Languages
Th
u
Fri
Sa
t
15:30
Su
n
☛
USER LOGIN // PERSONALISATION
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
15 16 17
24
18
16:00
16:30
17:00
17:30
R.T.E. (Radio Telefís
Éireann)
Sesame Street
Ol.
Clarence the Cross-Eyed Lion
Radio Éireann
. Three-o-One: The Sound of ...
Nua
..
Tógha..
01 January 1973 ☛
Mo
n
The Irish Press
Ireland in the EEC
18:00
Tu
e
W
ed
Th
u
Fri
Sa
t
---
Tu
e
W
ed
Th
u
06-07 January 1973 ☛
Mo
n
The Connacht Tribune
5 Accused of Misleading Villagers
Fri
Sa
t
Su
n
Tragedy Hits Holiday Group
The Y.
Coombe Hospital wins the
baby race
Music on th..
Official IRA
adm...
0
01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22
1
..
Roads’ Row Victory for Connemara
Hurling Star Drowned while Oyster Dredging
---
---
New
s
0
01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22
1
..
Martin McGuinness held at
Bridewell
---
The Kerryman
Su
n
“The Good ..
IVERNIA in London
DRI in Europe
Nora one of the firstPresentation
ß

12. Life Stories
Dr. Jane Gray, IQDA
Changing life patterns in Ireland, 1900s to present
"My mother used to make a ball and we used to play ball, she
used to make a hurl out of a bit of a board and make the
handle a bit thin and you could catch it, no shape or make it
only a bit of a board. And she used to make a ball out of a
soft set of turf and put an old sock around it"

13. REPOSITORY STRUCTURE

14. Open source components
Custom code to join them together

15. OAIS model

16. Search setup
Using Hydra – ruby on rails
Obvious to use Blacklight
Therefore use SOLR
Objects injested into Fedora Commons
Use the Solrizer gem to create the Solr index

18. Search setup
Object metadata all CC0
Search will return metadata on all records
Authorization system will restrict access to the objects
Multi-lingual data (English and Irish at the moment)
Indices for each language
Can search across specific or all

19. DATA PRESERVATION

20. Preservation strategy
Multi-site repository
Dublin and Maynooth (~25km separation)
Asynchronous replication
Ability to catch errors on the fly
Segregated storage
Master copies with surrogates for public access

21. CEPH features
Using CEPH as the underlying storage system
Provides Posix, S3 and Block access
Using S3 – potential to move to commercial cloud
Tiered storage and multi-site features
Erasure coding to reduce raw storage needs

22. Data representation
Groups of objects bundled using bagit format
Checksums built into the format for error detection
Useful for bulk transport of objects
Potential integration with DARIAH storage testbed

23. USER ACCESS

24. User Access
Primarily through the blacklight search interface
Other routes
• Curated collections and virtual galleries
• Georeferenced data – mapping
• Temporal data – timelines
• User definied collections
• DOI references in papers

26. DRI Presentation

27. •
•
•
•
•
•
•
•
•
http://projectblacklight.org/
http://projecthydra.org/
http://www.fedora-commons.org/
http://opennebula.org/
http://lucene.apache.org/solr/
http://www.ceph.com/
http://tools.ietf.org/html/draft-kunze-bagit
http://www.dri.ie
http://apps.dri.ie/locationLODer/