Tim Donohue DSpace + DuraCloud Integrations                                                                               ...
Basis for DSpace Integration1. DSpace AIP Backup & Restore (1.7 +)   (Initial DuraCloud use case: Backup & Restore)2. DSpa...
Intro to Archival Info Pkgs (1.7+)• Primary Use Cases  – Backup & Restore of DSpace Content     • All content or just part...
How to Backup DSpace (pre-1.7)                  Database                          Assetstore Folder Full Database         ...
How to Restore All (pre-1.7)                 Database                          Assetstore FolderFull Database             ...
How to Restore a Collection (pre-1.7)                       Database                          Assetstore FolderTemporary  ...
How to Restore a Collection (pre-1.7)                       Database                          Assetstore FolderTemporary  ...
Backup via Archival Info PkgsPackage for each  Community,Collection & Item        AIP backup       Licensed under Creative...
Restore All via Archival Info Pkgs Package for each   Community, Collection & Item         AIP backup        Licensed unde...
Restore a Collection via AIPs                                            1                          2   Collection AIP    ...
What’s in an AIP?                                                                 Content Files                License    ...
What’s in an AIP?RelatedObject AIPs                                                           Content Files          Licen...
The “Site” AIPTop-LevelCommunity  AIPs                                              METS                                  ...
What can AIPs restore?Restore All In-Archive Content (Files + Metadata)Restore All People & GroupsRestore All Permissions ...
Migrate a CollectionOne DSpace Install                                                           Another DSpace Install   ...
Migrate ContentOne DSpace Install                                                                                     (Fut...
DuraCloud Integration (1.7.x)                [1]                ./dspace packager -d                [2]                jav...
DuraCloud Integration (1.7.x)                [1]                java -jar retrievaltool.jar                [2]            ...
New: DSpace Replication Suite in 1.8Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA...
DSpace Curation System (1.7+)• Enables a basic ‘microservices’ approach to  curating DSpace objects• Anyone can build a ta...
DSpace Replication Suite (1.8)• A set of curation tasks geared towards  ‘replicating’ (backup/restore/audit) content• “Wra...
DuraCloud Integration (1.8.x)                                                              “Replication Task Suite”:      ...
DuraCloud Integration (1.8.x)                                                              “Replication Task Suite”:      ...
DSpace Replication Suite DemoLicensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)  ...
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)                         To req...
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)                         To req...
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)                         To req...
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)                         To req...
Known LimitationsCannot yet take advantage of DuraCloud  streaming capabilities (AIPs are zip files)Cannot yet take advant...
Next Steps• Working towards “unzipped” AIPs (1.8?)  – METS file & Content files stored in an AIP ‘folder’    but NOT zippe...
In Large Thanks to…• MIT : Richard Rodgers & Wendy Bossons  – Developed Curation Task Framework  – Developed initial Repli...
For More Information• Replication Task Suite:  – https://wiki.duraspace.org/display/DSPACE/Replic    ationTaskSuite• AIP B...
Photo/Icon Acknowledgments•   Package : http://www.flickr.com/photos/halfbisqued/2353845688/•   Harddrive & Terminal icons...
Upcoming SlideShare
Loading in …5
×

DSpace & DuraCloud Integrations

1,496 views
1,417 views

Published on

DSpace & DuraCloud Integrations talk, as presented as part of the DuraCloud Workshop at Open Repositories 2011 on June 6, 2011.

More Information on work presented in these slides can be found at:
* https://wiki.duraspace.org/display/DSPACE/ReplicationTaskSuite

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,496
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
33
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

DSpace & DuraCloud Integrations

  1. 1. Tim Donohue DSpace + DuraCloud Integrations DuraSpaceLicensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  2. 2. Basis for DSpace Integration1. DSpace AIP Backup & Restore (1.7 +) (Initial DuraCloud use case: Backup & Restore)2. DSpace Curation Task System (1.7 +)3. DSpace Replication Task Suite (1.8) Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  3. 3. Intro to Archival Info Pkgs (1.7+)• Primary Use Cases – Backup & Restore of DSpace Content • All content or just partial (Community/Collection/Item) – Migration/Export of DSpace Content • All content or just partial (Community/Collection/Item) – DuraCloud Integration Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  4. 4. How to Backup DSpace (pre-1.7) Database Assetstore Folder Full Database Folder Backup Backup Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  5. 5. How to Restore All (pre-1.7) Database Assetstore FolderFull Database Folder Backup Backup Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  6. 6. How to Restore a Collection (pre-1.7) Database Assetstore FolderTemporary Temporary Database Folder? Full Database Folder Backup Backup Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  7. 7. How to Restore a Collection (pre-1.7) Database Assetstore FolderTemporary Temporary Database Folder? Full Database Folder Backup Backup Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  8. 8. Backup via Archival Info PkgsPackage for each Community,Collection & Item AIP backup Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  9. 9. Restore All via Archival Info Pkgs Package for each Community, Collection & Item AIP backup Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  10. 10. Restore a Collection via AIPs 1 2 Collection AIP Items in Collection AIP backup Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  11. 11. What’s in an AIP? Content Files License or Logos Other Files METS in Bundles (DIM / MODS / PREMIS / (optional) METSRights) *Also a BagIt version in worksArchival Information Package (AIP) Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  12. 12. What’s in an AIP?RelatedObject AIPs Content Files License or Logos Other Files METS in Bundles (DIM / MODS / PREMIS / (optional) METSRights) Descriptive Metadata: DIM & MODS Tech/Preservation Metadata: PREMIS Rights Metadata: METSRights Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  13. 13. The “Site” AIPTop-LevelCommunity AIPs METS (DIM / MODS / PREMIS / METSRights) Special AIP for site-wide info/metadata: (e.g. Group Memberships, EPeople) Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  14. 14. What can AIPs restore?Restore All In-Archive Content (Files + Metadata)Restore All People & GroupsRestore All Permissions / Access RightsRestore Community / Collection Logos, Metadata, Rights & Item TemplatesRestore Community / Collection / Item HierarchyRestore In-Process / Incomplete ItemsRestore Collection OAI-PMH/ORE Harvest SettingsRestore all configuration files (dspace.cfg, etc.) Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  15. 15. Migrate a CollectionOne DSpace Install Another DSpace Install 2 1 Collection AIP Items in Collection Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  16. 16. Migrate ContentOne DSpace Install (Future work) 2 1 Collection AIP Items in Collection Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  17. 17. DuraCloud Integration (1.7.x) [1] ./dspace packager -d [2] java -jar synctool.jar 1Package for each Community,Collection & ItemLocal “Watch” Folder Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  18. 18. DuraCloud Integration (1.7.x) [1] java -jar retrievaltool.jar [2] ./dspace packager -r 2Package for each Community,Collection & Item Local Folder Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  19. 19. New: DSpace Replication Suite in 1.8Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  20. 20. DSpace Curation System (1.7+)• Enables a basic ‘microservices’ approach to curating DSpace objects• Anyone can build a task & share it.• Currently tasks must be written in Java – Working on JRuby & Jython integration (1.8?)• “Frees” admin tasks from Command Line – Can now run from Admin UI or CLI Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  21. 21. DSpace Replication Suite (1.8)• A set of curation tasks geared towards ‘replicating’ (backup/restore/audit) content• “Wraps” 1.7 DSpace AIP Backup & Restore Backup content to AIP (filesystem or DuraCloud) Restore/Replace from AIP Audit AIP (compare to DSpace content) Basic IO Tracking of AIP Upload/Downloads All replication tasks can be run via Admin UI Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  22. 22. DuraCloud Integration (1.8.x) “Replication Task Suite”: • Suite of Curation Tasks • One step Sync process ORCommand line Curation Tools • Via UI or CLI Package for each Community, 1 Collection & Item 1 Local Temp Folder (Cache) Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  23. 23. DuraCloud Integration (1.8.x) “Replication Task Suite”: • One step Retrieve process • Via UI or CLI ORCommand line Curation Tools • Also ‘auditing’ tools Package for each Community, 1 Collection & Item 1 Local Temp Folder (Cache) Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  24. 24. DSpace Replication Suite DemoLicensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  25. 25. Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  26. 26. Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  27. 27. Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  28. 28. Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  29. 29. Known LimitationsCannot yet take advantage of DuraCloud streaming capabilities (AIPs are zip files)Cannot yet take advantage of DuraCloud transformation services (AIPs are zip files) Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  30. 30. Next Steps• Working towards “unzipped” AIPs (1.8?) – METS file & Content files stored in an AIP ‘folder’ but NOT zipped up – Support for DuraCloud streaming, etc.• DSpace UI Streaming Integration (@mire)• ‘Auto-Sync’ options – Updates in DSpace -> DuraCloud (queued?) – Updates via DuraCloud services -> DSpace? Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  31. 31. In Large Thanks to…• MIT : Richard Rodgers & Wendy Bossons – Developed Curation Task Framework – Developed initial Replication Suite tasks• @mire : Mark Diggory – Look for @mire’s “Integrating DuraCloud Services in DSpace” talk on Friday at 3:30pm Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  32. 32. For More Information• Replication Task Suite: – https://wiki.duraspace.org/display/DSPACE/Replic ationTaskSuite• AIP Backup & Restore: – https://wiki.duraspace.org/display/DSDOC/AIP+Ba ckup+and+Restore• Curation Task System: – https://wiki.duraspace.org/display/DSDOC/Curati on+System Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org
  33. 33. Photo/Icon Acknowledgments• Package : http://www.flickr.com/photos/halfbisqued/2353845688/• Harddrive & Terminal icons: http://tango.freedesktop.org/Tango_Desktop_Project• Folder icon: http://www.openclipart.org/detail/13740• Database icon: http://www.openclipart.org/detail/68413• Zip Pkg icon: http://veryicon.com/icons/system/capital-icon-suite-mac/zip-10.html• File icons: http://veryicon.com/icons/system/rhor-v2-part-3/• Checkmark & Delete icons: http://veryicon.com/icons/system/on-stage/• Tools Icon: http://veryicon.com/icons/system/azullustre/ Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: info@duracloud.org

×