Your SlideShare is downloading. ×
0
The New DRS:
Plan for Metadata Migration
Harvard Library & Library Technology Services
February 26, 2014
Agenda
Welcome and introduction …... Franziska Frey, Tracey Robinson
Migration challenges .…………... Randy Stern
Creating th...
WELCOME & INTRODUCTION
Franziska Frey, Tracey Robinson
The DRS Advisory Group…
…provides oversight and guidance during the rollout
phase of the DRS2 project and ensures that the...
Why a New DRS?
• Upgrade to best-in-breed technologies
• Adopt digital preservation best practices
and standards
• Preserv...
Preservation Capability Before and
After the DRS2 Project
Level One Level Two Level Three Level Four
Storage & Geographic
...
Evolution of the DRS
2000 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 20122001
Current
DRS in
production
New DRS in
...
New DRS - Completed
2009 2010
2011 2012
convened
DRS
Advisory
Group
software in
production
2013 2014 2015
users trained,
p...
New DRS - Upcoming
2009 2010
2011 2012 2013 2014 2015
Infrastructure
Development
Metadata
Migration
& User
Adoption
metada...
MIGRATION CHALLENGES
Randy Stern
Why “Metadata” Migration?
Why not
“content” migration?
Pre-migration
DRS Content
Current
DRS
Database
Post-migration
DRS Content
Current
DRS
Database New DRS Database
New DRS
Index
New DRS Object
Descriptors
New DRS Data Model
• Not a simple metadata conversion
• A new DRS object is a logical intellectual
entity that unifies mul...
Object Descriptors
• METS files generated for each object
– Standards-based internal schemas (PREMIS,
MODS, MIX, etc.)
• M...
Technical Challenges
• Many formats
– Images, audio, text, digitized books, web
sites, documents, biomedical image stacks,...
Technical Challenges
• At 1 sec/file, 45 million files would take
520 days!
• We are designing the migration
software tool...
CREATING THE PLAN
Kate Bowers
Formulating a Migration Strategy
• Analysis of:
– DRS content
• Technical (relationships, etc. for building objects)
• Vol...
Migration Strategy Factors
• Combines needs of users with technical
requirements
• User sequencing will be based on:
– Cur...
WALKTHROUGH OF PLAN
Andrea Goethals
Migrating Content in 5 Stages
Migrate 1st : Tier 1 content
Migrate 2nd: Tier 2 content
Migrate 3rd: Tier 3 content
Migrate...
Migrating Content in 5 Stages
Migrate 1st : Tier 1 content
Migrate 2nd: Tier 2 content
Migrate 3rd: Tier 3 content
Migrate...
Migrating Content in 5 Stages
Migrate 1st : Tier 1 content
Migrate 2nd: Tier 2 content
Migrate 3rd: Tier 3 content
Migrate...
Migrating Content in 5 Stages
Tier Content
1 Text (Methodology, ESRI World File), Document, Color Profile, Target Image
2 ...
Migrating Content in 5 Stages
Tier Content
1 Text (Methodology, ESRI World File), Document, Color Profile, Target Image
2 ...
Migrating Content in 5 Stages
Tier Content
1 Text (Methodology, ESRI World File), Document, Color Profile, Target Image
2 ...
Tier 2: Sequence by DRS Owner Code
• Migrate just your unit’s PDS document and
still image content
• Minimize the amount o...
DRS Owner Codes to be Migrated
DIV.LIBR
FHCL.HOUGH
FHCL.MUSI
GSD.LIBR
RAD.ARCH
RAD.SCHL
FHCL.JUD
FHCL.FAL
FHCL.MAPS
FMUS.M...
Timing
• Current estimates:
–Building & testing migration tools: Now
–Begin Tier 1 content: Spring 2014
–Begin Tier 2 cont...
After Your Tier 2 Migration
• You, and anyone depositing on your
behalf, will begin depositing only to the
new DRS
• All o...
IMAGING SERVICES: MINIMIZING
DISRUPTION
Bill Comstock
Minimizing Disruption
• Testing by Imaging Services
• Uninterrupted services
• Migration sequencing
• Participating as a “...
Testing by Imaging Services
Alpha and beta testing:
• Depositing processes
• DRS content maintenance tools
–Searching and ...
Uninterrupted Services
Providing services before and after your
migration
• Content needs to be deposited
• Content needs ...
Migration Sequencing
• We will synchronize deposits with your
migration
–start depositing for you in the new DRS
after you...
Imaging Services as Pioneers
As pioneers, we:
• Learn to use the new tools
• Refine the new depositing workflows
• Identif...
WRAP-UP AND NEXT STEPS
Kate Bowers and Andrea Goethals
Nine Pioneers
• Limited number of first depositors to new
DRS
• Factors
– New DRS-ready content from new systems
• EAS (El...
First Deposit in the New DRS
大藏經 Da Zang Jing - Buddhist sutra, Qing dynasty (1644 -1911), China, Tibetan language
Email List
drs-migration@hulmail.harvard.edu
http://hul.harvard.edu/ois/systems/drs/drs2.html
Coming Attractions
• Open meetings
– Technical aspects brown bag (March)
– Digital preservation & DRS intro (Summer)
• Tra...
Q & A
Thanks!
The New DRS: Plan for Metadata Migration
Upcoming SlideShare
Loading in...5
×

The New DRS: Plan for Metadata Migration

260

Published on

Harvard Library and Library Technology Services review the new standard in digital preservation at the Library

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
260
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • The group is tasked to ensure that decisions during the project rollout are informed by the needs of active DRS depositors and content owners.
  • Minimizing disruption has been an emphasis of our planning.
  • Imaging Services staff have contributed to alpha and beta testing and have concentrated on what we know best.
  • Throughout the migration we want to make sure your work can continue.
  • Transcript of "The New DRS: Plan for Metadata Migration"

    1. 1. The New DRS: Plan for Metadata Migration Harvard Library & Library Technology Services February 26, 2014
    2. 2. Agenda Welcome and introduction …... Franziska Frey, Tracey Robinson Migration challenges .…………... Randy Stern Creating the plan ………………..... Kate Bowers Walkthrough of plan …………….. Andrea Goethals Imaging Services: minimizing disruption……….… Bill Comstock Wrap-up & next steps ………...… Kate Bowers, Andrea Goethals Q & A ……………………………….…... All
    3. 3. WELCOME & INTRODUCTION Franziska Frey, Tracey Robinson
    4. 4. The DRS Advisory Group… …provides oversight and guidance during the rollout phase of the DRS2 project and ensures that the user community of active DRS depositors and content owners contributes to decisions about the rollout. Amy Benson Kate Bowers Bill Comstock Franziska Frey (chair) Andrea Goethals Wendy Gogel Tracey Robinson Randy Stern
    5. 5. Why a New DRS? • Upgrade to best-in-breed technologies • Adopt digital preservation best practices and standards • Preserve metadata better • Improve collection management • Support preservation planning & activities • Improve access to content & metadata • Support more formats & genres
    6. 6. Preservation Capability Before and After the DRS2 Project Level One Level Two Level Three Level Four Storage & Geographic Location File Fixity and Data Integrity Information Security Metadata File Formats = already compliant = will be compliant after the DRS2 project Based on the NDSA Levels of Digital Preservation
    7. 7. Evolution of the DRS 2000 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 20122001 Current DRS in production New DRS in productionDRS enhancements New DRS infrastructure development 2013 2014 2015 New DRS metadata migration & user adoption
    8. 8. New DRS - Completed 2009 2010 2011 2012 convened DRS Advisory Group software in production 2013 2014 2015 users trained, phase 1 hardware in production migrated content to new hardware Infrastructure Development Metadata Migration & User Adoption Fedora assessment DuraCloud pilot test early release beta 1 beta 2 beta 3 first object deposited to the new DRS
    9. 9. New DRS - Upcoming 2009 2010 2011 2012 2013 2014 2015 Infrastructure Development Metadata Migration & User Adoption metadata migration tools created metadata migrated users moved
    10. 10. MIGRATION CHALLENGES Randy Stern
    11. 11. Why “Metadata” Migration? Why not “content” migration?
    12. 12. Pre-migration DRS Content Current DRS Database
    13. 13. Post-migration DRS Content Current DRS Database New DRS Database New DRS Index New DRS Object Descriptors
    14. 14. New DRS Data Model • Not a simple metadata conversion • A new DRS object is a logical intellectual entity that unifies multiple DRS files – Still image objects - archival and production masters, and deliverables including thumbnails – Audio objects - archival and production masters and deliverables – PDS objects - page image and text files
    15. 15. Object Descriptors • METS files generated for each object – Standards-based internal schemas (PREMIS, MODS, MIX, etc.) • Metadata gathered from multiple sources – Current DRS database – Every content file – HOLLIS records
    16. 16. Technical Challenges • Many formats – Images, audio, text, digitized books, web sites, documents, biomedical image stacks, opaque files • Unique migration rules per format – technical metadata, roles, relationships • Large (>5000 file) PDS documents • 45+ million DRS files
    17. 17. Technical Challenges • At 1 sec/file, 45 million files would take 520 days! • We are designing the migration software tools for parallel processing • We are configuring multiple servers to run the migration
    18. 18. CREATING THE PLAN Kate Bowers
    19. 19. Formulating a Migration Strategy • Analysis of: – DRS content • Technical (relationships, etc. for building objects) • Volume and type by repository –metadata for mapping –user activity in the DRS –survey of highest volume, active users –training and testing registration lists
    20. 20. Migration Strategy Factors • Combines needs of users with technical requirements • User sequencing will be based on: – Current deposit & administrative activity – Level of preparation (training and participation in beta testing)
    21. 21. WALKTHROUGH OF PLAN Andrea Goethals
    22. 22. Migrating Content in 5 Stages Migrate 1st : Tier 1 content Migrate 2nd: Tier 2 content Migrate 3rd: Tier 3 content Migrate 4th: Tier 4 content Migrate 5th: Tier 5 content
    23. 23. Migrating Content in 5 Stages Migrate 1st : Tier 1 content Migrate 2nd: Tier 2 content Migrate 3rd: Tier 3 content Migrate 4th: Tier 4 content Migrate 5th: Tier 5 content simpler objects more complex objects
    24. 24. Migrating Content in 5 Stages Migrate 1st : Tier 1 content Migrate 2nd: Tier 2 content Migrate 3rd: Tier 3 content Migrate 4th: Tier 4 content Migrate 5th: Tier 5 content dependencies between tiers dependencies within tiers
    25. 25. Migrating Content in 5 Stages Tier Content 1 Text (Methodology, ESRI World File), Document, Color Profile, Target Image 2 PDS Document, Still Image 3 Audio, Text (SMIL) 4 Web Harvest, Opaque Container 5 Biomedical Image; Google Document Container 1, 2, 3
    26. 26. Migrating Content in 5 Stages Tier Content 1 Text (Methodology, ESRI World File), Document, Color Profile, Target Image 2 PDS Document, Still Image 3 Audio, Text (SMIL) 4 Web Harvest, Opaque Container 5 Biomedical Image; Google Document Container 1, 2, 3
    27. 27. Migrating Content in 5 Stages Tier Content 1 Text (Methodology, ESRI World File), Document, Color Profile, Target Image 2 PDS Document, Still Image 3 Audio, Text (SMIL) 4 Web Harvest, Opaque Container 5 Biomedical Image; Google Document Container 1, 2, 3 Tiers 1, 3, 4, 5: Migrate across all DRS owner codes at one time Tier 2: Migrate one DRS owner code at a time
    28. 28. Tier 2: Sequence by DRS Owner Code • Migrate just your unit’s PDS document and still image content • Minimize the amount of time the content you manage the most is in 2 different systems
    29. 29. DRS Owner Codes to be Migrated DIV.LIBR FHCL.HOUGH FHCL.MUSI GSD.LIBR RAD.ARCH RAD.SCHL FHCL.JUD FHCL.FAL FHCL.MAPS FMUS.MCZ HLS.LIBR HUL.ARCH HUAM.MUSE HBS.BAKR VIT.BERE HUL.PRES FMUS.GRAY HPPM.PIRC DOAK.RESLIB DOAK.MUS FCOR.REISCH FMUS.ARN HLNC.LIBR ARB.AAHOD DOAK.LIBR FCOR.FORST FCOR.WOLBACH FMUS.FARL FMUS.HUH FMUS.ORC FMUS.PEAB HMS.COUNT HPRE.WARD HUAM.SARDIS HUL.GGL HUL.OIS FCOR.CARP FCOR.HCO FCOR.URI FHCL.CAB FHCL.COLL FHCL.DAVIS FHCL.ENV FHCL.FUNG FHCL.GOV FHCL.LITT FHCL.MED FHCL.SLV FHCL.TOZ FHCL.YENCH FMUS.SEM FMUS.WARE GSE.GUTMN KSG.LIBR
    30. 30. Timing • Current estimates: –Building & testing migration tools: Now –Begin Tier 1 content: Spring 2014 –Begin Tier 2 content: Summer 2014 • Units will be contacted about their Tier 2 migration schedule
    31. 31. After Your Tier 2 Migration • You, and anyone depositing on your behalf, will begin depositing only to the new DRS • All of your management tasks will be done only in the new DRS
    32. 32. IMAGING SERVICES: MINIMIZING DISRUPTION Bill Comstock
    33. 33. Minimizing Disruption • Testing by Imaging Services • Uninterrupted services • Migration sequencing • Participating as a “pioneer”
    34. 34. Testing by Imaging Services Alpha and beta testing: • Depositing processes • DRS content maintenance tools –Searching and assembling content for download –Editing PDS objects
    35. 35. Uninterrupted Services Providing services before and after your migration • Content needs to be deposited • Content needs to be searched • Content needs to be assembled • may need to be edited • may need to be downloaded
    36. 36. Migration Sequencing • We will synchronize deposits with your migration –start depositing for you in the new DRS after your Tier 2 content is migrated
    37. 37. Imaging Services as Pioneers As pioneers, we: • Learn to use the new tools • Refine the new depositing workflows • Identify bugs • Suggest improvements • Create a group of local experts that can support those that follow We’ll wear the scars so that you can stay pretty!
    38. 38. WRAP-UP AND NEXT STEPS Kate Bowers and Andrea Goethals
    39. 39. Nine Pioneers • Limited number of first depositors to new DRS • Factors – New DRS-ready content from new systems • EAS (Electronic Archiving Service), ACORN (Weissman Preservation Center conservation treatments), DASH (for ETD) – Prepared and trained staff – No content to migrate • HUA example: opaque objects
    40. 40. First Deposit in the New DRS 大藏經 Da Zang Jing - Buddhist sutra, Qing dynasty (1644 -1911), China, Tibetan language
    41. 41. Email List drs-migration@hulmail.harvard.edu
    42. 42. http://hul.harvard.edu/ois/systems/drs/drs2.html
    43. 43. Coming Attractions • Open meetings – Technical aspects brown bag (March) – Digital preservation & DRS intro (Summer) • Training and instruction – Refresher training – New training – Onsite assistance
    44. 44. Q & A Thanks!
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×