Transfers go in, not SIPs
Transfers go in, not SIPs•   Verify transfer compliance•   Rename with transfer UUID•   Assign file UUIDs to objects•   As...
Transfer directory structure (0.8)   Transfer_Name          /logs (only for logs created by Archivematica)          /metad...
Digitization metadata: excerpt
Ingest Planning: organization• Decide how to organize transfer contents :  logical or arbitrary groupings?• Don’t make AIP...
Ingest Planning: organizationTIFFs: arbitrary blocks of ~ 1,000    Audio & video: fewer files, fewer errors,for migration;...
Ingest Planning: Quality• Need time for quality control of digital  objects and metadata before ingest• Set aside problem ...
Ingest Planning: Quality• Check metadata completeness &  accuracy, image quality: no garbage in!• Started with export from...
Ingest Planning: Quality• Using custom MS Access form, volunteers  inspected master & derivative images next  to their des...
Ingest Planning: TIFFs• Separate files not to be  ingested and those to be  ingested as sub-items• Not for ingest: not own...
Assemble transfer objects:     no access files
Assemble transfer objects:    have access files
Tracking transfers
Configure Archivematica• Workflow (s): configure for each transfer unless  using default• AIP compression: will be the sam...
Workflow: processingMCP.xml• Overrides default processing xml files  used to process born-digital materials• Can customize...
Workflow: processingMCP.xml          Location Transfer_Name          /logs          /metadata          /objects      proce...
Workflow: processingMCP.xml          Example
Workflow: processingMCP.xml          Example
Workflow: processingMCP.xml          Example
Workflow: processingMCP.xml          Example
Workflow: processingMCP.xml          Example
Workflow: processingMCP.xml Example. No normalization
Workflow: processingMCP.xmlExample. Access normalization
Workflow: processingMCP.xml          Example
Setting AIP compression
Setting AIP compression
Normalization:Preservation planning tab
Normalization: TIFconvert "%fileFullName%" -sampling-factor 4:4:4 -quality 60 %outputDirectory                   %%prefix%...
Load Transfers
Structured directory on server
Staging area for transfers
Launching transfers
Monitoring using htop
Transfer failed: X marks it
Failed microservice: what went wrong?
Failed microservice message
Reporting: microservices
Reporting: Compress AIP           Greenwich Mean Time
Upload DIP directory• As they are created, DIPs appear here
Upload DIP directory• DIP objects in “objects” folder
Upload DIP directory• Inside objects folder, each DIP object with  new UUID
Completed AIPs
Inside the AIP directory
Open Terminal to checksum AIP bag
Checksum AIP bag
Copyto external drive  and then to   AIPstore
Exception: service copies
Exceptions: service directory  Transfer_Name           /logs           /metadata           /objects                 /acces...
Bare-metal Workflow: lots of copying!  Network                                        Network storage –                   ...
VM Workflow: less manual copying                                 automated   Network  storage – Digitized QA              ...
Using Archivematica 0.8 for Digitized Content
Using Archivematica 0.8 for Digitized Content
Upcoming SlideShare
Loading in …5
×

Using Archivematica 0.8 for Digitized Content

2,231 views

Published on

Shows production workflow currently in use for a bare-metal installation of Archivematica 0.8, without direct access to AIPstore or DIPstore. Screenshots show different Archivematica configurations available.

Published in: Technology, News & Politics
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,231
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
37
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Using Archivematica 0.8 for Digitized Content

  1. 1. Transfers go in, not SIPs
  2. 2. Transfers go in, not SIPs• Verify transfer compliance• Rename with transfer UUID• Assign file UUIDs to objects• Assign checksums and file sizes to objects• Verify metadata directory checksums• Generate METS.xml document• Extract packages• Scan for viruses• Sanitize objects file and directory names• Sanitize transfer name• Characterize and extract metadata• Create SIP(s)
  3. 3. Transfer directory structure (0.8) Transfer_Name /logs (only for logs created by Archivematica) /metadata checksum.md5 (must have this name) /submissionDocumentation (optional) info_about_digitization.xls /objects example.MKV transfer_name.csv /access (optional) example.MP4 (must have same name) transfer_name.csv processingMCP.xml
  4. 4. Digitization metadata: excerpt
  5. 5. Ingest Planning: organization• Decide how to organize transfer contents : logical or arbitrary groupings?• Don’t make AIP too big!• Organize files into single transfers – we did this on a network drive
  6. 6. Ingest Planning: organizationTIFFs: arbitrary blocks of ~ 1,000 Audio & video: fewer files, fewer errors,for migration; prob. logical groups easier to organize by fonds or series
  7. 7. Ingest Planning: Quality• Need time for quality control of digital objects and metadata before ingest• Set aside problem or “do later” files, such as those requiring rescanning
  8. 8. Ingest Planning: Quality• Check metadata completeness & accuracy, image quality: no garbage in!• Started with export from current db: current workflow is to only ingest items already described to item level• One item #, one file (and no more)• Filenames have to agree with item # and with csv ingested: spaces, capitalization, etc, including extensions
  9. 9. Ingest Planning: Quality• Using custom MS Access form, volunteers inspected master & derivative images next to their descriptions to ensure correct image• Image sizes double-checked in case any too small
  10. 10. Ingest Planning: TIFFs• Separate files not to be ingested and those to be ingested as sub-items• Not for ingest: not owned by us, should never have been digitized; or need descriptions• Sub-items: multi-page, need a procedure first
  11. 11. Assemble transfer objects: no access files
  12. 12. Assemble transfer objects: have access files
  13. 13. Tracking transfers
  14. 14. Configure Archivematica• Workflow (s): configure for each transfer unless using default• AIP compression: will be the same for all transfers processing at the same time• Normalization choices
  15. 15. Workflow: processingMCP.xml• Overrides default processing xml files used to process born-digital materials• Can customize to make the processing faster• Must always have exactly this name, even if contents vary
  16. 16. Workflow: processingMCP.xml Location Transfer_Name /logs /metadata /objects processingMCP.xml
  17. 17. Workflow: processingMCP.xml Example
  18. 18. Workflow: processingMCP.xml Example
  19. 19. Workflow: processingMCP.xml Example
  20. 20. Workflow: processingMCP.xml Example
  21. 21. Workflow: processingMCP.xml Example
  22. 22. Workflow: processingMCP.xml Example. No normalization
  23. 23. Workflow: processingMCP.xmlExample. Access normalization
  24. 24. Workflow: processingMCP.xml Example
  25. 25. Setting AIP compression
  26. 26. Setting AIP compression
  27. 27. Normalization:Preservation planning tab
  28. 28. Normalization: TIFconvert "%fileFullName%" -sampling-factor 4:4:4 -quality 60 %outputDirectory %%prefix%%fileName%%postfix%.jpg"
  29. 29. Load Transfers
  30. 30. Structured directory on server
  31. 31. Staging area for transfers
  32. 32. Launching transfers
  33. 33. Monitoring using htop
  34. 34. Transfer failed: X marks it
  35. 35. Failed microservice: what went wrong?
  36. 36. Failed microservice message
  37. 37. Reporting: microservices
  38. 38. Reporting: Compress AIP Greenwich Mean Time
  39. 39. Upload DIP directory• As they are created, DIPs appear here
  40. 40. Upload DIP directory• DIP objects in “objects” folder
  41. 41. Upload DIP directory• Inside objects folder, each DIP object with new UUID
  42. 42. Completed AIPs
  43. 43. Inside the AIP directory
  44. 44. Open Terminal to checksum AIP bag
  45. 45. Checksum AIP bag
  46. 46. Copyto external drive and then to AIPstore
  47. 47. Exception: service copies
  48. 48. Exceptions: service directory Transfer_Name /logs /metadata /objects /access /service processingMCP.xml
  49. 49. Bare-metal Workflow: lots of copying! Network Network storage – storage -Digitized QA AIPstore External drive External drive
  50. 50. VM Workflow: less manual copying automated Network storage – Digitized QA Network storage - AIPstore External drive

×