NYARC Web Archiving Program: Archivists Round Table of Metropolitan New York

1. QUALITY ASSURANCE ADMINISTRATIONNEED + VISION PRESERVATION QUESTIONS New York Art Resources Consortium Web Archiving Program Deborah Kempe Principal Investigator, Two-Year Mellon Grant “Making the Black Hole Gray” Sumitra Duncan NYARC Web Archiving Coordinator Celeste Brewer Seth Persons Molly Seegers NYARC Web Archiving Interns Karl-Rainer Blumenthal National Digital Stewardship Resident The Archivists Round Table of Metropolitan New York, Inc., June 15, 2015

2. QUALITY ASSURANCE ADMINISTRATIONNEED + VISION PRESERVATION QUESTIONS New York Art Resources Consortium Web Archiving Program Deborah Kempe Principal Investigator, Two-Year Mellon Grant “Making the Black Hole Gray” Sumitra Duncan NYARC Web Archiving Coordinator Celeste Brewer Seth Persons Molly Seegers NYARC Web Archiving Interns Karl-Rainer Blumenthal National Digital Stewardship Resident

3. NEED + VISION Why? The Digital Black Hole What? Deliverables How? Radical Collaboration Live Wayback Art ephemeraArt Analog Art Ephemera

4. NEED + VISION Why? The Digital Black Hole What? Deliverables How? Radical Collaboration Live Wayback

5. NEED + VISION Why? The Digital Black Hole What? Deliverables How? Radical Collaboration Live Wayback “The Web dwells in a never-ending present. It is — elementally — ethereal, ephemeral, unstable, and unreliable.” -Jill Lepore, The Cobweb: Can the Internet be archived? The New Yorker

7. NEED + VISION Why? The Digital Black Hole What? Deliverables How? Radical Collaboration Live Wayback “The Web dwells in a never-ending present. It is — elementally — ethereal, ephemeral, unstable, and unreliable.” -Jill Lepore, The Cobweb: Can the Internet be archived? The New Yorker

9. NEXT STEPS Why? The Digital Black Hole What? Deliverables How? Radical Collaboration Live Wayback GOALS FOR NYARC ● Rich and substantial digital resources in art history ● Seamless integration with other research materials ● APIs and judicious metadata ● Permanence and long-term preservation ● Scalable ● Extensible ● Sustainable AND FOR THE GREATER GOOD ● Networked collections ● Federal initiatives ● New approaches made possible through innovative thinking ● Tools for new research

11. ADMINISTRATION Live Wayback Staffing & Partnerships Workflow Elements Harvesting Access & Discovery

18. ADMINISTRATION Live Wayback Staffing & Partnerships Workflow Elements Harvesting Access & Discovery www.nyarc.org/webarchive

19. ADMINISTRATION Live Wayback Staffing & Partnerships Workflow Elements Harvesting Access & Discovery Discovery via Primo

21. QUALITY ASSURANCE Live Wayback Context Process Challenges Successes Test Crawl Adjust Scope Full Crawl of Site QA Process

22. QUALITY ASSURANCE Live Wayback Context Process Challenges Successes Patch Crawls ● capture embedded content that was missed in initial crawl ● patch URLs into overall crawl structures seamlessly

23. QUALITY ASSURANCE Live Wayback Context Process Challenges Successes Running patch crawls is time intensive and iterative. Each and every web page must be checked for missing URLs.

24. QUALITY ASSURANCE Live Wayback Context Process Challenges Successes Archive-It is not always able to tell when a URL is missing from a web page.

25. QUALITY ASSURANCE Live Wayback Context Process Challenges Successes Functionality of the web page must be checked in order to determine the extent of missing URLs.

26. QUALITY ASSURANCE Live Wayback Context Process Challenges Successes Testing the web page reveals unknown missing URLs and activates them for patch crawls.

27. QUALITY ASSURANCE Context Process Challenges Successes

28. QUALITY ASSURANCE Context Process Challenges Successes 2 months later... Unfortunately, by the time they engineered a solution, the content had already changed. However, we will be able to have successful crawls in the future.

29. QUALITY ASSURANCE Context Process Challenges Successes Large, Content Rich Websites

30. Context Process Challenges Successes QUALITY ASSURANCE 4 views of 755 exhibitions + 2 views of 1,206 artworks + 47 museum collections + other sections of the website = 5,473 clicks X 2 (both live and archived sites) = 10,946 clicks

31. QUALITY ASSURANCE Live Wayback Context Process Challenges Successes Video content hosted by a third-party may not playback even though the URLs have been patch crawled.

32. QUALITY ASSURANCE Live Wayback Context Process Challenges Successes Flash files (.swf) are triggered by time-based processes that require manual intervention (and patience). These files can conceal access to more files including other .swf, .jpegs, .html, and more.

33. QUALITY ASSURANCE Live Wayback Web archiving art resources has unique complications due to the use of images, video, dynamic content, and complex site structures. Context Process Challenges Successes Dynamic Content is often hidden behind php or javascript files which the crawler is unable to access. Sometimes the crawler is able to collect all of the necessary files, but is unable to display the viewing interface.

34. QUALITY ASSURANCE Live Wayback Context Process Challenges Successes

35. QUALITY ASSURANCE Live Wayback Context Process Challenges Successes

37. PRESERVATION Live Wayback Why preservation? Measuring success Remaining challenges

42. PRESERVATION Live Wayback Why preservation? Measuring success Remaining challenges <warc> <xml>

43. PRESERVATION Live Wayback Why preservation? Measuring success Remaining challenges Control Integrity Security Accessibility + + +

44. PRESERVATION Live Wayback Why preservation? Measuring success Remaining challenges Control Integrity Security Accessibility + + + Storage and Geographic Location File Fixity and Data Integrity Information Security Metadata & File Formats Levels of Preservation (NDSA, 2013)

45. PRESERVATION Live Wayback Why preservation? Measuring success Remaining challenges Storage and Geographic Location File Fixity and Data Integrity Information Security Metadata File Formats Levels of Preservation (NDSA, 2013) 3 421

49. PRESERVATION Live Wayback Why preservation? Measuring success Remaining challenges <warc> <xml>

53. Acknowledgements Presentation template and relationship diagram courtesy of Karl-Rainer Blumenthal Andy Goldsworthy artwork, courtesy of degine.blogspot.com An Art Resource in New York: The Collective Collection of the NYARC Art Museum Libraries, by Brian Lavoie, Ph.D. Senior Research Scientist, Günter Waibel Program Officer, OCLC Programs and Research, c2008. http://www.oclc. org/content/dam/research/publications/library/2008/2008-02.pdf http://hyperallergic.com/211250/moma-is-archiving-its-exhibition-websites-before-they- expire/ Icons from The Noun Project (https://thenounproject.com) ● “Vortex” by Eli Ratner ● “Light-bulb by Ian Mawle ● “Architect” by Luis Prado THANK YOU!

NYARC Web Archiving Program: Archivists Round Table of Metropolitan New York

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (11)

Similar to NYARC Web Archiving Program: Archivists Round Table of Metropolitan New York

Similar to NYARC Web Archiving Program: Archivists Round Table of Metropolitan New York (20)

Recently uploaded

Recently uploaded (20)

NYARC Web Archiving Program: Archivists Round Table of Metropolitan New York