Project to Production: The Web       Archiving Service                DLF Fall Forum, 2010                    Tracy Seneca...
Web-at-Risk: 2005-2009University of California Curation Center, California Digital Library
Today:•   Whirlwind tour•   Project to production•   Current / future work•   The DLF community            University of C...
Build unique archives                     for local research communities                                                  ...
Archive Ephemeral Content  University of California Curation Center, California Digital Library
Analyze Site Change                                                     Are there new                                     ...
Easy to Use : Simple Workflow   University of California Curation Center, California Digital Library
Tailor Capture Settings                                                           Focus on site, directory                ...
http://was.cdlib.orgUniversity of California Curation Center, California Digital Library
Archive Home PageUniversity of California Curation Center, California Digital Library
Automatic List of Sites University of California Curation Center, California Digital Library
Search Across All Sites or Target Just One        University of California Curation Center, California Digital Library
WAS Snapshot: October 2010                                                  Stats: Since January 2007                     ...
Project to Production• Scale  – Infrastructure: ensure continuity of service    through server upgrades• Policy  – Setting...
Engage the users(Needs assessment doesn’t stop)  University of California Curation Center, California Digital Library
“Production” is not just for us!• How does WAS fit into existing workflows?• Is there a ‘web archivist’? Student workers?F...
Archiving the State of California                                Learning from the archives  California State Agency Sites...
Archiving the Gulf oil spill        Improving support for collaboration          527 sites          9288 + captures       ...
LSU tags relevant sites in DeliciousCDL imports Delicious JSON feed into WAS     University of California Curation Center,...
Interoperability of CDL Services    University of California Curation Center, California Digital Library
University of California Curation Center, California Digital Library
DLF CommunityTools                              Common issues/developments• WAS                              • Memento   –...
In-depth demos: Nov 8th and 16th    Workflows discussion: Nov 5th• http://www.facebook.com/webarchiving• http://was.cdlib....
Upcoming SlideShare
Loading in …5
×

Project to Production: The Web Archiving Service

305
-1

Published on

Seneca, Tracy. Project to Production: The Web Archiving Service. Digital Library Federation Forum. Palo Alto, CA. 2010.

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
305
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • I’ve talked about this issue – as it applies to WAS - a couple times beforeLook back at older presentations to look at how we guessed we were going to support collaborationHow much has happened as we expectedWhat has happened that we didn’t anticipate at all
  • Archive sites not just to preserve static content, but to see things you couldn’t see on the live web.
  • You determine what’s relevant to you
  • See WAS handout for basic detailsIn use by UC campus libraries, Stanford university, NYU, Minnesota Historical Society, Water Resources Center archives. University of Michigan Bentley Historical Library coming on board in July.81 archives under construction (public access available to 25)Archiving all state of California Government Agency Sites“Contact Us” link from WAS website if you have questions, video demos, user guides available.Available both within and beyond the UC
  • Curator controls the appearance & description of public archives
  • Browseable list of sites automatically providedAny descriptive information you provided for a site would display here as well
  • Search by key word or URLLimit results to a particular siteLimit results by file type (PDF, HTML, images, audio, video, MS Office)
  • These figures include 2007 – 2008 pilot activity.Very high number of dark archives reflects:Caution making content public- very active new users whose content is still embargoed for rights considerations- some may be combined after migration to SOLR
  • “Web resources” means both entire web sites, sections of websites pertaining to the spill, and individual resources such as patent information for blowout preventers etc.117 of these sites were also included in the 2005 Hurricane Katrina Web archive. That archive is not yet publicly available; we hope to provide access concurrent with the oil spill archive.400+ of these sites were selected by Louisiana State University subject experts.
  • “Web resources” means both entire web sites, sections of websites pertaining to the spill, and individual resources such as patent information for blowout preventers etc.117 of these sites were also included in the 2005 Hurricane Katrina Web archive. That archive is not yet publicly available; we hope to provide access concurrent with the oil spill archive.400+ of these sites were selected by Louisiana State University subject experts.
  • Gave subject experts access to WAS: # site nominations: 0Gave subject experts access to external site nomination tool: # nominations: 6Pulled librarian-nominated sites from Delicious: 400+
  • CDL’s Digital Preservation Repository currently provides preservation services for the Web Archiving Service. Forthcoming migration will improve curatorial access to preservation reports and features and improve the fundamental design of WAS preservation.CDL’s eScholarship service has already migrated to Merritt storage.IMLS grant proposal to create a link between eScholarship and WAS to scrape, harvest and preserve all cited URLs in faculty publications.Preserve and provide access to cited references along with publications
  • Project to Production: The Web Archiving Service

    1. 1. Project to Production: The Web Archiving Service DLF Fall Forum, 2010 Tracy Seneca University of California Curation Center, California Digital Library
    2. 2. Web-at-Risk: 2005-2009University of California Curation Center, California Digital Library
    3. 3. Today:• Whirlwind tour• Project to production• Current / future work• The DLF community University of California Curation Center, California Digital Library
    4. 4. Build unique archives for local research communities Geographically focused archives support local research • Los Angeles • Monterey Bay • Orange County • San Diego • Santa BarbaraTopical archives support specialresearch collections• Guantanamo Bay / Tamiment Library• California Water Districts / WaterResources Center Archives University of California Curation Center, California Digital Library
    5. 5. Archive Ephemeral Content University of California Curation Center, California Digital Library
    6. 6. Analyze Site Change Are there new documents on this site? Are there documents in my archive that have been removed from the live site?University of California Curation Center, California Digital Library
    7. 7. Easy to Use : Simple Workflow University of California Curation Center, California Digital Library
    8. 8. Tailor Capture Settings Focus on site, directory or page Set appropriate frequency University of California Curation Center, California Digital Library
    9. 9. http://was.cdlib.orgUniversity of California Curation Center, California Digital Library
    10. 10. Archive Home PageUniversity of California Curation Center, California Digital Library
    11. 11. Automatic List of Sites University of California Curation Center, California Digital Library
    12. 12. Search Across All Sites or Target Just One University of California Curation Center, California Digital Library
    13. 13. WAS Snapshot: October 2010 Stats: Since January 2007 17 organizations 4,509 sites captured 33,619 captures run 21.7 terabytes 29 archives published University of California Curation Center, California Digital Library
    14. 14. Project to Production• Scale – Infrastructure: ensure continuity of service through server upgrades• Policy – Setting up service agreements University of California Curation Center, California Digital Library
    15. 15. Engage the users(Needs assessment doesn’t stop) University of California Curation Center, California Digital Library
    16. 16. “Production” is not just for us!• How does WAS fit into existing workflows?• Is there a ‘web archivist’? Student workers?Friday November 5th 10-11:30 amCurators from UC San Diego, NYU, Universityof Michigan and UC Berkeley share theirapproaches to web archiving.http://was.cdlib.orgWAS News University of California Curation Center, California Digital Library
    17. 17. Archiving the State of California Learning from the archives California State Agency Sites with robots no robots gone 2% 47% 51%307 sites1363 captures2.7 TBBegan October 2008 University of California Curation Center, California Digital Library
    18. 18. Archiving the Gulf oil spill Improving support for collaboration 527 sites 9288 + captures 1.6 TB Began May 5 University of California Curation Center, California Digital Library
    19. 19. LSU tags relevant sites in DeliciousCDL imports Delicious JSON feed into WAS University of California Curation Center, California Digital Library
    20. 20. Interoperability of CDL Services University of California Curation Center, California Digital Library
    21. 21. University of California Curation Center, California Digital Library
    22. 22. DLF CommunityTools Common issues/developments• WAS • Memento – UC libraries, NYU, Stanford • Policy• Archive-It – Can / should Facebook block – Columbia, Rice, Indiana… government pages from capture?• Other – Library of Congress, North Texas, Los Alamos • Collection • Collaboration • Imagination
    23. 23. In-depth demos: Nov 8th and 16th Workflows discussion: Nov 5th• http://www.facebook.com/webarchiving• http://was.cdlib.org > WAS News University of California Curation Center, California Digital Library

    ×