This presentation covers the general steps to restoring the US first website from a UNIX backup file. Presentation was given at IIPC GA 2015 at Stanford University.

1. IIPC GA 2015 – Stanford CA
Restoring US First Website
AHMED ALSUM
WEB PALEONTOLOGIST

2. IIPC GA 2015 – Stanford CA2

3. IIPC GA 2015 – Stanford CA
The Early WWW Exhibit on 2006
http://www.slac.stanford.edu/history/earlyweb/firstpages.shtml

4. IIPC GA 2015 – Stanford CA
Where are the pieces?

5. IIPC GA 2015 – Stanford CA
What we got from SLAC
• Yearly UNIX backup files.
• Mix of web pages and system artifacts like UNIX backup notes and
console history.

6. IIPC GA 2015 – Stanford CA
Where can we replay it at Stanford?
epa.gov

7. IIPC GA 2015 – Stanford CA
Reconstruction Strategy
Page Content
URI
Timestamp
CDX File WARC File
example.org 19980722050200 …. 22222 file1.warc
example.org 19991201081010 …. 33333 file1.warc
.
.
.
.
…..
……..
……….
……….
……….
……….

8. IIPC GA 2015 – Stanford CA
Page Content
CRAWLING THE ARCHIVE
 Determine a list of pages.
 Using wget command to combine these pages in WARC format.
 wget generates CDX file but with today’s date!
Page1.html
Page2.html
Page3.html
WARC File
Page1.html
…
…
Page2.html
…
…
Page3.html
…
…
wget
CDX File
Localhost/... 2014 …….
Localhost/... 2014 …….
Localhost/... 2014 …….

9. IIPC GA 2015 – Stanford CA
URI Challenges
 There was no one-one mapping between directory structure and URI.
 Host name changed through time.

10. IIPC GA 2015 – Stanford CA
URI Discovery
R
R
R R
R
Inlinks within the collection Backlinks in SE HTML Source code
Web Archives Publication about Early Web Interviews with SLAC staff

11. IIPC GA 2015 – Stanford CA
URI Discovery
FIRST SET OF URIS WERE ONE LEVEL
 File path: 1993/SLACVM/www/192/rl2971/default.html
 URL: http://slacvm.slac.stanford.edu/FIND/default.html
TO LIMIT ACCESS TO SLAC NETWORK
 http://slacvm.slac.stanford.edu/SLACONLY/something

12. IIPC GA 2015 – Stanford CA
Timestamp
 We depend on the UNIX back context file that listed the last-modified
date for each file.
 We updated the wget-generated CDX file with the old dates.

13. IIPC GA 2015 – Stanford CA
Access Problem
WE USED OPEN WAYBACK 2.0, BUT…
 For historical reason, the start-date was hard-coded to 1996, we’ve to
modify to start on 1991.
 Early websites have capital letters in the URI.
 SLAC early logo was on XBM format which becomes obsolete now.

14. IIPC GA 2015 – Stanford CA
Quality Assurance
WE SHARED THE RESTORED SITE WITH THE EARLY SLAC STAFF.

15. IIPC GA 2015 – Stanford CA
Results
Year Number of Discovered URI
1991 52
1992 119
1993 356
1994 663
1995 5264
1996 4911
1997 2026
1998 2378

16. IIPC GA 2015 – Stanford CA
SLAC homepage evolution

17. IIPC GA 2015 – Stanford CA
Home Page, we didn’t have homepage
1991 – DEFAULT PAGE
1995 – THREE HOMEPAGES
1998 – MODERN HOMEPAGE

18. IIPC GA 2015 – Stanford CA
First Under-construction page

19. IIPC GA 2015 – Stanford CA
Visit swap.stanford.edu