Project ‘The Digital City Revives’. A Case Study of Web Archaeology - A sneak preview: DIY Handbook for Web Archaeology
Tjarda de Haan, web archaeologist & guest e-curator Amsterdam Museum
Heritage Studies: Preservation and Presentation of the Moving Image
University of Amsterdam, 20 April 2017
Project ‘The Digital City Revives’. A Case Study of Web Archaeology
1. Project ‘The Digital City Revives’
A Case Study of Web Archaeology
-A sneak preview: DIY Handbook for Web Archaeology-
Tjarda de Haan, web archaeologist & guest e-curator Amsterdam Museum
Heritage Studies: Preservation and Presentation of the Moving Image
University of Amsterdam, 20 April 2017
2. DIY Handbook for Web Archaeology
• Introduction
o Starting points sharing knowledge, collaboration, participation
o Audience and objectives DIY
o How is this manual organized?
• Born digital cultural heritage
o DDS: treasury with (born) digital heritage
o Challenges of born digital heritage
• Getting started – A step-by-step manual
o Start plan, policy/strategy, document, blog, search partners
o Excavate collect stories, memories and data, crowdsourcing
o Reconstruct analyze data and make it accessible, deploy/develop web archaeological tools, reconstruct heritage
o Complete the collections archive DDS in e-depots, present
o Conclusions share knowledge
• Literature / references / sources
Project ‘DDS Revives’
3. 1. Introduction
2. Born digital cultural heritage
3. Getting started – A step-by-step manual
4. Literature / references / sources
Project ‘DDS Revives’
4. What is De Digitale Stad - The Digital City (1994-2001)?
• 1st (free) public domain virtual city in the world.
• 1st Dutch online community.
• 1st time internet (free) accessible to general public in the Netherlands.
• Grounded by a fluid group: independent media, hackers and the municipality of Amsterdam.
• Attracted international interest for the design: metaphor of a city to structure cyberspace.
• Good for the cyberreputation of the city of Amsterdam:
• CNN (1997): "For hundreds of years the city of Amsterdam has been a center of commercial trade, art and education.
Now it’s helping point the way in the information revolution too".
• Manuel Castells (The Internet Galaxy, 2001): "The most famous citizen computer network. (…) A new form of public
sphere combining local institutions, grassroots organisations, and computer networks in the development of cultural
expression and civic participation".
• Inhabitants (users): 1994: 12.000 - 1997: 60.000 - 1998: 80.000 - 2001: 150.000.
• Houses (homepages): 1996: 3.300 - 1997: 6.500 - 1998: 2.500 - 2001: 782
Project ‘DDS Revives’
5. Help! Our digital heritage is getting lost!
• "The world’s digital heritage is at risk of being
lost” and "It’s preservation is an urgent issue of
worldwide concern”, UNESCO in ‘Charter on the
preservation of the digital heritage’ (2003).
• "The Web was not designed to be preserved. The
average life of a Web page is about 100 days“,
Brewster Kahle, Internet Archive.
• “Humanity’s first steps into the digital world could
be lost to future historians. We face a "forgotten
generation, or even a forgotten century”, Vint
Cerf.
Project ‘DDS Revives’
6. Out of the box, collection 2.0
• The Amsterdam Museum had to cross boundaries and get out of its comfort-zone to break
new ground in dealing with digital heritage. To seek out new technologies, and new
disciplines.
• New areas: Different demands in acquiring and preservation of digitally created expressions
of culture or physical objects. How to preserve digital objects that are interactive,
networked, process-oriented and context-dependent?
• E-culture: Data are the new clay, scripts are the new shovels and the web is the youngest
layer of clay that we mine. Web archaeology is a new direction in e-culture in which we
excavate relatively new (born-digital) material, that has only recently been lost, with
relatively new (digital) tools. Both matter and methods to excavate and reconstruct our
digital past are very young and still developing.
Project ‘DDS Revives’
7. The Digital City: important heritage for Amsterdam
• Digital objects bring the story of Amsterdam to life
• The Amsterdam Museum has a long relationship with DDS
• The Digital City as a -pilot- digital object
• Why DDS?
– DDS marks a point in history
– DDS illustrates the role of Amsterdam in the Internet revolution
– DDS history has something typical Amsterdam
– DDS makes the development of Internet touchable
• Why do we do it again?
– Contributing to museum renovation
– Cooperation takes us further
– The importance of participation and co-creation
• The Digital City will definitely come alive!
• DDS can be preserved as heritage and DDS will contribute to bringing
the history of Amsterdam to life!
Project ‘DDS Revives’
8. 1. Introduction
2. Born digital cultural heritage
3. Getting started – A step-by-step manual
4. Literature / references / sources
Project ‘DDS Revives’
9. Challenges of born-digital heritage
• Material: information is complex and transitive by nature. By rapid obsolescence of
technology: missing hardware and software, lost documents and link rot. Dynamic character:
doesn’t always reach final status and often not clear what belongs to the object. Ethical
issues: regarding to privacy, copyright and licensing.
• Methodes: there is a difference between web archiving and web archaeology. Web
archiving is the equivalent of taking a photographic snapshot of an object, while we first
aim to recreate the digital object itself. And then: dynamic web archiving!
• Division of tasks: what we *badly* need is:
o Developing of web archaeological tools.
o Central repository for tools: for example to read obsolete media.
o Software library: who is archiving old software like Solaris, Windows, MacOs.
o Sustainable e-depot and infrastructure.
o Approach: no standards: given the urgency we can not wait. So just do it ... and trial and
error!
Project ‘DDS Revives’
10. Treasury with born-digital heritage
• DDS is an important historical source about the early years of the Web in
the Netherlands.
• Culture: DDS is a digital environment, a virtual city with inhabitants, which
brought forth objects, ideas and traditions.
• Technology: DDS is a complex information system with different
applications, through time. DDS is interactive, networked process-based
and context-dependent:
– 1994-1995: DDS 1.0 - telnet interface
– 1994-1995: DDS 2.0 - website: static HTML
– 1995-2001: DDS 3.0 - web system: static HTML and interactive web pages
DDS 1.0 - 15 January 1994 DDS 3.0 - 10 June 1995
DDS 2.0 - 1 October 1994
Project ‘DDS Revives’
11. 1. Introduction
2. Born digital cultural heritage
3. Getting started – A step-by-step manual
4. Literature / references / sources
Project ‘DDS Revives’
12. Project ‘DDS Revives’
Start: what do you do?
1. Start:
• Make a plan, develop strategy, document, blog, search partners
• Launch an Open History Lab and virtual museum
2. Excavate:
• Collect stories, memories and data
• Crowdsource & engage the community (Grave Diggers Party
(parties: plural!) & go to your audience: use social media)
3. Reconstruct:
• Analyse & make data readable/accessible
• Develop/deploy web archaeological tools
• Reconstruct DDS: 'historical true' & 'quick and dirty‘
4. Complete the collections:
• (Dynamic!) Archive DDS data in e-depots
• Let the Bytes Free & open it up to the audience: presentation!
5. Finish:
• Conclusions, evaluation, documentation, knowledge sharing
13. Goals of the project ‘DDS Revives’
• Reconstruct and preserve DDS;
• Provide insight into the (existing and new) processes, techniques and methods for born-
digital material and the context in which they are found, to excavate and reconstruct;
• Ask attention to the danger of ‘digital amnesia’;
• Provide museums and organizations with specialized knowledge about the reconstruction of
born-digital heritage and lower the threshold for future web archaeological projects.
Disseminating knowledge about new standards for archives on the storage of digital-born
heritage.
• Make digital cultural heritage DDS ‘future-proof’:
• Visible (content): promoting (re)use of DDS
• Usable (connection): improving (re)use of DDS collection by making it available by linking and enriching data.
• Preservable (services): maintain DDS sustainable and keep it accessible.
Project ‘DDS Revives’
16. Excavate: Collect stories, memories and data
• Search Wayback Machine - The Internet Archive is building a digital library of Internet sites
and other cultural artifacts in digital form. Go to: web.archive.org
• Search Momento - A search engine for web archives. Go to: timetravel.mementoweb.org
Project ‘DDS Revives’
17. Excavate: Collect stories, memories and data
• Search Delpher.nl - millions of digitized texts from Dutch newspapers, books, magazines and
radio. Texts from collections of scientific institutions, libraries and cultural heritage
institutions.
Project ‘DDS Revives’
18. Excavate: ‘Grave Diggers Party’
1. Working space The Archaeological Site re:DDS
• Workstations:
• Bring and upload your code.
• Digg in the Wayback Machine and store excavations in Historical (e-)Depot.
• Share your stories and memories in the Open History Lab https://hart.amsterdam/re-dds
• Tools:
• Computers: excavators.
• Storage: buckets.
• UNIX commands, mice: pades, pick-axe, trowels.
• Scripts : metal detectors.
• USB: find bags.
• Metadata: find cards.
2. Museum space Tourist Tours
• /Lost+found: ‘Cabinet of Curiosities’:
• Hardware: servers, terminals, modembanks, taperobots etc.
• Screenshots of DDS.
• Billboard’s:
• What is DDS? Where are you? What is this site? Why this site?
Project ‘DDS Revives’
19. Excavate: Collect stories, memories and data
Ask the experts (former users/inhabitants/creators):
• The appearances of the homepage
• Excerpts from the Metro (the Digital Underground)
• Terminals, servers, pictures of the first inhabitants, their avatars, the interfaces, the ideas suggested
• DDS running at Netscape on PCs of that time.
• All bytes that I'm lost
• Special things that are on photo's of the meetings. A coaster, a T-shirt etc
• The old site. As a tribute
• The ability to chat as in the time of DDS
• Large boards of DDS interface, a working DDS, a circle with computers where you can log in again and as
many DDS'ers :) You might also like to exhibit the homes of the residents, with the profiles
• Actually the atmosphere. But that will be difficult
• The Stone Age Computer.
• DDS 3.0 operating
• Bruine Kroeg (virtual bar)
Go to https://hart.amsterdam/whois
Project ‘DDS Revives’
20. Excavate: know what you are looking for
Archaeological remains of the city:
• Software - Interfaces (DDS 1,2,3 and 3.5), Freezes (1996 and DDS3).
• Hardware - Shaman, Alibaba, Aladdin, etc. Tape-robot, anybody?
• Special projects - web: DDS Webmix, Metro, cafés, etc.
• Special projects - hybrid: SMART TV, IjburgTV, Transparant Amsterdam, live.dds.nl, etc.
• Individual houses and squares - Thisbe, Plein van de Dood, etc.
Project ‘DDS Revives’
24. Reconstruction: From data to database
Three years HDC:
•FIRST: every participant signs a contract!
•Develop new methods to get more and different information
•Building datasets for scientists / researchers
•Building tools to query / research forensic tools, make searchable, filtering techniques, visualization etc
•Result: Working software so far: Avatar Generator & DDS3.0
•Next: Analysis of the content!
Project ‘DDS Revives’
25. 2 variants DDS3.0 (squares / houses interface):
Emulation:
•'DDS 3.0' - old code on new machines
•Recompile the original code
•Emulate hardware and/or software
•Starting point: restoring the original software
Replica:
•'DDS 4.0' - old data into new software
•Make-over with new technology
•Based on modern software
•Starting points: restore the user experience & recreate historical object & present it
without privacy and security issues
Project ‘DDS Revives’
27. Complete the collections: research
• From raw data to e-depot. How to preserve digital objects that are interactive, networked,
process-oriented and context-dependent? Using the Open Archive Information System
(OAIS):
o Ingest: How to read DDS data into an archive and what agreements can be made?
o Archival Storage: Which archives are reliable for the DDS data and how should the data be stored?
o Data management: How should DDS data be managed and which metadata are required?
o Administration: Who takes care of the maintenance, and what does that look like?
o Preservation planning: How do DDS data remain accessible in the future and how to manage this?
o Access: How can interested people get to the DDS data?
• How to present the DDS born-digital heritage in a museum context for future generations?
Project ‘DDS Revives’
28. Complete the collections: preserve and present
From excavation to e-depot: three preservation scenario’s:
1. Bit preservation. Only storage of excavated data: saving the ones and zeros without
sustainability actions. This backup can be used to create a reconstruction in the future.
2. Sustainable storage as dark-archive (= the master, designated community, read only).
Dynamic storage of all excavated and reconstructed DDS data. Starting point: full
preservation; storage of bits and preservation planning and action if necessary.
3. Sustainable storage ánd access. Dynamic storage of excavated and reconstructed DDS
3.0 data with two sub-scenarios:
o Providing access from the e-Depot
o Create a derivative variant for museum presentation
o The first benefits of long-term access, the second for the benefit of immediate use
Project ‘DDS Revives’
29. THANK YOU!!!!!
“Please do make sure that colleagues and contributors are made
aware of the impressive impact of their efforts and the high
regard with which we hold your work”. - William Kilbride, DPC
Because:
•The collaboration with the University of Amsterdam and also
with the community that used the website. That they were able
to dig out that data is extremely impressive.
•It preserves rich culture *and* it tells others how they can do it
too. But fundamentally this is a project all about people - at the
start, middle and end - and sometimes we forget this.
•This is an innovative and exciting project.
30. LOST & FOUND: DDS in the museum!
Project ‘DDS Revives’
31. And (again!) you celebrate with the community!
Project ‘DDS Revives’
32. Conclusions: not yet… we need you!
• Share your memories on https://hart.amsterdam/whois
• Do you have excavations? Mail t.dehaan@amsterdammuseum.nl
• Visit the Amsterdam Museum and immortalise yourself!
• Participate at the Heritage Hacking Labs! https://hart.amsterdam/reDDSagenda
Project ‘DDS Revives’
33. 1. Introduction
2. Born digital cultural heritage
3. Getting started – A step-by-step manual
4. Literature / references / sources
Project ‘DDS Revives’
34. Literature / references / sources
Get inspired:
• Digital Preservation Coalition: Digital Preservation Handbook:
http://www.dpconline.org/handbook/
• Charter on the Preservation of Digital Heritage: UNESCO
http://portal.unesco.org/en/ev.php-URL_ID=17721&URL_DO=DO_TOPIC&URL_SECTION=201.html
• Project "The Digital City Revives. A Case Study of Web Archaeology" in: Proceedings of
iPRES2016, 13th International Conference on Digital Preservation (July 2016).
http://www.ipres2016.ch/frontend/organizers/media/iPRES2016/_PDF/IPR16.Proceedings_3_Web_
• Wiki: Sustainable access to digital information: http://wiki.ncdd.nl (Dutch)
• Follow us: http://hart.amsterdammuseum.nl/re-dds (Dutch) - and take part!
Project ‘DDS Revives’
35. Thanks to:
Our partners (till now):
Amsterdam City Archives, Digital Heritage Netherlands, Digital Preservation Coalition, Dutch Cultural Coalition
for Digital Durability, Joost Flint, Karin Spaink, independent researcher, Dutch Computer Heritage
Foundation, International Institute of Social History, LIMA, National Coalition Digital Preservation,
National Library of the Netherlands, Netherlands Institute for Sound and Vision, Old inhabitants, (ex)
DDS employees and DDS affiliated web-archeologists, UvA Faculty of Science: dr. G. Alberts, Waag
Society.
Sources:
• Guidelines for the Preservating of Digital Heritage (March 2003).
http://unesdoc.unesco.org/images/0013/001300/130071e.pdf.
• IIPC: Web Archives: The Future(s), by Eric T. Meyer, Arthur Thomas, Ralph Schroeder (2011, University of Oxford)
http://netpreserve.org/events/Hague/Presentations/OII-IIPC.pdf.
• Charter on the Preservation of Digital Heritage: UNESCO http://portal.unesco.org/en/ev.php-
URL_ID=17721&URL_DO=DO_TOPIC&URL_SECTION=201.html.
• WIR SIND HIER – Opening TOTAL RECALL – The Evolution of Memory
http://www.aec.at/aeblog/en/2013/07/22/wir-sind-hier-opening-total-recall-the-evolution-of-memory/.
• The excavation of the Temple of Isis in Pompeii Photo: Bridgeman Art Library
http://www.telegraph.co.uk/history/pompeii/9852299/Pompeii-exhibition-a-timeline-of-Pompeii-and-
Herculaneum.html.
• Old files, by xkcd, https://xkcd.com/1360/ and see http://www.explainxkcd.com/wiki/index.php/Old_Files
• And more: https://del.icio.us/re_dds
Contact
• t.dehaan@amsterdammuseum.nl
Project ‘DDS Revives’
Editor's Notes
Gezicht op Amsterdam in vogelvlucht, 1538
Cornelis Anthonisz. (1500 - 1561) (schilder)
http://hdl.handle.net/11259/collection.38011
Van 24 januari tot 15 februari 1996 heeft een e-mail formulier online gestaan.
"Omdat het opslaan van e-mail een unieke mogelijkheid is voor toekomstige historici op een indruk te krijgen van taal, gebruiken, ideeën en leven van de Eerste Digitale Stad Bewoners, zullen we volgende week een formulier on-line zetten, waarin je je vrijwillig kunt aanmelden voor het wel bewaren in de freeze van de persoonlijke e-mail.”
Emulatie is het door middel van software nadoen van hardware en/of software. Virtual Machines (VMs) zijn programma's die dit kunnen. Al heb je een VM waar je historisch object op werkt dan kan je door enkel de VM software bruikbaar te houden de data toegankelijk houden, in tegenstelling tot alle dependencies van het object zelf want die zijn al werkend in de emulatie.
De emulatie, waar deels van de functionaliteit weer door hercompilatie aan de praat is gekregen. Dit betreft de complete originele structuur en de bestanden, zo ver mogelijk, onaangepast. Hier in zijn de e-mails wel aanwezig maar (nog) niet bereikbaar via de web pagina's.
Replica: Het namaken van het historisch object. Een replicatie probeert het gedrag van het historisch object na te doen. Dit is niet noodzakelijk via authentieke methoden. Zo maakte DDS in de tijd van de freeze geen gebruik van een database. De replica van DDS doet dit wel, echter als bezoeker aan de site zou je dit niet merken.
De replica heeft door middel van data mining technieken de gegevens uit de originele structuur gehaald en in een database gezet. Hier voor was geen email data nodig omdat deze versie alleen publiek toegankelijke data zou vertonen.
Het Digistad Cafe (IRC, Internet Relay Chat) - HyperNews - Nieuwsgroepen - E-mail - Webmix, SMART TV, Digitale Huiskamer, Digitale Stadsomroep - hybride media - De Metro
DDS = complex informatie systeem met verschillende applicaties, door de tijd heen.
Perl, C, Bash, Javascript, HTML, SUN OS, SPARC
DDS 1.0 telnet interface
DDS 2.0 website: statische HTML
DDS 3.0 - websysteem: statische HTML en interactieve webpagina’s
- Presentatie DDS uit c.a. 1999 (flash 2,6Mb) - https://www.dds.nl/dds/10jaar.php
D:\Werk\_AM\_Opdrachten\_ProjectplanDDS\2017 - NCDD\2017 - Week Digitaal Erfgoed\Presentatie (4 min)
- Webmix - Archeologische dienst: Deel 1 https://www.youtube.com/watch?v=7NMdWeMc40Y&t=3s (4 min 39 sec)
- SMART TV https://www.youtube.com/watch?time_continue=724&v=nCx887IW_wA (3”17”50)
- Wandeling door De Digitale Stad in video 1996 https://www.youtube.com/watch?v=Qde7d7ISgCI - Published on Jan 14, 2014. "opnames Nina Meilof, voor diverse presentaties over DDS door haarzelf. Zeer uitgebreide verzameling screenshots van pleinen, wijken, huizen. Begint met de openingspoort, eindigt met de plattegrond." (9 min 35 sec)