Memento: TimeGates, TimeBundles, and TimeMaps

The Memento Team Herbert Van de Sompel Michael L. Nelson Robert Sanderson Lyudmila Balakireva Scott Ainsworth Harihar Shankar Memento: TimeGates, TimeBundles, and TimeMaps Memento is partially funded by the Library of Congress

W3C Web Architecture: Resource – URI - Representation Resource Representation Represents URI Identifies dereference

W3C Web Architecture: Resource – URI - Representation dereference content negotiation Resource URI Identifies Representation 1 Represents Representation 2 Represents

How does Memento achieve this? In order to fully understand how Memento introduces a time dimension to the Web, we present a brief recap of Transparent Content Negotiation (conneg) in HTTP. RFC 2295. Transparent Content Negotiation in HTTP, http://www.ietf.org/rfc/rfc2295.txt

HTTP GET on URI A

GET with conneg on URI T – Server Choice – 302 Found – Step 1 transparently negotiable resource

GET with conneg on URI T – Server Choice – 302 Found – Step 2

GET with conneg on URI T – Server List – 406 Not Acceptable

How does Memento do This?
Component 1: Navigation towards an archived resource via its original resource, by leveraging content negotiation.

Terminology Intermission We introduce the term Memento to refer to an archived version of a resource. A Memento for a resource URI-R (as it existed) at time t i is a resource URI-M i [URI-R@t i ] for which the representation at any moment past its creation time t c is the same as the representation that was available from URI-R at time t i , with t c >= t i . Implicit in this definition is the notion that, once created, a Memento always keeps the same representation.

DT-conneg: Content Negotiation in the datetime dimension
RFC 2295 introduces conneg in the following dimensions: media type, language, compression, character set, e.g.:
HTTP Request:
Accept-Language: en-US
HTTP Response:
Content-Language: en-US
Inspired by RFC 2295, Memento introduces datetime conneg:
HTTP Request:
Accept-Datetime: Mon, 12 Oct 2009 14:20:33 GMT
HTTP Response:
Content-Datetime: Sun, 11 Oct 2009 11:18:05 GMT

Terminology Intermission We introduce the term TimeGate to refer to a transparently negotiable resource that supports the datetime dimension. A TimeGate for an original resource URI-R is a transparently negotiable resource URI-G[URI-R] for which all variant resources are Mementos URI-M i [URI-R@t i ] of the resource URI-R. Since multiple archives may host versions of URI-R, multiple TimeGates may exist for any given resource, i.e. one per archive.

A Memento HTTP Navigation involving an Aggregator
Scenario
www.digitalpreservation.gov points at TimeGate provided by an Aggregator
URI-R, URI-G, URI-M on different servers

Memento HTTP Flow HEAD R, Accept-Datetime 302  M, Vary, TCN, Link  R,B,M 200, Content-Datetime, Link  R,B,M GET G, Accept-Datetime GET M, Accept-Datetime 200, Link  G

Memento HTTP Flow: URI-R HEAD R, Accept-Datetime HEAD / HTTP/1.1 Host: www.digitalpreservation.gov Accept-Datetime: Sat, 10 Oct 2009 00:00:00 GMT Connection: close

Memento HTTP Flow: Success – URI-R HTTP/1.1 200 OK Date: Thu, 21 Jan 2010 00:02:12 GMT Server: Apache Link: <http://mementoproxy.lanl.gov/aggr/timegate/http://www.digitalpreservation.gov/> ; rel=“timegate” Content-Length: 255 Connection: close Content-Type: text/html; charset=iso-8859-1 200, Link  G

Memento HTTP Flow: URI-G GET G, Accept-Datetime GET /aggr/timegate/http://www.digitalpreservation.gov/ HTTP/1.1 Host: mementoproxy.lanl.gov Accept-Datetime: Sat, 10 Oct 2009 00:00:00 GMT Connection: close

Memento HTTP Flow: Success – URI-G 302  M, Vary, Link  R,B,M HTTP/1.1 302 Found Date: Thu, 21 Jan 2010 00:06:50 GMT Server: Apache TCN: choice Vary: negotiate, accept-datetime Location: http://wayback.archive-it.org/1610/20090928171405/http:// www.digitalpreservation.gov/ Link: <http://www.digitalpreservation.gov/>; rel="original", <http://mementoproxy.lanl.gov/aggr/timebundle/http://www.digitalpreservation.gov/>; rel="timebundle”, <http://wayback.archive -it.org/256/20051108162921/http://www.digitalpreservation.gov/>; rel=“first-memento”; datetime=“Tue, 08 Nov 2005 00:00:00 GMT”, <http://webcitation.org/query?id=1257028234035091>; rel=“next-memento”; datetime=”Sat, 31 Oct 2009 18:30:35 GMT”, <http://webcitation.org/query?id=1213058061345794>; rel=“prev-memento”; datetime="Mon, 09 Jun 2008 20:34:23 GMT”, <http://wayback.archive -it.org/256/20100120102000/http://www.digitalpreservation.gov/>; rel=“last-memento”; datetime=”Wed, 20 Jan 2010 10:20:00 GMT” Content-Length: 0 Connection: close

Memento HTTP Flow: URI-M GET M, Accept-Datetime GET /1610/20090928171405/http://www.digitalpreservation.gov/ HTTP/1.1 Host: wayback.archive-it.org Accept-Datetime: Sat, 10 Oct 2009 00:00:00 GMT Connection: close

Memento HTTP Flow: Success – URI-M 200, Content-Datetime, Link  R,B,M HTTP/1.1 200 OK Server: Apache-Coyote/1.1 X-Archive-Orig-Accept-Ranges: bytes … Content-Type: text/html;charset=utf-8 Content-Length: 23364 Date: Thu, 21 Jan 2010 00:09:40 GMT Content-Datetime: Mon, 28 Sep 2009 17:14:05 GMT Link: <http://www.digitalpreservation.gov/>; rel="original", <http://wayback.archive-it.org/web/timebundle/http://www.digitalpreservation.gov/>; rel="timebundle”, <http://wayback.archive -it.org/256/20051108162921/http://www.digitalpreservation.gov/>; rel=“first-memento”; datetime=“Tue, 08 Nov 2005 00:00:00 GMT”, <http://wayback.archive -it.org/256/20100120102000/http://www.digitalpreservation.gov/>; rel=“last-memento”; datetime=”Wed, 20 Jan 2010 10:20:00 GMT” Connection: close Link header values are local to wayback.archive-it.org and different than those provided by URI-G

300 Multiple Choices HTTP/1.1 300 Multiple Choices Server: Apache Content-Length: 705 Content-Type: text/html; charset=utf-8 Date: Thu, 21 Jan 2010 00:09:40 GMT TCN: list Vary: negotiate, accept-datetime Link: < http://en.wikipedia.org/Special:TimeBundle/http://en.wikipedia.org/wiki/DJ_Shadow >; rel="timebundle", < http://en.wikipedia.org/wiki/DJ_Shadow >; rel="original”, <http://en.wikipedia.org/w/index.php?title=DJ_Shadow&oldid=1493688>; rel=“first-memento”; datetime="Sun, 28 Sep 2003 01:42:00 GMT”, <http://en.wikipedia.org/w/index.php?title=DJ_Shadow&oldid=337446696>; rel=“last-memento”; datetime="Tue, 12 Jan 2010 19:55:00 GMT”, <http://en.wikipedia.org/w/index.php?title=DJ_Shadow&oldid=322586071>; rel=“prev-memento”; datetime="Wed, 28 Oct 2009 14:307:00 GMT”, <http://en.wikipedia.org/w/index.php?title=DJ_Shadow&oldid=326164283” ; rel=“next-memento”; datetime="Thu, 26 Nov 2009 23:50:00 GMT” Connection: close
Two scenarios that generate a 300 at the TimeGate:
A client requests a 300 using the “Negotiate: 1.0” request header
An archive has two or more Mementos with the same Datetime (HTTP only supports second-level granularity)

406 Not Acceptable
A client request for a Memento with a datetime outside the first and last values will generate a 406
For example a request in Wikipedia with:
Accept-DateTime: Mon, 31 May 1999 00:00:00 GMT
HTTP/1.1 406 Not Acceptable Server: Apache Content-Length: 709 Content-Type: text/html; charset=utf-8 Date: Thu, 21 Jan 2010 00:09:40 GMT Vary: negotiate, accept-datetime TCN: list Link: < http://en.wikipedia.org/Special:TimeBundle/http://en.wikipedia.org/wiki/DJ_Shadow >; rel="timebundle", < http://en.wikipedia.org/wiki/DJ_Shadow >; rel="original”, <http://en.wikipedia.org/w/index.php?title=DJ_Shadow&oldid=1493688>; rel=“first-memento”; datetime="Sun, 28 Sep 2003 01:42:00 GMT”, <http://en.wikipedia.org/w/index.php?title=DJ_Shadow&oldid=337446696>; rel=“last-memento”; datetime="Tue, 12 Jan 2010 19:55:00 GMT”, Connection: close

The Web with Time Dimension added by Memento

There are two components to the Memento Solution:
Component 2: A discovery API for archives that allows requesting a list of all archived versions it holds for a resource with a given URI.
Done

Mementos for any given URI-R are distributed across archives.
In order to get a correct perspective of available Mementos, different archives need to be consulted.
Can do so in distributed consultation mode (slooow), or by consulting an aggregator.
Why an API?

Terminology Intermission We introduce the term TimeBundle to refer to a resource via which an overview of all Mementos for an original resource URI-R is available.
A TimeBundle for a resource URI-R, is a resource URI-B[URI-R] that is an aggregation of:
All Mementos URI-Mi [URI-R@t i ] available from an archive,
The archive's TimeGate URI-G for URI-R,
The original resource URI-R itself.

Memento DT-conneg component

See OAI-ORE: http://www.openarchives.org/ore/1.0/toc/ Memento DT-conneg component

Memento DT-conneg component Memento discovery component

Recall URI-G Response… 302  M, Vary, Link  R,B,M HTTP/1.1 302 Found Date: Thu, 21 Jan 2010 00:06:50 GMT Server: Apache TCN: choice Vary: negotiate, accept-datetime Location: http://wayback.archive-it.org/1610/20090928171405/http:// www.digitalpreservation.gov/ Link: <http://www.digitalpreservation.gov/>; rel="original", <http://mementoproxy.lanl.gov/aggr/timebundle/http://www.digitalpreservation.gov/>; rel="timebundle”, <http://wayback.archive -it.org/256/20051108162921/http://www.digitalpreservation.gov/>; rel=“first-memento”; datetime=“Tue, 08 Nov 2005 00:00:00 GMT”, <http://webcitation.org/query?id=1257028234035091>; rel=“next-memento”; datetime=”Sat, 31 Oct 2009 18:30:35 GMT”, <http://webcitation.org/query?id=1213058061345794>; rel=“prev-memento”; datetime="Mon, 09 Jun 2008 20:34:23 GMT”, <http://wayback.archive -it.org/256/20100120102000/http://www.digitalpreservation.gov/>; rel=“last-memento”; datetime=”Wed, 20 Jan 2010 10:20:00 GMT” Content-Length: 0 Connection: close

Dereferencing URI-B % telnet mementoproxy.lanl.gov 80 Trying 204.121.6.37... Connected to ttt.lanl.gov. Escape character is '^]'. HEAD /aggr/timebundle/http://www.digitalpreservation.gov/ HTTP/1.1 Host: mementoproxy.lanl.gov Connection: close HTTP/1.1 303 See Other Date: Wed, 21 Jul 2010 03:09:46 GMT Server: Apache Location: http://mementoproxy.lanl.gov/aggr/timemap/rdf/http://www.digitalpreservation.gov/ Vary: Accept Connection: close Content-Type: text/plain; charset=UTF-8 Connection closed by foreign host.

RDF?! Yuck! % telnet mementoproxy.lanl.gov 80 Trying 204.121.6.37... Connected to ttt.lanl.gov. Escape character is '^]'. HEAD /aggr/timebundle/http://www.digitalpreservation.gov/ HTTP/1.1 Accept: application/rdf+xml; q=0.0 Host: mementoproxy.lanl.gov Connection: close HTTP/1.1 303 See Other Date: Wed, 21 Jul 2010 03:12:42 GMT Server: Apache Location: http://mementoproxy.lanl.gov/aggr/timemap/link/http://www.digitalpreservation.gov/ Vary: Accept Connection: close Content-Type: text/plain; charset=UTF-8 Connection closed by foreign host.

TimeMap http://mementoproxy.lanl.gov/aggr/timemap/rdf/http://www.digitialpreservation.gov/ http://mementoproxy.lanl.gov/aggr/timemap/link/http://www.digitialpreservation.gov/ <http://mementoproxy.lanl.gov/aggr/timebundle/http://www.digitalpreservation.gov/>;rel="timebundle", <http://www.digitalpreservation.gov/>;rel="original", <http://web.archive.org/web/20020802022406/www.digitalpreservation.gov/>;rel="first-memento";datetime="Fri, 02 Aug 2002 02:24:06 GMT", <http://web.archive.org/web/20020921111830/www.digitalpreservation.gov/>;rel="memento";datetime="Sat, 21 Sep 2002 11:18:30 GMT", <http://web.archive.org/web/20020924113650/www.digitalpreservation.gov/>;rel="memento";datetime="Tue, 24 Sep 2002 11:36:50 GMT", <http://web.archive.org/web/20020927005417/www.digitalpreservation.gov/>;rel="memento";datetime="Fri, 27 Sep 2002 00:54:17 GMT", … [deletia]… <http://webarchive.nationalarchives.gov.uk/20080911010610/http://www.digitalpreservation.gov/>;rel="memento";datetime="Thu, 11 Sep 2008 00:00:00 GMT", <http://web.archive.org/web/20090516160321/www.digitalpreservation.gov/>;rel="memento";datetime="Sat, 16 May 2009 16:03:21 GMT", <http://web.archive.org/web/20090616162603/www.digitalpreservation.gov/>;rel="memento";datetime="Tue, 16 Jun 2009 16:26:03 GMT", <http://web.archive.org/web/20090716162514/www.digitalpreservation.gov/>;rel="memento";datetime="Thu, 16 Jul 2009 16:25:14 GMT", <http://web.archive.org/web/20090816181051/www.digitalpreservation.gov/>;rel="memento";datetime="Sun, 16 Aug 2009 18:10:51 GMT", <http://web.archive.org/web/20090916193533/www.digitalpreservation.gov/>;rel="memento";datetime="Wed, 16 Sep 2009 19:35:33 GMT", <http://wayback.archive-it.org/1610/20090928171405/http://www.digitalpreservation.gov/>;rel="memento";datetime="Mon, 28 Sep 2009 0 0:00:00 GMT", <http://web.archive.org/web/20091016235112/www.digitalpreservation.gov/>;rel="memento";datetime="Fri, 16 Oct 2009 23:51:12 GMT", <http://webcitation.org/query?id=1257028234035091>;rel="memento";datetime="Sat, 31 Oct 2009 18:30:35 GMT", <http://web.archive.org/web/20091116214743/www.digitalpreservation.gov/>;rel="memento";datetime="Mon, 16 Nov 2009 21:47:43 GMT", <http://web.archive.org/web/20091216192113/www.digitalpreservation.gov/>;rel="memento";datetime="Wed, 16 Dec 2009 19:21:13 GMT", <http://web.archive.org/web/20100116192640/www.digitalpreservation.gov/>;rel="memento";datetime="Sat, 16 Jan 2010 19:26:40 GMT", <http://web.archive.org/web/20100216193825/www.digitalpreservation.gov/>;rel="memento";datetime="Tue, 16 Feb 2010 19:38:25 GMT", <http://web.archive.org/web/20100316200421/www.digitalpreservation.gov/>;rel="memento";datetime="Tue, 16 Mar 2010 20:04:21 GMT", <http://web.archive.org/web/20100416195253/www.digitalpreservation.gov/>;rel="memento";datetime="Fri, 16 Apr 2010 19:52:53 GMT", <http://web.archive.org/web/20100516200754/www.digitalpreservation.gov/>;rel="last-memento";datetime="Sun, 16 May 2010 20:07:54 GMT"

TimeBundle API: For Discovery, Cross-Archive Services
Archive uses common approaches to make TimeBundles/TimeMaps discoverable:
SiteMaps,
Atom Feeds,
OAI-PMH.
Aggregator harvests and merges TimeMaps. Based on this information, the Aggregator exposes its own TimeGates.
Cross-archive
Finer datetime granularity
Better chances of matching a client’s datetime preference.
Can become a shared target for redirection for many web servers.

There are two components to the Memento Solution:
Component 2: A discovery API for archives that allows requesting a list of all archived versions it holds for a resource with a given URI.
Done Done

Memento wants to make navigating the Web’s Past Easy
http://www.mementoweb.org
http://groups.google.com/group/memento-dev

Memento: TimeGates, TimeBundles, and TimeMaps

by Michael Nelson on Jul 21, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 36

Statistics

Memento: TimeGates, TimeBundles, and TimeMaps — Presentation Transcript

http://ws-dl.blogspot.com	33
http://ws-dl.blogspot.ca	3