This document evaluates caching policies for Memento TimeMaps. The authors observed changes to over 4,000 TimeMaps for 92 days and analyzed caching policies based on the changes. They found that TimeMaps either remained unchanged (77.4%) or increased in size through new archives or mementos (22.6%). An optimal TimeMap cache Time-To-Live of 15 days balances freshness with reduced load on archives by caching incrementally improving TimeMaps.

1. An Evaluation of Caching Policies for
Memento TimeMaps
Justin F. Brunelle and Michael L. Nelson
Old Dominion University
{jbrunelle, mln}@cs.odu.edu
JCDL 2013
Indianapolis, Indiana
07/2013

2. Discovering Archived
nasa.gov Pages
Archived Pages => mementos
Mementos identified by URI-M
Live Pages => resources
Resources identified by URI-R 2

3. 3

4. TimeMaps: Lists of mementos
<http://mementoproxy.lanl.gov/aggr/timegate/http://www.nasa.gov/>;rel="timegate", <http://www.nasa.gov/>;rel="original",
<http://api.wayback.archive.org/memento/19961231235847/http://www.nasa.gov/>;rel="first memento";datetime="Tue, 31 Dec 1996 23:58:47 GMT",
<http://api.wayback.archive.org/memento/19970605230559/http://www.nasa.gov/>;rel="memento";datetime="Thu, 05 Jun 1997 23:05:59 GMT",
<http://api.wayback.archive.org/memento/19970711094601/http://www.nasa.gov/>;rel="memento";datetime="Fri, 11 Jul 1997 09:46:01 GMT",
<http://api.wayback.archive.org/memento/19981202170636/http://www.nasa.gov/>;rel="memento";datetime="Wed, 02 Dec 1998 17:06:36 GMT",
<http://api.wayback.archive.org/memento/19981212031235/http://www.nasa.gov/>;rel="memento";datetime="Sat, 12 Dec 1998 03:12:35 GMT",
<http://api.wayback.archive.org/memento/19990116233500/http://nasa.gov/>;rel="memento";datetime="Sat, 16 Jan 1999 23:35:00 GMT",
<http://api.wayback.archive.org/memento/19990117063022/http://nasa.gov/>;rel="memento";datetime="Sun, 17 Jan 1999 06:30:22 GMT",
<http://api.wayback.archive.org/memento/19990125091025/http://nasa.gov/>;rel="memento";datetime="Mon, 25 Jan 1999 09:10:25 GMT",
<http://api.wayback.archive.org/memento/19990203005545/http://nasa.gov/>;rel="memento";datetime="Wed, 03 Feb 1999 00:55:45 GMT",
<http://api.wayback.archive.org/memento/20080903053412/http://www.nasa.gov/>;rel="memento";datetime="Wed, 03 Sep 2008 05:34:12 GMT",
<http://webarchive.nationalarchives.gov.uk/20080904014810/http://www.nasa.gov/>;rel="memento";datetime="Thu, 04 Sep 2008 00:00:00 GMT",
<http://api.wayback.archive.org/memento/20080904055742/http://www.nasa.gov/>;rel="memento";datetime="Thu, 04 Sep 2008 05:57:42 GMT",
<http://webarchive.nationalarchives.gov.uk/20080906134025/http://www.nasa.gov/>;rel="memento";datetime="Sat, 06 Sep 2008 00:00:00 GMT",
<http://api.wayback.archive.org/memento/20080906143204/http://www.nasa.gov/>;rel="memento";datetime="Sat, 06 Sep 2008 14:32:04 GMT",
<http://webarchive.nationalarchives.gov.uk/20080907124040/http://www.nasa.gov/>;rel="memento";datetime="Sun, 07 Sep 2008 00:00:00 GMT",
<http://api.wayback.archive.org/memento/20080907160232/http://www.nasa.gov/>;rel="memento";datetime="Sun, 07 Sep 2008 16:02:32 GMT",
<http://webarchive.nationalarchives.gov.uk/20120809003120/http://www.nasa.gov/>;rel="memento";datetime="Thu, 09 Aug 2012 00:00:00 GMT",
<http://webarchive.nationalarchives.gov.uk/20120814175606/http://www.nasa.gov/>;rel="memento";datetime="Tue, 14 Aug 2012 00:00:00 GMT",
<http://webarchive.nationalarchives.gov.uk/20120819212348/http://www.nasa.gov/>;rel="memento";datetime="Sun, 19 Aug 2012 00:00:00 GMT",
<http://webarchive.nationalarchives.gov.uk/20120826185010/http://www.nasa.gov/>;rel="memento";datetime="Sun, 26 Aug 2012 00:00:00 GMT",
<http://webarchive.nationalarchives.gov.uk/20120909230516/http://www.nasa.gov/>;rel="last memento";datetime="Sun, 09 Sep 2012 00:00:00 GMT"
<http://api.wayback.archive.org/memento/19961231235847/http://www.nasa.gov/>
;rel="first memento";datetime="Tue, 31 Dec 1996 23:58:47 GMT"
http://webarchive.nationalarchives.gov.uk/20080907124040/http://www.nasa.gov/
;rel="memento";datetime="Sun, 07 Sep 2008 00:00:00 GMT",
4

5. Aggregating TimeMapes
• Multiple archives
• Expensive
• Caching reduces
load on archives
• Write-through
Cache
Aggre-
gator
Sort
IA TM
AIT TM
HTTP
Cache
…
5

6. Aggregator Cache
• TimeMaps change
• Only want to cache better TimeMaps
– Bigger is better
• Ideally monotonically increasing
• Two extremes:
– Never cache (TTL=0)
– Never update in cache (TTL=92)
6

7. Agenda
7

8. Cache content measures
• |a| => # of archives
<http://api.wayback.archive.org/memento/19961231235847/http://www.nasa.gov/>
;rel="first memento";datetime="Tue, 31 Dec 1996 23:58:47 GMT”,
• |m| => # of mementos
<http://api.wayback.archive.org/memento/19961231235847/http://www.nasa.gov/>
;rel="first memento";datetime="Tue, 31 Dec 1996 23:58:47 GMT”,
8

9. Same TimeMap
• |a| == |a'|
• |m| == |m'|
All archives have reported the same mementos.
TimeMap T
9
mm mm
mm
TimeMap T'
mm mm
mm
|a| = 2; |m| = 3 |a| = 2; |m| = 3

10. Gained Archives, Gained Mementos
• |a| < |a`|
• |m| < |m`|
A new archive (WebCite) has just indexed and
reported a memento for the first time.
10
TimeMap T
mm mm
mm
TimeMap T'
mm mm
mm
mm
|a| = 2; |m| = 3 |a| = 3; |m| = 4

11. • |a| == |a`|
• |m| < |m`|
The Internet Archive has released a set of new
mementos.
11
TimeMap T
mm mm
mm
TimeMap T'
mm mm
mm mm
Same Archives, Gained Mementos
|a| = 2; |m| = 3 |a| = 2; |m| = 4

12. Lost Archives, Same Mementos
• |a| > |a`|
• |m| == |m`|
A redaction of 1 memento took place in the Internet Archive which
now does not report mementos for this resource.
The UK Web Archive has released 1 new memento for this resource.
1212
TimeMap T '
mm mm
mm
TimeMap T
mm
mm
mm
|a| = 3; |m| = 3 |a| = 2; |m| = 3

13. Lost Archives, Gained Mementos
• |a| > |a`|
• |m| < |m`|
A redaction of 2 mementos took place in the Internet Archive which
now does not report mementos for this resource.
The UK Government Web Archive has released 3 new mementos for
this resource.
13
TimeMap T
mm mm
mm
TimeMap T'
mm
mmmm
mm
|a| = 2; |m| = 3 |a| = 1; |m| = 4

14. Lost Archives, Lost Mementos
• |a| > |a`|
• |m| > |m`|
Archive-It has removed a collection, and no longer reports
those mementos. No other archives have new mementos
of those resources.
14
TimeMap T
mm mm
mm
TimeMap T'
mm
|a| = 2; |m| = 3 |a| = 1; |m| = 1

15. Gained Archives, Lost Mementos
• |a| < |a`|
• |m| > |m`|
A new archive (WebCite) has just indexed and reported 1 memento for
the first time.
A server error at the Internet Archive caused an omission of 2
mementos.
15
TimeMap T
mm mm
mm
|a| = 2; |m| = 4
TimeMap T'
mm
mm
mm
|a| = 3; |m| = 3
mm

16. Agenda
16

17. Experiment Design
• Eliminate caching from local Memento proxies
• Daily observations of 4,000 TimeMaps for 92 days in 2013
• TimeMaps analyzed for changes & cardinality
• Investigated caching policies
• Outages observed from Memento/archives/department
17

18. Observations
Occurrence Description Action
77.4% Unchanged TimeMap Do not update cache
19.7% Lost archives, lost mementos Do not update cache
2.4% Gained archives, gained mementos Update cache
0.4% Same archives, gained mementos Update cache
0.1% Gained archives, lost mementos Do not update cache
0.01% Lost archives, same mementos Update cache
0.01% Lost archives, gained mementos Update cache
18

19. Impact of Change in TimeMaps
• Caching transient errors
– Not returned or not archived?
19

20. Cardinality of TimeMaps
<http://mementoproxy.lanl.gov/aggr/timegate/http://www.nasa.gov/>;rel="timegate", <http://www.nasa.gov/>;rel="original",
<http://api.wayback.archive.org/memento/19961231235847/http://www.nasa.gov/>;rel="first memento";datetime="Tue, 31 Dec 1996 23:58:47 GMT",
<http://api.wayback.archive.org/memento/19970605230559/http://www.nasa.gov/>;rel="memento";datetime="Thu, 05 Jun 1997 23:05:59 GMT",
<http://api.wayback.archive.org/memento/19970711094601/http://www.nasa.gov/>;rel="memento";datetime="Fri, 11 Jul 1997 09:46:01 GMT",
<http://api.wayback.archive.org/memento/19981202170636/http://www.nasa.gov/>;rel="memento";datetime="Wed, 02 Dec 1998 17:06:36 GMT",
<http://api.wayback.archive.org/memento/19981212031235/http://www.nasa.gov/>;rel="memento";datetime="Sat, 12 Dec 1998 03:12:35 GMT",
<http://api.wayback.archive.org/memento/19990116233500/http://nasa.gov/>;rel="memento";datetime="Sat, 16 Jan 1999 23:35:00 GMT",
<http://api.wayback.archive.org/memento/19990117063022/http://nasa.gov/>;rel="memento";datetime="Sun, 17 Jan 1999 06:30:22 GMT",
<http://api.wayback.archive.org/memento/19990125091025/http://nasa.gov/>;rel="memento";datetime="Mon, 25 Jan 1999 09:10:25 GMT",
<http://api.wayback.archive.org/memento/19990203005545/http://nasa.gov/>;rel="memento";datetime="Wed, 03 Feb 1999 00:55:45 GMT",
…
|TM| ?
20

21. Strict vs. Loose Matching
• Different archive, URI-M, datetime- Strict: 2, Loose: 2
<http://api.wayback.archive.org/memento/20080509125659/http://flare.prefuse.org/>;rel="memento";
datetime="Fri, 09 May 2008 12:56:59 GMT",
<http://webarchive.nationalarchives.gov.uk/20080908074106/http://flare.prefuse.org/>;rel="memento";
datetime="Mon, 08 Sep 2008 00:00:00 GMT",
• Same archive, datetime, different URI-M- Strict: 3, Loose: 1
<http://web.archive.org/web/20101101060204/http://aarp.org:80/Health/>;rel="memento";
datetime="Mon, 01 Nov 2010 06:02:04 GMT",
<http://web.archive.org/web/20101101060204/http://www.aarp.org:80/Health/>;rel="memento";
datetime=“Mon, 01 Nov 2010 06:02:04 GMT",
<http://web.archive.org/web/20101101060204/http://www.aarp.org:80/health/>;rel="memento";
datetime=“Mon, 01 Nov 2010 06:02:04 GMT",
• Same archive, different URI-M, bad datetime- Strict: 2, Loose: 2
<http://wayback.archive-it.org/2342/20110321192906/http://www.apple.com/iphone/find-my-iphone-
setup/>...datetime="Mon, 21 Mar 2011 00:00:00 GMT"
<http://wayback.archive-it.org/2354/20110321035356/http://www.apple.com/iphone/find-my-iphone-
setup/>...datetime="Mon, 21 Mar 2011 00:00:00 GMT"
21

22. Strict vs. Loose: translate.google.com
22

23. Agenda
23

24. Testing
• TTLs [0, 92]
– 0: Thrashed cache, best freshness
– 92: First TimeMap cached, no replacement
• Policies
– Unconditional
• Cardinality ignored
– Conditional
• Replacements occur when cardinality is better
24

25. Evaluation
• Minimize cost values:
– Q – Queries to the archives
– MemDays – number of missed mementos/day
• Calculated MemDays: mementos missed/day
TTL: ∞
TTL: 0 MemDays
Q
25

26. MemDays
26
6
|TM|=10
MemDay
=8

27. Optimal TTLUnconditional
Conditional
Optimal TTL= 9
Optimal TTL= 15
27

28. Agenda
28

29. Conclusion & Future Work
• 3-month observation of 4,000 TimeMaps
• Change patterns studied
– 80.2% of TimeMaps monotonically increase
– Others decrease
• Optimal TTL = 15 days
• Cache Improvements:
– Saves requests to the archives
• Worth reinvestigating
– Changed Memento landscape 29

30. Backups
30

31. www.nasa.gov 1996 - 2012
31

32. Memento
Integrates the past and present web
Now
Always
Current
2008 2006 200120082010
32

33. 33

34. Cardinality
• Size of a TimeMap
– # Archives?
– # Date times?
• TimeMaps:
• Cardinality:
• Monotonic Increase:
34