Dissertation Defense

Using the Web Infrastructure
for Real Time Recovery
of Missing Web Pages
Dissertation Defense
Martin Klein
mklein@cs.odu.edu
Old Dominion University
Norfolk, VA
07/18/2011
Committee:
Dr. Michael L. Nelson (Advisor)
Dr. Yaohang Li
Dr. Michele C. Weigle
Dr. Mohammad Zubair
Dr. Robert Sanderson
Dr. Herbert Van de Sompel

Agenda
LSs for Web Pages
DF Estimation Techniques
TC-DF Correlation
Web Page Titles
Synchronicity
Link Neighborhood LSs
Book of the Dead
Web Page Tags
2
Motivation
Background

The Problem
3

The Problem - 404 Errors
Expected lifetime of a web page is 44 days[Kahle1997]
URIs inaccessible in CS papers: 23%-53%[Lawrence2001]
Inaccessible web pages: 67% after 4 years [Koehler2002]
Inaccessible objects in DLs: 3%[Nelson2002]
URIs inaccessible in high IF journals: 3.8% after 3 months; 13% after 27 months [Dellavalle2003]
URIs inaccessible in D-Lib Magazine: ~30%[McCown2005]
URIs inaccessible (and not archived) in scholarly articles: ~25%[Sanderson2011]
4

The Problem - 404 Errors
Are they really gone? Or just relocated?
Has anybody crawled and indexed it?
Do Google, Yahoo!, Bing have a copy of the page?
Has the page been archived by a web archive?
Information retrieval techniques needed to (re-)discover content
5

The Solution?
Search engines
Requires knowledge about content
Problem with homographs (jaguar, present, lead, M/mobile, etc)
Problem with very frequent terms/names (Michael Nelson, Eric Miller, etc)
Web archives
Helps for apple pie recipe but not for web page of transferred faculty, e.g.
6

Content Similarity
JCDL 2005
http://www.jcdl2005.org/
July 2005
http://www.jcdl2005.org/
Today
7

Content Similarity
Hypertext 2006
http://www.ht06.org/
August 2006
http://www.ht06.org/
Today
8

Content Similarity
PSP 2003
http://www.pspcentral.org/events/annual_meeting_2003.html
http://www.pspcentral.org/events/archive/annual_meeting_2003.html
August 2003
Today
9

Content Similarity
ECDL 1999
http://www.informatik.uni-trier.de/~ley/
db/conf/ercimdl/ercimdl99.html
http://www-rocq.inria.fr/EuroDL99/
October 1999
Today
10

Content Similarity
Greynet 1999
http://www.konbib.nl/infolev/greynet/2.5.htm
1999
Today
?
?
11

Research Questions (1)
The Problem
Based on the WI, can we use content- and link structure based methods to (re-)discover missing web pages in real time?
Investigated Methods:
Lexical signatures
Titles
Tags
Link neighborhood lexical signatures
12

Research Questions (2)
The Problem
What are the optimal characteristics of these methods (age, length, etc) with respect to retrieval performance?
Can we improve the performance by consolidating two or more methods?
Can we have a real-world implementation and evaluation of the above?
13

Agenda
LSs for Web Pages
TC-DF Correlation
Web Page Titles
Synchronicity
Book of the Dead
Web Page Tags
14
Motivation
Background

Memento, Web Infrastructure (WI)
15

Lexical Signatures (LSs)
First introduced by Phelps and Wilensky[Phelps2000]
Small set of terms capturing “aboutness” of a document, “lightweight” metadata
Resource
Abstract
10,000 terms
200 terms
16

Lexical Signature Generation
Following TF-IDF scheme first introduced by Spaerck Jones and Robertson [Jones1973]
Term frequency (TF):
“How often does this word appear in this document?”
Inverse document frequency (IDF):
“In how many documents does this word appear?”
17

Lexical Signatures -- Examples
18

Agenda
LSs for Web Pages
TC-DF Correlation
Web Page Titles
Synchronicity
Book of the Dead
Web Page Tags
19
A Comparison of Techniques for Estimating IDF Values to Generate LexicalSignatures for the Web(WIDM 2008)
Motivation
Background

Accurate IDF Values for LSs
Screen scraping the Google web interface
20

The Dataset
Local universe consisting of copies of URIs
from the Internet Archivebetween 1996 and 2007
21

The Idea
Use IDF values obtained from
Local collection of web pages
“screen scraping”SE result pages
Validate both methods against a baseline
Google N-Grams
Note: N-Grams provide term count (TC) and not DF values – ask me for details
22

LSs Example
Based on all 3 methods
URL: http://www.perfect10wines.com
Year: 2007
Union: 12 unique terms
23

Comparing LSs
Normalized term overlap
Assume term commutativity
k-term LSs normalized by k
Kendall Tau
Modified version since LSs to compare may contain different terms
M-Score
Penalizes discordance in higher ranks
24

Comparing LSs
Top 5, 10 and 15terms
LC – local universe
SC – screen scraping
NG – N-Grams
25

Conclusions
Both methods for the computation of IDF values provide accurate results
Compared to the Google N-Gram baseline
Screen scraping method seems preferable
Similarity scores are slightly higher
Feasible in real time!!!
Contribution:
Established well performing IDF estimation technique.
26

Agenda
LSs for Web Pages
TC-DF Correlation
Web Page Titles
Synchronicity
Book of the Dead
Web Page Tags
27
Revisiting Lexical Signatures to (Re-)Discover Web Pages(ECDL 2008)
Motivation
Background

The Idea
Evaluate Evolution of LSs over Time by
Generate LSs of URIs (from local universe mentioned above) over time
Conduct overlap analysis
Neither Phelps and Wilensky nor Park et al.[Park2004] did that
Park et al. just re-confirmed their findings after 6 months
28

LSs Over Time - Example
10-term LSs generated for
http://www.perfect10wines.com
29

LS Overlap Analysis
Rooted:
overlap between the LS of the year of the first observation in the IA and all LSs of the consecutive years that URI has been observed
Sliding:
overlap between two LSs of consecutive years starting with the first year and ending with the last
30

Evolution of LSs over Time
Rooted
Results:
Little overlap between the early years and more recent ones
Highest overlap in the first 1-2 years after creation of the LS
Rarely peaks after that – once terms are gone do not return
31

Evolution of LSs over Time
Sliding
Results:
Overlap increases over time
Seem to reach steady state around 2003
32

Performance of LSs
Idea:
Query LSs against Google search API
Identify URI in result set
For each URI it is possible that:
URI is returned as the top ranked result
URI is ranked somewhere between 2 and 10
URI is ranked somewhere between 11 and 100
URI is ranked somewhere beyond rank 100 considered as not returned
33

Performance of LSs wrt Length
Results:
2-, 3- and 4-term LSs perform poorly
5-, 6- and 7-term LSs seem best
Top mean rank (MR) value with 5 terms
Most top ranked with 7 terms
Binary pattern: either in top 10 or undiscovered
8 terms and beyond do not show improvement
34

Performance of LSs wrt Length
nDCG for LSs consisting of 2-15 terms
(mean over all years)
35

Performance of LSs over Time
nDCG for LSs consisting of 2, 5, 7 and 10 terms
36

Conclusions
LSs decay over time
Rooted: quickly after generation
Sliding: seem to stabilize
LSs older than 5 years perform poorly
5-, 6- and 7-term LSs seem to perform best
7 – most top ranked
5 – lowest mean rank
2..4 as well as 8+ term LSs are insufficient
Contribution:
Determined age and length limits for LSs.
37

Agenda
LSs for Web Pages
TC-DF Correlation
Web Page Titles
Synchronicity
Book of the Dead
Web Page Tags
38
Evaluating Methods to Rediscover Missing Web Pages from the Web Infrastructure(JCDL 2010)
Motivation
Background

59 copies
The Problem
The Problem
Internet Archive - Wayback Machine
www.aircharter-international.com
http://web.archive.org/web/*/http://www.aircharter-international.com
Lexical Signature
(TF/IDF)
Charter Aircraft Cargo Passenger Jet Air Enquiry
Title
ACMI, Private Jet Charter, Private Jet Lease, Charter Flight Service: Air Charter International
39

The Problem
The Problem
Lexical Signature
(TF/IDF)
Charter Aircraft Cargo Passenger Jet Air Enquiry
40

The Problem
The Problem
Title
ACMI, Private Jet Charter, Private Jet Lease, Charter Flight Service: Air Charter International
41

The Idea
Contributions
Compare performance of two automated methods to rediscover web pages
Lexical signatures (LSs)
Titles
Evaluate performance of combination of methods and suggest workflow for real time web page rediscovery
42

LS Retrieval Performance
LS Retrieval Performance
5- and 7-Term LSs
Yahoo! returns most URIs top ranked and leaves least undiscovered
Binary retrieval pattern, URI either within top 10 or undiscovered
43

Title Retrieval Performance
Title Retrieval Performance
Non-Quoted and Quoted Titles
Results at least as good as for LSs
Google and Yahoo! return more URIs for non-quoted titles
Same binary retrieval pattern
44

Combination of Methods
Combination of Methods
Top Results for Combination of Methods
45

Conclusions
Concluding Remarks
LSs and titles are suitable as search engine queries
Return 50%-70% URIs top ranked
BUT
Titles are cheaper to obtain, hence
Preferred primary method
5-term LSs secondary method
Results in 75% top ranked URIs
Contributions:
Provided evidence for suitability of titles and introduced web page discovery framework.
46

Agenda
LSs for Web Pages
TC-DF Correlation
Web Page Titles
Synchronicity
Book of the Dead
Web Page Tags
47
Is This a Good Title?(Hypertext 2010)
Motivation
Background

???
The Problem
The Problem
http://www.drbartell.com/
Lexical Signature
(TF/IDF)
Plastic Surgeon Reconstructive Dr Bartell Symbol University
48

The Problem
The Problem
http://www.drbartell.com/
Title
Thomas Bartell MD Board-Certified - Cosmetic Plastic Reconstructive Surgery
49

The Problem
The Problem
www.reagan.navy.mil
Lexical Signature
(TF/IDF)
Ronald USS MCSN Torrey Naval Sea Commanding
50

The Problem
The Problem
www.reagan.navy.mil
???
Title
Home Page
Is This a Good Title?
51

The Idea
Contributions
Display title evolution over time
Compare to content evolution
“Normalize” time as fixed size windows
Provide prediction model for title’s retrieval potential
52

Title and LS Retrieval Performance
Title (and LS) Retrieval Performance
Titles
5- and 7-Term LSs
Titles return more than 60% URIs top ranked
Binary retrieval pattern, URI either within top 10 or undiscovered
53

Title Evolution – Example I
Title Evolution - Example I
www.sun.com/solutions
1998-01-27
Sun Software Products Selector Guides - Solutions Tree
1999-02-20
Sun Software Solutions
2002-02-01
Sun Microsystems Products
2002-06-01
Sun Microsystems - Business & Industry Solutions
2003-08-01
Sun Microsystems - Industry & Infrastructure Solutions Sun Solutions
2004-02-02
Sun Microsystems – Solutions
2004-06-10
Gateway Page - Sun Solutions
2006-01-09
Sun Microsystems Solutions & Services
2007-01-03
Services & Solutions
2007-02-07
Sun Services & Solutions
2008-01-19
Sun Solutions
54

Title Evolution – Example II
Title Evolution - Example II
www.datacity.com/mainf.html
2002-10-16
computer company in Manassas Virginia sells Custom Built Computers with Removable Hard Drives Kits and Iomega 2GB Jaz Drives (jazz drives) October 2002 DataCity 800-326-5051 toll free
2006-03-14
Est1989 Computer company in Stafford Virginia sells Custom Built Secure Computers with DoD 5200.1-R Approved Removable Hard Drives, Hard Drive Kits and Iomega 2GB Jaz Drives (jazz drives), introduces the IllumiNite; lighted keyboard DataCity 800-326-5051 Service Disabled Veteran Owned Business SDVOB
2000-06-19
DataCityof Manassas Park Main Page
2000-10-12
DataCityof Manassas Park sells Custom Built Computers & Removable Hard Drives
2001-08-21
DataCitya computer company in Manassas Park sells Custom Built Computers & Removable Hard Drives
55

Title Evolution Over Time
How much do titles change over time?
Copies from fixed size time windows per year
Extract available titles of past 14 years
Compute normalized Levenshtein edit distance between titles of copies and baseline (today)(0 = identical;1 = completely dissimilar)
56

Title edit distance frequencies
Half the titles of available copies from recent years are (close to) identical
Decay from 2005 on (with fewer copies available)
4 year old title:40% chance to be unchanged
57

[0,0] - 122 times
[0,1] - over 1600 times
Title vs Document
Y: avg shingle value for all copies per URI
X: avg edit distance of corresponding titles
overlap indicated by:green: <10red: >90
Semi-transparent: total amount of points plotted
58

Title Performance Prediction
Title Performance Prediction
Quality prediction of title by
Number of nouns, articles etc.
Amount of title terms, characters [Ntoulas2006]
Observation of re-occurring terms in poorly performing titles - “Stop Titles”
home, index, home page, welcome, untitled document
The performance of any given title can be predicted as insufficient if it consists to 75% or more of a “Stop Title”!
59

Conclusions
Concluding Remarks
Titles change more slowly and less significantly over time than web page content
Not all titles equally good
If the majority of title terms are Stop Titles its quality can be predicted poor
Contribution:
Quantified title evolution and introduced stop titles.
60

Agenda
LSs for Web Pages
TC-DF Correlation
Web Page Titles
Synchronicity
Book of the Dead
Web Page Tags
61
Motivation
Background
Find, New, Copy, Web, Page - Tagging for the (Re-)Discovery of Web Pages(TPDL 2011)

The Problem
The Problem
We have seen that we have a good chance to rediscover missing pages with
Lexical signatures
Titles
BUT
What if no archived/cached copy can be found?
62

The Solution?
The Problem
Conferences
Digitallibraries
Conference
Library
Jcdl2005
63

The Idea
The Problem
Experimental evaluation of tag based query length cf. 5- or 7-term LSs
Test combination of methods to improve retrieval performance
Investigate “descriptive” power of tags
64

The Experiment
The Problem
Tags queried against the Yahoo! BOSS API
Same four retrieval cases introduced earlier
nDCG w/ binary relevance scoring
Mean Average Precision
65

The Experiment
The Problem
Combining methods
66

The Experiment
The Problem
Fact:
~50% of tags do not occur in page [Bischoff2008]
“Secret”:
~50% of tags do not occur in current version of page
ergo: How about previous versions?
67

Ghost Tags
The Problem
3,306 URIs w/ older copies
66.3% of our tags do not occur in page
4.9% of tags occur in previous version of page Ghost Tags
represent a previous version better than the current one
What kind of tags are these?
Important to the document, to the Delicious user?
68

Ghost Tags
The Problem
Document importance:
TF rank
User importance:
Delicious rank
Normalized rank:
0 - top
1 - bottom
69

Conclusions
Concluding Remarks
Tags can be used for search (if available)
Combining tags with titles and LSs gains URIs
Ghost Tags exist!
1/3 of them are important to the page and user
Contributions:
Added tags to web page discovery framework and introduced notion of Ghost Tags.
70

Agenda
LSs for Web Pages
TC-DF Correlation
Web Page Titles
Synchronicity
Book of the Dead
Web Page Tags
71
Motivation
Background
Rediscovering Missing Web Pages Using Link Neighborhood Lexical Signatures(JCDL 2011)

The Problem
The Problem
We have seen that we have a good chance to rediscover missing pages with
Lexical signatures
Titles
BUT
What if no archived/cached copy can be found?
Plan A: Tags
72

Plan B
The Problem
Link neighborhood Lexical Signatures (LNLSs)
is about
Computer
Dominion
Norfolk
Monarch
extract
73

The Idea
The Problem
Determine for well performing LNLS:
Length
Number of backlinks
Backlink levels
Radius of terms on backlink page
74

The Radius on a Backlink Page
The Problem
Entire page
Paragraph
Anchor text
75

The Dataset
309 URIs
28,325 first level
306,700 second level backlinks
Filter for language, file type, etc.
 12% discarded
Lexical signature generation
IDF values from Yahoo!
1..7 and 10 terms
Query Yahoo! API
Compute “goodness” (nDCG)
76

The Results
The Problem
1st and 2nd
level
level-radius-rank
better
77

The Results – Radius
The Problem
All Radii
level-radius-rank
78

The Results – Backlink Rank
The Problem
Ranks
10
100
1000
level-radius-rank
79

The Results – In Numbers
The Problem
GOOD
1-anchor-1000
WINNER
1-anchor-10
80

Conclusions
Concluding Remarks
Optimal link neighborhood lexical signatures:
Contain 4 terms
Parsed from top 10backlink pages
Include firstbacklink level only
Consider anchor text only
Contributions:
Added LNLS to web page discovery framework.
81

Agenda
LSs for Web Pages
TC-DF Correlation
Web Page Titles
Synchronicity
Book of the Dead
Web Page Tags
82
Motivation
Background
Synchronicity – Automatically Rediscover Missing Web Pages in Real Time
(JCDL 2011)

Synchronicity
Concluding Remarks
Firefox add-on
Triggers on 404 error
Rediscover page via:
Memento
Title
Lexical signature
Tags
Link neighborhood lexical signature
URI modification
http://bit.ly/no-more-404
83

Contributions
Concluding Remarks
Introduce reliable real-time approach to estimate IDF values
Workflow for generation of well performing lexical signatures
Performance evaluation of web page titles
Investigation of tags for web page discovery
Analysis of link neighborhood lexical signatures and their optimal parameter
Introduce Synchronicity implementing the entire framework
84

Concluding Remarks
85

Next Stop… New Mexico
Concluding Remarks
86

List of my Relevant Publications
Concluding Remarks
M.Klein, M.L.Nelson, “A Comparison of Techniques for Estimating IDF Values to Generate Lexical Signatures for the Web“, WIDM 2008, pp. 39-46
M.Klein, M.L.Nelson, “Revisiting Lexical Signatures to (Re-)Discover Web Pages”, ECDL 2008, pp. 371-382
M.Klein, M.L.Nelson, “Correlation of Term Count and Document Frequency for Google N-Grams“, ECIR 2009, pp. 620-627
M.Klein, M.L.Nelson, “Inter-Search Engine Lexical Signature Performance“, JCDL 2009, pp. 413-414
M.Klein, M.L.Nelson, “Investigating the Change of Web Pages Titles Over Time“, InDP 2009
M.Klein, J.Shipman, M.L.Nelson, “Is This a Good Title”, Hypertext 2010, pp. 3-12
M.Klein, M.L.Nelson, “Evaluating Methods to Rediscover Missing Web Pages from the Web Infrastructure”, JCDL 2010, pp. 59-68
M.Klein, J.Ware, M.L.Nelson, “Rediscovering Missing Web Pages Using Link Neighborhood Lexical Signatures”, JCDL 2011
M.Klein, M.Aly, M.L.Nelson, “Synchronicity - Automatically Rediscover Missing Web Pages in Real Time”, JCDL 2011
M.Klein, M.L.Nelson, “Find, New, Copy, Web, Page – Tagging for the (Re-)Discovery of Web Pages”, TPDL 2011 to appear
87

References
Concluding Remarks
Bischoff2008
K.Bischoff, C.Firan, W.Nejdl, R.Paiu, “Can All Tags Be Used for Search?” In: Proceedings of CIKM '08, pp.193-202, 2008
Dellavalle2003
R.P.Dellavalle, E.J.Hester, L.F.Heilig, A.L.Drake, J.W.Kuntzman, M.Graber, L.M.Schilling, “Information Science: Going, Going, Gone: Lost Internet References”, Science 302(5646), pp.787-788, 2003
Jones1973
K.Spärck Jones, “Index Term Weighting”, Information Storage and Retrieval, pp. 619-633, 1973
Kahle1997
B.Kahle, “Preserving the Internet”, Scientific American 276, pp.82-83, 1997
Koehler2002
W.C.Koehler, “Web Page Change and Persistence - A Four-Year Longitudinal Study”, JASIST 53(2), pp.162-171, 2002
Lawrence2001
S.Lawrence, D.M.Pennock, G.W.Flake, R.Krovetz, F.M.Coetzee, E.Glover, F.A.Nielsen, A.Kruger, C.L.Giles, “Persistence of Web References in Scientic Research”, Computer 34(2), pp.26-31, 2001
McCown2005
F.McCown, S.Chan, M.L.Nelson, J.Bollen, “The Availability and Persistence of Web References in D-Lib Magazine”, Proceedings of IWAW '05, 2005
Nelson2002
M.L.Nelson, B.D.Allen, “Object Persistence and Availability in Digital Libraries”, D-Lib Magazine 8(1), 2002
Ntoulas2006
A. Ntoulas, M.Najork, M.Manasse, D.Fetterly, “Detecting Spam Web Pages Through Content Analysis”, Proceedings of WWW ’06, pp 83-92, 2006
Park2004
S.T.Park, D.M.Pennock, C.L.Giles, R.Krovetz, “Analysis of Lexical Signatures for Improving Information Persistence on the World Wide Web”, TOIS 22(4), pp.540-572, 2004
Phelps2000
T.A.Phelps, R.Wilensky, “Robust Hyperlinks Cost Just Five Words Each”, technical report, UC Berkeley, 2000
Sanderson2011
R.Sanderson, M.Phillips, H.Van de Sompel, “Analyzing the Persistence of Referenced Web Resources with Memento”, Proceedings of OR '11, 2011
88

Using the Web Infrastructure
for Real Time Recovery
of Missing Web Pages
Martin Klein
mklein@cs.odu.edu
http://www.cs.odu.edu/~mklein/

Backup Slides

Future Work
91
“Story Telling” with Memento
Find more Stop Titles
Find more Ghost Tags
Identify “Stop Anchors”
Synchronicity 1.0
Web service
CMD line tool

Agenda
LSs for Web Pages
TC-DF Correlation
Web Page Titles
Synchronicity
Book of the Dead
Web Page Tags
92
Correlation of Term Count and Document Frequency for Google N-Grams(ECIR 2009)
Motivation
Background

The Problem
Need of a reliable source to accurately compute IDF values of web pages (in real time)
Shown, screen scraping works but
missing validation of baseline (Google N-Grams)
N-Grams seem suitable (recently created, based on web pages) but provide TC and not DF  what is their relationship?
93

94
Background
Google N-grams provide term count (TC) values
D1 = “Please, Please Me” D2 = “Can’t Buy Me Love”
D3 = “All You Need Is Love” D4 = “Long, Long, Long”
TC >= DF, but is there a correlation?
Can we use TC to estimate DF?

95
Experiment Results
Investigate correlation between TC and DF
within “Web as Corpus” (WaC)
Rank similarity of all terms

96
Experiment Results
Investigate correlation between TC and DF
within “Web as Corpus” (WaC)
Spearman’s ρ and Kendall τ

97
Experiment Results
Top 10 terms in decreasing order of their TF/IDF values
taken from http://ecir09.irit.fr
U = 14
∩ = 6
Strong indicator that TC can be used to estimate DF for web pages!
Google: screen scraping DF values from the Google web interface

98
Experiment Results
Show similarity between WaC based TC and
Google N-Gram based TC
TC frequencies
N-Grams have a threshold of 200

Experiment Results
Frequency of TC/DF Ratio Within the WaC
Integer Values
Two Decimals
One Decimal
99

Conclusions
TC and DF Ranks within the WaC show strong correlation
TC frequencies of WaC and Google N-Grams are very similiar
N-Grams are suitable for accurate IDF estimation for web pages
 Does not mean everything correlated to TC can be used as DF substitute!
100

Agenda
LSs for Web Pages
TC-DF Correlation
Web Page Titles
Synchronicity
Book of the Dead
Web Page Tags
101
Inter-Search Engine Lexical Signature Performance
(JCDL 2009)
Motivation
Background

Inter-Search EngineLexical Signature Performance
http://en.wikipedia.org/wiki/Elephant
Elephant
Tusks
Trunk
African
Loxodonta
Elephant, Asian, African
Species, Trunk
Elephant, African, Tusks
Asian, Trunk

Agenda
LSs for Web Pages
TC-DF Correlation
Web Page Titles
Synchronicity
Book of the Dead
Web Page Tags
104
Motivation
Background
Synchronicity – Automatically Rediscover Missing Web Pages in Real Time
(JCDL 2011)

Synchro…What?
Synchronicity
Experience of causally unrelated events occurring together in a meaningful manner
Events reveal underlying pattern, framework bigger than any of the synchronous systems
Carl Gustav Jung (1875-1961)
“meaningful coincidence”
Deschamps – de Fontgibu plumpudding example
picture from http://www.crystalinks.com/jung.html
105

Synchro…What?
Repo Man (1984)
http://www.imdb.com/title/tt0087995/
http://www.youtube.com/watch?v=X4HQyqc-aVU
106

Agenda
LSs for Web Pages
TC-DF Correlation
Web Page Titles
Synchronicity
Book of the Dead
Web Page Tags
107
Motivation
Background
(Not yet published)

Book of the Dead
Corpus of missing web pages
233 URIs returning status 404
Mechanical Turk to determine “aboutness”
Guess from URI string
Mementos for 161 URIs
Apply lexical signatures and title
108

5-term LSs
Titles
109
Experiment Results
Dice Similarity Coefficient
of Top 100 Results
D = 0
0.0 < D ≤ 0.3
0.3 < D ≤ 0.6
0.6 < D ≤ 1.0

5-term LSs
Titles
110
Experiment Results
Jaro Distance
of Top 100 Results
J = 0
0.0 < J ≤ 0.3
0.3 < J ≤ 0.6
0.6 < J ≤ 1.0

Book of the Dead
Mechanical Turk to determine relevance of results
Top 10 only
Relevant
Somewhat relevant
Not relevant
Broken URI
nDCG of top 10 results
111

5-term LSs
Titles
112
Experiment Results
Relevance of Top 10 Results

113
Experiment Results
nDCG of Top 10 Results

http://ws-dl.blogspot.com	272
http://www.blogger.com	2
http://ws-dl.blogspot.in	1
http://ws-dl.blogspot.fr	1
http://ws-dl.blogspot.co.uk	1

Dissertation Defense

by Martin Klein on Sep 13, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

5 Embeds 277

Statistics

Dissertation Defense — Presentation Transcript