Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Reference Rot and !
Link Decoration!
Martin Klein!
UCLA
martinklein0815@gmail.com
@mart1nkle1n
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
Hiberlink Team
• Los Alamos National Laborat...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
3
http://journals.plos.org/plosone/article?i...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
4
Reference Rot
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
5
Link Rot
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
6
“Entertaining” Link Rot
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
7
Ubiquitous Link Rot
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
8
Content Drift
http://dl00.org!
!
2000
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
9
Content Drift
http://dl00.org!
!
2004
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
10
Content Drift
http://dl00.org!
!
2005
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
11
Content Drift
http://dl00.org!
!
2008
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
12
NYT Coverage
Links in!
Supreme Court deci...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
13
Scholarly Communication
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
14
!Exist
!Exist
!Exist
Exist
Exist
Archived...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
Entrance Hiberlink
• These resources:
• Are ...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
16
Quantifying!
Reference Rot
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
Our Study
• Time frame of publications: Jan ...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
18
1997 1999 2001 2003 2005 2007 2009 2011
0...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
19
Link Rot in arXiv
1997 1999 2001 2003 200...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
20
1997 1999 2001 2003 2005 2007 2009 2011
1...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
21
Content Drift / Archival Status
Not Archi...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
22
PMC
Elsevier
arXiv
Not Archived
75.3%
Arc...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
23
Loss of Context
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
24
Loss of Context
all links active links
li...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
STM Article Extrapolation
25
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
STM Article Extrapolation
• Immune: article ...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
27
Immune vs not Immune STM Articles
0
10
20...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
STM Article Extrapolation
• Immune: article ...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
29
0
10
20
30
40
50
60
70
80
90
100
1997 199...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
30
An approach to solve !
Reference Rot
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
Robust Links
1.Create snapshot of linked res...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
Robust Links
1. Create snapshot of linked re...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
Page Creation Date
33
<!DOCTYPE html>
<html>...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
34
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
Robust Links
1. Create snapshot of linked re...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
Link Decoration
36
<a href="http://hiberlink...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
Link Decoration
37
<a href="http://hiberlink...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
38
http://robustlinks.mementoweb.org/demo/ur...
Reference Rot and Link Decoration!
@mart1nkle1n!
OAI9, Geneva, June 17th 2015
39
http://robustlinks.mementoweb.org/demo/ur...
Reference Rot and !
Link Decoration!
Martin Klein!
UCLA
martinklein0815@gmail.com
@mart1nkle1n
Upcoming SlideShare
Loading in …5
×

Reference Rot and Link Decoration

1,386 views

Published on

Reference Rot and Link Decoration
Presentation given at OAI9 based on "Scholarly Context Not Found: One in Five Articles Suffers from Reference Rot"
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0115253

Published in: Internet
  • Be the first to comment

  • Be the first to like this

Reference Rot and Link Decoration

  1. 1. Reference Rot and ! Link Decoration! Martin Klein! UCLA martinklein0815@gmail.com @mart1nkle1n
  2. 2. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 Hiberlink Team • Los Alamos National Laboratory • Research Library: (Martin Klein), (Robert Sanderson), Harihar Shankar, Herbert Van de Sompel! • University of Edinburgh • Edina: Peter Burnhill, Neil Mayo, Muriel Mewissen, Christine Rees, Tim Strickland, Richard Wincewicz • Language Technology Group: Beatrix Alex, Claire Grover, Colin Matheson, Richard Tobin, (Ke “Adam” Zhou) • Funding: Andrew W. Mellon Foundation 2
  3. 3. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 3 http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0115253
  4. 4. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 4 Reference Rot
  5. 5. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 5 Link Rot
  6. 6. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 6 “Entertaining” Link Rot
  7. 7. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 7 Ubiquitous Link Rot
  8. 8. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 8 Content Drift http://dl00.org! ! 2000
  9. 9. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 9 Content Drift http://dl00.org! ! 2004
  10. 10. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 10 Content Drift http://dl00.org! ! 2005
  11. 11. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 11 Content Drift http://dl00.org! ! 2008
  12. 12. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 12 NYT Coverage Links in! Supreme Court decisions:! ! • Link rot: 29%! ! • Reference rot: 49%
  13. 13. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 13 Scholarly Communication
  14. 14. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 14 !Exist !Exist !Exist Exist Exist Archived Archived !Archived Archived Archived
  15. 15. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 Entrance Hiberlink • These resources: • Are not necessarily under the custodianship of parties that care about long time integrity, access • Do not necessarily have the same sense of fixity like e.g., journal articles • Links to these resources are subject to Reference Rot: • Link Rot: Link stops working e.g., HTTP 404 • Content Drift: Linked content changes over time 15
  16. 16. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 16 Quantifying! Reference Rot
  17. 17. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 Our Study • Time frame of publications: Jan 1997 - Dec 2012 • Articles from arXiv, Elsevier, and PMC in XML and PDF format • Convert PDF to XML • Extract URIs to web at large resources • Store article’s publication date • URI live web test (trusted in 200 OK response) • URI archive lookup via Memento infrastructure 17 arXiv Elsevier PMC total articles 707, 667 2, 285, 000 595, 889 articles with HTTP references 142, 134 94, 645 156, 160 amount of HTTP references 346, 177 232, 712 480, 853
  18. 18. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 18 1997 1999 2001 2003 2005 2007 2009 2011 02000060000100000140000180000 articles URI references 1997 1999 2001 2003 2005 2007 2009 2011 050001500025000350004500055000 articles URI references 1997 1999 2001 2003 2005 2007 2009 2011 050000100000150000200000250000300000350000 articles URI references PMC Elsevier arXiv Our Corpora
  19. 19. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 19 Link Rot in arXiv 1997 1999 2001 2003 2005 2007 2009 2011 102030405060708090100 1000020000300004000050000 HTTP References Link Rot NumberofHTTPReferences LinkRotPercentage
  20. 20. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 20 1997 1999 2001 2003 2005 2007 2009 2011 102030405060708090100 1000020000300004000050000 HTTP References Link Rot NumberofHTTPReferences LinkRotPercentage 1997 1999 2001 2003 2005 2007 2009 2011 102030405060708090100 5000100001500020000250003000035000 HTTP References Link Rot NumberofHTTPReferences LinkRotPercentage 1997 1999 2001 2003 2005 2007 2009 2011 102030405060708090100 20000400006000080000100000120000 HTTP References Link Rot NumberofHTTPReferences LinkRotPercentage PMC Elsevier arXiv
  21. 21. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 21 Content Drift / Archival Status Not Archived 75.3% Archived 24.7% Rotten 26.0% Active 74.0% All Links • Archival status used as proxy • Availability of archived copy created within N days of article’s publication • N = 14 arXiv
  22. 22. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 22 PMC Elsevier arXiv Not Archived 75.3% Archived 24.7% Rotten 26.0% Active 74.0% All Links Not Archived 75.2% Archived 24.8% Rotten 32.7% Active 67.3% All Links Not Archived 74.5% Archived 25.5% Rotten 20.0% Active 80.0% All Links
  23. 23. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 23 Loss of Context
  24. 24. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 24 Loss of Context all links active links links archived! (14 days)
  25. 25. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 STM Article Extrapolation 25
  26. 26. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 STM Article Extrapolation • Immune: article contains no URIs to web at large resources • Healthy: none of the URIs to web at large resources suffer from link rot nor content drift • infected: at least one URI to web at large resources suffers from link rot or content drift 26
  27. 27. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 27 Immune vs not Immune STM Articles 0 10 20 30 40 50 60 70 80 90 100 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 Immune not Immune
  28. 28. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 STM Article Extrapolation • Immune: article contains no URIs to web at large resources • Healthy: none of the URIs to web at large resources suffer from reference rot • Infected: at least one URI to web at large resources suffers from reference rot 28
  29. 29. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 29 0 10 20 30 40 50 60 70 80 90 100 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 Immune Healthy Infected 1/5 articles suffers ! from ! Reference Rot! Immune, Healthy, Infected STM Articles
  30. 30. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 30 An approach to solve ! Reference Rot
  31. 31. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 Robust Links 1.Create snapshot of linked resources in a web archive when: • drafting work • submitting article • publishing article • aggregating article 31
  32. 32. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 Robust Links 1. Create snapshot of linked resources in a web archive 2. Convey creation date of your web page in machine-actionable manner 32
  33. 33. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 Page Creation Date 33 <!DOCTYPE html> <html> <head> <title> … </title> <meta itemprop="datePublished" content="2015-02-18" /> … </head> … </html>
  34. 34. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 34
  35. 35. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 Robust Links 1. Create snapshot of linked resources in a web archive 2. Convey creation date of your web page in machine- actionable manner 3. Decorate links with datetime of linking and URI of archived snapshot, in addition to resource’s original URI 35 http://robustlinks.mementoweb.org/spec/
  36. 36. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 Link Decoration 36 <a href="http://hiberlink.org/">http://hiberlink.org/</a>
  37. 37. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 Link Decoration 37 <a href="http://hiberlink.org/" ! data-versionurl="http://archive.is/Bvq2v" data-versiondate=“2014-11-01"> ! http://hiberlink.org/</a>
  38. 38. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 38 http://robustlinks.mementoweb.org/demo/uri_references_js.html
  39. 39. Reference Rot and Link Decoration! @mart1nkle1n! OAI9, Geneva, June 17th 2015 39 http://robustlinks.mementoweb.org/demo/uri_references_js.html
  40. 40. Reference Rot and ! Link Decoration! Martin Klein! UCLA martinklein0815@gmail.com @mart1nkle1n

×