SlideShare a Scribd company logo

Quantifying Orphaned Annotations in Hypothes.is

M
maturban

Web annotation has been receiving increased attention recently with the organization of the Open Annotation Collaboration and new tools for open annotation, such as Hypothes.is. In this paper, we investigate the prevalence of orphaned annotations, where a live Web page no longer contains the text that had previously been annotated in the Hypothes.is annotation system (containing 20,953 highlighted text annotations).

1 of 43
Download to read offline
Quantifying Orphaned
Annotations in Hypothes.is
Mohamed Aturban, Michael L. Nelson, and Michele C. Weigle
Department of Computer Science,
Old Dominion University,
Norfolk, VA 23529
1
TPDL 2015
Poznan, Poland
September 13-17, 2015
What is Web Annotation?
2
Handwritten Annotations Web Annotations
Haslhofer, B., Simon, R., Sanderson, R., Van de Sompel, H.: The Open Annotation Collaboration (OAC) model. In: Proceedings of the IEEE Workshop on Multimedia on the Web (MMWeb). pp. 5–9. IEEE (2011)
http://networkedlearningcollaborative.com/wp-content/uploads/2015/07/53e12bf10cf2d79877a53311.pdf
• Open Annotation Collaboration (OAC)
group defines an annotation as a set of
connected resources
What is Web Annotation?
3
• Open Annotation Collaboration (OAC)
group defines an annotation as a set of
connected resources
What is Web Annotation?
4
Why is Web Annotation Important?
• A collaborative tool
• Social criticism
• Education for students and teachers
• Scholarly and academic purposes
• Editing and publishing
Web annotations do not modify the original resource
5
Annotating Web Resources Using
Hypothes.is
6
Highlights
Notes
Replies
Tags
Target URI
The Annotation is Attached to the Live Web
7
http://www.wired.co.uk/news/archive/2014-02/06/tim-berners-lee-reclaim-the-
web/viewgallery/332234
August 2015 (from live web)
https://hypothes.is/a/tp5hnn4PTjuSg7h
bc7_cxQ
Annotation made in February 2014
Tags
A Review of Annotation Attachment to
the Live & Archived Web
8
Does an annotation attach to its
target’s live webpage?
Does an annotation attach to any
archived copies (Mementos) of the
target webpage
1 YES YES
2 YES NO
3 NO YES
4 NO NO
The Annotation is Attached to the Live Web
and to an Archived Copy (Memento) of the
Webpage
9
https://web.archive.org/web/20140207083733/http://
www.wired.co.uk/news/archive/2014-02/06/tim-
berners-lee-reclaim-the-web/viewgallery/332234
February 2014 (an archived copy)
http://www.wired.co.uk/news/archive/2014-
02/06/tim-berners-lee-reclaim-the-
web/viewgallery/332234
August 2015 (from live web)
https://hypothes.is/a/tp5hnn4PT
juSg7hbc7_cxQ
Annotation made in
February 2014
The Annotation is Attached to the Live Web but
No Mementos Are Available
10
http://tkbr.ccsp.sfu.ca/pub802/2015/01/more-
horsepower-to-wattpad/
August 2015 (from live web)
https://hypothes.is/a/o_2W8QwZR
Dm8w0F1dqRxUQ and
https://hypothes.is/a/NE8AT6R3Tn
6Qg6FeYy1C0w
Annotation made in
February 2015
The Annotation is
in Danger of being
Orphaned
The Annotation is Not Attached to the Live
Web but It is Attached to Mementos
11
https://web.archive.org/web/201412101210
18/http://climatefeedback.org/
December 2014 (an archived copy)
climatefeedback.org
Annotation made in December
2014
climatefeedback.org
August 2015 (from live web)
The Annotation is Not Attached to the Live
Web and No Mementos Are Available
12
http://renaissancejohnson.weebly.com/spensers-
wordcloud.html
August 2015 (from live web)
https://hypothes.is/a/wFLZKLGqS8
S3Zyfr_Rmu4w
Annotation in July 2015
The Annotation is
Orphaned
Four Different Cases For Annotation
Attachment
Does an annotation attach to its
target’s live webpage?
Does an annotation attach to any
archived copies (Mementos) of the
target webpage
1 YES YES
2 YES NO
3 NO YES
4 NO NO
13
In Danger of being Orphaned
Orphaned
Safe
Can Reattach to Memento
We Studied Hypothes.is
Annotations
•How many annotations are orphaned?
•How many annotations are in danger of
being orphaned?
•How many annotations can be reattached
to mementos in public web archives?
14
Related Work
• OAC introduced the idea to make annotations reusable
through different systems
• Annotations, as web resources, have unique URIs
• Sanderson and Van de Sompel introduced a framework
to make web annotations persistent over time
• Integrating features in the Open Annotation Data Model with
the Memento framework
• Reconstructing annotations for a given memento
• Retrieving mementos for a given annotation
• Kreymer's Browsertrix provides on-demand web
archiving
• Whenever an annotation is created, a copy of the related
webpage could be archived automatically
• Funded by Hypothes.is
15
Sanderson, R., Van de Sompel, H.: Making web annotations persistent over time. In: Proceedings of the 10th ACM/IEEE Joint Conference on Digital Libraries
(JCDL). pp. 1–10. ACM (2010)
http://blog.webrecorder.io/2015/06/open-annotation-fund-project.html
https://hypothes.is/blog/fund-on-demand-web-archiving-completion/
Annotations in Hypothes.is Are
Increasing
16
January 2015
(7744)
August 2015
( 33,946 )
January 2015 - dataset used in TPDL 2015 paper
August 2015 - dataset presented here and in arXiv version
Annotation Types in Hypothes.is
17
Highlighted
Text
Note Tags Number of
Annotations
 11,289
  9858
   9252
  1835
  1356
 348
 8
33,953
Annotation Types in Hypothes.is
18
Highlighted
Text
Note Tags Number of
Annotations
 11,289
  9858
   9252
  1835
  1356
 348
 8
33,953
We studied
20,953
annotations
that contain
highlighted
text
Several Academic Sites Use
Hypothes.is Widely
19
Number of Annotations
Contain Highlighted
Text
Host
1222 caseyboyle.net
1191 www.perseus.tufts.edu
887 rhetoric.eserver.org
875 networkedlearningcollaborative.com
749 sosol.perseids.org
733 tkbr.ccsp.sfu.ca
526 shakespeare.mit.edu
391 hypothes.is
356 renaissancejohnson.weebly.com
336 moodle2.wesleyan.edu
We Issued HTTP Head Requests for
Target URIs of All Annotations
20
Number of
Annotations
Status Code Example
18,167 200 OK http://www.w3.org/Talks/9704WWW6-tbl/slide16.htm
820 Unresolvable URIs file:///Users/peggy/Desktop/CIRCLE-youthvoting-individualPages.pdf
778 Timeout http://testbelfastgroup.digitalscholarship.emory.edu/
666 404 https://www.facebook.com/manunymous
190 Soft 4XX http://www.transmography.net/braineryworkshop/camperforce-by-joseph/
87 401 http://wiki.shuttleworthfoundation.org/~shuttlew/wiki/index.php?title=Nov_2
013_DW_Dogfood_prep
80 503 https://via.hypothes.is/http://b.pagekite.me/blog/2015-04-
27_Roadmap_to_v1.html
68 403 http://onlinelibrary.wiley.com/store/10.1002/2013EF000191/asset/eft214.pdf?
v=1&t=hppouayu&s=e3a980c9e2c6317987306d4e1d76c690c29fe758
48 410 https://www.scribd.com/word/removal/31126999
21 406 https://www.scribd.com/deleted/95457320
19 500 http://androidfrat.com/2015/01/the-new-usb-announcement-just-killed-the-
usb-super-position/
9 400, 416,
504, 520
https://ibpublishing.ibo.org/live-exist/rest/app/pub.xql?doc=EX_Instructions_
2013_e&part=10&chapter=4&page=1
Out of 33,946 Annotations,
We Investigated 20,133 Target URIs
21
{
"updated": "2014-02-10T22:51:03.920650+00:00",
"target": [
{
"source": "http://www.wired.co.uk/news/archive/2014-02/06/tim-berners-
lee-reclaim-the-web",
"selector": [
{ "endContainer":
"/div[3]/form[1]/div[2]/article[1]/div[1]/div[1]/div[2]/div[1]/div[1]/p[1]",
"endOffset": 187,
"type": "RangeSelector",
"startOffset": 0,
"startContainer":
“/div[3]/form[1]/div[2]/article[1]/div[1]/div[1]/div[2]/div[1]/div[1]/p[1]"
},
{ "exact": "Twenty-five years on from the web's inception, its creator has
urged the public to re-engage with its original design: a
decentralised internet that at its very core, remains open to all.",
"prefix": "anChris Woods / chrismwoods.com",
"type": "TextQuoteSelector",
"suffix": "Speaking with Wired editor Davi"
},
{ "start": 307,
"end": 494,
"type": "TextPositionSelector"
}]}],
"tags": ["w3", "re-decentrilization" ],
"text": "",
"created": "2014-02-10T22:51:03.920636+00:00",
"uri": "http://www.wired.co.uk/news/archive/2014-02/06/tim-berners-lee-
reclaim-the-web",
"user": "acct:aculich@hypothes.is",
"consumer": "00000000-0000-0000-0000-000000000000",
"id": "tp5hnn4PTjuSg7hbc7_cxQ",
"permissions": {
"admin": [ "acct:aculich@hypothes.is“ ],
…
An Annotation: JSON (L) and Visualized (R)
22
http://www.wired.co.uk/news/archive/2014-02/06/tim-berners-lee-
reclaim-the-web/viewgallery/332234
August 2015
• Compare an annotation’s highlighted text with a
live webpage’s content.
• Download annotation JSON from
Hypothes.is
• Extract the text from the annotation’s target
URI
• If the annotation’s highlighted text is found
in the webpage the annotation is
attached to the live web
23
Methodology
24
Only 78% of Highlighted Text
Annotations Attach to the Live Web
Discovering Mementos for All
Resolvable Target URIs
• Using LANL Memento Aggregator
• Considering only mementos with datetime
immediately before or after the annotation’s creation
date
• Four cases regarding the availability of mementos for
the annotation’s target URI:
• Mementos exist before and after the annotation’s creation
date
• Mementos exist only before the annotation’s creation date
• Mementos exist only after annotation’s creation date
• No mementos exist
25
Mementos Exist Before and After the
Annotation Creation Date
26
25% (4986) of resolvable target URIs
27
Mementos Exist Only Before the
Annotation Creation Date
12% (2477) of resolvable target URIs
28
7% (1397) of resolvable target URIs
Mementos Exist Only After the
Annotation Creation Date
29
No Mementos Exist
56% (11273) of resolvable target URIs
Are Annotations Attached to
Existing Mementos?
• Similar to checking if an annotation is attached to the
live web
• If the highlighted text is found in a memento the
annotation is attached and could be recovered.
• 8860 annotations have a target URI with at least one
memento
• Of these, 90% (7963) can be attached to a memento
30
Annotation Targets with Existing Mementos
Before and After the Annotation Creation Date
31
Attached to Live
Web Page
Attached to
Memento (Before)
Attached to
Memento (After)
Number of
Annotations
Yes Yes Yes 4091
Yes Yes No 93
Yes No Yes 100
Yes No No 182
No Yes Yes 251
No Yes No 69
No No Yes 44
No No No 156
4986 (Total)
Attached to Live
Web Page
Attached to
Memento (Before)
Attached to
Memento (After)
Number of
Annotations
Yes Yes Yes 4091
Yes Yes No 93
Yes No Yes 100
Yes No No 182
No Yes Yes 251
No Yes No 69
No No Yes 44
No No No 156
4986 (Total)
Annotation Targets with Existing Mementos
Before and After the Annotation Creation Date
32
In Danger
Orphaned
Annotation Targets with Existing Mementos
Only Before the Annotation Creation Date
33
Attached to Live
Web Page
Attached to
Memento (Before)
Number of
Annotations
Yes Yes 1984
Yes No 235
No Yes 133
No No 125
2477 (Total)
Attached to Live
Web Page
Attached to
Memento (Before)
Number of
Annotations
Yes Yes 1984
Yes No 235
No Yes 133
No No 125
2477 (Total)
Annotation Targets with Existing Mementos
Only Before the Annotation Creation Date
34
In Danger
Orphaned
Attached to Live
Web Page
Attached to
Memento (After)
Number of
Annotations
Yes Yes 1148
Yes No 101
No Yes 50
No No 98
1397 (Total)
Annotation Targets with Existing Mementos
Only After the Annotation Creation Date
35
Attached to Live
Web Page
Attached to
Memento (After)
Number of
Annotations
Yes Yes 1148
Yes No 101
No Yes 50
No No 98
1397 (Total)
Annotation Targets with Existing Mementos
Only After the Annotation Creation Date
36
In Danger
Orphaned
Annotation Targets with
No Existing Mementos
37
Attached to
Live Web Page
Number of
Annotations
Yes 7839
No 3434
11,273 (Total)
Attached to
Live Web Page
Number of
Annotations
Yes 7839
No 3434
11,273 (Total)
Annotation Targets with
No Existing Mementos
38
In Danger
Orphaned
How Many Orphaned Annotations
Does Hypothes.is Have?
• 19% (3813) of annotations are orphaned
• 41% (8357) of annotations are in danger of
being orphaned
• In total, 60% (12,170) of annotations are either
orphaned or in danger of being orphaned
39
How Many Annotations Can Be
Reattached Using Web Archives?
• Archives could only save 3% (547) of annotations that
would otherwise be orphaned
• 37% (7416) of annotations are safe -- attached to the
live web and also attached to one or more
mementos.
40
Archives Used to Attach Annotations
to Mementos
41
Archive Attached to
Live Web
Not Attached to
Live Web
web.archive.org 6997 (94.3%) 455 (83.1%)
archive.is 679 (9.15%) 39 (7.12%)
wayback.archive-it.org 562 (7.57%) 47 (8.59%)
github.com 80 (1.07%) 21 (3.83%)
wayback.vefsafn.is 71 (0.95%) 53 (9.68%)
arxiv.org 18 (0.24%) 0
webarchive.loc.gov 3 (0.04%) 0
webarchive.org.uk 4 (0.05%) 0
webarchive.nationalarchives.gov.uk 2 (0.02%) 0
discordia.wikia.com 1 (0.01%) 0
Total 8417 (113.4%) 615 (112.32%)
A single
annotation
may reattach
to mementos
from multiple
archives
The Status of Current Hypothes.is
Annotations
42
Highlighted Text Annotations
with Resolvable Target URIs :
20,133
Conclusion
• We analyzed the attachment of 20,953 highlighted
text annotations in Hypothes.is.
• 60% of annotations are orphaned or in danger of
being orphaned
• 7963 mementos from 10 different web archives
could be used to keep the remaining 40% of
annotations safe.
43
Archiving webpages at the time of annotation is
important to avoid orphaned annotations.

Recommended

Interoperability for web based scholarship
Interoperability for web based scholarshipInteroperability for web based scholarship
Interoperability for web based scholarshipHerbert Van de Sompel
 
Signposting Overview (Version November 2017)
Signposting Overview (Version November 2017)Signposting Overview (Version November 2017)
Signposting Overview (Version November 2017)Herbert Van de Sompel
 
Signposting for Repositories
Signposting for RepositoriesSignposting for Repositories
Signposting for RepositoriesMartin Klein
 
Achieving Link Integrity for Managed Collections
Achieving Link Integrity for Managed CollectionsAchieving Link Integrity for Managed Collections
Achieving Link Integrity for Managed CollectionsHerbert Van de Sompel
 
Persistent Identifiers and the Web: The Need for an Unambiguous Mapping
Persistent Identifiers and the Web: The Need for an Unambiguous MappingPersistent Identifiers and the Web: The Need for an Unambiguous Mapping
Persistent Identifiers and the Web: The Need for an Unambiguous MappingHerbert Van de Sompel
 
The web is rotting and what to do about it
The web is rotting and what to do about itThe web is rotting and what to do about it
The web is rotting and what to do about itHerbert Van de Sompel
 

More Related Content

What's hot

Discovering Scholarly Orphans Using ORCID
Discovering Scholarly Orphans Using ORCIDDiscovering Scholarly Orphans Using ORCID
Discovering Scholarly Orphans Using ORCIDMartin Klein
 
To the Rescue of the Orphans of Scholarly Communication
To the Rescue of the Orphans of Scholarly CommunicationTo the Rescue of the Orphans of Scholarly Communication
To the Rescue of the Orphans of Scholarly CommunicationMartin Klein
 
Persistent Identification: Easier Said than Done
Persistent Identification: Easier Said than DonePersistent Identification: Easier Said than Done
Persistent Identification: Easier Said than DoneHerbert Van de Sompel
 
FAIR Signposting: A KISS Approach to a Burning Issue
FAIR Signposting: A KISS Approach to a Burning IssueFAIR Signposting: A KISS Approach to a Burning Issue
FAIR Signposting: A KISS Approach to a Burning IssueHerbert Van de Sompel
 
Creating Topical Collections: Web Archives vs. Live Web
Creating Topical Collections:Web Archives vs. Live WebCreating Topical Collections:Web Archives vs. Live Web
Creating Topical Collections: Web Archives vs. Live WebMartin Klein
 
Hiberlink: Investigating Reference Rot, December 2013
Hiberlink: Investigating Reference Rot, December 2013Hiberlink: Investigating Reference Rot, December 2013
Hiberlink: Investigating Reference Rot, December 2013Herbert Van de Sompel
 
The Single Power of Link - Richard Wallis
The Single Power of Link - Richard WallisThe Single Power of Link - Richard Wallis
The Single Power of Link - Richard Wallistulipbiru64
 
Linked Data Snowball, or Why We Need Reconciliation
Linked Data Snowball, or Why We Need ReconciliationLinked Data Snowball, or Why We Need Reconciliation
Linked Data Snowball, or Why We Need ReconciliationRobert Sanderson
 
Robust Linking to Web Resources
Robust Linking to Web ResourcesRobust Linking to Web Resources
Robust Linking to Web ResourcesMartin Klein
 
Linked data radical change
Linked data   radical changeLinked data   radical change
Linked data radical changeRichard Wallis
 
Prototypes of pro-active approaches to support the archiving of web reference...
Prototypes of pro-active approaches to support the archiving of web reference...Prototypes of pro-active approaches to support the archiving of web reference...
Prototypes of pro-active approaches to support the archiving of web reference...EDINA, University of Edinburgh
 
Linked Data for Libraries: Experiments between Cornell, Harvard and Stanford
Linked Data for Libraries: Experiments between Cornell, Harvard and StanfordLinked Data for Libraries: Experiments between Cornell, Harvard and Stanford
Linked Data for Libraries: Experiments between Cornell, Harvard and StanfordSimeon Warner
 
Linked Data - Radical Change?
Linked Data -  Radical Change?Linked Data -  Radical Change?
Linked Data - Radical Change?Richard Wallis
 
Reference Rot and Link Decoration
Reference Rot and Link DecorationReference Rot and Link Decoration
Reference Rot and Link DecorationMartin Klein
 

What's hot (20)

PID Signposting Pattern
PID Signposting PatternPID Signposting Pattern
PID Signposting Pattern
 
Paul Evan Peters Lecture
Paul Evan Peters LecturePaul Evan Peters Lecture
Paul Evan Peters Lecture
 
Discovering Scholarly Orphans Using ORCID
Discovering Scholarly Orphans Using ORCIDDiscovering Scholarly Orphans Using ORCID
Discovering Scholarly Orphans Using ORCID
 
Creating Pockets of Persistence
Creating Pockets of PersistenceCreating Pockets of Persistence
Creating Pockets of Persistence
 
To the Rescue of the Orphans of Scholarly Communication
To the Rescue of the Orphans of Scholarly CommunicationTo the Rescue of the Orphans of Scholarly Communication
To the Rescue of the Orphans of Scholarly Communication
 
Persistent Identification: Easier Said than Done
Persistent Identification: Easier Said than DonePersistent Identification: Easier Said than Done
Persistent Identification: Easier Said than Done
 
FAIR Signposting: A KISS Approach to a Burning Issue
FAIR Signposting: A KISS Approach to a Burning IssueFAIR Signposting: A KISS Approach to a Burning Issue
FAIR Signposting: A KISS Approach to a Burning Issue
 
Creating Topical Collections: Web Archives vs. Live Web
Creating Topical Collections:Web Archives vs. Live WebCreating Topical Collections:Web Archives vs. Live Web
Creating Topical Collections: Web Archives vs. Live Web
 
Hiberlink: Investigating Reference Rot, December 2013
Hiberlink: Investigating Reference Rot, December 2013Hiberlink: Investigating Reference Rot, December 2013
Hiberlink: Investigating Reference Rot, December 2013
 
The Single Power of Link - Richard Wallis
The Single Power of Link - Richard WallisThe Single Power of Link - Richard Wallis
The Single Power of Link - Richard Wallis
 
Linked Data Snowball, or Why We Need Reconciliation
Linked Data Snowball, or Why We Need ReconciliationLinked Data Snowball, or Why We Need Reconciliation
Linked Data Snowball, or Why We Need Reconciliation
 
Robust Linking to Web Resources
Robust Linking to Web ResourcesRobust Linking to Web Resources
Robust Linking to Web Resources
 
Linked data radical change
Linked data   radical changeLinked data   radical change
Linked data radical change
 
Prototypes of pro-active approaches to support the archiving of web reference...
Prototypes of pro-active approaches to support the archiving of web reference...Prototypes of pro-active approaches to support the archiving of web reference...
Prototypes of pro-active approaches to support the archiving of web reference...
 
The Web We Want
The Web We WantThe Web We Want
The Web We Want
 
Linked Data for Libraries: Experiments between Cornell, Harvard and Stanford
Linked Data for Libraries: Experiments between Cornell, Harvard and StanfordLinked Data for Libraries: Experiments between Cornell, Harvard and Stanford
Linked Data for Libraries: Experiments between Cornell, Harvard and Stanford
 
Linked Data - Radical Change?
Linked Data -  Radical Change?Linked Data -  Radical Change?
Linked Data - Radical Change?
 
Library Linked Data and the Future of Bibliographic Control
Library Linked Data and the Future of Bibliographic ControlLibrary Linked Data and the Future of Bibliographic Control
Library Linked Data and the Future of Bibliographic Control
 
LIBRIS - Linked Library Data
LIBRIS - Linked Library DataLIBRIS - Linked Library Data
LIBRIS - Linked Library Data
 
Reference Rot and Link Decoration
Reference Rot and Link DecorationReference Rot and Link Decoration
Reference Rot and Link Decoration
 

Viewers also liked

TPDL 2015 - Profiling Web Archives
TPDL 2015 - Profiling Web ArchivesTPDL 2015 - Profiling Web Archives
TPDL 2015 - Profiling Web ArchivesSawood Alam
 
Impact Analysis of OCR Quality on Research Tasks in Digital Archives
Impact Analysis of OCR Quality on Research Tasks in Digital ArchivesImpact Analysis of OCR Quality on Research Tasks in Digital Archives
Impact Analysis of OCR Quality on Research Tasks in Digital ArchivesMyriam Traub
 
Characteristics of Social Media Stories
Characteristics of Social Media StoriesCharacteristics of Social Media Stories
Characteristics of Social Media StoriesYasmin AlNoamany, PhD
 
Detecting Off-Topic Pages in Web Archives
Detecting Off-Topic Pages in Web ArchivesDetecting Off-Topic Pages in Web Archives
Detecting Off-Topic Pages in Web ArchivesYasmin AlNoamany, PhD
 
Profile Serialization IIPC GA 2015
Profile Serialization IIPC GA 2015Profile Serialization IIPC GA 2015
Profile Serialization IIPC GA 2015Sawood Alam
 
Using Web Archives to Enrich the Live Web Experience Through Storytelling
Using Web Archives to Enrich the Live Web Experience Through StorytellingUsing Web Archives to Enrich the Live Web Experience Through Storytelling
Using Web Archives to Enrich the Live Web Experience Through StorytellingYasmin AlNoamany, PhD
 
Web Archiving: A Brief Introduction
Web Archiving: A Brief IntroductionWeb Archiving: A Brief Introduction
Web Archiving: A Brief IntroductionSawood Alam
 
Profiling Web Archives
Profiling Web ArchivesProfiling Web Archives
Profiling Web ArchivesSawood Alam
 
JCDL2015: How Well are Arabic Websites Archived?
JCDL2015: How Well are Arabic Websites Archived?JCDL2015: How Well are Arabic Websites Archived?
JCDL2015: How Well are Arabic Websites Archived?LulwahMA
 
Improving Accessibility of Archived Raster Dictionaries of Complex Script Lan...
Improving Accessibility of Archived Raster Dictionaries of Complex Script Lan...Improving Accessibility of Archived Raster Dictionaries of Complex Script Lan...
Improving Accessibility of Archived Raster Dictionaries of Complex Script Lan...Sawood Alam
 
Detecting Off-Topic Pages in Web Archives
Detecting Off-Topic Pages in Web ArchivesDetecting Off-Topic Pages in Web Archives
Detecting Off-Topic Pages in Web ArchivesYasmin AlNoamany, PhD
 
Web Archives and Data Challenges - Archives Unleashed
Web Archives and Data Challenges - Archives UnleashedWeb Archives and Data Challenges - Archives Unleashed
Web Archives and Data Challenges - Archives Unleashedmwe400
 
Social Feed Manager presentation at Archives Unleashed 3.0
Social Feed Manager presentation at Archives Unleashed 3.0Social Feed Manager presentation at Archives Unleashed 3.0
Social Feed Manager presentation at Archives Unleashed 3.0Justin Littman
 
Web Archiving Activities of ODU’s Web Science and Digital Library Research G...
Web Archiving Activities of ODU’s Web Science and Digital Library Research G...Web Archiving Activities of ODU’s Web Science and Digital Library Research G...
Web Archiving Activities of ODU’s Web Science and Digital Library Research G...Michael Nelson
 
Twitter Analysis: Fake News
Twitter Analysis: Fake  NewsTwitter Analysis: Fake  News
Twitter Analysis: Fake NewsErika Siregar
 
Good News/ Bad News
Good News/ Bad NewsGood News/ Bad News
Good News/ Bad NewsLulwahMA
 
Paragraph writing dale
Paragraph writing daleParagraph writing dale
Paragraph writing daleShadale Mackey
 
A portfolio walk through
A portfolio walk throughA portfolio walk through
A portfolio walk throughYiping Xia
 
Изучаем ФГОС дошкольного образования
Изучаем ФГОС дошкольного образованияИзучаем ФГОС дошкольного образования
Изучаем ФГОС дошкольного образованияdetskiy_sad_120_Samara
 

Viewers also liked (20)

TPDL 2015 - Profiling Web Archives
TPDL 2015 - Profiling Web ArchivesTPDL 2015 - Profiling Web Archives
TPDL 2015 - Profiling Web Archives
 
Impact Analysis of OCR Quality on Research Tasks in Digital Archives
Impact Analysis of OCR Quality on Research Tasks in Digital ArchivesImpact Analysis of OCR Quality on Research Tasks in Digital Archives
Impact Analysis of OCR Quality on Research Tasks in Digital Archives
 
Characteristics of Social Media Stories
Characteristics of Social Media StoriesCharacteristics of Social Media Stories
Characteristics of Social Media Stories
 
Detecting Off-Topic Pages in Web Archives
Detecting Off-Topic Pages in Web ArchivesDetecting Off-Topic Pages in Web Archives
Detecting Off-Topic Pages in Web Archives
 
Profile Serialization IIPC GA 2015
Profile Serialization IIPC GA 2015Profile Serialization IIPC GA 2015
Profile Serialization IIPC GA 2015
 
Using Web Archives to Enrich the Live Web Experience Through Storytelling
Using Web Archives to Enrich the Live Web Experience Through StorytellingUsing Web Archives to Enrich the Live Web Experience Through Storytelling
Using Web Archives to Enrich the Live Web Experience Through Storytelling
 
Web Archiving: A Brief Introduction
Web Archiving: A Brief IntroductionWeb Archiving: A Brief Introduction
Web Archiving: A Brief Introduction
 
Profiling Web Archives
Profiling Web ArchivesProfiling Web Archives
Profiling Web Archives
 
JCDL2015: How Well are Arabic Websites Archived?
JCDL2015: How Well are Arabic Websites Archived?JCDL2015: How Well are Arabic Websites Archived?
JCDL2015: How Well are Arabic Websites Archived?
 
Improving Accessibility of Archived Raster Dictionaries of Complex Script Lan...
Improving Accessibility of Archived Raster Dictionaries of Complex Script Lan...Improving Accessibility of Archived Raster Dictionaries of Complex Script Lan...
Improving Accessibility of Archived Raster Dictionaries of Complex Script Lan...
 
Detecting Off-Topic Pages in Web Archives
Detecting Off-Topic Pages in Web ArchivesDetecting Off-Topic Pages in Web Archives
Detecting Off-Topic Pages in Web Archives
 
Web Archives and Data Challenges - Archives Unleashed
Web Archives and Data Challenges - Archives UnleashedWeb Archives and Data Challenges - Archives Unleashed
Web Archives and Data Challenges - Archives Unleashed
 
Social Feed Manager presentation at Archives Unleashed 3.0
Social Feed Manager presentation at Archives Unleashed 3.0Social Feed Manager presentation at Archives Unleashed 3.0
Social Feed Manager presentation at Archives Unleashed 3.0
 
Web Archiving Activities of ODU’s Web Science and Digital Library Research G...
Web Archiving Activities of ODU’s Web Science and Digital Library Research G...Web Archiving Activities of ODU’s Web Science and Digital Library Research G...
Web Archiving Activities of ODU’s Web Science and Digital Library Research G...
 
Twitter Analysis: Fake News
Twitter Analysis: Fake  NewsTwitter Analysis: Fake  News
Twitter Analysis: Fake News
 
Good News/ Bad News
Good News/ Bad NewsGood News/ Bad News
Good News/ Bad News
 
Paragraph writing dale
Paragraph writing daleParagraph writing dale
Paragraph writing dale
 
A portfolio walk through
A portfolio walk throughA portfolio walk through
A portfolio walk through
 
Electricity
ElectricityElectricity
Electricity
 
Изучаем ФГОС дошкольного образования
Изучаем ФГОС дошкольного образованияИзучаем ФГОС дошкольного образования
Изучаем ФГОС дошкольного образования
 

Similar to Quantifying Orphaned Annotations in Hypothes.is

Ensuring the Integrity (& Continuity) of Our Record of Scholarship
Ensuring the Integrity (& Continuity) of Our Record of ScholarshipEnsuring the Integrity (& Continuity) of Our Record of Scholarship
Ensuring the Integrity (& Continuity) of Our Record of ScholarshipEDINA, University of Edinburgh
 
Connect the Dots: Bridging Silos of Information (WPCampus 2019)
Connect the Dots: Bridging Silos of Information (WPCampus 2019)Connect the Dots: Bridging Silos of Information (WPCampus 2019)
Connect the Dots: Bridging Silos of Information (WPCampus 2019)Elaine Shannon
 
Hiberlink: Prototypes of pro-active approaches to support the archiving of we...
Hiberlink: Prototypes of pro-active approaches to support the archiving of we...Hiberlink: Prototypes of pro-active approaches to support the archiving of we...
Hiberlink: Prototypes of pro-active approaches to support the archiving of we...EDINA, University of Edinburgh
 
Social Software in Libraries Workshop
Social Software in Libraries WorkshopSocial Software in Libraries Workshop
Social Software in Libraries WorkshopMeredith Farkas
 
Connect With Your Users: Communicate Using Social Software Tools
Connect With Your Users: Communicate Using Social Software ToolsConnect With Your Users: Communicate Using Social Software Tools
Connect With Your Users: Communicate Using Social Software ToolsRobFav
 
Bio2RDF presentation at Combine 2012
Bio2RDF presentation at Combine 2012Bio2RDF presentation at Combine 2012
Bio2RDF presentation at Combine 2012François Belleau
 
Organising and Managing Research
Organising and Managing ResearchOrganising and Managing Research
Organising and Managing ResearchDr. Vinayak Bharadi
 
Ontology and Ontology Libraries: a Critical Study
Ontology and Ontology Libraries: a Critical StudyOntology and Ontology Libraries: a Critical Study
Ontology and Ontology Libraries: a Critical StudyDebashisnaskar
 
Web crawlingchapter
Web crawlingchapterWeb crawlingchapter
Web crawlingchapterBorseshweta
 
Profile-based Dataset Recommendation for RDF Data Linking
Profile-based Dataset Recommendation for RDF Data Linking  Profile-based Dataset Recommendation for RDF Data Linking
Profile-based Dataset Recommendation for RDF Data Linking Mohamed BEN ELLEFI
 
Ontology and Ontology Libraries: a critical study
Ontology and Ontology Libraries: a critical studyOntology and Ontology Libraries: a critical study
Ontology and Ontology Libraries: a critical studyDebashisnaskar
 
Web Today, Good Tomorrow? Transactional archiving of web content
Web Today, Good Tomorrow? Transactional archiving of web contentWeb Today, Good Tomorrow? Transactional archiving of web content
Web Today, Good Tomorrow? Transactional archiving of web contentPeter Burnhill
 
A Data Citation Roadmap for Scholarly Data Repositories
A Data Citation Roadmap for Scholarly Data RepositoriesA Data Citation Roadmap for Scholarly Data Repositories
A Data Citation Roadmap for Scholarly Data RepositoriesLIBER Europe
 
Usage of Linked Data: Introduction and Application Scenarios
Usage of Linked Data: Introduction and Application ScenariosUsage of Linked Data: Introduction and Application Scenarios
Usage of Linked Data: Introduction and Application ScenariosEUCLID project
 
What Does DITA Have To Do With Wiki
What Does DITA Have To Do With WikiWhat Does DITA Have To Do With Wiki
What Does DITA Have To Do With WikiAnne Gentle
 
Verifiable, linked open knowledge that anyone can edit
Verifiable, linked open knowledge that anyone can editVerifiable, linked open knowledge that anyone can edit
Verifiable, linked open knowledge that anyone can editDario Taraborelli
 

Similar to Quantifying Orphaned Annotations in Hypothes.is (20)

Ensuring the Integrity (& Continuity) of Our Record of Scholarship
Ensuring the Integrity (& Continuity) of Our Record of ScholarshipEnsuring the Integrity (& Continuity) of Our Record of Scholarship
Ensuring the Integrity (& Continuity) of Our Record of Scholarship
 
Connect the Dots: Bridging Silos of Information (WPCampus 2019)
Connect the Dots: Bridging Silos of Information (WPCampus 2019)Connect the Dots: Bridging Silos of Information (WPCampus 2019)
Connect the Dots: Bridging Silos of Information (WPCampus 2019)
 
Hiberlink: Prototypes of pro-active approaches to support the archiving of we...
Hiberlink: Prototypes of pro-active approaches to support the archiving of we...Hiberlink: Prototypes of pro-active approaches to support the archiving of we...
Hiberlink: Prototypes of pro-active approaches to support the archiving of we...
 
Social Software in Libraries Workshop
Social Software in Libraries WorkshopSocial Software in Libraries Workshop
Social Software in Libraries Workshop
 
Lec1.pptx
Lec1.pptxLec1.pptx
Lec1.pptx
 
Connect With Your Users: Communicate Using Social Software Tools
Connect With Your Users: Communicate Using Social Software ToolsConnect With Your Users: Communicate Using Social Software Tools
Connect With Your Users: Communicate Using Social Software Tools
 
Open Annotation Model
Open Annotation ModelOpen Annotation Model
Open Annotation Model
 
Bio2RDF presentation at Combine 2012
Bio2RDF presentation at Combine 2012Bio2RDF presentation at Combine 2012
Bio2RDF presentation at Combine 2012
 
Reference Rot: Threat and Remedy
Reference Rot: Threat and RemedyReference Rot: Threat and Remedy
Reference Rot: Threat and Remedy
 
Organising and Managing Research
Organising and Managing ResearchOrganising and Managing Research
Organising and Managing Research
 
Ontology and Ontology Libraries: a Critical Study
Ontology and Ontology Libraries: a Critical StudyOntology and Ontology Libraries: a Critical Study
Ontology and Ontology Libraries: a Critical Study
 
Web crawlingchapter
Web crawlingchapterWeb crawlingchapter
Web crawlingchapter
 
Profile-based Dataset Recommendation for RDF Data Linking
Profile-based Dataset Recommendation for RDF Data Linking  Profile-based Dataset Recommendation for RDF Data Linking
Profile-based Dataset Recommendation for RDF Data Linking
 
Ontology and Ontology Libraries: a critical study
Ontology and Ontology Libraries: a critical studyOntology and Ontology Libraries: a critical study
Ontology and Ontology Libraries: a critical study
 
Web Today, Good Tomorrow? Transactional archiving of web content
Web Today, Good Tomorrow? Transactional archiving of web contentWeb Today, Good Tomorrow? Transactional archiving of web content
Web Today, Good Tomorrow? Transactional archiving of web content
 
A Data Citation Roadmap for Scholarly Data Repositories
A Data Citation Roadmap for Scholarly Data RepositoriesA Data Citation Roadmap for Scholarly Data Repositories
A Data Citation Roadmap for Scholarly Data Repositories
 
Reference Rot and E-Theses: Threat and Remedy
Reference Rot and E-Theses: Threat and RemedyReference Rot and E-Theses: Threat and Remedy
Reference Rot and E-Theses: Threat and Remedy
 
Usage of Linked Data: Introduction and Application Scenarios
Usage of Linked Data: Introduction and Application ScenariosUsage of Linked Data: Introduction and Application Scenarios
Usage of Linked Data: Introduction and Application Scenarios
 
What Does DITA Have To Do With Wiki
What Does DITA Have To Do With WikiWhat Does DITA Have To Do With Wiki
What Does DITA Have To Do With Wiki
 
Verifiable, linked open knowledge that anyone can edit
Verifiable, linked open knowledge that anyone can editVerifiable, linked open knowledge that anyone can edit
Verifiable, linked open knowledge that anyone can edit
 

Recently uploaded

Genetic Code. A comprehensive overview..pdf
Genetic Code. A comprehensive overview..pdfGenetic Code. A comprehensive overview..pdf
Genetic Code. A comprehensive overview..pdfmughalgumar440
 
Lecture 10 .Parasympathetic agents- Indirect acting.pptx
Lecture 10 .Parasympathetic agents- Indirect acting.pptxLecture 10 .Parasympathetic agents- Indirect acting.pptx
Lecture 10 .Parasympathetic agents- Indirect acting.pptxmanjusha kareppa
 
Nutritional Problems in India . pptx
Nutritional  Problems  in  India .  pptxNutritional  Problems  in  India .  pptx
Nutritional Problems in India . pptxsaviosha12
 
PSILOTUM : structure, morphology, anatomy, reproduction , life cycle etc.
PSILOTUM : structure, morphology, anatomy,  reproduction , life cycle etc.PSILOTUM : structure, morphology, anatomy,  reproduction , life cycle etc.
PSILOTUM : structure, morphology, anatomy, reproduction , life cycle etc.Silpa Selvaraj
 
Microbial Fermentation(Strain Improvement)
Microbial  Fermentation(Strain Improvement)Microbial  Fermentation(Strain Improvement)
Microbial Fermentation(Strain Improvement)Rachana Choudhary
 
Chemical Bonding and it's Types 001.pptx
Chemical Bonding and it's Types 001.pptxChemical Bonding and it's Types 001.pptx
Chemical Bonding and it's Types 001.pptxperiyar arts college
 
electrophoresis: types, advantages, disadvantages and applications.
electrophoresis: types, advantages, disadvantages and applications.electrophoresis: types, advantages, disadvantages and applications.
electrophoresis: types, advantages, disadvantages and applications.Silpa Selvaraj
 
Weak-lensing detection of intracluster filaments in the Coma cluster
Weak-lensing detection of intracluster filaments in the Coma clusterWeak-lensing detection of intracluster filaments in the Coma cluster
Weak-lensing detection of intracluster filaments in the Coma clusterSérgio Sacani
 
Seminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisSeminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisLinaMarcelaCharrisRa
 
Inroduction to LIPOSOMES & NIOSOMES.pptx
Inroduction to LIPOSOMES & NIOSOMES.pptxInroduction to LIPOSOMES & NIOSOMES.pptx
Inroduction to LIPOSOMES & NIOSOMES.pptxHarshadaa bafna
 
dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...
dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...
dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...dkNET
 
nanotechnology in forensic science......
nanotechnology in forensic science......nanotechnology in forensic science......
nanotechnology in forensic science......Madona Mathew
 
PINKY[1].pptx.priyanka.ppt.protein organization.
PINKY[1].pptx.priyanka.ppt.protein organization.PINKY[1].pptx.priyanka.ppt.protein organization.
PINKY[1].pptx.priyanka.ppt.protein organization.aishwaryabharadwaj31
 
Interpretation, measurement and mixed methods
Interpretation, measurement and mixed methodsInterpretation, measurement and mixed methods
Interpretation, measurement and mixed methodsTuukka Ylä-Anttila
 
Anatomy-Reproductive-System for grade 10
Anatomy-Reproductive-System for grade 10Anatomy-Reproductive-System for grade 10
Anatomy-Reproductive-System for grade 10rolanaribato30
 
Seminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisSeminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisLinaMarcelaCharrisRa
 
UKRAINIAN Kyiv Institute of Thermophysics Memorandum_English.pdf
UKRAINIAN Kyiv Institute of Thermophysics Memorandum_English.pdfUKRAINIAN Kyiv Institute of Thermophysics Memorandum_English.pdf
UKRAINIAN Kyiv Institute of Thermophysics Memorandum_English.pdfThane Heins
 
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...Uzay Emir
 
Ento-322, Agrochemicals for agriculture usee
Ento-322, Agrochemicals for agriculture useeEnto-322, Agrochemicals for agriculture usee
Ento-322, Agrochemicals for agriculture useeDrAnita Sharma
 
Introduction to Chromatography (Column chromatography)
Introduction to Chromatography (Column chromatography)Introduction to Chromatography (Column chromatography)
Introduction to Chromatography (Column chromatography)Ahmed Metwaly
 

Recently uploaded (20)

Genetic Code. A comprehensive overview..pdf
Genetic Code. A comprehensive overview..pdfGenetic Code. A comprehensive overview..pdf
Genetic Code. A comprehensive overview..pdf
 
Lecture 10 .Parasympathetic agents- Indirect acting.pptx
Lecture 10 .Parasympathetic agents- Indirect acting.pptxLecture 10 .Parasympathetic agents- Indirect acting.pptx
Lecture 10 .Parasympathetic agents- Indirect acting.pptx
 
Nutritional Problems in India . pptx
Nutritional  Problems  in  India .  pptxNutritional  Problems  in  India .  pptx
Nutritional Problems in India . pptx
 
PSILOTUM : structure, morphology, anatomy, reproduction , life cycle etc.
PSILOTUM : structure, morphology, anatomy,  reproduction , life cycle etc.PSILOTUM : structure, morphology, anatomy,  reproduction , life cycle etc.
PSILOTUM : structure, morphology, anatomy, reproduction , life cycle etc.
 
Microbial Fermentation(Strain Improvement)
Microbial  Fermentation(Strain Improvement)Microbial  Fermentation(Strain Improvement)
Microbial Fermentation(Strain Improvement)
 
Chemical Bonding and it's Types 001.pptx
Chemical Bonding and it's Types 001.pptxChemical Bonding and it's Types 001.pptx
Chemical Bonding and it's Types 001.pptx
 
electrophoresis: types, advantages, disadvantages and applications.
electrophoresis: types, advantages, disadvantages and applications.electrophoresis: types, advantages, disadvantages and applications.
electrophoresis: types, advantages, disadvantages and applications.
 
Weak-lensing detection of intracluster filaments in the Coma cluster
Weak-lensing detection of intracluster filaments in the Coma clusterWeak-lensing detection of intracluster filaments in the Coma cluster
Weak-lensing detection of intracluster filaments in the Coma cluster
 
Seminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisSeminario biología molecular Lina Charris
Seminario biología molecular Lina Charris
 
Inroduction to LIPOSOMES & NIOSOMES.pptx
Inroduction to LIPOSOMES & NIOSOMES.pptxInroduction to LIPOSOMES & NIOSOMES.pptx
Inroduction to LIPOSOMES & NIOSOMES.pptx
 
dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...
dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...
dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...
 
nanotechnology in forensic science......
nanotechnology in forensic science......nanotechnology in forensic science......
nanotechnology in forensic science......
 
PINKY[1].pptx.priyanka.ppt.protein organization.
PINKY[1].pptx.priyanka.ppt.protein organization.PINKY[1].pptx.priyanka.ppt.protein organization.
PINKY[1].pptx.priyanka.ppt.protein organization.
 
Interpretation, measurement and mixed methods
Interpretation, measurement and mixed methodsInterpretation, measurement and mixed methods
Interpretation, measurement and mixed methods
 
Anatomy-Reproductive-System for grade 10
Anatomy-Reproductive-System for grade 10Anatomy-Reproductive-System for grade 10
Anatomy-Reproductive-System for grade 10
 
Seminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisSeminario biología molecular Lina Charris
Seminario biología molecular Lina Charris
 
UKRAINIAN Kyiv Institute of Thermophysics Memorandum_English.pdf
UKRAINIAN Kyiv Institute of Thermophysics Memorandum_English.pdfUKRAINIAN Kyiv Institute of Thermophysics Memorandum_English.pdf
UKRAINIAN Kyiv Institute of Thermophysics Memorandum_English.pdf
 
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...
 
Ento-322, Agrochemicals for agriculture usee
Ento-322, Agrochemicals for agriculture useeEnto-322, Agrochemicals for agriculture usee
Ento-322, Agrochemicals for agriculture usee
 
Introduction to Chromatography (Column chromatography)
Introduction to Chromatography (Column chromatography)Introduction to Chromatography (Column chromatography)
Introduction to Chromatography (Column chromatography)
 

Quantifying Orphaned Annotations in Hypothes.is

  • 1. Quantifying Orphaned Annotations in Hypothes.is Mohamed Aturban, Michael L. Nelson, and Michele C. Weigle Department of Computer Science, Old Dominion University, Norfolk, VA 23529 1 TPDL 2015 Poznan, Poland September 13-17, 2015
  • 2. What is Web Annotation? 2 Handwritten Annotations Web Annotations Haslhofer, B., Simon, R., Sanderson, R., Van de Sompel, H.: The Open Annotation Collaboration (OAC) model. In: Proceedings of the IEEE Workshop on Multimedia on the Web (MMWeb). pp. 5–9. IEEE (2011) http://networkedlearningcollaborative.com/wp-content/uploads/2015/07/53e12bf10cf2d79877a53311.pdf
  • 3. • Open Annotation Collaboration (OAC) group defines an annotation as a set of connected resources What is Web Annotation? 3
  • 4. • Open Annotation Collaboration (OAC) group defines an annotation as a set of connected resources What is Web Annotation? 4
  • 5. Why is Web Annotation Important? • A collaborative tool • Social criticism • Education for students and teachers • Scholarly and academic purposes • Editing and publishing Web annotations do not modify the original resource 5
  • 6. Annotating Web Resources Using Hypothes.is 6 Highlights Notes Replies Tags Target URI
  • 7. The Annotation is Attached to the Live Web 7 http://www.wired.co.uk/news/archive/2014-02/06/tim-berners-lee-reclaim-the- web/viewgallery/332234 August 2015 (from live web) https://hypothes.is/a/tp5hnn4PTjuSg7h bc7_cxQ Annotation made in February 2014 Tags
  • 8. A Review of Annotation Attachment to the Live & Archived Web 8 Does an annotation attach to its target’s live webpage? Does an annotation attach to any archived copies (Mementos) of the target webpage 1 YES YES 2 YES NO 3 NO YES 4 NO NO
  • 9. The Annotation is Attached to the Live Web and to an Archived Copy (Memento) of the Webpage 9 https://web.archive.org/web/20140207083733/http:// www.wired.co.uk/news/archive/2014-02/06/tim- berners-lee-reclaim-the-web/viewgallery/332234 February 2014 (an archived copy) http://www.wired.co.uk/news/archive/2014- 02/06/tim-berners-lee-reclaim-the- web/viewgallery/332234 August 2015 (from live web) https://hypothes.is/a/tp5hnn4PT juSg7hbc7_cxQ Annotation made in February 2014
  • 10. The Annotation is Attached to the Live Web but No Mementos Are Available 10 http://tkbr.ccsp.sfu.ca/pub802/2015/01/more- horsepower-to-wattpad/ August 2015 (from live web) https://hypothes.is/a/o_2W8QwZR Dm8w0F1dqRxUQ and https://hypothes.is/a/NE8AT6R3Tn 6Qg6FeYy1C0w Annotation made in February 2015 The Annotation is in Danger of being Orphaned
  • 11. The Annotation is Not Attached to the Live Web but It is Attached to Mementos 11 https://web.archive.org/web/201412101210 18/http://climatefeedback.org/ December 2014 (an archived copy) climatefeedback.org Annotation made in December 2014 climatefeedback.org August 2015 (from live web)
  • 12. The Annotation is Not Attached to the Live Web and No Mementos Are Available 12 http://renaissancejohnson.weebly.com/spensers- wordcloud.html August 2015 (from live web) https://hypothes.is/a/wFLZKLGqS8 S3Zyfr_Rmu4w Annotation in July 2015 The Annotation is Orphaned
  • 13. Four Different Cases For Annotation Attachment Does an annotation attach to its target’s live webpage? Does an annotation attach to any archived copies (Mementos) of the target webpage 1 YES YES 2 YES NO 3 NO YES 4 NO NO 13 In Danger of being Orphaned Orphaned Safe Can Reattach to Memento
  • 14. We Studied Hypothes.is Annotations •How many annotations are orphaned? •How many annotations are in danger of being orphaned? •How many annotations can be reattached to mementos in public web archives? 14
  • 15. Related Work • OAC introduced the idea to make annotations reusable through different systems • Annotations, as web resources, have unique URIs • Sanderson and Van de Sompel introduced a framework to make web annotations persistent over time • Integrating features in the Open Annotation Data Model with the Memento framework • Reconstructing annotations for a given memento • Retrieving mementos for a given annotation • Kreymer's Browsertrix provides on-demand web archiving • Whenever an annotation is created, a copy of the related webpage could be archived automatically • Funded by Hypothes.is 15 Sanderson, R., Van de Sompel, H.: Making web annotations persistent over time. In: Proceedings of the 10th ACM/IEEE Joint Conference on Digital Libraries (JCDL). pp. 1–10. ACM (2010) http://blog.webrecorder.io/2015/06/open-annotation-fund-project.html https://hypothes.is/blog/fund-on-demand-web-archiving-completion/
  • 16. Annotations in Hypothes.is Are Increasing 16 January 2015 (7744) August 2015 ( 33,946 ) January 2015 - dataset used in TPDL 2015 paper August 2015 - dataset presented here and in arXiv version
  • 17. Annotation Types in Hypothes.is 17 Highlighted Text Note Tags Number of Annotations  11,289   9858    9252   1835   1356  348  8 33,953
  • 18. Annotation Types in Hypothes.is 18 Highlighted Text Note Tags Number of Annotations  11,289   9858    9252   1835   1356  348  8 33,953 We studied 20,953 annotations that contain highlighted text
  • 19. Several Academic Sites Use Hypothes.is Widely 19 Number of Annotations Contain Highlighted Text Host 1222 caseyboyle.net 1191 www.perseus.tufts.edu 887 rhetoric.eserver.org 875 networkedlearningcollaborative.com 749 sosol.perseids.org 733 tkbr.ccsp.sfu.ca 526 shakespeare.mit.edu 391 hypothes.is 356 renaissancejohnson.weebly.com 336 moodle2.wesleyan.edu
  • 20. We Issued HTTP Head Requests for Target URIs of All Annotations 20 Number of Annotations Status Code Example 18,167 200 OK http://www.w3.org/Talks/9704WWW6-tbl/slide16.htm 820 Unresolvable URIs file:///Users/peggy/Desktop/CIRCLE-youthvoting-individualPages.pdf 778 Timeout http://testbelfastgroup.digitalscholarship.emory.edu/ 666 404 https://www.facebook.com/manunymous 190 Soft 4XX http://www.transmography.net/braineryworkshop/camperforce-by-joseph/ 87 401 http://wiki.shuttleworthfoundation.org/~shuttlew/wiki/index.php?title=Nov_2 013_DW_Dogfood_prep 80 503 https://via.hypothes.is/http://b.pagekite.me/blog/2015-04- 27_Roadmap_to_v1.html 68 403 http://onlinelibrary.wiley.com/store/10.1002/2013EF000191/asset/eft214.pdf? v=1&t=hppouayu&s=e3a980c9e2c6317987306d4e1d76c690c29fe758 48 410 https://www.scribd.com/word/removal/31126999 21 406 https://www.scribd.com/deleted/95457320 19 500 http://androidfrat.com/2015/01/the-new-usb-announcement-just-killed-the- usb-super-position/ 9 400, 416, 504, 520 https://ibpublishing.ibo.org/live-exist/rest/app/pub.xql?doc=EX_Instructions_ 2013_e&part=10&chapter=4&page=1
  • 21. Out of 33,946 Annotations, We Investigated 20,133 Target URIs 21
  • 22. { "updated": "2014-02-10T22:51:03.920650+00:00", "target": [ { "source": "http://www.wired.co.uk/news/archive/2014-02/06/tim-berners- lee-reclaim-the-web", "selector": [ { "endContainer": "/div[3]/form[1]/div[2]/article[1]/div[1]/div[1]/div[2]/div[1]/div[1]/p[1]", "endOffset": 187, "type": "RangeSelector", "startOffset": 0, "startContainer": “/div[3]/form[1]/div[2]/article[1]/div[1]/div[1]/div[2]/div[1]/div[1]/p[1]" }, { "exact": "Twenty-five years on from the web's inception, its creator has urged the public to re-engage with its original design: a decentralised internet that at its very core, remains open to all.", "prefix": "anChris Woods / chrismwoods.com", "type": "TextQuoteSelector", "suffix": "Speaking with Wired editor Davi" }, { "start": 307, "end": 494, "type": "TextPositionSelector" }]}], "tags": ["w3", "re-decentrilization" ], "text": "", "created": "2014-02-10T22:51:03.920636+00:00", "uri": "http://www.wired.co.uk/news/archive/2014-02/06/tim-berners-lee- reclaim-the-web", "user": "acct:aculich@hypothes.is", "consumer": "00000000-0000-0000-0000-000000000000", "id": "tp5hnn4PTjuSg7hbc7_cxQ", "permissions": { "admin": [ "acct:aculich@hypothes.is“ ], … An Annotation: JSON (L) and Visualized (R) 22 http://www.wired.co.uk/news/archive/2014-02/06/tim-berners-lee- reclaim-the-web/viewgallery/332234 August 2015
  • 23. • Compare an annotation’s highlighted text with a live webpage’s content. • Download annotation JSON from Hypothes.is • Extract the text from the annotation’s target URI • If the annotation’s highlighted text is found in the webpage the annotation is attached to the live web 23 Methodology
  • 24. 24 Only 78% of Highlighted Text Annotations Attach to the Live Web
  • 25. Discovering Mementos for All Resolvable Target URIs • Using LANL Memento Aggregator • Considering only mementos with datetime immediately before or after the annotation’s creation date • Four cases regarding the availability of mementos for the annotation’s target URI: • Mementos exist before and after the annotation’s creation date • Mementos exist only before the annotation’s creation date • Mementos exist only after annotation’s creation date • No mementos exist 25
  • 26. Mementos Exist Before and After the Annotation Creation Date 26 25% (4986) of resolvable target URIs
  • 27. 27 Mementos Exist Only Before the Annotation Creation Date 12% (2477) of resolvable target URIs
  • 28. 28 7% (1397) of resolvable target URIs Mementos Exist Only After the Annotation Creation Date
  • 29. 29 No Mementos Exist 56% (11273) of resolvable target URIs
  • 30. Are Annotations Attached to Existing Mementos? • Similar to checking if an annotation is attached to the live web • If the highlighted text is found in a memento the annotation is attached and could be recovered. • 8860 annotations have a target URI with at least one memento • Of these, 90% (7963) can be attached to a memento 30
  • 31. Annotation Targets with Existing Mementos Before and After the Annotation Creation Date 31 Attached to Live Web Page Attached to Memento (Before) Attached to Memento (After) Number of Annotations Yes Yes Yes 4091 Yes Yes No 93 Yes No Yes 100 Yes No No 182 No Yes Yes 251 No Yes No 69 No No Yes 44 No No No 156 4986 (Total)
  • 32. Attached to Live Web Page Attached to Memento (Before) Attached to Memento (After) Number of Annotations Yes Yes Yes 4091 Yes Yes No 93 Yes No Yes 100 Yes No No 182 No Yes Yes 251 No Yes No 69 No No Yes 44 No No No 156 4986 (Total) Annotation Targets with Existing Mementos Before and After the Annotation Creation Date 32 In Danger Orphaned
  • 33. Annotation Targets with Existing Mementos Only Before the Annotation Creation Date 33 Attached to Live Web Page Attached to Memento (Before) Number of Annotations Yes Yes 1984 Yes No 235 No Yes 133 No No 125 2477 (Total)
  • 34. Attached to Live Web Page Attached to Memento (Before) Number of Annotations Yes Yes 1984 Yes No 235 No Yes 133 No No 125 2477 (Total) Annotation Targets with Existing Mementos Only Before the Annotation Creation Date 34 In Danger Orphaned
  • 35. Attached to Live Web Page Attached to Memento (After) Number of Annotations Yes Yes 1148 Yes No 101 No Yes 50 No No 98 1397 (Total) Annotation Targets with Existing Mementos Only After the Annotation Creation Date 35
  • 36. Attached to Live Web Page Attached to Memento (After) Number of Annotations Yes Yes 1148 Yes No 101 No Yes 50 No No 98 1397 (Total) Annotation Targets with Existing Mementos Only After the Annotation Creation Date 36 In Danger Orphaned
  • 37. Annotation Targets with No Existing Mementos 37 Attached to Live Web Page Number of Annotations Yes 7839 No 3434 11,273 (Total)
  • 38. Attached to Live Web Page Number of Annotations Yes 7839 No 3434 11,273 (Total) Annotation Targets with No Existing Mementos 38 In Danger Orphaned
  • 39. How Many Orphaned Annotations Does Hypothes.is Have? • 19% (3813) of annotations are orphaned • 41% (8357) of annotations are in danger of being orphaned • In total, 60% (12,170) of annotations are either orphaned or in danger of being orphaned 39
  • 40. How Many Annotations Can Be Reattached Using Web Archives? • Archives could only save 3% (547) of annotations that would otherwise be orphaned • 37% (7416) of annotations are safe -- attached to the live web and also attached to one or more mementos. 40
  • 41. Archives Used to Attach Annotations to Mementos 41 Archive Attached to Live Web Not Attached to Live Web web.archive.org 6997 (94.3%) 455 (83.1%) archive.is 679 (9.15%) 39 (7.12%) wayback.archive-it.org 562 (7.57%) 47 (8.59%) github.com 80 (1.07%) 21 (3.83%) wayback.vefsafn.is 71 (0.95%) 53 (9.68%) arxiv.org 18 (0.24%) 0 webarchive.loc.gov 3 (0.04%) 0 webarchive.org.uk 4 (0.05%) 0 webarchive.nationalarchives.gov.uk 2 (0.02%) 0 discordia.wikia.com 1 (0.01%) 0 Total 8417 (113.4%) 615 (112.32%) A single annotation may reattach to mementos from multiple archives
  • 42. The Status of Current Hypothes.is Annotations 42 Highlighted Text Annotations with Resolvable Target URIs : 20,133
  • 43. Conclusion • We analyzed the attachment of 20,953 highlighted text annotations in Hypothes.is. • 60% of annotations are orphaned or in danger of being orphaned • 7963 mementos from 10 different web archives could be used to keep the remaining 40% of annotations safe. 43 Archiving webpages at the time of annotation is important to avoid orphaned annotations.