Herbert Van de Sompel, profile picture
Uploaded byHerbert Van de Sompel
PPTX, PDF1,631 views

Perseverance on Persistence

The document discusses Herbert Van de Sompel's insights on digital persistence and interoperability challenges in the context of scholarly assets, emphasizing the need for a web-centric paradigm. It addresses issues like 'link rot' and 'content drift,' which hinder faithful navigation of archived web content, and proposes solutions like persistent identifiers (PIDs) and robust linking practices to manage these challenges. Furthermore, it highlights the importance of machine learning for routing web archive requests and presents the Memento framework for temporal access to historical web states.

Embed presentation

Downloaded 17 times
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Herbert Van de Sompel Los Alamos National Laboratory @hvdsomp Perseverance on Persistence a future-note about the past
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 OAI-ORE
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 2006 • OAI-ORE observation: Scholarly assets are rapidly becoming compound, consisting of multiple resources with various: • Relationships • Interdependencies • How to convey this compound-ness in an interoperable manner so that applications can access, consume such assets? http://www.openarchives.org/ore/1.0/toc
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Address interoperability challenges from the perspective of the web • The resource at the center of the universe • The notion of a repository (or even of a web server) does not exist in the architecture of the web • Neither the notion of a Digital Object • The tools of the interoperability trade are the primitives of the web ORE Insight 1 - Web-Centric Interoperability Paradigm
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Tools of the Web-Centric Interoperability Trade • Resource • URI • HTTP as the API: HEAD/GET, POST, PUT, DELETE • Representation • Media Type • Link • Content Negotiation • Typed Link • Controlled Vocabularies for Typed Links W3C Architecture of the World Wide Web RDF, RDFS, OWL
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 OAI-ORE in EDM Europeana v1.0 2009
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 The web-centric ORE approach allowed using off-the-shelf web tools to archive evolving compound objects • Evolving versions of Resource Maps, Aggregated Resources were captured in a web archive • But how to use the URI of the Aggregation or Resource Map to see the status of an Aggregation at a specific moment in the past? ORE Insight 2 – How to Access Temporal State of an Aggregation H. Van de Sompel (2007) Compound Information Object Prototype Demonstration https://www.dropbox.com/s/dd7xd427y90q4jx/CT_Watch_hvds_20070703.mov?dl=0
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 H. Van de Sompel, M. L. Nelson, R. Sanderson (2013) RFC7089 - HTTP Framework for Time- Based Access to Resource States – Memento. https://tools.ietf.org/html/rfc7089 Memento
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Tools of the Web-Centric Interoperability Trade – HTTP Stack • Resource • URI • HTTP as the API • Representation • Media Types • Link • Content Negotiation, e.g. for preferred Media Type • Typed Link • Controlled Vocabularies for Typed Links W3C Architecture of the World Wide Web HTTP Links, IANA link relation registry, community link relation types HATEOAS – Hypermedia As The Engine Of Application State http://en.wikipedia.org/wiki/HATEOAS
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Original Resource and Mementos
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Bridge from Present to Past
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Bridge from Present to Past
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Bridge from Past to Present
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 timegate Link: Link to Your Own History Can link to preferred web archive, but also: • Maintain your own resource version history • timegate link to your own history • Distributed management of resource history • Uniform access to resource history across systems • Follow links across systems subject to time
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 No timegate Link – Client Intelligence Client uses TimeGate of its preferred web archive, but: • Internet Archive is massive, yet substantial unique materials in other archives • Introduce aggregated TimeGate: Memento Aggregator
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Routing TimeGate Requests Using Machine Learning Bornand, N., Balakireva, L., Van de Sompel, H. (2016) Routing Memento Requests Using Binary Classifiers. JCDL16. https://arxiv.org/abs/1606.09136 • Memento Aggregator covers 20+ web archives • Distributed systems problem: As the number of archives (and incoming requests) grows, sending requests to each archive for every incoming request is not feasible • Response times • Load on distributed archives • After various optimization attempts, devised an approach using binary classifiers per web archive: • Trained on the basis of cached URIs, using URI features only • Operational since 2016: 80% reduction in # queries. 1/3 reduction in response times. Recall 85%
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 From Internet Archive TodayToday Select Date Mar 20 2007 Apr 03 2007 Various Memento Tools (client/server) https://github.com/machawk1/awesome-memento
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Pockets of Persistence
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Creating Pockets of Persistence • With Memento’s time travel capability in place, what would it take to support faithfully navigating the web of the Past? • There are two major forces that hinder achieving this goal: • Link rot: A link stops working all together • Content drift: The linked content changes over time and may eventually no longer be representative of the content that was originally linked • Without these forces at work, the web of the Present would be the same as the web of the Past • But that clearly is not the case
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Hyperlinks in Theory
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Hyperlinks in Reality
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Hyperlinks in Reality
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Link Rot
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Link Rot - PMC Martin Klein, Herbert Van de Sompel, Robert Sanderson, Harihar Shankar, et al. (2014) Scholarly context not found. In: PLOS ONE https://doi.org/10.1371/journal.pone.0115253
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Hyperlinks in Reality
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Content Drift
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Content Drift
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Content Drift http://icecube.wisc.edu/ on May 8 2009 (left) and August 27 2009 (right)
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 No Content Drift http://www.ifa.hawaii.edu/~cowie/k_table.html on June 9 1997 (left) and March 2016 (right)
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Content Drift - PMC Shawn Jones, Herbert Van de Sompel, Harihar Shankar, Martin Klein, et al. (2016) Scholarly context not found. In: PLOS ONE https://doi.org/10.1371/journal.pone.0167475
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Creating Pockets of Persistence • What would it take to really support faithfully navigating the web of the Past? • This challenge exists for the entire web. Some communities with well managed collections care about addressing it: • Scholarly communication • Cultural heritage • Legal publications • Journalism • Wikipedia • Why? • Link Rot: Quality of Service • Content Drift: integrity of the record, reliable evidence, revisiting the state of knowledge, transparency of editorial process, …
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 US Supreme Court Opinion – Link Rot Activism http://ssnat.com
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Two Types of Links from a Managed Collection
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Take 1 – PID Approach PID for B
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Managed Collection => Managed Collection
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 PID Approach Combat: • Link Rot: Link to PID; Redirect to current location • Content Drift: Mint a PID per version; Link to version PID With PID links: • Web of Present = Web of Past
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 URI References - PMC Herbert Van de Sompel, Martin Klein, and Shawn Jones (2016) Persistent URIs Must Be Used to Be Persistent. In: WWW2016. http://arxiv.org/1602.09102 Herbert Van de Sompel, Martin Klein, and Shawn Jones (2016) Persistent URIs Must Be Used to Be Persistent. In: WWW2016. http://arxiv.org/1602.09102
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 cite-as Relation Type Herbert Van de Sompel et al. (2018) cite-as: A Link Relation to Convey a Preferred URI for Referencing. https://datatracker.ietf.org/doc/draft-vandesompel-citeas/ http://signposting.org
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 PID Approach – Division of Labor
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Managed Collection => Web at Large
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 PID Approach  -
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Take 2 – Robust Links Approach
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Managed Collection => Web at Large
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Snapshot Approach Combat: • Link Rot & Content Drift: Custodian of A creates snapshot of B, in web archive or locally Regarding links: • Intuition suggests linking to the snapshot of B …
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Linking to Snapshot of B = Potentially Creating a Rotten Link • Existing practice for linking to snapshots: <a href=“URL of snapshot of B”> • Problems with existing practice: o Impossible to visit the original URI, if desired o Requires the permanent existence/uptime of the archive that holds the snapshot - One link rot problem replaced by another http://robustlinks.mementoweb.org/about/
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Permanent Existence/Uptime of Archives? Remnant of discontinued web archive http://mummify.it captured on February 14 2014 https://web.archive.org/web/20140214233752/https://www.mummify.it/
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Permanent Existence/Uptime of Archives? http://www.themoscowtimes.com/news/article/russia-bans-wayback-machine-internet-archive-over- islamic-state-video/510074.html
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Permanent Existence/Uptime of Archives? http://web.archive.org/web/20121101043952/http://vogin.nl on March 6 2017 at 15:59 CET
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Decorate the Link • Proposed practice for linking to captures: <a href=“URL of snapshot of B” data-originalurl=“B” data-versiondate=“datetime of snapshot of B”> <a href=“B” data-versionurl=“URL of snapshot of B” data-versiondate=“datetime of snapshot of B”> http://robustlinks.mementoweb.org/spec/
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Robust Links: Link Decoration in Action Van de Sompel H. & Nelson, M.L. (2015) Reminiscing about 15 years of interoperability efforts. In: D-Lib Magazine. https://doi.org/10.1045/november2015-vandesompel JavaScript makes the link decorations actionable
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Robust Links: Refuse to Die
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Snapshot Approach – Division of Labor
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Managed Collection => Managed Collection
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Cool URI Approach Combat: • Link Rot: Link to B; Redirect to current location • Content Drift: Generic URI; Version URIs With Cool URI links: • Tension between linking to generic URI and version URI
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Robust Links: Refuse to Die
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Cool URI Approach – Division of Labor
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Robust Links Approach  
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Summary    PID RLLabor -
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Robust Links for Linked Data? Sanderson, R., Ciccarese, P., and Young, B. (2017) Web Annotation Vocabulary W3C Recommendation 23 February 2017. https://www.w3.org/TR/annotation-vocab/
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Handling Resource Versions, Captures B B t1 B t2
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Systems with Resource Versions
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 DBpedia Snapshot Archive Using HDT, TPF, Memento Vander Sande, M., Verborgh, R., Hochstenbach, P., and Van de Sompel, H. (2017) Towards sustainable publishing and querying of distributed Linked Data archives. Temporal: subject URI access ; ?s ?p ?o queries ; SPARQL queries
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Memento Tracer http://tracer.mementoweb.org
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Resource Capture: Tension Between Scale and Quality • Web crawling: optimized for scale • Problems with capturing resources accessible via interactive affordances • webrecorder.io: optimized for quality • Personal archiving • User records web navigation session • Not used for archiving at scale • LOCKSS: optimized for scholarly journals • Pages in Publisher/Journal portals share lay-out, affordances • Heuristics per publisher/journal to improve capture quality
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Memento Tracer: New Sweet Spot Between Scale and Quality • ~ web crawling: server side process to capture resources • ~ LOCKSS: leverages insight that web publications in any given portal are based on same template: • share lay-out • share interactive affordances • ~ webrecorder.io: human guidance to achieve quality • But, with Memento Tracer: • user does not record a specific web publication • user records heuristics that apply to a class of web publications
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Memento Tracer
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 A Trace for slideshare Presentations { "portal_url_match": "(slideshare.net)/([^/]+)/([^/]+)", "actions": [{ "action_order": "1", "value": "div.j-next-btn.arrow-right", "type": "CSSSelector", "action": "repeated_click", "repeat_until": { "condition": "changes", "type": "resource_url" } }, { "action_order": "2", "value": "div.notranslate.transcript.add- padding-right.j-transcript a", "type": "CSSSelector", "action": "click" } ], …
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Memento Tracer: Experimental • Promising results, thus far • Currently investigating challenges, including: • User interface to support recording Traces for complex sequences of interactions. • Limitations of the browser event listener approach for recording Traces. • Language used to express Traces. • Organization of the shared repository for Traces. • Selection of a Trace for capturing a web publication in cases where different page layouts and interactive affordances are available for web publications that share a URI pattern.
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Demo: Recording a Trace for a Web Publication https://github.com/www.gorillatoolkit/pkg/mux
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Demo: Capturing another Web Publication Using the Trace https://github.com/mementoweb/node-solid-server
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Demo: Capturing another Web Publication Using the Trace https://github.com/mementoweb/node-solid-server
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Demo: Playing Back the Captured Web Publication Capture of https://github.com/mementoweb/node-solid-server
Herbert Van de Sompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Herbert Van de Sompel Los Alamos National Laboratory @hvdsomp Perseverance on Persistence a future-note about the past

More Related Content

PDF
International Image Interoperability Framework (IIIF). Sharing high resolutio...
byLIBIS
 
PDF
Linked Data at the German National Library
byReinhold Heuvelmann
 
PDF
A Framework for Aggregating Private and Public Web Archives
byjcdl2018
 
PDF
Establishing and Verifying Fixity of Archived Web Pages
bymaturban
 
PDF
Methodological Guidelines for Publishing Linked Data
byBoris Villazón-Terrazas
 
PDF
Early Chinese Periodicals Online (ECPO): From Digitization Towards Open Data....
byMatthias Arnold
 
PDF
GND and URIs: Integration and Identification
byReinhold Heuvelmann
 
PDF
It is hard to compute fixity on archived web pages
bymaturban
 
International Image Interoperability Framework (IIIF). Sharing high resolutio...
byLIBIS
 
Linked Data at the German National Library
byReinhold Heuvelmann
 
A Framework for Aggregating Private and Public Web Archives
byjcdl2018
 
Establishing and Verifying Fixity of Archived Web Pages
bymaturban
 
Methodological Guidelines for Publishing Linked Data
byBoris Villazón-Terrazas
 
Early Chinese Periodicals Online (ECPO): From Digitization Towards Open Data....
byMatthias Arnold
 
GND and URIs: Integration and Identification
byReinhold Heuvelmann
 
It is hard to compute fixity on archived web pages
bymaturban
 

What's hot

PDF
Martin Kaltenböck - OGD Linked Open Government Data
byDanube University Krems, Centre for E-Governance
 
PDF
Using Knowledge Graphs in Data Science - From Symbolic to Latent Representati...
byHeiko Paulheim
 
PDF
New Adventures in RDF2vec
byHeiko Paulheim
 
PDF
From Wikis to Knowledge Graphs
byHeiko Paulheim
 
ODP
Machine Learning with and for Semantic Web Knowledge Graphs
byHeiko Paulheim
 
ODP
Knowledge Graphs on the Web
byHeiko Paulheim
 
PPTX
Registration / Certification Interoperability Architecture (overlay peer-review)
byHerbert Van de Sompel
 
PDF
Knowledge Matters! The Role of Knowledge Graphs in Modern AI Systems
byHeiko Paulheim
 
ODP
Make Embeddings Semantic Again!
byHeiko Paulheim
 
PPT
LOCAH Project and Considerations of Linked Data Approaches
byAdrian Stevenson
 
PDF
Aleksandar Kapisoda: The semantic approach for tracking scientific publications
bySemantic Web Company
 
ODP
Mining the Web of Linked Data with RapidMiner
byHeiko Paulheim
 
PPTX
Enabling Personal Use of Web Archives
byMichele Weigle
 
PDF
The drawbridge to knowledge - Linking scholarly publications and research inf...
byLukas Koster
 
ODP
Machine Learning & Embeddings for Large Knowledge Graphs
byHeiko Paulheim
 
PPT
Open Data - The Fingal Perspective
byFingal Open Data
 
PDF
The Web as infrastructure for scholarly research and communication
byHerbert Van de Sompel
 
Martin Kaltenböck - OGD Linked Open Government Data
byDanube University Krems, Centre for E-Governance
 
Using Knowledge Graphs in Data Science - From Symbolic to Latent Representati...
byHeiko Paulheim
 
New Adventures in RDF2vec
byHeiko Paulheim
 
From Wikis to Knowledge Graphs
byHeiko Paulheim
 
Machine Learning with and for Semantic Web Knowledge Graphs
byHeiko Paulheim
 
Knowledge Graphs on the Web
byHeiko Paulheim
 
Registration / Certification Interoperability Architecture (overlay peer-review)
byHerbert Van de Sompel
 
Knowledge Matters! The Role of Knowledge Graphs in Modern AI Systems
byHeiko Paulheim
 
Make Embeddings Semantic Again!
byHeiko Paulheim
 
LOCAH Project and Considerations of Linked Data Approaches
byAdrian Stevenson
 
Aleksandar Kapisoda: The semantic approach for tracking scientific publications
bySemantic Web Company
 
Mining the Web of Linked Data with RapidMiner
byHeiko Paulheim
 
Enabling Personal Use of Web Archives
byMichele Weigle
 
The drawbridge to knowledge - Linking scholarly publications and research inf...
byLukas Koster
 
Machine Learning & Embeddings for Large Knowledge Graphs
byHeiko Paulheim
 
Open Data - The Fingal Perspective
byFingal Open Data
 
The Web as infrastructure for scholarly research and communication
byHerbert Van de Sompel
 

Similar to Perseverance on Persistence

PPTX
The web is rotting and what to do about it
byHerbert Van de Sompel
 
PPTX
"Web Archive services framework for tighter integration between the past and ...
byAhmed AlSum
 
PDF
En toen was er niets meer ....
byvoginip
 
PPT
Achieving Link Integrity for Managed Collections
byHerbert Van de Sompel
 
PPTX
Memento 101
byHerbert Van de Sompel
 
PPT
Linked Data - the Future for Open Repositories?
byAdrian Stevenson
 
PPT
Digital Preservation at ODU
byJustin Brunelle
 
PPT
Digital Preservation - ODU
byJustin Brunelle
 
PPTX
Reminiscing about interoperability
byHerbert Van de Sompel
 
PPTX
Collaborating web archives - Herbert van de Sompel
byNetwerk Digitaal Erfgoed
 
ODP
Charper.lawdi.20130531
bycharper
 
PPTX
Hiberlink: Prototypes of pro-active approaches to support the archiving of we...
byEDINA, University of Edinburgh
 
PPTX
Prototypes of pro-active approaches to support the archiving of web reference...
byEDINA, University of Edinburgh
 
PPTX
HIBERLINK: Reference Rot and Linked Data: Threat and Remedy
byPRELIDA Project
 
PPTX
Web archiving challenges and opportunities
byAhmed AlSum
 
PDF
ALIAOnline Practical Linked (Open) Data for Libraries, Archives & Museums
byJon Voss
 
PDF
MementoMap: A Web Archive Profiling Framework for Efficient Memento Routing
bySawood Alam
 
PPTX
Reference Rot
byShawn Jones
 
PPTX
2015-odu-ece-tools-for-past-web
byMichele Weigle
 
PPTX
Creating Pockets of Persistence
byHerbert Van de Sompel
 
The web is rotting and what to do about it
byHerbert Van de Sompel
 
"Web Archive services framework for tighter integration between the past and ...
byAhmed AlSum
 
En toen was er niets meer ....
byvoginip
 
Achieving Link Integrity for Managed Collections
byHerbert Van de Sompel
 
Memento 101
byHerbert Van de Sompel
 
Linked Data - the Future for Open Repositories?
byAdrian Stevenson
 
Digital Preservation at ODU
byJustin Brunelle
 
Digital Preservation - ODU
byJustin Brunelle
 
Reminiscing about interoperability
byHerbert Van de Sompel
 
Collaborating web archives - Herbert van de Sompel
byNetwerk Digitaal Erfgoed
 
Charper.lawdi.20130531
bycharper
 
Hiberlink: Prototypes of pro-active approaches to support the archiving of we...
byEDINA, University of Edinburgh
 
Prototypes of pro-active approaches to support the archiving of web reference...
byEDINA, University of Edinburgh
 
HIBERLINK: Reference Rot and Linked Data: Threat and Remedy
byPRELIDA Project
 
Web archiving challenges and opportunities
byAhmed AlSum
 
ALIAOnline Practical Linked (Open) Data for Libraries, Archives & Museums
byJon Voss
 
MementoMap: A Web Archive Profiling Framework for Efficient Memento Routing
bySawood Alam
 
Reference Rot
byShawn Jones
 
2015-odu-ece-tools-for-past-web
byMichele Weigle
 
Creating Pockets of Persistence
byHerbert Van de Sompel
 

More from Herbert Van de Sompel

PPTX
Researcher Pod: Scholarly Communication Using the Decentralized Web
byHerbert Van de Sompel
 
PPTX
Persistent Identification: Easier Said than Done
byHerbert Van de Sompel
 
PPTX
FAIR Signposting: A KISS Approach to a Burning Issue
byHerbert Van de Sompel
 
PPTX
Collecting the organizational scholarly record
byHerbert Van de Sompel
 
PPTX
To the Rescue of Scholarly Orphans
byHerbert Van de Sompel
 
PPTX
Almost two decades at LANL
byHerbert Van de Sompel
 
PPTX
Paul Evan Peters Lecture
byHerbert Van de Sompel
 
PPTX
Signposting Overview (Version November 2017)
byHerbert Van de Sompel
 
PPTX
Signposting Overview
byHerbert Van de Sompel
 
PPTX
PID Signposting Pattern
byHerbert Van de Sompel
 
PPTX
DBpedia Archive using Memento, Triple Pattern Fragments, and HDT
byHerbert Van de Sompel
 
PPTX
Interoperability for web based scholarship
byHerbert Van de Sompel
 
PPTX
ResourceSync Quick Overview
byHerbert Van de Sompel
 
PPTX
A Perspective on Archiving the Scholarly Record
byHerbert Van de Sompel
 
PPTX
ResourceSync Overview
byHerbert Van de Sompel
 
PPTX
Persistent Identifiers and the Web: The Need for an Unambiguous Mapping
byHerbert Van de Sompel
 
PPTX
Hiberlink: Investigating Reference Rot, December 2013
byHerbert Van de Sompel
 
PPTX
ResourceSync tutorial OAI8
byHerbert Van de Sompel
 
PDF
A Clean Slate?
byHerbert Van de Sompel
 
PDF
Paint-Yourself-In-The-Corner Infrastructure
byHerbert Van de Sompel
 
Researcher Pod: Scholarly Communication Using the Decentralized Web
byHerbert Van de Sompel
 
Persistent Identification: Easier Said than Done
byHerbert Van de Sompel
 
FAIR Signposting: A KISS Approach to a Burning Issue
byHerbert Van de Sompel
 
Collecting the organizational scholarly record
byHerbert Van de Sompel
 
To the Rescue of Scholarly Orphans
byHerbert Van de Sompel
 
Almost two decades at LANL
byHerbert Van de Sompel
 
Paul Evan Peters Lecture
byHerbert Van de Sompel
 
Signposting Overview (Version November 2017)
byHerbert Van de Sompel
 
Signposting Overview
byHerbert Van de Sompel
 
PID Signposting Pattern
byHerbert Van de Sompel
 
DBpedia Archive using Memento, Triple Pattern Fragments, and HDT
byHerbert Van de Sompel
 
Interoperability for web based scholarship
byHerbert Van de Sompel
 
ResourceSync Quick Overview
byHerbert Van de Sompel
 
A Perspective on Archiving the Scholarly Record
byHerbert Van de Sompel
 
ResourceSync Overview
byHerbert Van de Sompel
 
Persistent Identifiers and the Web: The Need for an Unambiguous Mapping
byHerbert Van de Sompel
 
Hiberlink: Investigating Reference Rot, December 2013
byHerbert Van de Sompel
 
ResourceSync tutorial OAI8
byHerbert Van de Sompel
 
A Clean Slate?
byHerbert Van de Sompel
 
Paint-Yourself-In-The-Corner Infrastructure
byHerbert Van de Sompel
 

Perseverance on Persistence

  • 1.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Herbert Van de Sompel Los Alamos National Laboratory @hvdsomp Perseverance on Persistence a future-note about the past
  • 2.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 OAI-ORE
  • 3.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 2006 • OAI-ORE observation: Scholarly assets are rapidly becoming compound, consisting of multiple resources with various: • Relationships • Interdependencies • How to convey this compound-ness in an interoperable manner so that applications can access, consume such assets? http://www.openarchives.org/ore/1.0/toc
  • 4.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Address interoperability challenges from the perspective of the web • The resource at the center of the universe • The notion of a repository (or even of a web server) does not exist in the architecture of the web • Neither the notion of a Digital Object • The tools of the interoperability trade are the primitives of the web ORE Insight 1 - Web-Centric Interoperability Paradigm
  • 5.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Tools of the Web-Centric Interoperability Trade • Resource • URI • HTTP as the API: HEAD/GET, POST, PUT, DELETE • Representation • Media Type • Link • Content Negotiation • Typed Link • Controlled Vocabularies for Typed Links W3C Architecture of the World Wide Web RDF, RDFS, OWL
  • 6.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18
  • 7.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 OAI-ORE in EDM Europeana v1.0 2009
  • 8.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 The web-centric ORE approach allowed using off-the-shelf web tools to archive evolving compound objects • Evolving versions of Resource Maps, Aggregated Resources were captured in a web archive • But how to use the URI of the Aggregation or Resource Map to see the status of an Aggregation at a specific moment in the past? ORE Insight 2 – How to Access Temporal State of an Aggregation H. Van de Sompel (2007) Compound Information Object Prototype Demonstration https://www.dropbox.com/s/dd7xd427y90q4jx/CT_Watch_hvds_20070703.mov?dl=0
  • 9.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 H. Van de Sompel, M. L. Nelson, R. Sanderson (2013) RFC7089 - HTTP Framework for Time- Based Access to Resource States – Memento. https://tools.ietf.org/html/rfc7089 Memento
  • 10.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Tools of the Web-Centric Interoperability Trade – HTTP Stack • Resource • URI • HTTP as the API • Representation • Media Types • Link • Content Negotiation, e.g. for preferred Media Type • Typed Link • Controlled Vocabularies for Typed Links W3C Architecture of the World Wide Web HTTP Links, IANA link relation registry, community link relation types HATEOAS – Hypermedia As The Engine Of Application State http://en.wikipedia.org/wiki/HATEOAS
  • 11.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Original Resource and Mementos
  • 12.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Bridge from Present to Past
  • 13.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Bridge from Present to Past
  • 14.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Bridge from Past to Present
  • 15.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 timegate Link: Link to Your Own History Can link to preferred web archive, but also: • Maintain your own resource version history • timegate link to your own history • Distributed management of resource history • Uniform access to resource history across systems • Follow links across systems subject to time
  • 16.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 No timegate Link – Client Intelligence Client uses TimeGate of its preferred web archive, but: • Internet Archive is massive, yet substantial unique materials in other archives • Introduce aggregated TimeGate: Memento Aggregator
  • 17.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Routing TimeGate Requests Using Machine Learning Bornand, N., Balakireva, L., Van de Sompel, H. (2016) Routing Memento Requests Using Binary Classifiers. JCDL16. https://arxiv.org/abs/1606.09136 • Memento Aggregator covers 20+ web archives • Distributed systems problem: As the number of archives (and incoming requests) grows, sending requests to each archive for every incoming request is not feasible • Response times • Load on distributed archives • After various optimization attempts, devised an approach using binary classifiers per web archive: • Trained on the basis of cached URIs, using URI features only • Operational since 2016: 80% reduction in # queries. 1/3 reduction in response times. Recall 85%
  • 18.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 From Internet Archive TodayToday Select Date Mar 20 2007 Apr 03 2007 Various Memento Tools (client/server) https://github.com/machawk1/awesome-memento
  • 19.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Pockets of Persistence
  • 20.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Creating Pockets of Persistence • With Memento’s time travel capability in place, what would it take to support faithfully navigating the web of the Past? • There are two major forces that hinder achieving this goal: • Link rot: A link stops working all together • Content drift: The linked content changes over time and may eventually no longer be representative of the content that was originally linked • Without these forces at work, the web of the Present would be the same as the web of the Past • But that clearly is not the case
  • 21.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Hyperlinks in Theory
  • 22.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Hyperlinks in Reality
  • 23.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Hyperlinks in Reality
  • 24.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Link Rot
  • 25.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Link Rot - PMC Martin Klein, Herbert Van de Sompel, Robert Sanderson, Harihar Shankar, et al. (2014) Scholarly context not found. In: PLOS ONE https://doi.org/10.1371/journal.pone.0115253
  • 26.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Hyperlinks in Reality
  • 27.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Content Drift
  • 28.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Content Drift
  • 29.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Content Drift http://icecube.wisc.edu/ on May 8 2009 (left) and August 27 2009 (right)
  • 30.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 No Content Drift http://www.ifa.hawaii.edu/~cowie/k_table.html on June 9 1997 (left) and March 2016 (right)
  • 31.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Content Drift - PMC Shawn Jones, Herbert Van de Sompel, Harihar Shankar, Martin Klein, et al. (2016) Scholarly context not found. In: PLOS ONE https://doi.org/10.1371/journal.pone.0167475
  • 32.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Creating Pockets of Persistence • What would it take to really support faithfully navigating the web of the Past? • This challenge exists for the entire web. Some communities with well managed collections care about addressing it: • Scholarly communication • Cultural heritage • Legal publications • Journalism • Wikipedia • Why? • Link Rot: Quality of Service • Content Drift: integrity of the record, reliable evidence, revisiting the state of knowledge, transparency of editorial process, …
  • 33.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 US Supreme Court Opinion – Link Rot Activism http://ssnat.com
  • 34.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Two Types of Links from a Managed Collection
  • 35.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Take 1 – PID Approach PID for B
  • 36.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Managed Collection => Managed Collection
  • 37.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 PID Approach Combat: • Link Rot: Link to PID; Redirect to current location • Content Drift: Mint a PID per version; Link to version PID With PID links: • Web of Present = Web of Past
  • 38.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18
  • 39.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 URI References - PMC Herbert Van de Sompel, Martin Klein, and Shawn Jones (2016) Persistent URIs Must Be Used to Be Persistent. In: WWW2016. http://arxiv.org/1602.09102 Herbert Van de Sompel, Martin Klein, and Shawn Jones (2016) Persistent URIs Must Be Used to Be Persistent. In: WWW2016. http://arxiv.org/1602.09102
  • 40.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 cite-as Relation Type Herbert Van de Sompel et al. (2018) cite-as: A Link Relation to Convey a Preferred URI for Referencing. https://datatracker.ietf.org/doc/draft-vandesompel-citeas/ http://signposting.org
  • 41.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 PID Approach – Division of Labor
  • 42.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Managed Collection => Web at Large
  • 43.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18
  • 44.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 PID Approach  -
  • 45.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Take 2 – Robust Links Approach
  • 46.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Managed Collection => Web at Large
  • 47.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Snapshot Approach Combat: • Link Rot & Content Drift: Custodian of A creates snapshot of B, in web archive or locally Regarding links: • Intuition suggests linking to the snapshot of B …
  • 48.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Linking to Snapshot of B = Potentially Creating a Rotten Link • Existing practice for linking to snapshots: <a href=“URL of snapshot of B”> • Problems with existing practice: o Impossible to visit the original URI, if desired o Requires the permanent existence/uptime of the archive that holds the snapshot - One link rot problem replaced by another http://robustlinks.mementoweb.org/about/
  • 49.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Permanent Existence/Uptime of Archives? Remnant of discontinued web archive http://mummify.it captured on February 14 2014 https://web.archive.org/web/20140214233752/https://www.mummify.it/
  • 50.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Permanent Existence/Uptime of Archives? http://www.themoscowtimes.com/news/article/russia-bans-wayback-machine-internet-archive-over- islamic-state-video/510074.html
  • 51.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Permanent Existence/Uptime of Archives? http://web.archive.org/web/20121101043952/http://vogin.nl on March 6 2017 at 15:59 CET
  • 52.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Decorate the Link • Proposed practice for linking to captures: <a href=“URL of snapshot of B” data-originalurl=“B” data-versiondate=“datetime of snapshot of B”> <a href=“B” data-versionurl=“URL of snapshot of B” data-versiondate=“datetime of snapshot of B”> http://robustlinks.mementoweb.org/spec/
  • 53.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Robust Links: Link Decoration in Action Van de Sompel H. & Nelson, M.L. (2015) Reminiscing about 15 years of interoperability efforts. In: D-Lib Magazine. https://doi.org/10.1045/november2015-vandesompel JavaScript makes the link decorations actionable
  • 54.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Robust Links: Refuse to Die
  • 55.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18
  • 56.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Snapshot Approach – Division of Labor
  • 57.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Managed Collection => Managed Collection
  • 58.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Cool URI Approach Combat: • Link Rot: Link to B; Redirect to current location • Content Drift: Generic URI; Version URIs With Cool URI links: • Tension between linking to generic URI and version URI
  • 59.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Robust Links: Refuse to Die
  • 60.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18
  • 61.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Cool URI Approach – Division of Labor
  • 62.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Robust Links Approach  
  • 63.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Summary    PID RLLabor -
  • 64.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Robust Links for Linked Data? Sanderson, R., Ciccarese, P., and Young, B. (2017) Web Annotation Vocabulary W3C Recommendation 23 February 2017. https://www.w3.org/TR/annotation-vocab/
  • 65.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Handling Resource Versions, Captures B B t1 B t2
  • 66.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Systems with Resource Versions
  • 67.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 DBpedia Snapshot Archive Using HDT, TPF, Memento Vander Sande, M., Verborgh, R., Hochstenbach, P., and Van de Sompel, H. (2017) Towards sustainable publishing and querying of distributed Linked Data archives. Temporal: subject URI access ; ?s ?p ?o queries ; SPARQL queries
  • 68.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Memento Tracer http://tracer.mementoweb.org
  • 69.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Resource Capture: Tension Between Scale and Quality • Web crawling: optimized for scale • Problems with capturing resources accessible via interactive affordances • webrecorder.io: optimized for quality • Personal archiving • User records web navigation session • Not used for archiving at scale • LOCKSS: optimized for scholarly journals • Pages in Publisher/Journal portals share lay-out, affordances • Heuristics per publisher/journal to improve capture quality
  • 70.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Memento Tracer: New Sweet Spot Between Scale and Quality • ~ web crawling: server side process to capture resources • ~ LOCKSS: leverages insight that web publications in any given portal are based on same template: • share lay-out • share interactive affordances • ~ webrecorder.io: human guidance to achieve quality • But, with Memento Tracer: • user does not record a specific web publication • user records heuristics that apply to a class of web publications
  • 71.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Memento Tracer
  • 72.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 A Trace for slideshare Presentations { "portal_url_match": "(slideshare.net)/([^/]+)/([^/]+)", "actions": [{ "action_order": "1", "value": "div.j-next-btn.arrow-right", "type": "CSSSelector", "action": "repeated_click", "repeat_until": { "condition": "changes", "type": "resource_url" } }, { "action_order": "2", "value": "div.notranslate.transcript.add- padding-right.j-transcript a", "type": "CSSSelector", "action": "click" } ], …
  • 73.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Memento Tracer: Experimental • Promising results, thus far • Currently investigating challenges, including: • User interface to support recording Traces for complex sequences of interactions. • Limitations of the browser event listener approach for recording Traces. • Language used to express Traces. • Organization of the shared repository for Traces. • Selection of a Trace for capturing a web publication in cases where different page layouts and interactive affordances are available for web publications that share a URI pattern.
  • 74.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Demo: Recording a Trace for a Web Publication https://github.com/www.gorillatoolkit/pkg/mux
  • 75.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Demo: Capturing another Web Publication Using the Trace https://github.com/mementoweb/node-solid-server
  • 76.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Demo: Capturing another Web Publication Using the Trace https://github.com/mementoweb/node-solid-server
  • 77.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Demo: Playing Back the Captured Web Publication Capture of https://github.com/mementoweb/node-solid-server
  • 78.
    Herbert Van deSompel @hvdsomp EuropeanaTech 2018, Rotterdam, The Netherlands, 15/05/18 Herbert Van de Sompel Los Alamos National Laboratory @hvdsomp Perseverance on Persistence a future-note about the past