The Promise of a Better Connected Digital World:
Data Registry Systems Without the Web
Philippe Cudré-Mauroux
eXascale Infolab, University of Fribourg
Switzerland
Christophe Guéret
VU University / DANS
The Netherlands
Verisign Labs Distinguished Speakers Series
Verisign Labs, Reston–USA
December 13, 2013

Entities
2

Entity Data
• Semi-structured,
interlinked
descriptions of
shared instances
–
–
–
–
–
–
Persons
Objects
Software
Locations
Sensors
…
3

Entities as Mediation
• Rising paradigm
– Store information at the entity granularity
– Integrate information by inter-linking entities
• Advantages?
– Coarser granularity compared to keywords
• More natural, e.g., brain functions similarly (or is it the other way
around?)
– Denormalized information compared to RDBMSs
• Schema-later, heterogeneity, sparsity
• Pre-computed joins, “Semantic” linking
• Drawbacks?
4

Prominence of Entity-Powered Apps
–
–
–
–
–
–
–
–
–
–
Collaborative Editing (Wikipedia’s wikidata)
Social Networks (Facebook’s Open Graph)
Serious Networks (LinkedIn’s Business Graph)
Web Search (Google’s Knowledge Graph)
Software Integration (Yahoo!’s WOO)
Question Answering (IBM’s Watson)
Dynamic Websites (BBC’s London Olympics)
Open Data (data.gov.uk, linkeddata.org)
Most of our own applications (exascale.info)
etc. etc.
5

Problem: Limited Access to Entities (1)
• 70+% of the world’s population has no or
very limited access to the Web
[Ahmed Shams 2013]
6

Problem: Limited Access to Entities (2)
• Even in developed countries, deploying
collaborative entity-editing platforms is
technically exceedingly challenging
– Local/Global QoS to serve arbitrary entity data
• Performance, scale-out
– Collaborative aspects
• Transactions, versioning, integration
– Offline / mobile concerns
• Caching / replication / serializability
7

Potential Building Blocks?
• … for a hybrid online/offline, collaborative entity
registry:
–
–
–
–
–
–
–
DNS3 (never meant for entity data)
DOA (awkward Web integration, limited features)
RDBMSs (ACID? Impedance mismatch, limited perf.)
P2P / decentralized CDNs (performance issues)
Native RDF Stores (too expressive; scalability / perf. issues)
(Structured) Inverted Indices (no transactions; slow updates)
noSQL key-value / document stores
(wrong PACELC trade-offs; (some) performance issues)
[Iliya Enchev 2012
ISWC 2013]
8

Our Solution: ERS, the
Entity Registry System
• Three-tier solution to deploy entity-powered apps
– Flexible
• Seamlessly reconcile entities in local / ad-hoc / global modes
– Collaborative
• Transactional consistency, data versioning
– Scalable
• Bridges, scale-out servers, tunable consistency
– Open-source
• https://github.com/ers-devs
9

ERS Architecture (1)
• Contributors: Contributors read and edit the contents of
the registry. They may create and delete entities, look for
entities, and contribute to the entities’ descriptions.
• Bridges: Bridges do not directly contribute to the
contents of the registry. They are used to connect
isolated closed networks and improve the availability of
the descriptions shared by the contributors.
• Aggregators: Some use-cases may require the
presence of global servers that contains a copy of all the
data provided individually by the contributors. The global
server provides a single entry point to the registry.
10

ERS Architecture (2)
www
www
11

Sample Deployment [Videos]
12

ERS Data & API
• Data: flexible RDF quads serialized as
– JSON documents (contributors, bridges)
– Key, value pairs (aggregators)
• Atomic & serializable operations through
various locking granularities
– Insert entity (IE), Insert property (IP), Update property (UP), Delete
property (DP), Delete entity (DE), Shallow entity copy (SC), Deep entity
copy (DC), Insert link between two entities (IL), Delete link between two
entities (DL)
=> Consistency
13

Unique Technical Features
I.
Seamless, best-effort entity synchronization
– Local, ad-hoc, global modes
II.
Fault-tolerance and decentralization
– Property replication, no single point of failure
III. Built-in versioning and provenance
– Collaborative entity editing made easy
IV. Linear scalability
– Tunable consistency levels
14

Performance: Distributed Locking (1)
• Decentralized, multi-granular locking protocol
for transactional consistency on top of
persistency layers
15

Performance: Distributed Locking (2)
• Fault-tolerant, though Paxos-like algorithms
limit horizontal scalability
16

Performance: Optimistic Concurrency (1)
• ERS typically operates on insert-heavy low-conflict workloads
– Most of the time new entities are inserted and properties added
• Goal: separate validation from write operations
– Per worker TX management
– Distributed ID generator for consistent commits
17

Tunable Consistency in ERS
• Weak Writes: For each TX a CID is acquired and a new record
is written, write is not validated
• Strong Writes: For each TX a CID is acquired and a new
record is written. After the write, a read verifies the visibility; if
the record is not visible the write is performed again
• Write validation: Forward chaining of records based on the
highest CID, last writer wins
Inserting 111 is
possible using weak
111
writes, but the write
cannot be validated
18

OC – Execution Stack
• Breakdown of a single
write operation with
tunable consistency

Performance: Optimistic Concurrency (2)
=> Linear scalability even for write-heavy workloads
20

Ongoing Deployments (1)
• Swiss-Dutch local/global social messaging
Cloud hosted Aggregator
Bridge in
Amsterdam
(VUA)
Contributors on
VUA internal
network
Bridge in
Switzerland
(Exascale)
Contributors on
Exascale Infolab
internal network
21

Ongoing Deployments (2)
• Test deployments on new affordable
devices
SmilePlug
(cloud-based learning)
Wandboard
(ultra low power computer)
Earl (backcountry survival tablet)

Ongoing Deployments (3)
• Entity-powered apps for the Sugar Learning
Platform
23

Ongoing Deployments (4)
• ERS for Ambient Assisted Living of elderly
persons in tropical environments
[AAL research group @ VU]
24

Conclusions
• The Web is becoming entity-centric
–
–
Land of opportunities for new registries
Urgent needs for developing countries
• ERS is a unique, open-source entity registry solution
supporting
–
–
–
–
Local / ad-hoc / global modes
Collaborative editing and entity versioning
Tunable consistency levels
Linear scalability
• Series of ongoing deployments
–
Stay tuned for more results and lessons learnt

Big Thanks to the whole ERS Team
Dutch Team @ DANS
Dr. Christophe Guéret
Swiss team @ XI
Prof. Dr. Philippe
Cudré-Mauroux
Dr. Marat Charlaganov
C. Dinu & Pepijn Kroes
Dr. Martin Grund
Teodor Macicas
… and to our MSc students:
–
Iliya Enchev and Ahmed Shams

And Special Thanks to…
• Scott Hollenbeck, Debra Anderson, Allison
Mankin & the Internet Infrastructures Grant
team
• Dr. Burt Kaliski and his team
• Vincenzo Russo, Benoit Perroud, Romain
Cholat and the whole Verisign Fribourg office
… for their continued support

References
•
P. Cudré-Mauroux, G. Demartini, D.E. Difallah, A.E. Mostafa, V. Russo, and M. Thomas.
A Demonstration of DNS3: a Semantic-Aware DNS Service. ISWC 2011.
•
P. Cudré-Mauroux, G. Demartini, I. Enchev, C. Gueret and B. Perroud: Downscaling
Entity Registries for Ad-Hoc Environments. Downscale 2012.
•
M. Charlaganov, P. Cudré-Mauroux, C. Dinu, C. Guéret, M. Grund, T. Macicas:
Demonstrating The Entity Registry System: Implementing 5-Star Linked Data Without the
Web. ISWC 2013.
•
P. Cudré-Mauroux, I. Enchev, S. Fundatureanu, P.T. Groth, A. Haque, A. Harth, F.
Keppmann, D.P. Miranker, J. Sequeda, M. Wylot: NoSQL Databases for RDF: An
Empirical Evaluation. ISWC 2013.
•
M. Charlaganov, P. Cudré-Mauroux, C. Dinu, C. Guéret, M. Grund, T. Macicas: The Entity
Registry System: Implementing 5-Star Linked Data Without the Web. CoRR abs 2013.
•
M. Charlaganov, P. Cudré-Mauroux, C. Dinu, C. Guéret, M. Grund, P. Kroes, and T.
Macicas: Collaboratively Editing an Entity Registry in Poorly Connected Environments.
CAiSE 2014 [submitted].
28

Further Entity Research @ XI
•
R. Prokofyev, G. Demartini and P. Cudré-Mauroux: Effective Named Entity Recognition
for Idiosyncratic Web Collections. WWW 2014.
•
G. Demartini, D.E. Difallah., and P. Cudré-Mauroux: Large-scale linked data integration
using probabilistic reasoning and crowdsourcing. The VLDB Journal, 2013.
•
A. Tonon, M. Catasta, G. Demartini, P. Cudré-Mauroux, and K. Aberer: TRank: Ranking
Entity Types Using the Web of Data. ISWC 2013.
•
A. Tonon, G. Demartini, and P. Cudré-Mauroux: Combining inverted indices and
structured search for ad-hoc object retrieval. SIGIR 2012.
•
G. Demartini, D.E. Difallah, and P. Cudré-Mauroux: ZenCrowd: leveraging probabilistic
reasoning and crowdsourcing techniques for large-scale entity linking. WWW 2012.
29

Thanks a lot for your attention
http://exascale.info
30