Collaboration is one of the most important topics regarding the evolution of the World Wide Web and thus also for the Web of Data. In scenarios of distributed collaboration on datasets it is necessary to provide support for multiple different versions of datasets to exist simultaneously, while also providing support for merging diverged datasets. In this paper we present an approach that uses SPARQL 1.1 in combination with the version control system Git, that creates commits for all changes applied to an RDF dataset containing multiple named graphs. Further the operations provided by Git are used to distribute the commits among collabora-tors and merge diverged versions of the dataset. We show the advantages of (public) Git repositories for RDF datasets and how this represents a way to collaborate on RDF data and consume it. With SPARQL 1.1 and Git in combination, users are given several opportunities to participate in the evolution of RDF data. Natanael Arndt | AKSW, Universität Leipzig Presentation at Semantics 2016 in Leipzig in the context with the results of the LEDS project

1. Distributed Collaboration on RDF Datasets
Using Git
Towards the Quit Store
Natanael Arndt, Norman Radtke and Michael Martin
SEMANTiCS 2016, Leipzig
September 14, 2016

2. Problem & Motivation
2 / 23

3. Problem & Motivation
Linked Datasets as of August 2014
Enterprise Workspace
clone
enrich
😱
Public LOD Cloud
3 / 23

4. Problem & Motivation
Remark (Co-Evolution)
The process of datasets simultaneously evolving separated from each
other while influencing each others evolution
4 / 23

5. Problem & Motivation
Usage of public LOD as background knowledge
Mobile use cases
In distributed collaboration on RDF datasets
⇒ Support for multiple versions of the same dataset at the same
time
5 / 23

6. Approach
The same problem exists for
source code repositories
Since around 10 years
distributed version control
system are solving this
problem
Multiple working copy exist at
the same time and can be
synchronized
Server/Client Server/Client
Server/Client
Server/Client
Server/Client
6 / 23

7. Approach
Git is successful in software
development
Server/Client Server/Client
Server/Client
Server/Client
Server/Client
7 / 23

8. Approach
Git is successful in software
development
We have decided to see if this
also works for RDF
Server/Client Server/Client
Server/Client
Server/Client
Server/Client
7 / 23

9. Approach
Git is successful in software
development
We have decided to see if this
also works for RDF
So we have put RDF into the
repositories
Server/Client Server/Client
Server/Client
Server/Client
Server/Client
7 / 23

10. Methodology
Quit Store
RDF
Quad Store
SPARQL 1.1 Interface
Query & Update Read/write interface
Translating read/write
operations to versioning
Synchronizes the store with
the current working copy
8 / 23

11. Methodology:
Serialization of RDF data
Multiple RDF serialization formats are available
For the versioning with Git we need:
Same RDF graph = same representation
Minimal difference between versions
Meaningful difference between version
⇒ We have chosen a canonicalized N-Quads serialization
9 / 23

12. Methodology:
Blank Nodes in Versioning
With RDF as exchange format, still blank nodes are a problem
Blank nodes identifiers only have a local scope
… are not persistent or portable identifiers
… are purely an artifact of the serialization
([Cyganiak et al., 2014] sections 3.4 and 3.5)
10 / 23

13. Methodology:
Blank Nodes in Versioning
With RDF as exchange format, still blank nodes are a problem
Blank nodes identifiers only have a local scope
… are not persistent or portable identifiers
… are purely an artifact of the serialization
We follow the recommendation of RDF 1.1, to replace blank
nodes with IRIs
([Cyganiak et al., 2014] sections 3.4 and 3.5)
10 / 23

14. Methodology:
Read/Write Interface
Quit Store
RDF
Quad Store
SPARQL 1.1 Interface
Query & Update
SPARQL 1.1 Query and Update
Query proxy providing a
SPARQL endpoint
Executes Queries on the Store
Triggers read or write
operations on the versioning
layer
11 / 23

15. Methodology:
Translating Read/Write Operations
Quit Store
RDF
Quad Store
SPARQL 1.1 Interface
Query & Update
SPARQL read/write operations
are transformed to commit,
merge, revert, push and pull
12 / 23

16. Methodology:
Commit
Is triggered by UPDATE Queries
The changed graphs are added and commited in a new Git
commit
A Commit contains lines resp. statements added/removed
A B
13 / 23

17. Methodology:
Commit
A commit is always referring to its predecessor not vice versa
We can also create two commits with the same predecessor
A B C
14 / 23

18. Methodology:
Commit
A commit is always referring to its predecessor not vice versa
We can also create two commits with the same predecessor
Branching/Forking
A B C
D
14 / 23

19. Methodology:
Merge
If the commits are diverged we need to synchronize the versions
A B C
D
15 / 23

20. Methodology:
Merge
If the commits are diverged we need to synchronize the versions
Create a commit with two predecessors
Still we need to actually consolidate the graphs
A B C
D
E
15 / 23

21. Methodology:
Merge
Using the default three-way-merge from git
<urn:ex:Tilia> a <urn:ex:Tree> .
<urn:ex:Tilia> <urn:ex:age> "1000"^^xsd:integer .
<urn:ex:Tilia> <urn:ex:label> "Linda"@de .
<urn:ex:Tilia> <urn:ex:label> "Tilia"@en .
16 / 23

22. Methodology:
Merge
Using the default three-way-merge from git
On syntactical level Git produces conflicts
Branch A
<urn:ex:Tilia> a <urn:ex:Tree> .
<urn:ex:Tilia> <urn:ex:age> "1000"^^xsd:integer .
+ <urn:ex:Tilia> <urn:ex:height> "40"^^xsd:integer .
<urn:ex:Tilia> <urn:ex:label> "Linda"@de .
<urn:ex:Tilia> <urn:ex:label> "Tilia"@en .
Branch B
<urn:ex:Tilia> a <urn:ex:Tree> .
<urn:ex:Tilia> <urn:ex:age> "1000"^^xsd:integer .
+ <urn:ex:Tilia> <urn:ex:label> "Linde"@de .
- <urn:ex:Tilia> <urn:ex:label> "Linda"@de .
<urn:ex:Tilia> <urn:ex:label> "Tilia"@en .
16 / 23

23. Methodology:
Merge
Using the default three-way-merge from git
On syntactical level Git produces conflicts
Git Merge:
<urn:ex:Tilia> a <urn:ex:Tree> .
<urn:ex:Tilia> <urn:ex:age> "1000"^^xsd:integer .
<<<<<<< HEAD
<urn:ex:Tilia> <urn:ex:height> "40"^^xsd:integer . =======
<urn:ex:Tilia> <urn:ex:label> "Linda"@de . <urn:ex:Tilia> <urn:ex:label> "Linde"@de .
>>>>>>> typo
<urn:ex:Tilia> <urn:ex:label> "Tilia"@en .
16 / 23

24. Methodology:
Merge
Using the default three-way-merge from git
On syntactical level Git produces conflicts
But actually there is no conflict
Conflicts have to be looked for on other levels
<urn:ex:Tilia> a <urn:ex:Tree> .
<urn:ex:Tilia> <urn:ex:age> "1000"^^xsd:integer .
<urn:ex:Tilia> <urn:ex:height> "40"^^xsd:integer .
<urn:ex:Tilia> <urn:ex:label> "Linde"@de .
<urn:ex:Tilia> <urn:ex:label> "Tilia"@en .
16 / 23

25. Methodology:
Revert
Reverting a commit undoes an earlier change
This is done by exchanging the add- and delete-set of statements
A B B−1
17 / 23

26. Implementation
File References
SPARQL 1.1 Interface
Public Git Repository
Local Git Repository
Query-Analyzer
Quad-Store
SPARQL Query
Update
Dump to files
Select
Parse files
Response
Written in Python, using Flask API as HTTP interface and RDFlib for
SPARQL and RDF
18 / 23

27. Integration
Quit Store has the role of managing the repository
Provide the read/write interface
Synchronize the repository and the store.
Quit Store
Quit Diff Δ( , )
Quit Merge
60%
Quit Notify
Quit Store
RDF
Quad Store
SPARQL 1.1 Interface
Query & Update
19 / 23

28. Integration
Quit Diff can calculate differences between commits
Trace provenance of statement
Transmit patches to collaborators.
Quit Store
Quit Diff Δ( , )
Quit Merge
60%
Quit Notify
Quit Store
RDF
Quad Store
SPARQL 1.1 Interface
Query & Update
19 / 23

29. Integration & Future Work
Quit Notify can actively inform other clones of updates
This enables distributed setups for collaboration and
synchronization.
Quit Store
Quit Diff Δ( , )
Quit Merge
60%
Quit Notify
Quit Store
RDF
Quad Store
SPARQL 1.1 Interface
Query & Update
20 / 23

30. Integration & Future Work
Quit Merge will implement various merge strategies for RDF
Detect conflicts in diverged versions.
Quit Store
Quit Diff Δ( , )
Quit Merge
60%
Quit Notify
Quit Store
RDF
Quad Store
SPARQL 1.1 Interface
Query & Update
20 / 23

31. Poster
Quit DiffNatanael Arndt
Norman Radtke
?LEDS
INKED NTERPRISE ATA ERVICESL E D S
P 34 21 / 23

32. Conclusion
With Quit we have presented a
methodology for
version control and tracking
provenance of contributions,
synchronization: clone, push and pull
by other participants, and
distributed collaboration on RDF
datasets (gitflow)
Hopefully this can help to utilize the
big ecosystem of methodologies and
tools around Git
Quit Store
Quit Diff Δ( , )
Quit Merge
60%
Quit Notify
Quit Store
RDF
Quad Store
SPARQL 1.1 Interface
Query & Update
22 / 23

33. Conclusion
With Quit we have presented a
methodology for
version control and tracking
provenance of contributions,
synchronization: clone, push and pull
by other participants, and
distributed collaboration on RDF
datasets (gitflow)
Hopefully this can help to utilize the
big ecosystem of methodologies and
tools around Git
Questions?
Natanael Arndt
<arndt@informatik.uni-leipzig.de>
Quit Store
Quit Diff Δ( , )
Quit Merge
60%
Quit Notify
Quit Store
RDF
Quad Store
SPARQL 1.1 Interface
Query & Update
22 / 23

34. References I
Cyganiak, R., Wood, D., and Lanthaler, M. (2014).
Rdf 1.1 concepts and abstract syntax.
https://www.w3.org/TR/2014/REC-rdf11-concepts-20140225/.
23 / 23