This document describes Doc2Graph, an open source tool that transforms JSON documents into a graph database. It discusses how Doc2Graph works, including converting JSON trees into a graph and reusing existing nodes. It also provides examples of using Doc2Graph with CouchbaseDB, MongoDB, and the Spotify API to import music data into Neo4j. The document concludes with information on Doc2Graph's configuration options.

2. SINCE 2013

3. NEO4J JDBC DRIVER

4. APOC Developers
Awesome
Procedures
On
Cypher

5. Graph analysis over JSON data
WEB API
JSON FILE

6. CONTENTS
• Document oriented DB and web API
• Polyglot persistence
• JSON
• How Doc2Graph works
• Connectors (CouchbaseDB, MongoDB)
• Spotify API example
• Configuration

7. Document oriented DBs and web APIs

8. Polyglot persistence
Document is an aggregation of information: data are duplicated
Graph is a relationship-oriented structure: data are connected
Different approach for different use case

9. JSON
Every (JSON) document is a tree structure
Every tree structure is a sub graph
Trees mixed together are a graph

10. JSON
{ id : “123”,
name: “doc”,
internalDoc : {
id: “abc”,
name : “iDoc”,
languages: [“EN”,”IT”],
}
}
id: “123”
name: “doc”
id: “abc”
name: “iDoc”
languages:
[‘EN’,’IT’]
[:IN_DOC]

11. JSON
{ id : “123”,
name: “doc”,
internalDoc : {
id: “abc”,
name : “iDoc”,
languages: [“EN”,”IT”],
},
refDoc : {
id: “efg”,
name : “extDoc”,
}
}
id: “123”
name: “doc”
id: “abc”
name: “iDoc”
languages:
[‘EN’,’IT’]
[:IN_DOC]
id: “efg”
name:“extDoc
[:IN_DOC]

12. Tree

13. Graph

14. Doc2Graph
• Automatic synch
• Customizable Transformation Logic
• Re-use existing nodes
• Import JSON from any kind of data source
• Open Source

15. Re-using existing nodes
id: “123”
name: “doc”
id: “abc”
name: “iDoc”
languages:
[‘EN’,’IT’]
[:IN_DOC]
id: “456”
name: “other”
[:IN_DOC]

16. How it works
A core installed into Neo4j: neo4j-json
A optional jar with custom rules
A configuration node
A connector to external data source

17. How it works
CALL json.upsert(‘document-key’,’{json: …}’);
CALL json.delete(‘document-key’);

18. CouchbaseDB connector

19. MongoDB connector

20. Spotify API example
https://api.spotify.com/v1/artists/{id}
https://api.spotify.com/v1/albums/{id}
https://api.spotify.com/v1/tracks/{id}

21. Spotify API example
1. neo4j start
2. java -jar target/neo4j-connector.jar -Dconf.file=neo4j-sync.propeties
3. ./cbdocloader -n localhost:8091 -u ad -p ps -b spotify artists.zip
4. ./cbdocloader -n localhost:8091 -u ad -p ps -b spotify tracks.zip
5. ./cbdocloader -n localhost:8091 -u ad -p ps -b spotify albums.zip

22. Spotify API example
{ "id" : "00D4HkSczu4DJP5PCkVOiX",
"name" : "Spaceship Coupe",
"type" : “track”,
"album" : {
"name" : "The 20/20 Experience - The Complete Experience",
"type" : “album",
"id" : "3Xli4SFIUo8GAr4f6yORHr",
"artists" : [ {
"id" : "31TPClRtHm23RisEBtV3X7",
"name" : "Justin Timberlake",
"type" : "artist",
} ],
"artists" : [ {
"id" : "31TPClRtHm23RisEBtV3X7",
"name" : "Justin Timberlake",
"type" : “artist"
} ]
}

23. Spotify API example
match p=(daddy:Artist {name:'Daddy Yankee'})--(alb:Album)--(oth:Artist)
return p

24. Spotify API example
{
"id" : "omar",
"type" : "user",
"name" : "Omar Rampado",
"preferred_album" : {
"id" : "3HArmgjL01OKFTLfU1HJiQ",
"name" : "Otra Cosa",
"type" : "album",
"href" : "https://api.spotify.com/v1/albums/3HArmgjL01OKFTLfU1HJiQ",
"uri" : "spotify:album:3HArmgjL01OKFTLfU1HJiQ",
"album_type" : "single",
"available_markets" : [“CO","DO","EC","PE","US"]
}
}

25. Spotify API example
match p=(daddy:Artist {name:'Daddy Yankee'})--(alb:Album)--(oth:Artist)
, (omar:User {id: 'omar'})--(alb)
return p, omar

26. Configuration
CREATE (n:JSON_CONFIG {
configuration: 'byNode'
,root_node_key_property:'_document_key'
,document_default_label:'DocNode'
,document_id_builder:'org.neo4j.helpers.json.document.impl.DocumentIdBuilderTypeId'
,document_relation_builder:'org.neo4j.helpers.json.document.impl.DocumentRelationBuilderH
asTypeArrayKey'
,document_label_builder:'org.neo4j.helpers.json.document.impl.DocumentLabelBuilderByType'
,log_discard_events: false
})

27. LINKS
http://www.larus-ba.it/neo4j/en/projects/doc2graph/
https://github.com/larusba/doc2graph