A Walk in Graph DatabasesPierre De Wilde4 May 2012Global Brain InstituteVUB - ECCO Group
The Law of the Hammer       If the only tool you have is a hammer,             everything looks like a nail.          Abra...
The Law of the Relational Database    If the only tool you have is a relational database,               everything looks l...
doesntOne size fits all                    Scalability issue                      Scale up                      Scale out ...
NoSQL ?! No SQL ? Not only SQL !                Scalability solutions                  Key-value stores                  C...
Query language for relational databases                  SQL               ISUD or CRUD
Traversal           graphQuery language for relational databases       Gremlin is a graph traversal language
Map of walk              Warm up                Property Graph                Graph Database              Walk with Gremli...
Map of walk              Warm up                Property Graph                Graph Database              Walk with Gremli...
Graph        G = (V, E)                 --. .-. .- .--. ....
One graph doesnt fit all    Marko A. Rodriguez and Peter Neubauer - Constructions from Dots and Lines - 2010
Property graph    A property graph is a directed, labeled, attributed, multi graph.
Anatomy of a vertex                      A vertex is composed of                      - an unique identifier (id)         ...
Anatomy of an edge                     An edge is composed of                     - an unique identifier (id)             ...
Map of walk              Warm up                Property Graph                Graph Database              Walk with Gremli...
Graph database
Key feature of a graph database      Index-free adjacency
Some graph database vendors                   Neo4j from Neo Technology                   http://neo4j.org/               ...
Map of walk              Warm up                Property Graph                Graph Database              Walk with Gremli...
TinkerPop      Open source project in the graph space
TinkerPop family          https://github.com/tinkerpop
Gremlin                           $ gremlin.sh                                    ,,,/                                    ...
Map of walk              Warm up                Property Graph                Graph Database              Walk with Gremli...
Connect to a graph database                    gremlin> g = new TinkerGraph(name)                    gremlin> g = new Neo4...
Add a vertex / an edge                     gremlin> v1 = g.addVertex()                     gremlin> v2 = g.addVertex()    ...
Update a vertex                  gremlin> v = g.getVertex(1)                  ==>v[1]                  gremlin> v.getPrope...
Update an edge                 gremlin> e = g.getEdge(8)                 ==>e[8][1-knows->4]                 gremlin> e.ge...
Remove a vertex                  gremlin> v = g.getVertex(3)                  ==>v[3]                  gremlin> g.removeVe...
Remove an edge                 gremlin> e = g.getEdge(10)                 ==>e[10][4-created->5]                 gremlin> ...
Disconnect from the graph database                   gremlin> g.shutdown()
Map of walk              Warm up                Property Graph                Graph Database              Walk with Gremli...
Graph traversal                  Jump                  - from vertex to edge                  - from edge to vertex       ...
Graph traversal: starting the traversal                      gremlin> g.v(1)                      ==>v[1]
Graph traversal: outgoing edges                    gremlin> g.v(1).outE                    ==>e[7][1-knows->2]            ...
Graph traversal: incoming vertices                    gremlin> g.v(1).outE.inV                    ==>v[2]                 ...
Graph traversal: outgoing edges (cont.)                     gremlin> g.v(1).outE.inV.outE                     ==>e[10][4-c...
Graph traversal: incoming vertices (cont.)                     gremlin> g.v(1).outE.inV.outE.inV                     ==>v[...
Graph traversal: ending the traversal                     gremlin> g.v(1).outE.inV.outE.inV.outE
Graph traversal: starting vertex                     gremlin> g.v(1)                     ==>v[1]
Graph traversal: adjacent vertices                     gremlin> g.v(1).out                     ==>v[2]                    ...
Graph traversal: adjacent vertices (cont.)                     gremlin> g.v(1).out.out                     ==>v[5]        ...
Graph traversal: starting vertex                     gremlin> g.v(1)                     ==>v[1]
Graph traversal: labeled outgoing edges                    gremlin> g.v(1).outE(created)                    ==>e[9][1-crea...
Graph traversal: labeled adjacent vertices                     gremlin> g.v(1).outE(created).inV                     ==>v[...
Graph traversal: labeled adjacent (cont.)                     gremlin> g.v(1).out(created).in(created)                    ...
Graph traversal and ...                      index                      transform                      filter             ...
Graph traversal and index                    gremlin> g.idx(vertices)[[name:marko]]                    ==>v[1]
Graph traversal and transform                    gremlin> g.v(1).outE.label.dedup                    ==>knows             ...
Graph traversal and filter                      gremlin> g.v(1).out(knows).age                      ==>27                 ...
Graph traversal and compute                   gremlin> g.v(1).outE.weight                   ==>0.5                   ==>1....
Graph traversal and manipulate                    gremlin> g.v(1).outE.sideEffect{it.                    weight+=0.1}.weig...
Graph traversal and loop                    gremlin> g.v(1).out.loop(1){it.loops<3}                    ==>v[5]            ...
Graph traversal and path                    gremlin> g.v(1).outE.inV.path                    ==>[v[1], e[7][1-knows->2], v...
Global traversal: in-degree distribution                      gremlin> m=[:].withDefault{0}; g.V.                      sid...
Walk is ending               flexible       Gremlin is a graph traversal language
Map of walk              Warm up                Property Graph                Graph Database              Walk with Gremli...
Linked Data          http://www.w3.org/DesignIssues/LinkedData.html
Linked Data cloud Linking Open Data cloud diagram, by Richard Cyganiak and Anja Jentzsch. http://lod-cloud.net/
Linked Data initiative                http://freeyourmetadata.org/
Linked Data and Gremlin                        +                     +gremlin> g = new SparqlRepositorySailGraph("http://d...
Map of walk              Warm up                Property Graph                Graph Database              Walk with Gremli...
Graph and Brain
Global Graph                     Internet             =>         net of computers   Word Wide Web                         ...
Thank you
http://tinkerpop.com         Logos created by Ketrina Yim for TinkerPop geeks        Images created by Flickr Creative Com...
Upcoming SlideShare
Loading in...5
×

A walk in graph databases v1.0

8,594

Published on

A walk in graph databases v1.0

  1. 1. A Walk in Graph DatabasesPierre De Wilde4 May 2012Global Brain InstituteVUB - ECCO Group
  2. 2. The Law of the Hammer If the only tool you have is a hammer, everything looks like a nail. Abraham Maslow - The Psychology of Science - 1966
  3. 3. The Law of the Relational Database If the only tool you have is a relational database, everything looks like a table. A Walk in Graph Databases - 2012
  4. 4. doesntOne size fits all Scalability issue Scale up Scale out Index-intensive issue Find data Join data
  5. 5. NoSQL ?! No SQL ? Not only SQL ! Scalability solutions Key-value stores Column databases Document databases Index-intensive solution Graph databases
  6. 6. Query language for relational databases SQL ISUD or CRUD
  7. 7. Traversal graphQuery language for relational databases Gremlin is a graph traversal language
  8. 8. Map of walk Warm up Property Graph Graph Database Walk with Gremlin Graph Manipulation Graph Traversal Stretching Linked Data Global Graph
  9. 9. Map of walk Warm up Property Graph Graph Database Walk with Gremlin Graph Manipulation Graph Traversal Stretching Linked Data Global Graph
  10. 10. Graph G = (V, E) --. .-. .- .--. ....
  11. 11. One graph doesnt fit all Marko A. Rodriguez and Peter Neubauer - Constructions from Dots and Lines - 2010
  12. 12. Property graph A property graph is a directed, labeled, attributed, multi graph.
  13. 13. Anatomy of a vertex A vertex is composed of - an unique identifier (id) - a collection of properties - a set of incoming edges (inE) - a set of outgoing edges (outE)
  14. 14. Anatomy of an edge An edge is composed of - an unique identifier (id) - an outgoing vertex (outV) - a label - an incoming vertex (inV) - a collection of properties
  15. 15. Map of walk Warm up Property Graph Graph Database Walk with Gremlin Graph Manipulation Graph Traversal Stretching Linked Data Global Graph
  16. 16. Graph database
  17. 17. Key feature of a graph database Index-free adjacency
  18. 18. Some graph database vendors Neo4j from Neo Technology http://neo4j.org/ OrientDB from Orient Technologies http://www.orientdb.org/ Dex from Sparsity-Technologies http://www.sparsity-technologies.com/dex InfiniteGraph from Objectivity, Inc. http://www.infinitegraph.com/
  19. 19. Map of walk Warm up Property Graph Graph Database Walk with Gremlin Graph Manipulation Graph Traversal Stretching Linked Data Global Graph
  20. 20. TinkerPop Open source project in the graph space
  21. 21. TinkerPop family https://github.com/tinkerpop
  22. 22. Gremlin $ gremlin.sh ,,,/ (o o) -----oOOo-(_)-oOOo----- gremlin> Gremlin is a graph traversal language
  23. 23. Map of walk Warm up Property Graph Graph Database Walk with Gremlin Graph Manipulation Graph Traversal Stretching Linked Data Global Graph
  24. 24. Connect to a graph database gremlin> g = new TinkerGraph(name) gremlin> g = new Neo4jGraph(name) gremlin> g = new OrientGraph(name) gremlin> g = new DexGraph(name) gremlin> g = new IGGraph(name)
  25. 25. Add a vertex / an edge gremlin> v1 = g.addVertex() gremlin> v2 = g.addVertex() ... gremlin> g.addEdge(v1, knows, v2) ... gremlin> g.loadGraphML(url)
  26. 26. Update a vertex gremlin> v = g.getVertex(1) ==>v[1] gremlin> v.getPropertyKeys() ==>age ==>name gremlin> v.getProperty(name) ==>marko gremlin> v.getProperty(age) ==>29 gremlin> v.setProperty(age,32) ==>32 gremlin> v.age ==>32 gremlin> v.name ==>marko
  27. 27. Update an edge gremlin> e = g.getEdge(8) ==>e[8][1-knows->4] gremlin> e.getPropertyKeys() ==>weight gremlin> e.getProperty(weight) ==>1.0 gremlin> e.setProperty(weigth,0.9) ==>0.9 gremlin> e.map() ==>weigth=0.9 ==>weight=1.0 gremlin> e.removeProperty(weigth) ==>0.9
  28. 28. Remove a vertex gremlin> v = g.getVertex(3) ==>v[3] gremlin> g.removeVertex(v) ==>null
  29. 29. Remove an edge gremlin> e = g.getEdge(10) ==>e[10][4-created->5] gremlin> g.removeEdge(e) ==>null
  30. 30. Disconnect from the graph database gremlin> g.shutdown()
  31. 31. Map of walk Warm up Property Graph Graph Database Walk with Gremlin Graph Manipulation Graph Traversal Stretching Linked Data Global Graph
  32. 32. Graph traversal Jump - from vertex to edge - from edge to vertex - from vertex to vertex
  33. 33. Graph traversal: starting the traversal gremlin> g.v(1) ==>v[1]
  34. 34. Graph traversal: outgoing edges gremlin> g.v(1).outE ==>e[7][1-knows->2] ==>e[9][1-created->3] ==>e[8][1-knows->4]
  35. 35. Graph traversal: incoming vertices gremlin> g.v(1).outE.inV ==>v[2] ==>v[4] ==>v[3]
  36. 36. Graph traversal: outgoing edges (cont.) gremlin> g.v(1).outE.inV.outE ==>e[10][4-created->5] ==>e[11][4-created->3]
  37. 37. Graph traversal: incoming vertices (cont.) gremlin> g.v(1).outE.inV.outE.inV ==>v[5] ==>v[3]
  38. 38. Graph traversal: ending the traversal gremlin> g.v(1).outE.inV.outE.inV.outE
  39. 39. Graph traversal: starting vertex gremlin> g.v(1) ==>v[1]
  40. 40. Graph traversal: adjacent vertices gremlin> g.v(1).out ==>v[2] ==>v[4] ==>v[3]
  41. 41. Graph traversal: adjacent vertices (cont.) gremlin> g.v(1).out.out ==>v[5] ==>v[3]
  42. 42. Graph traversal: starting vertex gremlin> g.v(1) ==>v[1]
  43. 43. Graph traversal: labeled outgoing edges gremlin> g.v(1).outE(created) ==>e[9][1-created->3]
  44. 44. Graph traversal: labeled adjacent vertices gremlin> g.v(1).outE(created).inV ==>v[3] gremlin> g.v(1).out(created) ==>v[3]
  45. 45. Graph traversal: labeled adjacent (cont.) gremlin> g.v(1).out(created).in(created) ==>v[1] ==>v[4] ==>v[6]
  46. 46. Graph traversal and ... index transform filter compute manipulate loop path
  47. 47. Graph traversal and index gremlin> g.idx(vertices)[[name:marko]] ==>v[1]
  48. 48. Graph traversal and transform gremlin> g.v(1).outE.label.dedup ==>knows ==>created gremlin> g.v(1).out(knows).name ==>vadas ==>josh
  49. 49. Graph traversal and filter gremlin> g.v(1).out(knows).age ==>27 ==>32 gremlin> g.v(1).out(knows).filter{it .age>30}.age ==>32
  50. 50. Graph traversal and compute gremlin> g.v(1).outE.weight ==>0.5 ==>1.0 ==>0.4 gremlin> g.v(1).outE.weight.mean() ==>0.6333333353201548
  51. 51. Graph traversal and manipulate gremlin> g.v(1).outE.sideEffect{it. weight+=0.1}.weight ==>0.6 ==>1.1 ==>0.5
  52. 52. Graph traversal and loop gremlin> g.v(1).out.loop(1){it.loops<3} ==>v[5] ==>v[3]
  53. 53. Graph traversal and path gremlin> g.v(1).outE.inV.path ==>[v[1], e[7][1-knows->2], v[2]] ==>[v[1], e[8][1-knows->4], v[4]] ==>[v[1], e[9][1-created->3], v[3]] gremlin> g.v(1).out.path ==>[v[1], v[2]] ==>[v[1], v[4]] ==>[v[1], v[3]]
  54. 54. Global traversal: in-degree distribution gremlin> m=[:].withDefault{0}; g.V. sideEffect{m[it.in.count()]+=1}. iterate(); m.sort() ==>0=2 ==>1=3 ==>3=1
  55. 55. Walk is ending flexible Gremlin is a graph traversal language
  56. 56. Map of walk Warm up Property Graph Graph Database Walk with Gremlin Graph Manipulation Graph Traversal Stretching Linked Data Global Graph
  57. 57. Linked Data http://www.w3.org/DesignIssues/LinkedData.html
  58. 58. Linked Data cloud Linking Open Data cloud diagram, by Richard Cyganiak and Anja Jentzsch. http://lod-cloud.net/
  59. 59. Linked Data initiative http://freeyourmetadata.org/
  60. 60. Linked Data and Gremlin + +gremlin> g = new SparqlRepositorySailGraph("http://dbpedia.org/sparql")gremlin> v = g.v( http://dbpedia.org/resource/Global_brain)==>v[http://dbpedia.org/resource/Global_brain]gremlin> v.out(http://www.w3.org/2000/01/rdf-schema#comment).has(lang,en).value==>The Global Brain is a metaphor for the worldwide intelligent network...gremlin> v.inE(http://dbpedia.org/ontology/knownFor).outV==>v[http://dbpedia.org/resource/Francis_Heylighen]gremlin> v.inE(http://dbpedia.org/ontology/knownFor).outV.outE(http://dbpedia.org/ontology/knownFor).inV==>v[http://dbpedia.org/resource/Self-organization]==>v[http://dbpedia.org/resource/Memetics]==>v[http://dbpedia.org/resource/Global_brain]
  61. 61. Map of walk Warm up Property Graph Graph Database Walk with Gremlin Graph Manipulation Graph Traversal Stretching Linked Data Global Graph
  62. 62. Graph and Brain
  63. 63. Global Graph Internet => net of computers Word Wide Web => web of documentsGiant Global Graph => graph of metadata I called this graph the Semantic Web, but maybe it should have been Giant Global Graph. Tim Berners-Lee - timbls blog - 2007
  64. 64. Thank you
  65. 65. http://tinkerpop.com Logos created by Ketrina Yim for TinkerPop geeks Images created by Flickr Creative Commons Artists Graphs created by Memotive Concept Mapping tool
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×