A walk in graph databases v1.0

9,567 views
9,258 views

Published on

A walk in graph databases v1.0

  1. 1. A Walk in Graph DatabasesPierre De Wilde4 May 2012Global Brain InstituteVUB - ECCO Group
  2. 2. The Law of the Hammer If the only tool you have is a hammer, everything looks like a nail. Abraham Maslow - The Psychology of Science - 1966
  3. 3. The Law of the Relational Database If the only tool you have is a relational database, everything looks like a table. A Walk in Graph Databases - 2012
  4. 4. doesntOne size fits all Scalability issue Scale up Scale out Index-intensive issue Find data Join data
  5. 5. NoSQL ?! No SQL ? Not only SQL ! Scalability solutions Key-value stores Column databases Document databases Index-intensive solution Graph databases
  6. 6. Query language for relational databases SQL ISUD or CRUD
  7. 7. Traversal graphQuery language for relational databases Gremlin is a graph traversal language
  8. 8. Map of walk Warm up Property Graph Graph Database Walk with Gremlin Graph Manipulation Graph Traversal Stretching Linked Data Global Graph
  9. 9. Map of walk Warm up Property Graph Graph Database Walk with Gremlin Graph Manipulation Graph Traversal Stretching Linked Data Global Graph
  10. 10. Graph G = (V, E) --. .-. .- .--. ....
  11. 11. One graph doesnt fit all Marko A. Rodriguez and Peter Neubauer - Constructions from Dots and Lines - 2010
  12. 12. Property graph A property graph is a directed, labeled, attributed, multi graph.
  13. 13. Anatomy of a vertex A vertex is composed of - an unique identifier (id) - a collection of properties - a set of incoming edges (inE) - a set of outgoing edges (outE)
  14. 14. Anatomy of an edge An edge is composed of - an unique identifier (id) - an outgoing vertex (outV) - a label - an incoming vertex (inV) - a collection of properties
  15. 15. Map of walk Warm up Property Graph Graph Database Walk with Gremlin Graph Manipulation Graph Traversal Stretching Linked Data Global Graph
  16. 16. Graph database
  17. 17. Key feature of a graph database Index-free adjacency
  18. 18. Some graph database vendors Neo4j from Neo Technology http://neo4j.org/ OrientDB from Orient Technologies http://www.orientdb.org/ Dex from Sparsity-Technologies http://www.sparsity-technologies.com/dex InfiniteGraph from Objectivity, Inc. http://www.infinitegraph.com/
  19. 19. Map of walk Warm up Property Graph Graph Database Walk with Gremlin Graph Manipulation Graph Traversal Stretching Linked Data Global Graph
  20. 20. TinkerPop Open source project in the graph space
  21. 21. TinkerPop family https://github.com/tinkerpop
  22. 22. Gremlin $ gremlin.sh ,,,/ (o o) -----oOOo-(_)-oOOo----- gremlin> Gremlin is a graph traversal language
  23. 23. Map of walk Warm up Property Graph Graph Database Walk with Gremlin Graph Manipulation Graph Traversal Stretching Linked Data Global Graph
  24. 24. Connect to a graph database gremlin> g = new TinkerGraph(name) gremlin> g = new Neo4jGraph(name) gremlin> g = new OrientGraph(name) gremlin> g = new DexGraph(name) gremlin> g = new IGGraph(name)
  25. 25. Add a vertex / an edge gremlin> v1 = g.addVertex() gremlin> v2 = g.addVertex() ... gremlin> g.addEdge(v1, knows, v2) ... gremlin> g.loadGraphML(url)
  26. 26. Update a vertex gremlin> v = g.getVertex(1) ==>v[1] gremlin> v.getPropertyKeys() ==>age ==>name gremlin> v.getProperty(name) ==>marko gremlin> v.getProperty(age) ==>29 gremlin> v.setProperty(age,32) ==>32 gremlin> v.age ==>32 gremlin> v.name ==>marko
  27. 27. Update an edge gremlin> e = g.getEdge(8) ==>e[8][1-knows->4] gremlin> e.getPropertyKeys() ==>weight gremlin> e.getProperty(weight) ==>1.0 gremlin> e.setProperty(weigth,0.9) ==>0.9 gremlin> e.map() ==>weigth=0.9 ==>weight=1.0 gremlin> e.removeProperty(weigth) ==>0.9
  28. 28. Remove a vertex gremlin> v = g.getVertex(3) ==>v[3] gremlin> g.removeVertex(v) ==>null
  29. 29. Remove an edge gremlin> e = g.getEdge(10) ==>e[10][4-created->5] gremlin> g.removeEdge(e) ==>null
  30. 30. Disconnect from the graph database gremlin> g.shutdown()
  31. 31. Map of walk Warm up Property Graph Graph Database Walk with Gremlin Graph Manipulation Graph Traversal Stretching Linked Data Global Graph
  32. 32. Graph traversal Jump - from vertex to edge - from edge to vertex - from vertex to vertex
  33. 33. Graph traversal: starting the traversal gremlin> g.v(1) ==>v[1]
  34. 34. Graph traversal: outgoing edges gremlin> g.v(1).outE ==>e[7][1-knows->2] ==>e[9][1-created->3] ==>e[8][1-knows->4]
  35. 35. Graph traversal: incoming vertices gremlin> g.v(1).outE.inV ==>v[2] ==>v[4] ==>v[3]
  36. 36. Graph traversal: outgoing edges (cont.) gremlin> g.v(1).outE.inV.outE ==>e[10][4-created->5] ==>e[11][4-created->3]
  37. 37. Graph traversal: incoming vertices (cont.) gremlin> g.v(1).outE.inV.outE.inV ==>v[5] ==>v[3]
  38. 38. Graph traversal: ending the traversal gremlin> g.v(1).outE.inV.outE.inV.outE
  39. 39. Graph traversal: starting vertex gremlin> g.v(1) ==>v[1]
  40. 40. Graph traversal: adjacent vertices gremlin> g.v(1).out ==>v[2] ==>v[4] ==>v[3]
  41. 41. Graph traversal: adjacent vertices (cont.) gremlin> g.v(1).out.out ==>v[5] ==>v[3]
  42. 42. Graph traversal: starting vertex gremlin> g.v(1) ==>v[1]
  43. 43. Graph traversal: labeled outgoing edges gremlin> g.v(1).outE(created) ==>e[9][1-created->3]
  44. 44. Graph traversal: labeled adjacent vertices gremlin> g.v(1).outE(created).inV ==>v[3] gremlin> g.v(1).out(created) ==>v[3]
  45. 45. Graph traversal: labeled adjacent (cont.) gremlin> g.v(1).out(created).in(created) ==>v[1] ==>v[4] ==>v[6]
  46. 46. Graph traversal and ... index transform filter compute manipulate loop path
  47. 47. Graph traversal and index gremlin> g.idx(vertices)[[name:marko]] ==>v[1]
  48. 48. Graph traversal and transform gremlin> g.v(1).outE.label.dedup ==>knows ==>created gremlin> g.v(1).out(knows).name ==>vadas ==>josh
  49. 49. Graph traversal and filter gremlin> g.v(1).out(knows).age ==>27 ==>32 gremlin> g.v(1).out(knows).filter{it .age>30}.age ==>32
  50. 50. Graph traversal and compute gremlin> g.v(1).outE.weight ==>0.5 ==>1.0 ==>0.4 gremlin> g.v(1).outE.weight.mean() ==>0.6333333353201548
  51. 51. Graph traversal and manipulate gremlin> g.v(1).outE.sideEffect{it. weight+=0.1}.weight ==>0.6 ==>1.1 ==>0.5
  52. 52. Graph traversal and loop gremlin> g.v(1).out.loop(1){it.loops<3} ==>v[5] ==>v[3]
  53. 53. Graph traversal and path gremlin> g.v(1).outE.inV.path ==>[v[1], e[7][1-knows->2], v[2]] ==>[v[1], e[8][1-knows->4], v[4]] ==>[v[1], e[9][1-created->3], v[3]] gremlin> g.v(1).out.path ==>[v[1], v[2]] ==>[v[1], v[4]] ==>[v[1], v[3]]
  54. 54. Global traversal: in-degree distribution gremlin> m=[:].withDefault{0}; g.V. sideEffect{m[it.in.count()]+=1}. iterate(); m.sort() ==>0=2 ==>1=3 ==>3=1
  55. 55. Walk is ending flexible Gremlin is a graph traversal language
  56. 56. Map of walk Warm up Property Graph Graph Database Walk with Gremlin Graph Manipulation Graph Traversal Stretching Linked Data Global Graph
  57. 57. Linked Data http://www.w3.org/DesignIssues/LinkedData.html
  58. 58. Linked Data cloud Linking Open Data cloud diagram, by Richard Cyganiak and Anja Jentzsch. http://lod-cloud.net/
  59. 59. Linked Data initiative http://freeyourmetadata.org/
  60. 60. Linked Data and Gremlin + +gremlin> g = new SparqlRepositorySailGraph("http://dbpedia.org/sparql")gremlin> v = g.v( http://dbpedia.org/resource/Global_brain)==>v[http://dbpedia.org/resource/Global_brain]gremlin> v.out(http://www.w3.org/2000/01/rdf-schema#comment).has(lang,en).value==>The Global Brain is a metaphor for the worldwide intelligent network...gremlin> v.inE(http://dbpedia.org/ontology/knownFor).outV==>v[http://dbpedia.org/resource/Francis_Heylighen]gremlin> v.inE(http://dbpedia.org/ontology/knownFor).outV.outE(http://dbpedia.org/ontology/knownFor).inV==>v[http://dbpedia.org/resource/Self-organization]==>v[http://dbpedia.org/resource/Memetics]==>v[http://dbpedia.org/resource/Global_brain]
  61. 61. Map of walk Warm up Property Graph Graph Database Walk with Gremlin Graph Manipulation Graph Traversal Stretching Linked Data Global Graph
  62. 62. Graph and Brain
  63. 63. Global Graph Internet => net of computers Word Wide Web => web of documentsGiant Global Graph => graph of metadata I called this graph the Semantic Web, but maybe it should have been Giant Global Graph. Tim Berners-Lee - timbls blog - 2007
  64. 64. Thank you
  65. 65. http://tinkerpop.com Logos created by Ketrina Yim for TinkerPop geeks Images created by Flickr Creative Commons Artists Graphs created by Memotive Concept Mapping tool

×