Traversing Graph Databases with Gremlin               Marko A. Rodriguez             Graph Systems Architect            ht...
Thank You SponsorsShort Slideshow + Live Demo = This Presentation
TinkerPop Productions1    1    TinkerPop (http://tinkerpop.com), Marko A. Rodriguez (http://markorodriguez.com), PeterNeub...
TinkerPop Graph Stack                            http://host:8182/graph/vertices/1  RESTful ServerTraversal Language Trave...
TinkerPop Graph Stack – Focus                     http://host:8182/graph/vertices/1Traversal Language Traversal Engine Gen...
Blueprints –> Pipes –> Groovy –> GremlinTo understand Gremlin, its important to understand Groovy,Blueprints, and Pipes.  ...
Gremlin is a Domain Specific Language• Gremlin 0.7+ uses Groovy as its host language.2• Gremlin 0.7+ takes advantage of Gro...
Gremlin is for Property Graphs                                                            name = "lop"                    ...
Gremlin is for Blueprints-enabled Graph Databases•   Blueprints can be seen as the JDBC for property graph databases.3•   ...
A Blueprints Detour - Implementations  TinkerGraph
A Blueprints Detour - Ouplementations     JUNG     Java Universal Network/Graph Framework
Gremlin Compiles Down to Pipes• Pipes is a data flow framework for evaluating lazy graph traversals.4• A Pipe extends Itera...
A Pipes Detour - Chained Iterators• This Pipeline takes objects of type A and turns them into objects of  type D.         ...
A Pipes Detour - Simple Example“What are the names of the people that marko knows?”                                     B ...
A Pipes Detour - Simple ExamplePipe<Vertex,Edge> pipe1 = new OutEdgesPipe("knows");Pipe<Edge,Vertex> pipe2 = new InVertexP...
A Pipes Detour - Pipes Library                       [ GRAPHS ]           [ SIDEEFFECTS ][ FILTERS ]            OutEdgesPi...
A Pipes Detour - Creating Pipespublic class NumCharsPipe extends AbstractPipe<String,Integer> {  public Integer processNex...
Now onto Gremlin proper...
The Gremlin Architecture   Neo4j   OrientDB   DEX
The Many Ways of Using Gremlin• Gremlin has a REPL to be run from the shell.• Gremlin can be natively integrated into any ...
Pipe = Step: 3 Generic StepsIn general a Pipe is a “step” in Gremlin. Gremlin is founded on a collectionof atomic steps. T...
Abstract/Derived/Inferred Adjacency                              outE                                         in          ...
Developers FOAF Import               Graph                                     Friend-Of-A-Friend                         ...
Integrating the JDK (Java API)• Groovy is the host language for Gremlin. Thus, the JDK is natively  available in a path ex...
Time for a demomarko$ gremlin         ,,,/         (o o)-----oOOo-(_)-oOOo-----gremlin>                             Gremli...
Graph Bootcamp      http://markorodriguez.com/services/workshops/graph-bootcamp/• June 23-24th in Chicago, Illinois.• Augu...
Upcoming SlideShare
Loading in...5
×

Traversing Graph Databases with Gremlin

9,545

Published on

A discussion of Blueprints, Pipes, and Gremlin. The presentation's second half was a live Gremlin tutorial/demo.

Published in: Technology
0 Comments
15 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
9,545
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
252
Comments
0
Likes
15
Embeds 0
No embeds

No notes for slide

Traversing Graph Databases with Gremlin

  1. 1. Traversing Graph Databases with Gremlin Marko A. Rodriguez Graph Systems Architect http://markorodriguez.com NoSQL New York City Meetup – May 16, 2011 Gremlin G = (V, E) May 10, 2011
  2. 2. Thank You SponsorsShort Slideshow + Live Demo = This Presentation
  3. 3. TinkerPop Productions1 1 TinkerPop (http://tinkerpop.com), Marko A. Rodriguez (http://markorodriguez.com), PeterNeubauer (http://www.linkedin.com/in/neubauer), Joshua Shinavier (http://fortytwo.net/), PavelYaskevich (https://github.com/xedin), Darrick Wiebe (http://ofallpossibleworlds.wordpress.com/), Stephen Mallette (http://stephen.genoprime.com/), Alex Averbuch (http://se.linkedin.com/in/alexaverbuch)
  4. 4. TinkerPop Graph Stack http://host:8182/graph/vertices/1 RESTful ServerTraversal Language Traversal Engine Graph-to-Object Mapper Generic Interface TinkerGraph Graph Database
  5. 5. TinkerPop Graph Stack – Focus http://host:8182/graph/vertices/1Traversal Language Traversal Engine Generic Interface TinkerGraph
  6. 6. Blueprints –> Pipes –> Groovy –> GremlinTo understand Gremlin, its important to understand Groovy,Blueprints, and Pipes. Bl ue s pr ip e in P ts
  7. 7. Gremlin is a Domain Specific Language• Gremlin 0.7+ uses Groovy as its host language.2• Gremlin 0.7+ takes advantage of Groovy’s meta-programming, dynamic typing, and closure properties.• Groovy is a superset of Java and as such, natively exposes the full JDK to Gremlin. 2 Groovy available at http://groovy.codehaus.org/.
  8. 8. Gremlin is for Property Graphs name = "lop" lang = "java" weight = 0.4 3 name = "marko" age = 29 created weight = 0.2 9 1 created 8 created 12 7 weight = 1.0 weight = 0.5 weight = 0.4 6 knows knows 11 name = "peter" age = 35 name = "josh" 4 age = 32 2 10 name = "vadas" age = 27 weight = 1.0 created 5 name = "ripple" lang = "java"A graph is composed of vertices (nodes), edges (links), and properties (keys/values).
  9. 9. Gremlin is for Blueprints-enabled Graph Databases• Blueprints can be seen as the JDBC for property graph databases.3• Provides a collection of interfaces for graph database providers to implement.• Provides tests to ensure the operational semantics of any implementation are correct.• Provides numerous “helper utilities” to make working with graph databases easy. Blueprints 3 Blueprints is available at http://blueprints.tinkerpop.com.
  10. 10. A Blueprints Detour - Implementations TinkerGraph
  11. 11. A Blueprints Detour - Ouplementations JUNG Java Universal Network/Graph Framework
  12. 12. Gremlin Compiles Down to Pipes• Pipes is a data flow framework for evaluating lazy graph traversals.4• A Pipe extends Iterator, Iterable and can be chained together to create processing pipelines.• A Pipe can be embedded into another Pipe to allow for nested processing. Pipes 4 Pipes is available at http://pipes.tinkerpop.com.
  13. 13. A Pipes Detour - Chained Iterators• This Pipeline takes objects of type A and turns them into objects of type D. D D A A A Pipe1 B Pipe2 C Pipe3 D D A D A PipelinePipe<A,D> pipeline = new Pipeline<A,D>(Pipe1<A,B>, Pipe2<B,C>, Pipe3<C,D>)
  14. 14. A Pipes Detour - Simple Example“What are the names of the people that marko knows?” B name=peter knows A knows C name=pavel name=marko created created D name=gremlin
  15. 15. A Pipes Detour - Simple ExamplePipe<Vertex,Edge> pipe1 = new OutEdgesPipe("knows");Pipe<Edge,Vertex> pipe2 = new InVertexPipe();Pipe<Vertex,String> pipe3 = new PropertyPipe<String>("name");Pipe<Vertex,String> pipeline = new Pipeline(pipe1,pipe2,pipe3);pipeline.setStarts(new SingleIterator<Vertex>(graph.getVertex("A")); InVertexPipe() OutEdgesPipe("knows") PropertyPipe("name") B name=peter knows A knows C name=pavel name=marko created created D name=gremlinHINT: The benefit of Gremlin is that this Java verbosity is reduced to g.v(‘A’).outE(‘knows’).inV.name.
  16. 16. A Pipes Detour - Pipes Library [ GRAPHS ] [ SIDEEFFECTS ][ FILTERS ] OutEdgesPipe AggregatorPipeAndFilterPipe InEdgesPipe GroupCountPipeBackFilterPipe OutVertexPipe CountPipeCollectionFilterPipe InVertexPip SideEffectCapPipeComparisonFilterPipe IdFilterPipeDuplicateFilterPipe IdPipe [ UTILITIES ]FutureFilterPipe LabelFilterPipe GatherPipeObjectFilterPipe LabelPipe PathPipeOrFilterPipe PropertyFilterPipe ScatterPipeRandomFilterPipe PropertyPipe PipelineRangeFilterPipe ...
  17. 17. A Pipes Detour - Creating Pipespublic class NumCharsPipe extends AbstractPipe<String,Integer> { public Integer processNextStart() { String word = this.starts.next(); return word.length(); }}When extending the base class AbstractPipe<S,E> all that is required isan implementation of processNextStart().
  18. 18. Now onto Gremlin proper...
  19. 19. The Gremlin Architecture Neo4j OrientDB DEX
  20. 20. The Many Ways of Using Gremlin• Gremlin has a REPL to be run from the shell.• Gremlin can be natively integrated into any Groovy class.• Gremlin can be interacted with indirectly through Java, via Groovy.• Gremlin has a JSR 223 ScriptEngine as well.
  21. 21. Pipe = Step: 3 Generic StepsIn general a Pipe is a “step” in Gremlin. Gremlin is founded on a collectionof atomic steps. The syntax is generally seen as step.step.step. Thereare 3 categories of steps.• transform: map the input to some output. S → T outE, inV, paths, copySplit, fairMerge, . . .• filter: either output the input or not. S → (S ∪ ∅) back, except, uniqueObject, andFilter, orFilter, . . .• sideEffect: output the input and yield a side-effect. S → S aggregate, groupCount, . . .
  22. 22. Abstract/Derived/Inferred Adjacency outE in outE V loop(2){..} back(3) {i t. sa la } ry } friend_of_a_friend_who_earns_less_than_friend_at_work• outE, inV, etc. is low-level graph speak (the domain is the graph).• codeveloper is high-level domain speak (the domain is software development).5 5 In this way, Gremlin can be seen as a DSL (domain-specific language) for creating DSL’s for yourgraph applications. Gremlin’s domain is “the graph.” Build languages for your domain on top of Gremlin(e.g. “software development domain”).
  23. 23. Developers FOAF Import Graph Friend-Of-A-Friend GraphDeveloper Imports Friends at Work Graph Graph You need not make Software Imports Friendship derived graphs explicit. Graph Graph You can, at runtime, compute them. Moreover, generate them locally, not Developer Created globally (e.g. ``Markos Employment Graph friends from work Graph relations"). This concept is related to automated reasoning and whether reasoned relations are inserted into the explicit graph or Explicit Graph computed at query time.
  24. 24. Integrating the JDK (Java API)• Groovy is the host language for Gremlin. Thus, the JDK is natively available in a path expression.gremlin> v.out(‘friend’){it.name.matches(‘M.*’)}.name==>Mattias==>Marko==>Matt...gremlin> x.out(‘rated’).transform{JSONParser.get(it.uri).stars}.mean()...gremlin> y.outE(‘rated’).filter{TextAnalysis.isElated(it.review)}.inV...
  25. 25. Time for a demomarko$ gremlin ,,,/ (o o)-----oOOo-(_)-oOOo-----gremlin> Gremlin G = (V, E) http://gremlin.tinkerpop.com http://groups.google.com/group/gremlin-users http://tinkerpop.com
  26. 26. Graph Bootcamp http://markorodriguez.com/services/workshops/graph-bootcamp/• June 23-24th in Chicago, Illinois.• August X-Yth in Denver, Colorado.• Book a private gig for your organization.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×