TinkerPop: a story of graphs, DBs, and graph DBs
Upcoming SlideShare
Loading in...5
×
 

TinkerPop: a story of graphs, DBs, and graph DBs

on

  • 3,039 views

intro to TinkerPop and the Aurelius Graph Cluster for the Graph DB Workshop, Texas Linux Festival 2014

intro to TinkerPop and the Aurelius Graph Cluster for the Graph DB Workshop, Texas Linux Festival 2014

Statistics

Views

Total Views
3,039
Views on SlideShare
2,218
Embed Views
821

Actions

Likes
8
Downloads
14
Comments
0

24 Embeds 821

http://nosql.mypopescu.com 315
http://feedly.com 286
https://twitter.com 62
http://www.newsblur.com 42
http://www.feedspot.com 20
http://feeds.feedburner.com 17
http://digg.com 14
http://www.inoreader.com 9
http://newsblur.com 8
http://feedreader.com 7
http://feedproxy.google.com 7
http://reader.aol.com 6
http://reader.berdnik.info 5
http://www.hanrss.com 4
http://www.slideee.com 4
http://fortytwo.net 3
http://my.organic.hu 2
http://xianguo.com 2
http://192.168.3.22 2
http://reader.inthemail.org 2
https://reader.aol.com 1
http://pupuserio.us 1
http://tweetedtimes.com 1
http://test.eklablog.info 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

TinkerPop: a story of graphs, DBs, and graph DBs TinkerPop: a story of graphs, DBs, and graph DBs Presentation Transcript

  • TinkerPop: a story of graphs, DBs, and graph DBs Joshua Shinavier and James Thornton Texas Linux Festival June 13th, 2014
  • Once, there was a thing
  • v(1) Let’s call it a vertex
  • The vertex had some metadata v(1) name: “Graph DB workshop”
  • We’ll call that a property v(1) name: “Graph DB workshop” You are here.
  • In fact, the vertex had multiple properties v(1) name: “Graph DB workshop” type: “Event”
  • The properties were of various types v(1) name: “Graph DB workshop” type: “Event” starts: 1402682400000 ends: 1402696800000
  • v(1) name: “Graph DB workshop” type: “Event” starts: 1402682400000 ends: 1402696800000 v(2) name: “Texas Linux Fest” type: “Event” starts: 1402664400000 ends: 1402808400000 Our vertex was not alone
  • Thus, an edge v(1) name: “Graph DB workshop” type: “Event” starts: 1402682400000 ends: 1402696800000 v(2) name: “Texas Linux Fest” type: “Event” starts: 1402664400000 ends: 1402808400000
  • The edge was directed… v(1) name: “Graph DB workshop” type: “Event” starts: 1402682400000 ends: 1402696800000 v(2) name: “Texas Linux Fest” type: “Event” starts: 1402664400000 ends: 1402808400000
  • …and labeled v(1) name: “Graph DB workshop” type: “Event” starts: 1402682400000 ends: 1402696800000 v(2) name: “Texas Linux Fest” type: “Event” starts: 1402664400000 ends: 1402808400000 partOf
  • The label types the relationship v(1) name: “Graph DB workshop” type: “Event” starts: 1402682400000 ends: 1402696800000 v(2) name: “Texas Linux Fest” type: “Event” starts: 1402664400000 ends: 1402808400000 partOf You are here, too.
  • v(1) name: “Graph DB workshop” type: “Event” starts: 1402682400000 ends: 1402696800000 v(2) name: “Texas Linux Fest” type: “Event” starts: 1402664400000 ends: 1402808400000 partOf v(3) name: “Chef Workshop” type: “Event” starts: 1402664400000 ends: 1402696800000 v(4) name: “Canonical Charm School” type: “Event” starts: 1402664400000 ends: 1402696800000 partOf partOf More vertices joined the fun…
  • v(1) name: “Graph DB workshop” type: “Event” starts: 1402682400000 ends: 1402696800000 v(2) name: “Texas Linux Fest” type: “Event” starts: 1402664400000 ends: 1402808400000 partOf v(3) name: “Chef Workshop” type: “Event” starts: 1402664400000 ends: 1402696800000 v(4) name: “Canonical Charm School” type: “Event” starts: 1402664400000 ends: 1402696800000 partOf partOf v(7) name: “TinkerPop suite” type: “Software” hasTopic v(8) name: “Aurelius Graph Cluster” type: “Software” hasTopic More labels, too
  • Now it was a labeled multigraph v(1) name: “Graph DB workshop” type: “Event” starts: 1402682400000 ends: 1402696800000 v(2) name: “Texas Linux Fest” type: “Event” starts: 1402664400000 ends: 1402808400000 partOf v(3) name: “Chef Workshop” type: “Event” starts: 1402664400000 ends: 1402696800000 v(4) name: “Canonical Charm School” type: “Event” starts: 1402664400000 ends: 1402696800000 partOf partOf v(6) name: “Joshua Shinavier” type: “Person” githubId: “joshsh” v(5) presentedBy presentedBy v(7) name: “TinkerPop suite” type: “Software” hasTopic v(8) name: “Aurelius Graph Cluster” type: “Software” hasTopic name: “James Thornton” type: “Person” githubId: “espeed”
  • A few more edges v(1) name: “Graph DB workshop” type: “Event” starts: 1402682400000 ends: 1402696800000 v(2) name: “Texas Linux Fest” type: “Event” starts: 1402664400000 ends: 1402808400000 partOf v(3) name: “Chef Workshop” type: “Event” starts: 1402664400000 ends: 1402696800000 v(4) name: “Canonical Charm School” type: “Event” starts: 1402664400000 ends: 1402696800000 partOf partOf v(6) name: “Joshua Shinavier” type: “Person” githubId: “joshsh” v(5) presentedBy presentedBy v(7) name: “TinkerPop suite” type: “Software” hasTopic v(8) name: “Aurelius Graph Cluster” type: “Software” contributesTo contributesTo hasTopic contributesTo name: “James Thornton” type: “Person” githubId: “espeed”
  • Some edges also had properties v(1) name: “Graph DB workshop” type: “Event” starts: 1402682400000 ends: 1402696800000 v(2) name: “Texas Linux Fest” type: “Event” starts: 1402664400000 ends: 1402808400000 partOf v(3) name: “Chef Workshop” type: “Event” starts: 1402664400000 ends: 1402696800000 v(4) name: “Canonical Charm School” type: “Event” starts: 1402664400000 ends: 1402696800000 partOf partOf v(6) name: “Joshua Shinavier” type: “Person” githubId: “joshsh” v(5) presentedBy presentedBy v(7) name: “TinkerPop suite” type: “Software” hasTopic v(8) name: “Aurelius Graph Cluster” type: “Software” contributesTo contributesTo hasTopic contributesTo weight: 0.2 weight: 0.8 name: “James Thornton” type: “Person” githubId: “espeed” weight: 1.0
  • We call this a Property Graph v(1) name: “Graph DB workshop” type: “Event” starts: 1402682400000 ends: 1402696800000 v(2) name: “Texas Linux Fest” type: “Event” starts: 1402664400000 ends: 1402808400000 partOf v(3) name: “Chef Workshop” type: “Event” starts: 1402664400000 ends: 1402696800000 v(4) name: “Canonical Charm School” type: “Event” starts: 1402664400000 ends: 1402696800000 partOf partOf v(6) name: “Joshua Shinavier” type: “Person” githubId: “joshsh” v(5) presentedBy presentedBy v(7) name: “TinkerPop suite” type: “Software” hasTopic v(8) name: “Aurelius Graph Cluster” type: “Software” contributesTo contributesTo hasTopic contributesTo weight: 0.2 weight: 0.8 name: “James Thornton” type: “Person” githubId: “espeed” weight: 1.0
  • Many graph DB data models are variations on this theme v(1) name: “Graph DB workshop” type: “Event” starts: 1402682400000 ends: 1402696800000 v(2) name: “Texas Linux Fest” type: “Event” starts: 1402664400000 ends: 1402808400000 partOf v(3) name: “Chef Workshop” type: “Event” starts: 1402664400000 ends: 1402696800000 v(4) name: “Canonical Charm School” type: “Event” starts: 1402664400000 ends: 1402696800000 partOf partOf v(6) name: “Joshua Shinavier” type: “Person” githubId: “joshsh” v(5) presentedBy presentedBy v(7) name: “TinkerPop suite” type: “Software” hasTopic v(8) name: “Aurelius Graph Cluster” type: “Software” contributesTo contributesTo hasTopic contributesTo weight: 0.2 weight: 0.8 name: “James Thornton” type: “Person” githubId: “espeed” weight: 1.0
  • Neo4j
  • OrientDB
  • Sparksee* * the graph database previously known as DEX
  • etc.
  • Enter • single Property Graph API supported by diverse graph database backends • choose your favorite, but avoid vendor lock-in • Blueprints : graph DB :: JDBC : RDBMS • implementations, “ouplementations”, test suites, and helper utilities are built on top
  • Blueprints implementations
  • Now we need a query language… • build it on the Blueprints API • query over any Blueprints-compatible DB • make it path-like, with side-effects • match abstract traversals through the graph, filtering, ranking, and mutating as you go • make it interactive. How about a REPL?
  • • a domain-specific language for traversing graphs • Turing-complete, permits access to the full JDK • has been adapted to various JVM languages • Gremlin : graph DB :: SQL : RDBMS… sort of Enter
  • Think “pipes and filters”
  • • Pipes: dataflow framework. The basis of Gremlin • Frames: Java bean framework for graphs • Furnace: Property Graph algorithms • Rexster: high-performance graph database server The rest of the TinkerPop family
  • TinkerPop is… • a developer group creating an open-source graph DB stack • a community of users and third-party implementors • a foundation for building high-performance graph applications of any size • model some data on your laptop • build massive clustered applications • open source, BSD licensed
  • A detailed guide to the rest of this workshop • intro to the Aurelius Graph Cluster • demos of graph tools and concepts • guided installation of tools • preview of TinkerPop3
  • Thanks!
  • The Aurelius Graph Cluster
  • In TinkerPop… • we adapt various graph DBs to a unified API • they become Property Graph databases
  • With AGC… • we adapt various high-performance databases to the Titan API • they become graph databases
  • Take your pick of CAP
  • Titan highlights • graphs, transactions scale with the number of machines in a cluster • geo, numeric range, and full text search for vertices and edges • support for either of two indexing backends • ElasticSearch, Lucene • native support for Blueprints, Rexster
  • Dealing with supernodes • Titan’s vertex-centric indices permit ordered querying from a vertex • e.g. retrieve “knows” edges… in order of “since” timestamp • iterates efficiently, even if there are thousands of edges
  • What about Faunus
  • Faunus… • is a Hadoop-based graph analytics engine • in Titan 0.5 will simply be called Titan/Hadoop • adds support for global distributed graph operations • applies (a subset of) Gremlin in a breadth-first fashion
  • Faunus inputs and outputs • Hadoop SequenceFile format (in/out) • Titan graph DB (in/out) • GraphSON format (in/out) • Rexster (in) • RDF (in) • Gremlin scripts (in/out)
  • Demo time
  • TinkerPop3
  • What’s new in TP3 • new Gremlin implementation which makes good use of Java 8 closures, enables introspection and optimization of traversals • new OLAP API with support for message passing systems like Giraph, Hama, Faunus, etc. • revamped I/O utilities with support for GraphSON, GraphML, and GremlinKryo • new server model, incl. remote execution of scripts via WebSocket API, server plugin support, customizable serialization formats
  • Gremlitron • Blueprints, Pipes, and Gremlin are all integrated in TinkerPop3 • Frames obsoleted by Gremlin DSLs • Furnace is Gremlin OLAP • Rexster is Gremlin Server
  • Try it out • at: • https://github.com/tinkerpop/tinkerpop3 • mailing list: • https://groups.google.com/forum/gremlin-users • we welcome your feedback and/or PRs
  • josh@fortytwo.net james.thornton@gmail.com http://tinkerpop.com