NoSQL Frankfurt 2010 - The GraphDB Landscape and sones

  • 17,077 views
Uploaded on

Achim Friedland has provided a very interesting overview of the graph databases products, the goals and some scenarios for graph databases, a brief comparison of property graphs with other models …

Achim Friedland has provided a very interesting overview of the graph databases products, the goals and some scenarios for graph databases, a brief comparison of property graphs with other models (relational databases, object-oriented, semantic web/RDF, and many other interesting aspects.
(via: http://nosql.mypopescu.com/post/1211252052/nosql-frankfurt-a-quick-review-of-the-conference)

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
17,077
On Slideshare
0
From Embeds
0
Number of Embeds
11

Actions

Shares
Downloads
357
Comments
2
Likes
25

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Photo: Gephi, flickr Frankfurt co-located to ICOODB 2010 28. September 2010 Frankfurt, Germany The GraphDB Landscape and sones Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 1
  • 2. Photo: Gephi, flickr Hashtags: #nosqlfr #graphdb #sones #neo4j Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 2
  • 3. Photo: litlnemo, flickr Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 3
  • 4. Photo: litlnemo, flickr • 43 years Object Orientation / OOP • 35 years Relational Databases / SQL • 21 years Semantic Web / RDF Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 4
  • 5. Photo: litlnemo, flickr • 43 years Object Orientation / OOP • 35 years Relational Databases / SQL • 21 years Semantic Web / RDF …but still we do not know where our files have gone! Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 5
  • 6. Photo: Chris Westermeyer, flickr Our wish list… Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 6
  • 7. Photo: litlnemo, flickr Simple management of structured, semi-structured and unstructured information Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 7
  • 8. Photo: litlnemo, flickr SQL Simple management of structured, semi-structured and unstructured XML/JSON information binaries Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 8
  • 9. Photo: Robbert van der Steeg, flickr Simple recursively linked information models Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 9
  • 10. Photo: Robbert van der Steeg, flickr Simple recursively linked information models • <a href = “…”> • metadata e.g. EXIF Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 10
  • 11. Photo: Khem, flickr Traversing linked information, finding shortest-paths, do semantic partitions Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 11
  • 12. Photo: NASA, flickr Recommendation and discovery of potentially interessting linked information Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 12
  • 13. Photo: NASA, flickr personal social item-related Recommendation and discovery of potentially interessting linked information Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 13
  • 14. Photo: squacco, flickr Consistency criteria and indices for simple attributes up to complex subgraph structures Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 14
  • 15. Photo: Cedrix Thual, flickr Versioned information and versioned information schemata Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 15
  • 16. Photo: Birger Hoppe, flickr Good integration into state-of-the- art programming concepts (no Object-Relational-Mappers!) Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 16
  • 17. Photo: Chuck “Caveman” Coker, flickr Can graph-databases help? Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 17
  • 18. Photo: Jeffrey Beall, flickr Graph-Databases? Graph = ( Vertics, Edges ) A graph database is a database that uses graph structures with nodes, edges, and properties to represent and store information. General graph databases that can store any graph are distinct from specialized graph databases such as triple stores and network databases. http://en.wikipedia.org/wiki/Graph_database Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 18
  • 19. Photo: Jeffrey Beall, flickr Graph-Databases? Graph = ( Vertics, Edges ) R. Angles, C. Gutierrez: Survey of graph database models, ACM Computing Surv. 40, 1 (Feb. 2008), 1-39 Adrian Silvescu, Donia Caragea, Anna Atramentov: Graph Databases, Technical Report. May 2002 Mark Gemis, Jan Paredaens, Inge Thyssens, und Jan Van den Bussche: GOOD: A Graph-Oriented Object Database System, 1993; In: Proceedings of the1993 ACM SIGMOD Intern. Conf. on Management of Data (Washington D.C., US, May 1993), 505-510. Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 19
  • 20. Photo: Gephi, flickr The Property-Graph Alice Friends Bob ID = 1 ID = 2 since = 2009/09/21 Alter = 21 Alter = 23 Properties are key-values pairs e.g. <String, Object> or <AttributeKey, Object> Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 20
  • 21. Photo: Gephi, flickr The Property-Graph vs. Relational Databases Relation Table A Table B Table Focus on structured tables and their relational consistency constraints. Does not auto-scale. Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 21
  • 22. Photo: Gephi, flickr The Property-Graph vs. Object-Oriented Model Reference / Pointer Object Object Focus on structured or dynamic objects. No (simple way for) edge properties. Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 22
  • 23. Photo: Gephi, flickr The Property-Graph vs. Semantic Web / RDF Predicate Subject Object Focus on triples or quads. No (simple way for) edge properties. Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 23
  • 24. Photo: Gephi, flickr The Property-Graph observes Friends Enemies Alice Bob Eve since = 2009/11/02 Hobby Hobby Enemies Crypto Guru Bruce A directed, attributed, multi-relational graph as central data structure for graph-databases. Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 24
  • 25. Photo: Jeffrey Beall, flickr The Property-Graph Currently no academically founded or well-defined graph data model, but… The Property-Graph Model http://wiki.github.com/tinkerpop/blueprints/property-graphmodel Rodriguez, M.A., Neubauer, P., “Constructions from Dots and Lines,” Bulletin of the American Society for Information Science and Technology, American Society for Information Science and Technology, volume 36, August 2010. …much more at http://markorodriguez.com ;) Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 25
  • 26. Photo: Gephi, flickr The Property-Graph // Neo4J example Node _alice = graphdb.createNode(); _alice.setProperty("Name", "Alice"); _alice.setProperty("Age", 21); Node _bob = graphdb.createNode(); _alice.setProperty("Name", „Bob"); _alice.setProperty("Age", 23); _alice.createRelationshipTo(_bob, Friends); _bob.createRelationshipTo(_alice, Friends); Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 26
  • 27. Photo: Gephi, flickr The Property-Graph // sones gql example CREATE VERTEX User ADD ATTRIBUTES (String Name, SET<User> Friends) INDICES (Name) INSERT INTO User VALUES (Name = "Alice", Age = 21) INSERT INTO User VALUES (Name = "Bob", Age = 23) LINK User(Name = ‘Alice') VIA Friends TO User(Name = ‘Bob') LINK User(Name = ‘Bob') VIA Friends TO User(Name = ‘Alice‘) Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 27
  • 28. Photo: Gephi, flickr Benefits of Graph-Databases • Explicit graph data model – No implicit model like within relational dbs – In the spirit of NoSQL: “One database for one data model” – GraphDB gets a change to understand the semantics and our intention Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 28
  • 29. Photo: Gephi, flickr Benefits of Graph-Databases • Index-free adjacency – No global adjacency index as each vertex acts as its own “mini-index” – The speed for traversing from one node to another is independed of the size of the graph ( Data locality and improved scalability ) – Optimized for random access patterns Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 29
  • 30. Photo: Gephi, flickr Graph-Database landscape Neo4J HyperGraphDB InfoGrid InfiniteGraph Filament OrientDB DEX VertexDB sones Marklogic? Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 30
  • 31. Photo: Gephi, flickr Data model Filament No Edge- VertexDB Properties Property-Graph InfoGrid DEX OrientDB Schemata Neo4J InfiniteGraph HyperGraphDB sones HyperGraphs Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 31
  • 32. Photo: Gephi, flickr Query methods Gremlin VertexDB Neo4J InfoGrid InfiniteGraph OrientDB Traverser API DEX Adapted SQL Filament HyperGraphDB HGQuery API sones GraphQL Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 32
  • 33. Photo: Gephi, flickr Programming languages Neo4J HyperGraphDB InfoGrid InfiniteGraph Filament Java DEX C# OrientDB C++ Lua sones VertexDB JS C Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 33
  • 34. Photo: Gephi, flickr License models AGPLv3 “Free Lunch” Neo4J InfoGrid DEX InfiniteGraph sones $$$ OrientDB HyperGraphDB LGPL Apache 2.0 Filament VertexDB BSD Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 34
  • 35. Photo: Gephi, flickr Protocols Neo4J HyperGraphDB InfoGrid InfiniteGraph OrientDB REST/JSON sones REST/XML DEX HTTP VertexDB Filament Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 35
  • 36. Photo: Gephi, flickr Neo4J • URL http://www.neo4j.org • Goals Embedded and spatial DBMS • APIs Java, JRuby, Ruby, Python, Scala, Clojure, C#, Erlang • Transactions Concurrent reads, Sync on nodes • Repl./Scaling Master-Slave, Master Failover • Persistency Proprietary ondisk format Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 36
  • 37. Photo: Gephi, flickr InfoGrid • URL http://www.infogrid.org • Goals Distributed GraphDB and Web- Application Framework • Transactions Yes • Repl./Scaling p2p-replication, partitioning • Persistency Proprietary, RDBMS, Hadoop MeshBase _GDB = StoreMeshBase.create(_MySQLStore); MeshObject _xkcd = _GDB.getMeshObjectLifecycleManager().createMeshObject(); _xkcd.setProperty("Name", "xkcd"); _xkcd.setProperty("Url", "http://www.xkcd.com"); _xkcd.relate(_good); Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 37
  • 38. Photo: Gephi, flickr DEX • URL http://www.dama.upc.edu • Goals High performance and integration of various information sources • Transactions n/a • Repl./Scaling n/a • Persistency Proprietary ondisk format long xkcd = _GDB.newNode(Resource); _GDB.setAttribute(xkcd, Name, "xkcd"); _GDB.setAttribute(xkcd, Url, "http://xkcd.com"); int Tags = _GDB.newUndirectedEdgeType("Tags"); long e2 = _GDB.newEdge(xkcd, good, Tags); Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 38
  • 39. Photo: Gephi, flickr HyperGraphDB • URL http://www.kobrix.com • Goals AI, NLP, Semantic Web • Transactions ACI(D), STM • Repl./Scaling p2p • Persistency Berkeley DB // Note: Website(), Tag() are POJO HGHandle _xkcd = _GDB.add(new Website("xkcd", "http://www.xkcd.com")); HGHandle _good = _GDB.add(new Tag("good"); HGHandle _xkcd2good = _GDB.add(new HGPlainLink(_xkcd, _good)); Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 39
  • 40. Photo: Gephi, flickr Infinite Graph • URL http://www.infinitegraph.com • Goals Very large graph databases • Transactions Distributed transactions • Repl./Scaling Objectivity/DB • Persistency Objectivity/DB // Note: Website(), Tag() extend BaseVertex; Edge() extends BaseEdge Website xkcd = new Website("xkcd", "http://xkcd.com/"); _GDB.addVertex(xkcd); Tag good = new Tag("good"); _GDB.addVertex(good); xkcd.addEdge (new Edge(), good, EdgeKind.BIDIRECTIONAL); Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 40
  • 41. Photo: Gephi, flickr OrientDB • URL http://www.orienttechnologies.com • Goals Providing a KV-Store, DocumentDB and a GraphDB in one solution • Transactions ACID, MVCC • Repl./Scaling n/a • Persistency Proprietary ondisk format OGraphVertex _xkcd = _GDB.createVertex(). set("Name", "xkcd"). set("Url", "http://www.xkcd.com"); OGraphVertex _good = _GDB.createVertex().set("Name", "good"); _xkcd.link (_good); Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 41
  • 42. Photo: Gephi, flickr sones • URL http://www.sones.de • Goals Management of linked (binary) data • Transactions ACID, MVCC (alpha) • Repl./Scaling p2p (alpha) • Persistency Proprietary file system Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 42
  • 43. Photo: Shayne Kaye, flickr sones Property-Hypergraph User Friend Bob ID = 2 since = 2009/09/21 Alter = 23 Alice SET<User> Friends ID = 1 SetMaxNumber = 12 Alter = 21 User Friend Carol ID = 3 since = 1997/04/11 Alter = 20 Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 43
  • 44. Photo: Shayne Kaye, flickr sones Property-Hypergraph User Friend Bob ID = 2 since = 2009/09/21 Alter = 23 Alice SET<User> Friends ID = 1 SetMaxNumber = 12 Alter = 21 User Friend Carol ID = 3 since = 1997/04/11 Alter = 20 Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 44
  • 45. Photo: Shayne Kaye, flickr sones Property-Hypergraph • Extensions to the Property Graph model • Multiple edges are be grouped by a hyperedge • Hyperedge may have their own properties • Properties may include code as data (e.g. C#: Func<…>, ExpressionTrees) • Allows calculations be done among the set of edges (GetMinWeight, SetMaxNumber, …) Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 45
  • 46. Photo: Shayne Kaye, flickr sones Special Edges and Properties LatestRevision Copy1 20100921… PropertyStream DefaultEdition Copy2 Vertex 20100918… Copy1 LatestRevision FileStream DefaultEdition 20100815… Copy1 Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 46
  • 47. Photo: Shayne Kaye, flickr sones Graph Query Language FROM User SELECT User.Friends.Friends.Name • “SQL for graphs” • Its goal is to provide a very user-friendly language for ad-hoc graph queries • Functions and aggregates are type-safe and can be extended by your own plug-ins, e.g. • SELECT COUNT(User.Friends) • SELECT User.Friends.Random(2) • SELECT User.Friends.Name.Substring(2,5) Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 47
  • 48. Photo: jonathanpercy, flickr Gremlin ./outE[@label=‘created’]/inV/inE[@label= ‘created’]/outV[g:except($_)]/@name • Get outgoing edges, where their label is equal to ‘created’ • Get their incoming vertices • Get incoming edges, where their labels equals ‘created’ • Get outgoing vertices, except the starting vertex • Get their name property Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 48
  • 49. For more information… achim@sones.de http://www.twitter.com/ahzf http://www.twitter.com/graphdbs Achim Friedland <achim@sones.de> NoSQL Frankfurt , 9/28/2010 49