Published on

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. An overview of InfiniteGraph, the distributed graph database. Darren Wood Chief Architect, InfiniteGraph
  2. 2. For today’s discussion… <ul><li>InfiniteGraph </li></ul><ul><ul><li>Background / History </li></ul></ul><ul><ul><li>Architecture </li></ul></ul><ul><ul><li>What we are working on </li></ul></ul><ul><li>Consistency Models </li></ul><ul><li>Challenges/Requirements of Distibution </li></ul><ul><li>Data model – Schema vs Schema-less </li></ul>Copyright © InfiniteGraph
  3. 3. Building on Objectivity/DB <ul><li>Derived from Objectivity/DB core </li></ul><ul><ul><li>Distributed Object Database </li></ul></ul><ul><ul><li>Native core (broad platform support) with C++, Java, C# and Python Bindings </li></ul></ul><ul><ul><li>Thousands of deployments, many are 24x7x365 </li></ul></ul><ul><ul><li>Markets : VLDB, Data Fusion / Metadata, Complex Object Models, Relationship Analytics </li></ul></ul>Copyright © InfiniteGraph
  4. 4. Relationship Analytics Experience <ul><li>Relationship Analytics </li></ul><ul><ul><li>Large part of Objy/DB Government business </li></ul></ul><ul><ul><li>Significant growth in the commercial space </li></ul></ul><ul><ul><li>Many OEM opportunities </li></ul></ul><ul><ul><li>Mostly based on graph theory </li></ul></ul><ul><ul><li>Hand coded algorithms </li></ul></ul>Copyright © InfiniteGraph
  5. 5. Needed something more… <ul><li>Objectivity Provides </li></ul><ul><ul><li>Powerful distributed object storage </li></ul></ul><ul><ul><li>Flexible class based persistence </li></ul></ul><ul><ul><li>Enterprise DB features </li></ul></ul><ul><li>But…. </li></ul><ul><ul><li>Lots of custom coding </li></ul></ul><ul><ul><li>Complex deployment </li></ul></ul><ul><ul><li>No generic algorithm framework </li></ul></ul>Copyright © InfiniteGraph
  6. 6. Enter InfiniteGraph <ul><li>Simple Graph focused API </li></ul><ul><li>Automated distribution and deployment </li></ul><ul><li>Mostly configuration driven </li></ul><ul><li>Java class based persistence </li></ul><ul><li>Property model support </li></ul><ul><li>Asynchronous navigation </li></ul><ul><li>Indexing framework </li></ul>Copyright © InfiniteGraph
  7. 7. Some code… Copyright © InfiniteGraph Vertex alice = myGraph.addVertex(new Person(“Alice”)); Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); bob.addEdge(new Call(timestamp), charlie); Alice Carlos Charlie Bob Meets Calls Pays Calls
  8. 8. Basic Architecture Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Blueprints User Apps Objectivity/DB Distributed Database Session / TX Management Placement
  9. 9. Copyright © InfiniteGraph
  10. 10. Targeting Large Graphs <ul><li>Graphs grow quickly </li></ul><ul><ul><li>Billions of phone calls / day in US </li></ul></ul><ul><ul><li>Emails, social media events, IP Traffic </li></ul></ul><ul><ul><li>Financial transactions </li></ul></ul><ul><li>Some analytics require navigation of large sections of the graph </li></ul><ul><li>Must distribute data and go parallel </li></ul>Copyright © InfiniteGraph
  11. 11. Consistency Models <ul><li>Trading off full consistency for performance </li></ul><ul><li>Relax locking, allow stale reads </li></ul><ul><li>Gain 100x edge ingest rates </li></ul><ul><li>Great for “social graph” applications </li></ul><ul><li>These tend to have relaxed consistency requirements </li></ul><ul><li>API allows choice per operation </li></ul>Copyright © InfiniteGraph
  12. 12. Pipelining Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Session / TX Management Placement (MDP) Placement (Pipelining) V 1 V 2 V 3 E 12 E 23 Pipeline Manager Staging Containers Pipeline Containers E(1->2) E(3->1) E(2->3) E(2->1) E(2->3) E(3->1) E(1->2) E(3->2) E(1->2) E(2->3) E(3->1) E(2->1) E(2->3) E(3->1) E(3->2) E(1->2)
  13. 13. Distributing Navigation <ul><li>Graph algorithms naturally branch </li></ul><ul><li>Breaking up the process is relatively simple </li></ul><ul><li>Orchestrating it is more challenging </li></ul>Copyright © InfiniteGraph Alice Carlos Charlie Bob Meets Calls Pays Dave Eve Chuck Calls Lives With Meets
  14. 14. Partitioned Graphs are Ugly Copyright © InfiniteGraph Distributed API Application(s) Partition 1 Partition 3 Partition 2 Partition ... n Processor Processor Processor Processor
  15. 15. Making it scale… <ul><li>A distributed data tier supports parallel IO </li></ul><ul><li>A distributed cache/processor tier for processing “navlets” </li></ul><ul><li>Ability to deal with remote data reads (fast) </li></ul><ul><li>Navlets hit processor/cache instances based on consistent hashing of their start vertex </li></ul>Copyright © InfiniteGraph
  16. 16. Flexible Data Models <ul><li>Schema isn’t for everyone </li></ul><ul><ul><li>Great for relatively static models </li></ul></ul><ul><ul><li>Support for inheritance helps </li></ul></ul><ul><ul><li>So does auto-evolution </li></ul></ul><ul><li>Looking to support document style elements </li></ul><ul><ul><li>Support for ad-hoc attributes </li></ul></ul><ul><ul><li>Ability to mix both approaches </li></ul></ul>Copyright © InfiniteGraph
  17. 17. Other Projects <ul><li>Blueprints / Gremlin support </li></ul><ul><li>Native Visualizer and Management Console </li></ul><ul><li>Built in Qualifiers </li></ul><ul><li>More Indexing Enhancements </li></ul><ul><li>Algorithms </li></ul><ul><li>Large scale samples </li></ul>Copyright © InfiniteGraph
  18. 18. Thank you ! Copyright © InfiniteGraph [email_address] Twitter - @infinitegraph