Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

InfiniteGraph

1,445 views

Published on

  • Be the first to comment

InfiniteGraph

  1. 1. An overview of InfiniteGraph, the distributed graph database. Darren Wood Chief Architect, InfiniteGraph
  2. 2. For today’s discussion… <ul><li>InfiniteGraph </li></ul><ul><ul><li>Background / History </li></ul></ul><ul><ul><li>Architecture </li></ul></ul><ul><ul><li>What we are working on </li></ul></ul><ul><li>Consistency Models </li></ul><ul><li>Challenges/Requirements of Distibution </li></ul><ul><li>Data model – Schema vs Schema-less </li></ul>Copyright © InfiniteGraph
  3. 3. Building on Objectivity/DB <ul><li>Derived from Objectivity/DB core </li></ul><ul><ul><li>Distributed Object Database </li></ul></ul><ul><ul><li>Native core (broad platform support) with C++, Java, C# and Python Bindings </li></ul></ul><ul><ul><li>Thousands of deployments, many are 24x7x365 </li></ul></ul><ul><ul><li>Markets : VLDB, Data Fusion / Metadata, Complex Object Models, Relationship Analytics </li></ul></ul>Copyright © InfiniteGraph
  4. 4. Relationship Analytics Experience <ul><li>Relationship Analytics </li></ul><ul><ul><li>Large part of Objy/DB Government business </li></ul></ul><ul><ul><li>Significant growth in the commercial space </li></ul></ul><ul><ul><li>Many OEM opportunities </li></ul></ul><ul><ul><li>Mostly based on graph theory </li></ul></ul><ul><ul><li>Hand coded algorithms </li></ul></ul>Copyright © InfiniteGraph
  5. 5. Needed something more… <ul><li>Objectivity Provides </li></ul><ul><ul><li>Powerful distributed object storage </li></ul></ul><ul><ul><li>Flexible class based persistence </li></ul></ul><ul><ul><li>Enterprise DB features </li></ul></ul><ul><li>But…. </li></ul><ul><ul><li>Lots of custom coding </li></ul></ul><ul><ul><li>Complex deployment </li></ul></ul><ul><ul><li>No generic algorithm framework </li></ul></ul>Copyright © InfiniteGraph
  6. 6. Enter InfiniteGraph <ul><li>Simple Graph focused API </li></ul><ul><li>Automated distribution and deployment </li></ul><ul><li>Mostly configuration driven </li></ul><ul><li>Java class based persistence </li></ul><ul><li>Property model support </li></ul><ul><li>Asynchronous navigation </li></ul><ul><li>Indexing framework </li></ul>Copyright © InfiniteGraph
  7. 7. Some code… Copyright © InfiniteGraph Vertex alice = myGraph.addVertex(new Person(“Alice”)); Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); bob.addEdge(new Call(timestamp), charlie); Alice Carlos Charlie Bob Meets Calls Pays Calls
  8. 8. Basic Architecture Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Blueprints User Apps Objectivity/DB Distributed Database Session / TX Management Placement
  9. 9. Copyright © InfiniteGraph
  10. 10. Targeting Large Graphs <ul><li>Graphs grow quickly </li></ul><ul><ul><li>Billions of phone calls / day in US </li></ul></ul><ul><ul><li>Emails, social media events, IP Traffic </li></ul></ul><ul><ul><li>Financial transactions </li></ul></ul><ul><li>Some analytics require navigation of large sections of the graph </li></ul><ul><li>Must distribute data and go parallel </li></ul>Copyright © InfiniteGraph
  11. 11. Consistency Models <ul><li>Trading off full consistency for performance </li></ul><ul><li>Relax locking, allow stale reads </li></ul><ul><li>Gain 100x edge ingest rates </li></ul><ul><li>Great for “social graph” applications </li></ul><ul><li>These tend to have relaxed consistency requirements </li></ul><ul><li>API allows choice per operation </li></ul>Copyright © InfiniteGraph
  12. 12. Pipelining Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Session / TX Management Placement (MDP) Placement (Pipelining) V 1 V 2 V 3 E 12 E 23 Pipeline Manager Staging Containers Pipeline Containers E(1->2) E(3->1) E(2->3) E(2->1) E(2->3) E(3->1) E(1->2) E(3->2) E(1->2) E(2->3) E(3->1) E(2->1) E(2->3) E(3->1) E(3->2) E(1->2)
  13. 13. Distributing Navigation <ul><li>Graph algorithms naturally branch </li></ul><ul><li>Breaking up the process is relatively simple </li></ul><ul><li>Orchestrating it is more challenging </li></ul>Copyright © InfiniteGraph Alice Carlos Charlie Bob Meets Calls Pays Dave Eve Chuck Calls Lives With Meets
  14. 14. Partitioned Graphs are Ugly Copyright © InfiniteGraph Distributed API Application(s) Partition 1 Partition 3 Partition 2 Partition ... n Processor Processor Processor Processor
  15. 15. Making it scale… <ul><li>A distributed data tier supports parallel IO </li></ul><ul><li>A distributed cache/processor tier for processing “navlets” </li></ul><ul><li>Ability to deal with remote data reads (fast) </li></ul><ul><li>Navlets hit processor/cache instances based on consistent hashing of their start vertex </li></ul>Copyright © InfiniteGraph
  16. 16. Flexible Data Models <ul><li>Schema isn’t for everyone </li></ul><ul><ul><li>Great for relatively static models </li></ul></ul><ul><ul><li>Support for inheritance helps </li></ul></ul><ul><ul><li>So does auto-evolution </li></ul></ul><ul><li>Looking to support document style elements </li></ul><ul><ul><li>Support for ad-hoc attributes </li></ul></ul><ul><ul><li>Ability to mix both approaches </li></ul></ul>Copyright © InfiniteGraph
  17. 17. Other Projects <ul><li>Blueprints / Gremlin support </li></ul><ul><li>Native Visualizer and Management Console </li></ul><ul><li>Built in Qualifiers </li></ul><ul><li>More Indexing Enhancements </li></ul><ul><li>Algorithms </li></ul><ul><li>Large scale samples </li></ul>Copyright © InfiniteGraph
  18. 18. Thank you ! Copyright © InfiniteGraph [email_address] Twitter - @infinitegraph

×