Graph Database Overview and Feature Update Darren Wood Chief Architect, InfiniteGraph
History <ul><li>Objectivity – Massively scalable, distributed object oriented database </li></ul><ul><ul><li>Used in Gover...
Graph Databases <ul><li>Key technical attributes </li></ul><ul><li>How Infinite Graph addresses these </li></ul><ul><li>Qu...
Graph Databases <ul><li>Optimized around data relationships </li></ul><ul><ul><li>Relationships as first class citizens </...
Distributed Graph Must Haves <ul><li>High performance distributed persistence </li></ul><ul><li>Ability to deal with remot...
Some Code Copyright © InfiniteGraph Vertex alice = myGraph.addVertex(new Person(“Alice”));  Vertex bob = myGraph.addVertex...
Physical Storage Comparison Copyright © InfiniteGraph Meetings P1 Place Time P2 Alice Denver 5-27-10 Bob Calls From Time D...
Query and Navigation <ul><li>Queries – but not as you know them </li></ul><ul><li>More like a rules based search and disco...
Navigation Example Copyright © InfiniteGraph // Create a qualifier that describes the target vertex Qualifier findCharlieP...
Management of Large Data Graphs <ul><li>Graphs grow quickly </li></ul><ul><ul><li>Billions of phone calls / day in US </li...
Basic Architecture Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Blueprin...
Feature Update Copyright © InfiniteGraph 2.0
Accelerated Ingest Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Session ...
Accelerated Ingest Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Session ...
InfiniteGraph Visualizer <ul><li>Really nice flexible graph viewer </li></ul><ul><li>Browser style navigation and history ...
InfiniteGraph Visualizer Copyright © InfiniteGraph
InfiniteGraph Visualizer Copyright © InfiniteGraph
Indexing Framework <ul><li>Focused on providing choice ! </li></ul><ul><li>Manual Indexes for grouping data </li></ul><ul>...
<ul><li>Automated Distributed Navigation </li></ul><ul><li>Stored Loadable Navigators </li></ul><ul><li>Visualizer Navigat...
Thankyou ! Copyright © InfiniteGraph [email_address]
Upcoming SlideShare
Loading in …5
×

Meetup: An Introduction to InfiniteGraph, and Connecting the Dots in Big Data.

909 views
853 views

Published on

On August 17, 2011, the InfiniteGraph team hosted a local Meetup attended by dozens of senior developers working on large scale enterprise and startup projects. Big Data problems are quickly presenting themselves in almost every area of computing from Social Network Analysis to File Processing. Many technologies, such as those in the NoSQL space were developed in response to the limitations of current storage systems as an effective mechanism to deal with these mountains of data. And much of that data is interconnected in ways that, when organized properly, gives interesting and often valuable information. InfiniteGraph was designed specifically to traverse complex relationships in big data, and provide the framework for products built to provide real-time network analysis, business decision support and relationship analytics. Speakers: Thomas Krafft, Director of Marketing, InfiniteGraph. Darren Wood, Chief Architect, InfiniteGraph. Mark Maagdenberg, Senior Field Engineer, InfiniteGraph.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
909
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
15
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Meetup: An Introduction to InfiniteGraph, and Connecting the Dots in Big Data.

  1. 1. Graph Database Overview and Feature Update Darren Wood Chief Architect, InfiniteGraph
  2. 2. History <ul><li>Objectivity – Massively scalable, distributed object oriented database </li></ul><ul><ul><li>Used in Government (DoD, Intelligence) </li></ul></ul><ul><ul><ul><li>Machine generated data such as sensor, acoustic… </li></ul></ul></ul><ul><ul><li>OEM Markets </li></ul></ul><ul><ul><ul><li>Either complex data models, or high ingest or both </li></ul></ul></ul><ul><li>Significant technical advantage in highly connected (many-to-many) data models </li></ul>Copyright © InfiniteGraph
  3. 3. Graph Databases <ul><li>Key technical attributes </li></ul><ul><li>How Infinite Graph addresses these </li></ul><ul><li>Query and navigation </li></ul><ul><li>Challenges/Requirements of Distribution </li></ul><ul><li>Practical applications </li></ul>Copyright © InfiniteGraph
  4. 4. Graph Databases <ul><li>Optimized around data relationships </li></ul><ul><ul><li>Relationships as first class citizens </li></ul></ul><ul><ul><li>Super fast traversal between entities </li></ul></ul><ul><ul><li>Rich/flexible annotation of connections </li></ul></ul><ul><li>Small focused API (typically not SQL) </li></ul><ul><ul><li>Natively work with concepts of Vertex/Edge </li></ul></ul><ul><ul><li>SQL has no concept of “navigation” </li></ul></ul><ul><ul><li>Most attempts based in SQL are convoluted </li></ul></ul>Copyright © InfiniteGraph
  5. 5. Distributed Graph Must Haves <ul><li>High performance distributed persistence </li></ul><ul><li>Ability to deal with remote data reads (fast) </li></ul><ul><li>Intelligent local cache of subgraphs </li></ul><ul><li>Distributed navigation processing </li></ul><ul><li>Distributed, multi-source concurrent ingest </li></ul><ul><li>Write modes supporting both strict and eventual consistency </li></ul>Copyright © InfiniteGraph
  6. 6. Some Code Copyright © InfiniteGraph Vertex alice = myGraph.addVertex(new Person(“Alice”)); Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); bob.addEdge(new Call(timestamp), charlie); Alice Carlos Charlie Bob Meets Calls Pays Calls
  7. 7. Physical Storage Comparison Copyright © InfiniteGraph Meetings P1 Place Time P2 Alice Denver 5-27-10 Bob Calls From Time Duration To Bob 13:20 25 Carlos Bob 17:10 15 Charlie Payments From Date Amount To Carlos 5-12-10 100000 Charlie Met 5-27-10 Alice Called 13:20 Bob Payed 100000 Carlos Charlie Called 17:10 Rows/Columns/Tables Relationship/Graph Optimized
  8. 8. Query and Navigation <ul><li>Queries – but not as you know them </li></ul><ul><li>More like a rules based search and discovery </li></ul><ul><li>Asynchronous Results </li></ul>Copyright © InfiniteGraph Alice Carlos Charlie Bob Meets Calls Pays Calls “ Find all paths between Alice and Charlie” “ Find all paths between Alice and Charlie – within 2 degrees” “ Find all paths between Alice and Charlie – events in May 2010”
  9. 9. Navigation Example Copyright © InfiniteGraph // Create a qualifier that describes the target vertex Qualifier findCharliePredicate = new VertexPredicate(personType, &quot;name == ’Charlie'&quot; ); // Construct a navigator which starts with Alice and uses a result qualifier // to find all paths in the graph to Charlie Navigator charlieFinder = alice.navigate( Guide.SIMPLE_BREADTH_FIRST, // default guide Qualifier.ANY, // no path constraints findCharliePredicate , // find paths ending with Charlie myResultHandler); // fire results to supplied handler // Start the navigator charlieFinder.start();
  10. 10. Management of Large Data Graphs <ul><li>Graphs grow quickly </li></ul><ul><ul><li>Billions of phone calls / day in US </li></ul></ul><ul><ul><li>Emails, social media events, IP Traffic </li></ul></ul><ul><ul><li>Financial transactions </li></ul></ul><ul><li>Some analytics require navigation of large sections of the graph </li></ul><ul><li>Each step (often) depends on the last </li></ul><ul><li>Must distribute data and go parallel </li></ul>Copyright © InfiniteGraph
  11. 11. Basic Architecture Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Blueprints User Apps Objectivity/DB Distributed Database Session / TX Management Placement
  12. 12. Feature Update Copyright © InfiniteGraph 2.0
  13. 13. Accelerated Ingest Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Session / TX Management Placement Standard Blocking Ingest/Placement (MDP Plugin) Objectivity/DB App-1 (Ingest V 1 ) App-2 (Ingest V 2 ) App-3 (Ingest V 3 ) V 1 V 2 V 3 App-1 (E 1 2 { V 1 V 2 }) App-2 (E 23 { V 2 V 3 }) App-3 E 12 E 23
  14. 14. Accelerated Ingest Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Session / TX Management Placement (Standard) Placement (Accelerated) V 1 V 2 V 3 E 12 E 23 Distributed Pipelines Staging Containers Pipeline Containers E(1->2) E(3->1) E(2->3) E(2->1) E(2->3) E(3->1) E(1->2) E(3->2) E(1->2) E(2->3) E(3->1) E(2->1) E(2->3) E(3->1) E(3->2) E(1->2)
  15. 15. InfiniteGraph Visualizer <ul><li>Really nice flexible graph viewer </li></ul><ul><li>Browser style navigation and history </li></ul><ul><li>Full index support – search your data </li></ul><ul><li>Display connections around a selected point </li></ul><ul><li>Fully customize display to your data model </li></ul><ul><li>Full data view via selection </li></ul>Copyright © InfiniteGraph
  16. 16. InfiniteGraph Visualizer Copyright © InfiniteGraph
  17. 17. InfiniteGraph Visualizer Copyright © InfiniteGraph
  18. 18. Indexing Framework <ul><li>Focused on providing choice ! </li></ul><ul><li>Manual Indexes for grouping data </li></ul><ul><li>Automatic Indexes for cross population </li></ul><ul><li>Query interface with qualification language </li></ul><ul><li>Pluggable query operators </li></ul><ul><li>External index support (Lucene) </li></ul>Copyright © InfiniteGraph
  19. 19. <ul><li>Automated Distributed Navigation </li></ul><ul><li>Stored Loadable Navigators </li></ul><ul><li>Visualizer Navigation Plugins </li></ul><ul><li>More Visualizer Enhancements </li></ul><ul><li>More Import/Export support </li></ul>Copyright © InfiniteGraph >> next
  20. 20. Thankyou ! Copyright © InfiniteGraph [email_address]

×