Graph Databases – What’s so Different ? Darren Wood Chief Architect, InfiniteGraph
Graph Databases <ul><li>Key technical attributes </li></ul><ul><li>How Infinite Graph addresses these </li></ul><ul><li>Qu...
Graph Databases <ul><li>Optimized around data relationships </li></ul><ul><ul><li>Relationships as first class citizens </...
Some Code Copyright © InfiniteGraph Vertex alice = myGraph.addVertex(new Person(“Alice”));  Vertex bob = myGraph.addVertex...
Physical Storage Comparison Copyright © InfiniteGraph Meetings P1 Place Time P2 Alice Denver 5-27-10 Bob Calls From Time D...
Query and Navigation <ul><li>Queries – but not as you know them </li></ul><ul><li>More like a rules based search and disco...
Navigation Example Copyright © InfiniteGraph // Create a qualifier that describes the target vertex Qualifier findCharlieP...
Management of Large Data Graphs <ul><li>Graphs grow quickly </li></ul><ul><ul><li>Billions of phone calls / day in US </li...
Distributing Navigation <ul><li>Graph algorithms naturally branch </li></ul><ul><li>Can be automated or guided </li></ul>C...
Partitioning – Typical Big Data Copyright © InfiniteGraph Distributed API Application(s) Partition 1 Partition 3 Partition...
Graph Partitioning <ul><li>Graph partitioning is not as simple </li></ul><ul><li>Graph operations are rarely partition bou...
Graph Partitioning – Reality ! Copyright © InfiniteGraph Distributed API Application(s) Partition 1 Partition 3 Partition ...
Distributed Graph Must Haves <ul><li>High performance distributed persistence </li></ul><ul><li>Ability to deal with remot...
Practical Applications Copyright © InfiniteGraph
Pathfinding <ul><ul><li>Intelligence, Police, Counter Terrorism </li></ul></ul><ul><ul><li>Financial Transactions and Frau...
Graph Analysis (Algorithms) Copyright © InfiniteGraph Bob Sam Degree Centrality Closeness and Betweeness Centrality Fred Jo
Graph Analysis (Algorithms) <ul><li>Social Networks </li></ul><ul><ul><li>Most connected participants </li></ul></ul><ul><...
Graph Analysis (Patterns) <ul><li>Crime (again) </li></ul><ul><ul><li>Recognize common patterns of activity </li></ul></ul...
Many Many More ! <ul><li>Spatial data </li></ul><ul><li>Defence / Situational Awareness </li></ul><ul><li>Sciences </li></...
Thankyou ! Copyright © InfiniteGraph [email_address] Twitter - @infinitegraph
Upcoming SlideShare
Loading in …5
×

New Data Technologies, Graph Computing and Relationship Discovery in the Enterprise - Darren

3,532 views

Published on

Presentation slides by Darren Wood, Lead Architect for InfiniteGraph, the distributed graph database.

Published in: Technology

New Data Technologies, Graph Computing and Relationship Discovery in the Enterprise - Darren

  1. 1. Graph Databases – What’s so Different ? Darren Wood Chief Architect, InfiniteGraph
  2. 2. Graph Databases <ul><li>Key technical attributes </li></ul><ul><li>How Infinite Graph addresses these </li></ul><ul><li>Query and navigation </li></ul><ul><li>Challenges/Requirements of Distribution </li></ul><ul><li>Practical applications </li></ul>Copyright © InfiniteGraph
  3. 3. Graph Databases <ul><li>Optimized around data relationships </li></ul><ul><ul><li>Relationships as first class citizens </li></ul></ul><ul><ul><li>Super fast traversal between entities </li></ul></ul><ul><ul><li>Rich/flexible annotation of connections </li></ul></ul><ul><li>Small focused API (typically not SQL) </li></ul><ul><ul><li>Natively work with concepts of Vertex/Edge </li></ul></ul><ul><ul><li>SQL has no concept of “navigation” </li></ul></ul><ul><ul><li>Most attempts based in SQL are convoluted </li></ul></ul>Copyright © InfiniteGraph
  4. 4. Some Code Copyright © InfiniteGraph Vertex alice = myGraph.addVertex(new Person(“Alice”)); Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); bob.addEdge(new Call(timestamp), charlie); Alice Carlos Charlie Bob Meets Calls Pays Calls
  5. 5. Physical Storage Comparison Copyright © InfiniteGraph Meetings P1 Place Time P2 Alice Denver 5-27-10 Bob Calls From Time Duration To Bob 13:20 25 Carlos Bob 17:10 15 Charlie Payments From Date Amount To Carlos 5-12-10 100000 Charlie Met 5-27-10 Alice Called 13:20 Bob Payed 100000 Carlos Charlie Called 17:10 Rows/Columns/Tables Relationship/Graph Optimized
  6. 6. Query and Navigation <ul><li>Queries – but not as you know them </li></ul><ul><li>More like a rules based search and discovery </li></ul><ul><li>Asynchronous Results </li></ul>Copyright © InfiniteGraph Alice Carlos Charlie Bob Meets Calls Pays Calls “ Find all paths between Alice and Charlie” “ Find all paths between Alice and Charlie – within 2 degrees” “ Find all paths between Alice and Charlie – events in May 2010”
  7. 7. Navigation Example Copyright © InfiniteGraph // Create a qualifier that describes the target vertex Qualifier findCharliePredicate = new VertexPredicate(personType, &quot;name == ’Charlie'&quot; ); // Construct a navigator which starts with Alice and uses a result qualifier // to find all paths in the graph to Charlie Navigator charlieFinder = alice.navigate( Guide.SIMPLE_BREADTH_FIRST, // default guide Qualifier.ANY, // no path constraints findCharliePredicate , // find paths ending with Charlie myResultHandler); // fire results to supplied handler // Start the navigator charlieFinder.start();
  8. 8. Management of Large Data Graphs <ul><li>Graphs grow quickly </li></ul><ul><ul><li>Billions of phone calls / day in US </li></ul></ul><ul><ul><li>Emails, social media events, IP Traffic </li></ul></ul><ul><ul><li>Financial transactions </li></ul></ul><ul><li>Some analytics require navigation of large sections of the graph </li></ul><ul><li>Each step (often) depends on the last </li></ul><ul><li>Must distribute data and go parallel </li></ul>Copyright © InfiniteGraph
  9. 9. Distributing Navigation <ul><li>Graph algorithms naturally branch </li></ul><ul><li>Can be automated or guided </li></ul>Copyright © InfiniteGraph Alice Carlos Charlie Bob Meets Calls Pays Dave Eve Chuck Calls Lives With Meets
  10. 10. Partitioning – Typical Big Data Copyright © InfiniteGraph Distributed API Application(s) Partition 1 Partition 3 Partition 2 Partition ... n Processor Processor Processor Processor
  11. 11. Graph Partitioning <ul><li>Graph partitioning is not as simple </li></ul><ul><li>Graph operations are rarely partition bound </li></ul><ul><li>Graphs are ‘alive’ </li></ul><ul><li>Repartitioning is expensive </li></ul><ul><li>Partitions must co-operate </li></ul>Copyright © InfiniteGraph
  12. 12. Graph Partitioning – Reality ! Copyright © InfiniteGraph Distributed API Application(s) Partition 1 Partition 3 Partition 2 Partition ... n Processor Processor Processor Processor
  13. 13. Distributed Graph Must Haves <ul><li>High performance distributed persistence </li></ul><ul><li>Ability to deal with remote data reads (fast) </li></ul><ul><li>Intelligent local cache of subgraphs </li></ul><ul><li>Distributed navigation processing </li></ul><ul><li>Distributed, multi-source concurrent ingest </li></ul><ul><li>Write modes supporting both strict and eventual consistency </li></ul>Copyright © InfiniteGraph
  14. 14. Practical Applications Copyright © InfiniteGraph
  15. 15. Pathfinding <ul><ul><li>Intelligence, Police, Counter Terrorism </li></ul></ul><ul><ul><li>Financial Transactions and Fraud </li></ul></ul><ul><ul><li>Discover Paths Between Entities </li></ul></ul><ul><ul><ul><li>One to One, Many to Many </li></ul></ul></ul><ul><ul><li>Constrain by </li></ul></ul><ul><ul><ul><li>Edge and Vertex Type </li></ul></ul></ul><ul><ul><ul><li>Edge and Vertex Attributes (including temporal) </li></ul></ul></ul><ul><ul><ul><li>Required Sequence of Events </li></ul></ul></ul><ul><ul><ul><li>Length (Hops), Total Edge Weight </li></ul></ul></ul>Copyright © InfiniteGraph
  16. 16. Graph Analysis (Algorithms) Copyright © InfiniteGraph Bob Sam Degree Centrality Closeness and Betweeness Centrality Fred Jo
  17. 17. Graph Analysis (Algorithms) <ul><li>Social Networks </li></ul><ul><ul><li>Most connected participants </li></ul></ul><ul><ul><li>Influencers </li></ul></ul><ul><ul><li>Important Syndicates or Sub-networks </li></ul></ul><ul><li>Central figures in crime organisations </li></ul><ul><li>Business Intelligence </li></ul><ul><ul><li>Discovering Knowledge Assets </li></ul></ul><ul><ul><li>Complex analytics </li></ul></ul>Copyright © InfiniteGraph
  18. 18. Graph Analysis (Patterns) <ul><li>Crime (again) </li></ul><ul><ul><li>Recognize common patterns of activity </li></ul></ul><ul><ul><li>Complex chains of interaction </li></ul></ul><ul><li>Security </li></ul><ul><ul><li>Recognize attack/threat patterns </li></ul></ul><ul><ul><li>Auditing / log analytics </li></ul></ul><ul><li>Targeting Advertising </li></ul><ul><ul><li>To specific browsing patterns </li></ul></ul>Copyright © InfiniteGraph
  19. 19. Many Many More ! <ul><li>Spatial data </li></ul><ul><li>Defence / Situational Awareness </li></ul><ul><li>Sciences </li></ul><ul><li>Health Care </li></ul><ul><li>Genealogy </li></ul><ul><li>Logistics </li></ul><ul><li>Tracking </li></ul>Copyright © InfiniteGraph
  20. 20. Thankyou ! Copyright © InfiniteGraph [email_address] Twitter - @infinitegraph

×