0
Graph Databases : Connecting the Dots in Big Data Darren Wood Chief Architect, InfiniteGraph
Relationships are  everywhere
Graph Databases <ul><li>Not Really Graph Problems </li></ul><ul><ul><li>Average age of my customers that purchased X </li>...
Graph Databases <ul><li>Optimized around data relationships </li></ul><ul><ul><li>Relationships as first class citizens </...
Physical Storage Comparison Copyright © InfiniteGraph Meetings P1 Place Time P2 Alice Denver 5-27-10 Bob Calls From Time D...
Simple API Copyright © InfiniteGraph Vertex alice = myGraph.addVertex(new Person(“Alice”));  Vertex bob = myGraph.addVerte...
Query and Navigation <ul><li>Queries – but not as you know them </li></ul><ul><li>More like a rules based search and disco...
Navigation Example Copyright © InfiniteGraph // Create a qualifier that describes the target vertex Qualifier findCharlieP...
Navigational Query Performance
Scaling Graphs – Getting Data In Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Exten...
Accelerated Ingest Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Session ...
Choose Your Own Consistency… Copyright © InfiniteGraph // Describe your requested model using policies PolicyChain  myPoli...
Indexing Framework <ul><li>Focused on providing choice ! </li></ul><ul><li>Manual Indexes for grouping data </li></ul><ul>...
InfiniteGraph Visualizer Copyright © InfiniteGraph
Scaling Graphs – Distributed Navigation <ul><li>Graph algorithms naturally branch </li></ul><ul><li>Requires orchestration...
Big Distributed Data (Traditional - Huge Generalization) Copyright © InfiniteGraph Distributed API Application(s) Partitio...
Big Distributed Data (Graph) Copyright © InfiniteGraph Distributed API Application(s) Partition 1 Partition 3 Partition 2 ...
Some customers and partners
Thankyou ! Copyright © InfiniteGraph [email_address]
Upcoming SlideShare
Loading in...5
×

NOSQL Now! Presentation, August 24, 2011: Graph Databases: Connecting the Dots in Big Data

573

Published on

Darren Wood is the Architect and Lead Developer of InfiniteGraph, the distributed graph database, produced by Objectivity, Inc. Darren has spent the majority of his career architecting and building distributed systems with an emphasis on elastic scalability and data management. Prior to joining Objectivity, Inc. in 2007, Darren held positions as a Senior Consultant with IONA Technologies and a Development Team Lead for Citect Australia. Darren holds a First Class Honors Degree in Computer Systems Engineering from the University of Technology in Sydney, Australia.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
573
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Relationships and connections are EVERYWHERE. Examples include CRM, Telecom, Intelligence, Research, Healthcare, Finance and yes, social networks too. But notice, it’s absolutely not just about social networks, in the Facebook sense. ANY application that needs to find connections and relationships separated by more than 2 degrees, is a good candidate for InfiniteGraph.
  • InfiniteGraph  (built on Objectivity/DB) is optimized for high speed traversal of complex relationships. Compared to traditional technologies, we return results faster by several orders of magnitude, and performance will not degrade or suddenly drop-off.
  • Transcript of "NOSQL Now! Presentation, August 24, 2011: Graph Databases: Connecting the Dots in Big Data"

    1. 1. Graph Databases : Connecting the Dots in Big Data Darren Wood Chief Architect, InfiniteGraph
    2. 2. Relationships are everywhere
    3. 3. Graph Databases <ul><li>Not Really Graph Problems </li></ul><ul><ul><li>Average age of my customers that purchased X </li></ul></ul><ul><ul><li>Which zip code buys the most of Y </li></ul></ul><ul><li>Graph Problems </li></ul><ul><ul><li>How is person A connected to person B </li></ul></ul><ul><ul><li>Can suspect Y be associated with location Z </li></ul></ul><ul><ul><li>Who are influencers within a social network ? </li></ul></ul>Copyright © InfiniteGraph
    4. 4. Graph Databases <ul><li>Optimized around data relationships </li></ul><ul><ul><li>Relationships as first class citizens </li></ul></ul><ul><ul><li>Super fast traversal between entities </li></ul></ul><ul><ul><li>Rich/flexible annotation of connections </li></ul></ul><ul><li>Small focused API (typically not SQL) </li></ul><ul><ul><li>Natively work with concepts of Vertex/Edge </li></ul></ul><ul><ul><li>SQL has no concept of “navigation” </li></ul></ul><ul><ul><li>Most attempts based in SQL are convoluted </li></ul></ul>Copyright © InfiniteGraph
    5. 5. Physical Storage Comparison Copyright © InfiniteGraph Meetings P1 Place Time P2 Alice Denver 5-27-10 Bob Calls From Time Duration To Bob 13:20 25 Carlos Bob 17:10 15 Charlie Payments From Date Amount To Carlos 5-12-10 100000 Charlie Met 5-27-10 Alice Called 13:20 Bob Payed 100000 Carlos Charlie Called 17:10 Rows/Columns/Tables Relationship/Graph Optimized
    6. 6. Simple API Copyright © InfiniteGraph Vertex alice = myGraph.addVertex(new Person(“Alice”)); Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); bob.addEdge(new Call(timestamp), charlie); Alice Carlos Charlie Bob Meets Calls Pays Calls
    7. 7. Query and Navigation <ul><li>Queries – but not as you know them </li></ul><ul><li>More like a rules based search and discovery </li></ul><ul><li>Asynchronous Results </li></ul>Copyright © InfiniteGraph Alice Carlos Charlie Bob Meets Calls Pays Calls “ Find all paths between Alice and Charlie” “ Find all paths between Alice and Charlie – within 2 degrees” “ Find all paths between Alice and Charlie – events in May 2010”
    8. 8. Navigation Example Copyright © InfiniteGraph // Create a qualifier that describes the target vertex Qualifier findCharliePredicate = new VertexPredicate(personType, &quot;name == ’Charlie'&quot; ); // Construct a navigator which starts with Alice and uses a result qualifier // to find all paths in the graph to Charlie Navigator charlieFinder = alice.navigate( Guide.SIMPLE_BREADTH_FIRST, // default guide Qualifier.ANY, // no path constraints findCharliePredicate , // find paths ending with Charlie myResultHandler); // fire results to supplied handler // Start the navigator charlieFinder.start();
    9. 9. Navigational Query Performance
    10. 10. Scaling Graphs – Getting Data In Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Session / TX Management Placement Standard Blocking Ingest/Placement (MDP Plugin) Objectivity/DB App-1 (Ingest V 1 ) App-2 (Ingest V 2 ) App-3 (Ingest V 3 ) V 1 V 2 V 3 App-1 (E 1 2 { V 1 V 2 }) App-2 (E 23 { V 2 V 3 }) App-3 E 12 E 23
    11. 11. Accelerated Ingest Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Session / TX Management Placement (Standard) Placement (Accelerated) V 1 V 2 V 3 E 12 E 23 Distributed Pipelines Staging Containers Pipeline Containers E(1->2) E(3->1) E(2->3) E(2->1) E(2->3) E(3->1) E(1->2) E(3->2) E(1->2) E(2->3) E(3->1) E(2->1) E(2->3) E(3->1) E(3->2) E(1->2)
    12. 12. Choose Your Own Consistency… Copyright © InfiniteGraph // Describe your requested model using policies PolicyChain myPolicies = new PolicyChain(new EdgePipeliningPolicy( true )); // Start a transaction with the policies you want Transaction tx = myGraph.beginTransaction( AccessMode.READ_WRITE, myPolicies); // This code doesn’t change, can be used with any policies alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); tx.commit();
    13. 13. Indexing Framework <ul><li>Focused on providing choice ! </li></ul><ul><li>Manual Indexes for grouping data </li></ul><ul><li>Automatic Indexes for cross population </li></ul><ul><li>Query interface with qualification language </li></ul><ul><li>Pluggable query operators </li></ul><ul><li>External index support (Lucene) </li></ul>Copyright © InfiniteGraph
    14. 14. InfiniteGraph Visualizer Copyright © InfiniteGraph
    15. 15. Scaling Graphs – Distributed Navigation <ul><li>Graph algorithms naturally branch </li></ul><ul><li>Requires orchestration of threads/agents </li></ul>Copyright © InfiniteGraph Alice Carlos Charlie Bob Meets Calls Pays Dave Eve Chuck Calls Lives With Meets
    16. 16. Big Distributed Data (Traditional - Huge Generalization) Copyright © InfiniteGraph Distributed API Application(s) Partition 1 Partition 3 Partition 2 Partition ... n Processor Processor Processor Processor
    17. 17. Big Distributed Data (Graph) Copyright © InfiniteGraph Distributed API Application(s) Partition 1 Partition 3 Partition 2 Partition ... n Processor Processor Processor Processor
    18. 18. Some customers and partners
    19. 19. Thankyou ! Copyright © InfiniteGraph [email_address]
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×