Your SlideShare is downloading. ×
  • Like
  • Save

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

New Data Technologies, Graph Computing and Relationship Discovery in the Enterprise - Darren

  • 3,157 views
Published

Presentation slides by Darren Wood, Lead Architect for InfiniteGraph, the distributed graph database.

Presentation slides by Darren Wood, Lead Architect for InfiniteGraph, the distributed graph database.

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
3,157
On SlideShare
0
From Embeds
0
Number of Embeds
4

Actions

Shares
Downloads
0
Comments
0
Likes
5

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Graph Databases – What’s so Different ? Darren Wood Chief Architect, InfiniteGraph
  • 2. Graph Databases
    • Key technical attributes
    • How Infinite Graph addresses these
    • Query and navigation
    • Challenges/Requirements of Distribution
    • Practical applications
    Copyright © InfiniteGraph
  • 3. Graph Databases
    • Optimized around data relationships
      • Relationships as first class citizens
      • Super fast traversal between entities
      • Rich/flexible annotation of connections
    • Small focused API (typically not SQL)
      • Natively work with concepts of Vertex/Edge
      • SQL has no concept of “navigation”
      • Most attempts based in SQL are convoluted
    Copyright © InfiniteGraph
  • 4. Some Code Copyright © InfiniteGraph Vertex alice = myGraph.addVertex(new Person(“Alice”)); Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); bob.addEdge(new Call(timestamp), charlie); Alice Carlos Charlie Bob Meets Calls Pays Calls
  • 5. Physical Storage Comparison Copyright © InfiniteGraph Meetings P1 Place Time P2 Alice Denver 5-27-10 Bob Calls From Time Duration To Bob 13:20 25 Carlos Bob 17:10 15 Charlie Payments From Date Amount To Carlos 5-12-10 100000 Charlie Met 5-27-10 Alice Called 13:20 Bob Payed 100000 Carlos Charlie Called 17:10 Rows/Columns/Tables Relationship/Graph Optimized
  • 6. Query and Navigation
    • Queries – but not as you know them
    • More like a rules based search and discovery
    • Asynchronous Results
    Copyright © InfiniteGraph Alice Carlos Charlie Bob Meets Calls Pays Calls “ Find all paths between Alice and Charlie” “ Find all paths between Alice and Charlie – within 2 degrees” “ Find all paths between Alice and Charlie – events in May 2010”
  • 7. Navigation Example Copyright © InfiniteGraph // Create a qualifier that describes the target vertex Qualifier findCharliePredicate = new VertexPredicate(personType, "name == ’Charlie'" ); // Construct a navigator which starts with Alice and uses a result qualifier // to find all paths in the graph to Charlie Navigator charlieFinder = alice.navigate( Guide.SIMPLE_BREADTH_FIRST, // default guide Qualifier.ANY, // no path constraints findCharliePredicate , // find paths ending with Charlie myResultHandler); // fire results to supplied handler // Start the navigator charlieFinder.start();
  • 8. Management of Large Data Graphs
    • Graphs grow quickly
      • Billions of phone calls / day in US
      • Emails, social media events, IP Traffic
      • Financial transactions
    • Some analytics require navigation of large sections of the graph
    • Each step (often) depends on the last
    • Must distribute data and go parallel
    Copyright © InfiniteGraph
  • 9. Distributing Navigation
    • Graph algorithms naturally branch
    • Can be automated or guided
    Copyright © InfiniteGraph Alice Carlos Charlie Bob Meets Calls Pays Dave Eve Chuck Calls Lives With Meets
  • 10. Partitioning – Typical Big Data Copyright © InfiniteGraph Distributed API Application(s) Partition 1 Partition 3 Partition 2 Partition ... n Processor Processor Processor Processor
  • 11. Graph Partitioning
    • Graph partitioning is not as simple
    • Graph operations are rarely partition bound
    • Graphs are ‘alive’
    • Repartitioning is expensive
    • Partitions must co-operate
    Copyright © InfiniteGraph
  • 12. Graph Partitioning – Reality ! Copyright © InfiniteGraph Distributed API Application(s) Partition 1 Partition 3 Partition 2 Partition ... n Processor Processor Processor Processor
  • 13. Distributed Graph Must Haves
    • High performance distributed persistence
    • Ability to deal with remote data reads (fast)
    • Intelligent local cache of subgraphs
    • Distributed navigation processing
    • Distributed, multi-source concurrent ingest
    • Write modes supporting both strict and eventual consistency
    Copyright © InfiniteGraph
  • 14. Practical Applications Copyright © InfiniteGraph
  • 15. Pathfinding
      • Intelligence, Police, Counter Terrorism
      • Financial Transactions and Fraud
      • Discover Paths Between Entities
        • One to One, Many to Many
      • Constrain by
        • Edge and Vertex Type
        • Edge and Vertex Attributes (including temporal)
        • Required Sequence of Events
        • Length (Hops), Total Edge Weight
    Copyright © InfiniteGraph
  • 16. Graph Analysis (Algorithms) Copyright © InfiniteGraph Bob Sam Degree Centrality Closeness and Betweeness Centrality Fred Jo
  • 17. Graph Analysis (Algorithms)
    • Social Networks
      • Most connected participants
      • Influencers
      • Important Syndicates or Sub-networks
    • Central figures in crime organisations
    • Business Intelligence
      • Discovering Knowledge Assets
      • Complex analytics
    Copyright © InfiniteGraph
  • 18. Graph Analysis (Patterns)
    • Crime (again)
      • Recognize common patterns of activity
      • Complex chains of interaction
    • Security
      • Recognize attack/threat patterns
      • Auditing / log analytics
    • Targeting Advertising
      • To specific browsing patterns
    Copyright © InfiniteGraph
  • 19. Many Many More !
    • Spatial data
    • Defence / Situational Awareness
    • Sciences
    • Health Care
    • Genealogy
    • Logistics
    • Tracking
    Copyright © InfiniteGraph
  • 20. Thankyou ! Copyright © InfiniteGraph [email_address] Twitter - @infinitegraph