Gluecon InfiniteGraph Presentation: Scaling the Social Graph in the Cloud

Scaling the [Social] Graph in the [Cloud] Darren Wood Lead Architect, InfiniteGraph

Graph Databases (Quickly) • Optimized around data relationships • Small focused API (typically not SQL) • Typical Use Cases : – Social Graph Analysis – Catching Bad Guys (see Booth 16) – Fraud / Financial (more bad guys) – Data Intensive Science – Web / Advertising Analytics Copyright © InfiniteGraph

Graph Databases (Almost Done) Vertex alice = myGraph.addVertex(new Person(“Alice”)); Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); bob.addEdge(new Call(timestamp), charlie); Alice Bob Carlos Charlie Meets Calls Pays Calls Copyright © InfiniteGraph

What’s So Difficult Then ? • Graphs grow quickly – Billions of phone calls / day in US – Emails, social media events, IP Traffic – Financial transactions • Some analytics require navigation of large sections of the graph • Each step (often) depends on the last • Must distribute data and go parallel Copyright © InfiniteGraph

Big Distributed Data (Traditional - Huge Generalization) Application(s) Distributed API Processor Processor Processor Processor Partition 1 Partition 2 Partition 3 Partition ...n Copyright © InfiniteGraph

Which of These Work ? • A carefully orchestrated combination of various options  • Can be tuned (degree of look ahead) • Healing graph can be expensive (write cost) • This can also be tuned/configured (external edge thresholds) Copyright © InfiniteGraph

http://blog.infinitegraph.com	243
http://www.nosqldatabases.com	42

Gluecon InfiniteGraph Presentation: Scaling the Social Graph in the Cloud

by InfiniteGraph on Jun 23, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 285

Statistics

Gluecon InfiniteGraph Presentation: Scaling the Social Graph in the Cloud — Presentation Transcript