Graphs, Edges & Nodes: Untangling the Social Web

Graphs, Edges &
Nodes
Untangling the social web.

Wednesday, March 9, 2011

What’s a graph?


Graph


Graph
10

19
9 7
2 15
7
3
12
13
9
6
6
4 3
5 7
4
14
1

4


Graph
11 10 10

19
6 9 7
2 15
7 21
3 8
12
15 13 13
17 9
22
6
6
3
4 4 3
2 5 7
4
6 14 9 12
1
10 4
19


Simple

At most one edge bet ween any pair of nodes.


Multigraph

Multiple edges bet ween vertices allowed.


Pseudograph

Self-loops are permitted.


G = (V, E)


What’s a node?
vertex
point
junction
0-simplex


What’s an edge?
arc
branch
line
link
1-simplex


Directed


Undirected


Data Structures


1

3

2

4

(Finite simple graph)


vertices

0 1 1 1

1 0 0 0
vertices
1 0 0 1

1 0 1 0

Adjacency Matrix
(2d array)


[1, 2, 3, 4]
2 1 1 1
3 4 3
4

Array entries (vertices) point to singly linked-lists


Visualizations


You are here.


(Graph does not include Justin Bieber)


Social Graphs


User-based item recommendations


People

Recommend items to me that are popular amongst my friends


People
(friends)



People
(friends)

Items



People
(me) (friends)

Items



2-step path on homogeneous bipartite
graph.


Strong Connection Problem (SCP)


There are many of these ‘fundamental’
graph units:

- tripartite graphs (user/asset/tag)
- folksonomies
- multicolor-multiparity graph
- etc.


Graph Storage
Engines


Neo4j
“An embedded, disk-based, fully transactional Java persistence engine that
stores data structured in graphs rather than in tables.”

http://neo4j.org


HypergraphDB
“A general purpose, extensible, portable, distributed, embeddable, open-source
data storage mechanism. It is a graph database designed specifically for
artificial intelligence and semantic web projects.”

http://kobrix.org/hgdb.jsp


Special Purpose
Storage Engines


FlockDB
“FlockDB is a database that stores graph data, but it isn't a database
optimized for graph-traversal operations. Instead, it's optimized for very
large adjacency lists, fast reads and writes, and page-able set arithmetic
queries.”

http://engineering.twitter.com/2010/05/introducing-flockdb.html


Redis
“Redis is an advanced key-value store. [...] the dataset is not volatile, and values
can be strings, exactly like in memcached, but also lists, sets, and ordered sets. All
this data types can be manipulated with atomic operations to push/pop elements,
add/remove elements, perform server side union, intersection, difference between
sets, etc.”

http://code.google.com/p/redis


A Redis Friends/
Followers Example


Redis makes you think in terms of datastructures,
and operations on those structures.


Set:
Finite (for our cases) collection of objects in which
order has no significance and multiplicity is generally
ignored.
S = { Alice, Bob, Carol }

List:
Finite (for our cases) collection of objects in which
order *is* significant and multiplicity is allowed.

L = [ X, Y, X, Z, Q]


Insert a user into a set

SET uid:1000:username jperras

Command Key Value


Use sets for denoting my followers/people
I follow.


Adding a new follower

SADD uid:1000:following 1001
SADD uid:1001:followers 1000

Command Key Value


Posting Updates

$r = Redis();
$postid = $r->incr("global:nextPostId");
$post = $User['id'] ."|". time() ."|". $status;
$r->set("post:$postid", $post);
$followers = $r->smembers("uid:".$User['id'].":followers");

if ($followers === false) $followers = Array();
$followers[] = $User['id']; /* Add the post to our own posts too */

foreach($followers as $fid) {
$r->push("uid:$fid:posts", $postid, false);
}
# Push the post on the timeline, and trim the timeline to the
# newest 1000 elements.
$r->push("global:timeline", $postid, false);
$r->ltrim("global:timeline",0,1000);


Common followers? - Set intersections!

SINTER users:1000:followers users:1000:followers

Command Key 1 Key 2


A MySQL Example
(simplified)


# Mutual Friends
select f.friend_id
from friends f
join friends m
on m.user_id = f.friend_id
and m.friend_id = f.user_id
where f.user_id = 1234

# Following (for directed graphs)
select f.friend_id
from friends f
left join friends m
and m.user_id is null;

# Followers (for directed graphs)
select m.friend_id
from friends f
left join friends m
where f.friend_id = 1234
and m.user_id is null


# Mutual Friends
select f.friend_id
from friends f
join friends m

# Following (for directed graphs)
select f.friend_id
from friends f
left join friends m

Not too bad. on m.user_id = f.friend_id
and m.user_id is null;

# Followers (for directed graphs)
select m.friend_id
from friends f
left join friends m
where f.friend_id = 1234
and m.user_id is null


Relational databases can work for the simplest
of cases, but are not always the best solution for
many graph operations/algorithms.


Graphs and graph-databases are only
going to be more and more useful.


However, graph algorithms are hard.

So don’t write your own.

And make sure you use a persistent storage engine
that is best suited for the type of queries
you will be performing.


Resources


Resources
The Algorithm Design Manual,
Steve S. Skiena
Programming Collective
Intelligence, Toby Segaran
Introduction to Algorithms,
Cormen, Leiserson, Rivest


@jperras


Photo Credits

Graph of the internet, circa 2003: http://www.duniacyber.com/freebies/education/what-
is-internet-lookslike/ (built from partial troll of public servers using traceroute)

My real friends for letting me use their Facebook profile images.


References

Large Scale Graph Algorithms (class lectures), Yuri Lifshits, Steklov Institute of
Mathematics at St. Petersburg

http://mathworld.wolfram.com/Set.html

Programming Collective Intelligence, Toby Segaran

The Algorithm Design Manual, Steve S. Skiena


Graphs, Edges & Nodes: Untangling the Social Web

More Related Content

Viewers also liked

Similar to Graphs, Edges & Nodes: Untangling the Social Web

More from ConFoo

Recently uploaded

Graphs, Edges & Nodes: Untangling the Social Web