SlideShare a Scribd company logo
1 of 74
Download to read offline
Graph Algebra
Graph operations in the language of linear algebra
1
Graph representation
1 2
3
2
Graph representation
Graph on top of:

1. tables (JanusGraph as on disk storage)
2. documents (ArangoDB)
Formal graph structure:
1. adjacency list (Neo4J, JanusGraph)
2. adjacency matrix (RedisGraph)
3
Adjacency matrix
0 1 1
0 0 1
0 0 0
A[i,j] = 1 if entity i is connected to j
0 otherwise.
4
Binary matrix
• 1 bit per cell
• Matrix addition binary OR

• Matrix multiplication binary AND
5
Binary matrix
1 bit per matrix cell
1,000,000 X 1,000,000
One trillion bits = 125GB
……………………………………………………….
……………………………………………………….
……………………………………………………….
……………………………………………………….
……………………………………………………….
……………………………………………………….
……………………………………………………….
……………………………………………………….
……………………………………………………….
……………………………………………………….
……………………………………………………….
……………………………………………………….
……………………………………………………….
……………………………………………………….
6
Real world graphs
Most real world graphs are sparse
Facebook’s friendship graph
2 billion users
338 friends for user on average
2,000,000,000 * 338 / 2,000,000,000^2
0.000000169% utilisation
7
Sparse matrix
• Tracks nonzeros

• Assume zero for untracked entries
8
GraphBLAS
• Standard building blocks for graph algorithms in the
language of linear algebra

• Sparse Matrix-Matrix multiply

• Sparse Vector-Matrix multiply
9
SuiteSparse:GraphBLAS
Graph algorithms via sparse linear algebra over semirings
via	traditional	Breadth-First-Search:	
				for	each	i	in	current	level	
										for	each	edge	(i,j)	
														if	j	is	new	
																	add	j	to	next	level	...
Find	next	BFS	level:	just	one	masked	matrix-vector	multiply
Tim Davis, Texas A&M University
via	semiring:	
			y<mask>=A*x
SuiteSparse:GraphBLAS
• traversing	nodes	and	edges	one	a	time:	no	scope	for	library	optimization	
• linear	algebra:	“bulk”	work	can	be	given	to	a	library	
• let	the	experts	write	the	library	kernels:	fast,	robust,	portable	performance	
• composable	linear	algebra:	associative,	distributive,	(AB)T=BTAT,	...
Tim Davis, Texas A&M University
Why	GraphBLAS?
Outline
Graph algorithms in the language of linear algebra
Consider C=A*B on a semiring
Semiring: add and multiply operators, and additive identity
Example: with OR-AND semiring: A and B are adjacency matrices of two graphs
C=A*B: contains edge (i, j) if nodes i and j share any neighbor in common
Shortest paths via MIN-PLUS semiring
Graph object is opaque; can exploit lazy evaluation
The GraphBLAS Spec: graphblas.org
SuiteSparse:GraphBLAS implementation and performance
Why graph algorithms with linear algebra?
powerful way of expressing graph algorithms with large, “bulk” operations on
adjaceny matrices. No need to chase nodes and edges.
linear algebra with semirings: composable operations, like (AB)C = A(BC)
lower software complexity: let the experts write the core graph kernels
simple object for complex problems: a sparse matrix with any data type, including
user-defined
security: encrypt/decrypt via linear algebra and binary operators
mathematically well-defined graph object, closed under operations
performance: serial, parallel, GPU, ... let the library optimize large “bulk”
graph/matrix operators
Breadth-first search example
A(i, j) = 1 for edge (j, i)
A is binary; dot (.) is zero for clarity.
. . . 1 . . .
1 . . . . . .
. . . 1 . 1 1
1 . . . . . 1
. 1 . . . . 1
. . 1 . 1 . .
. 1 . . . . .
Breadth-first search: initializations
v = zeros (n,1) ; // result
q = false (n,1) ; // current level
q (source) = true ;
v: q:
. .
. .
. .
. 1
. .
. .
. .
GrB assign (v, q, NULL, level, GrB ALL, n, NULL)
v <q> = level ; // assign level
v: q:
. .
. .
. .
1 1
. .
. .
. .
GrB mxv (q, v, NULL, GxB LOR LAND BOOL, A, q, desc)
first part of q<!v>=A*q:
t = A*q ;
GrB mxv (q, v, NULL, GxB LOR LAND BOOL, A, q, desc)
second part of q<!v>=A*q:
q = false (n,1) ;
q <!v> = t ;
v: t=A*q: q<!v>=t
. 1 1
. . .
. 1 1
1 . .
. . .
. . .
. . .
GrB assign (v, q, NULL, level, GrB ALL, n, NULL)
v <q> = level ; // assign level
v: q:
2 1
. .
2 1
1 .
. .
. .
. .
GrB mxv (q, v, NULL, GxB LOR LAND BOOL, A, q, desc)
first part of q<!v>=A*q:
t = A*q ;
GrB mxv (q, v, NULL, GxB LOR LAND BOOL, A, q, desc)
second part of q<!v>=A*q:
q = false (n,1) ;
q <!v> = t ;
v: t=A*q: q<!v>=t
2 . .
. 1 1
2 . .
1 1 .
. . .
. 1 1
. . .
GrB assign (v, q, NULL, level, GrB ALL, n, NULL)
v <q> = level ; // assign level
v: q:
2 .
3 1
2 .
1 .
. .
3 1
. .
GrB mxv (q, v, NULL, GxB LOR LAND BOOL, A, q, desc)
first part of q<!v>=A*q:
t = A*q ;
GrB mxv (q, v, NULL, GxB LOR LAND BOOL, A, q, desc)
second part of q<!v>=A*q:
q = false (n,1) ;
q <!v> = t ;
v: t=A*q: q<!v>=t
2 . .
3 . .
2 1 .
1 . .
. 1 1
3 . .
. 1 1
GrB assign (v, q, NULL, level, GrB ALL, n, NULL)
v <q> = level ; // assign level
v: q:
2 .
3 .
2 .
1 .
4 1
3 .
4 1
GrB mxv (q, v, NULL, GxB LOR LAND BOOL, A, q, desc)
first part of q<!v>=A*q:
t = A*q ;
GrB mxv (q, v, NULL, GxB LOR LAND BOOL, A, q, desc)
second part of q<!v>=A*q:
q = false (n,1) ;
q <!v> = t ;
v: t=A*q: q<!v>=t
2 . .
3 . .
2 1 .
1 1 .
4 1 .
3 1 .
4 . .
GraphBLAS operations: overview
operation MATLAB GraphBLAS
analog extras
matrix multiplication C=A*B 960 built-in semirings
element-wise, set union C=A+B any operator
element-wise, set intersection C=A.*B any operator
reduction to vector or scalar s=sum(A) any operator
apply unary operator C=-A C=f(A)
transpose C=A’
submatrix extraction C=A(I,J)
submatrix assignment C(I,J)=A zombies and pending tuples
C=A*B with 960 built-in semirings, and each matrix one of 11 types: GraphBLAS has
960 ⇥ 113 = 1, 277, 760 built-in versions of matrix multiply. MATLAB has 4. Arbitrary
user-defined types, operators, monoids, and semirings can be created at run time.
GraphBLAS objects
GrB_Type 11 built-in types, “any” user-defined type
GrB_UnaryOp unary operator such as z = x
GrB_BinaryOp binary operator such as z = x + y
GrB_Monoid associative operator like z = x + y with identity 0
GrB_Semiring a multiply operator and additive monoid
GrB_Vector like an n-by-1 matrix
GrB_Matrix a sparse m-by-n matrix
GrB_Descriptor parameter settings
all objects opaque; allows for internal optimization
matrices in compressed-sparse column (CSC) form, with sorted indices
non-blocking mode; matrix can have pending operations
all operations can take an optional mask: like a bulk if statement, ChMi = ...
and an optional accumulator operator: C = C ...
GraphBLAS operations
GrB_mxm matrix-matrix multiply ChMi = C AB
GrB_vxm vector-matrix multiply w0
hm0
i = w0
u0
A
GrB_mxv matrix-vector multiply whmi = w Au
GrB_eWiseMult element-wise, ChMi = C (A ⌦ B)
set union whmi = w (u ⌦ v)
GrB_eWiseAdd element-wise, ChMi = C (A B)
set intersection whmi = w (u v)
GrB_extract extract submatrix ChMi = C A(i, j)
whmi = w u(i)
GrB_assign assign submatrix C(i, j)hMi = C(i, j) A
w(i)hmi = w(i) u
GrB_apply apply unary operator ChMi = C f (A)
whmi = w f (u)
GrB_reduce reduce to vector whmi = w [ j A(:, j)]
reduce to scalar s = s [ ij A(i, j)]
GrB_transpose transpose ChMi = C A0
Operations: C(I,J)=A, submatrix/subgraph assignment
hardest function to implement
modifies C in place
costly to modify the matrix/graph, so operations are left pending
zombies: edges/entries still in graph/matrix but marked for deletion
pending tuples: unsorted list of edges/entries to be added to graph/matrix
Building a graph: all at once
Creating a matrix from list of tuples: fast in GraphBLAS:
for (int k = 0 ; k < nz ; k++)
{
I [k] = simple_rand_i ( ) % nrows ;
J [k] = simple_rand_i ( ) % ncols ;
X [k] = simple_rand_x ( ) ;
}
GrB_Matrix A ;
GrB_Matrix_new (&A, GrB_FP64, nrows, ncols) ;
GrB_Matrix_build (A, I, J, X, nz, GrB_SECOND_FP64) ;
Just as fast in MATLAB:
for k = 1:nz
I (k) = randi (nrows) ;
J (k) = randi (ncols) ;
X (k) = rand ( ) ;
end
A = sparse (I,J,X, nrows,ncols) ;
Building a graph: incremental
One element at a time: fast in GraphBLAS:
GrB_Matrix A ;
GrB_Matrix_new (&A, GrB_FP64, nrows, ncols) ;
for (int k = 0 ; k < nz ; k++)
{
GrB_Index i = simple_rand_i ( ) % nrows ;
GrB_Index j = simple_rand_i ( ) % ncols ;
double x = simple_rand_x ( ) ;
// A (i,j) = x
GrB_Matrix_setElement (A, x, i, j) ;
}
Impossibly slow in MATLAB:
A = sparse (nrows,ncols) ; % an empty sparse matrix
for k = 1:nz
i = randi (nrows) ;
j = randi (ncols) ;
A (i,j) = rand ( ) ;
end
GraphBLAS performance: C(I,J)=A
Submatrix assignment
Example: C is the Freescale2 matrix, 3 million by 3 million with 14.3 million
nonzeros
I = randperm (n,5500)
J = randperm (n,7000)
A = random sparse matrix with 38,500 nonzeros
C(I,J) = A
87 seconds in MATLAB
0.74 seconds in GraphBLAS, without exploiting blocking mode, via GrB_assign
Summary
GraphBLAS: graph algorithms in the language of linear algebra
“Sparse-anything” matrices, including user-defined types
matrix multiplication with any semiring
operations: C=A*B, C=A+B, reduction, transpose, accumulator/mask, submatrix
extraction and assigment
performance: most operations just as fast as MATLAB, submatrix assignment
100x or faster.
Version 2.0.1 available at suitesparse.com, Debian, Ubuntu, Mac HomeBrew, ...
RedisGraph
37
Friend of friend
MATCH (src)-[:friend]->(f)-[:friend]-(fof)
WHERE src.age > 30
RETURN fof
src f fof
friend friend
38
Execution plan
MATCH

(src)-[:friend]->(f)-[:friend]->(fof)
WHERE src.age > 30
RETURN fof
Index scan
Expand
Expand
Project
src.age > 30
(src)-[:friend]->(f)
(f)-[:friend]->(fof)
RETURN fof
39
Execution plan
Index scan
Expand
Expand
Project
Entity ID 5
40
src.age > 30
(src)-[:friend]->(f)
(f)-[:friend]->(fof)
RETURN fof
Execution plan
Index scan
Expand
Expand
Project
5 connected to 2
41
src.age > 30
(src)-[:friend]->(f)
(f)-[:friend]->(fof)
RETURN fof
Execution plan
Index scan
Expand
Expand
Project
2 connected to 9
42
src.age > 30
(src)-[:friend]->(f)
(f)-[:friend]->(fof)
RETURN fof
Execution plan
Index scan
Expand
Expand
ProjectProject 9
43
src.age > 30
(src)-[:friend]->(f)
(f)-[:friend]->(fof)
RETURN fof
Execution plan
Index scan
Expand
Expand
Project
2 connected to 1
44
src.age > 30
(src)-[:friend]->(f)
(f)-[:friend]->(fof)
RETURN fof
Execution plan
Index scan
Expand
Expand
ProjectProject 1
45
src.age > 30
(src)-[:friend]->(f)
(f)-[:friend]->(fof)
RETURN fof
Execution plan
Index scan
Expand
Expand
Project
2 depleted
46
src.age > 30
(src)-[:friend]->(f)
(f)-[:friend]->(fof)
RETURN fof
Execution plan
Index scan
Expand
Expand
Project
5 depleted
47
src.age > 30
(src)-[:friend]->(f)
(f)-[:friend]->(fof)
RETURN fof
Execution plan
Index scan
Expand
Expand
Project
Entity ID 8
48
src.age > 30
(src)-[:friend]->(f)
(f)-[:friend]->(fof)
RETURN fof
Execution plan
• Serial

• Random memory access 

• Discovers one entity at a time
49
RedisGraph &
GraphBLAS
50
OpenCypher
to

linear algebra expression
51
MATCH

(src)-[:friend]->(f)-[:friend]->(fof)
WHERE src.age > 30
RETURN fof
=
Age_Filter * Friendship * Friendship
52
1
5
4
2
63
53
0 0 0 0 0 0
0 1 0 0 0 0
0 0 1 0 0 0
0 0 0 1 0 0
0 0 0 0 0 0
0 0 0 0 0 1
Age Filter
0 1 0 0 1 0
0 0 1 0 0 0
1 0 0 0 1 1
0 0 0 0 1 0
1 0 1 0 0 0
0 1 0 1 0 0
Friendships
0 1 0 0 1 0
0 0 1 0 0 0
1 0 0 0 1 1
0 0 0 0 1 0
1 0 1 0 0 0
0 1 0 1 0 0
Friendships
* *
54
Matrix multiplication
is associative
(A*B)*C = A*(B*C)
55
0 1 0 0 1 0
0 0 1 0 0 0
1 0 0 0 1 1
0 0 0 0 1 0
1 0 1 0 0 0
0 1 0 1 0 0
Friendships
0 1 0 0 1 0
0 0 1 0 0 0
1 0 0 0 1 1
0 0 0 0 1 0
1 0 1 0 0 0
0 1 0 1 0 0
Friendships
*
1 0 1 0 0 0
1 0 0 0 1 1
1 1 1 1 1 0
1 0 1 0 0 0
1 1 0 0 1 1
0 0 1 0 1 0
Friendships ^2
=
NNZ = 18
56
Age Filter
0 1 0 0 1 0
0 0 1 0 0 0
1 0 0 0 1 1
0 0 0 0 1 0
1 0 1 0 0 0
0 1 0 1 0 0
Friendships
0 0 0 0 0 0
0 0 1 0 0 0
1 0 0 0 1 1
0 0 0 0 1 0
0 0 0 0 0 0
0 1 0 1 0 0
Filtered friendships

src > 30
* =
NNZ = 7
0 0 0 0 0 0
0 1 0 0 0 0
0 0 1 0 0 0
0 0 0 1 0 0
0 0 0 0 0 0
0 0 0 0 0 1
57
0 1 0 0 1 0
0 0 1 0 0 0
1 0 0 0 1 1
0 0 0 0 1 0
1 0 1 0 0 0
0 1 0 1 0 0
Friendships
0 0 0 0 0 0
0 0 1 0 0 0
1 0 0 0 1 1
0 0 0 0 1 0
0 0 0 0 0 0
0 1 0 1 0 0
Filtered friendships

src > 30
* =
0 0 0 0 0 0
1 0 0 0 1 1
1 1 1 1 1 0
1 0 1 0 0 0
0 0 0 0 0 0
0 0 1 0 1 0
FOF
58
0 0 0 0 0 0
1 0 0 0 1 1
1 1 1 1 1 0
1 0 1 0 0 0
0 0 0 0 0 0
0 0 1 0 1 0
1
5
4
2
63
59
0 0 0 0 0 0
1 0 0 0 1 1
1 1 1 1 1 0
1 0 1 0 0 0
0 0 0 0 0 0
0 0 1 0 1 0
1
5
4
2
63
60
Friend of friend
variable length
MATCH (src)-[:friend*2..4]->(fof)
WHERE src.age > 30
RETURN fof
src F2 fof
friend
F3 F4
61
MATCH (src)-[:friend*2..4]->(fof)
WHERE src.age > 30
RETURN fof
=
Age_Filter * (Friendship^2 + Friendship^3 + Friendship^4)


=
M = AF;

R = 0;

For i=0; i < 3; i++

M = M*F

R = R+M
62
0 0 0 0 0 0
0 1 0 0 0 0
0 0 1 0 0 0
0 0 0 1 0 0
0 0 0 0 0 0
0 0 0 0 0 1
Age filter
1 1 1 0 1 1
1 1 1 1 1 1
1 1 1 1 1 1
1 1 1 0 1 1
1 1 1 1 1 1
1 0 1 0 1 1
Friendships^2 + Friendships^3
1 1 1 0 1 1
0 0 0 0 0 0
0 0 0 0 0 0
1 1 1 0 1 1
0 0 0 0 0 0
1 0 1 0 1 1
Friendships
* =
63
1
5
4
2
63
1 1 1 0 1 1
0 0 0 0 0 0
0 0 0 0 0 0
1 1 1 0 1 1
0 0 0 0 0 0
1 0 1 0 1 1
64
Additional algorithms
• Connected Components

• Shortest paths

• Minimum spanning tree
65
Graph distribution
Block multiplication
A*B=C
A B C
A1
A3 A4
A2 B1 B2
B4B3
C1 C2
C3 C4
66
Graph distribution
Block multiplication
A*B=C
A B C
A1
A3 A4
A2 B1 B2
B4B3
A1*B1+

A2*B3
A1*B2+

A2*B4
A3*B1+

A4*B3
A3*B2+

A4*B4
67
Parallelize
• CuSPARSE - GPU

• OpenMP - CPU
68
Benchmarks
69
Benchmarking graph databases on the problem of community detection paper

Reports a comprehensive comparative evaluation

between three popular graph databases, Titan, OrientDB and Neo4j.

For evaluation they’ve used real data derived from the SNAP dataset collection.

All experiments were run on an Intel Core i7 at 3.5Ghz with 16GB of main memory

and a 1.4 TB hard disk, the OS being Ubuntu Linux 12.04 (64bit).

We’ve performed the same benchmarks against RedisGraph, using inferior hardware.
Benchmarks
70
Massive Insertion Workload (MIW)
Create the graph database and configure it for massive loading.

Populate it with a particular dataset.

Measure the time for the creation of the whole graph.
All the measurements are in seconds

Dataset contains 1134890 nodes and 2987624 edges
RedisGraph
Titan
OrientDB
Neo4j
0 75 150 225 300
24.69
252.15
104.27
0.53
Benchmarks
71
Query Workload FindNeighbours (FN)

finds the neighbours of all nodes
All the measurements are in seconds

Dataset contains 1134890 nodes and 2987624 edges
RedisGraph
Titan
OrientDB
Neo4j
0 7.5 15 22.5 30
4.51
9.34
20.71
0.05
Benchmarks
72
Query Workload FindAdjacentNodes (FA)

finds the adjacent nodes of all edges.
All the measurements are in seconds

Dataset contains 1134890 nodes and 2987624 edges
RedisGraph
Titan
OrientDB
Neo4j
0 12.5 25 37.5 50
1.46
6.15
42.82
0.05
Benchmarks
73
Query Workload FindShortestPath (FS)

Finds the shortest path between the first node and 100 randomly picked nodes.
All the measurements are in seconds

Dataset contains 1134890 nodes and 2987624 edges
RedisGraph
Titan
OrientDB
Neo4j
0 7.5 15 22.5 30
0.08
23.47
24.87
0.001
Thank You
@roilipman

davis@tamu.edu
74

More Related Content

What's hot

Introduction to Mongodb execution plan and optimizer
Introduction to Mongodb execution plan and optimizerIntroduction to Mongodb execution plan and optimizer
Introduction to Mongodb execution plan and optimizerMydbops
 
Introducing BinarySortedMultiMap - A new Flink state primitive to boost your ...
Introducing BinarySortedMultiMap - A new Flink state primitive to boost your ...Introducing BinarySortedMultiMap - A new Flink state primitive to boost your ...
Introducing BinarySortedMultiMap - A new Flink state primitive to boost your ...Flink Forward
 
Apache Calcite: A Foundational Framework for Optimized Query Processing Over ...
Apache Calcite: A Foundational Framework for Optimized Query Processing Over ...Apache Calcite: A Foundational Framework for Optimized Query Processing Over ...
Apache Calcite: A Foundational Framework for Optimized Query Processing Over ...Julian Hyde
 
Introduction to Apache Calcite
Introduction to Apache CalciteIntroduction to Apache Calcite
Introduction to Apache CalciteJordan Halterman
 
Combining a Knowledge Graph and Graph Algorithms to Find Hidden Skills at NASA
Combining a Knowledge Graph and Graph Algorithms to Find Hidden Skills at NASACombining a Knowledge Graph and Graph Algorithms to Find Hidden Skills at NASA
Combining a Knowledge Graph and Graph Algorithms to Find Hidden Skills at NASANeo4j
 
MongoDB .local Toronto 2019: Tips and Tricks for Effective Indexing
MongoDB .local Toronto 2019: Tips and Tricks for Effective IndexingMongoDB .local Toronto 2019: Tips and Tricks for Effective Indexing
MongoDB .local Toronto 2019: Tips and Tricks for Effective IndexingMongoDB
 
Deep Dive into Stateful Stream Processing in Structured Streaming with Tathag...
Deep Dive into Stateful Stream Processing in Structured Streaming with Tathag...Deep Dive into Stateful Stream Processing in Structured Streaming with Tathag...
Deep Dive into Stateful Stream Processing in Structured Streaming with Tathag...Databricks
 
Achieve Blazing-Fast Ingest Speeds with Apache Arrow
Achieve Blazing-Fast Ingest Speeds with Apache ArrowAchieve Blazing-Fast Ingest Speeds with Apache Arrow
Achieve Blazing-Fast Ingest Speeds with Apache ArrowNeo4j
 
Apache Calcite Tutorial - BOSS 21
Apache Calcite Tutorial - BOSS 21Apache Calcite Tutorial - BOSS 21
Apache Calcite Tutorial - BOSS 21Stamatis Zampetakis
 
Inside MongoDB: the Internals of an Open-Source Database
Inside MongoDB: the Internals of an Open-Source DatabaseInside MongoDB: the Internals of an Open-Source Database
Inside MongoDB: the Internals of an Open-Source DatabaseMike Dirolf
 
Connecting and using PostgreSQL database with psycopg2 [Python 2.7]
Connecting and using PostgreSQL database with psycopg2 [Python 2.7]Connecting and using PostgreSQL database with psycopg2 [Python 2.7]
Connecting and using PostgreSQL database with psycopg2 [Python 2.7]Dinesh Neupane
 
Understanding and tuning WiredTiger, the new high performance database engine...
Understanding and tuning WiredTiger, the new high performance database engine...Understanding and tuning WiredTiger, the new high performance database engine...
Understanding and tuning WiredTiger, the new high performance database engine...Ontico
 
GraphFrames: Graph Queries In Spark SQL
GraphFrames: Graph Queries In Spark SQLGraphFrames: Graph Queries In Spark SQL
GraphFrames: Graph Queries In Spark SQLSpark Summit
 
Using Delta Lake to Transform a Legacy Apache Spark to Support Complex Update...
Using Delta Lake to Transform a Legacy Apache Spark to Support Complex Update...Using Delta Lake to Transform a Legacy Apache Spark to Support Complex Update...
Using Delta Lake to Transform a Legacy Apache Spark to Support Complex Update...Databricks
 
PostgreSQL High_Performance_Cheatsheet
PostgreSQL High_Performance_CheatsheetPostgreSQL High_Performance_Cheatsheet
PostgreSQL High_Performance_CheatsheetLucian Oprea
 

What's hot (20)

Introduction to Mongodb execution plan and optimizer
Introduction to Mongodb execution plan and optimizerIntroduction to Mongodb execution plan and optimizer
Introduction to Mongodb execution plan and optimizer
 
Presto overview
Presto overviewPresto overview
Presto overview
 
Introducing BinarySortedMultiMap - A new Flink state primitive to boost your ...
Introducing BinarySortedMultiMap - A new Flink state primitive to boost your ...Introducing BinarySortedMultiMap - A new Flink state primitive to boost your ...
Introducing BinarySortedMultiMap - A new Flink state primitive to boost your ...
 
Sql query patterns, optimized
Sql query patterns, optimizedSql query patterns, optimized
Sql query patterns, optimized
 
Apache Calcite: A Foundational Framework for Optimized Query Processing Over ...
Apache Calcite: A Foundational Framework for Optimized Query Processing Over ...Apache Calcite: A Foundational Framework for Optimized Query Processing Over ...
Apache Calcite: A Foundational Framework for Optimized Query Processing Over ...
 
Introduction to Apache Calcite
Introduction to Apache CalciteIntroduction to Apache Calcite
Introduction to Apache Calcite
 
Combining a Knowledge Graph and Graph Algorithms to Find Hidden Skills at NASA
Combining a Knowledge Graph and Graph Algorithms to Find Hidden Skills at NASACombining a Knowledge Graph and Graph Algorithms to Find Hidden Skills at NASA
Combining a Knowledge Graph and Graph Algorithms to Find Hidden Skills at NASA
 
PostgreSQL
PostgreSQLPostgreSQL
PostgreSQL
 
MongoDB .local Toronto 2019: Tips and Tricks for Effective Indexing
MongoDB .local Toronto 2019: Tips and Tricks for Effective IndexingMongoDB .local Toronto 2019: Tips and Tricks for Effective Indexing
MongoDB .local Toronto 2019: Tips and Tricks for Effective Indexing
 
Deep Dive into Stateful Stream Processing in Structured Streaming with Tathag...
Deep Dive into Stateful Stream Processing in Structured Streaming with Tathag...Deep Dive into Stateful Stream Processing in Structured Streaming with Tathag...
Deep Dive into Stateful Stream Processing in Structured Streaming with Tathag...
 
Achieve Blazing-Fast Ingest Speeds with Apache Arrow
Achieve Blazing-Fast Ingest Speeds with Apache ArrowAchieve Blazing-Fast Ingest Speeds with Apache Arrow
Achieve Blazing-Fast Ingest Speeds with Apache Arrow
 
Apache Calcite Tutorial - BOSS 21
Apache Calcite Tutorial - BOSS 21Apache Calcite Tutorial - BOSS 21
Apache Calcite Tutorial - BOSS 21
 
Inside MongoDB: the Internals of an Open-Source Database
Inside MongoDB: the Internals of an Open-Source DatabaseInside MongoDB: the Internals of an Open-Source Database
Inside MongoDB: the Internals of an Open-Source Database
 
Connecting and using PostgreSQL database with psycopg2 [Python 2.7]
Connecting and using PostgreSQL database with psycopg2 [Python 2.7]Connecting and using PostgreSQL database with psycopg2 [Python 2.7]
Connecting and using PostgreSQL database with psycopg2 [Python 2.7]
 
Deep Dive on Amazon DynamoDB
Deep Dive on Amazon DynamoDBDeep Dive on Amazon DynamoDB
Deep Dive on Amazon DynamoDB
 
Understanding and tuning WiredTiger, the new high performance database engine...
Understanding and tuning WiredTiger, the new high performance database engine...Understanding and tuning WiredTiger, the new high performance database engine...
Understanding and tuning WiredTiger, the new high performance database engine...
 
GraphFrames: Graph Queries In Spark SQL
GraphFrames: Graph Queries In Spark SQLGraphFrames: Graph Queries In Spark SQL
GraphFrames: Graph Queries In Spark SQL
 
Recursive Query Throwdown
Recursive Query ThrowdownRecursive Query Throwdown
Recursive Query Throwdown
 
Using Delta Lake to Transform a Legacy Apache Spark to Support Complex Update...
Using Delta Lake to Transform a Legacy Apache Spark to Support Complex Update...Using Delta Lake to Transform a Legacy Apache Spark to Support Complex Update...
Using Delta Lake to Transform a Legacy Apache Spark to Support Complex Update...
 
PostgreSQL High_Performance_Cheatsheet
PostgreSQL High_Performance_CheatsheetPostgreSQL High_Performance_Cheatsheet
PostgreSQL High_Performance_Cheatsheet
 

Similar to RedisConf18 - Lower Latency Graph Queries in Cypher with Redis Graph

Intro matlab and convolution islam
Intro matlab and convolution islamIntro matlab and convolution islam
Intro matlab and convolution islamIslam Alabbasy
 
CSCI 2033 Elementary Computational Linear Algebra(Spring 20.docx
CSCI 2033 Elementary Computational Linear Algebra(Spring 20.docxCSCI 2033 Elementary Computational Linear Algebra(Spring 20.docx
CSCI 2033 Elementary Computational Linear Algebra(Spring 20.docxmydrynan
 
Tall-and-skinny Matrix Computations in MapReduce (ICME MR 2013)
Tall-and-skinny Matrix Computations in MapReduce (ICME MR 2013)Tall-and-skinny Matrix Computations in MapReduce (ICME MR 2013)
Tall-and-skinny Matrix Computations in MapReduce (ICME MR 2013)Austin Benson
 
Parallel Evaluation of Multi-Semi-Joins
Parallel Evaluation of Multi-Semi-JoinsParallel Evaluation of Multi-Semi-Joins
Parallel Evaluation of Multi-Semi-JoinsJonny Daenen
 
Fundamentals of Image Processing & Computer Vision with MATLAB
Fundamentals of Image Processing & Computer Vision with MATLABFundamentals of Image Processing & Computer Vision with MATLAB
Fundamentals of Image Processing & Computer Vision with MATLABAli Ghanbarzadeh
 
Multinomial Logistic Regression with Apache Spark
Multinomial Logistic Regression with Apache SparkMultinomial Logistic Regression with Apache Spark
Multinomial Logistic Regression with Apache SparkDB Tsai
 
Alpine Spark Implementation - Technical
Alpine Spark Implementation - TechnicalAlpine Spark Implementation - Technical
Alpine Spark Implementation - Technicalalpinedatalabs
 
CS 354 Transformation, Clipping, and Culling
CS 354 Transformation, Clipping, and CullingCS 354 Transformation, Clipping, and Culling
CS 354 Transformation, Clipping, and CullingMark Kilgard
 
Subtle Asynchrony by Jeff Hammond
Subtle Asynchrony by Jeff HammondSubtle Asynchrony by Jeff Hammond
Subtle Asynchrony by Jeff HammondPatrick Diehl
 
Problemas resueltos de funciones lineales ccesa007
Problemas resueltos de  funciones lineales ccesa007Problemas resueltos de  funciones lineales ccesa007
Problemas resueltos de funciones lineales ccesa007Demetrio Ccesa Rayme
 
MATLAB Questions and Answers.pdf
MATLAB Questions and Answers.pdfMATLAB Questions and Answers.pdf
MATLAB Questions and Answers.pdfahmed8651
 
mat lab introduction and basics to learn
mat lab introduction and basics to learnmat lab introduction and basics to learn
mat lab introduction and basics to learnpavan373
 
Tall-and-skinny Matrix Computations in MapReduce (ICME colloquium)
Tall-and-skinny Matrix Computations in MapReduce (ICME colloquium)Tall-and-skinny Matrix Computations in MapReduce (ICME colloquium)
Tall-and-skinny Matrix Computations in MapReduce (ICME colloquium)Austin Benson
 
CS 354 Pixel Updating
CS 354 Pixel UpdatingCS 354 Pixel Updating
CS 354 Pixel UpdatingMark Kilgard
 

Similar to RedisConf18 - Lower Latency Graph Queries in Cypher with Redis Graph (20)

Intro to matlab
Intro to matlabIntro to matlab
Intro to matlab
 
Intro matlab and convolution islam
Intro matlab and convolution islamIntro matlab and convolution islam
Intro matlab and convolution islam
 
CSCI 2033 Elementary Computational Linear Algebra(Spring 20.docx
CSCI 2033 Elementary Computational Linear Algebra(Spring 20.docxCSCI 2033 Elementary Computational Linear Algebra(Spring 20.docx
CSCI 2033 Elementary Computational Linear Algebra(Spring 20.docx
 
Tall-and-skinny Matrix Computations in MapReduce (ICME MR 2013)
Tall-and-skinny Matrix Computations in MapReduce (ICME MR 2013)Tall-and-skinny Matrix Computations in MapReduce (ICME MR 2013)
Tall-and-skinny Matrix Computations in MapReduce (ICME MR 2013)
 
Parallel Evaluation of Multi-Semi-Joins
Parallel Evaluation of Multi-Semi-JoinsParallel Evaluation of Multi-Semi-Joins
Parallel Evaluation of Multi-Semi-Joins
 
Fundamentals of Image Processing & Computer Vision with MATLAB
Fundamentals of Image Processing & Computer Vision with MATLABFundamentals of Image Processing & Computer Vision with MATLAB
Fundamentals of Image Processing & Computer Vision with MATLAB
 
Python grass
Python grassPython grass
Python grass
 
Multinomial Logistic Regression with Apache Spark
Multinomial Logistic Regression with Apache SparkMultinomial Logistic Regression with Apache Spark
Multinomial Logistic Regression with Apache Spark
 
Alpine Spark Implementation - Technical
Alpine Spark Implementation - TechnicalAlpine Spark Implementation - Technical
Alpine Spark Implementation - Technical
 
Matlab1
Matlab1Matlab1
Matlab1
 
CS 354 Transformation, Clipping, and Culling
CS 354 Transformation, Clipping, and CullingCS 354 Transformation, Clipping, and Culling
CS 354 Transformation, Clipping, and Culling
 
Subtle Asynchrony by Jeff Hammond
Subtle Asynchrony by Jeff HammondSubtle Asynchrony by Jeff Hammond
Subtle Asynchrony by Jeff Hammond
 
Revision1schema C programming
Revision1schema C programmingRevision1schema C programming
Revision1schema C programming
 
Problemas resueltos de funciones lineales ccesa007
Problemas resueltos de  funciones lineales ccesa007Problemas resueltos de  funciones lineales ccesa007
Problemas resueltos de funciones lineales ccesa007
 
Introduction to MATLAB
Introduction to MATLABIntroduction to MATLAB
Introduction to MATLAB
 
MATLAB Questions and Answers.pdf
MATLAB Questions and Answers.pdfMATLAB Questions and Answers.pdf
MATLAB Questions and Answers.pdf
 
mat lab introduction and basics to learn
mat lab introduction and basics to learnmat lab introduction and basics to learn
mat lab introduction and basics to learn
 
Tall-and-skinny Matrix Computations in MapReduce (ICME colloquium)
Tall-and-skinny Matrix Computations in MapReduce (ICME colloquium)Tall-and-skinny Matrix Computations in MapReduce (ICME colloquium)
Tall-and-skinny Matrix Computations in MapReduce (ICME colloquium)
 
CS 354 Pixel Updating
CS 354 Pixel UpdatingCS 354 Pixel Updating
CS 354 Pixel Updating
 
Dsp manual
Dsp manualDsp manual
Dsp manual
 

More from Redis Labs

Redis Day Bangalore 2020 - Session state caching with redis
Redis Day Bangalore 2020 - Session state caching with redisRedis Day Bangalore 2020 - Session state caching with redis
Redis Day Bangalore 2020 - Session state caching with redisRedis Labs
 
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020Redis Labs
 
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...Redis Labs
 
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020Redis Labs
 
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...Redis Labs
 
Redis for Data Science and Engineering by Dmitry Polyakovsky of Oracle
Redis for Data Science and Engineering by Dmitry Polyakovsky of OracleRedis for Data Science and Engineering by Dmitry Polyakovsky of Oracle
Redis for Data Science and Engineering by Dmitry Polyakovsky of OracleRedis Labs
 
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020Redis Labs
 
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Redis Labs
 
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...Redis Labs
 
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...Redis Labs
 
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...Highly Available Persistent Session Management Service by Mohamed Elmergawi o...
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...Redis Labs
 
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...Redis Labs
 
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Redis Labs
 
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs
 
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs
 
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs
 
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020Redis Labs
 
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...Redis Labs
 
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...Redis Labs
 
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...Redis Labs
 

More from Redis Labs (20)

Redis Day Bangalore 2020 - Session state caching with redis
Redis Day Bangalore 2020 - Session state caching with redisRedis Day Bangalore 2020 - Session state caching with redis
Redis Day Bangalore 2020 - Session state caching with redis
 
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020
 
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...
 
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020
 
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...
 
Redis for Data Science and Engineering by Dmitry Polyakovsky of Oracle
Redis for Data Science and Engineering by Dmitry Polyakovsky of OracleRedis for Data Science and Engineering by Dmitry Polyakovsky of Oracle
Redis for Data Science and Engineering by Dmitry Polyakovsky of Oracle
 
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020
 
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
 
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...
 
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...
 
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...Highly Available Persistent Session Management Service by Mohamed Elmergawi o...
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...
 
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...
 
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
 
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020
 
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020
 
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020
 
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020
 
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...
 
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...
 
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...
 

Recently uploaded

Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...BookNet Canada
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Kaya Weers
 
Efficiencies in RPA with UiPath and CyberArk Technologies - Session 2
Efficiencies in RPA with UiPath and CyberArk Technologies - Session 2Efficiencies in RPA with UiPath and CyberArk Technologies - Session 2
Efficiencies in RPA with UiPath and CyberArk Technologies - Session 2DianaGray10
 
Automation Ops Series: Session 3 - Solutions management
Automation Ops Series: Session 3 - Solutions managementAutomation Ops Series: Session 3 - Solutions management
Automation Ops Series: Session 3 - Solutions managementDianaGray10
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneUiPathCommunity
 
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...BookNet Canada
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesBernd Ruecker
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024TopCSSGallery
 
full stack practical assignment msc cs.pdf
full stack practical assignment msc cs.pdffull stack practical assignment msc cs.pdf
full stack practical assignment msc cs.pdfHulkTheDevil
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFMichael Gough
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
Arti Languages Pre Seed Pitchdeck 2024.pdf
Arti Languages Pre Seed Pitchdeck 2024.pdfArti Languages Pre Seed Pitchdeck 2024.pdf
Arti Languages Pre Seed Pitchdeck 2024.pdfwill854175
 
Transport in Open Pits______SM_MI10415MI
Transport in Open Pits______SM_MI10415MITransport in Open Pits______SM_MI10415MI
Transport in Open Pits______SM_MI10415MIRomil Mishra
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfROWELL MARQUINA
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
Women in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationWomen in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationDianaGray10
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Bitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactiveBitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactivestartupro
 

Recently uploaded (20)

Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)
 
Efficiencies in RPA with UiPath and CyberArk Technologies - Session 2
Efficiencies in RPA with UiPath and CyberArk Technologies - Session 2Efficiencies in RPA with UiPath and CyberArk Technologies - Session 2
Efficiencies in RPA with UiPath and CyberArk Technologies - Session 2
 
Automation Ops Series: Session 3 - Solutions management
Automation Ops Series: Session 3 - Solutions managementAutomation Ops Series: Session 3 - Solutions management
Automation Ops Series: Session 3 - Solutions management
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyone
 
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architectures
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024
 
full stack practical assignment msc cs.pdf
full stack practical assignment msc cs.pdffull stack practical assignment msc cs.pdf
full stack practical assignment msc cs.pdf
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
Arti Languages Pre Seed Pitchdeck 2024.pdf
Arti Languages Pre Seed Pitchdeck 2024.pdfArti Languages Pre Seed Pitchdeck 2024.pdf
Arti Languages Pre Seed Pitchdeck 2024.pdf
 
Transport in Open Pits______SM_MI10415MI
Transport in Open Pits______SM_MI10415MITransport in Open Pits______SM_MI10415MI
Transport in Open Pits______SM_MI10415MI
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdf
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
Women in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationWomen in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automation
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Bitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactiveBitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactive
 

RedisConf18 - Lower Latency Graph Queries in Cypher with Redis Graph

  • 1. Graph Algebra Graph operations in the language of linear algebra 1
  • 3. Graph representation Graph on top of:
 1. tables (JanusGraph as on disk storage) 2. documents (ArangoDB) Formal graph structure: 1. adjacency list (Neo4J, JanusGraph) 2. adjacency matrix (RedisGraph) 3
  • 4. Adjacency matrix 0 1 1 0 0 1 0 0 0 A[i,j] = 1 if entity i is connected to j 0 otherwise. 4
  • 5. Binary matrix • 1 bit per cell • Matrix addition binary OR
 • Matrix multiplication binary AND 5
  • 6. Binary matrix 1 bit per matrix cell 1,000,000 X 1,000,000 One trillion bits = 125GB ………………………………………………………. ………………………………………………………. ………………………………………………………. ………………………………………………………. ………………………………………………………. ………………………………………………………. ………………………………………………………. ………………………………………………………. ………………………………………………………. ………………………………………………………. ………………………………………………………. ………………………………………………………. ………………………………………………………. ………………………………………………………. 6
  • 7. Real world graphs Most real world graphs are sparse Facebook’s friendship graph 2 billion users 338 friends for user on average 2,000,000,000 * 338 / 2,000,000,000^2 0.000000169% utilisation 7
  • 8. Sparse matrix • Tracks nonzeros • Assume zero for untracked entries 8
  • 9. GraphBLAS • Standard building blocks for graph algorithms in the language of linear algebra • Sparse Matrix-Matrix multiply • Sparse Vector-Matrix multiply 9
  • 10. SuiteSparse:GraphBLAS Graph algorithms via sparse linear algebra over semirings via traditional Breadth-First-Search: for each i in current level for each edge (i,j) if j is new add j to next level ... Find next BFS level: just one masked matrix-vector multiply Tim Davis, Texas A&M University via semiring: y<mask>=A*x
  • 11. SuiteSparse:GraphBLAS • traversing nodes and edges one a time: no scope for library optimization • linear algebra: “bulk” work can be given to a library • let the experts write the library kernels: fast, robust, portable performance • composable linear algebra: associative, distributive, (AB)T=BTAT, ... Tim Davis, Texas A&M University Why GraphBLAS?
  • 12. Outline Graph algorithms in the language of linear algebra Consider C=A*B on a semiring Semiring: add and multiply operators, and additive identity Example: with OR-AND semiring: A and B are adjacency matrices of two graphs C=A*B: contains edge (i, j) if nodes i and j share any neighbor in common Shortest paths via MIN-PLUS semiring Graph object is opaque; can exploit lazy evaluation The GraphBLAS Spec: graphblas.org SuiteSparse:GraphBLAS implementation and performance
  • 13. Why graph algorithms with linear algebra? powerful way of expressing graph algorithms with large, “bulk” operations on adjaceny matrices. No need to chase nodes and edges. linear algebra with semirings: composable operations, like (AB)C = A(BC) lower software complexity: let the experts write the core graph kernels simple object for complex problems: a sparse matrix with any data type, including user-defined security: encrypt/decrypt via linear algebra and binary operators mathematically well-defined graph object, closed under operations performance: serial, parallel, GPU, ... let the library optimize large “bulk” graph/matrix operators
  • 14.
  • 15. Breadth-first search example A(i, j) = 1 for edge (j, i) A is binary; dot (.) is zero for clarity. . . . 1 . . . 1 . . . . . . . . . 1 . 1 1 1 . . . . . 1 . 1 . . . . 1 . . 1 . 1 . . . 1 . . . . .
  • 16. Breadth-first search: initializations v = zeros (n,1) ; // result q = false (n,1) ; // current level q (source) = true ; v: q: . . . . . . . 1 . . . . . .
  • 17. GrB assign (v, q, NULL, level, GrB ALL, n, NULL) v <q> = level ; // assign level v: q: . . . . . . 1 1 . . . . . .
  • 18. GrB mxv (q, v, NULL, GxB LOR LAND BOOL, A, q, desc) first part of q<!v>=A*q: t = A*q ;
  • 19. GrB mxv (q, v, NULL, GxB LOR LAND BOOL, A, q, desc) second part of q<!v>=A*q: q = false (n,1) ; q <!v> = t ; v: t=A*q: q<!v>=t . 1 1 . . . . 1 1 1 . . . . . . . . . . .
  • 20. GrB assign (v, q, NULL, level, GrB ALL, n, NULL) v <q> = level ; // assign level v: q: 2 1 . . 2 1 1 . . . . . . .
  • 21. GrB mxv (q, v, NULL, GxB LOR LAND BOOL, A, q, desc) first part of q<!v>=A*q: t = A*q ;
  • 22. GrB mxv (q, v, NULL, GxB LOR LAND BOOL, A, q, desc) second part of q<!v>=A*q: q = false (n,1) ; q <!v> = t ; v: t=A*q: q<!v>=t 2 . . . 1 1 2 . . 1 1 . . . . . 1 1 . . .
  • 23. GrB assign (v, q, NULL, level, GrB ALL, n, NULL) v <q> = level ; // assign level v: q: 2 . 3 1 2 . 1 . . . 3 1 . .
  • 24. GrB mxv (q, v, NULL, GxB LOR LAND BOOL, A, q, desc) first part of q<!v>=A*q: t = A*q ;
  • 25. GrB mxv (q, v, NULL, GxB LOR LAND BOOL, A, q, desc) second part of q<!v>=A*q: q = false (n,1) ; q <!v> = t ; v: t=A*q: q<!v>=t 2 . . 3 . . 2 1 . 1 . . . 1 1 3 . . . 1 1
  • 26. GrB assign (v, q, NULL, level, GrB ALL, n, NULL) v <q> = level ; // assign level v: q: 2 . 3 . 2 . 1 . 4 1 3 . 4 1
  • 27. GrB mxv (q, v, NULL, GxB LOR LAND BOOL, A, q, desc) first part of q<!v>=A*q: t = A*q ;
  • 28. GrB mxv (q, v, NULL, GxB LOR LAND BOOL, A, q, desc) second part of q<!v>=A*q: q = false (n,1) ; q <!v> = t ; v: t=A*q: q<!v>=t 2 . . 3 . . 2 1 . 1 1 . 4 1 . 3 1 . 4 . .
  • 29. GraphBLAS operations: overview operation MATLAB GraphBLAS analog extras matrix multiplication C=A*B 960 built-in semirings element-wise, set union C=A+B any operator element-wise, set intersection C=A.*B any operator reduction to vector or scalar s=sum(A) any operator apply unary operator C=-A C=f(A) transpose C=A’ submatrix extraction C=A(I,J) submatrix assignment C(I,J)=A zombies and pending tuples C=A*B with 960 built-in semirings, and each matrix one of 11 types: GraphBLAS has 960 ⇥ 113 = 1, 277, 760 built-in versions of matrix multiply. MATLAB has 4. Arbitrary user-defined types, operators, monoids, and semirings can be created at run time.
  • 30. GraphBLAS objects GrB_Type 11 built-in types, “any” user-defined type GrB_UnaryOp unary operator such as z = x GrB_BinaryOp binary operator such as z = x + y GrB_Monoid associative operator like z = x + y with identity 0 GrB_Semiring a multiply operator and additive monoid GrB_Vector like an n-by-1 matrix GrB_Matrix a sparse m-by-n matrix GrB_Descriptor parameter settings all objects opaque; allows for internal optimization matrices in compressed-sparse column (CSC) form, with sorted indices non-blocking mode; matrix can have pending operations all operations can take an optional mask: like a bulk if statement, ChMi = ... and an optional accumulator operator: C = C ...
  • 31. GraphBLAS operations GrB_mxm matrix-matrix multiply ChMi = C AB GrB_vxm vector-matrix multiply w0 hm0 i = w0 u0 A GrB_mxv matrix-vector multiply whmi = w Au GrB_eWiseMult element-wise, ChMi = C (A ⌦ B) set union whmi = w (u ⌦ v) GrB_eWiseAdd element-wise, ChMi = C (A B) set intersection whmi = w (u v) GrB_extract extract submatrix ChMi = C A(i, j) whmi = w u(i) GrB_assign assign submatrix C(i, j)hMi = C(i, j) A w(i)hmi = w(i) u GrB_apply apply unary operator ChMi = C f (A) whmi = w f (u) GrB_reduce reduce to vector whmi = w [ j A(:, j)] reduce to scalar s = s [ ij A(i, j)] GrB_transpose transpose ChMi = C A0
  • 32. Operations: C(I,J)=A, submatrix/subgraph assignment hardest function to implement modifies C in place costly to modify the matrix/graph, so operations are left pending zombies: edges/entries still in graph/matrix but marked for deletion pending tuples: unsorted list of edges/entries to be added to graph/matrix
  • 33. Building a graph: all at once Creating a matrix from list of tuples: fast in GraphBLAS: for (int k = 0 ; k < nz ; k++) { I [k] = simple_rand_i ( ) % nrows ; J [k] = simple_rand_i ( ) % ncols ; X [k] = simple_rand_x ( ) ; } GrB_Matrix A ; GrB_Matrix_new (&A, GrB_FP64, nrows, ncols) ; GrB_Matrix_build (A, I, J, X, nz, GrB_SECOND_FP64) ; Just as fast in MATLAB: for k = 1:nz I (k) = randi (nrows) ; J (k) = randi (ncols) ; X (k) = rand ( ) ; end A = sparse (I,J,X, nrows,ncols) ;
  • 34. Building a graph: incremental One element at a time: fast in GraphBLAS: GrB_Matrix A ; GrB_Matrix_new (&A, GrB_FP64, nrows, ncols) ; for (int k = 0 ; k < nz ; k++) { GrB_Index i = simple_rand_i ( ) % nrows ; GrB_Index j = simple_rand_i ( ) % ncols ; double x = simple_rand_x ( ) ; // A (i,j) = x GrB_Matrix_setElement (A, x, i, j) ; } Impossibly slow in MATLAB: A = sparse (nrows,ncols) ; % an empty sparse matrix for k = 1:nz i = randi (nrows) ; j = randi (ncols) ; A (i,j) = rand ( ) ; end
  • 35. GraphBLAS performance: C(I,J)=A Submatrix assignment Example: C is the Freescale2 matrix, 3 million by 3 million with 14.3 million nonzeros I = randperm (n,5500) J = randperm (n,7000) A = random sparse matrix with 38,500 nonzeros C(I,J) = A 87 seconds in MATLAB 0.74 seconds in GraphBLAS, without exploiting blocking mode, via GrB_assign
  • 36. Summary GraphBLAS: graph algorithms in the language of linear algebra “Sparse-anything” matrices, including user-defined types matrix multiplication with any semiring operations: C=A*B, C=A+B, reduction, transpose, accumulator/mask, submatrix extraction and assigment performance: most operations just as fast as MATLAB, submatrix assignment 100x or faster. Version 2.0.1 available at suitesparse.com, Debian, Ubuntu, Mac HomeBrew, ...
  • 38. Friend of friend MATCH (src)-[:friend]->(f)-[:friend]-(fof) WHERE src.age > 30 RETURN fof src f fof friend friend 38
  • 39. Execution plan MATCH
 (src)-[:friend]->(f)-[:friend]->(fof) WHERE src.age > 30 RETURN fof Index scan Expand Expand Project src.age > 30 (src)-[:friend]->(f) (f)-[:friend]->(fof) RETURN fof 39
  • 40. Execution plan Index scan Expand Expand Project Entity ID 5 40 src.age > 30 (src)-[:friend]->(f) (f)-[:friend]->(fof) RETURN fof
  • 41. Execution plan Index scan Expand Expand Project 5 connected to 2 41 src.age > 30 (src)-[:friend]->(f) (f)-[:friend]->(fof) RETURN fof
  • 42. Execution plan Index scan Expand Expand Project 2 connected to 9 42 src.age > 30 (src)-[:friend]->(f) (f)-[:friend]->(fof) RETURN fof
  • 43. Execution plan Index scan Expand Expand ProjectProject 9 43 src.age > 30 (src)-[:friend]->(f) (f)-[:friend]->(fof) RETURN fof
  • 44. Execution plan Index scan Expand Expand Project 2 connected to 1 44 src.age > 30 (src)-[:friend]->(f) (f)-[:friend]->(fof) RETURN fof
  • 45. Execution plan Index scan Expand Expand ProjectProject 1 45 src.age > 30 (src)-[:friend]->(f) (f)-[:friend]->(fof) RETURN fof
  • 46. Execution plan Index scan Expand Expand Project 2 depleted 46 src.age > 30 (src)-[:friend]->(f) (f)-[:friend]->(fof) RETURN fof
  • 47. Execution plan Index scan Expand Expand Project 5 depleted 47 src.age > 30 (src)-[:friend]->(f) (f)-[:friend]->(fof) RETURN fof
  • 48. Execution plan Index scan Expand Expand Project Entity ID 8 48 src.age > 30 (src)-[:friend]->(f) (f)-[:friend]->(fof) RETURN fof
  • 49. Execution plan • Serial • Random memory access • Discovers one entity at a time 49
  • 52. MATCH
 (src)-[:friend]->(f)-[:friend]->(fof) WHERE src.age > 30 RETURN fof = Age_Filter * Friendship * Friendship 52
  • 54. 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 Age Filter 0 1 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 1 0 0 0 0 1 0 1 0 1 0 0 0 0 1 0 1 0 0 Friendships 0 1 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 1 0 0 0 0 1 0 1 0 1 0 0 0 0 1 0 1 0 0 Friendships * * 54
  • 56. 0 1 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 1 0 0 0 0 1 0 1 0 1 0 0 0 0 1 0 1 0 0 Friendships 0 1 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 1 0 0 0 0 1 0 1 0 1 0 0 0 0 1 0 1 0 0 Friendships * 1 0 1 0 0 0 1 0 0 0 1 1 1 1 1 1 1 0 1 0 1 0 0 0 1 1 0 0 1 1 0 0 1 0 1 0 Friendships ^2 = NNZ = 18 56
  • 57. Age Filter 0 1 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 1 0 0 0 0 1 0 1 0 1 0 0 0 0 1 0 1 0 0 Friendships 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 Filtered friendships
 src > 30 * = NNZ = 7 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 57
  • 58. 0 1 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 1 0 0 0 0 1 0 1 0 1 0 0 0 0 1 0 1 0 0 Friendships 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 Filtered friendships
 src > 30 * = 0 0 0 0 0 0 1 0 0 0 1 1 1 1 1 1 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 FOF 58
  • 59. 0 0 0 0 0 0 1 0 0 0 1 1 1 1 1 1 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 1 5 4 2 63 59
  • 60. 0 0 0 0 0 0 1 0 0 0 1 1 1 1 1 1 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 1 5 4 2 63 60
  • 61. Friend of friend variable length MATCH (src)-[:friend*2..4]->(fof) WHERE src.age > 30 RETURN fof src F2 fof friend F3 F4 61
  • 62. MATCH (src)-[:friend*2..4]->(fof) WHERE src.age > 30 RETURN fof = Age_Filter * (Friendship^2 + Friendship^3 + Friendship^4) 
 = M = AF;
 R = 0;
 For i=0; i < 3; i++
 M = M*F
 R = R+M 62
  • 63. 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 Age filter 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 0 1 0 1 1 Friendships^2 + Friendships^3 1 1 1 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 1 1 0 0 0 0 0 0 1 0 1 0 1 1 Friendships * = 63
  • 64. 1 5 4 2 63 1 1 1 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 1 1 0 0 0 0 0 0 1 0 1 0 1 1 64
  • 65. Additional algorithms • Connected Components • Shortest paths • Minimum spanning tree 65
  • 66. Graph distribution Block multiplication A*B=C A B C A1 A3 A4 A2 B1 B2 B4B3 C1 C2 C3 C4 66
  • 67. Graph distribution Block multiplication A*B=C A B C A1 A3 A4 A2 B1 B2 B4B3 A1*B1+
 A2*B3 A1*B2+
 A2*B4 A3*B1+
 A4*B3 A3*B2+
 A4*B4 67
  • 68. Parallelize • CuSPARSE - GPU • OpenMP - CPU 68
  • 69. Benchmarks 69 Benchmarking graph databases on the problem of community detection paper Reports a comprehensive comparative evaluation
 between three popular graph databases, Titan, OrientDB and Neo4j. For evaluation they’ve used real data derived from the SNAP dataset collection. All experiments were run on an Intel Core i7 at 3.5Ghz with 16GB of main memory
 and a 1.4 TB hard disk, the OS being Ubuntu Linux 12.04 (64bit). We’ve performed the same benchmarks against RedisGraph, using inferior hardware.
  • 70. Benchmarks 70 Massive Insertion Workload (MIW) Create the graph database and configure it for massive loading. Populate it with a particular dataset. Measure the time for the creation of the whole graph. All the measurements are in seconds
 Dataset contains 1134890 nodes and 2987624 edges RedisGraph Titan OrientDB Neo4j 0 75 150 225 300 24.69 252.15 104.27 0.53
  • 71. Benchmarks 71 Query Workload FindNeighbours (FN)
 finds the neighbours of all nodes All the measurements are in seconds
 Dataset contains 1134890 nodes and 2987624 edges RedisGraph Titan OrientDB Neo4j 0 7.5 15 22.5 30 4.51 9.34 20.71 0.05
  • 72. Benchmarks 72 Query Workload FindAdjacentNodes (FA)
 finds the adjacent nodes of all edges. All the measurements are in seconds
 Dataset contains 1134890 nodes and 2987624 edges RedisGraph Titan OrientDB Neo4j 0 12.5 25 37.5 50 1.46 6.15 42.82 0.05
  • 73. Benchmarks 73 Query Workload FindShortestPath (FS)
 Finds the shortest path between the first node and 100 randomly picked nodes. All the measurements are in seconds
 Dataset contains 1134890 nodes and 2987624 edges RedisGraph Titan OrientDB Neo4j 0 7.5 15 22.5 30 0.08 23.47 24.87 0.001