Propel your performance:
AgensGraph,
the multi-model database
AgensGraph from the perspective of the developer
Junseok Yang
Senior Research Engineer, Bitnine Global
jsyang@bitnine.net
https://github.com/bitnine-oss/agensgraph
What We Want
What We Want
Graph * Relational * Document
What We Want
Graph * Relational * Document
Graph Query Language * SQL
What We Want
Graph * Relational * Document
Graph Query Language * SQL
ACID Properties
Don't Reinvent the Wheel
ACID Properties
JSON
SQL
Relational Data Model
Development Goal
Graph Engine
Graph Query Language
Performance
Vertex and Edge - Storage
1
{ name: 'Tom' } { name: 'Jerry' }
2
3
FRIEND_OF
CHARACTER CHARACTER
Vertex and Edge - Storage
{ name: 'Tom' }1 2 { name: 'Jerry' }3 1 2 { }
I P I S E P I P
1
{ name: 'Tom' } { name: 'Jerry' }
2
3
FRIEND_OF
CHARACTER CHARACTER
Label - Storage
{ name: 'Tom' }1
2 { name: 'Jerry' }
CHARACTER
3 1 2 { }
FRIEND_OF
Label Hierarchy - Storage
{ name: 'Tom' }1
2 { name: 'Jerry' }
CHARACTER
3 1 2 { }
FRIEND_OF
{ name: 'Tom' }1 2 { name: 'Jerry' }
CAT MOUSE
CHARACTER
Graph - Storage
Tom
Jerry
Spike
...
graphgraph
database
We Need a Graph Query Language
WITH RECURSIVE friends(path, last, len) AS (
-- non-recursive term
SELECT ARRAY[start], end, 1
FROM FRIEND_OF
WHERE start = 1 -- starting point
UNION
-- recursive term
SELECT n.path || n.last, f.end, n.len + 1
FROM friends AS n JOIN FRIEND_OF AS f ON n.last = f.start
WHERE n.len < 2 AND -- change this to set different length
f.start <> ALL (n.path) -- edge uniqueness check
)
SELECT path FROM friends;
An example query to retrieve friend and friend-of-friend relationship using SQL
Using SQL to retrieve graph data is painful!
Candidate Languages
Cypher
Alternative
The Language We Chose
MATCH (:CHARACTER {name: 'Tom'})-[:FRIEND_OF*1..2]->(f)
RETURN f;
g.V().hasLabel('CHARACTER').has('name', 'Tom')
.outE().hasLabel('FRIEND_OF').inV.loop('f'){it.loops < 2}
Cypher
Alternative
The Language We Chose
MATCH (:CHARACTER {name: 'Tom'})-[:FRIEND_OF*1..2]->(f)
RETURN f;
g.V().hasLabel('CHARACTER').has('name', 'Tom')
.outE().hasLabel('FRIEND_OF').inV.loop('f'){it.loops < 2}
Cypher
Alternative
Cypher * SQL
SELECT n.name
FROM history,
(MATCH (n) RETURN n) AS emp
WHERE history.year > n.year::int AND
history.event = 'AgensGraph';
Cypher in SQL
MATCH (n)
WHERE n.year::int <
(SELECT year
FROM history
WHERE event = 'AgensGraph')
RETURN n.name;
SQL in Cypher
name
---------
Junseok
(1 row)
year event
1996 PostgreSQL
2016 AgensGraph
history
Junseok
2015
Ryan
2016
Write Queries - Challenges
MATCH (n)
CREATE (n)-[r]->(m);
v1
v2
MATCH CREATE
n
Write Queries - Challenges
MATCH (n)
CREATE (n)-[r]->(m);
v2
MATCH CREATE
n
v1
Write Queries - Challenges
MATCH (n)
CREATE (n)-[r]->(m);
v2
MATCH CREATE
n
Write Queries - Challenges
MATCH (n)
CREATE (n)-[r]->(m);
MATCH CREATE
n
v2
Write Queries - Challenges
MATCH (n)
CREATE (n)-[r]->(m);
GraphWrite
n
Scan on Parent Label - Challenges
Scan on "CHARACTER"
Scan on "CAT"
Scan on "MOUSE"
Append
MATCH (n:CHARACTER) WHERE id(n) = ?
RETURN n;
CHARACTER
CAT MOUSE
Label Hierarchy
planning
Scan on Parent Label - Challenges
Scan on "CHARACTER"
Scan on "CAT"
Scan on "MOUSE"
Append
MATCH (n:CHARACTER) WHERE id(n) = ?
RETURN n;
CHARACTER
CAT MOUSE
Label Hierarchy
planning
scan
Scan on Parent Label - Challenges
Scan on "CHARACTER"
Scan on "CAT"
Scan on "MOUSE"
Append
MATCH (n:CHARACTER) WHERE id(n) = ?
RETURN n;
CHARACTER
CAT MOUSE
Label Hierarchy
planning
2 1
2
Scan
1
3
Scan on Parent Label - Challenges
Scan on "CHARACTER"
Scan on "CAT"
Scan on "MOUSE"
Append
MATCH (n:CHARACTER) WHERE id(n) = ?
RETURN n;
CHARACTER
CAT MOUSE
Label Hierarchy
planning
2 1
2
1
3
Skip scan
Variable Length Relationships - Challenges
MATCH (n)-[r*1..2]->(m)
RETURN m;
Variable Length Relationships - Challenges
MATCH (n)-[r*1..2]->(m)
RETURN m;
Step 1
Variable Length Relationships - Challenges
MATCH (n)-[r*1..2]->(m)
RETURN m;
Step 1
Step 2
Variable Length Relationships - Challenges
MATCH (n)-[r*1..2]->(m)
RETURN m;
Level 1
Parent Node
Variable Length Relationships - Challenges
MATCH (n)-[r*1..2]->(m)
RETURN m;
Level 2
Level 1
Parent Node
Variable Length Relationships - Challenges
MATCH (n)-[r*1..2]->(m)
RETURN m;
Level 2
Level 1
Parent Node
LDBC Benchmark
http://www.ldbcouncil.org/
Members
Oracle Labs, IBM, Huawei, SAP, Sparsity, OpenLink Software, Ontotext, Neo Technology
Graphalytics Benchmark
Semantic Publishing Benchmark
Social Network Benchmark
Performance Comparisons - LDBC Benchmark
* We had optimized the two database as much as we can.
* The benchmark results can vary according to configurations.
10x faster
(appx.)
Compet
itor
(x)
Performance Comparisons - LDBC Benchmark
* We had optimized the two database as much as we can.
* The benchmark results can vary according to configurations.
20x faster
(appx.)
Compet
itor
(x)
Performance Comparisons - LDBC Benchmark
* We had optimized the two database as much as we can.
* The benchmark results can vary according to the configurations.
Compet
itor
(x)
Performance Comparisons - LDBC Benchmark
50x faster
(appx.)
* We had optimized the two database as much as we can.
* The benchmark results can vary according to configurations.
Compet
itor
(x)
Cybercrime Analytic System - Use Case
Faster than the Fastest
● We are working on
○ Improving the performance of shortestpath() function
○ Providing more catalog tables
agensgraph.com
https://github.com/bitnine-oss/agensgraph
Apache License, Version 2.0
Thank you!

Propel your Performance: AgensGraph, the multi-model database

  • 1.
    Propel your performance: AgensGraph, themulti-model database AgensGraph from the perspective of the developer Junseok Yang Senior Research Engineer, Bitnine Global jsyang@bitnine.net https://github.com/bitnine-oss/agensgraph
  • 2.
  • 3.
    What We Want Graph* Relational * Document
  • 4.
    What We Want Graph* Relational * Document Graph Query Language * SQL
  • 5.
    What We Want Graph* Relational * Document Graph Query Language * SQL ACID Properties
  • 6.
    Don't Reinvent theWheel ACID Properties JSON SQL Relational Data Model
  • 7.
    Development Goal Graph Engine GraphQuery Language Performance
  • 8.
    Vertex and Edge- Storage 1 { name: 'Tom' } { name: 'Jerry' } 2 3 FRIEND_OF CHARACTER CHARACTER
  • 9.
    Vertex and Edge- Storage { name: 'Tom' }1 2 { name: 'Jerry' }3 1 2 { } I P I S E P I P 1 { name: 'Tom' } { name: 'Jerry' } 2 3 FRIEND_OF CHARACTER CHARACTER
  • 10.
    Label - Storage {name: 'Tom' }1 2 { name: 'Jerry' } CHARACTER 3 1 2 { } FRIEND_OF
  • 11.
    Label Hierarchy -Storage { name: 'Tom' }1 2 { name: 'Jerry' } CHARACTER 3 1 2 { } FRIEND_OF { name: 'Tom' }1 2 { name: 'Jerry' } CAT MOUSE CHARACTER
  • 12.
  • 13.
    We Need aGraph Query Language WITH RECURSIVE friends(path, last, len) AS ( -- non-recursive term SELECT ARRAY[start], end, 1 FROM FRIEND_OF WHERE start = 1 -- starting point UNION -- recursive term SELECT n.path || n.last, f.end, n.len + 1 FROM friends AS n JOIN FRIEND_OF AS f ON n.last = f.start WHERE n.len < 2 AND -- change this to set different length f.start <> ALL (n.path) -- edge uniqueness check ) SELECT path FROM friends; An example query to retrieve friend and friend-of-friend relationship using SQL Using SQL to retrieve graph data is painful!
  • 14.
  • 15.
    The Language WeChose MATCH (:CHARACTER {name: 'Tom'})-[:FRIEND_OF*1..2]->(f) RETURN f; g.V().hasLabel('CHARACTER').has('name', 'Tom') .outE().hasLabel('FRIEND_OF').inV.loop('f'){it.loops < 2} Cypher Alternative
  • 16.
    The Language WeChose MATCH (:CHARACTER {name: 'Tom'})-[:FRIEND_OF*1..2]->(f) RETURN f; g.V().hasLabel('CHARACTER').has('name', 'Tom') .outE().hasLabel('FRIEND_OF').inV.loop('f'){it.loops < 2} Cypher Alternative
  • 17.
    Cypher * SQL SELECTn.name FROM history, (MATCH (n) RETURN n) AS emp WHERE history.year > n.year::int AND history.event = 'AgensGraph'; Cypher in SQL MATCH (n) WHERE n.year::int < (SELECT year FROM history WHERE event = 'AgensGraph') RETURN n.name; SQL in Cypher name --------- Junseok (1 row) year event 1996 PostgreSQL 2016 AgensGraph history Junseok 2015 Ryan 2016
  • 18.
    Write Queries -Challenges MATCH (n) CREATE (n)-[r]->(m); v1 v2 MATCH CREATE n
  • 19.
    Write Queries -Challenges MATCH (n) CREATE (n)-[r]->(m); v2 MATCH CREATE n v1
  • 20.
    Write Queries -Challenges MATCH (n) CREATE (n)-[r]->(m); v2 MATCH CREATE n
  • 21.
    Write Queries -Challenges MATCH (n) CREATE (n)-[r]->(m); MATCH CREATE n v2
  • 22.
    Write Queries -Challenges MATCH (n) CREATE (n)-[r]->(m); GraphWrite n
  • 23.
    Scan on ParentLabel - Challenges Scan on "CHARACTER" Scan on "CAT" Scan on "MOUSE" Append MATCH (n:CHARACTER) WHERE id(n) = ? RETURN n; CHARACTER CAT MOUSE Label Hierarchy planning
  • 24.
    Scan on ParentLabel - Challenges Scan on "CHARACTER" Scan on "CAT" Scan on "MOUSE" Append MATCH (n:CHARACTER) WHERE id(n) = ? RETURN n; CHARACTER CAT MOUSE Label Hierarchy planning scan
  • 25.
    Scan on ParentLabel - Challenges Scan on "CHARACTER" Scan on "CAT" Scan on "MOUSE" Append MATCH (n:CHARACTER) WHERE id(n) = ? RETURN n; CHARACTER CAT MOUSE Label Hierarchy planning 2 1 2 Scan 1 3
  • 26.
    Scan on ParentLabel - Challenges Scan on "CHARACTER" Scan on "CAT" Scan on "MOUSE" Append MATCH (n:CHARACTER) WHERE id(n) = ? RETURN n; CHARACTER CAT MOUSE Label Hierarchy planning 2 1 2 1 3 Skip scan
  • 27.
    Variable Length Relationships- Challenges MATCH (n)-[r*1..2]->(m) RETURN m;
  • 28.
    Variable Length Relationships- Challenges MATCH (n)-[r*1..2]->(m) RETURN m; Step 1
  • 29.
    Variable Length Relationships- Challenges MATCH (n)-[r*1..2]->(m) RETURN m; Step 1 Step 2
  • 30.
    Variable Length Relationships- Challenges MATCH (n)-[r*1..2]->(m) RETURN m; Level 1 Parent Node
  • 31.
    Variable Length Relationships- Challenges MATCH (n)-[r*1..2]->(m) RETURN m; Level 2 Level 1 Parent Node
  • 32.
    Variable Length Relationships- Challenges MATCH (n)-[r*1..2]->(m) RETURN m; Level 2 Level 1 Parent Node
  • 33.
    LDBC Benchmark http://www.ldbcouncil.org/ Members Oracle Labs,IBM, Huawei, SAP, Sparsity, OpenLink Software, Ontotext, Neo Technology Graphalytics Benchmark Semantic Publishing Benchmark Social Network Benchmark
  • 34.
    Performance Comparisons -LDBC Benchmark * We had optimized the two database as much as we can. * The benchmark results can vary according to configurations. 10x faster (appx.) Compet itor (x)
  • 35.
    Performance Comparisons -LDBC Benchmark * We had optimized the two database as much as we can. * The benchmark results can vary according to configurations. 20x faster (appx.) Compet itor (x)
  • 36.
    Performance Comparisons -LDBC Benchmark * We had optimized the two database as much as we can. * The benchmark results can vary according to the configurations. Compet itor (x)
  • 37.
    Performance Comparisons -LDBC Benchmark 50x faster (appx.) * We had optimized the two database as much as we can. * The benchmark results can vary according to configurations. Compet itor (x)
  • 38.
  • 39.
    Faster than theFastest ● We are working on ○ Improving the performance of shortestpath() function ○ Providing more catalog tables
  • 40.