HBaseCon 2015: S2Graph - A Large-scale Graph Database with HBase

HBaseCon
S2Graph :
A large-scale graph database
with Hbase
daumkakao
Doyoung Yoon x Taejin Chin
2
DaumKakao
A Mobile Lifestyle Platform
1. KakaoTalk
a. Mobile Messenger replacing SMS
b. ‘KaTalkHe’ is being used as a verb in Korea like ‘Googling’
c. 96% of Korean smartphone users are using KakaoTalk
d. 170M users worldwide
e. 3B messages / day
3
KakaoTalk
Social
Platform
DaumKakao
A Mobile Lifestyle Platform
KakaoStory
KakaoGroup
Daum Cafe
Contents
Platform
KakaoTopic
KakaoPage
KakaoGame
Commerce
Platform
KakaoPick
KakaoMusic
Marketing
Platform
Yellow IDMedia Daum
Daum tvPot
Local
Platform
Daum Map
Daum Webtoon
Personal
Platform
Sol calendarKakaoPlace
Sol Mail
Zap
Plus FriendGift Shop
Digital Item Store KakaoStyle
KakaoHome
Sol Group
Story Plus
Daum Cluod
Biggest mobile SNS
in Korea
96% of Korean
smartphone users are
using KakaoTalk messenger,
170 million users
worldwide)
4
Our Social Graph
Message
length : 9
Write
length : 3
Read
impact :3
Coupon
price : 10
Present
price : 3
affinity 6affinity: 9
affinity 3
affinity 3
affinity 4
affinity 1
affinity 2
affinity 2
affinity 9
Friend
Group
size : 6
Emoticon
count : 7
Eat
rating : 4
View
count : 8
Play
level: 6
Pick
withFriend : 3
Advertise
ctr : 0.32
Search
keyword
: “HBase"
Listen
count : 6
Like
count : 7
Comment
length : 15
affinity 3
5
Our Social Graph
Message
length : 9
Write
length : 3
affinity 6affinity: 9
affinity 3
affinity 3
affinity 4
affinity 1
affinity 2
affinity 2
affinity 9
Friend
Play
level: 6
Pick
withFriend : 3
Advertise
ctr : 0.32
Search
keyword
: “HBase"
Listen
count : 6
C
l
affinity 3
Message ID : 201
Ad ID : 603
Music ID
Item ID : 13
Post ID : 97
Game ID : 1984
6
Technical Challenges
1. Large social graph constantly changing
a. Scale
more than,
social network: 10 billion edges, 200 million vertices, 50 million update on existing edges.
user activities: 400 million new edges per day
7
Technical Challenges (cont)
2. Low latency for breadth first search traversal on connected data.
a. performance requirement
peak graph-traversing query per second: 20000
response time: 100ms
8
Technical Challenges (cont)
3. Update should be applied to graph in real time for viral effect
Person A
Post
Fast Person B
Comment
Person C
Sharing
Person D
Mention
Fast Fast
9
Technical Challenges (cont)
4. Support for Dynamic Ranking logic
a. push strategy: hard to change data ranking logic dynamically.
b. pull strategy: can try various data ranking logic
10
Before
Each app server should know each DB’s sharding logic.
Highly inter-connected architecture
Friend relationship SNS feeds Blog user activities Messaging
Messaging
App
SNS
App
Blog
App
11
After
SNS
App
Blog
App
Messaging
App
S2Graph DB
stateless app servers
12
S2Graph : Distributed Online GraphDB
1. Low-latency
2.Graph-traversable
3.Scalable
4.Eventually consistent
5.Asynchronous, non-blocking
13
Why We Choose HBase?
1. High Availability
2.Scalability
3.Low latency
4.High concurrency
5.Fault tolerant
6.Integration with HDFS
7.Distributed operation
14
The Data Model
1. Columns
2. Labels
3. Directions
4. Index Properties
5. Non-index Properties 1 3comment
4
know created
name = “josh”

age = 32
edge 2 source vertex
vertex 1 out edges
edge 2 target vertex
2
edge 2 label
vertex 2 in edges
vertex 4 id
vertex 4 properties
date = 20150507
edge 5 properties
5
15
How to store the data - Edge
Logical View
1. Snapshot edges : Up-to-date status of edge
column
row
Tgt Vertex ID1 Tgt Vertex ID2 Tgt Vertex ID3
Src Vertex ID1 Properties Properties Properties
Src Vertex ID2 Properties Properties Properties
a. Fetching an edge between two specific vertex
b. Lookup Table to reach indexed edges for update, increment, delete operations
16
How to store the data - Edge
Logical View
2. Indexed edges : Edges with index
column
row
Index Values | Tgt Vertex ID1 Index Values | Tgt Vertex ID2
Src Vertex ID1 Non-index Properties Non-index Properties
a. Fetches edges originating from a certain vertex in order of index
17
How to store the data - Edge
Physical View - table schema
1. Snapshot Edge
a. Rowkey
Murmur Hash Src Vertex ID Label ID Direction Index Sequence Is Inverted
16 bit variable length 30 bit 2 bit 7bit 1 bit
Vertex IDs can be encoded with 8 bit header + byte array (long, integer, short, byte, string)
18
How to store the data - Edge
Physical View - table schema
1. Snapshot Edge
b. Qualifier
Target Vertex ID
variable length
c. Value
All Property Key Value Pairs
variable length
19
How to store the data - Edge
Physical View - table schema
2. Indexed Edge
a. Rowkey
Murmur Hash Src Vertex ID Label ID Direction Index Sequence Is Inverted
16 bit variable length 30 bit 2 bit 7bit 1 bit
Vertex IDs can be encoded with 8 bit header + byte array (long, integer, short, byte, string)
20
How to store the data - Edge
Physical View - table schema
2. Indexed Edge
b. Qualifier
Index Property Values Tgt Vertex ID
variable length variable length
c. Value
Non-index Property Key Value Pairs
variable length
21
How to store the data - Vertex
Logical View
1. Vertex : Up-to-date status of Vertex
column
row
Property Key1 Property Key2
Src Vertex ID1 Value1 Value2
Vertex ID2 Value1 Value2
22
How to store the data - Vertex
Physical View - table schema
1. Vertex : Up-to-date status of Vertex
a. Rowkey
Murmur Hash Column ID Vertex ID
16 bit integer(32bit) variable length
b. Qualifier
Property Key
Byte(8 bit)
c. Value
Property Value
variable length
23
How to read the data - GetEdges
Using a custom query DSL on top of HTTP
curl -XPOST localhost:9000/graphs/getEdges -H 'Content-Type: Application/json' -d '
{
"srcVertices": [{"serviceName": "s2graph", "columnName": "account_id", "id":1}],
"steps": [
[{"label": "friends", "direction": "out", "limit": 100}], // step
[{"label": "hear", "direction": "out", "limit": 10}]
]
}
'
Steps = a list of Step
Step = contains the labels to traverse
and how to rank them in the result
Step 1
friend 1
hear
time: 20140502
hear
time: 20140712
hear
time: 20141116
Friends Friends
friend 2
User 1
Step 2
Don’t let go let it be let it go
24
How to read the data - GetEdges Example
Friend list
curl -XPOST localhost:9000/graphs/getEdges -H 'Content-Type: Application/json' -d '
{
"srcVertices": [{"serviceName": "s2graph", "columnName": "account_id", "id":1}],
"steps": [
[{"label": "friends", "direction": "out", "limit": 100}], // step
]
}
'
friend 1 friend 2
User 1
Friends Friends
25
How to read the data - GetEdges Example
Songs my friends have listened
curl -XPOST localhost:9000/graphs/getEdges -H 'Content-Type: Application/json' -d '
{
"srcVertices": [{"serviceName": "s2graph", "columnName": "account_id", "id":1}],
"steps": [
[{"label": "friends", "direction": "out", "limit": 50, “scoring”: {“score”: 1.0}],
[{"label": "listen", "direction": "out", "limit": 10}]
]
}
'
friend 1
Friends Friends
friend 2
Don’t let go let it be let it go
hear
time: 20140502
hear
time: 20140712
hear
time: 20141116
User 1
Reference : https://github.com/daumkakao/s2graph#1-definition
26
How to read the data - GetEdges Example
Similar songs to songs that I have listened to.
curl -XPOST localhost:9000/graphs/getEdges -H 'Content-Type: Application/json' -d '
{
"srcVertices": [{"serviceName": "s2graph", "columnName": "account_id", "id":1}],
"steps": [
[{"label": "listen", "direction": "out", "limit": 50}],
[{"label": "similar_song", "direction": "out", "limit": 10, “scoring”: {“score”: 1.0}]
]
}
User 1
Don’t let go let it be let it go
hear
time: 20140502
hear
time: 20140712
hear
time: 20141116
let it bleed Hey jude Do you wanna
build a snowman?
similar_song
similarity: 0.3
similar_song
similarity: 0.4
similar_song
similarity: 0.6
27
How to read the data - GetVertices
curl -XPOST localhost:9000/graphs/getVertices -H 'Content-Type: Application/json' -d '
[
{"serviceName": "s2graph", "columnName": "account_id", "ids": [1, 2, 3]},
{"serviceName": "kakaomusic", "columnName": "user_id", "ids": [1, 2, 3]}
]
'
User 1
{created_at:20070812,
updated_at:20150507}
User 2
{created_at:201206132,
updated_at:20140505}
28
How to write the data - Insert
curl -XPOST localhost:9000/graphs/edges/insert -H 'Content-Type: Application/json' -d '
[
{"from":1,"to":2,"label":"graph_test","props":{"time":-1, "weight":10},"timestamp":1417616431},
]
'
User 1 User 2
29
How to write the data - Delete
curl -XPOST localhost:9000/graphs/edges/delete -H 'Content-Type: Application/json' -d '
[
{"from":1,"to":2,"label":"graph_test","timestamp":1417616431},
{"from":1,"to":3,"label":"graph_test","timestamp":1417616431},
]
'
User 1 User 2
30
How to write the data - Update
curl -XPOST localhost:9000/graphs/edges/update -H 'Content-Type: Application/json' -d '
[
{"from":1,"to":2,"label":"graph_test","timestamp":1417616431, "props": {"is_hidden": true, “status”: 200},
{"from":1,"to":3,"label":"graph_test","timestamp":1417616431, "props": {"status": -500}
]
User 1 User 2
friend
{is_hidden:true,
status:200}
31
REST API Spec.
Read
1. getEdges
2. checkEdge
3. getEdgesCount
4. getVertices
Write
1. insert
2. delete
3. update
4. increment
Management
1. create service (vertex type)
2. create label (edge type)
3. add Index
32
HBase Table Configuration
1. setDurability(Durability.ASYNC_WAL)
2. setCompressionType(Compression.Algorithm.LZ4)
3. setBloomFilterType(BloomType.Row)
4. setDataBlockEncoding(DataBlockEncoding.FAST_DIFF)
5. setBlockSize(32768)
6. setBlockCacheEnabled(true)
7. pre-split by (Intger.MaxValue / regionCount). regionCount = 120 when create table(on 20 region server).
33
HBase Cluster Configuration
• each machine: 8core, 32G memory
• hfile.block.cache.size: 0.6
• hbase.hregion.memstore.flush.size: 128MB
• otherwise use default value from CDH 5.3.1
34
Overall Architecture
S2graphClients
35
Compare to other Online GraphDBs
Titan (v0.4.2)
a. Pros

- Rich API and easy to setup

- Relatively large community

- Transaction handling
b. Cons

- Using it’s own ID system; less efficient for graph traversal (details in next slide)

- Index data stored on one region (hotspot) with strong consistency option

- Not many references on Titan with HBase comparing to other storages
36
Compare to Titan
Titan is less efficient for graph traversal
- For following 1 normal graph traversal query,
Vertex(“userID:1”).out(“friends”).limit(10).out(“friends”).limit(10)
User 1
friends
friends
37
Compare to Titan (cont)
Vertex(“userID:1”).out(“friend”).limit(10).out(“friend”).limit(10)
Titan S2graph
# of read requests 

on HBase
112 =
1 (Vertex Lookup : a)
+ 1 (1st step edges : b)
+ 10 (2nd step edges : c)
+ 100 (Destination Vertices : d)
11 =
1 (1step edges : e)
+ 10 (2nd step edges : f)
Titan S2graph
B
A
C
D
e
f
38
Performance
1. Test data
a. Total # of Edges: 9,000,000,000
b. Average # of adjacent edges per vertex: 500
c. Seed vertex: vertices that has more than 100 adjacent edges.
2. Test environment
a. Zookeeper server: 3
b. HBase Masterserver: 2
c. HBase Regionserver: 20
d. App server: 8 core, 16GB Ram
39
- Benchmark Query : src.out(“friend”).limit(50).out(“friend”).limit(10)
- Total concurrency: 20 * # of app server
Performance
2. Linear scalability
Latency
0
50
100
150
200
QPS
0
1,000
2,000
3,000
4,000
# of app server
1 2 4 8
QPS(Query Per Second) Latency(ms)
51515047
3,097
1,567
803
42147 50 51 51
# of app server
1 2 3 4 5 6 7 8
50010001500200025003000
QPS
40
Performance
3. Varying width of traverse (tested with a single server)
Latency
0
75
150
225
300
QPS
0
500
1,000
1,500
2,000
Limit on first step
20 40 80 100
QPS Latency(ms)
97
75
43
23 203266
457
943
23
43
75
97
- Benchmark Query : src.out(“friend”).limit(x).out(“friend”).limit(10)
- Total concurrency = 20 * 1(# of app server)
Performance
3. Varying width of traverse (tested with a single server)
Latency
0
75
150
225
300
QPS
0
500
1,000
1,500
2,000
Limit on first step
20 40 80 100
QPS Latency(ms)
97
75
43
23 203266
457
943
23
43
75
97
- Benchmark Query : src.out(“friend”).limit(x).out(“friend”).limit(10)
- Total concurrency = 20 * 1(# of app server)
42
- All query touch 1000 edges.
- each step` limit is on x axis.
- Can expect performance with given query`s search space.
Performance
4. Different query path(different I/O pattern)
Latency
0
37.5
75
112.5
150
QPS
0
80
160
240
320
400
limits on path
10 -> 100 100 -> 10 10 -> 10 -> 10 2 -> 5 -> 10 -> 10 2 -> 5 -> 2 -> 5 -> 10
QPS Latency(ms)
5667716867
352.2
298.1280272.5297
67 68 71 67 56
43
Performance
5. Write throughput per operation on single app server
Insert operation
Latency
0
1.25
2.5
3.75
5
Request per second
8000 16000 800000
44
Performance
6. write throughput per operation on single app server
Update(increment/update/delete) operation
Latency
0
2
4
6
8
Request per second
2000 4000 6000
45
Stats
1. HBase cluster per IDC (2 IDC)
- 3 Zookeeper Server
- 2 HBase Master
- 20 HBase Slave
2. App server per IDC
- 10 server for write-only
- 20 server for query only
3. Real traffic
- read: over 10K request per second
- now mostly 2 step queries with limit 100 on first step.
- write: over 5k request per second
* Deep traversal queries are not counted since it is in test stage for production
46
47
Through HBase !
48
Now Available As an Open Source
- https://github.com/daumkakao/s2graph
- Finding a mentor
Contact
- Taejin Chin : taejin.chin@gmail.com
- Doyoung Yoon : shom83@gmail.com
49
Latency
0
50
100
150
200
QPS
0
500
1,000
1,500
2,000
# of app server
1 2 3 4 5
Native Client QPS
Native Client Latency(ms)
174186189
177178
570
429315224112
178 177
189 186 174
- Benchmark Query : src.out(“friend”).limit(50).out(“friend”).limit(10)
- Test seed edges have adjacent edges more than 100: 30millions
- Total concurrency: 20 * # of app server
Appendix
Latency
0
50
100
150
200
QPS
0
500
1,000
1,500
2,000
# of app server
1 2 3 4 5
Asnychbase QPS
Asynchbase Latency(ms)
5351505047
1,895
1,567
1,192
803
421
47 50 50 51 53
3.5x performance improvement using Asynchbase
1 of 49

Recommended

Hbase status quo apache-con europe - nov 2012 by
Hbase status quo   apache-con europe - nov 2012Hbase status quo   apache-con europe - nov 2012
Hbase status quo apache-con europe - nov 2012Chris Huang
657 views40 slides
HBaseCon 2013: Apache Drill - A Community-driven Initiative to Deliver ANSI S... by
HBaseCon 2013: Apache Drill - A Community-driven Initiative to Deliver ANSI S...HBaseCon 2013: Apache Drill - A Community-driven Initiative to Deliver ANSI S...
HBaseCon 2013: Apache Drill - A Community-driven Initiative to Deliver ANSI S...Cloudera, Inc.
4.7K views11 slides
HBaseCon 2015: HBase @ CyberAgent by
HBaseCon 2015: HBase @ CyberAgentHBaseCon 2015: HBase @ CyberAgent
HBaseCon 2015: HBase @ CyberAgentHBaseCon
5.5K views90 slides
Strata Conference + Hadoop World NY 2016: Lessons learned building a scalable... by
Strata Conference + Hadoop World NY 2016: Lessons learned building a scalable...Strata Conference + Hadoop World NY 2016: Lessons learned building a scalable...
Strata Conference + Hadoop World NY 2016: Lessons learned building a scalable...Sumeet Singh
1.1K views71 slides
The Hidden Life of Spark Jobs by
The Hidden Life of Spark JobsThe Hidden Life of Spark Jobs
The Hidden Life of Spark JobsDataWorks Summit
1K views33 slides
A Graph Service for Global Web Entities Traversal and Reputation Evaluation B... by
A Graph Service for Global Web Entities Traversal and Reputation Evaluation B...A Graph Service for Global Web Entities Traversal and Reputation Evaluation B...
A Graph Service for Global Web Entities Traversal and Reputation Evaluation B...HBaseCon
4.1K views54 slides

More Related Content

What's hot

HBaseCon 2012 | HBase for the Worlds Libraries - OCLC by
HBaseCon 2012 | HBase for the Worlds Libraries - OCLCHBaseCon 2012 | HBase for the Worlds Libraries - OCLC
HBaseCon 2012 | HBase for the Worlds Libraries - OCLCCloudera, Inc.
3.9K views12 slides
Real-Time Video Analytics Using Hadoop and HBase (HBaseCon 2013) by
Real-Time Video Analytics Using Hadoop and HBase (HBaseCon 2013)Real-Time Video Analytics Using Hadoop and HBase (HBaseCon 2013)
Real-Time Video Analytics Using Hadoop and HBase (HBaseCon 2013)Suman Srinivasan
10.9K views14 slides
Rolling Out Apache HBase for Mobile Offerings at Visa by
Rolling Out Apache HBase for Mobile Offerings at Visa Rolling Out Apache HBase for Mobile Offerings at Visa
Rolling Out Apache HBase for Mobile Offerings at Visa HBaseCon
2.6K views39 slides
Introduction to the Hadoop Ecosystem (IT-Stammtisch Darmstadt Edition) by
Introduction to the Hadoop Ecosystem (IT-Stammtisch Darmstadt Edition)Introduction to the Hadoop Ecosystem (IT-Stammtisch Darmstadt Edition)
Introduction to the Hadoop Ecosystem (IT-Stammtisch Darmstadt Edition)Uwe Printz
10.7K views127 slides
2013 Nov 20 Toronto Hadoop User Group (THUG) - Hadoop 2.2.0 by
2013 Nov 20 Toronto Hadoop User Group (THUG) - Hadoop 2.2.02013 Nov 20 Toronto Hadoop User Group (THUG) - Hadoop 2.2.0
2013 Nov 20 Toronto Hadoop User Group (THUG) - Hadoop 2.2.0Adam Muise
3.1K views59 slides
HBaseCon 2013: HBase SEP - Reliable Maintenance of Auxiliary Index Structures by
HBaseCon 2013: HBase SEP - Reliable Maintenance of Auxiliary Index StructuresHBaseCon 2013: HBase SEP - Reliable Maintenance of Auxiliary Index Structures
HBaseCon 2013: HBase SEP - Reliable Maintenance of Auxiliary Index StructuresCloudera, Inc.
4.1K views16 slides

What's hot(20)

HBaseCon 2012 | HBase for the Worlds Libraries - OCLC by Cloudera, Inc.
HBaseCon 2012 | HBase for the Worlds Libraries - OCLCHBaseCon 2012 | HBase for the Worlds Libraries - OCLC
HBaseCon 2012 | HBase for the Worlds Libraries - OCLC
Cloudera, Inc.3.9K views
Real-Time Video Analytics Using Hadoop and HBase (HBaseCon 2013) by Suman Srinivasan
Real-Time Video Analytics Using Hadoop and HBase (HBaseCon 2013)Real-Time Video Analytics Using Hadoop and HBase (HBaseCon 2013)
Real-Time Video Analytics Using Hadoop and HBase (HBaseCon 2013)
Suman Srinivasan10.9K views
Rolling Out Apache HBase for Mobile Offerings at Visa by HBaseCon
Rolling Out Apache HBase for Mobile Offerings at Visa Rolling Out Apache HBase for Mobile Offerings at Visa
Rolling Out Apache HBase for Mobile Offerings at Visa
HBaseCon2.6K views
Introduction to the Hadoop Ecosystem (IT-Stammtisch Darmstadt Edition) by Uwe Printz
Introduction to the Hadoop Ecosystem (IT-Stammtisch Darmstadt Edition)Introduction to the Hadoop Ecosystem (IT-Stammtisch Darmstadt Edition)
Introduction to the Hadoop Ecosystem (IT-Stammtisch Darmstadt Edition)
Uwe Printz10.7K views
2013 Nov 20 Toronto Hadoop User Group (THUG) - Hadoop 2.2.0 by Adam Muise
2013 Nov 20 Toronto Hadoop User Group (THUG) - Hadoop 2.2.02013 Nov 20 Toronto Hadoop User Group (THUG) - Hadoop 2.2.0
2013 Nov 20 Toronto Hadoop User Group (THUG) - Hadoop 2.2.0
Adam Muise3.1K views
HBaseCon 2013: HBase SEP - Reliable Maintenance of Auxiliary Index Structures by Cloudera, Inc.
HBaseCon 2013: HBase SEP - Reliable Maintenance of Auxiliary Index StructuresHBaseCon 2013: HBase SEP - Reliable Maintenance of Auxiliary Index Structures
HBaseCon 2013: HBase SEP - Reliable Maintenance of Auxiliary Index Structures
Cloudera, Inc.4.1K views
HBase Backups by HBaseCon
HBase BackupsHBase Backups
HBase Backups
HBaseCon6.7K views
Building a Versatile Analytics Pipeline on Top of Apache Spark with Mikhail C... by Databricks
Building a Versatile Analytics Pipeline on Top of Apache Spark with Mikhail C...Building a Versatile Analytics Pipeline on Top of Apache Spark with Mikhail C...
Building a Versatile Analytics Pipeline on Top of Apache Spark with Mikhail C...
Databricks6K views
Introduction to the Hadoop Ecosystem (codemotion Edition) by Uwe Printz
Introduction to the Hadoop Ecosystem (codemotion Edition)Introduction to the Hadoop Ecosystem (codemotion Edition)
Introduction to the Hadoop Ecosystem (codemotion Edition)
Uwe Printz1.4K views
HBaseCon 2015: HBase and Spark by HBaseCon
HBaseCon 2015: HBase and SparkHBaseCon 2015: HBase and Spark
HBaseCon 2015: HBase and Spark
HBaseCon8.7K views
Apache Drill and Zeppelin: Two Promising Tools You've Never Heard Of by Charles Givre
Apache Drill and Zeppelin: Two Promising Tools You've Never Heard OfApache Drill and Zeppelin: Two Promising Tools You've Never Heard Of
Apache Drill and Zeppelin: Two Promising Tools You've Never Heard Of
Charles Givre8.1K views
Hoodie - DataEngConf 2017 by Vinoth Chandar
Hoodie - DataEngConf 2017Hoodie - DataEngConf 2017
Hoodie - DataEngConf 2017
Vinoth Chandar1.2K views
Running Apache Spark on a High-Performance Cluster Using RDMA and NVMe Flash ... by Databricks
Running Apache Spark on a High-Performance Cluster Using RDMA and NVMe Flash ...Running Apache Spark on a High-Performance Cluster Using RDMA and NVMe Flash ...
Running Apache Spark on a High-Performance Cluster Using RDMA and NVMe Flash ...
Databricks2.8K views
Hadoop Operations Powered By ... Hadoop (Hadoop Summit 2014 Amsterdam) by Adam Kawa
Hadoop Operations Powered By ... Hadoop (Hadoop Summit 2014 Amsterdam)Hadoop Operations Powered By ... Hadoop (Hadoop Summit 2014 Amsterdam)
Hadoop Operations Powered By ... Hadoop (Hadoop Summit 2014 Amsterdam)
Adam Kawa26.9K views
Keynote: Getting Serious about MySQL and Hadoop at Continuent by Continuent
Keynote: Getting Serious about MySQL and Hadoop at ContinuentKeynote: Getting Serious about MySQL and Hadoop at Continuent
Keynote: Getting Serious about MySQL and Hadoop at Continuent
Continuent1.3K views
HBaseCon 2013: How (and Why) Phoenix Puts the SQL Back into NoSQL by Cloudera, Inc.
HBaseCon 2013: How (and Why) Phoenix Puts the SQL Back into NoSQLHBaseCon 2013: How (and Why) Phoenix Puts the SQL Back into NoSQL
HBaseCon 2013: How (and Why) Phoenix Puts the SQL Back into NoSQL
Cloudera, Inc.5.3K views
Apache Drill - Why, What, How by mcsrivas
Apache Drill - Why, What, HowApache Drill - Why, What, How
Apache Drill - Why, What, How
mcsrivas7.9K views
Mercury: Hybrid Centralized and Distributed Scheduling in Large Shared Clusters by DataWorks Summit
Mercury: Hybrid Centralized and Distributed Scheduling in Large Shared ClustersMercury: Hybrid Centralized and Distributed Scheduling in Large Shared Clusters
Mercury: Hybrid Centralized and Distributed Scheduling in Large Shared Clusters
DataWorks Summit949 views

Similar to HBaseCon 2015: S2Graph - A Large-scale Graph Database with HBase

MongoDB Stich Overview by
MongoDB Stich OverviewMongoDB Stich Overview
MongoDB Stich OverviewMongoDB
430 views38 slides
[263] s2graph large-scale-graph-database-with-hbase-2 by
[263] s2graph large-scale-graph-database-with-hbase-2[263] s2graph large-scale-graph-database-with-hbase-2
[263] s2graph large-scale-graph-database-with-hbase-2NAVER D2
9.7K views51 slides
Graph Database Use Cases - StampedeCon 2015 by
Graph Database Use Cases - StampedeCon 2015Graph Database Use Cases - StampedeCon 2015
Graph Database Use Cases - StampedeCon 2015StampedeCon
833 views74 slides
Graph database Use Cases by
Graph database Use CasesGraph database Use Cases
Graph database Use CasesMax De Marzi
58.9K views74 slides
Evolving your Data Access with MongoDB Stitch - Drew Di Palma by
Evolving your Data Access with MongoDB Stitch - Drew Di PalmaEvolving your Data Access with MongoDB Stitch - Drew Di Palma
Evolving your Data Access with MongoDB Stitch - Drew Di PalmaMongoDB
252 views60 slides
Scalding big ADta by
Scalding big ADtaScalding big ADta
Scalding big ADtab0ris_1
2.2K views32 slides

Similar to HBaseCon 2015: S2Graph - A Large-scale Graph Database with HBase(20)

MongoDB Stich Overview by MongoDB
MongoDB Stich OverviewMongoDB Stich Overview
MongoDB Stich Overview
MongoDB430 views
[263] s2graph large-scale-graph-database-with-hbase-2 by NAVER D2
[263] s2graph large-scale-graph-database-with-hbase-2[263] s2graph large-scale-graph-database-with-hbase-2
[263] s2graph large-scale-graph-database-with-hbase-2
NAVER D29.7K views
Graph Database Use Cases - StampedeCon 2015 by StampedeCon
Graph Database Use Cases - StampedeCon 2015Graph Database Use Cases - StampedeCon 2015
Graph Database Use Cases - StampedeCon 2015
StampedeCon833 views
Graph database Use Cases by Max De Marzi
Graph database Use CasesGraph database Use Cases
Graph database Use Cases
Max De Marzi58.9K views
Evolving your Data Access with MongoDB Stitch - Drew Di Palma by MongoDB
Evolving your Data Access with MongoDB Stitch - Drew Di PalmaEvolving your Data Access with MongoDB Stitch - Drew Di Palma
Evolving your Data Access with MongoDB Stitch - Drew Di Palma
MongoDB252 views
Scalding big ADta by b0ris_1
Scalding big ADtaScalding big ADta
Scalding big ADta
b0ris_12.2K views
Composable Data Processing with Apache Spark by Databricks
Composable Data Processing with Apache SparkComposable Data Processing with Apache Spark
Composable Data Processing with Apache Spark
Databricks538 views
Introducing MongoDB Stitch, Backend-as-a-Service from MongoDB by MongoDB
Introducing MongoDB Stitch, Backend-as-a-Service from MongoDBIntroducing MongoDB Stitch, Backend-as-a-Service from MongoDB
Introducing MongoDB Stitch, Backend-as-a-Service from MongoDB
MongoDB2.5K views
Java/Scala Lab: Борис Трофимов - Обжигающая Big Data. by GeeksLab Odessa
Java/Scala Lab: Борис Трофимов - Обжигающая Big Data.Java/Scala Lab: Борис Трофимов - Обжигающая Big Data.
Java/Scala Lab: Борис Трофимов - Обжигающая Big Data.
GeeksLab Odessa413 views
Eagle6 mongo dc revised by MongoDB
Eagle6 mongo dc revisedEagle6 mongo dc revised
Eagle6 mongo dc revised
MongoDB661 views
Eagle6 Enterprise Situational Awareness by MongoDB
Eagle6 Enterprise Situational AwarenessEagle6 Enterprise Situational Awareness
Eagle6 Enterprise Situational Awareness
MongoDB736 views
MongoDB Stitch Introduction by MongoDB
MongoDB Stitch IntroductionMongoDB Stitch Introduction
MongoDB Stitch Introduction
MongoDB976 views
RedisConf18 - Redis Memory Optimization by Redis Labs
RedisConf18 - Redis Memory OptimizationRedisConf18 - Redis Memory Optimization
RedisConf18 - Redis Memory Optimization
Redis Labs3.3K views
Swift distributed tracing method and tools v2 by zhang hua
Swift distributed tracing method and tools v2Swift distributed tracing method and tools v2
Swift distributed tracing method and tools v2
zhang hua1.2K views
Perchè potresti aver bisogno di un database NoSQL anche se non sei Google o F... by Codemotion
Perchè potresti aver bisogno di un database NoSQL anche se non sei Google o F...Perchè potresti aver bisogno di un database NoSQL anche se non sei Google o F...
Perchè potresti aver bisogno di un database NoSQL anche se non sei Google o F...
Codemotion1.5K views
Reactive Data Centric Architectures with DDS by Angelo Corsaro
Reactive Data Centric Architectures with DDSReactive Data Centric Architectures with DDS
Reactive Data Centric Architectures with DDS
Angelo Corsaro1.8K views
Socialite, the Open Source Status Feed by MongoDB
Socialite, the Open Source Status FeedSocialite, the Open Source Status Feed
Socialite, the Open Source Status Feed
MongoDB3.4K views
viWave Study Group - Introduction to Google Android Development - Chapter 23 ... by Ted Chien
viWave Study Group - Introduction to Google Android Development - Chapter 23 ...viWave Study Group - Introduction to Google Android Development - Chapter 23 ...
viWave Study Group - Introduction to Google Android Development - Chapter 23 ...
Ted Chien856 views
Best Practices for Building and Deploying Data Pipelines in Apache Spark by Databricks
Best Practices for Building and Deploying Data Pipelines in Apache SparkBest Practices for Building and Deploying Data Pipelines in Apache Spark
Best Practices for Building and Deploying Data Pipelines in Apache Spark
Databricks1.9K views

More from HBaseCon

hbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes by
hbaseconasia2017: Building online HBase cluster of Zhihu based on Kuberneteshbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
hbaseconasia2017: Building online HBase cluster of Zhihu based on KubernetesHBaseCon
3.9K views36 slides
hbaseconasia2017: HBase on Beam by
hbaseconasia2017: HBase on Beamhbaseconasia2017: HBase on Beam
hbaseconasia2017: HBase on BeamHBaseCon
1.3K views26 slides
hbaseconasia2017: HBase Disaster Recovery Solution at Huawei by
hbaseconasia2017: HBase Disaster Recovery Solution at Huaweihbaseconasia2017: HBase Disaster Recovery Solution at Huawei
hbaseconasia2017: HBase Disaster Recovery Solution at HuaweiHBaseCon
1.4K views21 slides
hbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinterest by
hbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinteresthbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinterest
hbaseconasia2017: Removable singularity: a story of HBase upgrade in PinterestHBaseCon
936 views42 slides
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程 by
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程HBaseCon
1.1K views21 slides
hbaseconasia2017: Apache HBase at Netease by
hbaseconasia2017: Apache HBase at Neteasehbaseconasia2017: Apache HBase at Netease
hbaseconasia2017: Apache HBase at NeteaseHBaseCon
754 views27 slides

More from HBaseCon(20)

hbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes by HBaseCon
hbaseconasia2017: Building online HBase cluster of Zhihu based on Kuberneteshbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
hbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
HBaseCon3.9K views
hbaseconasia2017: HBase on Beam by HBaseCon
hbaseconasia2017: HBase on Beamhbaseconasia2017: HBase on Beam
hbaseconasia2017: HBase on Beam
HBaseCon1.3K views
hbaseconasia2017: HBase Disaster Recovery Solution at Huawei by HBaseCon
hbaseconasia2017: HBase Disaster Recovery Solution at Huaweihbaseconasia2017: HBase Disaster Recovery Solution at Huawei
hbaseconasia2017: HBase Disaster Recovery Solution at Huawei
HBaseCon1.4K views
hbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinterest by HBaseCon
hbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinteresthbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinterest
hbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinterest
HBaseCon936 views
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程 by HBaseCon
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
HBaseCon1.1K views
hbaseconasia2017: Apache HBase at Netease by HBaseCon
hbaseconasia2017: Apache HBase at Neteasehbaseconasia2017: Apache HBase at Netease
hbaseconasia2017: Apache HBase at Netease
HBaseCon754 views
hbaseconasia2017: HBase在Hulu的使用和实践 by HBaseCon
hbaseconasia2017: HBase在Hulu的使用和实践hbaseconasia2017: HBase在Hulu的使用和实践
hbaseconasia2017: HBase在Hulu的使用和实践
HBaseCon878 views
hbaseconasia2017: 基于HBase的企业级大数据平台 by HBaseCon
hbaseconasia2017: 基于HBase的企业级大数据平台hbaseconasia2017: 基于HBase的企业级大数据平台
hbaseconasia2017: 基于HBase的企业级大数据平台
HBaseCon701 views
hbaseconasia2017: HBase at JD.com by HBaseCon
hbaseconasia2017: HBase at JD.comhbaseconasia2017: HBase at JD.com
hbaseconasia2017: HBase at JD.com
HBaseCon828 views
hbaseconasia2017: Large scale data near-line loading method and architecture by HBaseCon
hbaseconasia2017: Large scale data near-line loading method and architecturehbaseconasia2017: Large scale data near-line loading method and architecture
hbaseconasia2017: Large scale data near-line loading method and architecture
HBaseCon598 views
hbaseconasia2017: Ecosystems with HBase and CloudTable service at Huawei by HBaseCon
hbaseconasia2017: Ecosystems with HBase and CloudTable service at Huaweihbaseconasia2017: Ecosystems with HBase and CloudTable service at Huawei
hbaseconasia2017: Ecosystems with HBase and CloudTable service at Huawei
HBaseCon683 views
hbaseconasia2017: HBase Practice At XiaoMi by HBaseCon
hbaseconasia2017: HBase Practice At XiaoMihbaseconasia2017: HBase Practice At XiaoMi
hbaseconasia2017: HBase Practice At XiaoMi
HBaseCon1.8K views
hbaseconasia2017: hbase-2.0.0 by HBaseCon
hbaseconasia2017: hbase-2.0.0hbaseconasia2017: hbase-2.0.0
hbaseconasia2017: hbase-2.0.0
HBaseCon1.8K views
HBaseCon2017 Democratizing HBase by HBaseCon
HBaseCon2017 Democratizing HBaseHBaseCon2017 Democratizing HBase
HBaseCon2017 Democratizing HBase
HBaseCon897 views
HBaseCon2017 Removable singularity: a story of HBase upgrade in Pinterest by HBaseCon
HBaseCon2017 Removable singularity: a story of HBase upgrade in PinterestHBaseCon2017 Removable singularity: a story of HBase upgrade in Pinterest
HBaseCon2017 Removable singularity: a story of HBase upgrade in Pinterest
HBaseCon646 views
HBaseCon2017 Quanta: Quora's hierarchical counting system on HBase by HBaseCon
HBaseCon2017 Quanta: Quora's hierarchical counting system on HBaseHBaseCon2017 Quanta: Quora's hierarchical counting system on HBase
HBaseCon2017 Quanta: Quora's hierarchical counting system on HBase
HBaseCon608 views
HBaseCon2017 Transactions in HBase by HBaseCon
HBaseCon2017 Transactions in HBaseHBaseCon2017 Transactions in HBase
HBaseCon2017 Transactions in HBase
HBaseCon1.8K views
HBaseCon2017 Highly-Available HBase by HBaseCon
HBaseCon2017 Highly-Available HBaseHBaseCon2017 Highly-Available HBase
HBaseCon2017 Highly-Available HBase
HBaseCon1.1K views
HBaseCon2017 Apache HBase at Didi by HBaseCon
HBaseCon2017 Apache HBase at DidiHBaseCon2017 Apache HBase at Didi
HBaseCon2017 Apache HBase at Didi
HBaseCon996 views
HBaseCon2017 gohbase: Pure Go HBase Client by HBaseCon
HBaseCon2017 gohbase: Pure Go HBase ClientHBaseCon2017 gohbase: Pure Go HBase Client
HBaseCon2017 gohbase: Pure Go HBase Client
HBaseCon1.7K views

Recently uploaded

Playwright Retries by
Playwright RetriesPlaywright Retries
Playwright Retriesartembondar5
5 views1 slide
Understanding HTML terminology by
Understanding HTML terminologyUnderstanding HTML terminology
Understanding HTML terminologyartembondar5
7 views8 slides
Quality Assurance by
Quality Assurance Quality Assurance
Quality Assurance interworksoftware2
5 views6 slides
FIMA 2023 Neo4j & FS - Entity Resolution.pptx by
FIMA 2023 Neo4j & FS - Entity Resolution.pptxFIMA 2023 Neo4j & FS - Entity Resolution.pptx
FIMA 2023 Neo4j & FS - Entity Resolution.pptxNeo4j
17 views26 slides
Flask-Python.pptx by
Flask-Python.pptxFlask-Python.pptx
Flask-Python.pptxTriloki Gupta
9 views12 slides
tecnologia18.docx by
tecnologia18.docxtecnologia18.docx
tecnologia18.docxnosi6702
5 views5 slides

Recently uploaded(20)

Understanding HTML terminology by artembondar5
Understanding HTML terminologyUnderstanding HTML terminology
Understanding HTML terminology
artembondar57 views
FIMA 2023 Neo4j & FS - Entity Resolution.pptx by Neo4j
FIMA 2023 Neo4j & FS - Entity Resolution.pptxFIMA 2023 Neo4j & FS - Entity Resolution.pptx
FIMA 2023 Neo4j & FS - Entity Resolution.pptx
Neo4j17 views
tecnologia18.docx by nosi6702
tecnologia18.docxtecnologia18.docx
tecnologia18.docx
nosi67025 views
Gen Apps on Google Cloud PaLM2 and Codey APIs in Action by Márton Kodok
Gen Apps on Google Cloud PaLM2 and Codey APIs in ActionGen Apps on Google Cloud PaLM2 and Codey APIs in Action
Gen Apps on Google Cloud PaLM2 and Codey APIs in Action
Márton Kodok16 views
AI and Ml presentation .pptx by FayazAli87
AI and Ml presentation .pptxAI and Ml presentation .pptx
AI and Ml presentation .pptx
FayazAli8714 views
Ports-and-Adapters Architecture for Embedded HMI by Burkhard Stubert
Ports-and-Adapters Architecture for Embedded HMIPorts-and-Adapters Architecture for Embedded HMI
Ports-and-Adapters Architecture for Embedded HMI
Burkhard Stubert29 views
Introduction to Git Source Control by John Valentino
Introduction to Git Source ControlIntroduction to Git Source Control
Introduction to Git Source Control
John Valentino7 views
JioEngage_Presentation.pptx by admin125455
JioEngage_Presentation.pptxJioEngage_Presentation.pptx
JioEngage_Presentation.pptx
admin1254558 views

HBaseCon 2015: S2Graph - A Large-scale Graph Database with HBase

  • 1. S2Graph : A large-scale graph database with Hbase daumkakao Doyoung Yoon x Taejin Chin
  • 2. 2 DaumKakao A Mobile Lifestyle Platform 1. KakaoTalk a. Mobile Messenger replacing SMS b. ‘KaTalkHe’ is being used as a verb in Korea like ‘Googling’ c. 96% of Korean smartphone users are using KakaoTalk d. 170M users worldwide e. 3B messages / day
  • 3. 3 KakaoTalk Social Platform DaumKakao A Mobile Lifestyle Platform KakaoStory KakaoGroup Daum Cafe Contents Platform KakaoTopic KakaoPage KakaoGame Commerce Platform KakaoPick KakaoMusic Marketing Platform Yellow IDMedia Daum Daum tvPot Local Platform Daum Map Daum Webtoon Personal Platform Sol calendarKakaoPlace Sol Mail Zap Plus FriendGift Shop Digital Item Store KakaoStyle KakaoHome Sol Group Story Plus Daum Cluod Biggest mobile SNS in Korea 96% of Korean smartphone users are using KakaoTalk messenger, 170 million users worldwide)
  • 4. 4 Our Social Graph Message length : 9 Write length : 3 Read impact :3 Coupon price : 10 Present price : 3 affinity 6affinity: 9 affinity 3 affinity 3 affinity 4 affinity 1 affinity 2 affinity 2 affinity 9 Friend Group size : 6 Emoticon count : 7 Eat rating : 4 View count : 8 Play level: 6 Pick withFriend : 3 Advertise ctr : 0.32 Search keyword : “HBase" Listen count : 6 Like count : 7 Comment length : 15 affinity 3
  • 5. 5 Our Social Graph Message length : 9 Write length : 3 affinity 6affinity: 9 affinity 3 affinity 3 affinity 4 affinity 1 affinity 2 affinity 2 affinity 9 Friend Play level: 6 Pick withFriend : 3 Advertise ctr : 0.32 Search keyword : “HBase" Listen count : 6 C l affinity 3 Message ID : 201 Ad ID : 603 Music ID Item ID : 13 Post ID : 97 Game ID : 1984
  • 6. 6 Technical Challenges 1. Large social graph constantly changing a. Scale more than, social network: 10 billion edges, 200 million vertices, 50 million update on existing edges. user activities: 400 million new edges per day
  • 7. 7 Technical Challenges (cont) 2. Low latency for breadth first search traversal on connected data. a. performance requirement peak graph-traversing query per second: 20000 response time: 100ms
  • 8. 8 Technical Challenges (cont) 3. Update should be applied to graph in real time for viral effect Person A Post Fast Person B Comment Person C Sharing Person D Mention Fast Fast
  • 9. 9 Technical Challenges (cont) 4. Support for Dynamic Ranking logic a. push strategy: hard to change data ranking logic dynamically. b. pull strategy: can try various data ranking logic
  • 10. 10 Before Each app server should know each DB’s sharding logic. Highly inter-connected architecture Friend relationship SNS feeds Blog user activities Messaging Messaging App SNS App Blog App
  • 12. 12 S2Graph : Distributed Online GraphDB 1. Low-latency 2.Graph-traversable 3.Scalable 4.Eventually consistent 5.Asynchronous, non-blocking
  • 13. 13 Why We Choose HBase? 1. High Availability 2.Scalability 3.Low latency 4.High concurrency 5.Fault tolerant 6.Integration with HDFS 7.Distributed operation
  • 14. 14 The Data Model 1. Columns 2. Labels 3. Directions 4. Index Properties 5. Non-index Properties 1 3comment 4 know created name = “josh”
 age = 32 edge 2 source vertex vertex 1 out edges edge 2 target vertex 2 edge 2 label vertex 2 in edges vertex 4 id vertex 4 properties date = 20150507 edge 5 properties 5
  • 15. 15 How to store the data - Edge Logical View 1. Snapshot edges : Up-to-date status of edge column row Tgt Vertex ID1 Tgt Vertex ID2 Tgt Vertex ID3 Src Vertex ID1 Properties Properties Properties Src Vertex ID2 Properties Properties Properties a. Fetching an edge between two specific vertex b. Lookup Table to reach indexed edges for update, increment, delete operations
  • 16. 16 How to store the data - Edge Logical View 2. Indexed edges : Edges with index column row Index Values | Tgt Vertex ID1 Index Values | Tgt Vertex ID2 Src Vertex ID1 Non-index Properties Non-index Properties a. Fetches edges originating from a certain vertex in order of index
  • 17. 17 How to store the data - Edge Physical View - table schema 1. Snapshot Edge a. Rowkey Murmur Hash Src Vertex ID Label ID Direction Index Sequence Is Inverted 16 bit variable length 30 bit 2 bit 7bit 1 bit Vertex IDs can be encoded with 8 bit header + byte array (long, integer, short, byte, string)
  • 18. 18 How to store the data - Edge Physical View - table schema 1. Snapshot Edge b. Qualifier Target Vertex ID variable length c. Value All Property Key Value Pairs variable length
  • 19. 19 How to store the data - Edge Physical View - table schema 2. Indexed Edge a. Rowkey Murmur Hash Src Vertex ID Label ID Direction Index Sequence Is Inverted 16 bit variable length 30 bit 2 bit 7bit 1 bit Vertex IDs can be encoded with 8 bit header + byte array (long, integer, short, byte, string)
  • 20. 20 How to store the data - Edge Physical View - table schema 2. Indexed Edge b. Qualifier Index Property Values Tgt Vertex ID variable length variable length c. Value Non-index Property Key Value Pairs variable length
  • 21. 21 How to store the data - Vertex Logical View 1. Vertex : Up-to-date status of Vertex column row Property Key1 Property Key2 Src Vertex ID1 Value1 Value2 Vertex ID2 Value1 Value2
  • 22. 22 How to store the data - Vertex Physical View - table schema 1. Vertex : Up-to-date status of Vertex a. Rowkey Murmur Hash Column ID Vertex ID 16 bit integer(32bit) variable length b. Qualifier Property Key Byte(8 bit) c. Value Property Value variable length
  • 23. 23 How to read the data - GetEdges Using a custom query DSL on top of HTTP curl -XPOST localhost:9000/graphs/getEdges -H 'Content-Type: Application/json' -d ' { "srcVertices": [{"serviceName": "s2graph", "columnName": "account_id", "id":1}], "steps": [ [{"label": "friends", "direction": "out", "limit": 100}], // step [{"label": "hear", "direction": "out", "limit": 10}] ] } ' Steps = a list of Step Step = contains the labels to traverse and how to rank them in the result Step 1 friend 1 hear time: 20140502 hear time: 20140712 hear time: 20141116 Friends Friends friend 2 User 1 Step 2 Don’t let go let it be let it go
  • 24. 24 How to read the data - GetEdges Example Friend list curl -XPOST localhost:9000/graphs/getEdges -H 'Content-Type: Application/json' -d ' { "srcVertices": [{"serviceName": "s2graph", "columnName": "account_id", "id":1}], "steps": [ [{"label": "friends", "direction": "out", "limit": 100}], // step ] } ' friend 1 friend 2 User 1 Friends Friends
  • 25. 25 How to read the data - GetEdges Example Songs my friends have listened curl -XPOST localhost:9000/graphs/getEdges -H 'Content-Type: Application/json' -d ' { "srcVertices": [{"serviceName": "s2graph", "columnName": "account_id", "id":1}], "steps": [ [{"label": "friends", "direction": "out", "limit": 50, “scoring”: {“score”: 1.0}], [{"label": "listen", "direction": "out", "limit": 10}] ] } ' friend 1 Friends Friends friend 2 Don’t let go let it be let it go hear time: 20140502 hear time: 20140712 hear time: 20141116 User 1 Reference : https://github.com/daumkakao/s2graph#1-definition
  • 26. 26 How to read the data - GetEdges Example Similar songs to songs that I have listened to. curl -XPOST localhost:9000/graphs/getEdges -H 'Content-Type: Application/json' -d ' { "srcVertices": [{"serviceName": "s2graph", "columnName": "account_id", "id":1}], "steps": [ [{"label": "listen", "direction": "out", "limit": 50}], [{"label": "similar_song", "direction": "out", "limit": 10, “scoring”: {“score”: 1.0}] ] } User 1 Don’t let go let it be let it go hear time: 20140502 hear time: 20140712 hear time: 20141116 let it bleed Hey jude Do you wanna build a snowman? similar_song similarity: 0.3 similar_song similarity: 0.4 similar_song similarity: 0.6
  • 27. 27 How to read the data - GetVertices curl -XPOST localhost:9000/graphs/getVertices -H 'Content-Type: Application/json' -d ' [ {"serviceName": "s2graph", "columnName": "account_id", "ids": [1, 2, 3]}, {"serviceName": "kakaomusic", "columnName": "user_id", "ids": [1, 2, 3]} ] ' User 1 {created_at:20070812, updated_at:20150507} User 2 {created_at:201206132, updated_at:20140505}
  • 28. 28 How to write the data - Insert curl -XPOST localhost:9000/graphs/edges/insert -H 'Content-Type: Application/json' -d ' [ {"from":1,"to":2,"label":"graph_test","props":{"time":-1, "weight":10},"timestamp":1417616431}, ] ' User 1 User 2
  • 29. 29 How to write the data - Delete curl -XPOST localhost:9000/graphs/edges/delete -H 'Content-Type: Application/json' -d ' [ {"from":1,"to":2,"label":"graph_test","timestamp":1417616431}, {"from":1,"to":3,"label":"graph_test","timestamp":1417616431}, ] ' User 1 User 2
  • 30. 30 How to write the data - Update curl -XPOST localhost:9000/graphs/edges/update -H 'Content-Type: Application/json' -d ' [ {"from":1,"to":2,"label":"graph_test","timestamp":1417616431, "props": {"is_hidden": true, “status”: 200}, {"from":1,"to":3,"label":"graph_test","timestamp":1417616431, "props": {"status": -500} ] User 1 User 2 friend {is_hidden:true, status:200}
  • 31. 31 REST API Spec. Read 1. getEdges 2. checkEdge 3. getEdgesCount 4. getVertices Write 1. insert 2. delete 3. update 4. increment Management 1. create service (vertex type) 2. create label (edge type) 3. add Index
  • 32. 32 HBase Table Configuration 1. setDurability(Durability.ASYNC_WAL) 2. setCompressionType(Compression.Algorithm.LZ4) 3. setBloomFilterType(BloomType.Row) 4. setDataBlockEncoding(DataBlockEncoding.FAST_DIFF) 5. setBlockSize(32768) 6. setBlockCacheEnabled(true) 7. pre-split by (Intger.MaxValue / regionCount). regionCount = 120 when create table(on 20 region server).
  • 33. 33 HBase Cluster Configuration • each machine: 8core, 32G memory • hfile.block.cache.size: 0.6 • hbase.hregion.memstore.flush.size: 128MB • otherwise use default value from CDH 5.3.1
  • 35. 35 Compare to other Online GraphDBs Titan (v0.4.2) a. Pros
 - Rich API and easy to setup
 - Relatively large community
 - Transaction handling b. Cons
 - Using it’s own ID system; less efficient for graph traversal (details in next slide)
 - Index data stored on one region (hotspot) with strong consistency option
 - Not many references on Titan with HBase comparing to other storages
  • 36. 36 Compare to Titan Titan is less efficient for graph traversal - For following 1 normal graph traversal query, Vertex(“userID:1”).out(“friends”).limit(10).out(“friends”).limit(10) User 1 friends friends
  • 37. 37 Compare to Titan (cont) Vertex(“userID:1”).out(“friend”).limit(10).out(“friend”).limit(10) Titan S2graph # of read requests 
 on HBase 112 = 1 (Vertex Lookup : a) + 1 (1st step edges : b) + 10 (2nd step edges : c) + 100 (Destination Vertices : d) 11 = 1 (1step edges : e) + 10 (2nd step edges : f) Titan S2graph B A C D e f
  • 38. 38 Performance 1. Test data a. Total # of Edges: 9,000,000,000 b. Average # of adjacent edges per vertex: 500 c. Seed vertex: vertices that has more than 100 adjacent edges. 2. Test environment a. Zookeeper server: 3 b. HBase Masterserver: 2 c. HBase Regionserver: 20 d. App server: 8 core, 16GB Ram
  • 39. 39 - Benchmark Query : src.out(“friend”).limit(50).out(“friend”).limit(10) - Total concurrency: 20 * # of app server Performance 2. Linear scalability Latency 0 50 100 150 200 QPS 0 1,000 2,000 3,000 4,000 # of app server 1 2 4 8 QPS(Query Per Second) Latency(ms) 51515047 3,097 1,567 803 42147 50 51 51 # of app server 1 2 3 4 5 6 7 8 50010001500200025003000 QPS
  • 40. 40 Performance 3. Varying width of traverse (tested with a single server) Latency 0 75 150 225 300 QPS 0 500 1,000 1,500 2,000 Limit on first step 20 40 80 100 QPS Latency(ms) 97 75 43 23 203266 457 943 23 43 75 97 - Benchmark Query : src.out(“friend”).limit(x).out(“friend”).limit(10) - Total concurrency = 20 * 1(# of app server)
  • 41. Performance 3. Varying width of traverse (tested with a single server) Latency 0 75 150 225 300 QPS 0 500 1,000 1,500 2,000 Limit on first step 20 40 80 100 QPS Latency(ms) 97 75 43 23 203266 457 943 23 43 75 97 - Benchmark Query : src.out(“friend”).limit(x).out(“friend”).limit(10) - Total concurrency = 20 * 1(# of app server)
  • 42. 42 - All query touch 1000 edges. - each step` limit is on x axis. - Can expect performance with given query`s search space. Performance 4. Different query path(different I/O pattern) Latency 0 37.5 75 112.5 150 QPS 0 80 160 240 320 400 limits on path 10 -> 100 100 -> 10 10 -> 10 -> 10 2 -> 5 -> 10 -> 10 2 -> 5 -> 2 -> 5 -> 10 QPS Latency(ms) 5667716867 352.2 298.1280272.5297 67 68 71 67 56
  • 43. 43 Performance 5. Write throughput per operation on single app server Insert operation Latency 0 1.25 2.5 3.75 5 Request per second 8000 16000 800000
  • 44. 44 Performance 6. write throughput per operation on single app server Update(increment/update/delete) operation Latency 0 2 4 6 8 Request per second 2000 4000 6000
  • 45. 45 Stats 1. HBase cluster per IDC (2 IDC) - 3 Zookeeper Server - 2 HBase Master - 20 HBase Slave 2. App server per IDC - 10 server for write-only - 20 server for query only 3. Real traffic - read: over 10K request per second - now mostly 2 step queries with limit 100 on first step. - write: over 5k request per second * Deep traversal queries are not counted since it is in test stage for production
  • 46. 46
  • 48. 48 Now Available As an Open Source - https://github.com/daumkakao/s2graph - Finding a mentor Contact - Taejin Chin : taejin.chin@gmail.com - Doyoung Yoon : shom83@gmail.com
  • 49. 49 Latency 0 50 100 150 200 QPS 0 500 1,000 1,500 2,000 # of app server 1 2 3 4 5 Native Client QPS Native Client Latency(ms) 174186189 177178 570 429315224112 178 177 189 186 174 - Benchmark Query : src.out(“friend”).limit(50).out(“friend”).limit(10) - Test seed edges have adjacent edges more than 100: 30millions - Total concurrency: 20 * # of app server Appendix Latency 0 50 100 150 200 QPS 0 500 1,000 1,500 2,000 # of app server 1 2 3 4 5 Asnychbase QPS Asynchbase Latency(ms) 5351505047 1,895 1,567 1,192 803 421 47 50 50 51 53 3.5x performance improvement using Asynchbase