HBaseCon 2017: Community-Driven Graph with JanusGraph (updated)

1. Note: This is a updated version of the original presentation at HBaseCon West 2017.
Jing Chen He • jinghe@us.ibm.com • Apache HBase PMC • JanusGraph TSC
Jason Plurad • pluradj@us.ibm.com • Apache TinkerPop PMC • JanusGraph TSC
HBaseCon West 2017 • June 12, 2017
Community-Driven Graphs with
JanusGraph

2. Agenda
Property Graphs
Graph Framework and Graph DB
Graph Community
Introduction to JanusGraph
JanusGraph with HBase
2 #HBaseCon

3. Graph
 Born for relationship!
 Intuitive modeling
 Expressive querying
 Native analysis
 Property Graph
3 #HBaseCon
https://tinkerpop.apache.org/docs/3.2.4/reference/#intro

4. Graph Use Cases
 Social network analysis
 Recommendation engines
 Knowledge graphs
 Internet of things
 Fraud detection system
 Meta data and Master data management
4 #HBaseCon

5. Graph Computing and Graph Database
 Graph Computing and Processing framework
 OLTP - Local traversal, real time.
 OLAP - Entire data set accessed, long running, parallel and batch processing.
 Apache TinkerPop - OLTP and OLAP (TinkerPop SparkGraphComputer)
 Apache Spark GraphX - OLAP
 Graph Database
 Graph Computing and Processing + Data Storage
 Example  Neo4J - Integrated graph processing and storage in one.
 Example  JanusGraph - Graph processing layer on top of external storage.
5 #HBaseCon

6. Apache TinkerPop
 Open source, vendor-agnostic,
graph computing framework
 Gremlin graph traversal language
6
Apache TinkerPop™
Maintainer Apache
Software
Foundation
License Apache
Latest Release 3.2.4
February 2017
https://tinkerpop.apache.org
#HBaseCon

7. Gremlin Graph Traversal Language
7 #HBaseCon
https://tinkerpop.apache.org/gremlin.html

8. TinkerPop Stack
8 #HBaseCon
https://tinkerpop.apache.org/docs/3.2.4/reference/#_graph_system_integration

9. Graph Landscape
9 #HBaseCon
https://tinkerpop.apache.org/gremlin.html#oltp-and-olap-traversals

10. Graph Landscape
10 #HBaseCon
https://tinkerpop.apache.org/gremlin.html#oltp-and-olap-traversals

11.  Scalable graph database distributed on
multi-machine clusters with pluggable storage
and indexing
 Fully-compliant with Apache TinkerPop graph
computing framework
 Vendor-neutral, open community with
open governance
– Founding members: Expero, Google, GRAKN.AI,
Hortonworks, IBM
– Latest members: Amazon, Netflix, Orchestral
Developments, Uber
11
JanusGraph™
Maintainer Linux
Foundation
License Apache
Latest
Release
0.2.0
Oct, 2017
https://janusgraph.org
#HBaseCon

12. 12 #HBaseCon
Architecture
Google Cloud Bigtable
http://docs.janusgraph.org/latest/arch-overview.html

13. 13 #HBaseCon
Storage Model
http://docs.janusgraph.org/latest/data-model.html#_janusgraph_data_layout

14. 14 #HBaseCon
Storage Model
http://docs.janusgraph.org/latest/data-model.html#_individual_edge_layout

15. 15 #HBaseCon
with HBase
 HBase – Perfect Storage Backend for JanusGraph
Big enough for your biggest graph!
The storage model
Read and write speed
Scalability and partitioning
Strong consistency
Tight integration with Hadoop Ecosystem
Great open community!
http://docs.janusgraph.org/latest/hbase.html

16. 16 #HBaseCon
with HBase
 HBase – Perfect Storage Backend for JanusGraph
Simple configuration!
 conf/janusgraph-hbase-solr.properties
 storage.backend=hbase
 storage.hostname=zookeeper-host1,zookeeper-host2,zookeeper-host3
 storage.hbase.table=janusgraph
 storage.hbase.ext.zookeeper.znode.parent=/hbase
 storage.hbase.ext.hbase.zookeeper.property.clientPort=2181
 Just open your graph!
 graph=JanusGraphFactory.open('conf/janusgraph-hbase-solr.properties')
Optional
Optional

17. 17 #HBaseCon
with HBase
 HBase – Perfect Storage Backend for JanusGraph
Throw in an Index Backend for better performance
 conf/janusgraph-hbase-solr.properties
 index.search.backend=solr
 index.search.solr.mode=cloud
 index.search.solr.zookeeper-url=zookeeper-host1:2181/solr,zookeeper-
host2:2181/solr,zookeeper-host3:2181/solr
 index.search.solr.configset=janusgraph

18. 18 #HBaseCon
with HBase
 HBase – Perfect Storage Backend for JanusGraph
 Look into more details
 Stores to Column Families
 Edge store  e
 Index store  g
 ID store  i
 System Transaction log store  l
 System Management log store  m
 System property store  s
 …
 CF attributes can be set. E.g. compression, TTL.

19. 19 #HBaseCon
with HBase
 HBase – Perfect Storage Backend for JanusGraph
 Look into more details
g.V().has("name", "Alice").out("knows").out("knows").values("name")
Execution Plan to
Backend Store and
Index
Edge Store
Index Store
Index
provider
(ES or
Solr)
Gremlin TraversalStrategy
Optimization
JanusGraph Optimization

20. 20 #HBaseCon
with HBase
 HBase – Perfect Storage Backend for JanusGraph
Look into more details
 A store (column family) is always specified.
 Get or Multi Get
 Batch to mutate
 Key range scan
 ColumnRangeFilter
 ColumnPaginationFilter
 HBase tuning
Edge Store
Index Store

21. 21 #HBaseCon
with Google Cloud Bigtable
 Bigtable implements the HBase 1.0 client API
Need the latest version of the bigtable-hbase-1.x artifact.
 storage.backend=hbase
 storage.hbase.ext.hbase.client.connection.impl=
com.google.cloud.bigtable.hbase1_x.BigtableConnection
 storage.hbase.ext.google.bigtable.project.id=
<Google Cloud Platform project id>
 storage.hbase.ext.google.bigtable.instance.id=<Bigtable instance id>

22. Thank you!