HBaseCon2017 Community-Driven Graphs with JanusGraph

HBaseCon
Jing Chen He • jinghe@us.ibm.com • Apache HBase PMC • JanusGraph TSC
Jason Plurad • pluradj@us.ibm.com • Apache TinkerPop PMC • JanusGraph TSC
HBaseCon West 2017 • June 12, 2017
Community-Driven Graphs with
JanusGraph
Agenda
Property Graphs
Graph Community
Introduction to JanusGraph
JanusGraph with HBase
2 #HBaseCon
Graph
 Born for relationship!
 Intuitive modeling
 Expressive querying
 Native analysis
3 #HBaseCon
https://tinkerpop.apache.org/docs/3.2.4/reference/#intro
Graph Data Use Cases
 Social network analysis
 Configuration management database
 Master data management
 Recommendation engines
 Knowledge graphs
 Internet of things
 Cybersecurity attack analysis
4 #HBaseCon
Apache TinkerPop
 Open source, vendor-agnostic,
graph computing framework
 Gremlin graph traversal language
5
Apache TinkerPop™
Maintainer Apache
Software
Foundation
License Apache
Latest Release 3.2.4
February 2017
https://tinkerpop.apache.org
#HBaseCon
Gremlin Graph Traversal Language
6 #HBaseCon
https://tinkerpop.apache.org/gremlin.html
TinkerPop Stack
7 #HBaseCon
https://tinkerpop.apache.org/docs/3.2.4/reference/#_graph_system_integration
Graph Landscape
8 #HBaseCon
https://tinkerpop.apache.org/gremlin.html#oltp-and-olap-traversals
 Scalable graph database distributed on
multi-machine clusters with pluggable storage
and indexing
 Fully-compliant with Apache TinkerPop graph
computing framework
 Vendor-neutral, open community with
open governance
– Founding members: Expero, Google, GRAKN.AI,
Hortonworks, IBM
– Latest members: Amazon, Netflix, Orchestral
Developments, Uber
9
JanusGraph™
Maintainer Linux
Foundation
License Apache
Latest
Release
0.1.0
April 2017
https://janusgraph.org
#HBaseCon
10 #HBaseCon
Architecture
Google Cloud Bigtable
http://docs.janusgraph.org/latest/arch-overview.html
11 #HBaseCon
Storage Model
http://docs.janusgraph.org/latest/data-model.html#_janusgraph_data_layout
12 #HBaseCon
Storage Model
http://docs.janusgraph.org/latest/data-model.html#_individual_edge_layout
13 #HBaseCon
with HBase
 HBase – Perfect Storage Backend for JanusGraph
Big enough for your biggest graph!
The storage model
Read and write speed
Scalability and partitioning
Strong consistency
Tight integration with Hadoop Ecosystem
Great open community!
http://docs.janusgraph.org/latest/hbase.html
14 #HBaseCon
with HBase
 HBase – Perfect Storage Backend for JanusGraph
Simple configuration!
 conf/janusgraph-hbase-solr.properties
 storage.backend=hbase
 storage.hostname=zookeeper-host1,zookeeper-host2,zookeeper-host3
 storage.hbase.table=janusgraph
 storage.hbase.ext.zookeeper.znode.parent=/hbase
 storage.hbase.ext.hbase.zookeeper.property.clientPort=2181
 Just open your graph!
 graph=JanusGraphFactory.open('conf/janusgraph-hbase-solr.properties')
Optional
Optional
15 #HBaseCon
with HBase
 HBase – Perfect Storage Backend for JanusGraph
Throw in an Index Backend for better performance
 conf/janusgraph-hbase-solr.properties
 index.search.backend=solr
 index.search.solr.mode=cloud
 index.search.solr.zookeeper-url=zookeeper-host1:2181/solr,zookeeper-
host2:2181/solr,zookeeper-host3:2181/solr
 index.search.solr.configset=janusgraph
16 #HBaseCon
with HBase
 HBase – Perfect Storage Backend for JanusGraph
Look into more details
 Stores to Column Families
 Edge store  e
 Index store  g
 ID store  i
 Transaction log store  l
 System property store  s
 CF attributes can be set. E.g. compression, TTL.
17 #HBaseCon
with HBase
 HBase – Perfect Storage Backend for JanusGraph
Look into more details
g.V().has("name", "Alice").out("knows").out("knows").values("name")
Query Plan to
Backend Store and
Index
Edge Store
Index Store
Index
provider
18 #HBaseCon
with HBase
 HBase – Perfect Storage Backend for JanusGraph
Look into more details
 A store (column family) is always specified.
 Get or Multi Get
 Batch to mutate
 Key range scan
 ColumnRangeFilter
 ColumnPaginationFilter
 HBase tuning
Edge Store
Index Store
19 #HBaseCon
with Google Cloud Bigtable
 Bigtable implements the HBase 1.0 client API
Need the latest version of the bigtable-hbase-1.0 artifact.
 storage.backend=hbase
 storage.hbase.ext.hbase.client.connection.impl=
com.google.cloud.bigtable.hbase1_0.BigtableConnection
 storage.hbase.ext.google.bigtable.project.id=
<Google Cloud Platform project id>
 storage.hbase.ext.google.bigtable.instance.id=<Bigtable instance id>
Thank you!
1 of 20

Recommended

Large Scale Graph Analytics with JanusGraph by
Large Scale Graph Analytics with JanusGraphLarge Scale Graph Analytics with JanusGraph
Large Scale Graph Analytics with JanusGraphP. Taylor Goetz
19.1K views67 slides
Hudi: Large-Scale, Near Real-Time Pipelines at Uber with Nishith Agarwal and ... by
Hudi: Large-Scale, Near Real-Time Pipelines at Uber with Nishith Agarwal and ...Hudi: Large-Scale, Near Real-Time Pipelines at Uber with Nishith Agarwal and ...
Hudi: Large-Scale, Near Real-Time Pipelines at Uber with Nishith Agarwal and ...Databricks
3.8K views28 slides
RESTful API Design, Second Edition by
RESTful API Design, Second EditionRESTful API Design, Second Edition
RESTful API Design, Second EditionApigee | Google Cloud
180.1K views106 slides
RaptorX: Building a 10X Faster Presto with hierarchical cache by
RaptorX: Building a 10X Faster Presto with hierarchical cacheRaptorX: Building a 10X Faster Presto with hierarchical cache
RaptorX: Building a 10X Faster Presto with hierarchical cacheAlluxio, Inc.
352 views15 slides
One Data Lake, Many Uses: Enabling Multi-Tenant Analytics with Amazon EMR (AN... by
One Data Lake, Many Uses: Enabling Multi-Tenant Analytics with Amazon EMR (AN...One Data Lake, Many Uses: Enabling Multi-Tenant Analytics with Amazon EMR (AN...
One Data Lake, Many Uses: Enabling Multi-Tenant Analytics with Amazon EMR (AN...Amazon Web Services
1.8K views37 slides
Hoodie - DataEngConf 2017 by
Hoodie - DataEngConf 2017Hoodie - DataEngConf 2017
Hoodie - DataEngConf 2017Vinoth Chandar
1.2K views83 slides

More Related Content

What's hot

Zeus: Uber’s Highly Scalable and Distributed Shuffle as a Service by
Zeus: Uber’s Highly Scalable and Distributed Shuffle as a ServiceZeus: Uber’s Highly Scalable and Distributed Shuffle as a Service
Zeus: Uber’s Highly Scalable and Distributed Shuffle as a ServiceDatabricks
1.7K views43 slides
Hadoop Security Architecture by
Hadoop Security ArchitectureHadoop Security Architecture
Hadoop Security ArchitectureOwen O'Malley
30.2K views30 slides
Radical Speed for SQL Queries on Databricks: Photon Under the Hood by
Radical Speed for SQL Queries on Databricks: Photon Under the HoodRadical Speed for SQL Queries on Databricks: Photon Under the Hood
Radical Speed for SQL Queries on Databricks: Photon Under the HoodDatabricks
1.1K views48 slides
Monitoring_with_Prometheus_Grafana_Tutorial by
Monitoring_with_Prometheus_Grafana_TutorialMonitoring_with_Prometheus_Grafana_Tutorial
Monitoring_with_Prometheus_Grafana_TutorialTim Vaillancourt
9.1K views62 slides
Presto by
PrestoPresto
PrestoKnoldus Inc.
5.1K views42 slides
Trino: A Ludicrously Fast Query Engine - Pulsar Summit NA 2021 by
Trino: A Ludicrously Fast Query Engine - Pulsar Summit NA 2021Trino: A Ludicrously Fast Query Engine - Pulsar Summit NA 2021
Trino: A Ludicrously Fast Query Engine - Pulsar Summit NA 2021StreamNative
536 views18 slides

What's hot(20)

Zeus: Uber’s Highly Scalable and Distributed Shuffle as a Service by Databricks
Zeus: Uber’s Highly Scalable and Distributed Shuffle as a ServiceZeus: Uber’s Highly Scalable and Distributed Shuffle as a Service
Zeus: Uber’s Highly Scalable and Distributed Shuffle as a Service
Databricks1.7K views
Hadoop Security Architecture by Owen O'Malley
Hadoop Security ArchitectureHadoop Security Architecture
Hadoop Security Architecture
Owen O'Malley30.2K views
Radical Speed for SQL Queries on Databricks: Photon Under the Hood by Databricks
Radical Speed for SQL Queries on Databricks: Photon Under the HoodRadical Speed for SQL Queries on Databricks: Photon Under the Hood
Radical Speed for SQL Queries on Databricks: Photon Under the Hood
Databricks1.1K views
Monitoring_with_Prometheus_Grafana_Tutorial by Tim Vaillancourt
Monitoring_with_Prometheus_Grafana_TutorialMonitoring_with_Prometheus_Grafana_Tutorial
Monitoring_with_Prometheus_Grafana_Tutorial
Tim Vaillancourt9.1K views
Trino: A Ludicrously Fast Query Engine - Pulsar Summit NA 2021 by StreamNative
Trino: A Ludicrously Fast Query Engine - Pulsar Summit NA 2021Trino: A Ludicrously Fast Query Engine - Pulsar Summit NA 2021
Trino: A Ludicrously Fast Query Engine - Pulsar Summit NA 2021
StreamNative536 views
Getting Started Monitoring with Prometheus and Grafana by Syah Dwi Prihatmoko
Getting Started Monitoring with Prometheus and GrafanaGetting Started Monitoring with Prometheus and Grafana
Getting Started Monitoring with Prometheus and Grafana
Syah Dwi Prihatmoko3.5K views
Designing APIs with OpenAPI Spec by Adam Paxton
Designing APIs with OpenAPI SpecDesigning APIs with OpenAPI Spec
Designing APIs with OpenAPI Spec
Adam Paxton6K views
Design your application using Persistent Graphs and OrientDB by Luca Garulli
Design your application using Persistent Graphs and OrientDBDesign your application using Persistent Graphs and OrientDB
Design your application using Persistent Graphs and OrientDB
Luca Garulli21.8K views
Apache Tez: Accelerating Hadoop Query Processing by DataWorks Summit
Apache Tez: Accelerating Hadoop Query Processing Apache Tez: Accelerating Hadoop Query Processing
Apache Tez: Accelerating Hadoop Query Processing
DataWorks Summit31.7K views
Hadoop Security Today and Tomorrow by DataWorks Summit
Hadoop Security Today and TomorrowHadoop Security Today and Tomorrow
Hadoop Security Today and Tomorrow
DataWorks Summit5.5K views
A Deep Dive into Query Execution Engine of Spark SQL by Databricks
A Deep Dive into Query Execution Engine of Spark SQLA Deep Dive into Query Execution Engine of Spark SQL
A Deep Dive into Query Execution Engine of Spark SQL
Databricks6.6K views
Golang basics for Java developers - Part 1 by Robert Stern
Golang basics for Java developers - Part 1Golang basics for Java developers - Part 1
Golang basics for Java developers - Part 1
Robert Stern1.3K views
Server monitoring using grafana and prometheus by Celine George
Server monitoring using grafana and prometheusServer monitoring using grafana and prometheus
Server monitoring using grafana and prometheus
Celine George2K views
Enabling ABAC with Accumulo and Ranger integration by DataWorks Summit
Enabling ABAC with Accumulo and Ranger integrationEnabling ABAC with Accumulo and Ranger integration
Enabling ABAC with Accumulo and Ranger integration
DataWorks Summit1.1K views
An Introduction to Prometheus (GrafanaCon 2016) by Brian Brazil
An Introduction to Prometheus (GrafanaCon 2016)An Introduction to Prometheus (GrafanaCon 2016)
An Introduction to Prometheus (GrafanaCon 2016)
Brian Brazil3.4K views
Top 5 Mistakes When Writing Spark Applications by Spark Summit
Top 5 Mistakes When Writing Spark ApplicationsTop 5 Mistakes When Writing Spark Applications
Top 5 Mistakes When Writing Spark Applications
Spark Summit26.4K views

Similar to HBaseCon2017 Community-Driven Graphs with JanusGraph

HBaseCon 2017: Community-Driven Graph with JanusGraph (updated) by
HBaseCon 2017: Community-Driven Graph with JanusGraph (updated)HBaseCon 2017: Community-Driven Graph with JanusGraph (updated)
HBaseCon 2017: Community-Driven Graph with JanusGraph (updated)Jing Chen (Jerry) He
94 views22 slides
Attack on graph by
Attack on graphAttack on graph
Attack on graphScott Miao
1.1K views40 slides
HBase, no trouble by
HBase, no troubleHBase, no trouble
HBase, no troubleLINE Corporation
1.5K views38 slides
Software architectures for the cloud by
Software architectures for the cloudSoftware architectures for the cloud
Software architectures for the cloudGeorgios Gousios
516 views45 slides
関西DB勉強会 (SAP HANA, express edition) by
関西DB勉強会 (SAP HANA, express edition)関西DB勉強会 (SAP HANA, express edition)
関西DB勉強会 (SAP HANA, express edition)Koji Shinkubo
1.7K views86 slides
Architecting applications with Hadoop - Fraud Detection by
Architecting applications with Hadoop - Fraud DetectionArchitecting applications with Hadoop - Fraud Detection
Architecting applications with Hadoop - Fraud Detectionhadooparchbook
6.5K views150 slides

Similar to HBaseCon2017 Community-Driven Graphs with JanusGraph(20)

HBaseCon 2017: Community-Driven Graph with JanusGraph (updated) by Jing Chen (Jerry) He
HBaseCon 2017: Community-Driven Graph with JanusGraph (updated)HBaseCon 2017: Community-Driven Graph with JanusGraph (updated)
HBaseCon 2017: Community-Driven Graph with JanusGraph (updated)
Attack on graph by Scott Miao
Attack on graphAttack on graph
Attack on graph
Scott Miao1.1K views
Software architectures for the cloud by Georgios Gousios
Software architectures for the cloudSoftware architectures for the cloud
Software architectures for the cloud
Georgios Gousios516 views
関西DB勉強会 (SAP HANA, express edition) by Koji Shinkubo
関西DB勉強会 (SAP HANA, express edition)関西DB勉強会 (SAP HANA, express edition)
関西DB勉強会 (SAP HANA, express edition)
Koji Shinkubo1.7K views
Architecting applications with Hadoop - Fraud Detection by hadooparchbook
Architecting applications with Hadoop - Fraud DetectionArchitecting applications with Hadoop - Fraud Detection
Architecting applications with Hadoop - Fraud Detection
hadooparchbook6.5K views
Big Data, Analytics and Machine Learning on AWS Lambda - SRV402 - re:Invent 2017 by Amazon Web Services
Big Data, Analytics and Machine Learning on AWS Lambda - SRV402 - re:Invent 2017Big Data, Analytics and Machine Learning on AWS Lambda - SRV402 - re:Invent 2017
Big Data, Analytics and Machine Learning on AWS Lambda - SRV402 - re:Invent 2017
Amazon Web Services1.2K views
Spectrum Scale - Diversified analytic solution based on various storage servi... by Wei Gong
Spectrum Scale - Diversified analytic solution based on various storage servi...Spectrum Scale - Diversified analytic solution based on various storage servi...
Spectrum Scale - Diversified analytic solution based on various storage servi...
Wei Gong673 views
How to develop Big Data Pipelines for Hadoop, by Costin Leau by Codemotion
How to develop Big Data Pipelines for Hadoop, by Costin LeauHow to develop Big Data Pipelines for Hadoop, by Costin Leau
How to develop Big Data Pipelines for Hadoop, by Costin Leau
Codemotion1.9K views
MongoDB et Hadoop by MongoDB
MongoDB et HadoopMongoDB et Hadoop
MongoDB et Hadoop
MongoDB2.3K views
Apache Tez - A unifying Framework for Hadoop Data Processing by DataWorks Summit
Apache Tez - A unifying Framework for Hadoop Data ProcessingApache Tez - A unifying Framework for Hadoop Data Processing
Apache Tez - A unifying Framework for Hadoop Data Processing
DataWorks Summit2.6K views
Big Data Day LA 2015 - What's new and next in Apache Tez by Bikas Saha of Hor... by Data Con LA
Big Data Day LA 2015 - What's new and next in Apache Tez by Bikas Saha of Hor...Big Data Day LA 2015 - What's new and next in Apache Tez by Bikas Saha of Hor...
Big Data Day LA 2015 - What's new and next in Apache Tez by Bikas Saha of Hor...
Data Con LA775 views
Apache Hadoop India Summit 2011 talk "Hadoop Map-Reduce Programming & Best Pr... by Yahoo Developer Network
Apache Hadoop India Summit 2011 talk "Hadoop Map-Reduce Programming & Best Pr...Apache Hadoop India Summit 2011 talk "Hadoop Map-Reduce Programming & Best Pr...
Apache Hadoop India Summit 2011 talk "Hadoop Map-Reduce Programming & Best Pr...
Big Data Analytics Projects - Real World with Pentaho by Mark Kromer
Big Data Analytics Projects - Real World with PentahoBig Data Analytics Projects - Real World with Pentaho
Big Data Analytics Projects - Real World with Pentaho
Mark Kromer3.4K views
Flink Forward SF 2017: Malo Deniélou - No shard left behind: Dynamic work re... by Flink Forward
Flink Forward SF 2017: Malo Deniélou -  No shard left behind: Dynamic work re...Flink Forward SF 2017: Malo Deniélou -  No shard left behind: Dynamic work re...
Flink Forward SF 2017: Malo Deniélou - No shard left behind: Dynamic work re...
Flink Forward267 views

More from HBaseCon

hbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes by
hbaseconasia2017: Building online HBase cluster of Zhihu based on Kuberneteshbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
hbaseconasia2017: Building online HBase cluster of Zhihu based on KubernetesHBaseCon
3.9K views36 slides
hbaseconasia2017: HBase on Beam by
hbaseconasia2017: HBase on Beamhbaseconasia2017: HBase on Beam
hbaseconasia2017: HBase on BeamHBaseCon
1.3K views26 slides
hbaseconasia2017: HBase Disaster Recovery Solution at Huawei by
hbaseconasia2017: HBase Disaster Recovery Solution at Huaweihbaseconasia2017: HBase Disaster Recovery Solution at Huawei
hbaseconasia2017: HBase Disaster Recovery Solution at HuaweiHBaseCon
1.4K views21 slides
hbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinterest by
hbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinteresthbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinterest
hbaseconasia2017: Removable singularity: a story of HBase upgrade in PinterestHBaseCon
936 views42 slides
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程 by
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程HBaseCon
1.1K views21 slides
hbaseconasia2017: Apache HBase at Netease by
hbaseconasia2017: Apache HBase at Neteasehbaseconasia2017: Apache HBase at Netease
hbaseconasia2017: Apache HBase at NeteaseHBaseCon
754 views27 slides

More from HBaseCon(20)

hbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes by HBaseCon
hbaseconasia2017: Building online HBase cluster of Zhihu based on Kuberneteshbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
hbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
HBaseCon3.9K views
hbaseconasia2017: HBase on Beam by HBaseCon
hbaseconasia2017: HBase on Beamhbaseconasia2017: HBase on Beam
hbaseconasia2017: HBase on Beam
HBaseCon1.3K views
hbaseconasia2017: HBase Disaster Recovery Solution at Huawei by HBaseCon
hbaseconasia2017: HBase Disaster Recovery Solution at Huaweihbaseconasia2017: HBase Disaster Recovery Solution at Huawei
hbaseconasia2017: HBase Disaster Recovery Solution at Huawei
HBaseCon1.4K views
hbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinterest by HBaseCon
hbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinteresthbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinterest
hbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinterest
HBaseCon936 views
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程 by HBaseCon
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
HBaseCon1.1K views
hbaseconasia2017: Apache HBase at Netease by HBaseCon
hbaseconasia2017: Apache HBase at Neteasehbaseconasia2017: Apache HBase at Netease
hbaseconasia2017: Apache HBase at Netease
HBaseCon754 views
hbaseconasia2017: HBase在Hulu的使用和实践 by HBaseCon
hbaseconasia2017: HBase在Hulu的使用和实践hbaseconasia2017: HBase在Hulu的使用和实践
hbaseconasia2017: HBase在Hulu的使用和实践
HBaseCon878 views
hbaseconasia2017: 基于HBase的企业级大数据平台 by HBaseCon
hbaseconasia2017: 基于HBase的企业级大数据平台hbaseconasia2017: 基于HBase的企业级大数据平台
hbaseconasia2017: 基于HBase的企业级大数据平台
HBaseCon701 views
hbaseconasia2017: HBase at JD.com by HBaseCon
hbaseconasia2017: HBase at JD.comhbaseconasia2017: HBase at JD.com
hbaseconasia2017: HBase at JD.com
HBaseCon828 views
hbaseconasia2017: Large scale data near-line loading method and architecture by HBaseCon
hbaseconasia2017: Large scale data near-line loading method and architecturehbaseconasia2017: Large scale data near-line loading method and architecture
hbaseconasia2017: Large scale data near-line loading method and architecture
HBaseCon598 views
hbaseconasia2017: Ecosystems with HBase and CloudTable service at Huawei by HBaseCon
hbaseconasia2017: Ecosystems with HBase and CloudTable service at Huaweihbaseconasia2017: Ecosystems with HBase and CloudTable service at Huawei
hbaseconasia2017: Ecosystems with HBase and CloudTable service at Huawei
HBaseCon683 views
hbaseconasia2017: HBase Practice At XiaoMi by HBaseCon
hbaseconasia2017: HBase Practice At XiaoMihbaseconasia2017: HBase Practice At XiaoMi
hbaseconasia2017: HBase Practice At XiaoMi
HBaseCon1.8K views
hbaseconasia2017: hbase-2.0.0 by HBaseCon
hbaseconasia2017: hbase-2.0.0hbaseconasia2017: hbase-2.0.0
hbaseconasia2017: hbase-2.0.0
HBaseCon1.8K views
HBaseCon2017 Democratizing HBase by HBaseCon
HBaseCon2017 Democratizing HBaseHBaseCon2017 Democratizing HBase
HBaseCon2017 Democratizing HBase
HBaseCon897 views
HBaseCon2017 Removable singularity: a story of HBase upgrade in Pinterest by HBaseCon
HBaseCon2017 Removable singularity: a story of HBase upgrade in PinterestHBaseCon2017 Removable singularity: a story of HBase upgrade in Pinterest
HBaseCon2017 Removable singularity: a story of HBase upgrade in Pinterest
HBaseCon646 views
HBaseCon2017 Quanta: Quora's hierarchical counting system on HBase by HBaseCon
HBaseCon2017 Quanta: Quora's hierarchical counting system on HBaseHBaseCon2017 Quanta: Quora's hierarchical counting system on HBase
HBaseCon2017 Quanta: Quora's hierarchical counting system on HBase
HBaseCon608 views
HBaseCon2017 Transactions in HBase by HBaseCon
HBaseCon2017 Transactions in HBaseHBaseCon2017 Transactions in HBase
HBaseCon2017 Transactions in HBase
HBaseCon1.8K views
HBaseCon2017 Highly-Available HBase by HBaseCon
HBaseCon2017 Highly-Available HBaseHBaseCon2017 Highly-Available HBase
HBaseCon2017 Highly-Available HBase
HBaseCon1.1K views
HBaseCon2017 Apache HBase at Didi by HBaseCon
HBaseCon2017 Apache HBase at DidiHBaseCon2017 Apache HBase at Didi
HBaseCon2017 Apache HBase at Didi
HBaseCon996 views
HBaseCon2017 gohbase: Pure Go HBase Client by HBaseCon
HBaseCon2017 gohbase: Pure Go HBase ClientHBaseCon2017 gohbase: Pure Go HBase Client
HBaseCon2017 gohbase: Pure Go HBase Client
HBaseCon1.7K views

Recently uploaded

From chaos to control: Managing migrations and Microsoft 365 with ShareGate! by
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!From chaos to control: Managing migrations and Microsoft 365 with ShareGate!
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!sammart93
9 views39 slides
Info Session November 2023.pdf by
Info Session November 2023.pdfInfo Session November 2023.pdf
Info Session November 2023.pdfAleksandraKoprivica4
11 views15 slides
Vertical User Stories by
Vertical User StoriesVertical User Stories
Vertical User StoriesMoisés Armani Ramírez
12 views16 slides
Igniting Next Level Productivity with AI-Infused Data Integration Workflows by
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Safe Software
257 views86 slides
Transcript: The Details of Description Techniques tips and tangents on altern... by
Transcript: The Details of Description Techniques tips and tangents on altern...Transcript: The Details of Description Techniques tips and tangents on altern...
Transcript: The Details of Description Techniques tips and tangents on altern...BookNet Canada
135 views15 slides
Case Study Copenhagen Energy and Business Central.pdf by
Case Study Copenhagen Energy and Business Central.pdfCase Study Copenhagen Energy and Business Central.pdf
Case Study Copenhagen Energy and Business Central.pdfAitana
16 views3 slides

Recently uploaded(20)

From chaos to control: Managing migrations and Microsoft 365 with ShareGate! by sammart93
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!From chaos to control: Managing migrations and Microsoft 365 with ShareGate!
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!
sammart939 views
Igniting Next Level Productivity with AI-Infused Data Integration Workflows by Safe Software
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Safe Software257 views
Transcript: The Details of Description Techniques tips and tangents on altern... by BookNet Canada
Transcript: The Details of Description Techniques tips and tangents on altern...Transcript: The Details of Description Techniques tips and tangents on altern...
Transcript: The Details of Description Techniques tips and tangents on altern...
BookNet Canada135 views
Case Study Copenhagen Energy and Business Central.pdf by Aitana
Case Study Copenhagen Energy and Business Central.pdfCase Study Copenhagen Energy and Business Central.pdf
Case Study Copenhagen Energy and Business Central.pdf
Aitana16 views
HTTP headers that make your website go faster - devs.gent November 2023 by Thijs Feryn
HTTP headers that make your website go faster - devs.gent November 2023HTTP headers that make your website go faster - devs.gent November 2023
HTTP headers that make your website go faster - devs.gent November 2023
Thijs Feryn21 views
Black and White Modern Science Presentation.pptx by maryamkhalid2916
Black and White Modern Science Presentation.pptxBlack and White Modern Science Presentation.pptx
Black and White Modern Science Presentation.pptx
maryamkhalid291616 views
TouchLog: Finger Micro Gesture Recognition Using Photo-Reflective Sensors by sugiuralab
TouchLog: Finger Micro Gesture Recognition  Using Photo-Reflective SensorsTouchLog: Finger Micro Gesture Recognition  Using Photo-Reflective Sensors
TouchLog: Finger Micro Gesture Recognition Using Photo-Reflective Sensors
sugiuralab19 views
AMAZON PRODUCT RESEARCH.pdf by JerikkLaureta
AMAZON PRODUCT RESEARCH.pdfAMAZON PRODUCT RESEARCH.pdf
AMAZON PRODUCT RESEARCH.pdf
JerikkLaureta19 views
Empathic Computing: Delivering the Potential of the Metaverse by Mark Billinghurst
Empathic Computing: Delivering  the Potential of the MetaverseEmpathic Computing: Delivering  the Potential of the Metaverse
Empathic Computing: Delivering the Potential of the Metaverse
Mark Billinghurst476 views
Perth MeetUp November 2023 by Michael Price
Perth MeetUp November 2023 Perth MeetUp November 2023
Perth MeetUp November 2023
Michael Price19 views
Lilypad @ Labweek, Istanbul, 2023.pdf by Ally339821
Lilypad @ Labweek, Istanbul, 2023.pdfLilypad @ Labweek, Istanbul, 2023.pdf
Lilypad @ Labweek, Istanbul, 2023.pdf
Ally3398219 views
STPI OctaNE CoE Brochure.pdf by madhurjyapb
STPI OctaNE CoE Brochure.pdfSTPI OctaNE CoE Brochure.pdf
STPI OctaNE CoE Brochure.pdf
madhurjyapb13 views

HBaseCon2017 Community-Driven Graphs with JanusGraph

  • 1. Jing Chen He • jinghe@us.ibm.com • Apache HBase PMC • JanusGraph TSC Jason Plurad • pluradj@us.ibm.com • Apache TinkerPop PMC • JanusGraph TSC HBaseCon West 2017 • June 12, 2017 Community-Driven Graphs with JanusGraph
  • 2. Agenda Property Graphs Graph Community Introduction to JanusGraph JanusGraph with HBase 2 #HBaseCon
  • 3. Graph  Born for relationship!  Intuitive modeling  Expressive querying  Native analysis 3 #HBaseCon https://tinkerpop.apache.org/docs/3.2.4/reference/#intro
  • 4. Graph Data Use Cases  Social network analysis  Configuration management database  Master data management  Recommendation engines  Knowledge graphs  Internet of things  Cybersecurity attack analysis 4 #HBaseCon
  • 5. Apache TinkerPop  Open source, vendor-agnostic, graph computing framework  Gremlin graph traversal language 5 Apache TinkerPop™ Maintainer Apache Software Foundation License Apache Latest Release 3.2.4 February 2017 https://tinkerpop.apache.org #HBaseCon
  • 6. Gremlin Graph Traversal Language 6 #HBaseCon https://tinkerpop.apache.org/gremlin.html
  • 9.  Scalable graph database distributed on multi-machine clusters with pluggable storage and indexing  Fully-compliant with Apache TinkerPop graph computing framework  Vendor-neutral, open community with open governance – Founding members: Expero, Google, GRAKN.AI, Hortonworks, IBM – Latest members: Amazon, Netflix, Orchestral Developments, Uber 9 JanusGraph™ Maintainer Linux Foundation License Apache Latest Release 0.1.0 April 2017 https://janusgraph.org #HBaseCon
  • 10. 10 #HBaseCon Architecture Google Cloud Bigtable http://docs.janusgraph.org/latest/arch-overview.html
  • 13. 13 #HBaseCon with HBase  HBase – Perfect Storage Backend for JanusGraph Big enough for your biggest graph! The storage model Read and write speed Scalability and partitioning Strong consistency Tight integration with Hadoop Ecosystem Great open community! http://docs.janusgraph.org/latest/hbase.html
  • 14. 14 #HBaseCon with HBase  HBase – Perfect Storage Backend for JanusGraph Simple configuration!  conf/janusgraph-hbase-solr.properties  storage.backend=hbase  storage.hostname=zookeeper-host1,zookeeper-host2,zookeeper-host3  storage.hbase.table=janusgraph  storage.hbase.ext.zookeeper.znode.parent=/hbase  storage.hbase.ext.hbase.zookeeper.property.clientPort=2181  Just open your graph!  graph=JanusGraphFactory.open('conf/janusgraph-hbase-solr.properties') Optional Optional
  • 15. 15 #HBaseCon with HBase  HBase – Perfect Storage Backend for JanusGraph Throw in an Index Backend for better performance  conf/janusgraph-hbase-solr.properties  index.search.backend=solr  index.search.solr.mode=cloud  index.search.solr.zookeeper-url=zookeeper-host1:2181/solr,zookeeper- host2:2181/solr,zookeeper-host3:2181/solr  index.search.solr.configset=janusgraph
  • 16. 16 #HBaseCon with HBase  HBase – Perfect Storage Backend for JanusGraph Look into more details  Stores to Column Families  Edge store  e  Index store  g  ID store  i  Transaction log store  l  System property store  s  CF attributes can be set. E.g. compression, TTL.
  • 17. 17 #HBaseCon with HBase  HBase – Perfect Storage Backend for JanusGraph Look into more details g.V().has("name", "Alice").out("knows").out("knows").values("name") Query Plan to Backend Store and Index Edge Store Index Store Index provider
  • 18. 18 #HBaseCon with HBase  HBase – Perfect Storage Backend for JanusGraph Look into more details  A store (column family) is always specified.  Get or Multi Get  Batch to mutate  Key range scan  ColumnRangeFilter  ColumnPaginationFilter  HBase tuning Edge Store Index Store
  • 19. 19 #HBaseCon with Google Cloud Bigtable  Bigtable implements the HBase 1.0 client API Need the latest version of the bigtable-hbase-1.0 artifact.  storage.backend=hbase  storage.hbase.ext.hbase.client.connection.impl= com.google.cloud.bigtable.hbase1_0.BigtableConnection  storage.hbase.ext.google.bigtable.project.id= <Google Cloud Platform project id>  storage.hbase.ext.google.bigtable.instance.id=<Bigtable instance id>