This document discusses using clusterdock, an open-source container orchestration framework, to deploy and manage Apache HBase clusters on Docker. It provides an overview of Docker and clusterdock, describes how the HBase topology uses clusterdock to build and start HBase clusters quickly, and demos running an HBase integration test on a clusterdock cluster. It also discusses plans to use clusterdock for running HBase integration tests upstream and improving the release process.

1. HBase on Docker with
clusterdock
Dima Spivak

2. Overview
Introduction to Docker (5 min or less!)
clusterdock
Topology for Apache HBase
Demo!
hbase-it upstream
In progress

3. Introduction to Docker
Virtual machines Docker containers
https://www.docker.com/what-docker

4. Introduction to Docker
Isolated namespaces, process trees
Daemon with REST API and a popular client
docker run …
Tooling
Networking
Images
Registries
Docker Swarm, Docker Compose, ...

5. clusterdock
Docker tooling designed to support one-process-per-container workflow
No mature tooling for “treat container like a host”
Python-based container-orchestration framework
Started as a hackathon at Cloudera, now open-source
https://github.com/cloudera/clusterdock

6. clusterdock
Build clusters once, start clusters
repeatedly
Abstractions for cluster-centric entities (e.g.
Cluster, Node, NodeGroup)
cluster.ssh(‘wget …’)
workers_nodegroup.ssh(‘cp …’)
Topology defines how to build/start clusters

7. clusterdock

8. clusterdock
profile.cfg
actions.py

9. clusterdock | Topology for Apache HBase
Started in HBASE-12721, now lives in dev-support
Supports building and starting clusters from binary tarballs or from source
Once image is built, starting a 5-node cluster takes less than 1 minute

10. clusterdock | Topology for Apache HBase
https://github.com/apache/hbase/blob/master/dev-support/apache_hbase_topology/actions.py

11. clusterdock | Topology for Apache HBase
configurations.cfg <configuration>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://node-1.cluster/hbase</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>node-1.cluster</value>
</property>
<property>
<name>hbase.zookeeper.property.datadir</name>
<value>/usr/local/zookeeper</value>
</property>
</configuration>

12. clusterdock | Topology for Apache HBase
source /dev/stdin <<< "$(curl -sL http://tiny.cloudera.com/clusterdock.sh)"
CLUSTERDOCK_TOPOLOGY_IMAGE=apache-docker-hbase.bintray.io/dev/clusterdock:apache_hbase_topology
clusterdock_run ./bin/start_cluster --registry-url apache-docker-hbase.bintray.io --namespace dev
apache_hbase --hbase-version=1.2.3 --hadoop-version=2.5.1 --java-version=7u79 --secondary-nodes='node-{2..5}'
clusterdock_ssh node-1.cluster

13. DEMO

15. hbase-it upstream
Upstream system testing framework
It works
Finds a lot of serious (e.g. data loss) bugs
Fault-injection via chaos monkey
… but no one runs it upstream
Setting up clusters is hard
Tests take a long time to run
Manual
clusterdock can help

16. hbase-it upstream
Build and push clusterdock clusters of
HBase dev branches
master, branch-1.3, branch-1.2, …
Use these pre-built clusters for testing
Can also improve release process
HBASE-16559
Ultimate goal: make passing ITs a quality
gate for new features

17. DEMO
clusterdock_ssh node-1.cluster
hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList -m serverKilling loop 1 4 250000000 dir 4

19. In progress
Documentation
Moving to infrastructure that can support longer runs
Expanded matrix
More chaos monkeys, configuration options, Java versions, ...
Finding a permanent home for clusterdock

20. @dimaspivak
dimaspivak@apache.org