Distributed databases can make so many things easier for a developer... but not always for DevOps. OK, almost never for DevOps. Kubernetes has come to the rescue with an easy application orchestration! It’s straightforward to do the orchestration leaning on relational databases as a data layer. However, it’s becoming a bit trickier to do the same when a distributed SQL database or other kind of distributed storage is used instead. In this talk you will learn how Kubernetes can orchestrate distributed database like Apache Ignite, in particular: - Cluster Assembling - database nodes auto-discovery in Kubernetes. - Database Resilience - automated horizontal scalability. - Database Availability - what’s the role of Kubernetes and the database. - Utilizing both RAM and disk - set up Apache Ignite in a way to get in-memory performance with the durability of disk.

1. © 2017 GridGain Systems, Inc.
In-Memory Performance
Durability of Disk

2. © 2017 GridGain Systems, Inc.
Distributed Database DevOps Dilemmas?
Kubernetes to the Rescue
Denis Magda
Apache Ignite PMC Chair
GridGain Director of Product Management

3. © 2017 GridGain Systems, Inc.
• Apache Ignite
• Clustering and Deployment
• Orchestration Specificities
• Kubernetes Deployment
• Scaling the Cluster
• Demo
Agenda

4. © 2017 GridGain Systems, Inc.
Apache Ignite In-Memory Computing Platform
Memory-Centric Database
Ignite Native Persistence
(Flash, SSD, Intel 3D XPoint)
Third-Party Persistence
(RDBMS, HDFS, NoSQL)
SQL Transactions Compute Services MLStreamingKey/Value
IoTFinancial
Services
Pharma &
Healthcare
E-CommerceTravel &
Logistics
Telco

5. © 2017 GridGain Systems, Inc.
Clustering
• Server Nodes
• Act as containers for data and computations
• Generally started as standalone processes
• Client Nodes
• Provide a cluster entry point to run operations
• Embedded in applications code

6. © 2017 GridGain Systems, Inc.
Deployment
• Nodes are logical entities
• Runs in a JVM process
• Many nodes in a single JVM process
• On-Premise and Cloud
• Physical server or VM
• AWS, Azure, Google Compute Engine
• Kubernetes, Mesos, YARN

7. © 2017 GridGain Systems, Inc.
IP Finder
• Networking subsystem component
• Knows IP addresses of cluster members
• Multiple Implementation
• Multicast
• Static (predefined IPs)
• Cloud (AWS, Azure, etc.)
• Kubernetes!

8. © 2017 GridGain Systems, Inc.
Orchestration Specificities
Cluster Discovery
• Kubernetes to the rescue!
• Nodes IP addresses are unknown
• IP addresses change after restarts
In-Cluster Communication
• Based on Ignite facilities
• Peer-to-peer
• Self-Healing
• Self-Contained
+

9. © 2017 GridGain Systems, Inc.
Kubernetes Deployment
• Kubernetes Lookup Service
• Maintains a list of the Ignite IPs
• Kubernetes IP Finder
• Communicates to the service
• Ignite pod obtains existing addresses
• The pod joins the cluster using an address
Server Node
ON-DISK
Kubernetes
Lookup
Service
1
2

10. © 2017 GridGain Systems, Inc.
Create the Lookup Service
apiVersion: v1
kind: Service
metadata:
name: ignite
spec:
clusterIP: None
ports:
- port: 9042 # custom value.
selector:
app: ignite
kubectl create -f ignite-service.yaml

11. © 2017 GridGain Systems, Inc.
Deploy Apache Ignite Cluster
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: ignite-cluster
spec:
# Start two Ignite nodes by default.
replicas: 2
template:
metadata:
labels:
app: ignite
spec:
containers:
# Custom Ignite pod name.
- name: ignite-node
image: apacheignite/ignite:2.3.0
env:
- name: OPTION_LIBS
value: ignite-kubernetes
- name: CONFIG_URI
value:
https://raw.githubusercontent.com/apache/ignite/master/modules/kubern
etes/config/example-kube.xml
kubectl create -f ignite-deployment.yaml

12. © 2017 GridGain Systems, Inc.
Scaling the Cluster
kubectl scale --replicas=5 -f ignite-deployment.yaml
From:
To:

13. © 2017 GridGain Systems, Inc.
Demo

14. © 2017 GridGain Systems, Inc.
Any Questions?
Thank you for joining us. Follow the conversation.
http://ignite.apache.org
#apacheignite
#denismagda