The session will revolve around the merits and demerits of running SMACK stack over Kubernetes and DCOS. We will start with first discussing how SMACK stack operates in a non distributed environment and identify the problems faced. We will then take a look at how SMACK stack could be deployed and used in distributed environments like Kubernetes and DCOS, the leading distributed operating system/orchestration tools. The session will end with a face of between DCOS and Kubernetes and we would try to identify which of the two is best to host SMACK stack over it. Keeping in mind the 45 mints duration of the session, the following are the pointers we would stress upon in our discussion : 1). The old school way (10 Minutes) a. Overview of how the SMACK stack used to run in a non distributed environment b. It's advantages c. It's disadvantages 2). Overview of running SMACK stack over DCOS (10 minutes) a. Advantages and ease of use of SMACK stack in DCOS b. Any disadvantage/drawback 3). Overview of running SMACK stack over Kubernetes (10 minutes) a. Advantages and ease of use of SMACK stack in Kubernetes b. Any disadvantage/drawback 4). Compare/contrast performance of SMACK stack over DCOS and Kubernetes (15 minutes)

1. Distributing the SMACK stack -
DCOS VS Kubernetes
Presented By :
Sahil Sawhney
@lihas5

2. © 2018 Minimal – Presentation template
2
Sahil Sawhney
Senior Software Consultant
@
Knoldus Inc
About Me

3. 3
Agenda
What and why
SMACK Stack Conclusions and
understandings
Running SMACK
stack over K8
Deploying
SMACK
Old School
Running
SMACK stack
over DCOS
Compare And
Contrast the
deployments

4. 4
❏ Spark
❏ Mesos
❏ Akka
❏ Cassandra
❏ Kafka

5. 5
Why SMACK Stack
❏ Developing applications that process voluminous real time data
❏ Moving from data at rest to data in motion
❏ Ensuring the composition of Reactive Systems abiding reactive
manifesto
❏ Membership of elite club of cutting edge technologies of this era.

6. 6
Deploying SMACK Stack
(A Case Study)

7. 7
Chapter 1

8. 8
The Vanilla
Systems

9. 9
Advantages Disadvantages
❏ Simple Configuration cluster are
created by using cluster nodes IP
(Public and Private).
❏ Better in a scenario when user base is
limited and not subjected to a rapid
growth.
❏ Can't think of many
❏ Tightly coupled, if a node goes down,
replacement node would require
config changes.
❏ Configuration management is a quite
challenging task, must be taken care
of manually.
❏ Application scaling might require
some down time and could be a
hectic process.

10. 10
Chapter 2
DC/OS is a distributed operating
system based on the Apache Mesos
(distributed systems kernel). It :
❏ Automates resource management
❏ Schedules process placement
❏ Facilitates inter-process
communication
❏ Simplifies the installation and
management of distributed services

11. 11
Let's Go On Shopping ...

12. 12
How Do I Specify My
Cluster Components?

13. 13
Deploying Services On DCOS

14. 14
Deploying Akka Http / Java Like Services

15. 15
Deployment Summary
❏ Spark is deployed via catalog, the spark job submitted to DCOS
determine the executors and accordingly resources are allocated.
❏ Mesos comes pre-packaged
❏ Akka application could be deployed using marathon like configuration.
❏ Cassandra and Kafka could be availed from catalog and their node count
could be determined.

16. 16
Chapter 3
Kubernetes is an open-source system
initially used by google for
❏ Automating Deployment
❏ Scaling
❏ Managing containerized applications.

17. 17
Excuse Me, Where Is Mister ‘M’ ?

18. 18
Kubernetes Operator
A Kubernetes operator consists of two components:
❏ Custom resource definition
❏ Custom controllers.
It's an extension of Kubernetes that allows us to define custom objects or
resources using Kubernetes that our controller can then listen to for any
changes to the resource definition

19. 19
Deploying Spark
Spark 2.3 introduced Kubernetes
(1.7 +) as its native cluster
manager.

20. 20
Deploying Cassandra
Cassandra (3.11 +) introduced
Kubernetes (1.8 +) as its Operator.

21. 21
Deploying Kafka
Confluent Operator automates:
❏ Provisioning
❏ Management
❏ Operations of Confluent
Platform
Using the Kubernetes Operator
API
Steps ->
❏ Create Kafka Operator
❏ Create Zookeeper Operator
❏ Create Kafka Cluster

22. 22
❏ Are we now ready to nail it ?
❏ Have we achieved “THE Stack” ?
❏ Are we now ready to jump on
production ?
❏ Or is there a catch ?

23. 23
Should We run Database /
Kafka in Docker / Distributed
Environment ?

24. 24
The Dev Scenario

25. 25
The Prod / QA Scenario

26. 26
References
❏ https://medium.com/pintail-labs/what-is-the-smack-stack-31bc85131a9a
❏ https://www.reactivemanifesto.org
❏ https://blog.softwaremill.com/running-akka-cluster-on-kubernetes-e4cd2913e951

27. 27
Thank You