Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Distributing the SMACK stack - Kubernetes VS DCOS - Sahil Sawhney (Knoldus Inc.)


Published on

The session will revolve around the merits and demerits of running SMACK stack over Kubernetes and DCOS. We will start with first discussing how SMACK stack operates in a non distributed environment and identify the problems faced. We will then take a look at how SMACK stack could be deployed and used in distributed environments like Kubernetes and DCOS, the leading distributed operating system/orchestration tools. The session will end with a face of between DCOS and Kubernetes and we would try to identify which of the two is best to host SMACK stack over it. Keeping in mind the 45 mints duration of the session, the following are the pointers we would stress upon in our discussion :

1). The old school way (10 Minutes)
a. Overview of how the SMACK stack used to run in a non distributed environment
b. It's advantages
c. It's disadvantages
2). Overview of running SMACK stack over DCOS (10 minutes)
a. Advantages and ease of use of SMACK stack in DCOS
b. Any disadvantage/drawback
3). Overview of running SMACK stack over Kubernetes (10 minutes)
a. Advantages and ease of use of SMACK stack in Kubernetes
b. Any disadvantage/drawback
4). Compare/contrast performance of SMACK stack over DCOS and Kubernetes (15 minutes)

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Distributing the SMACK stack - Kubernetes VS DCOS - Sahil Sawhney (Knoldus Inc.)

  1. 1. Distributing the SMACK stack - DCOS VS Kubernetes Presented By : Sahil Sawhney @lihas5
  2. 2. © 2018 Minimal – Presentation template 2 Sahil Sawhney Senior Software Consultant @ Knoldus Inc About Me
  3. 3. 3 Agenda What and why SMACK Stack Conclusions and understandings Running SMACK stack over K8 Deploying SMACK Old School Running SMACK stack over DCOS Compare And Contrast the deployments
  4. 4. 4 ❏ Spark ❏ Mesos ❏ Akka ❏ Cassandra ❏ Kafka
  5. 5. 5 Why SMACK Stack ❏ Developing applications that process voluminous real time data ❏ Moving from data at rest to data in motion ❏ Ensuring the composition of Reactive Systems abiding reactive manifesto ❏ Membership of elite club of cutting edge technologies of this era.
  6. 6. 6 Deploying SMACK Stack (A Case Study)
  7. 7. 7 Chapter 1
  8. 8. 8 The Vanilla Systems
  9. 9. 9 Advantages Disadvantages ❏ Simple Configuration cluster are created by using cluster nodes IP (Public and Private). ❏ Better in a scenario when user base is limited and not subjected to a rapid growth. ❏ Can't think of many ❏ Tightly coupled, if a node goes down, replacement node would require config changes. ❏ Configuration management is a quite challenging task, must be taken care of manually. ❏ Application scaling might require some down time and could be a hectic process.
  10. 10. 10 Chapter 2 DC/OS is a distributed operating system based on the Apache Mesos (distributed systems kernel). It : ❏ Automates resource management ❏ Schedules process placement ❏ Facilitates inter-process communication ❏ Simplifies the installation and management of distributed services
  11. 11. 11 Let's Go On Shopping ...
  12. 12. 12 How Do I Specify My Cluster Components?
  13. 13. 13 Deploying Services On DCOS
  14. 14. 14 Deploying Akka Http / Java Like Services
  15. 15. 15 Deployment Summary ❏ Spark is deployed via catalog, the spark job submitted to DCOS determine the executors and accordingly resources are allocated. ❏ Mesos comes pre-packaged ❏ Akka application could be deployed using marathon like configuration. ❏ Cassandra and Kafka could be availed from catalog and their node count could be determined.
  16. 16. 16 Chapter 3 Kubernetes is an open-source system initially used by google for ❏ Automating Deployment ❏ Scaling ❏ Managing containerized applications.
  17. 17. 17 Excuse Me, Where Is Mister ‘M’ ?
  18. 18. 18 Kubernetes Operator A Kubernetes operator consists of two components: ❏ Custom resource definition ❏ Custom controllers. It's an extension of Kubernetes that allows us to define custom objects or resources using Kubernetes that our controller can then listen to for any changes to the resource definition
  19. 19. 19 Deploying Spark Spark 2.3 introduced Kubernetes (1.7 +) as its native cluster manager.
  20. 20. 20 Deploying Cassandra Cassandra (3.11 +) introduced Kubernetes (1.8 +) as its Operator.
  21. 21. 21 Deploying Kafka Confluent Operator automates: ❏ Provisioning ❏ Management ❏ Operations of Confluent Platform Using the Kubernetes Operator API Steps -> ❏ Create Kafka Operator ❏ Create Zookeeper Operator ❏ Create Kafka Cluster
  22. 22. 22 ❏ Are we now ready to nail it ? ❏ Have we achieved “THE Stack” ? ❏ Are we now ready to jump on production ? ❏ Or is there a catch ?
  23. 23. 23 Should We run Database / Kafka in Docker / Distributed Environment ?
  24. 24. 24 The Dev Scenario
  25. 25. 25 The Prod / QA Scenario
  26. 26. 26 References ❏ ❏ ❏
  27. 27. 27 Thank You