Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Tales of Training: Scaling CodeLabs with Swarm Mode and Docker-Compose

410 views

Published on

Why does any "code lab workshop" or live demo are always such a challenge?
A wise sysadmin once told me: “Get your hand dirty with the production to learn”.

So I want to tell you a story of getting hand dirties, by creating a code lab environment considered as production.

This story will show that we can build a reproducible environment for code-labs workshops, by using the Docker “tools”: the Engine, Swarm Mode, Docker-Compose, Moby, LinuxKit.
Following the spirit of “Play With Docker”, but generalized at any service collection, this Codelab toolkit has been used on code-labs workshops of 120+ people.

That path was not a free lunch, but the lessons learned will give you an idea on how a training environment can be efficiently done with Compose and Swarm Mode, by treating it as a “production” platform, tackling the plumbing “youth” limitations for the better of your use case.

As a trainer, I never learned so much than building something to teach people someone else: this the story I want to tell you, the tale of using Docker as a tool of MASSIVE KNOWLEDGE SHARING, which is the root of growing our industry together.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Tales of Training: Scaling CodeLabs with Swarm Mode and Docker-Compose

  1. 1. Tales of Training: Scaling CodeLabs with Swarm Mode and Compose Damien DUPORTAL Training Engineer - CloudBees @DamienDuportal - github.com/dduportal
  2. 2. Agenda What is this talk about? ● Trench Story ● Drinking champagne ● Training / CodeLabs ● VM workload ● Container workload ● Orchestration What is this talk NOT about? ● Toolings wars: ○ Kubernetes vs. Swarm ○ AWS vs. Azure ● Showing code (20 min only!) ○ Help yourself (GitHub): http://bit.ly/2yoYG0v
  3. 3. Drinking my own champagne Learn by TRYING: ● Experimentation never wastes time ● Build something that works and iterate
  4. 4. “CodeLab”? ● Part of a training (learning experience) ● Practical Exercise ● Using an isolated environment ● Dedicated to a single user ● Synonym of “workshop” or “hands on lab”
  5. 5. CodeLab == Production Constraints: ● Network ● Configuration diversity ● User Error (Learning process) ● Time boxed Sessions
  6. 6. CodeLab VM Content: ● Jenkins Training: ○ Store Git SCM (Gitea - Go) ○ Jenkins Master + Build Agents (JVMs) ○ Web Command Line (TTYDs - Python) ○ WebIDE (Codiad - PHP) ○ Store Artifacts (Artifactory - JVM) ○ Docker Registry (Go)
  7. 7. Challenges ● CodeLab defined by service list ○ Docker-compose ● Portability ○ Docker as blueprint ● Explicit definition ○ Definition as code
  8. 8. Step 1: VMs Consul Servers AWS EC2 VM Host OS: Ubuntu Docker & Compose CodeLab Service Stack Consul AWS EC2 VM Host OS: Ubuntu Docker & Compose CodeLab Service Stack Consul AWS EC2 VM Host OS: Ubuntu Docker & Compose CodeLab Service Stack Consul AWS EC2 VM Host OS: Ubuntu Docker & Compose CodeLab Service Stack Consul AWS EC2 VM Host OS: Ubuntu Docker & Compose CodeLab Service Stack Consul AWS EC2 VM Host OS: Ubuntu Docker & Compose CodeLab Service Stack Consul SSH Bastion Reverse Proxies Register into Consul End User Trainer Deploys with Terraform Load-Balancer Reconfigure with Consul-Template
  9. 9. Step 1: Take Aways It works! Easy to begin with and to operate Cost (1 attendee per VM) TTP (Time To Play)
  10. 10. Road to Native Orchestration ● Solving Costs Issue: More attendees per VM ● Solving TTP Issue: ○ Moving to container-based workload (Look at PWD speed!) ○ Un-tie Infra from Services
  11. 11. Step 2.1: Provide Infra SSH Bastion“Infra Admin” Deploys with Terraform Swarm Worker (EC2 VM) Swarm Worker (EC2 VM) Swarm Worker (EC2 VM) Swarm Worker (EC2 VM) Swarm Worker (EC2 VM) Swarm Worker (EC2 VM) Swarm Manager (EC2 VM) Swarm Manager (EC2 VM) Swarm Manager (EC2 VM)
  12. 12. Step 2.2: Provide Services SSH Bastion“Infra Admin” Deploys with Docker Swarm Worker (EC2 VM) Swarm Worker (EC2 VM) Swarm Worker (EC2 VM) Swarm Worker (EC2 VM) Swarm Worker (EC2 VM) Swarm Worker (EC2 VM) Swarm Manager (EC2 VM) Swarm Manager (EC2 VM) Swarm Manager (EC2 VM) Deploys with Docker Trainer CodeLab Service Stack Support Services Support Services
  13. 13. Step 2.3: Benefits Swarm Worker (EC2 VM) Swarm Worker (EC2 VM) Swarm Worker (EC2 VM) Swarm Worker (EC2 VM) Swarm Worker (EC2 VM) Swarm Worker (EC2 VM) Swarm Manager (EC2 VM) Swarm Manager (EC2 VM) Swarm Manager (EC2 VM) CodeLab Service Stack Support Services Support Services End User
  14. 14. Step 2: Positive Side Effect ● Relying on Docker ecosystem ● Less side-kicks to maintain
  15. 15. Step 2: Nice problems to have ● CodeLab Scaling Pattern ○ (Deploy N times, NOT Scaling) ● Monitoring / Metrics ● Resource Management
  16. 16. CodeLab Scaling Pattern ● Compose and Swarm: ○ Define a stack of services ○ Scale each horizontally ● Concern here: ○ Deploy N times the same stack
  17. 17. Monitoring/Metrics It is NOT a challenge ● Should have done it at Step 1 ● Feedback loop ● A LOT of “Working Out of the box” tools
  18. 18. Resource Management ● Resource sharing patterns: ○ VM workloads: 1 stack for 1 VM resources ○ Container workload: N stacks for N VMs resources ● JVM pattern: Capping Memory (OOM) ● Need to measure and iterate ○ Automation
  19. 19. Step 2: Take Aways ● It still works! ● Easier to begin with and to operate ● Cost: 4x less (4 attendees per VM) ● TTP: ~10 s instead of ~10 min ● Maintaining: 5 tools instead of 8
  20. 20. Thank you and enjoy DockerCon!

×