Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Journey to Docker Production: Evolving Your Infrastructure and Processes - Bret Fisher, Independent Docker Consultant

535 views

Published on

DevOps in the Real World is far from perfect, and we're all somewhere on the path to one day writing that "Amazing-Hacker-News-Post about your chat-bot fully-automated micro-service infrastructure." But until then, how can you *really* start using containers today, in meaningful ways that impact yours and your customers productivity? This session is designed for practitioners who are looking for ways to get started now with Docker and Swarm in production. No Docker 101 here, this is for helping you be successful on your way to Dockerizing your production systems. Attendees will get tactics, example configs, real working infrastructure designs, and see the (sometimes messy) internals of Docker in production today.

Published in: Technology

Journey to Docker Production: Evolving Your Infrastructure and Processes - Bret Fisher, Independent Docker Consultant

  1. 1. Journey to Docker Production Bret Fisher Independent DevOps Sysadmin Docker Captain
  2. 2. • Docker Dev+Ops user since 2014 • Used for my co-founded startup • Now, fill time container ops indi consultant • Helped over dozen orgs go prod Who Am I?
  3. 3. • You want Docker in production • Containerizing existing infrastructure/apps • Need to break down barriers of getting into Docker production • Often these are "decision barriers", lets make some good defaults • Sometimes it's made-up requirements that aren't required day one • Hopefully I will help you today with all of these! Why Are We Here?
  4. 4. • Containerize As-Is • What To Focus On First • Building Prod Layer by Layer • Agile Infrastructure Planning • Other Ways To Make An Impact Agenda
  5. 5. Production Docker: A New Hope "Containerize as-is!"
  6. 6. New to containers? Limit other changes Sounds obvious, but many container projects start out too big in scope Problems you likely don't need to solve before container day one • Fully automatic CI/CD • Dynamic performance scaling • Containerizing all of an existing solution at once • Service discovery (dynamic updates to LB's, DNS, etc.) • Starting with persistent data Limit Your Simultaneous Innovation
  7. 7. • Microservice conversion isn't required • 12 Factor is a horizon we're always chasing • Don't let these ideals delay containerization • Maybe we need a 12 Factor 12-step program Legacy Apps Work In Containers Too At its core, a container is just a process
  8. 8. You need a good foundation What To Focus On First
  9. 9. is the path to the dark side A Bad Dockerfile...
  10. 10. Good Dockerfiles are more important than fancy orchestration • It's your new build and environment documentation • At first, focus on "it works" rather then fancy or lean (or multi-stage) • Use FROM Official distros that are most familiar What To Focus On First: Dockerfiles
  11. 11. Dockerfile Production Anti-patterns New teams often have anti-patterns or wrong defaults Lets look at some of these shall we:
  12. 12. • Anti: Not storing unique data in volumes Dockerfile Anti-pattern: Trapping Data 👍😺 • Not to be confused with Bind Mounts for sharing data with host
  13. 13. • Anti: Letting image builds pull FROM latest Dockerfile Anti-pattern: No Version Pinning 👍😻 • Anti: Letting image builds install latest packages • You don't do this with your code, why do it with dependencies?
  14. 14. • Anti: Not changing defaults in container like you would on a VM • e.g. php.ini, mysql.conf.d, java memory Dockerfile Anti-pattern: Leaving Default Config 👍😽 • NOTE: Don't just copy over config files from VM's into image
  15. 15. • Anti: Copying in environment config at image build Dockerfile Anti-pattern: Environment Specific 👎🙀 • Goal: Single Dockerfile with default ENV's, and overwrite per- environment at run time
  16. 16. Building Prod Layer by Layer
  17. 17. • Stick with what you know at first • Do some basic performance testing of your existing and planned setup. Even if just defaults with sample data. You will learn lots! • Recent Docker Inc. and HPE whitepaper on MySQL benchmark • (authored by yours truly, and others) • bretfisher.com/dockercon17 Containers-on-VM or Container-on-Bare-Metal Do either, or both. Lots of pros/cons to either
  18. 18. • Docker is kernel and storage driver dependent • Innovations/fixes are still happening here • "Minimum" version != "best" version • No pre-existing opinion? Ubuntu 16.04.2 LTS (latest LTS) • Popular, well-tested with Docker • One of latest LTS kernels out there • Wide storage driver support • Get correct Docker for your distro from store.docker.com Linux Distribution/Release Matters
  19. 19. • Don't make a decision based on image size (remember it's SIS) • At first: match your existing deployment process • Consider changing to Alpine later, maybe much later Container Base Distribution: Which One?
  20. 20. Swarm Architectures
  21. 21. • General advice on how to start small and grow big • Based off learnings from • Docker internal testing • Docker reference architectures • Real world deployments • Swarm3k lessons learned Good Defaults: Swarm Architectures
  22. 22. • "docker swarm init" done! • This happens every day with VM's, why not with containers • Why bother with swarm? Keeping same workflow across environments, secrets, etc. 1-Node Swarm Baby Swarm
  23. 23. • All Managers and Workers • One node can fail • Great when on a very small budget • pet projects • test/CI setups 3-Node Swarm Minimum for HA
  24. 24. • All Managers and Workers • Two nodes can fail • My minimum for production that affects business $$ 5-Node Swarm Better High-Availability
  25. 25. • 5 dedicated managers • Maybe some workers in DMZ • Anything beyond 5 nodes, stick with 5 Managers and rest workers • Control container placement with labels + constraints 10-Node Swarm Separating Out Managers
  26. 26. > docker node update --label-add data-drive=ssd worker1 > docker node update --label-add net-segment=dmz1 worker2 Constraints with Swarm Services > docker service create --constraint node.labels.data-drive==ssd mysql
  27. 27. • 5 dedicated managers • Resize managers as you grow • Multiple worker subnets on Private/DMZ • Control container placement with labels + constraints 100-Node Swarm Aww she's all grown up!
  28. 28. • Different hardware configurations • Different subnets or security groups • Solution: Use node labels with container constraints Not-So-Good Reasons for Multiple Swarms
  29. 29. • Geographical boundaries between nodes (e.g. different regions) • Security boundaries for compliance (PCI, etc.) • Personnel boundaries for Swarm management using Docker API • (could mitigate Docker EE UCP, auth plugin) Good Reasons for Multiple Swarms
  30. 30. Do You Have An Externally Driven Deadline?
  31. 31. • Beware the "Not Implemented Here" Syndrome • Similar to "not invented here" • Opportunities for Outsourcing • Where it can still be a challenge to implement and maintain • And, where the SaaS/On-prem market is mature • For Your Consideration • Image registry • Log aggregation, storage, search, and alerting • 17.05 gets service logs, good for getting started • Monitoring and alerting Outsource Well-Defined Plumbing
  32. 32. Other Docker Products that Compliment Docker CE • Swarm/Stacks/Secrets/Routing Mesh • CLI Management Docker for AWS/Azure • Cloud Resource Template • Moby Instance OS Docker EE • DTR/UCP • GUI Mgmt/RBAC/Layer 7 Proxy/Image Cache More Effort More Decisions Less Effort Less Decisions
  33. 33. Remember To Be Agile, You'll Build The Next One Better
  34. 34. • Already have good infrastructure automation (one click VM)? • Maybe you have great VM autoscale with single-service VM's? • Like the security boundary of the VM OS? You Want Even Less Ops Change? Lets accelerate this even more
  35. 35. • Least amount of infrastructure change while letting you: • Run on Dockerfiles recipes rather then Puppet etc. • Improve your Docker management skills • Simplify your VM OS build • Moby Project & LinuxKit will likely make this more popular • Windows is doing similar with Hyper-V Containers • Linux is doing similar with Intel Clear Containers One Container Per VM Why don't we talk about this more?
  36. 36. Other Ways To Make An Impact
  37. 37. • Use containers for tech support mock environments • Temp test environments, customer demos • Any process where you "wait for human to deploy/configure VM" Other Ways To Make An Impact Docker can be used internally and still add value
  38. 38. Thank You! My New Docker Udemy Course 90% off for DockerCon: dockermastery.com slides: bretfisher.com/dockercon17

×