Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Webinar: Code Faster on Kubernetes

258 views

Published on

What’s the key to successfully adopting microservices on Kubernetes?
Building a development workflow that helps developers code faster.

In this webinar, we introduce the principles of a cloud-native development workflow where individual teams build and ship software independently from each other.

Published in: Software
  • Be the first to comment

Webinar: Code Faster on Kubernetes

  1. 1. Code Faster on Kubernetes (Service Oriented Development) Rafael Schloming - @rschloming Philip Lombardi - @TheBigLombowski
  2. 2. datawire.io How do I break up my monolith? How do I architect my app with microservices? What infrastructure do I need in place before I can benefit from microservices? 2
  3. 3. datawire.io ● Building a cloud application using microservices in 2015 ● Distributed systems engineers ● Deeply studied microservices Architectures & Technology 3 Microservices at Datawire ...
  4. 4. datawire.io Control Plane and Data Plane 4 Sidecar Proxies Health checks Observability L4/L7 routing ... Control Plane Discovery AuthN/AuthZ Routing policy ... Sidecar Proxies Health checks Observability L4/L7 routing ... Sidecar Proxies Health checks Observability L4/L7 routing ...
  5. 5. datawire.io Debugging Velocity (or lack thereof) 5
  6. 6. datawire.io Tooling Architecture Process!!! 6
  7. 7. datawire.io Debugging our Pipeline 7
  8. 8. datawire.io Velocity comes from Process, not Architecture 8
  9. 9. datawire.io Service Oriented Architecture Service Oriented Development 9
  10. 10. datawire.io 10 Stability/Maturity Velocity Prototype Production Mission critical
  11. 11. datawire.io 11 Stability/Maturity Velocity Prototype Production Mission critical
  12. 12. datawire.io A single process is inefficient (Forces a single Stability vs Velocity Tradeoff) 12
  13. 13. datawire.io 13 Define Code Test Release Prod
  14. 14. datawire.io 14 Define Code Test Release Prod Centralized process ● Specialized teams ● Fixed policies (e.g., release criteria)
  15. 15. datawire.io A single process doesn’t scale 15
  16. 16. datawire.io How do I break up my monolith? How do I break up my process? 16
  17. 17. datawire.io 17 Microservices lets you run multiple processes!
  18. 18. datawire.io Microservices is a distributed development architecture workflow. 18
  19. 19. datawire.io 19 Stability/Maturity Velocity Prototype Production Mission critical ● How do I get to Continuous Deployment incrementally? ● How do I limit the scope of PCI (audit process)? ● How do I ship feature X as fast as possible?
  20. 20. datawire.io Microservices is ... ● Multiple processes ○ Including your existing process! ○ Processes designed for different stability/velocity tradeoffs ● Simultaneous processes 20
  21. 21. datawire.io Doing things this way shifts how people operate! ● Requires both organizational and technical changes 21
  22. 22. datawire.io Organizational Implementation 22
  23. 23. datawire.io Education ● Everyone exposed to full dev cycle Communication ● Nobody speaks the same language Delegation ● Small teams own big important parts You gotta give in order to get 23
  24. 24. datawire.io But you get a lot 24 Education ● Specialists become generalists -> Better holistic systems ● Learning, personal growth -> Job satisfaction Communication ● Conflict -> Collaboration Delegation ● Massive organizational scale
  25. 25. datawire.io Create self-sufficient, autonomous software teams. 25
  26. 26. datawire.io Why self-sufficiency and autonomy? ● Self-sufficient ○ Team does not need to rely on other teams to achieve its goals ● Autonomy ○ Team is able to independently make (process) decisions on how to achieve its goals 26
  27. 27. datawire.io Eliminate centralized specialist functions 27 Centralized architecture Centralized infrastructure / ops* (You might need a platform team)
  28. 28. datawire.io Think Spinoff 28
  29. 29. datawire.io 29 Monolith
  30. 30. datawire.io 30 Monolith Microservice Team
  31. 31. datawire.io Technical Implementation 31
  32. 32. datawire.io Control Plane and Data Plane 32 Sidecar Proxies Health checks Observability L4/L7 routing ... Control Plane Discovery AuthN/AuthZ Routing policy ... Sidecar Proxies Health checks Observability L4/L7 routing ... Sidecar Proxies Health checks Observability L4/L7 routing ...
  33. 33. datawire.io People operate the control plane 33 Sidecar Proxies Health checks Observability L4/L7 routing ... Control Plane Discovery AuthN/AuthZ Routing policy ... Sidecar Proxies Health checks Observability L4/L7 routing ... Sidecar Proxies Health checks Observability L4/L7 routing ... Control Plane UI People
  34. 34. datawire.io Guiding goal: make our team productive 34 Tools for Generalists ● Generalist UX != Specialist UX ● Specialist: All the things ● Generalist: Simple, Complete, Discoverable Tools for Education ● Build on familiar concepts ● Safe defaults ● Great feedback Tools for Autonomous Developers ● Fit multiple processes
  35. 35. datawire.io The Datawire Story... ● We have been building cloud apps as microservices since 2015 ● Way too small to have an actual dedicated ops / platform engineer 35
  36. 36. datawire.io The Datawire Story... ● But…! Good news I did that in a previous life. ● Problem…! I’m supposed to be writing product code. 36
  37. 37. datawire.io The Datawire Story... I wanted to do self-service so any developer could scratch their own itch without bugging me. 37
  38. 38. datawire.io Some design considerations... ● Polyglot programming shop ● Lots of different toolchains ● Distributed-ish engineering organization ● Not everyone has a strong background in microservices and infrastructure ● Several mission critical services 38
  39. 39. datawire.io The Datawire Story... Also I wanted to do it “right”... so I architected the shit out the problem with tools! 39
  40. 40. datawire.io 40 Expected Reaction… :D
  41. 41. datawire.io 41 … Actual Reaction D:
  42. 42. datawire.io Late 2015… 2016... Change was slow and painful before adopting better starting principles... 42
  43. 43. datawire.io It’s about the people and process, dummy ● Turns out it is not a tooling problem… ● It is a people and process problem… ● Engineers hate adopting new tools when the reason is not compelling… especially if the new tools make their life harder. ● All the tooling in the world does not stop people from continuing their existing behaviors… ● Every tool has a cost because every tool can and will be used slightly differently by different people. 43
  44. 44. datawire.io “Did you read the README?” ~ “No, but...” 44 ● A README isn’t enough. ● Developers are bad at RTFM and WTFM. ● Lots of arguments over people doing things differently from how the README stated… ● Lots of arguments also over people not knowing how to do things because the README did not state anything about <X>.
  45. 45. datawire.io Back to the drawing board... ● We wanted a common build/deployment mechanism that would work for everyone and anything (including CI, multiple languages, etc.) ● Started with Docker as it gave a common way to package and ship code. ● Kubernetes had been on my radar since 2014 but seemed finally ready for mass adoption at this point so we picked that to run containers. 45
  46. 46. datawire.io Our Kubernetes Journey... ● We built tools based on two things: a. Pain b. Common patterns we discovered across all our repositories 46
  47. 47. datawire.io 1. Code a change on a branch (e.g. dev/awesome-feature) 2. Package the build toolchain into the Docker image itself! 3. Run a single command and a new Docker image is built… no need to setup tools on developers machines. Build process is codified into Dockerfile. Step 1: Provide fast repeatable builds for everyone 47
  48. 48. datawire.io Step 1: I said everyone and I meant everyone... 48
  49. 49. datawire.io ● Commit (or don’t commit) a change on a branch (e.g. dev/awesome-feature) ● Run command and presto…! Updated code is running on the cluster without disturbing any other deployment. ● Support any number of parallel deployments of the same codebase Step 2: Provide fast self-service deploy 49
  50. 50. datawire.io Step 1 + 2 Recap ● Docker and Kubernetes provide the backbone of build and deployment. ● We found ourselves mostly doing the same thing (docker build ..., kubectl [apply|create] ...) so we codified it into tools that work fast and have a small learning surface area for devs. ● Nothing “magical” about what the tools do and they can be easily bypassed if necessary. 50
  51. 51. datawire.io ● Add a little metadata to a Kubernetes service manifest ● Ambassador listens and picks up change and configures Envoy ● Makes it super easy to do stuff like expose a branch (dev/awesome-feature) as https://dev-awesome-feature.datawire.io Step 3: Make it easy to reach the changed code 51
  52. 52. datawire.io Hell no. (But it’s soooooo much better than before) Perfect? 52
  53. 53. datawire.io What’s left? ● Self-service bootstrapping ● Self-service stateful infrastructure ● Self-service external to Kubernetes infrastructure (e.g. Amazon RDS) ● Monitoring And Logging 53
  54. 54. datawire.io All the Things 54
  55. 55. datawire.io Capabilities aren’t Enough, UX Matters 55
  56. 56. datawire.io Process and People factors drive the UX 56
  57. 57. datawire.io Build the tools your teams need now 57 Tools for Generalists ● Generalist UX != Specialist UX ● Generalist: Simple, Complete, Discoverable Tools for Education ● Build on familiar concepts ● Safe defaults ● Great feedback Tools for Autonomous Developers ● Fit multiple processes
  58. 58. datawire.io Demo 58
  59. 59. datawire.io Thank you! 59 Build your own dev workflow https://www.datawire.io/faster Tweet Us! @datawireio @thebiglombowski @rschloming Email Us! rhs@datawire.io plombardi@datawire.io

×