Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Loading in …3
×
1 of 46

Monoliths To Microservices - Practical Tips For CI, CD And DevOps in the Microservice World

4

Share

Download to read offline

Video: https://www.youtube.com/watch?v=NVb7aljfKYo&feature=youtu.be&t=6657

As the popularity of microservice architecture grows, CI and CD becomes more and more critical. Systems which contain dozens of services and diverse databases can get very challenging to manage. In this presentation I’ll take you through some hands on patterns which you can use to help you better manage your microservices!

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Monoliths To Microservices - Practical Tips For CI, CD And DevOps in the Microservice World

  1. 1. Monoliths to Microservices Practical Tips For CI, CD and DevOps in the Microservices world @dwmkerr Dave Kerr Senior Expert, McKinsey & Company
  2. 2. Who am I? @dwmkerr
  3. 3. I am a technology consultant. @dwmkerr
  4. 4. Which is closer to this… @dwmkerr
  5. 5. What am I going to talk about? @dwmkerr
  6. 6. The Project. @dwmkerr …or ‘come back WebSphere, all is forgiven’.
  7. 7. 2016 – new digital banking application launched to critical acclaim @dwmkerr
  8. 8. Digital garage setup, experienced product team, streets paved with gold. @dwmkerr
  9. 9. An effective product team will rapidly surface technological and organisational issues. @dwmkerr
  10. 10. I once caught a monolith thiiiiiiiis big
  11. 11. Solution Rebuild Stack Refresh 105 people 60 devs 5 tech leads 2 architects 9 months React Native Node.js Kubernetes GraphQL DevOps 2.0 Hiring Overhaul New HR processes > 50 new hires Sometimes there are no easy answers… Extensive Training Backend Frontend Scrum / Agile PMO Let’s talk about this… @dwmkerr
  12. 12. Our vision is to replace large complicated systems with small and simple units Mobile Web Monolith Integrator ESB SC CI/CD Artefacts APM Logs Core Cardmanagement Transfers Fraud ATMswitch Interbankswitch FileManagement …etc… Map/ Reduce …etc… Spark Data lake Internal Gateway API Gateway Bill payments Profile & preferences ReferralsP2P payments 2FA, TX signing, soft- Token Contacts & groups Statements Device management Cards Identity management Notifications & inbox Accounts & transfers Mobile Web Existing server applications Enterprise integration Core platforms & databases Service zones Devops zone Analytics zone ExistingRetired New Don’t forget to mix in some
  13. 13. People were nervous about all of the changes… @dwmkerr
  14. 14. @dwmkerr
  15. 15. Hic sunt dracones.…practical tips for devops in the microservice world…
  16. 16. Tip: Keep pipelines simple @dwmkerr
  17. 17. Tip: Keep pipelines simple. Build: turn code into artifacts Deploy: put artifacts somewhe @dwmkerr
  18. 18. Source http://www.dwmkerr.com/simple-continuous-integration-for-docker-images/
  19. 19. Tip: Keep pipelines simple... by embracing make. Step 1: rip out logic from your build system to create primitive commands @dwmkerr
  20. 20. Tip: Keep pipelines simple... by embracing make. Step 2: call primitive commands from your build system @dwmkerr
  21. 21. Tip: Keep pipelines simple... by embracing make. Step 3: when things get complicated, put them into ./scripts @dwmkerr
  22. 22. Tip: Keep pipelines simple... by embracing make. Demo github.com/dwmkerr/beautifully-simple-app-ci @dwmkerr
  23. 23. Tip: Craft Dockerfiles for your projects @dwmkerr
  24. 24. Tip: Craft Dockerfiles for your projects A project’s baseline image for a Node 8 microservice. Handles: build-time proxies, NPM registry, non-root users, version tracking and allows the base OS to be changed. This Dockerfile is sneaky – no file system changes, only instructions for later builds!
  25. 25. Tip: Craft Dockerfiles for your projects
  26. 26. Tip: Craft Dockerfiles for your projects Demo github.com/dwmkerr/docker-dynamodb @dwmkerr
  27. 27. @dwmkerr Tip: Use the sandbox …and share your toys!
  28. 28. Tip: Use the sandbox Code Sample github.com/dwmkerr/terraform-aws-openshift @dwmkerr
  29. 29. Tip: Use the sandbox Demo github.com/dwmkerr/terraform-aws-openshift @dwmkerr
  30. 30. @dwmkerr Don’t forget to share your toys! …it’ll help you find bugs, get more ideas and build a group to collaborate with.
  31. 31. @dwmkerr …one more thing…
  32. 32. @dwmkerr Don’t forget to fall madly in love with Terraform!
  33. 33. @dwmkerr Tip: Make a microservice manifest …say that fast five times…
  34. 34. Tip: Make a microservice manifest
  35. 35. Tip: Make a microservice manifest
  36. 36. Tip: The developer’s machine comes first @dwmkerr
  37. 37. Tip: The developer’s machine comes first • Never sacrifice developer experience • Use docker-compose liberally • You should be able to deploy to production from anyone’s machine, as long as they have credentials @dwmkerr
  38. 38. Tip: Co-locate code and deployment configuration @dwmkerr
  39. 39. Tip: Co-locate microservice code & deployment config • jenkins / circle / gitlab / whatever • k8s / openshift / swarm / whatever • src / test • README! @dwmkerr
  40. 40. Tip: Co-locate microservice code & deployment config If I change the logic for my healthcheck, does my config change? @dwmkerr
  41. 41. Tip: Love the shell @dwmkerr
  42. 42. @dwmkerr npm install -g tldr …if I can leave you with one gift github.com/dwmkerr/effective-shell
  43. 43. Thanks! A lot of this stuff is on my blog and GitHub. @dwmkerr dwmkerr.com github.com/dwmkerr linkedin.com/in/dwmkerr

Editor's Notes

  • I do three things:
    Advise on strategy, typical McKinsey stuff
    Lead our engineering practice
    A lot of coding
  • Lessons Learned: 2 mins
    23/25 mins

    You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale.

    Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs.

    Lesson: Know what works. Stateless, or self organising homogenous nodes
  • 5m mark:
    Serving clients across the world, deploying in airports, writing code.
  • 1. In this case: disjointed teams
    2. Slow deployments
    3. Black-box solutions
    4. Fast UI / Slow Backend
    5. Monkey patching
  • Monolithic data stores
  • Big transformation project
    C-level sponsorship
    Capability building
  • 10m mark

    Before, a single monolith database
    After – microservices
    Kafka / GraphQL
  • Lessons Learned: 2 mins
    23/25 mins

    You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale.

    Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs.

    Lesson: Know what works. Stateless, or self organising homogenous nodes
  • Talk the tech stack over
    15m
  • We got a lot of stuff wrong, and we are still getting a lot of stuff wrong.
  • Understand the commodity step vs the custom step
    Don’t try to get it right at the beginning, cause you probably won’t
  • Everything you see here is hard to test.
    Bring it into simple scripts, in fact use makefiles.
  • Lessons Learned: 2 mins
    21/25 mins

    Pipelines can get incredibly over-complicated very quickly.

    Most of the time, you need to sets of pipelines:

    1. Turn source code into artifacts.
    2. Put artifacts somewhere

    Be agile, think MVP, separate build/orchestrate
  • Demo:
    0. show the README and table
    show how there is a makefile in each project
    run the first two makefiles
    Show the circle file
    Show the travis file
  • 1. Note: Dockerfile has no app specifics, only environment specifics.
  • First, show the Dockerfile on Docker Hub
    Then, show the makefile
    Then, show the circle file
  • Lessons Learned: 2 mins
    23/25 mins

    You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale.

    Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs.

    Lesson: Know what works. Stateless, or self organising homogenous nodes
  • You have an amazing sandbox, use it!
    Get genuinely good at ops things, like networking, route tables, DNS etc
  • 1. First show the README.md file, explain what the project is for
    2.
  • Lessons Learned: 2 mins
    23/25 mins

    You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale.

    Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs.

    Lesson: Know what works. Stateless, or self organising homogenous nodes
  • Lessons Learned: 2 mins
    23/25 mins

    You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale.

    Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs.

    Lesson: Know what works. Stateless, or self organising homogenous nodes
  • Lessons Learned: 2 mins
    23/25 mins

    You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale.

    Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs.

    Lesson: Know what works. Stateless, or self organising homogenous nodes
  • Lessons Learned: 2 mins
    23/25 mins

    You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale.

    Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs.

    Lesson: Know what works. Stateless, or self organising homogenous nodes
  • Lessons Learned: 2 mins
    21/25 mins

    Pipelines can get incredibly over-complicated very quickly.

    Most of the time, you need to sets of pipelines:

    1. Turn source code into artifacts.
    2. Put artifacts somewhere

    Be agile, think MVP, separate build/orchestrate
  • Lessons Learned: 2 mins
    23/25 mins

    You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale.

    Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs.

    Lesson: Know what works. Stateless, or self organising homogenous nodes
  • No proxies
    No shit tools
    Docker-compose for databases, external services, etc. Docker-compose up to bring up dependencies.
    No environment rubbish in my day to day life please
  • Lessons Learned: 2 mins
    23/25 mins

    You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale.

    Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs.

    Lesson: Know what works. Stateless, or self organising homogenous nodes
  • Microservices aint micro unless they contain their config
  • Microservices aint micro unless they contain their config
  • Lessons Learned: 2 mins
    23/25 mins

    You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale.

    Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs.

    Lesson: Know what works. Stateless, or self organising homogenous nodes
  • Don’t forget, I’m writing a book!
  • Lessons Learned: 2 mins
    23/25 mins

    You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale.

    Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs.

    Lesson: Know what works. Stateless, or self organising homogenous nodes
  • ×