Recommended
Recommended
More Related Content
Recently uploaded
Recently uploaded (20)
Featured
Featured (20)
Monoliths To Microservices - Practical Tips For CI, CD And DevOps in the Microservice World
- 1. Monoliths to Microservices Practical Tips For CI, CD and DevOps in the Microservices world @dwmkerr Dave Kerr Senior Expert, McKinsey & Company
- 3. I am a technology consultant. @dwmkerr
- 5. Which is closer to this… @dwmkerr
- 7. What am I going to talk about? @dwmkerr
- 8. The Project. @dwmkerr …or ‘come back WebSphere, all is forgiven’.
- 9. 2016 – new digital banking application launched to critical acclaim @dwmkerr
- 10. Digital garage setup, experienced product team, streets paved with gold. @dwmkerr
- 11. An effective product team will rapidly surface technological and organisational issues. @dwmkerr
- 12. I once caught a monolith thiiiiiiiis big
- 13. Solution Rebuild Stack Refresh 105 people 60 devs 5 tech leads 2 architects 9 months React Native Node.js Kubernetes GraphQL DevOps 2.0 Hiring Overhaul New HR processes > 50 new hires Sometimes there are no easy answers… Extensive Training Backend Frontend Scrum / Agile PMO Let’s talk about this… @dwmkerr
- 14. Our vision is to replace large complicated systems with small and simple units Mobile Web Monolith Integrator ESB SC CI/CD Artefacts APM Logs Core Cardmanagement Transfers Fraud ATMswitch Interbankswitch FileManagement …etc… Map/ Reduce …etc… Spark Data lake Internal Gateway API Gateway Bill payments Profile & preferences ReferralsP2P payments 2FA, TX signing, soft- Token Contacts & groups Statements Device management Cards Identity management Notifications & inbox Accounts & transfers Mobile Web Existing server applications Enterprise integration Core platforms & databases Service zones Devops zone Analytics zone ExistingRetired New Don’t forget to mix in some
- 15. People were nervous about all of the changes… @dwmkerr
- 16. @dwmkerr
- 17. Hic sunt dracones.…practical tips for devops in the microservice world…
- 18. Tip: Keep pipelines simple @dwmkerr
- 19. Tip: Keep pipelines simple. Build: turn code into artifacts Deploy: put artifacts somewhe @dwmkerr
- 21. Tip: Keep pipelines simple... by embracing make. Step 1: rip out logic from your build system to create primitive commands @dwmkerr
- 22. Tip: Keep pipelines simple... by embracing make. Step 2: call primitive commands from your build system @dwmkerr
- 23. Tip: Keep pipelines simple... by embracing make. Step 3: when things get complicated, put them into ./scripts @dwmkerr
- 24. Tip: Keep pipelines simple... by embracing make. Demo github.com/dwmkerr/beautifully-simple-app-ci @dwmkerr
- 25. Tip: Craft Dockerfiles for your projects @dwmkerr
- 27. Tip: Craft Dockerfiles for your projects A project’s baseline image for a Node 8 microservice. Handles: build-time proxies, NPM registry, non-root users, version tracking and allows the base OS to be changed. This Dockerfile is sneaky – no file system changes, only instructions for later builds!
- 28. Tip: Craft Dockerfiles for your projects
- 29. Tip: Craft Dockerfiles for your projects Demo github.com/dwmkerr/docker-dynamodb @dwmkerr
- 30. @dwmkerr Tip: Use the sandbox …and share your toys!
- 31. Tip: Use the sandbox Code Sample github.com/dwmkerr/terraform-aws-openshift @dwmkerr
- 32. Tip: Use the sandbox Demo github.com/dwmkerr/terraform-aws-openshift @dwmkerr
- 33. @dwmkerr Don’t forget to share your toys! …it’ll help you find bugs, get more ideas and build a group to collaborate with.
- 35. @dwmkerr Don’t forget to fall madly in love with Terraform!
- 36. @dwmkerr Tip: Make a microservice manifest …say that fast five times…
- 37. Tip: Make a microservice manifest
- 38. Tip: Make a microservice manifest
- 39. Tip: The developer’s machine comes first @dwmkerr
- 40. Tip: The developer’s machine comes first • Never sacrifice developer experience • Use docker-compose liberally • You should be able to deploy to production from anyone’s machine, as long as they have credentials @dwmkerr
- 41. Tip: Co-locate code and deployment configuration @dwmkerr
- 42. Tip: Co-locate microservice code & deployment config • jenkins / circle / gitlab / whatever • k8s / openshift / swarm / whatever • src / test • README! @dwmkerr
- 43. Tip: Co-locate microservice code & deployment config If I change the logic for my healthcheck, does my config change? @dwmkerr
- 44. Tip: Love the shell @dwmkerr
- 45. @dwmkerr npm install -g tldr …if I can leave you with one gift github.com/dwmkerr/effective-shell
- 46. Thanks! A lot of this stuff is on my blog and GitHub. @dwmkerr dwmkerr.com github.com/dwmkerr linkedin.com/in/dwmkerr
Editor's Notes
- I do three things: Advise on strategy, typical McKinsey stuff Lead our engineering practice A lot of coding
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- 5m mark: Serving clients across the world, deploying in airports, writing code.
- 1. In this case: disjointed teams 2. Slow deployments 3. Black-box solutions 4. Fast UI / Slow Backend 5. Monkey patching
- Monolithic data stores
- Big transformation project C-level sponsorship Capability building
- 10m mark Before, a single monolith database After – microservices Kafka / GraphQL
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- Talk the tech stack over 15m
- We got a lot of stuff wrong, and we are still getting a lot of stuff wrong.
- Understand the commodity step vs the custom step Don’t try to get it right at the beginning, cause you probably won’t
- Everything you see here is hard to test. Bring it into simple scripts, in fact use makefiles.
- Lessons Learned: 2 mins 21/25 mins Pipelines can get incredibly over-complicated very quickly. Most of the time, you need to sets of pipelines: 1. Turn source code into artifacts. 2. Put artifacts somewhere Be agile, think MVP, separate build/orchestrate
- Demo: 0. show the README and table show how there is a makefile in each project run the first two makefiles Show the circle file Show the travis file
- 1. Note: Dockerfile has no app specifics, only environment specifics.
- First, show the Dockerfile on Docker Hub Then, show the makefile Then, show the circle file
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- You have an amazing sandbox, use it! Get genuinely good at ops things, like networking, route tables, DNS etc
- 1. First show the README.md file, explain what the project is for 2.
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- Lessons Learned: 2 mins 21/25 mins Pipelines can get incredibly over-complicated very quickly. Most of the time, you need to sets of pipelines: 1. Turn source code into artifacts. 2. Put artifacts somewhere Be agile, think MVP, separate build/orchestrate
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- No proxies No shit tools Docker-compose for databases, external services, etc. Docker-compose up to bring up dependencies. No environment rubbish in my day to day life please
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- Microservices aint micro unless they contain their config
- Microservices aint micro unless they contain their config
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- Don’t forget, I’m writing a book!
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes