Advertisement
Check these out next
Monoliths To Microservices - Practical Tips For CI, CD And DevOps in the Microservice World
Oct. 26, 2017•0 likes•997 views
4 likes
Be the first to like this
Show More
views
Total views
0
On Slideshare
0
From embeds
0
Number of embeds
0
Download to read offline
Report
Technology
Video: https://www.youtube.com/watch?v=NVb7aljfKYo&feature=youtu.be&t=6657 As the popularity of microservice architecture grows, CI and CD becomes more and more critical. Systems which contain dozens of services and diverse databases can get very challenging to manage. In this presentation I’ll take you through some hands on patterns which you can use to help you better manage your microservices!
Dave KerrFollow
Advertisement
Advertisement
Advertisement
Recommended
What does devops culture mean for engineersDave Kerr
Build a ChatBot in 10 minutes!Dave Kerr
The Student's Guide to LinkedInLinkedIn
Different Roles in Machine Learning CareerIntellipaat
Defining a Tech Project Vision in Eight Quick Steps pdfTechSoup
The Hero's Journey (For movie fans, Lego fans, and presenters!)Dan Roam
Monoliths To Microservices - Practical Tips For CI, CD And DevOps in the Microservice World
- Monoliths to Microservices Practical Tips For CI, CD and DevOps in the Microservices world @dwmkerr Dave Kerr Senior Expert, McKinsey & Company
- Who am I? @dwmkerr
- I am a technology consultant. @dwmkerr
- Which is closer to this… @dwmkerr
- What am I going to talk about? @dwmkerr
- The Project. @dwmkerr …or ‘come back WebSphere, all is forgiven’.
- 2016 – new digital banking application launched to critical acclaim @dwmkerr
- Digital garage setup, experienced product team, streets paved with gold. @dwmkerr
- An effective product team will rapidly surface technological and organisational issues. @dwmkerr
- I once caught a monolith thiiiiiiiis big
- Solution Rebuild Stack Refresh 105 people 60 devs 5 tech leads 2 architects 9 months React Native Node.js Kubernetes GraphQL DevOps 2.0 Hiring Overhaul New HR processes > 50 new hires Sometimes there are no easy answers… Extensive Training Backend Frontend Scrum / Agile PMO Let’s talk about this… @dwmkerr
- Our vision is to replace large complicated systems with small and simple units Mobile Web Monolith Integrator ESB SC CI/CD Artefacts APM Logs Core Cardmanagement Transfers Fraud ATMswitch Interbankswitch FileManagement …etc… Map/ Reduce …etc… Spark Data lake Internal Gateway API Gateway Bill payments Profile & preferences ReferralsP2P payments 2FA, TX signing, soft- Token Contacts & groups Statements Device management Cards Identity management Notifications & inbox Accounts & transfers Mobile Web Existing server applications Enterprise integration Core platforms & databases Service zones Devops zone Analytics zone ExistingRetired New Don’t forget to mix in some
- People were nervous about all of the changes… @dwmkerr
- @dwmkerr
- Hic sunt dracones.…practical tips for devops in the microservice world…
- Tip: Keep pipelines simple @dwmkerr
- Tip: Keep pipelines simple. Build: turn code into artifacts Deploy: put artifacts somewhe @dwmkerr
- Source http://www.dwmkerr.com/simple-continuous-integration-for-docker-images/
- Tip: Keep pipelines simple... by embracing make. Step 1: rip out logic from your build system to create primitive commands @dwmkerr
- Tip: Keep pipelines simple... by embracing make. Step 2: call primitive commands from your build system @dwmkerr
- Tip: Keep pipelines simple... by embracing make. Step 3: when things get complicated, put them into ./scripts @dwmkerr
- Tip: Keep pipelines simple... by embracing make. Demo github.com/dwmkerr/beautifully-simple-app-ci @dwmkerr
- Tip: Craft Dockerfiles for your projects @dwmkerr
- Tip: Craft Dockerfiles for your projects A project’s baseline image for a Node 8 microservice. Handles: build-time proxies, NPM registry, non-root users, version tracking and allows the base OS to be changed. This Dockerfile is sneaky – no file system changes, only instructions for later builds!
- Tip: Craft Dockerfiles for your projects
- Tip: Craft Dockerfiles for your projects Demo github.com/dwmkerr/docker-dynamodb @dwmkerr
- @dwmkerr Tip: Use the sandbox …and share your toys!
- Tip: Use the sandbox Code Sample github.com/dwmkerr/terraform-aws-openshift @dwmkerr
- Tip: Use the sandbox Demo github.com/dwmkerr/terraform-aws-openshift @dwmkerr
- @dwmkerr Don’t forget to share your toys! …it’ll help you find bugs, get more ideas and build a group to collaborate with.
- @dwmkerr …one more thing…
- @dwmkerr Don’t forget to fall madly in love with Terraform!
- @dwmkerr Tip: Make a microservice manifest …say that fast five times…
- Tip: Make a microservice manifest
- Tip: Make a microservice manifest
- Tip: The developer’s machine comes first @dwmkerr
- Tip: The developer’s machine comes first • Never sacrifice developer experience • Use docker-compose liberally • You should be able to deploy to production from anyone’s machine, as long as they have credentials @dwmkerr
- Tip: Co-locate code and deployment configuration @dwmkerr
- Tip: Co-locate microservice code & deployment config • jenkins / circle / gitlab / whatever • k8s / openshift / swarm / whatever • src / test • README! @dwmkerr
- Tip: Co-locate microservice code & deployment config If I change the logic for my healthcheck, does my config change? @dwmkerr
- Tip: Love the shell @dwmkerr
- @dwmkerr npm install -g tldr …if I can leave you with one gift github.com/dwmkerr/effective-shell
- Thanks! A lot of this stuff is on my blog and GitHub. @dwmkerr dwmkerr.com github.com/dwmkerr linkedin.com/in/dwmkerr
Editor's Notes
- I do three things: Advise on strategy, typical McKinsey stuff Lead our engineering practice A lot of coding
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- 5m mark: Serving clients across the world, deploying in airports, writing code.
- 1. In this case: disjointed teams 2. Slow deployments 3. Black-box solutions 4. Fast UI / Slow Backend 5. Monkey patching
- Monolithic data stores
- Big transformation project C-level sponsorship Capability building
- 10m mark Before, a single monolith database After – microservices Kafka / GraphQL
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- Talk the tech stack over 15m
- We got a lot of stuff wrong, and we are still getting a lot of stuff wrong.
- Understand the commodity step vs the custom step Don’t try to get it right at the beginning, cause you probably won’t
- Everything you see here is hard to test. Bring it into simple scripts, in fact use makefiles.
- Lessons Learned: 2 mins 21/25 mins Pipelines can get incredibly over-complicated very quickly. Most of the time, you need to sets of pipelines: 1. Turn source code into artifacts. 2. Put artifacts somewhere Be agile, think MVP, separate build/orchestrate
- Demo: 0. show the README and table show how there is a makefile in each project run the first two makefiles Show the circle file Show the travis file
- 1. Note: Dockerfile has no app specifics, only environment specifics.
- First, show the Dockerfile on Docker Hub Then, show the makefile Then, show the circle file
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- You have an amazing sandbox, use it! Get genuinely good at ops things, like networking, route tables, DNS etc
- 1. First show the README.md file, explain what the project is for 2.
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- Lessons Learned: 2 mins 21/25 mins Pipelines can get incredibly over-complicated very quickly. Most of the time, you need to sets of pipelines: 1. Turn source code into artifacts. 2. Put artifacts somewhere Be agile, think MVP, separate build/orchestrate
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- No proxies No shit tools Docker-compose for databases, external services, etc. Docker-compose up to bring up dependencies. No environment rubbish in my day to day life please
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- Microservices aint micro unless they contain their config
- Microservices aint micro unless they contain their config
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
- Don’t forget, I’m writing a book!
- Lessons Learned: 2 mins 23/25 mins You cannot simply transplant existing projects onto a platform like kubernetes and expect it to scale. Anecdote: a week of time lost while the team try and get a complex multi-database solution working on Kubernetes, a day to run it on VMs. Lesson: Know what works. Stateless, or self organising homogenous nodes
Advertisement