Conf 2018 Track 3 - Microservices - What I've learned after a year building systems
•Download as PPTX, PDF•
0 likes•121 views
The document discusses lessons learned from building microservice systems. It covers why to use microservices to scale complex systems independently. Key aspects include identifying bounded context services, messaging between services, whether services should be thin and temporal or fat with duplicated data. Testing and deploying distributed systems poses challenges. Monitoring, logging, and configuration management are vital to maintaining live microservice systems.
Recommended
More Related Content
Similar to Conf 2018 Track 3 - Microservices - What I've learned after a year building systems
Similar to Conf 2018 Track 3 - Microservices - What I've learned after a year building systems (20)
More from TechExeter
More from TechExeter (20)
Recently uploaded
Recently uploaded (20)
Conf 2018 Track 3 - Microservices - What I've learned after a year building systems
- 1. Microservices WHAT I’VE LEARNED AFTER A YEAR BUILDING SYSTEMS www.designcoderelease.blogspot.com@NathanGloyn
- 2. @NathanGloyn www.designcoderelease.blogspot.com Who am I? o Independent consultant o 20+ years experience o Primarily .Net & JS
- 3. @NathanGloyn www.designcoderelease.blogspot.com Disclaimer Photo by Clker-Free-Vector-Images on RobinHiggins under CC0 1.0
- 4. @NathanGloyn www.designcoderelease.blogspot.com Agenda Why use microservices? Aspects of microservices Development Testing Data Deployment The reality of a live system
- 5. @NathanGloyn www.designcoderelease.blogspot.com Why use microservices? Photo by Pexels on RobinHiggins under CC0 1.0
- 6. @NathanGloyn www.designcoderelease.blogspot.com Why use microservices? o Don’t! o Scale – nice problem to have o Complexity o Independent development o Technology agnostic
- 7. @NathanGloyn www.designcoderelease.blogspot.com Aspects – Different way of thinking Photo by geralt on pixabay under CC0 1.0
- 8. @NathanGloyn www.designcoderelease.blogspot.com Aspects – Different way of thinking
- 9. @NathanGloyn www.designcoderelease.blogspot.com Aspects – Different way of thinking
- 10. @NathanGloyn www.designcoderelease.blogspot.com Aspects – Different way of thinking
- 11. @NathanGloyn www.designcoderelease.blogspot.com Aspects – Identifying services Photo by quimono on RobinHiggins under CC0 1.0
- 12. @NathanGloyn www.designcoderelease.blogspot.com Aspects – Identifying services o How do you decide what a service consists of? o Domain driven design – Bounded Context o Beware mini-monoliths
- 13. @NathanGloyn www.designcoderelease.blogspot.com Aspects – messaging Photo by quimono on pxhere under CC0 1.0
- 14. @NathanGloyn www.designcoderelease.blogspot.com Aspects – messaging MessageBus
- 15. @NathanGloyn www.designcoderelease.blogspot.com Aspects – messaging M e s s a g e B u s
- 16. @NathanGloyn www.designcoderelease.blogspot.com Aspects – Fat vs Thin Fat man by hahacokies9 on piq under CC BY 3.0 Angry Video Game Nerd by itlivesinthewall on piq under CC BY 3.0 Vs
- 17. @NathanGloyn www.designcoderelease.blogspot.com Aspects – Fat vs Thin o Thin service • Only holds data relating to itself • Calls other services directly as it needed • Temporally coupled o Fat Service • Holds copies of data from other services that it would need to use • Resilient to other service failures • Data coupled
- 18. @NathanGloyn www.designcoderelease.blogspot.com Aspects – Distributed system Photo by geralt on pixabay under CC0 1.0
- 19. @NathanGloyn www.designcoderelease.blogspot.com Aspects – Distributed system o Likely to use multiple services for single “action” o Additional complexity o Fixing errors
- 20. @NathanGloyn www.designcoderelease.blogspot.com Development – Security Photo by Pexels on pixabay under CC0 1.0
- 21. @NathanGloyn www.designcoderelease.blogspot.com Development – Security o HTTPS o Work across all services o Authorization not just authentication
- 22. @NathanGloyn www.designcoderelease.blogspot.com Development – Source code repositories or
- 23. @NathanGloyn www.designcoderelease.blogspot.com Development – Technology Stack Photo by Jeppe Hove Jensen on Unsplash
- 24. @NathanGloyn www.designcoderelease.blogspot.com Development – shared functionality Photo by Jodi McKinney on The Creative Life In Between
- 25. @NathanGloyn www.designcoderelease.blogspot.com Development - UI Photo by Pankaj Patel on Unsplash
- 26. @NathanGloyn www.designcoderelease.blogspot.com Testing CAF Pyro A-Bombby Midlan Airport under CC BY-ND 2.0
- 28. @NathanGloyn www.designcoderelease.blogspot.com Live system – Configuration Cog wheels by Ky0n Cheng under CC0 1.0
- 29. @NathanGloyn www.designcoderelease.blogspot.com Live system – Data Photo by xresch on pixabay under CC0 1.0
- 30. @NathanGloyn www.designcoderelease.blogspot.com Live system – Messaging Green, Blue Yellow and White by Jocelyn Kinghorn used under CC BY-SA 2.0
- 31. @NathanGloyn www.designcoderelease.blogspot.com Live system – Logging Gotwood? by alx_chief used under CC BY-SA 2.0
- 32. @NathanGloyn www.designcoderelease.blogspot.com Live system – Monitoring Binoculars portrait (dscn4659_mod_vign_sm) by gerlos used under CC BY-ND 2.0
- 33. @NathanGloyn www.designcoderelease.blogspot.com Live system – Custom tools Photo by Carlos Irineu da Costa on Unsplash
- 34. @NathanGloyn www.designcoderelease.blogspot.com Summary Different way of thinking Choice of technology not necessarily important but at same time can be crucial Service identification is important and if should be fat or thin dependent upon resilience Build security in from the start Put configuration in one place You will need to alter data in production – know how you are going to do this Log, log and log some more Monitor the system not just individual services
Editor's Notes
- If you’ve built microservices things I’m talking about you probably already know Problems I came across and solutions that were discovered may not be a problem for you or my solution may not be applicable for your circumstances
- Don’t use microservices, just because it’s the “new hotness” doesn’t mean the pattern fits your problem Most people will look to Microservices to solve scaling as they are “easy” to scale, but scaling is a nice problem to have but you have to be successful first Complexity that comes with a distributed system – support, deployment Benefits Independent development – people will point to being able to have multiple teams develop separate parts far easier Technology agnostic
- New paradigm, new way to build Bringing old way of thinking can end up building mini-monoliths
- This is a classic, canonical , microservice - a single “endpoint” that connects to and uses its own data store
- You might have couple of services that share a datastore, the difference being both perform specific independent functions
- This may look like microservices but is closer to a distributed monolith due to the fact that they all share a single database. The main issue is that due to the shared dependency if you want to scale the services the data store can very quickly become a bottle neck if it isn’t able to scale to the same extent, not to mention possibility of collision between data being updated and need to resolve correct values.
- Domain driven design Bounded Context – related concepts e.g. student details from Connections which was not linked to users or security Thoughtworks article on extracting service from monolith - https://martinfowler.com/articles/extract-data-rich-service.html InfoQ article on decomposing monolith - https://www.infoq.com/articles/microservices-startup
- You will need messaging May have messaging on client only
- A server side message bus ties all of the services together allowing them to communicate but without being explicitly coupled to each other Examples: .Net - NServiceBus, Rebus Node – BusMQ Java – EventBus, ApacheCamel RabbitMQ, ZeroMQ,
- Client side message bus – all communication between the services is done client side Postal.js is an example of an in memory message bus written in JavaScript - https://github.com/postaljs/postal.js
- Both types are coupled, just in different ways Temporal coupling – means that if the other service isn’t there then the process may fail if the information/operation being executed is critical to the process Data coupling – if the data gets out of sync service can be executing against invalid data which could then have a domino effect throughout the system
- With multiple services you will often find that to accomplish what appears to the user as one action to call multiple services Everything is just that bit more complicated with a distributed system do not dismiss the additional complexity it will bring This can become more of an issue if you need the system to process some data, to correct a mistake for instance, and you need to be able to orchestrate many compensatory actions.
- Https by default Needs to work across all services Build in from the beginning Ensure authorization in place, not just authentication Existing security products – IdentityServer on .Net, Keycloak on Java, Passport on node, Auth0 etc
- Single repo can make it easy to develop but additional complexity around deployment Multiple repo separates the code making it easier to build and release harder to develop
- Nothing to stop you using what ever technology suits your service If using multiple technologies make sure you have people in the organisation that are able to dev and support those technologies Word of caution related to security – if you do not continue with a particular stack you still need to monitor that stack for vulnerabilities you need to fix
- Package manager Any changes to existing packages or new packages have to be published which may take time e.g. test, PR, build, deploy Pin version unless you are very confident that any update will not effect you
- Interface can end up having an impact on the services, if you design a CRUD based interface more likely you will create services that fit that. Task based UI will help create services related to the actions the users want to perform and often CRUD based services won’t fit this model at all. Single interface means you have to redeploy the entire thing if any one thing changes – monolith front end. Web pack “code splitting” is just chunking up a single interface Micro frontends is the latest name for composable UI where parts of the UI are loaded as necessary, additional complexity around loading parts of the UI as needed and reacting to user actions With many services involved you will be likely to have to deal with Eventual consistency where the system doesn’t have an immediate answer – how do you handle this in the UI?
- Integration testing of the system, or parts/services that interact, is harder – you want a test environment that mirrors your live. Automate as much of the integration testing as you can
- Deploy all services at the same time – fundamentally doing it wrong Deploying individual services – if using single repo then more complex build & release PaaS Vs Container Vs serverless Having the ability to decouple any data migrations from deployment can make life easier if using fat services UI Deployment – ideally completely separate from the backend service, if it can be broken into components and deployed piecemeal even better
- Progression from: config transformed at build/release external config in environment e.g. Azure Cloud Config external config system e.g. Consul, Zookeeper are examples of external configuration systems Enable configuration to be separate from the service – that way you can store the config in one place only Can be single point of failure – ensure you have a default config a service can work from
- Data needed will frequently change over the life of the system Document DB’s not immune to issues, likelihood is you will need to make changes to documents You will need a plan to migrate/alter data – fat services make this more difficult as you need to then run your changes in multiple services If the data is calculated then you will need to be able to perform the calculation outside of the service Multiple tenants – database per tenant or partition the data, – if using database per tenant is that for every service? If so you will end up with a lot of databases to manage
- Format of messages will change over time You want to avoid having to deploy multiple services each time a message changes Version the messages, ensure that services use explicit version Little overhead as need to understand when all services have moved off a version so it can be retired
- Need to be able to look at the system as a whole – ensure a correlation id is included in all log entries to make it easier to do so Centralised logging makes this easier – ELK stack or paid product e.g. splunk Combine with auditing to give a full picture of what is happening Need to be able to look at the logs in context so not just logging the errors as you need everything that was happening at that time Don’t just log server side you need to be logging from the UI as well to get complete picture of what the user is doing
- Can be harder since multiple services make up the system but you don’t necessarily need all of them Ensure you record metrics around performance of the system so that you identify if something is going wrong More difficult than with monolith as all services could be degraded but performance of the system is still ok, baseline your system and then work out what you would consider a problem and add alerts Golden Signals Latency (response time), Traffic (requests per second), Errors (Error rate), Saturation (how overloaded something is queue depth, concurrency, etc) Focus on things that directly impact the user experience good infoq article on these here https://www.infoq.com/articles/monitoring-SRE-golden-signals
- Don’t try and do everything through services – keep services to what they do and build custom tools to help with ad-hoc tasks e.g. replay messages, bulk load data
- Pick the wrong message bus for instance and you could end up with complications