Monolith to microservices - our journey
•Download as PPTX, PDF•
1 like•492 views
This document summarizes a company's journey from a monolithic architecture to microservices from February 2014 to present day. It started with 2 apps on a single backend API in February 2014, growing to 8 apps by December 2014 running on an ever-growing monolithic backend. To address scalability and development challenges, the company transitioned to a microservices architecture running containerized services communicating over RabbitMQ. They also implemented centralized logging and monitoring with FlightControl.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (18)
Similar to Monolith to microservices - our journey
Similar to Monolith to microservices - our journey (20)
Recently uploaded
Recently uploaded (20)
Monolith to microservices - our journey
- 1. Monolith to Microservices – Our Journey
- 2. February 2014 • 2 Presentation Layers: • Customer Services Backend • Simple Marketing Website • Single Backend API • Hosting: • 3 apps on Heroku • 1 engineer • c. £80 / day in transactions API MySQL (Heroku addon) Customer services Marketing site
- 3. March 2014 • 4 Presentation Layers: • Customer Services Backend • Online Booking Website • Hotel Booking Interface • Therapist Mobile Website • Single Backend API • Hosting: • 5 apps on Heroku • 1 engineer • c. £250 / day in transactions API MySQL (Heroku addon) Neo4J (Heroku addon) Customer services Online booking Hotel booking Therapist site
- 4. December 2014 • More Presentation Layers: • Customer Services Backend • Online Booking Website • Hotel Booking Interface • Therapist Responsive Website • Therapist Mobile Apps • Consumer iOS App • Consumer Android App • Hotel Group Login Interface • Ever-growing Monolithic Backend API • Hosting: • 8 apps on Heroku Starting to get expensive… • 2 engineers • Over £1.5k / day in transactions Customer services Online booking API MySQL (Heroku addon) Hotel booking Therapist site Neo4J (Heroku addon) redis (Heroku addon) Hotel groups Consumer iOS Consumer Android Therapist Android Therapist iOS Native apps RabbitMQ(EC2)
- 5. Time for a new architecture…
- 6. Monoliths Pros • Simple ops tasks (deployment, scaling) • Very fast development • Single tech stack (easy to understand) Cons • Harder to add new features with stability confidence • Overwhelming to develop • Long build +test times • Long term commitment to tech stack • Larger instance sizes
- 7. Microservices Pros • Tiny applications = tiny build + test time • Pick and choose stack for each case + easy to rewrite a service • More efficient scaling • Independent deploy processes • Prevent faults in one service affecting entire platform Cons • More applications to deploy • Inter-service communication has to be dealt with • Harder to aggregate logs and trace across systems • Not necessary for a simple codebase / platform
- 8. Where to host? • Practically zero learning curve • Expensive to run at large scale • No access to OS / VM • Git-push deploys can cause issues when bugs arise (CI / build pipeline) • Super configurable networking layer (security!) • Easy to go ‘back to OS’ for trickier setups • Attractive startup programme • Great services infrastructure (RDS / EC etc)
- 9. More applications to deploy • Problem: – More applications means more ops work to do to deploy and scale efficiently • Solution: – Containerize your apps (stop thinking about instances) – Automate your deployment, uptime management and scaling
- 10. Empire (PAAS) emp deploy user/repository:tag Deployments emp set VARIABLE=value -a app-name Environment Variables emp scale web=3:CPU:RAM -a app-name Scaling emp ps -a app-name Process State emp rollback v21 -a app-name Fast Release Rollback github.com/remind101/empire
- 11. Inter-service communication • Problem – Maintaining knowledge of traditional REST API endpoints & DNS names can grow out of control with lots of services involved • Solution – Use a centralised service bus (e.g. RabbitMQ)
- 12. RabbitMQ Service Bus Server application Client application npmjs.com/package/rabbitr
- 13. Logs and tracing through multiple layers • Problem – You need to view logs for all your applications – Nested requests can become difficult to trace • Solution – Use a standardised logging & tracing solution that can link requests through multiple application levels FlightControl
- 15. Now Customer services Online booking MySQL (RDS) Hotel booking Therapist site redis (ElastiCache) Hotel groups Consumer iOS Consumer Android Therapist Android Therapist iOS RabbitMQ Cluster on EC2 Therapist API gateway Customer + Hotel API gateway Microservice A NodeJS MySQL (RDS) redis (ElastiCache) Microservice B NodeJS Microservice C PHP MySQL (RDS) redis (ElastiCache) Microservice D NodeJS Postgres (RDS) Microservice E NodeJS Andmore… S3
Editor's Notes
- Good morning everyone, my name is Jack Tang and I'm the co founder and CEO of UrbanMassage.com We launched urban massage with a simple goal: giving our customers a solution to conveniently and easily treat themselves to all the healing properties a massage can offer. We firmly believe that looking after your body and mind is more important than ever in today’s fast-paced world, and we are dedicated to making wellbeing accessible, affordable and available for everyone. This is why we've built urbanmassage.com where you can book our vetted and qualified therapists to your home within the hour. Our technology has given us an unfair advantage to the growth and loyalty of our customers for the past 18 months. Now I pass you to my co founder and CTO Giles Williams to tell you a little more about our technical infrastructure
- To give an idea of the scaling of the product that lead us to look into a new architecture
- Added online booking for customers, an interface for hotels to book via, plus a website for therapists to manage their appointments Also started expermenting with Neo4J for our logistics logic Up to 5 apps on heroku now
- Just coming out of beta By this point we’re up to 8 apps on heroku, we’ve started exploring other Heroku addons for caching etc Now also supporting native apps for customers and therapists You can see we’ve started simplifying parts of our API by using RabbitMQ to perform some pub/sub work Eg. Create a booking controller, used to contain logic to email customer, sms therapist, clear stats caches, send a notification to our customer service chat room Now broken out into separate modules that subscribe to a ‘booking created’ event
- At this point we realised we needed to investigate other possibilities for our backend architecture so started comparing our current monolithic system to a micro services approach
- * Deploy and scale commands are simple – you only have one application to worry about * At first, development is pretty quick, you can just keep bolting features on top * Having a single tech stack can help if you only have one or two people on the project --- * It becomes hard to decouple parts of the system so changes in one area don’t affect other areas * Takes a while for developers to learn enough about the platform to be productive * Your builds and tests will take a while as you have to rebuild everything at once * It’s hard to experment with new technologies * No control over which parts of the app you scale, so you end up scaling everything disproportionaly
- Quick to run build and tests - More fine grained control over which parts of the ecosystem you scale Easy to deploy incremental changes to parts of the platform Sneaky prod bugs won’t bring whole platform down – e.g. customer entity output doesn’t affect customer support - Not necessary for simple codebases – website with a form that sends an email
- Heroku Pro: * If you know git, you can deploy your work Heroku cons: * Becomes expensive as you scale – you get more bang for your buck using say a t2.micro or small on EC2 than a dyno for the same price * Sometimes you need to run something custom, such as OpenOffice on an instance * Having to commit code if a bug crops up and you need to do a really quick fix can become an issue as your team scales AWS: * Ensuring good security between services is a lot easier when you have more control over the network – basically everything is publicly exposed on Heroku * Custom VM setups are possible * Really great startup programme including options of credit, support and advice along the way * Services infrastructure provides Heroku-addon style ease for managing DBs, cache, static file storage etc -------- Now that we’ve pretty much decided on AWS as our IaaS provider, let’s walk through a few of the issues we mentioned with the MicroServices archicture and some of the solutions we came up with
- Read problem….. * Containerisation can help you guarantee portability of your apps between your local dev machine and hosted environments * Automation will stop you having to delve into instances every time you want to deploy, or an instance drops out of a cluster
- Mention: Hackathon (date TBC) CTO slack group