Successfully reported this slideshow.

Moving to microservices – a technology and organisation transformational journey

23

Share

Loading in …3
×
1 of 40
1 of 40

More Related Content

Similar to Moving to microservices – a technology and organisation transformational journey

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Moving to microservices – a technology and organisation transformational journey

  1. 1. Moving to microservices – a technology and organisation transformational journey Boyan Dimitrov, Platform Automation Lead @ Hailo @nathariel
  2. 2. FROM LONDON TO TOKYO, FROM MADRID TO OSAKA.
  3. 3. The beginning…
  4. 4. Back in 2011 we started simple System overview: • Started on AWS • PHP frontend and Java backend applications • Built and supported by a small team: 3-4 backend engineers MySQL
  5. 5. And then we started expanding rapidly • City-specific environments • Branching the code base • Manually building infrastructure and configuration MySQL PHP Java MySQL PHP Java MySQL PHP Java
  6. 6. As we grew, getting features out became a challenge We quickly found out that extending monoliths is hard: • Hard to maintain the codebase • Any new feature took weeks to deliver • Hard to scale the dev teams Failure to deliver business value Performance
  7. 7. Operating a monolith in the cloud got even harder A lot of development and ops time wasted in firefighting: • Lack of automation • Multiple SPOFs • No proper monitoring MySQLMonolith Monolith • Unclear responsibilities • No well defined escalation or ownership process – many non actionable alerts transformed into “all hands on deck” actions
  8. 8. So in 2013 we ended up doing… Monolith
  9. 9. We wanted to build a global platform • Everything had to be automated – any workflow, any action • Everything had to be resilient and self-healing – regardless of the failure source: infrastructure, network or code • Each service had to be responsible for one thing and one thing only Looked at what we did wrong and redesigned it: Key challenges: • We decided to move from PHP & Java to Go • We had to build everything from scratch but move all our production traffic without any downtime • We had to change our culture as we go
  10. 10. eu-west-1 Message Bus+ API Gateway C* us-east-1 API Gateway C* Message Bus+ Go Service Go Service Go Service Go Service Go Service Go Service Go Service Go Service Go Service Go Service Go Service Go Service
  11. 11. We started with the building blocks Logic Storage Library for abstracting service-to-service comms service-layer Handler platform-layer Self-configuring external service adapters A service under the hood: • Service to service communication libs • Discovery • Configuration • A/B testing capabilities • Monitoring & Instrumentation • … and much more Any service gets for free:
  12. 12. In preparation for the migration Introducing a smart API Gateway made our life easy: • Let us do a transparent, seamless migration from user perspective • Gave us a lot of flexibility about how we route our traffic • Enabled us to build a lot of failover capabilities API Gateway Monolith
  13. 13. And then we started breaking down our monoliths • We aimed to get production traffic on the new platform as quickly as possible • We identified the low hanging fruit first and rewrote them • We kept iterating on our platform and building more tools as we needed API Gateway Decouple
  14. 14. At present we have • Microservices ecosystem (99.9% written in Go) • Designed specifically for the cloud – different building blocks and components will constantly be in flux, broken or unavailable • 1000+ AWS instances spanning multiple regions • 200+ services in production
  15. 15. The Platform Troll a platform by Swinsto101 / CC BY-SA 3.0 / Desaturated from original
  16. 16. TeVPC Auto Scaling S3 OrchestrationEnv DNS Release AutoScaling Discovery Monitoring CFEC2 Route 53 Redshift ComputeEIP Routing Core Platform Provisioning Login Services Cloud Provider Whisper Config
  17. 17. • Lowest level building blocks • We mostly use basic PaaS components and services as they cover most of our needs • We expect every underlying component to fail and we designed for this TeVPC Auto Scaling S3 CFEC2 Route 53 Redshift Cloud Provider
  18. 18. eu-west-1 Proxy Layer Message Bus eu-west-1a Services eu-west-1b eu-west-1c Shared Infra RabbitMQ RabbitMQ RabbitMQ API API API Go Go Go x many C* NSQ ZK C* NSQ ZK C* NSQ ZK x many x many
  19. 19. • We use auto scaling groups for everything  Guarantees each component can be rebuilt automatically  Including our database clusters that run on ephemeral storage ( we do keep 6 copies of each piece of data in 2 regions ) • Minimum of 3 AZs in every region • Every workflow is automated • Every component has to be self-healing and scalable Core principles
  20. 20. • Our “cloud provider abstraction” layer • Main purpose is infrastructure and workflow automation and discovery • Has a global view of everything happening across our infrastructure • Provides additional capabilities on top of AWS • The only services directly aware of our cloud provider specifics – gives us a lot of flexibility and let us introduce changes quickly OrchestrationEnv DNS Release AutoScalingComputeEIP Whisper
  21. 21. Everything in our platform emits events So naturally we want to capture all external events as well!
  22. 22. Whisper Service It’s all about event driven compute – think Lambda but within our platform Events Events Hundreds of publishers & subscribers NSQ Topics Events External sources Actions To subscribe to any new event source we have to only change a single service
  23. 23. Provides the most essential platform functions for every service: • Service Discovery • Service Provisioning • Routing & Load Balancing • Authentication/Authorization • Monitoring • Configuration Discovery Monitoring Routing Core Platform Provisioning Login Config
  24. 24. • Self-contained units of execution • Built around business capabilities or domain objects • Small enough to be rewritten in a few days • Independently scalable • They are all about adding business value Services
  25. 25. All good but did we make our development any faster?
  26. 26. Up and running in seconds Vetted & Tested & Built ~ 100 – 140 sec Setup ~ 1 sec Trigger a build ~ 1 sec
  27. 27. Deploying a service service name version auto scaling group
  28. 28. Deploying a service
  29. 29. Service A Service A Traffic 5% 95% Smart traffic shaping
  30. 30. LIVE We can get a completely new service in production in hours Hailo Warp Speed ( measured in many μs or something )
  31. 31. What about operating our platform?
  32. 32. Live traffic tracing
  33. 33. Microservices are all about the tooling!
  34. 34. Live request tracing
  35. 35. Live request tracing
  36. 36. Key Learnings • Automate everything – it enabled us to do more with less • Identify your KPIs and track them • Invest in tooling: the complexity in a microservices architecture is not in your application code anymore - it is in the thousands of service interactions! • Empowering our engineers increases your velocity tremendously! • Moving to microservices is a journey – make sure you take everyone onboard!
  37. 37. Thanks! @nathariel boyan@hailocab.com @HailoTech

Editor's Notes

  • Seamless user experience
  • From London to Singapore, From Barcelona to Tokyo
  • Cloud
  • ×