Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

"Microservices Lessons Learned" talk at Voxxed Days Microservices, Paris

213 views

Published on

The journey from monolith to microservices is different for every organization. A variety of challenges come with introducing microservices itself, but also organizational circumstances impacting the transformation that needed to be considered.

In this talk I would like to share some microservices lessons learned from a startup perspective - and in hindsight, what to watch out for if starting the journey again.

Published in: Technology

"Microservices Lessons Learned" talk at Voxxed Days Microservices, Paris

  1. 1. Microservices Lessons Learned CTO at Just Software @JustSocialApps Susanne Kaiser Independent Tech Consultant @suksr @suksr#VoxxedMicroservices
  2. 2. Each Journey is Different “People try to copy Netflix, but they can only copy what they see. They copy the results, not the process.” Adrian Cockcroft, AWS VP Cloud Architect, former Netflix Chief Cloud Architect @suksr#VoxxedMicroservices
  3. 3. Each Journey is Different Team Structure Skillset Size Journey Affecting Circumstances @suksr#VoxxedMicroservices
  4. 4. Each Journey is Different Team Structure Skillset Size Journey Legacy Maintenance effort Runtime environment Affecting Circumstances @suksr#VoxxedMicroservices
  5. 5. Each Journey is Different Team Structure Skillset Size Journey Legacy Maintenance effort Runtime environment Strategy New Features Timeline / Milestones Affecting Circumstances @suksr#VoxxedMicroservices
  6. 6. Background JUST DRIVE JUST CONNECT JUST LIST JUST WIKI JUST PEOPLE JUST NEWS JUST SOCIAL @suksr#VoxxedMicroservices
  7. 7. One team Single Unit One collaboration product One technology stack At the Beginning A Monolith in Every Aspect @suksr#VoxxedMicroservices
  8. 8. Background After an Evolving Time @suksr#VoxxedMicroservices
  9. 9. Background After an Evolving Time Productivity suffered @suksr#VoxxedMicroservices
  10. 10. Background After an Evolving Time Productivity suffered Usability/UX suffered @suksr#VoxxedMicroservices
  11. 11. Background After an Evolving Time Productivity suffered Usability/UX suffered New features released slowly @suksr#VoxxedMicroservices
  12. 12. JUST DRIVE JUST CONNECT JUST LIST JUST WIKI JUST PEOPLE JUST NEWS JUST SOCIAL Background Separate Collaboration Apps @suksr#VoxxedMicroservices
  13. 13. Background Separate, Autonomous Teams Well-defined responsibilites @suksr#VoxxedMicroservices
  14. 14. Background In The Long Run Organisation Product Software @suksr#VoxxedMicroservices
  15. 15. Background Our Motivation for Microservices Autonomous teams Develop independently Deploy independently Work at different parts independently Scale independently At different speed @suksr#VoxxedMicroservices
  16. 16. Identify Bounded Contexts Decomposition Strategy High cohesion within a service Loose coupling between services @suksr#VoxxedMicroservices
  17. 17. Identify Bounded Contexts Decomposition Strategy High cohesion within a service Loose coupling between services Bounded Context Related behaviour Semantic boundary around domain model Well-defined business function @suksr#VoxxedMicroservices
  18. 18. Identify Bounded Contexts Decomposition Strategy JUST DRIVE JUST CONNECT JUST LIST JUST WIKIJUST PEOPLE JUST NEWS Bounded Contexts @suksr#VoxxedMicroservices
  19. 19. JUST DRIVE Decomposition Strategy Co-Existing Service From Scratch @suksr#VoxxedMicroservices
  20. 20. JUST DRIVE Decomposition Strategy Co-Existing Service From Scratch JUST PEOPLE @suksr#VoxxedMicroservices
  21. 21. Decomposition Strategy Co-Existing Service From Scratch owns document state REST API Application-Service Domain-Model DB Adapter Monolith JUST DRIVE @suksr#VoxxedMicroservices
  22. 22. Decomposition Strategy Co-Existing Service From Scratch owns document state owns profile state document created by author Monolith REST API Application-Service Domain-Model DB Adapter @suksr#VoxxedMicroservices
  23. 23. owns document state owns profile state Events local copy of author Message Broker Decomposition Strategy Co-Existing Service From Scratch REST API Application-Service Domain-Model DB Adapter Message Broker Adapter Monolith publish subscribe @suksr#VoxxedMicroservices
  24. 24. DB Adapter Message Broker Adapter Application-Service Domain-Model REST API Domain-Event Good approach in general, but we did too many steps at once New UI New Business Logic New Data Structure => Not optimal to start with vs. Decomposition Strategy Co-Existing Service From Scratch @suksr#VoxxedMicroservices
  25. 25. Incremental Top Down Extracting Web App Decomposition Strategy Monolith REST API REST API REST Client @suksr#VoxxedMicroservices
  26. 26. Monolith REST API Extracting Business Logic Application-Service Domain-Model DB Adapter REST Client Incremental Top Down Decomposition Strategy Monolith uses extracted business logic @suksr#VoxxedMicroservices
  27. 27. Monolith Splitting Data Storage Incremental Top Down Decomposition Strategy REST API Application-Service Domain-Model DB Adapter Events Message Broker Message Broker Adapter publish subscribe @suksr#VoxxedMicroservices
  28. 28. Which One First? vs. Easy to Extract Changing Frequently Different Resource Consumption Early experiences w/ Microservices Greatest benefit after extraction Decomposition Strategy @suksr#VoxxedMicroservices
  29. 29. Cross-Cutting Concerns Authorization JUST DRIVE JUST WIKI Fine-grained authorization Inter-service dependency @suksr#VoxxedMicroservices
  30. 30. Cross-Cutting Concerns I have a new service that needs authorization. Where is the authz service I could use? Not there, yet. Sorry! Ok, then I am putting my code to the place where authz handling exists … to the monolith. Feeding the monolith Re-implementing authz w/ every new service Ok, then I am implementing authz in my local service. Authorization @suksr#VoxxedMicroservices
  31. 31. Cross-Cutting Concerns Handle Them Early Feeding the monolith Re-implementing authz w/ every new service Handle Cross-Cutting Concerns Early @suksr#VoxxedMicroservices
  32. 32. Cross-Cutting Concerns Avoid A Distributed Monolith Authz Service @suksr#VoxxedMicroservices
  33. 33. Cross-Cutting Concerns Avoid A Distributed Monolith Authz Servicetranslate One stable common contract translate translate @suksr#VoxxedMicroservices
  34. 34. Service Interaction Request-Driven / Event-Driven command query Events Message Broker publish subscribe command query Request-Driven Hybrid Events Message Broker publish subscribe Event-Driven @suksr#VoxxedMicroservices
  35. 35. How To Manage Shared Data? Hybrid Model Message Broker REST API Remote query directly to source Events for notification @suksr#VoxxedMicroservices
  36. 36. Event Driven State Transfer Message Broker Local copy of profile data ProfileUpdatedEvent How To Manage Shared Data? Events for data duplication @suksr#VoxxedMicroservices
  37. 37. Source Of Truth How To Manage Shared Data? Internal source of truth External source of truth Multiple sources of truth Single source of truth Events as first-class citizens “Traditional” Event-Driven System Event Store Event Log @suksr#VoxxedMicroservices
  38. 38. Messaging System Storage System Streaming Platform Message Broker Event Log Event Stream @suksr#VoxxedMicroservices
  39. 39. How To Manage Shared Data? Kafka Streams Unbounded, ordered sequence of data records Key-value pairContinuously updating @suksr#VoxxedMicroservices
  40. 40. How To Manage Shared Data? Kafka Streams Kafka Topic Lightweight embedded state store (disk-backed) Service Running in the same process of Microservice Loaded on startup of Microservice Streams make data available wherever it’s needed @suksr#VoxxedMicroservices
  41. 41. How To Manage Shared Data? Kafka Streams Kafka Streams API join filter group by aggregate etc. Changelog of state changes KStream KTable Snapshot of the latest value for each key Kafka Stream-Table Duality (key1, value1), (key2, value2), (key 1, value 3) key1 value3→ key2 value2→ @suksr#VoxxedMicroservices
  42. 42. How To Manage Data? Materialized Views w/ Kafka Streams Kafka Table for enrichment Document Service Stream REST API Materialized View as State Store Stream-Table-Join @suksr#VoxxedMicroservices
  43. 43. How To Manage Shared Data? Event Streams as a Shared Source of Truth Events for notification Events for data duplication ● Simple integration ● Remote query => increasing coupling ● Eliminating remote query => better decoupling ● Local copy => better autonomy ● Duplicating effort to maintain local dataset ● Simple integration ● Remote query => increasing coupling Event streams as a shared source of truth ● Eliminating local copy => reduces duplicating effort ● Pushes data to where it’s needed ● Increases pluggability ● Low barrier to entry for new service @suksr#VoxxedMicroservices
  44. 44. Hardware Data Store API API-Gateway Service Discovery Load-Balancer Message Broker Timeout-Handling Retries Idempotency Bulkheads Circuit Breaker Config-Mngmt. Monitoring Log Aggreation Metrics Distributed Tracing Health Checks SCM O/SVirtualization Container Runtime Checkout TestBuild CI/CD Pipeline Deploy µService Backup Recovery @suksr#VoxxedMicroservices Infrastructure Complexities
  45. 45. Build the things that differentiate you Offload the things that don’t @suksr#VoxxedMicroservices
  46. 46. Hardware O/S Virtualization Container Runtime Managed Services O/S Orchestration Data Store µService Offload by getting common building blocks managed by cloud providers
  47. 47. Start small Lessons Learned Handle cross-cutting concerns early Avoid a distributed monolith Be aware of affecting circumstances & Each journey is different :) Design event-driven & consider event streams as shared source of truth @suksr#VoxxedMicroservices Consider managed services to offload infrastructure complexities
  48. 48. Susanne Kaiser Independent Tech Consultant @suksr CTO at Just Software @JustSocialApps

×