Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Kafka: Legacy microservices
Similar to Kafka: Legacy microservices (20)
Recently uploaded
Recently uploaded (20)
Kafka: Legacy microservices
- 1. 1 Kafka: Smoothly migrate your legacy monolith to your new micro-services oriented architecture. Brice LEPORINI - brice@confluent.io @blep
- 2. 2 Me, myself and I... Brice LEPORINI, 44 Systems Engineer @Confluent Formerly developper : Java, Scala and JS (under threat only)
- 3. 33 Once upon a time, in the retailing business...
- 4. 4 FCC is a fictive retailing company, operating physical points of sales as well as an online platform FCC is currently running its platform since more than 10 years ● Desktop web application ● Mobile APIs ● Back-office applications Its architecture is based on a monolithic application running a RDBMS Key figures $1 B revenue for 2018 40k+ orders per week 1k+ point of sales Aggressive revenue growth target challenges are expected by FCC’s top management for the next 3 years
- 5. 5
- 6. 6 Who said big bang?
- 7. 77 Changing the engine on a plane while in flight
- 8. 88 Gradual migration No big bang Strangler application pattern Features are gradually swapped with new MS Canary release
- 9. 9 The legacy and the micro-services have to run side by side. They can modify concurrently same data. Conflictless!
- 10. 10 Step 1 : push changes from legacy to MS @blep
- 11. 11 A sample from a broader physical model @blep
- 12. 12 Step 1 : push changes from legacy to MS @blep
- 13. 1313 Demo time!
- 14. 14 Why Kafka? • Scalability • Retention • Durability • Replication • Security • Resiliency • Throughput • Ordering • Exactly Once Semantic • Transaction • Idempotency • Immutability • … @blep
- 15. 15 Step 1 : push changes from legacy to MS @blep
- 16. 16 @blep
- 17. 17 Step 1 : push changes from legacy to MS @blep
- 18. 18 A few words about duality of Streams and Tables @blep
- 19. 19 Step 1 : push changes from legacy to MS @blep
- 20. 20 Why Kafka Streams? Higher level API: transformation, filtering Stateful features: ● Aggregations ● Joins ● Windowing All are resilient and able to scale out! @blep
- 21. 2121 Demo time!
- 22. 22 @blep
- 23. 23 Step 2: push changes from MS to legacy @blep
- 24. 24 Step 2: push changes from MS to legacy @blep
- 25. 25 Entity state in MS environment = Last entity found in the snapshot + Events without validation
- 26. 26 Yes it is a kind of CQRS + ES usage CQRS = Command Query Responsability Segregation ES = Event Sourcing
- 27. 27 @blep
- 28. 28 @blep
- 29. 29 I tried to sell you a dream! CDC highly needs your attention in different ways depending on the DB you use JSON usage makes data governance at risk Needs more code to support updates This code for presentation purpose ! There’s still a lot of corner cases to cover! @blep
- 30. 3030 But when you cover all cases, this kind of architecture rocks!
- 31. 31 In conclusion No big bang ✅ Gradual migration ✅ Ability to rollback ✅ No production stop ✅ Derisked data migration ✅
Editor's Notes
- I’m going to make the case that databases and software architectures are only 50% complete, that there is a fundamental missing ingredient that we don’t represent and that capturing this will be a major leap forward in how we think about the architecture of data systems and of companies and that Kafka is they key building block spurring this change. This is a little bit of an outrageous claim. After all there is nothing really new in computer science and we’ve been building systems for a while, if there was something this fundamental we’d know about it, right? Well in a way we do. What I’m talking about is of course....