Enabling independent teams by creating decoupled data flows
•Download as PPTX, PDF•
0 likes•102 views
This document outlines an organization's journey to enable independent teams by decoupling data flows. It began with all development done by one centralized team with tight dependencies. To scale, the organization split into multiple teams but they remained tightly coupled through a shared database. The organization is now moving to an event-driven architecture with decoupled services communicating asynchronously through event streams. This will allow independent teams, improved scalability, and reduced operational costs by removing dependencies between services. Challenges include ensuring idempotency and handling events at least once.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Enabling independent teams by creating decoupled data flows
Similar to Enabling independent teams by creating decoupled data flows (20)
More from confluent
More from confluent (20)
Recently uploaded
Recently uploaded (20)
Enabling independent teams by creating decoupled data flows
- 1. Enabling independent teams by creating decoupled data flows Sven Herzing Chief Technology Officer
- 2. About us We’re on a mission to build MENA’s most effective Product- Led organization ● Largest delivery + quick commerce player in MENA ● Active in 9 markets ● Headquartered in Dubai ● 2 tech hubs
- 3. 350+ Team, 50+ Nationalities, 2 tech hubs
- 4. Let’s take a step back and start at the beginning ● Founded in 2004 in Kuwait ● Fast paced startup, development all done by one team ● Everybody know all moving pieces, focus is on fast delivery
- 5. Let’s take a step back and start at the beginning Business team request Delivery team Business team request Business team request
- 6. The organization grew Business team request Delivery team Business team request Business team request Delivery team Delivery team
- 7. High Level Architecture Vendor Management Consumer API CMS Management Main Database iOS Android Web
- 8. Delivery Hero enters the picture In 2016, talabat became part of the larger delivery hero group In 2019, we started our journey into a tech and product organization
- 9. Our journey timeline 2019 Goal: Move from “delivery team” model and building a tech & product organisation Milestones: - Team size is at 80 - We process 140 orders per minute 2020 Goal: Transform towards self-organised, empowered teams model Milestones: - Team grows to 210 - We process 300 orders per minute 2021 Goal: Building the right thing, in the right way, where speed is enabled by engineering culture and tech excellence Milestones: - Team grows to 250 (mid year) - We process 500 orders per minute
- 10. Architecture evolves as you grow Vendor Management Consumer API CMS Management Main Database iOS Android Web Service A Service B
- 11. Architecture evolves integrate with global services Consumer API Main Database Order Transmission Logistics Restaurant tools Driver app ● Order Transmission dispatches orders to global services ● Writes update from global services back to main database that downstream services have the data
- 12. Our journey timeline 2019 Goal: Move from “delivery team” model and building a tech & product organisation Milestones: - Team size is at 80 - We process 140 orders per minute 2020 Goal: Transform towards self-organised, empowered teams model Milestones: - Team grows to 210 - We process 300 orders per minute 2021 Goal: Building the right thing, in the right way, where speed is enabled by engineering culture and tech excellence Milestones: - Team grows to 250 (mid year) - We process 500 orders per minute
- 13. Functional scope is growing Consumer API Main Database Order Transmission Logistics Restaurant tools Driver app Rewards Order information Reorders
- 14. Functional scope is growing ● Orders are placed in main database ● Record update in database to distribute status update ○ Read on demand or DB polling on read replica ● Tight coupling on DB schema between teams ● Scheduled time for deployments due to DDL changes ● Very limited scalability
- 16. Our journey timeline 2019 Goal: Move from “delivery team” model and building a tech & product organisation Milestones: - Team size is at 80 - We process 140 orders per minute 2020 Goal: Transform towards self-organised, empowered teams model Milestones: - Team grows to 210 - We process 300 orders per minute 2021 Goal: Building the right thing, in the right way, where speed is enabled by engineering culture and tech excellence Milestones: - Team grows to 250 (mid year) - We process 500 orders per minute
- 17. A closer look on our dependencies ● Most services need order and vendor data to function ● Teams need to alter database schemas to achieve their objectives ● Most code is dependent on the existence of the monolithic database ● A bit more philosophical, but the database was our orchestrator Adding more people to the team didn’t make us faster, but made the problem more pressing
- 18. What we are aiming for ● Allow our teams to build services with fewer dependencies and act independently ● Enable operational speed by the decoupling of functionalities ● Allowing our organization to scale in order volume and team size ● Reducing operational cost
- 19. Rethinking scalability ● Database coupling hinder speed ● HTTP request fanout doesn’t scale ● Queues work, but it needs one for each service ● Data is still required ● Coupling between teams and services need to be resolved
- 20. Some principles for services ● Low coupling, high cohesion ● No service can depend on another service for data availability, functionality or uptime ● Events are first class citizens ● Choreography over orchestration The full list is actually 10, but we don’t need all of them right now
- 21. The journey to decoupling information Consumer API Main Database Order Transmission Central service integration Rewards Order information Nexus order service Orders (shadow mode) Orders Database Compare Orders To ensure data consistency Reorders User service
- 22. The journey to decoupling information Consumer API Main Database Order Transmission Central service integration Rewards Order information Nexus order service Orders Orders Database Reorders User service Write orders For compatibility
- 23. The journey to decoupling information Consumer API Main Database Order Transmission Central service integration Rewards Order information Nexus order service Orders Orders Database Reorders User service Write orders For compatibility Write orders For compatibility Order Stream ● Create fat events that satisfy needs of downstream services ● Use event schema registry to ensure all events conform to schema Order information
- 24. The journey to decoupling information Consumer API Main Database Order Transmission Central service integration Rewards Order information Nexus order service Orders Orders Database Reorders User service Write orders For compatibility Write orders For compatibility Order Stream Order information
- 25. Why this matters ● Allow our teams to build services with fewer dependencies and act independently ○ Events have full data, reducing dependencies ○ If others teams need data, it will be provided on a stream and can be cached locally ● Enable operational speed by the decoupling of functionalities ○ All services operate fully independent, not central DDL couples them ○ Event schema ensure structure while enabling decoupling ○ If a service is unavailable, events can be consumed later
- 26. Why this matters II ● Allowing our organization to scale in order volume and team size ○ No single database with all data is required ○ Downstream services can be added independently without adding fanout http requests at the origin ● Reducing operational cost ○ Local database can be smaller ○ Data retention can be optimized for each service
- 27. New Challenges ● Idempotency handling is required ● At least once semantics are more complex ● Event Schemas need more upfront thinking in design to make the best out of the solution
- 28. Future ● Each vertical can have their own order service, creating multiple producers of events on the same stream ● The source of truth will be the stream ● Disaster recovery will become simplified in setup by reducing data dependencies
- 29. Thank you.
Editor's Notes
- Tell a bit about the background 8 years in the region Talabat Careem Dubizzle
- Today our team is a multi national team that grew from humble beginnings with a few engineers in Kuwait to around 75 in 2019 Experience massive growth between 2019 and 2022
- Like many companies, it worked like this
- And as the organiation grew, it still worked like this
- High level architecture. All centered around a monolithic database Restaurant data and content into the DB, read on the consumer api
- Team size grows, it becomes harder to work in the single codebase Service are all around the world a big thing Services start to emerge, but data still coupled to the main database Easier to work on smaller code Load is still on a database, mitigated by creating read replicas Tight coupling on database level
- But let’s have a look at a different view, Database based synchronization and update mechanism Polling on main DB by order transmission, updates existing order records It works and is simple But Single point of failure Doesn’t scale well Everybody needs to have in depth knowledge of the whole system Tight coupling
- Functional scope grows Order information still centralized Updates distributed to via database updates
- A complex connected system has connections between areas. They define how well a team can work independently Each column is a team and each card is an ask from a different team Commitments are hard to keep when extra work is needed to support others
- You start to think about why we get more and more dependencies, while progress starts to feel slower Point on the right center is orders The big spot in upper center is vendor information
- Those are actually from the dubizzle times Full list Low coupling, high cohesion No service can dependent on antoher for uptime, functionality or data availability On transactions can span at most one service No vertical can impact the stability of another Bounded contexts are defined by business realms and not CRUD apis Choregraphy over orchestration Events are first class citizens
- Create a new service that can process orders, but keep the old one Process orders as before, updates to services as still in the same order object The service has no longer access to all types of information an order object might need, so the model need to contain all this data for each order Examples are user information Vendor address or opening hours Driver details
- Create a new service that can process orders, but keep the old one Process orders as before, updates to services as still in the same order object The service has no longer access to all types of information which an order object might need, so the model need to contain all this data for each order Examples are user information Vendor address or opening hours Driver details Avoid requests from Transmission to user service to fetch user data
- Create a new service that can process orders, but keep the old one Process orders as before, updates to services as still in the same order object The service has no longer access to all types of information which an order object might need, so the model need to contain all this data for each order Examples are user information Vendor address or opening hours Driver details Avoid requests from Transmission to user service to fetch user data
- Create a new service that can process orders, but keep the old one Process orders as before, updates to services as still in the same order object The service has no longer access to all types of information which an order object might need, so the model need to contain all this data for each order Examples are user information Vendor address or opening hours Driver details Avoid requests from Transmission to user service to fetch user data