Tudip's CTO Mr. Tushar Apshankar spoke about 'Database design for containerized microservices' in NASSCOM product conclave at Pune. Here is Slide Share for the same:)

1. 1
Tudip Technologies
Continuously Delivering Digital Solutions with
Integrity | Innovation | Serenity
https://tudip.com/

2. Database Design for
Containerized Microservices

3. The world where we live in...
Organizations which design
systems … are constrained to
produce designs which are copies
of the communication structures of
these organizations.
- M. Conway
Corollary by
James and Neil:
Therefore: Make sure the
organization is compatible
with the product architecture.
“
”

4. © 2019 Tudip Technologies. All Rights Reserved.
OTT System in Microservice Architecture

5. What is Microservice Architecture?
● As per Martin Fowler, services in a
microservice architecture (MSA) are often
processes that communicate over a network to
fulfill a goal using technology-agnostic
protocols such as HTTP.
● Services are independently deployable.
● Services are organized around fine-grained
business capabilities.
● Services can be implemented using different
programming languages, databases, hardware
and software environment, depending on what
fits best.
● This does not mean that a single microservice
is written in a patchwork of programming
languages.

6. © 2019 Tudip Technologies. All Rights Reserved.
• Multiple Components
• Built For Business, instead of technology
• Simple Routing
• Decentralized
• Failure Resistant
• Evolutionary
Salient features of Microservices

7. © 2019 Tudip Technologies. All Rights Reserved.
Loose coupling is the operative word, that means microservices should be able
to be modified without requiring changes in other microservices.
Overall Microservices

8. © 2019 Tudip Technologies. All Rights Reserved.
Microservices Pros
• Freedom to independently develop and
deploy services
• A microservice can be developed by a
fairly small team
• Code for different services can be written
in different languages (though
practitioners discourage it)
• Easy integration and automatic
deployment
• Easy to understand and modify for
developers, thus can help a new team
member become productive quickly
• The developers can make use of the
latest technologies
• The code is organized around business
capabilities
• When change is required in a certain part
of the application, only the related service
can be modified and redeployed—no
need to modify and redeploy the entire
application
• Better fault isolation: if one microservice
fails, the other will continue to work
• Easy to scale and integrate with third-
party services
• No long-term commitment to technology
stack

9. © 2019 Tudip Technologies. All Rights Reserved.
Microservices Cons
• Due to distributed deployment, testing
can become complicated and tedious
• Increasing number of services can result
in information barriers
• The architecture brings additional
complexity as the developers have to
mitigate fault tolerance, network latency,
and deal with a variety of message
formats as well as load balancing
• Being a distributed system, it can result in
duplication of effort
• When number of services increases,
integration and managing whole products
can become complicated
• In addition to several complexities of
monolithic architecture, the developers
have to deal with the additional
complexity of a distributed system
• Developers have to put additional effort
into implementing the mechanism of
communication between the services
• Handling use cases that span more than
one service without using distributed
transactions is not only tough but also
requires communication and cooperation
between different teams

10. © 2019 Tudip Technologies. All Rights Reserved.
How Micro a Microservice should be?
Amazon's policy is that the team implementing a microservice
should be small enough that they can be fed by two-pizzas.

11. © 2019 Tudip Technologies. All Rights Reserved.
Is it all Rosy?

12. © 2019 Tudip Technologies. All Rights Reserved.
The Hardest Part About Microservices:
Database Design

13. © 2019 Tudip Technologies. All Rights Reserved.
Yes, database is the DOG!

14. © 2019 Tudip Technologies. All Rights Reserved.
Broadly, you have 2 options!

15. © 2019 Tudip Technologies. All Rights Reserved.
Option # 1, Monolithic Database

16. © 2019 Tudip Technologies. All Rights Reserved.
Option #1.1, Variation of Monolithic Database

17. © 2019 Tudip Technologies. All Rights Reserved.
Option #2, Multiple Databases

18. © 2019 Tudip Technologies. All Rights Reserved.
Problems in Monolithic Database
• You can NOT scale a service without scaling the database
• You can not deploy a service because change in 1 schema, going to affect other
microservices and hence you can not do Continuous Integration/Continuous
Deployments
• You would be stuck with an RDBMS OR NoSQL and can NOT pick what works the
best
• And hence, service preference of the DB would be completely lost

19. © 2019 Tudip Technologies. All Rights Reserved.
Well, what about 2PC (Phase Commits)
• 2PC, is a mechanism for implementing a transaction across different software
components (multiple databases, message queues etc.)
• Well, if that is how you implement, the ghosts of limitations of Monolithic Database are
going to Haunt You!

20. © 2019 Tudip Technologies. All Rights Reserved.
Again, monolithic database is a DOG!

21. © 2019 Tudip Technologies. All Rights Reserved.
The Hardest Part About Microservices:
Your Data
https://blog.christianposta.com/microservices/the-hardest-part-about-microservices-data/
But challenges of the Database per Service..

22. © 2019 Tudip Technologies. All Rights Reserved.
Multiple Databases has its own challenges!
● Complexity of managing multiple SQL and NoSQL databases
● Implementing business transactions that span multiple services is not straightforward.
Moreover, many modern (NoSQL) databases don’t support them. The best solution is
to use the Saga pattern. Services publish events when they update data. Other
services subscribe to events and update their data in response.
● Implementing queries that join data that is now in multiple databases is challenging.

23. © 2019 Tudip Technologies. All Rights Reserved.
Saga
• Implement each business transaction that spans multiple services as a saga.
• A saga is a sequence of local transactions.
• Each local transaction updates the database and publishes a message or event to
trigger the next local transaction in the saga. If a local transaction fails because it
violates a business rule then the saga executes a series of compensating
transactions that undo the changes that were made by the preceding local
transactions.

24. © 2019 Tudip Technologies. All Rights Reserved.
Saga Continues

25. © 2019 Tudip Technologies. All Rights Reserved.
2 ways to coordinating the Saga
• There are two ways of coordination sagas:
• Choreography - each local transaction publishes domain events that trigger
local transactions in other services
• Orchestration - an orchestrator (object) tells the participants what local
transactions to execute

26. © 2019 Tudip Technologies. All Rights Reserved.
Here is choreography-based Saga

27. © 2019 Tudip Technologies. All Rights Reserved.
Choreography-based saga

28. © 2019 Tudip Technologies. All Rights Reserved.
Orchestration-based saga

29. © 2019 Tudip Technologies. All Rights Reserved.
And here is an Orchestra

30. © 2019 Tudip Technologies. All Rights Reserved.
And then there is always CQRS

31. © 2019 Tudip Technologies. All Rights Reserved.
CQRS: Command Query Responsibility Segregation

32. © 2019 Tudip Technologies. All Rights Reserved.
How did it work in one of our OTT project?

33. © 2019 Tudip Technologies. All Rights Reserved.
What all we have used in this application?
● Saga
● Along with CQRS

34. © 2019 Tudip Technologies. All Rights Reserved.
References
• https://en.wikipedia.org/wiki/Microservices
• https://en.wikipedia.org/wiki/Conway%27s_law
• https://microservices.io/patterns/data/event-sourcing.html
• https://auth0.com/blog/introduction-to-microservices-part-4-dependencies/
• https://stackoverflow.com/questions/41640621/data-sharing-between-micro-services
• https://www.baeldung.com/transactions-across-microservices

35. 35
Thank You
Know more about Tudip here!