Distributed transactions in SOA and Microservices
•
7 likes•1,422 views
We are familiar with transactions in databases, messaging systems and caches. When messaging system works fine, but database transaction fails - we are in big trouble! How to make all this guys work as a team? - What need to be used is distributed transactions or compensation mechanisms. On this talk we see both approaches and discuss how to take advantage of them when we are building Microservices architecture. See code examples here: https://github.com/kslisenko/distributed-transactions
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Distributed transactions in SOA and Microservices
Similar to Distributed transactions in SOA and Microservices (20)
More from Constantine Slisenka
More from Constantine Slisenka (11)
Recently uploaded
Recently uploaded (20)
Distributed transactions in SOA and Microservices
- 1. 1CONFIDENTIAL DISTRIBUTED TRANSACTIONS IN SOA AND MICROSERVICES KANSTANTSIN SLISENKA LEAD SOFTWARE ENGINEER NOV 9, 2017
- 2. 2CONFIDENTIAL kanstantsin_slisenka@epam.com Kanstantsin Slisenka Java Team Lead Financial services, trading solutions Speaker at: Minsk Java Tech Talks, IT Week, SEC Online, Java Professionals github.com/kslisenko
- 3. 3CONFIDENTIAL AGENDA Local transactions1 Distributed transactions2 Compensations: SAGA pattern3 Live demo4
- 4. 4CONFIDENTIAL WHY DO WE NEED TRANSACTIONS?
- 5. 5CONFIDENTIAL WHY DO WE NEED TRANSACTIONS? New order created Payment received Product shipped Data must be in known state anytime
- 6. 6CONFIDENTIAL Application Business workflows Infrastructure Transactions in file systems, databases, message brokers, caches, application servers
- 7. 7CONFIDENTIAL Transaction processing is based on logging of states and transitions HOW IT WORKS
- 8. 8CONFIDENTIAL Transaction log • Always in known state • States and transitions are logged • Log can be used for recovery DATABASE TRANSACTION AS STATE MACHINE begin in progress commit rollback INSERT UPDATE DELETE commitbegin
- 9. 9CONFIDENTIAL ACID TRANSACTION GUARANTEES Atomicity All or nothing Write-ahead log
- 10. 10CONFIDENTIAL ACID TRANSACTION GUARANTEES Atomicity All or nothing Write-ahead log Consistency Data always in valid state Constraints
- 11. 11CONFIDENTIAL ACID TRANSACTION GUARANTEES Atomicity All or nothing Write-ahead log Consistency Data always in valid state Constraints Isolation Visibility of concurrent actions Locking
- 12. 12CONFIDENTIAL ACID TRANSACTION GUARANTEES Consistency Data always in valid state Constraints Isolation Visibility of concurrent actions Locking Durability Changes become permanent Write-ahead log Atomicity All or nothing Write-ahead log
- 13. 13CONFIDENTIAL Transactions guarantee the data to always be in valid state
- 16. 16CONFIDENTIAL Book trip 1. Book hotel 2. Book flight 3. Hire rental car 4. Record booking
- 17. 17CONFIDENTIAL TRAVEL BOOKING SYSTEM DBBackend browser hotels flights cars bookings
- 18. 18CONFIDENTIAL TRAVEL BOOKING TRANSACTION hotels flights cars bookings DB Action Rollback action
- 19. 19CONFIDENTIAL TRAVEL BOOKING TRANSACTION Begin hotels flights cars bookings DB Action Rollback action begin
- 20. 20CONFIDENTIAL TRAVEL BOOKING TRANSACTION Begin hotels flights cars bookings DB Action Rollback action begin room_id=15 status=booked, traveler=Kostia room_id=15 status=free, traveler=null LOCK UPDATE
- 21. 21CONFIDENTIAL TRAVEL BOOKING TRANSACTION Begin hotels flights cars bookings DB Action Rollback action begin room_id=15 status=booked, traveler=Kostia room_id=15 status=free, traveler=null seat_id=17c status=booked, traveler=Kostia seat_id=17c status=free, traveler=null LOCK UPDATE LOCK UPDATE
- 22. 22CONFIDENTIAL TRAVEL BOOKING TRANSACTION Begin hotels flights cars bookings DB Action Rollback action begin room_id=15 status=booked, traveler=Kostia room_id=15 status=free, traveler=null seat_id=17c status=booked, traveler=Kostia seat_id=17c status=free, traveler=null car_num=1234AA7 status=booked, traveler=Kostia car_num=1234AA7 status=free, traveler=null LOCK UPDATE LOCK UPDATE LOCK UPDATE
- 23. 23CONFIDENTIAL TRAVEL BOOKING TRANSACTION Begin hotels flights cars bookings DB Action Rollback action begin room_id=15 status=booked, traveler=Kostia room_id=15 status=free, traveler=null seat_id=17c status=booked, traveler=Kostia seat_id=17c status=free, traveler=null car_num=1234AA7 status=booked, traveler=Kostia car_num=1234AA7 status=free, traveler=null INSERT traveler=Kostia, room_id=15, seat_id=17c, car_num=1234AA7 DELETE traveler=Kostia, room_id=15, seat_id=17c, car_num=1234AA7 LOCK UPDATE LOCK UPDATE LOCK UPDATE LOCK UPDATE
- 24. 24CONFIDENTIAL TRAVEL BOOKING TRANSACTION Begin hotels flights cars bookings DB Action Rollback action begin room_id=15 status=booked, traveler=Kostia room_id=15 status=free, traveler=null seat_id=17c status=booked, traveler=Kostia seat_id=17c status=free, traveler=null car_num=1234AA7 status=booked, traveler=Kostia car_num=1234AA7 status=free, traveler=null INSERT traveler=Kostia, room_id=15, seat_id=17c, car_num=1234AA7 DELETE traveler=Kostia, room_id=15, seat_id=17c, car_num=1234AA7 commit LOCK UPDATE LOCK UPDATE LOCK UPDATE LOCK UPDATE Commit
- 25. 25CONFIDENTIAL TRAVEL BOOKING TRANSACTION Begin hotels flights cars bookings DB UPDATE UPDATE UPDATE UPDATE Rollback Action Rollback action begin room_id=15 status=booked, traveler=Kostia room_id=15 status=free, traveler=null seat_id=17c status=booked, traveler=Kostia seat_id=17c status=free, traveler=null car_num=1234AA7 status=booked, traveler=Kostia car_num=1234AA7 status=free, traveler=null INSERT traveler=Kostia, room_id=15, seat_id=17c, car_num=1234AA7 DELETE traveler=Kostia, room_id=15, seat_id=17c, car_num=1234AA7 rollback LOCK LOCK LOCK LOCK
- 26. 26CONFIDENTIAL NOT ONLY IN DATABASES TRANSACTIONS ARE
- 27. 27CONFIDENTIAL • Isolation levels • Locking • Commit logs • Transactions – ONE and TWO phase
- 28. 28CONFIDENTIAL Cache Messages Transactions Subscriptions ACKs, … • Transacted sessions • JMS attributes – JMSXConsumerTXID – JMSXProducerTXID
- 29. 29CONFIDENTIAL AGENDA Local transactions1 Distributed transactions2 Compensations: SAGA pattern3 Live demo4
- 33. 33CONFIDENTIAL
- 34. 34CONFIDENTIAL
- 37. 37CONFIDENTIAL WEB-SERVICE AS STATE MACHINE /reserve /confirm /cancel reserved confirmed cancelled /reserve /confirm /cancel timeout 2 1 Idempotent operations
- 39. 39CONFIDENTIAL 2-PHASE COMMIT ALGORITHM begin in progress commit rollback prepare All changes saved to log Changes copied to storage Changes cancelled 2 1
- 40. 40CONFIDENTIAL JTA AND XA SPECIFICATION Java application
- 41. 41CONFIDENTIAL Application server JTA AND XA SPECIFICATION Transaction Manager Java application
- 42. 42CONFIDENTIAL Application server JTA AND XA SPECIFICATION Resource manager Resource manager Resource manager Transaction Manager DB JMS ? Java application JDBC CUSTOM
- 43. 43CONFIDENTIAL JTA AND XA SPECIFICATION Resource manager Resource manager Resource manager Application server Transaction Manager DB ? Java application begin begin begin
- 44. 44CONFIDENTIAL Application server JTA AND XA SPECIFICATION Resource manager Resource manager Resource manager Transaction Manager DB ? Java application update update update
- 45. 45CONFIDENTIAL Application server JTA AND XA SPECIFICATION Resource manager Resource manager Resource manager Transaction Manager DB ? Java application prepare prepare prepare
- 46. 46CONFIDENTIAL Application server JTA AND XA SPECIFICATION Resource manager Resource manager Resource manager Transaction Manager DB ? Java application commit commit commit
- 47. 47CONFIDENTIAL TM does all the job for us, but: • Needs too many messages • Doesn’t scale well • Not all vendors support XA
- 48. 48CONFIDENTIAL AGENDA Local transactions1 Distributed transactions2 Compensations: SAGA pattern3 Live demo4
- 50. 50CONFIDENTIAL SAGA EXAMPLE Book hotel Book flight Book car Cancel hotel Cancel flight Confirm booking Fail! Retry Retry failed Start Finish Skip
- 53. 53CONFIDENTIAL ACID Availability Consistency Isolation Durability BASE Basic Availability Soft state Eventual consistency (Trading consistency for availability) 2-PHASE COMMIT SAGA
- 54. 54CONFIDENTIAL System is partitioned CAP THEOREM CONSISTENCY AVAILABILITY Consistency and Availability System is single node
- 55. 55CONFIDENTIAL AGENDA Local transactions1 Distributed transactions2 Compensations: SAGA pattern3 Live demo4
- 59. 59CONFIDENTIAL Car service Frontend Tier 1 Backend cache Tier 2 /bookTrip Agency DB Hotels DB Flight service requests responses JMS JMS Distributed transaction JMS endpoint Remote call Local call JMS queue /getTrips github.com/kslisenko/distributed-transactions
- 60. 60CONFIDENTIAL Frontend Tier 1 Backend cache Tier 2 /bookTrip requests responses Car service Agency DB Hotels DB Flight service JMS JMS Distributed transaction JMS endpoint Remote call Local call JMS queue /getTrips github.com/kslisenko/distributed-transactions
- 61. 61CONFIDENTIAL Frontend Tier 1 Backend cache Tier 2 /bookTrip requests responses Car service Agency DB Hotels DB Flight service JMS JMS JMS Distributed transaction JMS endpoint Remote call Local call JMS queue /getTrips github.com/kslisenko/distributed-transactions
- 62. 62CONFIDENTIAL Frontend Tier 1 Backend cache Tier 2 /bookTrip requests responses Car service Agency DB Hotels DB Flight serviceJMS JMS JMS JMS /getTrips Distributed transaction JMS endpoint Remote call Local call JMS queue github.com/kslisenko/distributed-transactions
- 64. 64CONFIDENTIAL CONCLUSION 1. State machines everywhere 2. Transactions everywhere 3. Design for failure 4. Rely on tools or handle by own