Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How Orwell built a geo-distributed Bank-as-a-Service with microservices

139 views

Published on

Orwell Group shares how they leveraged microservices, an event driven architecture and both master and reference data management methodologies to build a new banking system for high retail banking customers and corporate banks requiring cross border payments and cash flow management – and scaled it to handle customers with millions of clients. In particular they explain how they built a high availability, geo-distributed and consistent platform on top of MariaDB. The result was a secure and distributed platform with high cost efficiency, and the data accuracy and consistency needed to create high quality data pipelines from transactions to analytics and ensure regulatory compliance (e.g., GDPR).

Published in: Software
  • Be the first to comment

  • Be the first to like this

How Orwell built a geo-distributed Bank-as-a-Service with microservices

  1. 1. The material in this presentation has been prepared by Orwell Group Holding Limited as general background information only about Orwell’s activities current as at the date of this presentation. Information in this presentation, including forecast financial information, should not be considered as advice or a recommendation in relation to holding, purchasing or selling securities in Orwell Group Holding Limited or any subsidiaries. This presentation may contain forward looking statements including statements regarding our intent, belief or current expectations with respect to Orwell’s businesses and operations, market conditions, results of operation and financial condition, and risk management practices. Readers are cautioned not to place undue reliance on forward looking statements. Forecasts, estimates and examples are subject to uncertainty and contingencies outside of our control. Orwell Banking Platform Payment Processing Cash management Compliance Payment Network as as Service How did we build a geo-distributed Bank- as-a-Service with microservices Manoj Chugh Chief Architect Raul Rodrigo Barco Engineering Head, Madrid
  2. 2. About Orwell 01 02 03 04 05 Regulated Cash Management with Borderless Banking Model Breakthrough Payments Technology Open Banking and PSD2 compliant with New Payments Architecture White Label for Banks Suite of Treasury, Payments and Loyalty solutions for Corporates
  3. 3. Challenge
  4. 4. Challenges  Complex estates that have been running since the late 60’s and 70’  How to manage change in these large estates to migrate to newer technology. Legacy Financial Services • User value experience and speed. • Users have different behaviour • Users expect availability of their data 24x7x365 Users • Secure and control data access and movement • Reduce risk (liquidity, credit, market, fraud, counterparty) Security, Risk & Controls • Anti-money laundering, Sanctions and Fraud • Reporting, strong security, safeguarding of money Regulators • Create niche products • Reduce the cost and Increase agility • Business continuity Product & Business Owners
  5. 5. How can we address the challenges and provide business agility with bleeding edge technology at best possible cost The journey is as important as the end state.Legacy
  6. 6. Solution
  7. 7. Key principles Append-only immutable log is the central layer for the distributed architecture Kappa Architecture Domain Driven, Single Responsibility with Ubiquitous Language and Bounded Context Domain Driven Microservices Command Query Responsibility Segregation & Event Sourcing Storing events rather than state. CQRS allows highly performant architecture. Failure is expected. Every component must be continuously available. Continuous Availability 01 02 03 04 Orwell’s adoption of technology Principles
  8. 8. Kappa Architecture Style • Kappa Architecture is a software architecture pattern. Rather than using a relational DB like SQL or a key-value store like Cassandra, the canonical data store in a Kappa Architecture system is an Apache Kafka append-only immutable log. • From the log, data is streamed through a computational system and fed into auxiliary stores for serving.
  9. 9. Microservices Design Domain Based Microservices Awareness of Microservices’ Trade- off  Must use some of the microservices principles that allows isolation but must avoid the possible silos that could be created. Awareness of Legacy function  The system must support and maintain the same level of functionality of exiting institutions or legacy systems will never be turn off. Awareness of Legacy Data  Old systems usually means bad data quality of missing data. Any new system must support this fact.
  10. 10. Data Infrastructure Design Our Data Infrastructure is based upon separation of concerns Write efficient (Consistency) Read efficient (Availability)  Financial Institutions on average will have above a 90% reads to 10% writes.  Payments are write operations but not updates.  Customer don’t change their details frequently, no one moves address multiple time time a day. Read/write performance is all about trade-offs and balance
  11. 11. Data Model Design  This allows the definitions of products with customized characteristics that can be built and test in a matter of days or weeks.  The model is designed to allow complex relationships between participants e.g. a company having users that manage their own self-service banking products Data Model is based upon Master Data Management & Party-Contract-Product Held principles
  12. 12. Geo-distributed multi-cloud Local Registry System Make Payment USA Instance ID Local Registry System UK Instance ID Account Key (64Bit) Instance ID Account Naming Structure to allow Routing Automated Discovery of Network participants Local Registry System Broadcast – 128bit Account Number Hash Germany Instance ID Make Payment DHT Based Network UUIDs: That are customized identifiers Like email The system is designed to have multiple instances across multiple geographies and multiple public clouds. Each system is instance aware.
  13. 13. How do we use MariaDB
  14. 14. Why MariaDB in our technology stack Accuracy Flexible information model Common understanding Write Performance We needed a flexible information model for customer and product information, that humans easily understand. Consistency and accuracy of financial information is highly important. The form of information to consume in microservice is dependent on the consumption model but we all need a common understanding. We don’t have a high performance requirement for this dataset as customers don’t change telephone number 50000 a second.
  15. 15. MariaDB Setup
  16. 16. MariaDB Change Data Capture Orwell’s unique solution
  17. 17. Orwell CDCClient implements MaxScale CDC Protocol - Connection andAuthentication • Client connects to MaxScale CDC protocol listener. • Send the authentication message which includes the user and the SHA1 of the password Registration • SendingUUID • Specify the output format (AVRO or JSON) for data retrieval. Data Request • SendCDC commands to retrieve router statistics or to query for data events CDC client features: • One client manage all changes events in different tables (configured in zookeeper) • Handle not existing avro-schema files • High availability (Leader-election pattern) MariaDB Change Data Capture
  18. 18. CDC Continuous Availability
  19. 19. Change Data Capture Demo
  20. 20. - Prometheus: Monitoring / Alerting - Grafana: Dashboard Node Cluster Definition: • 3 Instances • Monitoring Status instances • Monitoring Leader/Followers Leader/Followers • Monitoring Leader Elections CDC Resilience
  21. 21. - Prometheus: Monitoring - Grafana: Dashboard CDC Instance Nodes: • Tables • Events processed • OK • Expired • Duplicates CDC Metrics
  22. 22. Conclusion
  23. 23. Conclusion  Orwell combined best of two worlds (MariaDB SQL & NoSQL) in microservices CQRS architecture to delivering a hyperscale & highly-available solution  Our approach of using Domain based microservices allows a loosely coupled component architecture, all of which can have different scalability profiles  Use of Master Data Management techniques & JSON has allowed flexibility and creation of Metadata to support dataset changes  Flexible information model has reduced time to market for creating new products  MariaDB has been a useful General purpose RDBMS not just for the data consistency requirements but also for use with Hadoop applications such as HIVE Metastore, Apache Ranger and Apache Ambari.  With Orwell’s unique CDC solution, we can not only keep SQL & NoSQL in sync but also recover the failures with no down time to the service
  24. 24. The material in this presentation has been prepared by Orwell Group Holding Limited as general background information only about Orwell’s activities current as at the date of this presentation. Information in this presentation, including forecast financial information, should not be considered as advice or a recommendation in relation to holding, purchasing or selling securities in Orwell Group Holding Limited or any subsidiaries. This presentation may contain forward looking statements including statements regarding our intent, belief or current expectations with respect to Orwell’s businesses and operations, market conditions, results of operation and financial condition, and risk management practices. Readers are cautioned not to place undue reliance on forward looking statements. Forecasts, estimates and examples are subject to uncertainty and contingencies outside of our control. Thanks

×