Integration 5.0: From REST APIs and
Message Queues, to Event-Driven
Microservices
Fransiscus Kaurrany
EVP, Group Architecture & IT Service Quality, Bank Central Asia
Damien Wong
Vice President, APAC (Asia Pacific & Japan), Confluent
Fireside Chat
EVP, Group Architecture
& IT Service Quality
Bank Central Asia
VP, APAC
Confluent
Enterprises (especially Banks) are moving from
software users, to becoming software
Faster
application
development &
time to market
Enable teams
to build
independently
Reduce
infrastructure
cost and
complexity
Eliminate
dependencies in
feature delivery
Scale services
with smaller,
agile teams
“In many ways, we see ourselves as a technology company with a banking license.” —
Michael Corbat, Citi CEO
Evolution of Monolith
APP APP APP APP
Monolith
Large self contained application
Complex app with highly interdependent parts
Poor developer experience and productivity
Slow feature delivery
Difficult to deploy, impacting other systems
Requires replication of entire application
Microservices
Multiple smaller, single function apps
Independently deployable and upgradable
Built around solving business capabilities
Can be built w/ different programming languages
Scalable and agile = faster feature delivery
Leverages newer development platforms
To Microservices
Existing approaches to developing microservices
Messaging Queues
(rabbitMQ, ActiveMQ)
● Message brokers act as a centralized
messaging service through which all
of the microservices communicate
● Brokers handle messaging queueing,
HA, reliable communication between
services
● Messages received in a queue rather
than dropped and processed later
Communication via
HTTP REST API
● Benefits:
○ Simple set up
○ Efficient message delivery
● Synchronous communication
○ Client sends request and waits
for response
Challenges with REST
based microservices
Fulfillme
nt service
Stock
service
Order
service
Return
service
Payment
service
UI service
GUI
● Difficult to enforce standards across
services
● Does not scale if servers are
synchronous
● Risky inter-service dependencies
● Services required to maintain state
● Complex management and version
compatibility—slows development
● Requires load balancing
Challenges with legacy
messaging queues
Microservice
Producer
Legacy
Messaging
Queue
Queue 1
Queue 2
Microservice
A
Microservice
B
● Extremely difficult to scale
● Lacks message persistence—if
message delivery fails, its
unavailable for replay
● Low throughput and high
latency
● Need prior knowledge of
consumers
● Expensive MQ and mainframe
technology costs
●
Fulfillment
service
Stock
service
Order
service
Return
service
Payment
service
UI service
GUI
Why build with
Confluent
● Completely decoupled microservices
● Single standard for
inter-communication
● Maintains version compatibility
● Asynchronous services
development
● State persistent on single platform
for replay
● Distributed and highly scalable
● Process data in flight and real-time
● Deployment flexibility -- on prem, in
cloud, or hybrid
Confluent enables a new class of event driven
microservices
Why does it make sense?
Microservices
Distributed
Semi-loose with API versioning
Requires a consistent distributed
system
Synchronous
Event Driven Microservices
Central, synchronized through events
Completely decoupled (Fire and forget)
The streaming platform reduces the
dependencies to external system
Reactive (asynchronous)
State
Coupling
E2E Testing
Execution Model
Fireside Chat
EVP, Group Architecture
& IT Service Quality
Bank Central Asia
VP, APAC
Confluent
Challenges
● We need to scale faster to meet growing customer demands, so we
need platform that can support it
● System reliability & maintenance challenges
● Customer Engagement was impacted due to performance
● It become a challenge for the team to support various technologies
● BCA is largest bank in ASEAN by
Market Cap.
● Processing average 27 million
transactions per day
● 98% of total transaction services
via Digital Channels
Moving to
Event Driven
Architecture
Use Cases/Solutions
● Monolithic to microservices application transformation
● Modernise architecture for more flexibility & agility to meet
customer needs
● Real time customer notifications and marketing
● Real-time event streaming enables Digital Services to drive
advantage
● Going forward, leveraging AI/ML with Event Streaming Platform for
real time analytics
● Expanded use cases for fraud detection, marketing offers /
promotions & ATM security
Integration 5.0
Fransiscus Kaurrany
EVP, Group Architecture & IT Service Quality
The
Evo-Revolution
BCA Integration Architecture
tech
Proprietary Platform
Proprietary Gateway
std. &
comm.
ISO 8583
Sync
app
ATM and Classic EDC
pro & cons
(+) Less Tightly Couple
(-) Proprietary
Technology
tech
Open Platform
Service Bus
Monolithic
Rule Base Engine
std. & comm.
Web Service (SOAP)
Sync
app
Web and Mobile
Channel
pro & cons
(+) > Less Tightly Couple
(+) Open Standard for FE
tech
Cloud Platform
API GW
Messaging Queue
Microservices
std. &
comm.
REST API
Sync + Async
app
OMNI Channel
Android POS
Digital Branch
pro & cons
(+) >> Less Tightly Couple
(+) Open Standard
(+) Light Conn. < 500K
data
tech
Cloud Platform
API GW
Confluent Kafka
Microservices
std. & comm.
REST API + ISO
20022
Sync + Async
app
NRT Analytics and ML
Fraud Detection System
IoT
pro & cons
(+) Loosely Couple
(+) Open Standard
(+) NRT Parallel Process
(+) Massive Data
5.0
Event-Driven
4.0
API GW & MQ
3.0
EAI/SOA/ESB
tech
Socket
EJB / JCA
Screen Scrapping
std. &
comm.
No single standard
Sync
app
Classic Branch
Application
pro & cons
(-) Tightly Couple
2.0
Switching
1.0
Point to Point
The 5.0
Journey
BCA Integration Architecture
The Architecture
From 1.0 to 5.0 Integration
1.0 Point to Point
socket
ISO8583
screen
scraping
2.0 Switching
Base2
4
ATM 3rd
PartyED
C
Core Banking
* All ISO8583 base messaging
3.0
EAI/SOA/ESB
Internet Banking
Other Web App
EAI
* Using SOAP
CICS
Transaction
Gateway
ISO8583
*depend
Mobile Banking
The Architecture
From 1.0 to 5.0 Integration, continue
4.0 API Gateway & MQ
Base2
4
Core Banking
Front End
EAI
API
Gateway
MQ
3rd
Party
Kafk
a
BIG
DATA
AI/ML Engine
Applications
5.0 Event-Driven
Thank You
Interacting
2
Join the Confluent Slack Channel
https://launchpass.com/confluentcommunity
Local meetups
https://www.confluent.io/community/
KafkaSummit 2020
https://kafka-summit.org/
API Days Singapore

API Days Singapore

  • 1.
    Integration 5.0: FromREST APIs and Message Queues, to Event-Driven Microservices Fransiscus Kaurrany EVP, Group Architecture & IT Service Quality, Bank Central Asia Damien Wong Vice President, APAC (Asia Pacific & Japan), Confluent
  • 2.
    Fireside Chat EVP, GroupArchitecture & IT Service Quality Bank Central Asia VP, APAC Confluent
  • 3.
    Enterprises (especially Banks)are moving from software users, to becoming software Faster application development & time to market Enable teams to build independently Reduce infrastructure cost and complexity Eliminate dependencies in feature delivery Scale services with smaller, agile teams “In many ways, we see ourselves as a technology company with a banking license.” — Michael Corbat, Citi CEO
  • 4.
    Evolution of Monolith APPAPP APP APP Monolith Large self contained application Complex app with highly interdependent parts Poor developer experience and productivity Slow feature delivery Difficult to deploy, impacting other systems Requires replication of entire application Microservices Multiple smaller, single function apps Independently deployable and upgradable Built around solving business capabilities Can be built w/ different programming languages Scalable and agile = faster feature delivery Leverages newer development platforms To Microservices
  • 5.
    Existing approaches todeveloping microservices Messaging Queues (rabbitMQ, ActiveMQ) ● Message brokers act as a centralized messaging service through which all of the microservices communicate ● Brokers handle messaging queueing, HA, reliable communication between services ● Messages received in a queue rather than dropped and processed later Communication via HTTP REST API ● Benefits: ○ Simple set up ○ Efficient message delivery ● Synchronous communication ○ Client sends request and waits for response
  • 6.
    Challenges with REST basedmicroservices Fulfillme nt service Stock service Order service Return service Payment service UI service GUI ● Difficult to enforce standards across services ● Does not scale if servers are synchronous ● Risky inter-service dependencies ● Services required to maintain state ● Complex management and version compatibility—slows development ● Requires load balancing
  • 7.
    Challenges with legacy messagingqueues Microservice Producer Legacy Messaging Queue Queue 1 Queue 2 Microservice A Microservice B ● Extremely difficult to scale ● Lacks message persistence—if message delivery fails, its unavailable for replay ● Low throughput and high latency ● Need prior knowledge of consumers ● Expensive MQ and mainframe technology costs ●
  • 8.
    Fulfillment service Stock service Order service Return service Payment service UI service GUI Why buildwith Confluent ● Completely decoupled microservices ● Single standard for inter-communication ● Maintains version compatibility ● Asynchronous services development ● State persistent on single platform for replay ● Distributed and highly scalable ● Process data in flight and real-time ● Deployment flexibility -- on prem, in cloud, or hybrid Confluent enables a new class of event driven microservices
  • 9.
    Why does itmake sense? Microservices Distributed Semi-loose with API versioning Requires a consistent distributed system Synchronous Event Driven Microservices Central, synchronized through events Completely decoupled (Fire and forget) The streaming platform reduces the dependencies to external system Reactive (asynchronous) State Coupling E2E Testing Execution Model
  • 10.
    Fireside Chat EVP, GroupArchitecture & IT Service Quality Bank Central Asia VP, APAC Confluent
  • 11.
    Challenges ● We needto scale faster to meet growing customer demands, so we need platform that can support it ● System reliability & maintenance challenges ● Customer Engagement was impacted due to performance ● It become a challenge for the team to support various technologies ● BCA is largest bank in ASEAN by Market Cap. ● Processing average 27 million transactions per day ● 98% of total transaction services via Digital Channels Moving to Event Driven Architecture Use Cases/Solutions ● Monolithic to microservices application transformation ● Modernise architecture for more flexibility & agility to meet customer needs ● Real time customer notifications and marketing ● Real-time event streaming enables Digital Services to drive advantage ● Going forward, leveraging AI/ML with Event Streaming Platform for real time analytics ● Expanded use cases for fraud detection, marketing offers / promotions & ATM security
  • 12.
    Integration 5.0 Fransiscus Kaurrany EVP,Group Architecture & IT Service Quality
  • 13.
  • 14.
    tech Proprietary Platform Proprietary Gateway std.& comm. ISO 8583 Sync app ATM and Classic EDC pro & cons (+) Less Tightly Couple (-) Proprietary Technology tech Open Platform Service Bus Monolithic Rule Base Engine std. & comm. Web Service (SOAP) Sync app Web and Mobile Channel pro & cons (+) > Less Tightly Couple (+) Open Standard for FE tech Cloud Platform API GW Messaging Queue Microservices std. & comm. REST API Sync + Async app OMNI Channel Android POS Digital Branch pro & cons (+) >> Less Tightly Couple (+) Open Standard (+) Light Conn. < 500K data tech Cloud Platform API GW Confluent Kafka Microservices std. & comm. REST API + ISO 20022 Sync + Async app NRT Analytics and ML Fraud Detection System IoT pro & cons (+) Loosely Couple (+) Open Standard (+) NRT Parallel Process (+) Massive Data 5.0 Event-Driven 4.0 API GW & MQ 3.0 EAI/SOA/ESB tech Socket EJB / JCA Screen Scrapping std. & comm. No single standard Sync app Classic Branch Application pro & cons (-) Tightly Couple 2.0 Switching 1.0 Point to Point The 5.0 Journey BCA Integration Architecture
  • 15.
    The Architecture From 1.0to 5.0 Integration 1.0 Point to Point socket ISO8583 screen scraping 2.0 Switching Base2 4 ATM 3rd PartyED C Core Banking * All ISO8583 base messaging 3.0 EAI/SOA/ESB Internet Banking Other Web App EAI * Using SOAP CICS Transaction Gateway ISO8583 *depend Mobile Banking
  • 16.
    The Architecture From 1.0to 5.0 Integration, continue 4.0 API Gateway & MQ Base2 4 Core Banking Front End EAI API Gateway MQ 3rd Party Kafk a BIG DATA AI/ML Engine Applications 5.0 Event-Driven
  • 17.
  • 18.
    Interacting 2 Join the ConfluentSlack Channel https://launchpass.com/confluentcommunity Local meetups https://www.confluent.io/community/ KafkaSummit 2020 https://kafka-summit.org/