Agile integration - Connecting the dots of everything-distributed-architecture. A modern approach of integration with API management, service mesh, fuse, kafka and more

1. Agile integration - Connecting the dots
of everything-distributed-architecture
A modern approach with API management and more
Senior Middleware Architect
& Flying Elephants Mentor
Red Hat Professional Services

2. 50 YEARS OF EVOLUTION

3. JUST IN 1970...

5. BILLION
INTERCONNECTED
DEVICES
26
TRILLION
STABLE
CONNECTIONS
215
MILLION
CONNECTIONS
PER SECOND
63
Source: Gartner, Building Platforms for a Digital Society: Key Insights From the 2016 Gartner Symposium/Itxpo Keynote, Nov 2016. ID#G00317081.
BY THE YEAR 2020 THERE WILL BE MORE THAN

6. BILLION
B2B/B2C PER DAY
TRANSACTIONS
450
BILLION
CONNECTED
SMART DEVICES
50
ZETTABYTES
ACCUMULATED
BIG DATA VOLUME
44
Source: Gartner, Building Platforms for a Digital Society: Key Insights From the 2016 Gartner Symposium/Itxpo Keynote, Nov 2016. ID#G00317081.
BY THE YEAR 2020 THERE WILL BE MORE THAN

8. BUSINESS PROCESS MANAGEMENT
ENTERPRISE SERVICE BUS
MESSAGE BROKERING
APPLICATION / DATA
TRADITIONAL MULTI-TIER ARCHITECTURE

9. “ESB architectures hinder the operational
and architectural agility of integration tasks.
Technical professionals should use a
distributed integration platform to enable
containerized, agile integration, based on
DevOps principles.”
Source: Enabling Agile Integration With a Distributed Integration Platform, September 20189

10. AND THEN...

11. AND THEN...

12. ● Simple
● Light-weight
● Boundary
● Replaceable
● Independent
microservice
API
MICROSERVICES
DO ONE THING, BUT DO IT WELL

13. SIMPLE PIPELINE
GRANULARITY AND SECURITY

14. microservice by Fuse
Client
API
(Interface,
Contracts)
Routing
Protocol
Endpoints
Transforma
tion
SMART END-POINT
USE INTEGRATION WHERE NEEDED, RATHER THAN CENTRALIZING

15. DECENTRALIZED/DISTRIBUTED BPM
Pricing Service
Stock Service
Customer Status Service
Order Service
Promotion Service
Payment Service
Shipping Service
BPM AS A SERVICE. USE BPM WHEN YOU NEED.

16. RESTFul
HTTP
Swagger
Documentations
JSON
XML
3scaleAPIManagement
●
●
●
●
API ORIENTED
EXPOSE DNA OF YOUR BUSINESS

17. Kafka A-MQ Messaging
EVENTS
EVENT-BASED
RELIABLE ASYNCHRONIZED DELIVERY

18. 1. Data Synchronization, Staleness & Consistency
2. Security
3. Services Communication
4. Discovery
5. Cyclic Dependencies of Services
6. Distributed Logging and Debugging
7. Testing
8. Monitoring & Performance
9. DevOps Support
10. Fault Tolerance
10 CHALLENGES OF MSA

19. REFERENCE ARCHITECTURE

20. Agile Integration Reference Architecture

21. Agile Integration Reference Architecture
Gateway Layer
API exposure
1

22. Agile Integration Reference Architecture
Gateway Layer
API exposure
Composition Layer
Business µ-services
2
1

23. Agile Integration Reference Architecture
Base Microservices
µ-services
Gateway Layer
API exposure
Composition Layer
Business µ-services
2
1
3

24. Agile Integration Reference Architecture
Base Microservices
µ-services
Gateway Layer
API exposure
Composition Layer
Business µ-services
2
1
3
Capabilities
Messaging
DataVirtualization
Caching
IntegrationPatterns

25. Distributed Integration in Containers
Containers

26. composition layer
base layer
common
µs µs µs µs
µs µs
Service exposure
Internal Services/Data-Sources External Services (e.g Partners)
µsµs
Caching
Routing
Data transformation
Event triggers
Content Enrichment
µs µs µs µs
Messaging
API policies
Versioning
External Entities
(apps, partners, ...)
Integration layer
26

27. Service exposure
API policies
Versioning
External Entities
(apps, partners, ...)
Provides:
1. Access control
2. Usage Policies
3. Traffic Analytics
4. Developer onboarding
5. ...
GATEWAY LAYER1
Why?
● applies “Separation-of-Concerns”
● abstracts definition from implementation
● Resolves client access and connectivity
● favours no “lock-in”
● open layer: other systems in the
ecosystem can also register.
27

28. Service exposure
API policies
Versioning
External Entities
(apps, partners, ...)
28

29. COMPOSITION LAYERcomposite µs
⇒ Aggregations, splits, content
transformation, events,
caching, pattern-based
integration, etc.
2
Why?
● Typically define business functions.
● Resolve non-trivial or complex
integrations.
29

30. BASE LAYER
base µs
Organised into application
domains (based on
bounded context)
3
Why?
● Abstracts and absorbs backend
complexity
● API-fy legacy backends
● Simplify/Normalise access downwards
● Reusable and scalable units
30

31. 31
&

32. BACKENDS LAYER
backends
Internal or external
endpoints the integration
layer uses.
L
Why?
● Internal/external legacy systems still
need to be accessible
● Non-containerised systems also under
this layer
32

33. Agile Integration Reference Architecture
Base Microservices Layer
- µs organised into application domains (based on bounded context)
- Legacy integration (Think Gartner’s Bi-Modal IT)
Gateway Layer
- API Exposure / Separation of concerns
- Access control / Policies / Analytics
Composition Layer
- Composition of µs to provide business functions
- Pattern-based integration, aggregations, splits, content transformation,
caching, etc.
- Event-based communication, lightweight messaging
2
1
3
Capabilities
IntegrationPatterns
Messaging
Otherµsruntimes
Caching

34. Red Hat’s Agile Integration Concept
DISTRIBUTED
INTEGRATION
CLOUD-NATIVE SOLUTIONS
LEAN ARTIFACTS,
INDIVIDUALLY DEPLOYABLE
CONTAINER-BASED SCALING &
HIGH AVAILABILITY
SCALABILITY
PORTABILITY
CONTAINERS
LIGHTWEIGHT
PATTERN BASED
EVENT-ORIENTED
COMMUNITY-SOURCED
RE-USABILITY
APIs
WELL-DEFINED, REUSABLE,
& WELL-MANAGED
ENDPOINTS
ECOSYSTEM LEVERAGE
Culture Processes Tools Automation
Business Process
Management
Data
Management
IoT
Enablement
Enterprise
Applications
Mobile
Enablement
FLEXIBILITY
VERSATILITY

35. HA Proxy
SERVICES COMMUNICATION,
DISCOVERY & SECURITY
I - Internal/private PODs
communication managed by
service mesh (Istio)
II - Local APIs exposed via
APICast. Basic security.
III - External APIs exposed via
APICast. Advanced security.
Integration/Orchestration with
Fuse/AMQ/AMQ Streams
Develop namespace
APICast
Pod
Pod
ApService
Pod
Pod
ApService
Pod
Pod
ApService
Pod
Pod
I I I
II II
3csale namespace
3scale
3scSrvc
Pod
Pod3scSrvc
Pod
PodAPICast
Pod
Pod
“DMZ” namespace
APICast
Pod
Pod
III

36. Red Hat’s Full API Lifecycle Management

37. How Red Hat can help you handle
the Full API Lifecycle Management

38. Thank You
Senior Middleware Architect
& Flying Elephants Mentor
Red Hat Professional Services
ypreiger@redhat.com