Discuss the important role of integration in Digital Transformation.

1. Integration in the age of Digital Transformation
WSO2 Workshop
Auckland, New Zealand.
Dassana Wijesekara | dassana@wso2.com
Associate Director – Solution Architecture
22nd June 2017

2. Agenda
▪ Digital Transformation and Integration
▪ Centralized Integration
▪ Future Integration needs
▪ Microservices and Integration
▪ Decentralized Integration – Micro-Integration
▪ State of the art integration middleware
▪ WSO2’s Next generation integration platform with Ballerina
2

3. Background
...
3

4. Digital Transformation and Enterprise Integration
4
▪ DT in a nutshell
o Evolving Business Models
o Focus on customer experience
o Optimize Operation
▪ Brown-field reality requires integration
o Hence Enterprise Integration is a key step towards
Digital Transformation.

5. Understanding Integration
▪ Penetration of IT in to businesses is on the rise.
▪ Systems and apps mounts critical business processes
▪ Enterprises adopting many systems and apps in many flavours
▪ Disparate software applications need to work together to produce
a unified set of functionality.
▪ Plumbing different software applications and forming new
software solutions is referred to as ‘enterprise integration’.
5

6. Centralized Integration
6
● A central integration middleware that connects apps, services & data
● Conventional SOA/ESB approach
● Legacy systems to cloud services connected with services, data,
messages and processes.

7. Centralized Integration
WSO2 Enterprise Integrator (EI)6.x
7

8. Centralized Integration
8
▪ WSO2 Enterprise Integrator 6.1.1 – Runtime Structure
o ESB and DSS as a single runtime
o Message Broker
o Business Process Server
o Application Server
o Analytics (primarily ESB analytics)

9. Future Integration Needs
...
9

10. Future Integration needs
10
● Growth and diversity of Integration needs
○ APIs and SaaS
○ Internet of Things
○ On-premise applications
■ B2B, Proprietary, Legacy systems

11. Future Integration needs
11
● Agility and ease of Integration
○ Minimum integration skills and operational overhead.
○ Customize existing integrations rapidly.
○ Visual modeling, debugging, troubleshooting
○ Analytics – Statistics, message tracing
○ Error handling.
○ Streamlined development life cycle.

12. Future Integration needs
12
● Orchestration – Implementing complex orchestration logics
○ Proliferation of services, APIs and applications to integrate.
○ Complexity of the orchestration logic increases
○ Need a simple and agile development of orchestration logic –
Visual modeling tools.

13. Future Integration needs
13
● Orchestration – Implementing complex orchestration logics
○ Use case : Netflix API: Single API call to nested, conditional,
parallel execution of multiple backend network.

14. Future Integration needs
14
● Integrating applications, services, data, APIs and identity
○ There’s a broad integration challenge than the traditional ESB
related integration.
○ Integration Server, Data Integration, Identity Bus, API
GW/Composition

15. Future Integration needs
15
▪ Performance
o No of transactions and latency
o Ever increasing growth of traffic.
Source : http://blog.mailchimp.com/10m-api-calls-per-day-more/

16. Future Integration needs
16
● Performance
○ No of transactions and latency per container
○ Startup Time
○ Memory footprint
○ Distribution size
○ Average CPU consumption, Load Average

17. Future Integration needs
17
▪ Scalability
o Container based scaling
o Scaling based on the integration solution
o E.g : Ability to scale a given integration solution without
scaling others integration solutions.

18. Integration beyond ESB
...
18

19. Can Conventional Integration Middleware Survive ?
▪ Drastic changes in modern enterprise integration needs.
▪ Integration space has widened : sensors, devices, apps, systems,
cloud, government and many more.
▪ Not compatible with architectural paradigms (e.g : server less
architecture, microservices, in container architecture)
▪ All integration middleware available on the market provides more
or less same capabilities.
19

20. Future Integration Needs
● Microservices and ESB/Integration Middleware
○ Increasing adaptation of Microservices Architecture
○ Microservices does not encourage the conventional centralized
integration middleware.
○ Integration middleware is not container native.
20

21. Does Microservices architecture rejects ESB ?
● “Smart Endpoints and Dumb Pipes”
○ No ESB !
○ All the business and routing logic need to be moved to
endpoints.
21From ESB to ‘Smart endpoints and dumb pipes’ (source : YOW! 2016 — Microservices by Martin Fowler)

22. Does Microservices architecture rejects ESB ?
● Strict dogmatic adherence to “Smart Endpoints and Dumb Pipes”
will result in Point to Point (P2P) connectivity between services
and consumers.
22

23. Microservices and Enterprise Integration – In Practice
23
● Netflix API
“The Netflix API is the “front
door” to the Netflix ecosystem
of microservices. As requests
come from devices, the API
provides the logic of
composing calls to all services
that are required to construct a
response.”
Source : https://medium.com/netflix-techblog/engineering-trade-offs-and-the-netflix-api-re-architecture

24. Microservices and Enterprise Integration – In Practice
24
● Uber
Uber uses ‘Edge Services’ which
are exposed to the external
client/mobile applications and the
service orchestration logic is
burnt into the edge service.
Source : https://www.infoq.com/presentations/uber-darwin

25. Microservices and Enterprise Integration – In Practice
25
● Paypal
The API façade layer exposes
Paypal business functionalities
to various internal and external
client applications and
orchestration logic resides at
that layer.
Source : https://www.infoq.com/presentations/paypal-api-evolution

26. Microservices and Enterprise Integration – In Practice
26
● Microservices still requires an integration layer that can implement
service/API orchestrations and connect to any other external
system.
● Use monolithic centralized integration middleware?
- No !
● Need of ‘Decentralized Integration Middleware’

27. Pragmatic Microservices Architecture
27

28. Decentralized Integration
28
● No centralized integration bus/ESB.
● Build independent integration scenarios using an integration
framework :
○ Micro-Integrations
○ Integration Microservices

29. Micro-Integration/Integration Microservices
29
● Build a specific integration scenario and run that scenario
independently with a lightweight integration framework.
● One integration scenario per each runtime.
● Runtime is extremely lightweight fully container native.
● Develop, deploy and scale each integration scenarios
Independently.

30. Micro-integration Patterns
30
● Integrating Microservices

31. Micro-integration Patterns
31
● Integrating applications, services and data.

32. Micro-integration Patterns
32
● Integrating Microservices and other conventional applications

33. Scaling Micro-integration
33
● Scaling Micro-integrations
5 – Instances
2 – Instances
10 – Instances

34. Types of Micro-integration

35. The Actual Realization
...
35

36. Next generation WSO2 Integration Platform
Addressing the future Integration needs.

37. Ballerina
37
▪ A new programming language, that is designed and
optimized for integration, with text and graphical
syntaxes
▪ Sequence diagram programming model
▪ Modern, network-aware, data-aware, security-aware
programming system inspired by Java, Go, Maven, ...

38. Ballerina
38
▪ Graphical/textual parity with ability to switch back and forth
▪ Common integration capabilities are baked into the
language:
• Deep HTTP/REST/Swagger alignment
• Connectors for Web APIs and non-HTTP APIs
• Support for JSON, XML, (No)SQL data and mapping
▪ No magic – no weird syntax exceptions, everything is
derived from a few key language concepts
▪ Maximize developer productivity and abstraction clarity
▪ Support high performance implementations: low latency,
low memory and fast startup

39. Ballerina Concepts
39

40. Ballerina Concepts
40

41. Ballerina Concepts
41

42. Ballerina Concepts
42

43. Ballerina Concepts
43

44. State of the art Integration with Ballerina
44
▪ Light-weight container native runtime — Micro-
integrations.

45. State of the art Integration with Ballerina
45
▪ Light-weight container native runtime — Micro-
integrations.

46. State of the art Integration with Ballerina
46
▪ Revolutionized Graphical and textual representation
o Innovative textual and graphical programming syntax
based on Sequence diagram metaphor
o The graphical and textual syntaxes would maintain parity
and would be 100% interchangeable.
o Unlike GPLs (Java, Node.js), Ballerina is designed and
optimized for integration - ease of integration

47. State of the art Integration with Ballerina
47
▪ Orchestrations, Choreography and Multi-party
interactions
o Simpler to model complex orchestrations

48. State of the art Integration with Ballerina
48
▪ Orchestrations, Choreography and Multi-party interactions
o Multi-party interactions, Parallel-executions

49. State of the art Integration with Ballerina
49
▪ Support diverse integration requirements
o Client and server connectors for HTTP 1.1/2,
WebSockets, JMS, (S)FTP(S) and more
o Client connectors for BasicAuth, OAuth, AmazonAuth
o Connectors for Web APIs: Twitter, GMail, LinkedIn,
Facebook, Lambda Functions
o EIPs – As language features
o Graphical Data/Type mapping

50. State of the art Integration with Ballerina
50
▪ Performance
o Ballerina uses latest WSO2 Netty HTTP transport (over
5x of performance improvement compared to traditional
HTTP transport implementations)
o Super-fast startup and low memory foot print. (100%
container native)
o Fully non-blocking execution.

51. State of the art Integration with Ballerina
51
▪ Streamlined design, development, testing and
deployment life cycle
“Technology for developing, maintaining, testing, deploying, and governing
interfaces between applications, machines, or databases” (source : Forrester
TechRadar™: Integration Technologies 2015

52. State of the art Integration : Ballerina
52
▪ Streamlined design, development, testing and
deployment life cycle
o Configuration over code myth?
o Ballerina is a programming language- Hence it support
all the generic software development lifecycle stages.
o Packaging, sharing and executing
o Ballerina composer : Graphical/textual editor/debugger
o Testerina and Dockerina
o Docker – support

53. WSO2 Enterprise Integrator 7.0
53
▪ EI 7.0 : Ballerina replaces the ESB, Data Services
runtime.
▪ EI will have two parallel versions for a long time.
▪ Migration tool will be able to handle about 80% of ESB
and Data Services to Ballerina.
▪ We will continue to evolve and support Enterprise
Integrator v6!
- EI v6 will have continuing minor releases
- ESB users will not be left hanging

54. Conclusion
54
▪ Integration middleware is not disappearing…
▪ Rather growing to cover broad integration scenarios.
▪ Micro-integration will rule the world.
▪ Next generation WSO2 Integration Platform is
addressing those new paradigm shifts in Enterprise
Integration with Ballerina.
▪ Ballerina 1.0 GA will be available as a fully fledged
production-ready programming language for building
integrations

55. THANK YOU
wso2.com