The Role of Enterprise Integration in
Digital Transformation
Kasun Indrasiri
Director – Integration Technology
WSO2
Agenda
• Digital Transformation and Integration
• Centralized Integration
• Future Integration needs
• Microservices and Integration
• Decentralized Integration – Micro-Integration
• WSO2’s Next generation integration platform with Ballerina
Digital Transformation and Enterprise Integration
• DT in a nutshell
– Evolving Business Models
– Focus on customer experience
– Optimize Operation
• Brown-field reality requires integration
– Hence Enterprise Integration is a key step towards Digital
Transformation.
Enterprise Integration – In a nutshell
• “Technology for developing, maintaining, testing, deploying, and
governing interfaces between applications, machines, or
databases” – Forrester 2015
State of Integration Technologies Landscape
• Application and data integration technologies are moving to a
dynamic space – Cloud, Mobile, APIs, IoT, Convergence of Data
and Application Integration
Centralized Integration
• A central integration middleware that connects application,
services and data
• Conventional SOA/ESB approach
• Legacy systems to cloud services connected with services, data,
messages and processes.
Centralized Integration
• WSO2 Enterprise Integrator 6.0
Centralized Integration
• WSO2 Enterprise Integrator 6.0 – Runtime Structure
– ESB and DSS as a single runtime
– Message Broker
– Business Process Server
– Application Server
– Analytics (primarily ESB analytics)
Can Conventional Integration Middleware survive?
• Drastic changes in modern Enterprise Integration needs.
• Not compatible with new architectural paradigms (e.g. Container
Architecture, Microservices)
• Almost all integration middleware solution more or less offer the
same set of functionalities for a decade or so.
Future Integration needs
• Growth and diversity of Integration needs
– APIs and SaaS
– Internet of Things
– On-premise applications
• B2B, Proprietary, Legacy systems
Future Integration needs
• Agility and ease of Integration
– Minimum integration skills and operational overhead.
– Customize existing integrations rapidly.
– Visual modeling, debugging, trouble shooting
– Analytics – Statistics, message tracing
– Error handling.
– Streamlined development life cycle.
Future Integration needs
• Orchestration – Implementing complex orchestration logics
– Proliferation of services, APIs and applications to integrate.
– Complexity of the orchestration logic increases
– Need a simple and agile development of orchestration logic – Visual
modeling tools.
Future Integration needs
• Orchestration – Implementing complex orchestration logics
– Netflix API: Single API call to nested, conditional, parallel execution
of multiple backend network.
Future Integration needs
• Integrating applications, services, data, APIs and identity
– There’s a broad integration challenge than the traditional ESB
related integration.
– Integration Server, Data Integration, Identity Bus, API
GW/Composition
Future Integration needs
• Performance
– No of transactions and latency
– Ever increasing growth of traffic.
• E.g: Growth of API calls in 1 year
Source : http://blog.mailchimp.com/10m-api-calls-per-day-more/
Future Integration needs
• Performance (in containers)
– No of transactions and latency per container
– Startup Time
– Memory foot print
– Distribution size
– Average CPU consumption, Load Average
Future Integration needs
• Scalability
– Container based scaling
– Scaling based on the integration solution
• E.g : Ability to scale a given integration solution without scaling
others integration solutions.
Future Integration needs
• Microservices and ESB/Integration Middleware
– Increasing adaptation of Microservices Architecture
– Microservices do not encourage the conventional centralized
integration middleware.
– Integration middleware solutions are not container native.
Microservices architecture rejects ESB?
• Smart endpoints and Dumb pipes.
– No ESB!
– All the business logic and routing logic should be moved to the
endpoints.
Microservices architecture rejects ESB?
• Smart endpoints and Dumb pipes => Point to point Integration
Microservices and Enterprise Integration – In Practice
• Netflix API
– Single API call to nested, conditional, parallel execution of multiple
backend networks.
– Netflix API layer contains the routing logic.
Microservices and Enterprise Integration – In Practice
• Uber API
– Edge services contains the orchestration logic
Decentralized Integration
• No centralized integration bus/ESB.
• Build independent integration scenarios using integration
framework -> Micro-Integrations/Integration Microservices
Micro-Integration/Integration Microservices
• Build a specific integration scenario and run that scenario
independently with a light-weight integration framework.
• One integration scenario per each runtime.
• Runtime is extremely lightweight fully container native.
• Develop, deploy and scale each integration scenarios
Independently.
Micro-integration
• Integrating Microservices
Micro-integration
• Integrating applications, services and data.
Micro-integration
• Integrating Microservices and other conventional applications
Micro-integration
• Scaling Micro-integrations
5 – Instances
2 – Instances
10 – Instances
Types of Micro-Integrations
APIs and Integration
• API expose the internal business functionalities.
• API GW is the central component that serves API traffic and the
central location that security, throttling and caching
• API Gateway is the new ‘Monolith’
APIs and Integration
• API Gateway and Integration – Layered Architecture
Source : http://techblog.netflix.com/2016/08/engineering-trade-offs-and-netflix-api.html
APIs and Integration
• API Gateway and Integration – Layered Architecture
APIs and Integration
• APIs and Micro-integrations
How WSO2 address these modern Integration
Requirements
• Building a integration platform that addresses the future
Integration needs.
• We don’t want to build just a ‘new ESB’.
– But a reusable framework which is shared among similar
integration solutions – applications, services, APIs, data, identity
etc.
• A visual and textual representation.
Next generation WSO2 Integration Platform
• Addressing the future Integration needs.
Ballerina
• A new programming language with text and graphical syntaxes
• Sequence diagram programming model
• Modern, network-aware, data-aware, security-aware
programming system inspired by Java, Go, Maven, ...
Ballerina
• Graphical/textual parity with ability to switch back and forth
• Common integration capabilities are baked into the language:
– Deep HTTP/REST/Swagger alignment
– Connectors for Web APIs and non-HTTP APIs
– Support for JSON, XML, (No)SQL data and mapping
• No magic – no weird syntax exceptions, everything is derived from a
few key language concepts
• Maximize developer productivity and abstraction clarity
• Support high performance implementations: low latency, low memory
and fast startup
Ballerina Concepts
Ballerina
• Complex Orchestrations made simple.
– Sequence diagram metaphor is a simplest way to represent
orchestrations.
Ballerina
• Thinking in ‘Parallel’
– Multi-worker execution
Ballerina
• Connecting anything with anything.
• Client and server connectors for HTTP 1.1/2, WebSockets, JMS,
(S)FTP(S) and more
• Client connectors for BasicAuth, OAuth, AmazonAuth
• Connectors for Web APIs: Twitter, GMail, LinkedIn, Facebook,
Lambda Functions,
Ballerina
• Enterprise Integration Patterns
– Most EIPs are integral part of the Ballerina Programming Language
– if-else, fork-join, for-each etc.
Ballerina
• Type Mapping
– Visual Type mapping
– First class support for type mapping at the language level.
Ballerina
• Performance
– Ballerina uses latest WSO2 Netty HTTP transport (over 5x of
performance improvement compared to traditional HTTP transport
implementations)
– Super-fast startup and low memory foot print.
– Fully non-blocking execution.
Ballerina
• Debugging
– Visual debugging
Ballerina
• Scaling
– Container native
– Independent scaling of each micro-integration
Enterprise Integrator 7.0
• EI 7.0 : Ballerina replaces the ESB, Data Services runtime.
• EI will have two parallel versions for a long time.
• Migration tool will be able to handle about 80% of ESB and Data
Services to Ballerina.
• We will continue to evolve and support Enterprise Integrator v6!
– EI v6 will have continuing minor releases
– ESB users will not be left hanging
WSO2 Hybrid Integration Platform
• We focus on On-premise integration and iPaaS
– WSO2 Integration Cloud – ESB 5 in the cloud with DSS
Source : Gartner
Summary
• Integration middleware is not disappearing…
• Rather growing to cover broad integration scenarios.
• Micro-integration will rule the world.
• Next generation WSO2 Integration Platform is addressing those
new paradigm shifts in Enterprise Integration with Ballerina.
Thank You!

WSO2Con USA 2017: The Role of Enterprise Integration in Digital Transformation

  • 1.
    The Role ofEnterprise Integration in Digital Transformation Kasun Indrasiri Director – Integration Technology WSO2
  • 2.
    Agenda • Digital Transformationand Integration • Centralized Integration • Future Integration needs • Microservices and Integration • Decentralized Integration – Micro-Integration • WSO2’s Next generation integration platform with Ballerina
  • 3.
    Digital Transformation andEnterprise Integration • DT in a nutshell – Evolving Business Models – Focus on customer experience – Optimize Operation • Brown-field reality requires integration – Hence Enterprise Integration is a key step towards Digital Transformation.
  • 4.
    Enterprise Integration –In a nutshell • “Technology for developing, maintaining, testing, deploying, and governing interfaces between applications, machines, or databases” – Forrester 2015
  • 5.
    State of IntegrationTechnologies Landscape • Application and data integration technologies are moving to a dynamic space – Cloud, Mobile, APIs, IoT, Convergence of Data and Application Integration
  • 6.
    Centralized Integration • Acentral integration middleware that connects application, services and data • Conventional SOA/ESB approach • Legacy systems to cloud services connected with services, data, messages and processes.
  • 7.
    Centralized Integration • WSO2Enterprise Integrator 6.0
  • 8.
    Centralized Integration • WSO2Enterprise Integrator 6.0 – Runtime Structure – ESB and DSS as a single runtime – Message Broker – Business Process Server – Application Server – Analytics (primarily ESB analytics)
  • 9.
    Can Conventional IntegrationMiddleware survive? • Drastic changes in modern Enterprise Integration needs. • Not compatible with new architectural paradigms (e.g. Container Architecture, Microservices) • Almost all integration middleware solution more or less offer the same set of functionalities for a decade or so.
  • 10.
    Future Integration needs •Growth and diversity of Integration needs – APIs and SaaS – Internet of Things – On-premise applications • B2B, Proprietary, Legacy systems
  • 11.
    Future Integration needs •Agility and ease of Integration – Minimum integration skills and operational overhead. – Customize existing integrations rapidly. – Visual modeling, debugging, trouble shooting – Analytics – Statistics, message tracing – Error handling. – Streamlined development life cycle.
  • 12.
    Future Integration needs •Orchestration – Implementing complex orchestration logics – Proliferation of services, APIs and applications to integrate. – Complexity of the orchestration logic increases – Need a simple and agile development of orchestration logic – Visual modeling tools.
  • 13.
    Future Integration needs •Orchestration – Implementing complex orchestration logics – Netflix API: Single API call to nested, conditional, parallel execution of multiple backend network.
  • 14.
    Future Integration needs •Integrating applications, services, data, APIs and identity – There’s a broad integration challenge than the traditional ESB related integration. – Integration Server, Data Integration, Identity Bus, API GW/Composition
  • 15.
    Future Integration needs •Performance – No of transactions and latency – Ever increasing growth of traffic. • E.g: Growth of API calls in 1 year Source : http://blog.mailchimp.com/10m-api-calls-per-day-more/
  • 16.
    Future Integration needs •Performance (in containers) – No of transactions and latency per container – Startup Time – Memory foot print – Distribution size – Average CPU consumption, Load Average
  • 17.
    Future Integration needs •Scalability – Container based scaling – Scaling based on the integration solution • E.g : Ability to scale a given integration solution without scaling others integration solutions.
  • 18.
    Future Integration needs •Microservices and ESB/Integration Middleware – Increasing adaptation of Microservices Architecture – Microservices do not encourage the conventional centralized integration middleware. – Integration middleware solutions are not container native.
  • 19.
    Microservices architecture rejectsESB? • Smart endpoints and Dumb pipes. – No ESB! – All the business logic and routing logic should be moved to the endpoints.
  • 20.
    Microservices architecture rejectsESB? • Smart endpoints and Dumb pipes => Point to point Integration
  • 21.
    Microservices and EnterpriseIntegration – In Practice • Netflix API – Single API call to nested, conditional, parallel execution of multiple backend networks. – Netflix API layer contains the routing logic.
  • 22.
    Microservices and EnterpriseIntegration – In Practice • Uber API – Edge services contains the orchestration logic
  • 23.
    Decentralized Integration • Nocentralized integration bus/ESB. • Build independent integration scenarios using integration framework -> Micro-Integrations/Integration Microservices
  • 24.
    Micro-Integration/Integration Microservices • Builda specific integration scenario and run that scenario independently with a light-weight integration framework. • One integration scenario per each runtime. • Runtime is extremely lightweight fully container native. • Develop, deploy and scale each integration scenarios Independently.
  • 25.
  • 26.
  • 27.
    Micro-integration • Integrating Microservicesand other conventional applications
  • 28.
    Micro-integration • Scaling Micro-integrations 5– Instances 2 – Instances 10 – Instances
  • 29.
  • 30.
    APIs and Integration •API expose the internal business functionalities. • API GW is the central component that serves API traffic and the central location that security, throttling and caching • API Gateway is the new ‘Monolith’
  • 31.
    APIs and Integration •API Gateway and Integration – Layered Architecture Source : http://techblog.netflix.com/2016/08/engineering-trade-offs-and-netflix-api.html
  • 32.
    APIs and Integration •API Gateway and Integration – Layered Architecture
  • 33.
    APIs and Integration •APIs and Micro-integrations
  • 34.
    How WSO2 addressthese modern Integration Requirements • Building a integration platform that addresses the future Integration needs. • We don’t want to build just a ‘new ESB’. – But a reusable framework which is shared among similar integration solutions – applications, services, APIs, data, identity etc. • A visual and textual representation.
  • 35.
    Next generation WSO2Integration Platform • Addressing the future Integration needs.
  • 36.
    Ballerina • A newprogramming language with text and graphical syntaxes • Sequence diagram programming model • Modern, network-aware, data-aware, security-aware programming system inspired by Java, Go, Maven, ...
  • 37.
    Ballerina • Graphical/textual paritywith ability to switch back and forth • Common integration capabilities are baked into the language: – Deep HTTP/REST/Swagger alignment – Connectors for Web APIs and non-HTTP APIs – Support for JSON, XML, (No)SQL data and mapping • No magic – no weird syntax exceptions, everything is derived from a few key language concepts • Maximize developer productivity and abstraction clarity • Support high performance implementations: low latency, low memory and fast startup
  • 38.
  • 39.
    Ballerina • Complex Orchestrationsmade simple. – Sequence diagram metaphor is a simplest way to represent orchestrations.
  • 40.
    Ballerina • Thinking in‘Parallel’ – Multi-worker execution
  • 41.
    Ballerina • Connecting anythingwith anything. • Client and server connectors for HTTP 1.1/2, WebSockets, JMS, (S)FTP(S) and more • Client connectors for BasicAuth, OAuth, AmazonAuth • Connectors for Web APIs: Twitter, GMail, LinkedIn, Facebook, Lambda Functions,
  • 42.
    Ballerina • Enterprise IntegrationPatterns – Most EIPs are integral part of the Ballerina Programming Language – if-else, fork-join, for-each etc.
  • 43.
    Ballerina • Type Mapping –Visual Type mapping – First class support for type mapping at the language level.
  • 44.
    Ballerina • Performance – Ballerinauses latest WSO2 Netty HTTP transport (over 5x of performance improvement compared to traditional HTTP transport implementations) – Super-fast startup and low memory foot print. – Fully non-blocking execution.
  • 45.
  • 46.
    Ballerina • Scaling – Containernative – Independent scaling of each micro-integration
  • 47.
    Enterprise Integrator 7.0 •EI 7.0 : Ballerina replaces the ESB, Data Services runtime. • EI will have two parallel versions for a long time. • Migration tool will be able to handle about 80% of ESB and Data Services to Ballerina. • We will continue to evolve and support Enterprise Integrator v6! – EI v6 will have continuing minor releases – ESB users will not be left hanging
  • 48.
    WSO2 Hybrid IntegrationPlatform • We focus on On-premise integration and iPaaS – WSO2 Integration Cloud – ESB 5 in the cloud with DSS Source : Gartner
  • 49.
    Summary • Integration middlewareis not disappearing… • Rather growing to cover broad integration scenarios. • Micro-integration will rule the world. • Next generation WSO2 Integration Platform is addressing those new paradigm shifts in Enterprise Integration with Ballerina.
  • 50.