Integration in the age of DevOps
5/11/16
Christian Posta
@christianposta
Christian Posta
Principal Middleware Specialist/Architect
Twitter: @christianposta
Blog: http://blog.christianposta.com
Email: christian@redhat.com
•  Author “Microservices for Java developers”
O’Reilly (June 2016)
•  Committer on Apache Camel, Apache
ActiveMQ, Fabric8, many others
•  Worked with large Microservices, web-scale,
unicorn company
•  Blogger, speaker about DevOps, integration,
and microservices
Cloud Native Architectures
Cloud Native Architectures
•  Trying to incorporate new technology?
•  Trying to copy what others (Netflix, Amazon) are
doing?
•  Tactical automation?
•  Created a “DevOps” team?
•  Exploring cloud services?
•  Build/deploy automation?
•  OpenSource?
•  Piecemeal integration?
How are you keeping up with change?
Cloud Native Architectures
If	change	is	happening	on	the	outside		
faster	than	on	the	inside	the	end	is	in	sight.	
Jack Welch, former CEO, GE
Cloud Native Architectures
S&P company life expectancy
Cloud Native Architectures
Source: Dave Gray, The Connected Company
Source: Dave Gray, The Connected Company
DevOps is NOT about efficiency. DevOps is a reorg.
Cloud Native Architectures
•  Faster software delivery
•  Own database (data)
•  Faster innovation
•  Scalability
•  Right technology for the
problem
•  Test individual services
•  Isolation
•  Individual deployments
Microservices helps solve the problem of
“how do we decouple our services and
teams to move quickly at scale to
deliver business value”
Cloud Native Architectures
•  If my services are isolated at the process
level, I’m doing #microservices
I’m doing microservices if…
•  If I use REST/Thrift/ProtoBuf instead of
SOAP, I’m doing #microservices
•  If I use JSON, I’m doing #microservices
•  If I use Docker / SpringBoot / Dropwizard /
embedded Jetty, I’m doing #microservices
People	try	to	copy	Net7lix,	but	they	can	only		
copy	what	they	see.	They	copy	the		
results,	not	the	process.	
Adrian Cockcroft, former Chief Cloud Architect, Netflix
Call	it	what	you	want;	SOA,	microservices	–		
it’s	all	just	distributed	systems	
Christian Posta – Red Hat
Cloud Native Architectures
Cloud Native Architectures
Fallacies of distributed computing
•  Reliable networking
•  Latency is zero
•  Bandwidth is infinite
•  Network is secure
•  Topology doesn’t change
•  Single administrator
•  Transport cost is zero
•  Network is homogenous
https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing
Cloud Native Architectures
If we have to take into account the network,
do we still need integration?
Cloud Native Architectures
We’ll just do reactive, event-driven
distributed systems… still need integration?
Cloud Native Architectures
Yes; we need reliable integration!
•  REST, RPC
•  Messaging (ActiveMQ, JMS, AMQP, STOMP, Kafka, etc)
•  Legacy (SOAP, mainframe, file processing, proprietary)
•  Managed file processing
•  Streaming
•  Message transformation
•  EIPs
Heavy Lifting: Apache Camel for
Microservices
Cloud Native Architectures
Apache Camel to the rescue!
•  Small Java library
•  Distributed-system swiss-army knife!
•  Powerful EIPs
•  Declarative DSL
•  Embeddable into any JVM (EAP, Karaf, Tomcat, Spring
Boot, Dropwizard, Wildfly Swarm, no container, etc)
•  Very popular (200+ components for “dumb pipes”)
Real developers ride Camels!
Cloud Native Architectures
Apache Camel to the rescue!
•  Automatic retries, backoff algorithms
•  Dynamic routing
•  Powerful testing/mocking framework
•  Circuit breakers
•  Backpressure mechanisms
•  Beautiful REST DSL with built in Swagger support
REST DSL
public class OrderProcessorRouteBuilder extends RouteBuilder {	
	
@Override	
public void configure() throws Exception {	
	
rest().post(“/order/socks”)	
	.description(“New Order for pair of socks”)	
	.consumes(“application/json”)	
	.route()	
	 	.to(“activemq:topic:newOrder”)	
	 .log(“received new order ${body.orderId}”)	
.to(“ibatis:storeOrder?statementType=Insert”);	
}	
}
Cloud Native Architectures
Quick Demo of Camel
Cloud Native Architectures
Cloud Native Architectures
Typical problems developing microservices
•  How to run them all locally?
•  How to package them (dependency management)
•  How to test?
•  Vagrant? VirtualBox? VMs?
•  Specify configuration
•  Process isolation
•  Service discovery
•  Multiple versions?
Cloud Native Architectures
Shared infrastructure platforms headaches
•  Different teams
•  Different rates of change
•  VM sprawl
•  Configuration drift
•  Isolation / multi-tenancy
•  Performance
•  Real-time vs batch
•  Compliance
•  Security
•  Technology choices
Cloud Native Architectures
Immutable infrastructure/deploys
•  “we’ll just put it back in Ansible/Chef/Puppet/Salt”
•  Avoid chucking binaries / configs together and hope!
•  Cattle vs Pets
•  Don’t change it; replace it
•  System created fully from automation; avoid drift
•  Eliminate manual configuration/intervention
Docker / Linux Containers
Kubernetes
•  Developer focused workflow
•  Enterprise ready
•  Higher level abstraction above containers for
delivering technology and business value
•  Build/deployment triggers
•  Software Defined Networking (SDN)
•  Docker native format/packaging
•  CLI/Web based tooling
OpenShift
Red Hat OpenShift
Public
PaaS
Service
On-premise
or Private
PaaS
Software
Open
Source
PaaS
Project
Cloud Native Architectures
Cloud Native Architectures
Fuse Integration Services for OpenShift
•  Set of tools for integration developers
•  Package your Fuse/Camel services as Docker images
•  Run locally on CDK (container development kit)
•  Manage them with Kubernetes/OpenShift
•  Flat class loader JVMs
•  Supports Spring, CDI, Blueprint
•  Plugs-in to your existing build/release ecosystem
(Jenkins/Maven/Nexus/Gitlab,etc)
Cloud Native Architectures
Typical problems developing microservices
•  How to run them all locally?
•  How to package them
•  How to test?
•  Vagrant? VirtualBox? VMs?
•  Specify configuration
•  Process isolation
•  Service discovery
•  Multiple versions?
Cloud Native Architectures
Cloud Native Architectures
Quick Demo?
•  Trying to incorporate new technology?
•  Trying to copy what others (Netflix, Amazon) are
doing?
•  Tactical automation?
•  Created a “DevOps” team?
•  Exploring cloud services?
•  Build/deploy automation?
•  OpenSource?
•  Piecemeal integration?
How are you keeping up with change?
Cloud Native Architectures
•  100% open source, ASL 2.0
•  Technology agnostic (java, nodejs, python,
golang, etc)
•  Built upon decades of industry practices
•  1-click automation
•  Cloud native (on premise, public cloud, hybrid)
•  Complex build/deploy pipelines (human
workflows, approvals, chatops, etc)
•  Comprehensive integration inside/outside the
platform
What if you could do all of this right now
with an open-source platform?
•  Docker native, built on top of
Kubernetes
•  1-click CI/CD
•  Logging, Metrics
•  ChatOps
•  API Management
•  iPaaS/Integration
•  Chaos Monkey
•  Lots and lots of tooling/
libraries to make developing
cloud-native applications
easier
http://fabric8.io
Christian Posta
Principal Middleware Specialist/Architect
Twitter: @christianposta
Blog: http://blog.christianposta.com
Email: christian@redhat.com
Questions, Discussion, Demo!

Chicago Microservices Integration Talk

  • 1.
    Integration in theage of DevOps 5/11/16 Christian Posta @christianposta
  • 2.
    Christian Posta Principal MiddlewareSpecialist/Architect Twitter: @christianposta Blog: http://blog.christianposta.com Email: christian@redhat.com •  Author “Microservices for Java developers” O’Reilly (June 2016) •  Committer on Apache Camel, Apache ActiveMQ, Fabric8, many others •  Worked with large Microservices, web-scale, unicorn company •  Blogger, speaker about DevOps, integration, and microservices Cloud Native Architectures
  • 3.
  • 4.
    •  Trying toincorporate new technology? •  Trying to copy what others (Netflix, Amazon) are doing? •  Tactical automation? •  Created a “DevOps” team? •  Exploring cloud services? •  Build/deploy automation? •  OpenSource? •  Piecemeal integration? How are you keeping up with change? Cloud Native Architectures
  • 5.
  • 6.
    Cloud Native Architectures Source:Dave Gray, The Connected Company
  • 7.
    Source: Dave Gray,The Connected Company
  • 8.
    DevOps is NOTabout efficiency. DevOps is a reorg.
  • 9.
    Cloud Native Architectures • Faster software delivery •  Own database (data) •  Faster innovation •  Scalability •  Right technology for the problem •  Test individual services •  Isolation •  Individual deployments
  • 10.
    Microservices helps solvethe problem of “how do we decouple our services and teams to move quickly at scale to deliver business value” Cloud Native Architectures
  • 11.
    •  If myservices are isolated at the process level, I’m doing #microservices I’m doing microservices if… •  If I use REST/Thrift/ProtoBuf instead of SOAP, I’m doing #microservices •  If I use JSON, I’m doing #microservices •  If I use Docker / SpringBoot / Dropwizard / embedded Jetty, I’m doing #microservices
  • 12.
  • 14.
  • 15.
    Cloud Native Architectures Fallaciesof distributed computing •  Reliable networking •  Latency is zero •  Bandwidth is infinite •  Network is secure •  Topology doesn’t change •  Single administrator •  Transport cost is zero •  Network is homogenous https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing
  • 16.
    Cloud Native Architectures Ifwe have to take into account the network, do we still need integration?
  • 17.
    Cloud Native Architectures We’lljust do reactive, event-driven distributed systems… still need integration?
  • 18.
    Cloud Native Architectures Yes;we need reliable integration! •  REST, RPC •  Messaging (ActiveMQ, JMS, AMQP, STOMP, Kafka, etc) •  Legacy (SOAP, mainframe, file processing, proprietary) •  Managed file processing •  Streaming •  Message transformation •  EIPs
  • 19.
    Heavy Lifting: ApacheCamel for Microservices
  • 20.
    Cloud Native Architectures ApacheCamel to the rescue! •  Small Java library •  Distributed-system swiss-army knife! •  Powerful EIPs •  Declarative DSL •  Embeddable into any JVM (EAP, Karaf, Tomcat, Spring Boot, Dropwizard, Wildfly Swarm, no container, etc) •  Very popular (200+ components for “dumb pipes”)
  • 21.
  • 22.
    Cloud Native Architectures ApacheCamel to the rescue! •  Automatic retries, backoff algorithms •  Dynamic routing •  Powerful testing/mocking framework •  Circuit breakers •  Backpressure mechanisms •  Beautiful REST DSL with built in Swagger support
  • 23.
    REST DSL public classOrderProcessorRouteBuilder extends RouteBuilder { @Override public void configure() throws Exception { rest().post(“/order/socks”) .description(“New Order for pair of socks”) .consumes(“application/json”) .route() .to(“activemq:topic:newOrder”) .log(“received new order ${body.orderId}”) .to(“ibatis:storeOrder?statementType=Insert”); } }
  • 24.
  • 25.
  • 26.
    Cloud Native Architectures Typicalproblems developing microservices •  How to run them all locally? •  How to package them (dependency management) •  How to test? •  Vagrant? VirtualBox? VMs? •  Specify configuration •  Process isolation •  Service discovery •  Multiple versions?
  • 27.
    Cloud Native Architectures Sharedinfrastructure platforms headaches •  Different teams •  Different rates of change •  VM sprawl •  Configuration drift •  Isolation / multi-tenancy •  Performance •  Real-time vs batch •  Compliance •  Security •  Technology choices
  • 29.
    Cloud Native Architectures Immutableinfrastructure/deploys •  “we’ll just put it back in Ansible/Chef/Puppet/Salt” •  Avoid chucking binaries / configs together and hope! •  Cattle vs Pets •  Don’t change it; replace it •  System created fully from automation; avoid drift •  Eliminate manual configuration/intervention
  • 30.
    Docker / LinuxContainers
  • 31.
  • 32.
    •  Developer focusedworkflow •  Enterprise ready •  Higher level abstraction above containers for delivering technology and business value •  Build/deployment triggers •  Software Defined Networking (SDN) •  Docker native format/packaging •  CLI/Web based tooling OpenShift
  • 33.
    Red Hat OpenShift Public PaaS Service On-premise orPrivate PaaS Software Open Source PaaS Project
  • 34.
  • 35.
    Cloud Native Architectures FuseIntegration Services for OpenShift •  Set of tools for integration developers •  Package your Fuse/Camel services as Docker images •  Run locally on CDK (container development kit) •  Manage them with Kubernetes/OpenShift •  Flat class loader JVMs •  Supports Spring, CDI, Blueprint •  Plugs-in to your existing build/release ecosystem (Jenkins/Maven/Nexus/Gitlab,etc)
  • 36.
    Cloud Native Architectures Typicalproblems developing microservices •  How to run them all locally? •  How to package them •  How to test? •  Vagrant? VirtualBox? VMs? •  Specify configuration •  Process isolation •  Service discovery •  Multiple versions?
  • 37.
  • 38.
  • 39.
    •  Trying toincorporate new technology? •  Trying to copy what others (Netflix, Amazon) are doing? •  Tactical automation? •  Created a “DevOps” team? •  Exploring cloud services? •  Build/deploy automation? •  OpenSource? •  Piecemeal integration? How are you keeping up with change? Cloud Native Architectures
  • 40.
    •  100% opensource, ASL 2.0 •  Technology agnostic (java, nodejs, python, golang, etc) •  Built upon decades of industry practices •  1-click automation •  Cloud native (on premise, public cloud, hybrid) •  Complex build/deploy pipelines (human workflows, approvals, chatops, etc) •  Comprehensive integration inside/outside the platform What if you could do all of this right now with an open-source platform?
  • 41.
    •  Docker native,built on top of Kubernetes •  1-click CI/CD •  Logging, Metrics •  ChatOps •  API Management •  iPaaS/Integration •  Chaos Monkey •  Lots and lots of tooling/ libraries to make developing cloud-native applications easier http://fabric8.io
  • 46.
    Christian Posta Principal MiddlewareSpecialist/Architect Twitter: @christianposta Blog: http://blog.christianposta.com Email: christian@redhat.com Questions, Discussion, Demo!