Event Driven
Reference Architecture	
  
John	
  Mathon	
  
VP, Enterprise Evangelism
and Product Strategy
WSO2	
  Inc.	
  
	
  
“How do you share information between
users and applications in a networked
world?”
Platform 1 Platform 2 and 3
Publish/Subscribe Dream
Joe Mary FrankExchange Clearing
Event Driven Computing was born
Subject
Filter
Subject Filter Subject Filter Subject Filter Subject Filter
Real-Time Simultaneous Delivery to all Subscribers
Auto-Discovery and Inclusion
Self-Describing Data
What Point-to-Point looks like after a while
Extremely Brittle
Extremely NON-AGILE
Alternative EDA – Google - Polling has delays
Web Site A
The Cloud
Search
Indexed Site A
For Google it can still be a month before you are indexed
My dream: What if the cloud was Publish/Subscribe?
Web Site A
The Cloud
Search A
Traffic Data GPS info
MappingYelp
The Common EDA Pattern
- Hub and Spoke
Message
Broker
ServerA
AppC
ServerB
ServerC
AppBAppA
Technology is in Rapid Evolution
100 Billion API calls/minute
20,000 Public/80,000 Private APIs
Projects doubling
Every 18 months
1.5 Billion Smartphones
$100 Billion IaaS, PaaS, SaaS, iPaaS, BaaS, DaaS, …
Billions of Users
& IoT
Advantages of Publish/Subscribe Event
Driven Architecture
• Reduced Complexity
– Easily mediated to many
purposes
– Easier Programming
– Reduced Complexity for
Publisher and Subscriber
• Multiple Levels of failure
recovery
– Local distribution even if
network slow or
disconnected
– Action Locally when
possible
– Increased Redundancy
• Reuse of Events
– Things become
current
– Things become smart
• Shared Intelligence
– Reduced complexity of
individual service
• Reduced Load
– Only responsible to
deliver once
• Higher Agility
– Easy to replace
services
– Easy to provide
multiple versions of
services
EDA spurred a set of standard tools
Platform 2 Distributed SOA EDA
The
Network
Mediation/ESB
Activity
Monitor
Message Broker
Business
Process
Data Services
Registry
Discovery
Pattern Detection
Complex Event Processing
Logical Completeness – Use the Best Tool
Dimensionality
X: Stateful or not
Y: batch or real time
Z: low change or high change
X’: Long or short running
Platform 3
The Connected World
• Mobile – 1.5 Billion Smartphones
– 600,000 Mobile Apps
• IoT – Billions of devices
• Social – Billions of users
• APIs – 100 Billion API Calls/Minute
• Devops / PaaS – deliver new software daily
• Cloud – IaaS, PaaS, SaaS, iPaaS, CaaS, BaaS, DaaS -
$130 billion and growing fast
What is needed is a glue for NEW and OLD technology
Cassandra/
Hbase/
Mongo
Social API/App
Store
API Management
IoT
Platform 3
+API Management, BIG Data, Cloud, Social, Mobile,
Internet of Things, Device Management, Security everywhere
Business
Process
Pattern Detection
Complex Event ProcessingThe Cloud
Activity
Monitor +
SOCIAL
BIGDATA
Data Services
RDB +
BigData
Mediation/ESB +
APIs + Cloud
Adapters Social
Message Broker
+ BigData
Gov Registry
+ Social
Enterprise
Store + Mobile
Mgt
IOT Paas / SaaS /
DevOps Cloud
Integration
Identity Management
API Management
Mobile Device
Management
Non-Unitary Evolution
• Cloudification - Virtual Enterprise Evolution
• API Evolution
• Mobile / IoT Evolution
• BigData – Digitization Evolution
Cloudification – Virtual Enterprise Evolution
• Mostly this is about cost reduction
– SaaS Applications
– Personal Cloud Services
– IaaS
– PaaS
– iPaaS
– Security and Identity Management
– Cloud Management
– Best Practices


Complete Cloud Platform



Identity and Access Management




Enterprise App Management
It’s like a
catalogue!
API Evolution
• Mostly this is about top line growth and agility
– Re-factor the Enterprise
– Existing Services made into “API’s”
– Build Mobile Apps
– Offer Services to external parties
– Integration
– Reuse
APIs Taking Over
100 Billion API Calls/Minute
Uninteresting App if it doesn’t use the Cloud
Smartphone1
In a car
Smartphone2
In a restaurant
Waze Uber Yelp
Medical
Monitor
Service
Smartphone4
On the street
Smartphone3
In a taxi
Tesla Glucose
Monitor
Drone
GPS
Emergency
Vehicle
Search
United
Airlines
API Management – A must
• External API Delivery
– You can hack this yourself but will ultimately want a real API
Management solution
• Internal API socialization and reuse
– Enterprise Refactoring
– Key to Enterprise Renewal, increasing innovation
• Management of External APIs
– What are allowed services to use? Governance
• Management of all services
– Quality of Service, Load, accounting, insight into users
Reuse/Re-Factoring Enterprise

API Management



Publish and Socialize to facilitate Reuse
BigData – Digitization Evolution
• Increase Top Line
– Increased sensitivity to customer
– Understanding the customer
– Reaching the customer
– Maximum efficiency in marketing and
sales efforts
– Better serving the customer
BigData	
  Lambda	
  Architecture	
  
	
  Best	
  Prac>ce	
  /	
  Best	
  of	
  both	
  worlds	
  
http://srinathsview.blogspot.co.uk/2014/03/
implementing-bigdata-lambda.html
Mobile / IoT Evolution – 3 Use Cases!
• For the Factory / Enterprise
– Cost Reduction / Improved Efficiency / Automation
• For the Employees
– Productivity of Employees
• For the Customer
– Connected Business
– New Interaction Paradigms
– Increase Stickiness
Event Driven Computing New Era
Platform 3
“How do you share information between users and applications
in a connected Social IoT Mobile world?”
Social IoT
Mobile
Apps
IoT and
Mobile Mgt
API
Mgt and Reuse
BigData
Mediation / Broker
Micro
Services
Analytics
PaaS/DevOps
Security
Perf Mgt
Scaling
EDA Publish / Subscribe Mesh Network
IoT
IoT
IoT
IoT
IoT
IoT
IoT
IoT
IoT
Router1
Router2
No Single Point of Failure / Like the Internet
Functionality = Combination of all Services
Mesh1
Mesh2 Mesh3
Cloud Control and Aggregation
Mesh1
The Cloud
Mesh2
Customers
Employees
UI
IOT Reference Architecture
Device
API MgtDev Mgt
Service Bus BPM
BigData
Long Term
Analysis
Activity Mgt
Cloud
CEP
Cloud Orchestration/Integration
Commu
nication
Prov
ider
OS
Application
ESB, Security
OS
ESB, IM, PEP
Data Infrastructure
Hub
Orchestration
Integration
Rules Engine
Data
Dev Mgt
Rules Engine
User Dashboard
IoT Mgt
Dashboard


IoT Reference Architecture 



EDA Example Architectures
IoT,
Mobile
Trading Floor
Trader Market Data
Visualization
Risk AnalyticsMarket Data Source
Trader Order Placement
Trader Analytics
Visualization
Order Processing
Message Broker / Information Bus
What’s EDA on a trading floor?
Message Broker
Topics
Mediation/ESB
DSS to RDB
Data
FeedData
FeedData
FeedData
FeedData
FeedData
Feed
Trader Order
Entry
Trader Market
View
Trader
Analytics
Message Broker
Queues
Activity
Monitor
Activity Logs
Complex Event
Processor
Risk Desk
Operations
Desk
Business
Process
EDA Architecture: Ufer Taxis
Uber User App
Uber Driver
App
Meetup
Mobile App
United Airline
Mobile App
Mediation/ESB
Business
Processes
DSS to RDB
Message Broker
Payments
Square /
Zuora …Activity
Monitor
DSS Big Data
Activity Log
API and App
Store
Analytics
Peak Demand
Pricing
Dispute Handling
City Configuration
Reserve Service
Monitor Service
Driver Onboard
Driver Rating
Customer Onboard
Driver Interaction
Driver Selection
Driver Payments
Customer Payments
Avail Query
Online Sales
Mediation/ESB
DSS to RDB
Inventory
SKU / UPCs
Pricing
Consumer
Web App
Partner / OEM
Message Broker
Queues
Activity
Monitor
Activity Logs
Complex Event
Processor
Consumer
Mobile App
Business
Process
App
Server
Shipping
API Management
Health Enrollment –Insurance State or HMO
Mediation/ESB
DSS to RDB
Patient Data
Patient Web
Application
Patient Data
Sources
Message Broker
Queues
Activity
Monitor
Activity Logs
Complex Event
Processor
Patient Mobile
Application
Business
Process
App
ServerAPI Management
Patient
Medical
Records
Patient Data
SourcesPatient Data
Sources
Connected Car
API Mgt Dev Mgt
Upgrade
Cust Svc Service Bus
Design
BPM
BigData
Long Term
Analysis
Activity Mgt
Service Bus
Polling LoggingEvents
Brakes
Brakes
Brakes
Brakes
Console
Phone
Watch
3rd Party App /
Device3rd Party App /
Device3rd Party App /
Device
Brakes
Hi Priority
Planning
Lo Priority Apps
Cell
Ven Cloud
CEP - Service
Connected Construction
API MgtDev Mgt
Cust Svc
Service Bus
Design
BPM
BigData
Long Term
Analysis
Activity Mgt
Phone
Vehicles
Tools
Hi Priority PlanningLo Priority
Cloud
CEP
Building Sites
Vehicles
Vehicles
Phone
Phone
Tools
Tools
Tools
Data Gather
DeviceData Gather
Devices
Wharehouse
Wharehouses
Tools
Data Gather
Device
Contractors
Design Contractor
Planning
Tool
Connected Construction
WSO2 “Platform 3.0” Connected Enterprise
Architecture Gives you:
• Event Driven everywhere respond instantly
• Legacy Integration and renewal
• APIs instantly, managed and scalable
• Cloud Architecture and Deployment Flexibility and Speed
• Big Data Everywhere
• Easy Integration to Social
• Export APIs
• Easy Integration to payments, social, gps
WSO2 Advantage:
75 % of your service is
already written and
integrated with the latest
new technology from
BigData, Cloud, Social,
APIs is scalable and
secure




Connected-Health Reference Architecture 



About the Author
• 10 patents, publish / subscribe, multipath low latency high reliability messaging,
Content management in file systems, peer to peer messaging over WAN… others
• founder and CTO of TIBCO
• WSO2 Enterprise Product Strategy and Evangelism
• Twitter: @john_mathon
• Blog: CloudRamblings johnmathon.wordpress.com
Thank	
  You!	
  

WSO2Con EU 2015: Reference Architecture for EDA

  • 1.
    Event Driven Reference Architecture   John  Mathon   VP, Enterprise Evangelism and Product Strategy WSO2  Inc.    
  • 2.
    “How do youshare information between users and applications in a networked world?” Platform 1 Platform 2 and 3
  • 3.
    Publish/Subscribe Dream Joe MaryFrankExchange Clearing Event Driven Computing was born Subject Filter Subject Filter Subject Filter Subject Filter Subject Filter Real-Time Simultaneous Delivery to all Subscribers Auto-Discovery and Inclusion Self-Describing Data
  • 4.
    What Point-to-Point lookslike after a while Extremely Brittle Extremely NON-AGILE
  • 5.
    Alternative EDA –Google - Polling has delays Web Site A The Cloud Search Indexed Site A For Google it can still be a month before you are indexed
  • 6.
    My dream: Whatif the cloud was Publish/Subscribe? Web Site A The Cloud Search A Traffic Data GPS info MappingYelp
  • 7.
    The Common EDAPattern - Hub and Spoke Message Broker ServerA AppC ServerB ServerC AppBAppA
  • 8.
    Technology is inRapid Evolution 100 Billion API calls/minute 20,000 Public/80,000 Private APIs Projects doubling Every 18 months 1.5 Billion Smartphones $100 Billion IaaS, PaaS, SaaS, iPaaS, BaaS, DaaS, … Billions of Users & IoT
  • 9.
    Advantages of Publish/SubscribeEvent Driven Architecture • Reduced Complexity – Easily mediated to many purposes – Easier Programming – Reduced Complexity for Publisher and Subscriber • Multiple Levels of failure recovery – Local distribution even if network slow or disconnected – Action Locally when possible – Increased Redundancy • Reuse of Events – Things become current – Things become smart • Shared Intelligence – Reduced complexity of individual service • Reduced Load – Only responsible to deliver once • Higher Agility – Easy to replace services – Easy to provide multiple versions of services
  • 10.
    EDA spurred aset of standard tools Platform 2 Distributed SOA EDA The Network Mediation/ESB Activity Monitor Message Broker Business Process Data Services Registry Discovery Pattern Detection Complex Event Processing
  • 11.
    Logical Completeness –Use the Best Tool Dimensionality X: Stateful or not Y: batch or real time Z: low change or high change X’: Long or short running
  • 12.
    Platform 3 The ConnectedWorld • Mobile – 1.5 Billion Smartphones – 600,000 Mobile Apps • IoT – Billions of devices • Social – Billions of users • APIs – 100 Billion API Calls/Minute • Devops / PaaS – deliver new software daily • Cloud – IaaS, PaaS, SaaS, iPaaS, CaaS, BaaS, DaaS - $130 billion and growing fast
  • 13.
    What is neededis a glue for NEW and OLD technology Cassandra/ Hbase/ Mongo Social API/App Store API Management IoT
  • 14.
    Platform 3 +API Management,BIG Data, Cloud, Social, Mobile, Internet of Things, Device Management, Security everywhere Business Process Pattern Detection Complex Event ProcessingThe Cloud Activity Monitor + SOCIAL BIGDATA Data Services RDB + BigData Mediation/ESB + APIs + Cloud Adapters Social Message Broker + BigData Gov Registry + Social Enterprise Store + Mobile Mgt IOT Paas / SaaS / DevOps Cloud Integration Identity Management API Management Mobile Device Management
  • 15.
    Non-Unitary Evolution • Cloudification -Virtual Enterprise Evolution • API Evolution • Mobile / IoT Evolution • BigData – Digitization Evolution
  • 16.
    Cloudification – VirtualEnterprise Evolution • Mostly this is about cost reduction – SaaS Applications – Personal Cloud Services – IaaS – PaaS – iPaaS – Security and Identity Management – Cloud Management – Best Practices
  • 17.
  • 18.
    Identity and AccessManagement 
 

  • 19.
  • 20.
    API Evolution • Mostly thisis about top line growth and agility – Re-factor the Enterprise – Existing Services made into “API’s” – Build Mobile Apps – Offer Services to external parties – Integration – Reuse
  • 21.
    APIs Taking Over 100Billion API Calls/Minute Uninteresting App if it doesn’t use the Cloud Smartphone1 In a car Smartphone2 In a restaurant Waze Uber Yelp Medical Monitor Service Smartphone4 On the street Smartphone3 In a taxi Tesla Glucose Monitor Drone GPS Emergency Vehicle Search United Airlines
  • 22.
    API Management –A must • External API Delivery – You can hack this yourself but will ultimately want a real API Management solution • Internal API socialization and reuse – Enterprise Refactoring – Key to Enterprise Renewal, increasing innovation • Management of External APIs – What are allowed services to use? Governance • Management of all services – Quality of Service, Load, accounting, insight into users
  • 23.
  • 24.
    Publish and Socializeto facilitate Reuse
  • 25.
    BigData – DigitizationEvolution • Increase Top Line – Increased sensitivity to customer – Understanding the customer – Reaching the customer – Maximum efficiency in marketing and sales efforts – Better serving the customer
  • 26.
    BigData  Lambda  Architecture    Best  Prac>ce  /  Best  of  both  worlds   http://srinathsview.blogspot.co.uk/2014/03/ implementing-bigdata-lambda.html
  • 27.
    Mobile / IoTEvolution – 3 Use Cases! • For the Factory / Enterprise – Cost Reduction / Improved Efficiency / Automation • For the Employees – Productivity of Employees • For the Customer – Connected Business – New Interaction Paradigms – Increase Stickiness
  • 28.
    Event Driven ComputingNew Era Platform 3 “How do you share information between users and applications in a connected Social IoT Mobile world?” Social IoT Mobile Apps IoT and Mobile Mgt API Mgt and Reuse BigData Mediation / Broker Micro Services Analytics PaaS/DevOps Security Perf Mgt Scaling
  • 29.
    EDA Publish /Subscribe Mesh Network IoT IoT IoT IoT IoT IoT IoT IoT IoT Router1 Router2 No Single Point of Failure / Like the Internet Functionality = Combination of all Services Mesh1 Mesh2 Mesh3
  • 30.
    Cloud Control andAggregation Mesh1 The Cloud Mesh2 Customers Employees
  • 31.
    UI IOT Reference Architecture Device APIMgtDev Mgt Service Bus BPM BigData Long Term Analysis Activity Mgt Cloud CEP Cloud Orchestration/Integration Commu nication Prov ider OS Application ESB, Security OS ESB, IM, PEP Data Infrastructure Hub Orchestration Integration Rules Engine Data Dev Mgt Rules Engine User Dashboard IoT Mgt Dashboard
  • 32.
  • 33.
  • 34.
  • 35.
    Trading Floor Trader MarketData Visualization Risk AnalyticsMarket Data Source Trader Order Placement Trader Analytics Visualization Order Processing Message Broker / Information Bus
  • 36.
    What’s EDA ona trading floor? Message Broker Topics Mediation/ESB DSS to RDB Data FeedData FeedData FeedData FeedData FeedData Feed Trader Order Entry Trader Market View Trader Analytics Message Broker Queues Activity Monitor Activity Logs Complex Event Processor Risk Desk Operations Desk Business Process
  • 37.
    EDA Architecture: UferTaxis Uber User App Uber Driver App Meetup Mobile App United Airline Mobile App Mediation/ESB Business Processes DSS to RDB Message Broker Payments Square / Zuora …Activity Monitor DSS Big Data Activity Log API and App Store Analytics Peak Demand Pricing Dispute Handling City Configuration Reserve Service Monitor Service Driver Onboard Driver Rating Customer Onboard Driver Interaction Driver Selection Driver Payments Customer Payments Avail Query
  • 38.
    Online Sales Mediation/ESB DSS toRDB Inventory SKU / UPCs Pricing Consumer Web App Partner / OEM Message Broker Queues Activity Monitor Activity Logs Complex Event Processor Consumer Mobile App Business Process App Server Shipping API Management
  • 39.
    Health Enrollment –InsuranceState or HMO Mediation/ESB DSS to RDB Patient Data Patient Web Application Patient Data Sources Message Broker Queues Activity Monitor Activity Logs Complex Event Processor Patient Mobile Application Business Process App ServerAPI Management Patient Medical Records Patient Data SourcesPatient Data Sources
  • 40.
    Connected Car API MgtDev Mgt Upgrade Cust Svc Service Bus Design BPM BigData Long Term Analysis Activity Mgt Service Bus Polling LoggingEvents Brakes Brakes Brakes Brakes Console Phone Watch 3rd Party App / Device3rd Party App / Device3rd Party App / Device Brakes Hi Priority Planning Lo Priority Apps Cell Ven Cloud CEP - Service
  • 41.
    Connected Construction API MgtDevMgt Cust Svc Service Bus Design BPM BigData Long Term Analysis Activity Mgt Phone Vehicles Tools Hi Priority PlanningLo Priority Cloud CEP Building Sites Vehicles Vehicles Phone Phone Tools Tools Tools Data Gather DeviceData Gather Devices Wharehouse Wharehouses Tools Data Gather Device Contractors Design Contractor Planning Tool Connected Construction
  • 42.
    WSO2 “Platform 3.0”Connected Enterprise Architecture Gives you: • Event Driven everywhere respond instantly • Legacy Integration and renewal • APIs instantly, managed and scalable • Cloud Architecture and Deployment Flexibility and Speed • Big Data Everywhere • Easy Integration to Social • Export APIs • Easy Integration to payments, social, gps WSO2 Advantage: 75 % of your service is already written and integrated with the latest new technology from BigData, Cloud, Social, APIs is scalable and secure
  • 43.
  • 44.
    About the Author • 10patents, publish / subscribe, multipath low latency high reliability messaging, Content management in file systems, peer to peer messaging over WAN… others • founder and CTO of TIBCO • WSO2 Enterprise Product Strategy and Evangelism • Twitter: @john_mathon • Blog: CloudRamblings johnmathon.wordpress.com
  • 45.