Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Impact of Messaging Standards on Event-Driven Architecture and IoT


Published on

Sumeet Puri (@puri_sumeet) discusses the state and future of messaging standards and protocols such as JMS, MQTT, AMQP and REST, and explains how each affects event-driven architecture as applied to enterprise integration, big data analytics, private and public clouds, and the Internet of Things. Use cases included.

Interested in hearing the narrative in addition to seeing the slides? Contact us!

Published in: Engineering

The Impact of Messaging Standards on Event-Driven Architecture and IoT

  1. 1. © Solace Systems CONFIDENTIAL Sumeet Puri VP, Head of Technology, International The Impact of Messaging Standards on Event-Driven Architecture and IoT
  2. 2. 2 CONFIDENTIAL Diversified Blue-Chip Customer Base Disruptive Products in Rapidly Growing Market Solace Systems Company Overview  Better  Faster  Cheaper o Solace Systems is a provider of hardware and software end-to-end message routers o Solace hardware and software appliances are widely deployed in various Banking, Payments, Telco, IoT, Insurance, Digital and Big Data Systems o Solace appliances are used for Simplicity, Robustness (HA & DR), Performance and Lower TCO o Founded in 2001, HQ in Ottawa, Canada
  3. 3. In Only Out In In Out Out Only Data Movement Patterns – Magic Quadrant
  4. 4. Imagine there were no TCP/IP Still, the missing data movement pieces: Pub Sub | Persistence | Shock Absorption | App Throttling Messaging Standards Networking : Messaging :: TCP/IP : MQTT/AMQP…JMS
  5. 5. 5 CONFIDENTIAL Messaging as a Digital Backbone Not Just Messaging but Data Movement Omni Channel - Mobility Straight Through Processing Integrating Micro services - Faster Time To Market Big Data In Motion Event Driven Architecture REST API GWs No Web Server Data Movement Wiretap for Analytics Slow Consumer Handling Fanout Low Latency Sync and Async REST <-> JMS is a big deal REST Assured Multi protocol Simple Architecture Easy, Fully Integrated Administration & Security Real-Time & Very High Volume
  6. 6. 6 CONFIDENTIAL Aspects of a Digital | IoT Ecosystem The Architecture Cube Infrastructure Fog/ Gateway Layer Cloud/ Datacenter Edge Layer Cloud/ Datacenter Core Layer Functional Capture Analyze Store React Predict CICD&Device Management Security Network Fog Cloud Edge Core Messaging Multi Protocol Standards Aggregation Throttling Connection Scaling Filtering & Routing Fan in -Burst Handling Persistence Slow Consumer Handling Fanout – Lambda Stream Things & Apps Fog Computing Legacy Integration Real Time Analytics GUIs and Dashboards Legacy Gateways Business Logic & Integration [Big] Data Analytics [Big] Data Store Management andMonitoring
  7. 7. 7 CONFIDENTIAL The Waiter Pattern - Sync Waiter Restaurants Don’t Work This Way!! Take Order Go To Kitchen Wait for Dish Serve
  8. 8. 8 CONFIDENTIAL The Waiter Pattern – The Efficient, Normal Waiter o Waiters take the order, validate and move to the next table o Food is served via a callback o This proven pattern is the only one with which mobile apps scale So why do ESBs/API GWs work differently? Because they were built with HTTP shortcuts Take Next Order Submit To Kitchen Dish Ready? Serve
  9. 9. 9 CONFIDENTIAL Order Taking – The SOA Way An order is an order is an order – Payment, Telco, Mining 250ms 2s 2s 3s DWHReportingBusiness Logic Validations and Check Response Time = ~8s if you are lucky ESB/API Stack – Everything Synchronous, over SOAP/HTTP Any problem in the kitchen has direct user impact Poor throttling, load balancing, no isolation Task Time 1 Submit Income 2 Validate .25s 3 Business Logic 2s 4 Reporting 2s 5 Data Ware House 3s 6 Total Response Time ~8s 6 1 2 3 4 5
  10. 10. 10 CONFIDENTIAL Waiter Pattern – The Event Driven Way Much Higher Throughput, Much Better Response Time and Stability 250ms 1s 2s 2s DWHReportingBusiness Logic Validations and Check Response Time = 300ms Kitchen Order Taking and Ack Shock Absorber Task Time 1 Submit Income 2 Validate .25s 3 Reply and Queue 20us Total Response Time 300us 4 Business Logic 2s 5 Reporting 2s 6 Data Ware House 3s Kitchen Decoupled = Happy Customer Throttling, Load Balancing, Slow Consumer Handling 6 1 2 3 4 5 Topic to Q bridging Waiter
  11. 11. 11 CONFIDENTIAL Use Case - Relationship Managers Tab Pub Sub Req Rep Analytics Core Banking ESB (if needed) DWH Market Data TOMS
  12. 12. 12 CONFIDENTIAL Use Case - Messaging as a Digital Backbone Banking, Trading, Payments Straight Through Processing JSON/REST or JMS/MQTT Legacy Gateway REST message: URL for event routing http://mybank/pay/merchId /curr/type/xyz/xyz 0101011000101010111101010101001010101011010 Data In Motion Complex Events Processing Fraud Detection JMSTopic: mybank/pay/> Visual Analytics (Splunk) JMSTopictoQueue: mybank/POS/Monitor /> LosslessQ ShockAbsorber Solace Data At Rest Analytics (Hadoop) JMSTopic: mybank/> Lossless Q Shock Absorber Topic to Q mapping Core Business Logic Connectivity Data Store Cache ESB/Transform ESB/Transform JMS Topic: mybank/pay/*/fx/> mybank/pay/*/aud/> Wildcard routing on REST URLs/Topics Filtering and Topic (subject) Routing with Wildcards Persistence – never lose a message Publish Subscribe - Parallel, Async Processing Multi Protocol REST <> JMS <>AMQP <> MQTT Burst Handling and Throttling – Big Data Ingest Lambda Architecture – Speed and Batch
  13. 13. 13 CONFIDENTIAL Messaging Backbone for Hybrid Cloud Allows Cloud Transformation – Move gradually to the cloud PaaS – Private Cloud IaaS Bosh Provisioning Service Broker MaaS Catalog App App App App App App App Custom Apps Application Platforms Oracle OSB WebSphere App Server DataPower TIBCO BW Jboss Fuse MuleSoft App App App App App App AppApp App AppApp Application portability  Ubiquitous Connectivity  Best-of-breed Messaging App
  14. 14. 14 CONFIDENTIAL B2B DMZ Use Case Banking Event-Driven Architecture Big “Data Lake” Repository & Real-Time Analytics Retail Bus MortgagesCredit Cards Insurance Wealth Mgmt Domain Service Bus Wholesale MobileOnline Micropayments • Bank Machines • Branches • Telephone Dedicated Networks Central Services Integration Services Data Governance Business Benefits • Agility • Low TCO • Robustness • Innovation • Time to Market • Realtime services Orchestration Realtime 360 degree Customer View DR Payments Public Cloud Private Cloud
  15. 15. 15 CONFIDENTIAL Smart Cities: Bus Monitoring REST/MQTT Regulator’s Dashboard Commuter ’s Apps and RIAs Analytics DWH ESB (if needed) CEP JMS Legacy REST Edge CoreFog Legacy Gateway REST/MQTT UseCase o Collect real-time location, loading information and more from 5,000 buses o Optimize bus movement, fleet management, avoid bunching, forecast arrival time and more WhySolace o Scale: smaller footprint o Reliability and robustness o Integration with TIBCO ESB and CEP o Low Latency for predictive analytics o Native support for REST, MQTT, JMS and more without any bridges
  16. 16. 16 CONFIDENTIAL Smart Cities: LTA Next Generation ERP Routing Server Payments Analytics 0 0 0 • 200K Concurrent Connections • MQTT Termination • Request/reply for firmware upgrades • Push Notifications • Persistent Messaging • Shock Absorber • High Throughput • Fanout • Slow Consumer Handling MQTT JMS Edge CoreFog UseCase o Connect 1.5M vehicles (200K at a time) in real time for pay per use road tolls o Suggest better routes, traffic management – real time and long term and value added services/analytics WhySolace o Smaller footprint o Reliability and robustness o Subscription management, MQTT request reply, topic to queue bridging, fanout o Advanced security and monitoring/event generation o Cost half ½ of competition (IBM) MHI OBU
  17. 17. 17 CONFIDENTIAL Smart Cities: Building/Asset Monitoring Preventive Maintenance Solace Bridge o Collect information from building sensors – Preventive Maintenance and Optimization o Lift Open/Close, Lighting, Water, Electricity etc o Limited block pilot, to scale to all blocks Modern Sensors Aggregator – Collect data from legacy sensors at Residential Blocks. Building Block REST/MQTT ESB Core Business Logic (Event Driven) Scheduling ERP Big Data Store and Analytics Edge CoreFog UseCase o Getting real time/batch data from Housing buildings, such as lift data, pipelines, electricals etc. o For security and preventive maintenance WhySolace o Single fabric for Fog, Edge and Core o Scalability – easy to scale the pilot to all blocks o Aggregation per block as needed, every block is fiber connected and has a ruggedized server o Legacy bridging with ESBs o Feeding Big Data store for longer term predictive maintenance and cost/contract optimization o Real time security feeds as necessary, such as roof lock compromise
  18. 18. 18 CONFIDENTIAL UseCase o Real time QC Data sourcing and analytics to reduce recall times o Same data bus for bidirectional contract manufacturer communication WhySolace o Moving from DB-driven bus to Solace cuts recall time from days to minutes o WAN: Move data from 21 factories to HQ using less bandwidth, with lower latency and no loss o Ability to easily add big Data ingestion o Performance, speed and simplicity Smart Manufacturing (Industry 4.0): Real Time Movement of QC Data MES mes/main mes/measure MAIN mes/equip Master System mes/master DB History System mes/history DB OIP ENG’RClient PCClient Mobile Solace API java Solace API C Solace API C# Manufacturer Data At Rest Analytics Real Time Analytics (Recall) WAN Optimized Lossless Data Transfer Head Quarters QC Data Demand Data Control Data Visual Analytics Solac e Solace API HTML5
  19. 19. 19 CONFIDENTIAL G2B DMZ Use Case Smart Cities Event-Driven Architecture Big “Data Lake” Repository & Real-Time Analytics Home AffairsDefence Environment and Industry Housing and Buildings Infocom Dev Authority MobileOnline G2C • Kiosk Machines • Departments • Telephone Dedicated Networks Central Services Integration Services Data Governance Business Benefits • Agility • Low TCO • Robustness • Innovation • Time to Market • Realtime services Orchestration Realtime 360 degree Customer View DR Land Transport Gov Cloud Private Cloud
  20. 20. 20 CONFIDENTIAL ESB - ADAPTERS / TRANSFORMATION (only if needed) Use Case – Telco – Never Miss an Order Customer 360 via a Digital Event Backbone NGIN Service Consumers Process Layer CEP, Analytics, Big Data Lakes etc New systems can be added with standard protocols Core Business Logic South Bound Systems Presentation Layer North Bound Systems Mobile/Devices Thick Client Browser/RIA Sensors PCRF Mobile First and Consumer Apps Glue Service Layer ESB Other Frameworks New systems just connects and publish/subscribe events MDMPostPaid BillingCore Systems CEPFraud Detection Event Data Subscribers IN-Memory Data Grid Security Monitoring Governance Messaging +API Gateway Data River – Solace Enterprise Message Bus HTTP, REST, mqtt Data Lakes B2B Links Omni Channels Added seamlessly
  21. 21. 21 CONFIDENTIAL B2B MQTT, REST, Flat Files, SFTP, etc. DMZ Use Case Telco Event-Driven Architecture Enterprise Messaging Bus Web & Mobile Streaming, JavaScript, REST etc. MobileOnline Customer 360 • TopUp Machines • Branches Dedicated Networks Central Services Integration Services Governance and Analytics Orchestration &Transformation Benefits of Solace-Based Bus • Simplicity: Single HA pair vs many distributed software brokers, single software load vs multiple from different vendors, forcing security patch updates etc. • Pub/Sub functionality: Complete separation of pubs from subs, full Event Driven Architecture • Robustness: Slow consumers handling, fast failover • Scalability/Performance: One appliance pair meets growing needs for years • Manageability: Always on monitoring, centralized security JavaJMS REST DR Public Cloud HDFS Private Cloud PCRF NGIN API Gateway Provisio ning Mobile First Platform Server CBS CEP IN-Memory Cache NODE 1 NODE 2
  22. 22. 22 CONFIDENTIAL Why Solace? o Zero Down Time, Zero Message Loss ‐ Active Active DR o Micro Second Latency at Very High Throughput o Standards – APIs, REST, MQTT, JMS, AMQP on the way – interoperable at wireline level o Experience and References – Digital, Cloud, IoT, Big Data o Simplicity and Lower TCO Primary Data Center Backup Data Center SAN SAN WAN DR Link TCP Connection
  23. 23. 23 CONFIDENTIAL REST Assured! Zero Downtime, Zero Message Loss Lossless REST - Never lose an Order Simple – No Web/App Servers Performance & Reliability Standards Based, Cloud Ready
  24. 24. © Solace Systems CONFIDENTIAL Thank You