Actionable Insights - Thompson


Published on

Leveraging your Connectivity, Big Data and Predictive Analytics Infrastructure to Drive Top Line Revenue.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Actionable Insights - Thompson

  1. 1. © 2014 IBM Corporation Actionable Insights Leveraging your Connectivity, Big Data and Predictive Analytics infrastructure to drive top line revenue Ben Thompson Chief Architect IBM Integration Bus July 2014
  2. 2. © 2014 IBM Corporation Capture what’s happening in real-time Generate model of future Predict most likely outcome Proactively optimize business Tap into relevant data with context . . Unleash real-time data flowing throughout enterprise Actionable Insights What’s the Big Idea?
  3. 3. © 2014 IBM Corporation An Example of Actionable Insight MQTT DICOM PACS Predicting Deciding Integrating Imaging Modality Patient Report Monitoring Electronic Medical Record Alert Doctor ODBC JDBC SMS
  4. 4. © 2014 IBM Corporation Industrial Process Control – Machinery Failure OPC OPC Predicting Deciding Integrating Power Consumption Monitoring Temperature SCADA SAP BAPI Vibration RPM Order Part
  5. 5. © 2014 IBM Corporation Solutions at Scale What Big Data means for … Connected Appliances Connected Cars Smartphones Internet TVs Home Hubs Smart Meters Home health devices Connectivity & Integration The Internet of Things Analytics
  6. 6. © 2014 IBM Corporation
  7. 7. © 2014 IBM Corporation MQTT A transport for driving the Internet of Things Lossy or Constrained Network Lossy or Constrained Network Real-World Aware Business Processing High volumes of data/events 1999 Invented by Dr. Andy Stanford-Clark (IBM), Arlen Nipper (now Cirrus Link Solutions) 2011 - Eclipse PAHO MQTT open source project 2004 open community 2013 – MQTT Technical Committee formed Cimetrics, Cisco, Eclipse, dc-Square, Eurotech, IBM, INETCO Landis & Gyr, LSI, Kaazing, M2Mi, Red Hat, Solace, Telit Comms, Software AG, TIBCO, WSO2 • MQTT is a lightweight publish-subscribe protocol with reliable bi-directional message delivery • Open Source • Standards
  8. 8. © 2014 IBM Corporation Passing Data from Things into the Enterprise The Power of MQTT and IBM MessageSight • MQTT’s very compact wire format, results in lower network costs than an HTTP equivalent • Lightweight footprint – protocol will run on low power devices • Clients: C = 80kb; Java = 100kb JavaScript = 80kb • Recovery, store and forward, and publish/subscribe are all provided by the MQTT implementations, and don’t have to be coded into application logic • Simple set of verbs, easy for developers to learn • Easy integration with Systems of Record Lower development costsLower development costs Lower running costsLower running costs • Near real-time push of information • Minimal battery usage • Store and forward messaging • Exactly once delivery (where required) • MQTT’s Event-Driven design point means that a single server can support a million connected users or devices • Publish/Subscribe allows additional functionality to be added without change to existing application code More Flexibility and ScaleMore Flexibility and Scale Improved User ExperienceImproved User Experience
  9. 9. © 2014 IBM Corporation Action HTTP MQTT Get single piece of data 302 bytes 69 bytes (<4 times) Send single piece of data 320 bytes 47 bytes (<6 times) Get 100 pieces of data 12600 bytes 2445 bytes (<5 times) Send 100 pieces of data 14100 bytes 2126 bytes (<6 times) Characteristics HTTP MQTT Style Document-centric, request/response Data-centric, publish/subscribe Verbs GET/POST/POST/DELETE complex spec Pub/Sub/Unsub simple protocol, easy to learn Message size Large message, lots of data in headers 2 bytes in minimum header Quality of Service None, requires custom coding in application 3 levels: best-effort, at-least-once, exactly once Data distribution No distribution mechanism (1-to-1 only) 1-to-none, 1-to-1, 1-to-n Deliver Relevant Information Optimizing network with event-driven notification
  10. 10. © 2014 IBM Corporation • Analytics is the discovery and communication of meaningful patterns in data • Predictive analytics uses statistical techniques to build a model that describes key relationships in data • Predictive models are applied to new observations to estimate the likelihood or values of unknown (usually future) events Predictive Analytics Discovering trends in real-time data in flight 15 Petabytes of big data generated daily 95%of Mobile traffic is data by 2015 15bdevices connected by 2020 420m wearable health monitors by 2014 Big Data from Internet of Things
  11. 11. © 2014 IBM Corporation Analysing the Past, Present and Future Discovering trends in real-time data in flight PAST • Applying analytical techniques to past, archived events • Correlation & Filtering • Advanced Queries – “where is my transaction?” • Data Analyser and Observer! PRESENT • Analysing current in-flight events • Correlation, aggregation, metrics • Calculation of KPIs, real-time dashboard display • Reporter and Observer! FUTURE • Predictive Analytics • Invoke predictive models, trained on past data • Trigger actions based on predicted outcomes • Participant!
  12. 12. © 2014 IBM Corporation Monitoring the Past and Present with IIB Accounting & Statistics, Monitoring and Record & Replay
  13. 13. © 2014 IBM Corporation Message flow statistics - One record is created for each message flow in a server: – Message flow, Server (Execution Group) and Node (Broker) name and UUID – Type of data collected (snapshot or archive) – Processor and elapsed time spent processing messages – Processor and elapsed time spent waiting for input – Number of messages processed – Minimum, maximum, and average message sizes – Number of threads available and maximum assigned at any time – Number of messages committed and backed out – Accounting origin Thread statistics - One record is created for each thread assigned to the message flow: – Thread number (this has no significance and is for identification only) – Processor and elapsed time spent processing messages – Processor and elapsed time spent waiting for input – Number of messages processed – Minimum, maximum, and average message sizes Node statistics - One record is created for each node in the message flow: – Node name and Node type (for example MQInput) – Processor time spent processing messages – Elapsed time spent processing messages – Number of times the node is invoked – Number of messages processed – Minimum, maximum, and average message sizes Terminal statistics - One record is created for each terminal on a node: – Terminal name and Terminal type (Input or Output) – Number of times that a message is propagated to the terminal Accounting & Statistics Real-time publication of summarized system performance data
  14. 14. © 2014 IBM Corporation Publication + Subscription Monitoring your Integrations Publication of actual payload data for later analysis
  15. 15. © 2014 IBM Corporation Business Transaction Monitoring versus Business Activity Monitoring “Archive available for later queries” versus “Real-time view relating to pre-defined KPIs” Both monitoring capabilities have their pros and cons: – Real-time view gives you quicker insight but no post-event searching – Archived view slower to produce insight but complex post-event searching is easy
  16. 16. © 2014 IBM Corporation Data Analysis Iterative Build-time Analysis of large XML documents Create a Data Analysis project, select a set of sample XML documents for analysis, and IIB will generate a Data Analysis Model. Views and filters are provided to navigate through the complex content in a variety of ways. Revealed elements whose content relates to a known code set translation (defined in a glossary) are highlighted. Create Target Model (drag and drop items from the Data Analysis Model to the Target Model). Make further edits to theTarget Model (either for output messages or output to a database). Generate graphical maps which will convert input instance XML documents into instance XML documents which conform to the Target Model. Generate maps for inserts into a database. Use the generated subflow and associated resources in the normal way within the IIB Toolkit
  17. 17. © 2014 IBM Corporation Right-click menu from a “Focus Element” in the Data Analysis Model offers highlighting options. Choosing Highlight All Coexisting Elements: The percentage in the square brackets [nn%] shows the percentage of the instance documents containing the Focus Element which also include the element in question. 66.7% of those instance documents containing TopLevel_Element2 also contain TopLevel_Element1 Data Analysis Tools Highlight co-existing elements
  18. 18. © 2014 IBM Corporation Descendant “0” (Volume Element itself!) Descendant “1” Elements (e.g. Appendix) Descendant “2” Elements (e.g. Bibliography) Descendant “3” Elements (e.g. Bibliography) Descendant “4” Elements (e.g. Name, Author) Data Analysis Tools Highlighting the Min and Max Depth of Descendants
  19. 19. © 2014 IBM Corporation Manufacturing Industry Scenarios Discovering trends in real-time data in flight Rig Mine Factory ππππr2 h
  20. 20. © 2014 IBM Corporation A Pattern for MessageSight Integration
  21. 21. © 2014 IBM Corporation Provide business insight during integration data flows – e.g. intelligent decision making; score then action in-flight request based on a business rule – User creates (e.g.) if-then-else rules – The bus acts on these rules in flow, e.g. for business level routing New Decision Service node – Identifies inputs to business rules from in-flight data • e.g. the customers order from whole request • e.g. the item price from key fields… – Invokes the built-in rule engine – Captures rules output for downstream processing Create rules directly inside Integration Bus toolkit – Significant rules authoring facility built-in – Automatic package & deploy with integration assets – Dynamically reconfigure business rule – Optionally refer to business rules on external ODM decision server – Exploit separate full ODM Decision Center for BRMS scenarios Embedded rules engine for high performance – Rule is executed in the same OS process as integration data flow – Rule update notification ensures consistent rule execution – Optional governance of rules through remote ODM Decision Center Decision Management
  22. 22. © 2014 IBM Corporation Applying Analytics to In-flight Data Analytics node for model based decision making – Find & express patterns in data with analytics models – Analytics equivalent to Business Decision node • Pluggable engine for e.g. R, SPSS, SAS… – 2 key scenarios are “model score” and “model trend” – e.g. %buy additional item, SKU lower than expected Define the model in tools – This is a high value skill; understand & express behaviour – Use historic dataset; this is typically offline scenario – Both built-in tooling and external model import/reference Deploy/Change the Model – Model is encoded into integration flow logic – Deployed with integration solution – Analytics policy for dynamic change without redeploy – Optionally packaged as part of Shared Library Support Using the model in real time – Act on these models in integration flow – Scoring: Synchronous use of model score real-time data – Observing: Compare models in real-time for divergence Key, related considerations – Shared Libraries required with dynamic linkage • All Applications using library “see” re-deploy
  23. 23. © 2014 IBM Corporation Analytics Node Demand is growing for analytics to be a real-time activity As data flows through the enterprise, IIB has visibility to score it against a predictive model Data Scientist Role – Prepares a model based on an analytics engine. – For example R, SPSS, SAS Integration Developer Role – Formats a data stream and applies it to a model Analytics Node – R Scalar variable types: double, integer, character (string), logical (Boolean) – Data frames can be considered like database tables, consisting of labelled and typed columns and unlimited rows Configuration of input and output parameters – XPath expressions point to locations in the input and output trees – Direction of Parameter allows a single properties table to control tree copying and return results from the scoring process Score
  24. 24. © 2014 IBM Corporation Healthcare Industry Scenarios Discovering trends in real-time data in flight Operational – KPI’s – Retrospective view of performance Clinical insights – Real Time Analytic Processing – Interventive care from insight into longitudinal care records Cognitive Analytics – Assisted treatment/diagnosis Data Baby!
  25. 25. © 2014 IBM Corporation Almost 25% of the population is over 65, and that number is growing Medical advances mean people are living longer Services for the elderly account for almost 50% of the social services budget Many more elderly people are choosing to remain at home, even when they are alone Ensure their safety and provide needed services but the city had to find a cost effective way to know when its people needed help A mesh-network of sensors that monitor the home environment— temperature, CO2, water leaks, etc.—of elderly citizens living alone Additional home remote medical interaction with medical professionals, saving trips to the doctor It all works with a little help from “angels” (relatives or friends of the user) who are alerted if there is a problem A new model of social and health service that operates on existing budgets and resources, even as the elderly population increases Provides a technological, but still human, system of care via the remote “angels”—the user can be independent, but not feel alone Social service and health staff can concentrate on people who really need a physical presence with them, while those in the monitoring program maintain an excellent quality of life
  26. 26. © 2014 IBM Corporation Waste Management Combining the Internet of Things, Big Data, Analytics and Mobile! • Weight and type of waste • Excess of waste • Optimization of the collection path • Exception management (bins in wrong places, need of additional bins, replacement of bins etc.) • Send/receive working orders to/from SAP Central Acquisition System Field Management System SAP DB2 IIB WAS (J2EE app.) MQTT client Worklight Application GPS BPM ODM MessageSight HTTP(s) MQTT GPRS/3G RFID reader
  27. 27. © 2014 IBM Corporation Slope aware power train optimization Flooding/Slippery risk aware Driving alert 100 Dynamic/Variable Speed Limit alert & speed control Bus Signal status aware speed control going thru crossing Height/load limit aware fleet driving alert & detouring Accident/congestion aware detouring & navigation Dynamic parking space availability navigation Passenger crowd aware bus dynamic speed management Environment pollution surveillance traffic fencing control & fleet alert ! ! Co2 ! :-) !! Low Bridge The Connected Car Location Awareness: tracking where things are and how things move!
  28. 28. © 2014 IBM Corporation