Big Data LDN 2018: ACCELERATING YOUR ANALYTICS JOURNEY WITH REAL-TIME AI

© Copyright 2000-2018 TIBCO Software Inc.
Accelerating Your Analytics
Journey With Real-Time AI
Michael O’Connell
Chief Analytics Officer
TIBCO
Twitter: @MichOConnell

3
AI Success Stories
FINANCE:
Fraud Detection
MANUFACTURING:
Anomaly Detection
HEALTHCARE:
Patient Diagnosis
TRANSPORTATION:
Demand Forecasting
E-COMMERCE:
Recommender Systems
RETAIL:
Inventory Optimization

4
Data Flow: Event — Data — Insight
1/ All Data Begin as
Real-Time Events
2/ Analytics on
Accumulated Data
3/ Insights are Perishable
=> Take Action !!

5
My AI Timeline
1991 - 1997 2007 2008 - 2010 2012 2016 2018
KBS & Statistics Machine Learning Data Mining KDD Big Data Artificial Augmented
Ensembles Intelligence Analytics

6© Copyright 2000-2018 TIBCO Software Inc.
Recommender Systems
Suggestion Engines
E-COMMERCE:
Recommender Systems

7
Typical Applications of
Recommender Systems
Consumer Recommender Apps
• Movies (Netflix)
• Songs (Spotify)
• Books (Goodreads)
• eCommerce Products (Amazon)
AI Suggestion Engines
• Collaborative Filtering
• Factor Analysis / Latent Variables
• Content-based Filtering

8
Netflix “The Goal is to Become HBO before
they Become US”
2007 2011 2013 2018
$1M competition House of Cards Netflix on Top
Recommender Deal with NO PILOT Beat HBO
2018 : $8B on Original Programming
Most Emmy Award Noms – beat HBO
2007 : BellKor win Netflix $1M challenge
New AI Suggestion Engine (ML Ensemble)
2012 : 113 Pilots, $400M, 13 to season 2
Netflix grabbed House of Cards – NO PILOT

9
Spotfire A(X) Analytics Experience – Today!
Augmented
Search & AI-Powered Insights
Start in seconds, instant insights
Automated
Automagical Dataflows
Author & audit with automatically
recorded dataflow steps
Agile
Reimagined User Interface
Agile exploration made even easier
Accelerated
Real-time Insights
Real-time awareness and action

10
Spotfire X - AI Recommendations
Variable Relationships Algorithm
• User selects target variable
• AI algorithm discovers variable relationships to target
• Spotfire displays in order of strength - best practices graphics
• 3 clicks to brush-linked dashboard
Search = NLQ
• User searches multiple data tables + relationships
• Spotfire displays appropriate best-practice graphics
• Brush-linked dashboard entirely from chat interaction
Users and Data
• Broad user persona
• Scalable – wide data / Flexible – iterative, simple curation
Auto insight discovery & display

11
Spotfire AI Recommendations: AirBNB NYC
1. Target = Price 2. AI Suggestions = City, Room Type

12
Spotfire AI Recommendations: Telco Churn
1. Target = Customer Status 2. AI Suggestions = Churner ‘friends’, overcharging, handset

13
Spotfire X: AI Suggestions
Automated, Augmented insight discovery & display
Variable Relationships Algorithm
• User selects Target Variable
• AI algorithm
• Discovers variable relationships to target
• Spotfire displays in order of strength
• Best practices graphics
• 4 clicks to brush-linked dashboard
Users and Data
• Scalable – including wide data
• Flexible – iterative, rapid configuration

14
Spotfire X: NLQ Search
Augmented NLQ search & display
Search = NLQ
• User searches multiple data tables +
relationships
• Dashboard and data panel search
areas
• NLQ displays appropriate (best-
practice) graphs
• Brush-linked dashboard
constructed entirely from chat
interaction
Users and Data

Select a Target See Graphs
Data
preparation
Feature
relationships
Ranking
+
MORE LIKE THIS shows extra graphs with the same features
Adds graph to the page
Analyst Business User
Tell me about “Target”
Continue on the journey: adorn, manipulate, explore, drill
Spotfire Recommendations Visual Analytics

Anomaly Detection
MANUFACTURING:
Anomaly Detection

17
Typical Applications of
Anomaly Detection
IoT & Engineering
• Formula One
• Production Surveillance
• Oil-Rig Drilling Optimization
• Predictive Maintenance (PdM, CbM)
• Maverick Lot Detection
Financial Services
• Trade Surveillance
• Fraud Detection
Healthcare & Pharmaceutical
• Clinical Trial safety signals
• Patient risk – cardiac arrest, sepsis,
surgery infection
Key Issue – Understand Variability

19
Data for Data Science
Data created during a weekend is 400GB+
Drivers – Teams, Base Times
Outings – Tyres, Fuel
Tyres – Type, Condition
PitStop – Lap, Issues
Lap and Sector – Car position
GPS on all Cars – Resolution in cm

20
Performance Optimization
Race Data
• Every car in every race
• GPS in cm
Mercedes Car Data
• R&D testing
Analytics
• Engineering optimization
• Race analysis
• Race strategy
TIBCO & Mercedes-AMG Petronas
Motorsport Analytics

21
Driver and Car

Recent Races - Barcelona

Recent Races - Monaco

© Copyright 2000-2018 TIBCO Software Inc. -
CONFIDENTIAL

25
Anomaly Detection
Oil Rigs
Production Surveillance
Drilling Optimization

26
Oil & Gas Production
Transformer
ESP Controller
Well head sensors
P
P P
Controller
VSD
RTURTU
Remote Terminal Unit
Antenna
• Electric Submersible Pumps
• Real-time data
• Electrical parameters
• Downhole pressures and
temperatures
• Surface pressures
• All parameters monitored 24x7
• Prevent failures (extend equipment
life)
• Enhance production (keep systems
running)

27
Industrial IoT: Insight Step
TIBCO Spotfire Analysis
1. Navigate to equipment with issues
Focus on best time range
2. Calculate features to define anomaly
Include data prep steps
3. Calculate control limits on reference set
Normal functioning assets
4. Backtest rules/limits on data history
Minimize false positive rate

28
Industrial IoT: Action Step
TIBCO Spotfire Action
1. Read / process data from sensors
Clean data, calculate features
2. Test features against thresholds
Features, thresholds from Spotfire
3. Send notifications
Immediate action / long term trend
4. Write clean data & notifications to ODS
Identify bad actors from notifications
Ongoing Spotfire analysis of insights
Event Sequence
1. Problem detected
2. Message sent to field staff phone
3. Field staff shut down equipment

29
Value
• Extend equipment life
• Increase production / revenue
• Systematize data flow – real-time analytics
• Identify bad actors – prioritize operations
• Spread organization best practices digitally
• SME into digital form
• Standard workflows and operational practices
• >$x00M savings / year in ESPs
• Applications in other artificial lift systems

Production Surveillance
Control Center

32
Wellbore Trajectory Correction
• Real-time monitor the actual well path; propose a well path back to plan well path once
the actual well is deviated from plan well path
• Real-time determination of the “Best Yet-to-Drill Path”- Spotfire
Source: Z. Liu, R. Samuel, SPE-170861-MS

Understanding Variability
Neural Net Autoencoder
Reconstruction Error
vs Time
Cluster
Incidents
Root Cause
Low rank approximation
to high dimensional space

Real-time Equipment Monitoring
Global View
• Data from MES, Equip sensors
• Monitor Factory Metrics: OEE
Factory View
• Monitor Equipment Status
• Predictive maintenance
Sensor View
Statistical Process Control:
• Univariate, Multivariate,
Algorithmic anomaly detection

36
Autoencoder for Anomaly Detection:
Spotfire & TIBCO Data Science & StreamBase

HEALTHCARE:
Patient Diagnosis
Patient Diagnosis

38
Patient Diagnosis &
Intervention
Analytics & Real-Time Surveillance
• Cardiac Arrest Monitoring (UCM)
• Surgical Site Infections (U Iowa)

39
The Surgical Site Infection Problem
• Cost of hospital acquired infections
in US is in excess of $45 billion
• Readmissions lead to $41.3 billion in
costs
• SSI #1 cause of HAI - 21.8%
• SSI #1 cause of readmission in
surgical patients - 38.8%
Magill, S.S., et al. (2014). Multistate point-prevalence survey of health care-associated infections. N Engl J Med 370, 1198–1208.
Kassin, M.T., et al. (2012). Risk Factors for 30-Day Hospital Readmission among General Surgery Patients. J Am Coll Surg 215, 322–330.

40
Outcomes — Better than a Pill
58% reduction in SSI in 2 years
74% reduction in SSI in 3 years
Estimated cost reduction during hospitalization:
$2.2 million for every 300 patients undergoing surgery
Gbegnon, A., et al. (2014). Predicting Surgical Site Infections in Real-Time. KDD Proceedings
Cromwell, J. (2018). Machine Learning and High Definition Care: More Powerful than a Pill. TIBCO NOW Proceedings, community.tibco.com

41
Data Flow
TIBCO
Data Science

42
Important Variables, Model Summary
Variable K- NMI P value
SSIs 1 < 0.001
*Duration 0.035 < 0.001
*EBL 0.043 < 0.001
*Ostomy 0.055 < 0.001
Procedure category 0.029 < 0.001
Surgeon 0.021 < 0.001
Total procedures 0.028 < 0.001
*Transfusion 0.028 < 0.001
*Wound class 0.014 < 0.001
Zip code 0.032 0.13
Location 0.013 0.11
OUTCOME
No SSI SSI
No SSI TN=88 FN=5
SSI FP=27 TP=6
* = variable collected in OR in real-time

43
End-State Architecture
EMR
IntegrationLayer
API
Management
Offered
API
Offered
API
Offered
API
Data
Repository
Statistical
Model
Statistical
Model
Analytics
Server
Statistical
Model
Provider Environment Service Environment
TIBCO SaaS Offering
BusinessWorks
Container
Edition
Statistica
Spotfire

In the News

Data Science + Data Viz
AI for Social Good

46
Impact of parking citations on low-income
drivers in San Francisco
TIBCO DATA SCIENCE (ML on big data) with TIBCO SPOTFIRE Visual Analytics
TIBCO Products:
• TIBCO Data Science
• TIBCO Spotfire
Business objective: Discover insights and
make recommendations to SF Office of
Financial Justice for policy review
Collaboration between Tipping Point, SF
Office of Financial Justice and TIBCO
Delivers:
• Big Data – data prep and analytics at
scale
• Visualization (in-db and or in-memory)
of reduced dataset
• Simplified Interface for Business Users
• Web / cloud deployment (AWS)

54
Change in Policy!

55
Spotfire A(X) Analytics Experience
Augmented
Search & AI-Powered Insights
Start in seconds, instant insights
Automated
Automagical Dataflows
Author & audit with automatically
recorded dataflow steps
Agile
Reimagined User Interface
Agile exploration made even easier
Accelerated
Real-time Insights
Real-time awareness and action

TIBCO Cloud Integration
TIBCO Cloud LiveApps
TIBCO Cloud Messaging
TIBCO Cloud Spotfire
- With TIBCO Runtime R
TIBCO Cloud Mashery
TIBCO Connected Intelligence Cloud
TIBCO Cloud
Manage all your TIBCO cloud services from one location.
Leverage services within services.
One solution for your cloud needs.
Customer Servicing Billing
Account Management
Single Sign-On
Service RegistryTeam Management

Community and Resources
Ecosystem to help customers succeed with analytics and drive adoption
community.tibco.com
community.tibco.com

59
Questions & Contact
Thank you!
Michael O’Connell
Chief Analytics Officer
TIBCO
moconnell@tibco.com
@MichOConnell

Big Data LDN 2018: ACCELERATING YOUR ANALYTICS JOURNEY WITH REAL-TIME AI

More Related Content

What's hot

Similar to Big Data LDN 2018: ACCELERATING YOUR ANALYTICS JOURNEY WITH REAL-TIME AI

More from Matt Stubbs

Recently uploaded

Big Data LDN 2018: ACCELERATING YOUR ANALYTICS JOURNEY WITH REAL-TIME AI