Applying R in Streaming
and Business Intelligence Applications
Lou Bajuk-Yorgan
Sr. Dir., Product Management
TIBCO Software
lbajuk@tibco.com
@loubajuk
© Copyright 2000-2014 TIBCO Software Inc.
1
Analytic Challenges for Enterprises
• Big Data
– More and more data, and the expectation to
do something with it
• Competitive Pressures
– Deeper insights into data--Apply Advanced
Analytics
– Smarter Decisions--Broaden analytic usage
to wider community beyond Data Scientists
– Faster Decisions—both human and
automated
• Agile response to evolving opportunities
and threats
– Answers (and the questions to ask) change
rapidly
• Agile
– Easy prototyping of new models
and analysis
• Deeper insights
– Huge array of analytic methods
available
– The “best” method to solve a
given problem is likely available
• Performance
• Not designed for real time or Big Data
applications
• Broader usage
• Hard for non-Data Scientist to use
directly
• Challenging to integrate into enterprise
applications
• Performance, commercial support and
Intellectual Property concerns
• Compromises which impact Agility
• Recode in a new, less agile environment
• Rewrite, use specialized R packages to
solve one problem better
R can help… …but has it’s own challenges
What would the ideal solution look like?
• A single environment that would allow you to prototype in R, and deploy to
production in R
– Without recoding, without delay, without compromises
– Enable agile response to changing opportunities and threats
Requires
• Analytic flexibility, power and breadth of R
• High performance, scalable, robust platform
• Easy to embed in Business Intelligence, Real time and custom applications
• Fully supported for mission critical applications
• Allows R users to continue to work in their preferred development environments (e.g., RStudio)
TIBCO Enterprise Runtime for R (TERR)
• Unique, enterprise-grade engine for the R
language, built from the ground up by TIBCO
– Based on TIBCO’s long history and expertise with S+
– Better performance and memory management than open
source R
• Designed for R language compatibility
– Wide range of built-in analytic methods
– Compatible with thousands of CRAN packages (dplyr,
data.table, etc.)
• Designed for commercial embeddability
– TIBCO licensed & supported product
– Not GPL, not a repackaging of the Open source R engine
• TERR extends the reach of R in the enterprise
– Develop code in open source R
– Deploy on a commercially-supported and robust platform
– Without the delay and cost of rewriting your code
– Embed in Data Discovery, BI and real time applications
Example 1: Embedded TERR in Spotfire
• Spotfire: Data Discovery and Visualization platform for Business Users and Analysts
– Separate analytics platform, independent of TERR/R
• Easily enhance Spotfire analyses and applications with R language scripts
– Extend the impact of the Data Scientist/R by making their analytic insights available to a wider audience
Write R code directly in Spotfire;
TERR executes locally or on server
Manage TERR analytics locally or
in Server to reuse across
community
Deploy TERR-powered
applications to the web
Illustrating the power of embedded Advanced Analytics
Advanced Analytic Applications in Spotfire
Customer Churn:
• Retain your most profitable customers
• Increase upsell, decrease churn
Fraud Detection:
• Reduce losses due to fraudulent
transactions
Supply Chain Optimization:
• Anticipate peaks and lulls
• Optimize distribution centers
HR Planning:
• Predict employee attrition and optimize
retention
• Real-time advanced analytics
– Apply predictive model in response to some triggering event
 Sensors on industrial equipment trending negative; customer walks into your store or purchases online,
etc.
– Trigger the right decision in response
 Extend a mobile offer to a customer; stop a fraudulent transaction in process; alert the equipment
operator or shut down the equipment
Example 2: TERR and Streaming Data
Model
Develop model
Deploy via TERR in
TIBCO Streambase
Act
Automatically monitor
real-time transactions
Automatically trigger
action
Analyze
Analyze data in Spotfire
Uncover patterns,
trends & correlations
• Oil & Gas Extraction
– Maintenance Downtime and Equipment failures
are costly
– Engineers track sensor data to find leading
indicators
• Temperature, vibration, etc.
• Engineers usually use ad hoc rules on leading
indicators
– R/TERR used to develop predictive models for
preventative maintenance
– Deployed in real-time systems, alert when
maintenance recommended
Predictive Maintenance for Oil & Gas
© Copyright 2000-2013 TIBCO Software Inc.
• Port Congestion Detection
– Real time system triggers TERR
– Analyzes port congestion
– Recommends reduction of speed if
no berths available
• Maritime Abnormality Detection
– Based on Automatic Identification
System info, TERR calculates
likelihood of deviation from normal
sailing routes
– Alerts carrier & operator
Transportation and Logistics Optimization
Use TERR in your familiar tools
RStudio IDE
– Free, open source IDE widely used by the
R Community
– Fully compatible with TERR Developer
Edition
KNIME
– Free, open source workflow tool for data
management and analysis
– TERR fully compatible with KNIME
Interactive R Statistics Integration nodes
TERR is R for the Enterprise
• Develop code in open source R, deploy on commercially-supported, and
robust platforms
– Without recoding, without compromises
– Save time & money, quickly respond to new threats and opportunities
• Tightly & efficiently embed R language functionality
• Extend the power of R to a wider audience, more applications
• TERR Community at community.tibco.com
– Resources, Documentation, R compatibility, FAQs, Forums
– Predictive Analytics overview and resources
• Free TERR Developer Edition
– Full version of TERR engine for testing code prior to deployment
– Supported through TIBCO Community, download via tap.tibco.com
• Spotfire Free Trial: http://spotfire.tibco.com/trial
• R Consortium Founding Member www.r-consortium.org
Learn more and Try it yourself

R in BI and Streaming Applications for useR 2016

  • 1.
    Applying R inStreaming and Business Intelligence Applications Lou Bajuk-Yorgan Sr. Dir., Product Management TIBCO Software lbajuk@tibco.com @loubajuk © Copyright 2000-2014 TIBCO Software Inc. 1
  • 2.
    Analytic Challenges forEnterprises • Big Data – More and more data, and the expectation to do something with it • Competitive Pressures – Deeper insights into data--Apply Advanced Analytics – Smarter Decisions--Broaden analytic usage to wider community beyond Data Scientists – Faster Decisions—both human and automated • Agile response to evolving opportunities and threats – Answers (and the questions to ask) change rapidly
  • 3.
    • Agile – Easyprototyping of new models and analysis • Deeper insights – Huge array of analytic methods available – The “best” method to solve a given problem is likely available • Performance • Not designed for real time or Big Data applications • Broader usage • Hard for non-Data Scientist to use directly • Challenging to integrate into enterprise applications • Performance, commercial support and Intellectual Property concerns • Compromises which impact Agility • Recode in a new, less agile environment • Rewrite, use specialized R packages to solve one problem better R can help… …but has it’s own challenges
  • 4.
    What would theideal solution look like? • A single environment that would allow you to prototype in R, and deploy to production in R – Without recoding, without delay, without compromises – Enable agile response to changing opportunities and threats Requires • Analytic flexibility, power and breadth of R • High performance, scalable, robust platform • Easy to embed in Business Intelligence, Real time and custom applications • Fully supported for mission critical applications • Allows R users to continue to work in their preferred development environments (e.g., RStudio)
  • 5.
    TIBCO Enterprise Runtimefor R (TERR) • Unique, enterprise-grade engine for the R language, built from the ground up by TIBCO – Based on TIBCO’s long history and expertise with S+ – Better performance and memory management than open source R • Designed for R language compatibility – Wide range of built-in analytic methods – Compatible with thousands of CRAN packages (dplyr, data.table, etc.) • Designed for commercial embeddability – TIBCO licensed & supported product – Not GPL, not a repackaging of the Open source R engine • TERR extends the reach of R in the enterprise – Develop code in open source R – Deploy on a commercially-supported and robust platform – Without the delay and cost of rewriting your code – Embed in Data Discovery, BI and real time applications
  • 6.
    Example 1: EmbeddedTERR in Spotfire • Spotfire: Data Discovery and Visualization platform for Business Users and Analysts – Separate analytics platform, independent of TERR/R • Easily enhance Spotfire analyses and applications with R language scripts – Extend the impact of the Data Scientist/R by making their analytic insights available to a wider audience Write R code directly in Spotfire; TERR executes locally or on server Manage TERR analytics locally or in Server to reuse across community Deploy TERR-powered applications to the web
  • 7.
    Illustrating the powerof embedded Advanced Analytics
  • 8.
    Advanced Analytic Applicationsin Spotfire Customer Churn: • Retain your most profitable customers • Increase upsell, decrease churn Fraud Detection: • Reduce losses due to fraudulent transactions Supply Chain Optimization: • Anticipate peaks and lulls • Optimize distribution centers HR Planning: • Predict employee attrition and optimize retention
  • 9.
    • Real-time advancedanalytics – Apply predictive model in response to some triggering event  Sensors on industrial equipment trending negative; customer walks into your store or purchases online, etc. – Trigger the right decision in response  Extend a mobile offer to a customer; stop a fraudulent transaction in process; alert the equipment operator or shut down the equipment Example 2: TERR and Streaming Data Model Develop model Deploy via TERR in TIBCO Streambase Act Automatically monitor real-time transactions Automatically trigger action Analyze Analyze data in Spotfire Uncover patterns, trends & correlations
  • 10.
    • Oil &Gas Extraction – Maintenance Downtime and Equipment failures are costly – Engineers track sensor data to find leading indicators • Temperature, vibration, etc. • Engineers usually use ad hoc rules on leading indicators – R/TERR used to develop predictive models for preventative maintenance – Deployed in real-time systems, alert when maintenance recommended Predictive Maintenance for Oil & Gas © Copyright 2000-2013 TIBCO Software Inc.
  • 11.
    • Port CongestionDetection – Real time system triggers TERR – Analyzes port congestion – Recommends reduction of speed if no berths available • Maritime Abnormality Detection – Based on Automatic Identification System info, TERR calculates likelihood of deviation from normal sailing routes – Alerts carrier & operator Transportation and Logistics Optimization
  • 12.
    Use TERR inyour familiar tools RStudio IDE – Free, open source IDE widely used by the R Community – Fully compatible with TERR Developer Edition KNIME – Free, open source workflow tool for data management and analysis – TERR fully compatible with KNIME Interactive R Statistics Integration nodes
  • 13.
    TERR is Rfor the Enterprise • Develop code in open source R, deploy on commercially-supported, and robust platforms – Without recoding, without compromises – Save time & money, quickly respond to new threats and opportunities • Tightly & efficiently embed R language functionality • Extend the power of R to a wider audience, more applications
  • 14.
    • TERR Communityat community.tibco.com – Resources, Documentation, R compatibility, FAQs, Forums – Predictive Analytics overview and resources • Free TERR Developer Edition – Full version of TERR engine for testing code prior to deployment – Supported through TIBCO Community, download via tap.tibco.com • Spotfire Free Trial: http://spotfire.tibco.com/trial • R Consortium Founding Member www.r-consortium.org Learn more and Try it yourself

Editor's Notes

  • #6 … What is TERR?
  • #8 Since the demo wraps up with the idea of deploying the model to real time systems, it is a good segue
  • #9 Supply Chain Optimization: simulate production and shipping scenarios to anticipate peaks and lulls HR Retention: Predict employee attrition and optimize retention
  • #10 Example use case: real-time correlations for action Automated manufacturing yield analysis. Analyze manufacturing data in Spotfire Deploy model Compare live data to models of good behavior When actual manufacturing usage breaks the model, Spotfire used to understand why