Extending the Reach of R to the Enterprise with TERR and Spotfire

1,918 views
1,693 views

Published on

An overview of how TIBCO integrates dynamic, interactive visual applications in Spotfire with predictive and advanced analytics in the R language, using TIBCO Enterprise Runtime for R--our R-compatible, enterprise-grade platform for the R language.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,918
On SlideShare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Extending the Reach of R to the Enterprise with TERR and Spotfire

  1. 1. Extending the Reach of R to the Enterprise Lou Bajuk-Yorgan Sr. Dir., Product Management TIBCO Spotfire 1 © Copyright 2000-2014 TIBCO Software Inc.
  2. 2. Extending the Reach of R to the Enterprise • TIBCO, S+, and embracing R in Spotfire • Challenges of R for Enterprise applications • TIBCO Enterprise Runtime for R (TERR) • Benefits for organizations (and individuals) who use R • Examples of TERR integration and performance • Learn more and try it yourself -2
  3. 3. Our Journey to TERR • John Chambers developed the S language at Bell Labs – Starting in the mid 70’s • Insightful (Statsci) founded to commercial S as S+ in 1987 – The “plus”: statistical libraries, documentation, and support – Later focus on commercial users, ease of use, server integration • R: development begun by Ross Ihaka and Robert Gentleman at University of Auckland in mid 90’s • Insightful acquired by TIBCO in 2008 – Spotfire (for Data Discovery and Visualization) acquired in 2007 • Focus shifted to applying Predictive Analytics in Spotfire – Step 1: Embrace R -3
  4. 4. Predictive Analytics with Spotfire Easily provide targeted, relevant predictive analytics to business users to improve decision making • Ensure compliance & proper usage • Share best practices and consistent workflows • Get the answer & do “What If?” analyses when needed • Leverage investments in R, S+, SAS, MATLAB, … Powerful Predictive Analytics tools for Spotfire analysts • Integrated into Spotfire workflows • Easily create, evaluate, and share Predictive Models • Add Forecasts with a single click Benefits of Predictive Analytics to a spectrum of users • Increase confidence & effectiveness in decision-making – – – Reduce uncertainty Discover meaningful patterns, important data Maximize ROI • Anticipate and react to emerging trends • Reduce/manage risk – • Scenario planning, forecasts, fraud detection Forecast specific behavior, preemptively act on it – Increase upsell, decrease churn
  5. 5. Embracing R • Spotfire Statistics Server – Integration of R & S+ into Spotfire applications • – Later added SAS® & MATLAB® Leverage the interactive visualizations of Spotfire • Contribute to the R community • Well received—but our Enterprise customers need more – – -5 R provides tremendous benefits to statisticians But large enterprises are often challenged to leverage that value
  6. 6. Enterprise Challenges for R • Core R engine struggles with Big Data – – • R was not built for enterprise usage and integration – – • Built as an academic tool for research and teaching Software vendors attempting to use R in ways it was never intended GPL great for statisticians, but limits enterprise innovation and investment – – • Customers don’t use R, or reimplement R code in specialized libraries or other languages Lose agility & consistency, delay time to production, lose opportunities Viral open source licensing risks commercial IP Large vendors avoid tight integration due to open source concerns Free to acquire, but costly to maintain – – Version incompatibilities, variable quality in packages Lack of enterprise-level technical support 6 © Copyright 2000-2013 TIBCO Software Inc.
  7. 7. TIBCO Enterprise Runtime for R (TERR) • Unique, enterprise-quality implementation of the R language – Fundamentally different • • • – TIBCO IP: Not open source/GPL • • • – Independent implementation Licensable for embedding and redistribution by partners Enables implementation of transparent big data handling Broad compatibility with R functions and 1400+ CRAN packages • • New architecture, developed from the ground up Based on our long history and expertise with S+ Faster, more robust and more memory-efficient than R Ongoing effort to broaden our coverage of R Extends the Reach of R to the Enterprise – – – Develop in R, deploy on TERR Rapidly iterate prototyping to production without recoding/retesting—more rapidly respond to changing business conditions Easily integrate R-language analytics consistently across organization—into grids, BI applications, event-driven analytics, etc. © Copyright 2000-2013 TIBCO Software Inc.
  8. 8. Leveraging TERR TERR in Spotfire TERR in Statistics Services Embeddable TERR Engine Ad hoc tools and interactive applications powered by advanced analytics • Spotfire Analytics platform: interactive visualization & data discovery, easily build and share applications, broad data access, etc. Distributed analytics • Managed pools of engines • Load balancing, queuing, failover, parallelization, etc. • High level APIs for loose integration, data i/o (C#, Java) • Central management of analytics, R packages Custom (tight) integration, batch, existing grids, etc. • Faster than R, more robust, better memory management, fully supported • Low level APIs for tight integration • Integrated into TIBCO products: CEP, Cloud Compute, … © Copyright 2000-2013 TIBCO Software Inc.
  9. 9. Providing Value for individuals who use R • Not seeking to displace R from statistician’s desktops – • Contribute to the R community – – • As we port from S+ or develop for TERR • Supports “Develop in Open Source R, Deploy on TERR” • E.g., splusTimeSeries, splusTimeDate, sjdbc TERR Developer Edition – – – -9 Sponsor useR conferences, contribute to R Foundation Contribute bug reports and propose fixes to R core Contribute packages to CRAN – • Enterprise platform for the deployment and integration of your work—without having to rewrite it! Full version of TERR engine for testing code prior to deployment • Compatible with RStudio & ESS Emacs Free for non-production use Supported through Community site
  10. 10. Example 1: TERR vs. R Raw Performance One specific example • Non-optimal, non-vectorized, real-world R script • For loop with row by row processing for (i in seq(1,length=nrow(df))) { …process each customer record… } Results • TERR is ~35x faster for 50K rows, 150x faster for 500K rows • No code modification required We are looking for more real-world performance tests! • On average 2-10x faster than R in microtests
  11. 11. Example 2: Spotfire Forecast Tool • Forecast Tool – Easily add Forecasts to Visualizations by right click menu – Advanced users can tune settings – Uses embedded TERR engine • Benefits – Extend the power of Predictive Analytics for ad hoc analysis to all Spotfire users – Easy entry point to Spotfire Predictive Analytics
  12. 12. TERR integration with TIBCO StreamBase • Event-Driven analysis in TIBCO Spotfire Event Analytics – • Apply predictive models in real-time decision making – – – – • Process monitoring, analysis, and optimization Best marketing offer Customer churn Predictive Maintenance Yield optimization Rapidly develop and iterate models in production – Respond to changing opportunities and threats 12
  13. 13. TIBCO Cloud Compute Grid • High performance computing on the cloud – Available on TIBCO Cloud Marketplace – TERR, Java and .NET computations • Robust DataSynapse GridServer architecture – Used by Wall Street to manage 10K’s nodes – Java, .NET, and REST APIs (JSON) • Perfect for pure computational work – Vastly easier to use for applications like Monte Carlo simulations than Map-Reduce – Run complex statistical models multiple orders of magnitude faster than open source R on a single computer – Unparalleled scalability without upfront capital investment • Easy to get started – Uses your Amazon EC2 account
  14. 14. Demos • TERR in Spotfire – Fraud Detection Application – Data Functions: using the R language in Spotfire – Forecast Tool
  15. 15. Learn more and Try it yourself • TERR Community at TIBCOmmunity.com – – – – • TERR Developer Edition – – • Full version of TERR engine for testing code prior to deployment Supported through TIBCOmmunity, download via tap.tibco.com TIBCO Cloud Compute Grid – • Resources, FAQs, Forums Details of R coverage Product documentation & download More info at spotfire.tibco.com/terr https://marketplace.cloud.tibco.com We want your feedback and input! – – – Real world performance tests Package & R coverage prioritization Via TERR Community, or contact me lbajuk@tibco.com or @loubajuk

×