Predictive Analytics Leads to Successful
Recommendations and Direct Revenue Maximization
Christos Tryfonas, Ph.D.
CTO, Cloud & Big Data Analytics
                                                               INSTANT INTELLIGENCE




Web:	
  www.cetas.net	
  
Twi)er:	
  @CetasAnaly/cs	
  
Blog:	
  www.cetas.net/blog	
  
YouTube:	
  www.youtube.com/CetasAnaly/cs	
  
                                                © 2009 VMware Inc. All rights reserved
INSTANT INTELLIGENCE




      Transform IT
          + Transform Business
              + Transform Yourself



      Leverage Cetas to Transform
    Real-Time and Predictive Analytics

2
Infrastructure, Apps and now Data…                           INSTANT INTELLIGENCE




                             Build     Run
     Private
               Public
                            Analyze   Manage




Simplify Infrastructure   Simplify App Platform   Simplify Data
     With Cloud              Through PaaS             with
        (IaaS)                                      Analytics
                                                  as-a-Service


 3
Trend: New Data Growing at 60% Y/Y                                                                          INSTANT INTELLIGENCE




Exabytes of information stored                                                              20 Zetta by 2015

                                                                                            1 Yotta by 2030

                                                                                            Yes, you are part
                                                                                            of the yotta
                                                                          audio	
           generation…
                                                                 digital	
  tv	
  
                                                             digital	
  photos	
  
                                                  camera	
  phones,	
  rfid	
  
                                            medical	
  imaging,	
  sensors	
  
                     satellite	
  images,	
  logs,	
  scanners,	
  twi)er	
  
       cad/cam,	
  appliances,	
  machine	
  data,	
  digital	
  movies	
  



                                                                             Source: The Information Explosion, 2009


4
Data Growth in the Enterprise   INSTANT INTELLIGENCE




5
Trend: Value from Data Exceeds Hardware Cost                           INSTANT INTELLIGENCE




§  Value from the intelligence of data analytics now outstrips the cost
     of hardware
     •  Hadoop enables the use of 10x lower cost hardware
     •  Hardware cost halving every 18 months
                                                            Value
                    Big Iron:
                    $40k/CPU

                                                                    Commodity
                                                                    Cluster:
                                                                    $1k/CPU
                                          Cost




 6
The Big Data Problem                                              INSTANT INTELLIGENCE




    Velocity            Volume              Variety         Value


                                                              $

 10’s of Billions   From Terabytes to   Multi-Structured   Business
of Daily Records        Petabytes                          Insights




7
Trend: Big Data Analytics – Driven by Real-World Benefit   INSTANT INTELLIGENCE




8
Big Data Use Cases   INSTANT INTELLIGENCE




9
Big Data Analytics ROI                                                                                  INSTANT INTELLIGENCE




Source: Nucleus Research
http://www.enterpriseappstoday.com/business-intelligence/the-more-pervasive-business-analytics-roi-big-data.html

   10
Big Data Technology Stack   INSTANT INTELLIGENCE




11
A Holistic View of a Big Data System                                     INSTANT INTELLIGENCE




                Real-Time
                 Streams


                       Real-Time
                       Processing
                                           Analytics
Ingestion
    &                                      Big Query,
   ETL                      Real Time
                                            Search,           Batch
                            Database
                             (HBase,         Index          Processing
                            Cassandra)    (Google, Spunk,
                                              Cetas)




                            Unstructured Data (HDFS)



12
The Unified Analytics Cloud Platform                                         INSTANT INTELLIGENCE




          Mahout      R                                       Cetas/VMware
                               Analytics Tools
              Splunk                                  SAS or IBM SPSS

         Map Reduce              Developer           Spring
                                                                 PaaS
                 Python         Frameworks       Cloud Foundry

        Cassandra                                      HBase
                    HDFS     Database/DataStores
                 Greenplum                               Voldemort

        Hadoop
                                Data Platform           Data PaaS
            VMware /Cetas



             vSphere         Cloud Infrastructure
                                                       Private
                                                                   Public




13
A Unified Analytics Cloud Significantly Simplifies Infrastructure                  INSTANT INTELLIGENCE




                                   §  Simplify
                                       •  Single Hardware Infrastructure
                                       •  Faster/Easier provisioning
SQL or NoSQL Cluster




                                      Big SQL       NoSQL        Hadoop        Analytics
                                                                                Engine
      Analytics L Cluster


                                                Unified Analytics Infrastructure

                                                     Private
                                                               Public
 Hadoop Cluster

                                    §  Optimize
                                        •  Shared Resources = higher utilization
        Decision Support Cluster
                                        •  Elastic resources = faster on-demand
                                           access
 14
Predictive Analytics Objectives                    INSTANT INTELLIGENCE




§  Optimization of one or more target functions
  •  Example: Increase Customer Engagement
§  Vertical Specific
  •  Increase Customer Satisfaction
  •  Maximize Customer/User Engagement
  •  Minimize Churn Rate
  •  Maximize Revenue




15
Sample Applications of Predictive Analytics                        INSTANT INTELLIGENCE




§  eCommerce
    •  Customer interaction
    •  Revenue Maximization        Statistical /
    •  Inventory management       Regression
§  Mobile                           Models
    •  Capacity Planning
§  Ad/Publishing
    •  Ad serving                                Machine
    •  Customer Retention Management             Learning
                                               (NN, Bayesian,
§  Gaming                                          etc.)
    •  Ad serving
    •  Virtual Good Placement
    •  In-App Purchasing
                                                            Social Graph
§  IT
                                                              Algorithms
    •  Capacity Planning
    •  Alerting and Diagnosis

16
Taking Predictive Analytics to logical conclusion                          INSTANT INTELLIGENCE




§  Currently predictive analytics is in Silos
  •  Data Scientists w/ SAS, SPSS, Custom models
  •  Localized/Sampled Datasets
  •  Building Models using Open Source Modeling Tools: Hadoop, Mahout, R
§  Actionable Analytics
  •  Recommendation Engines (Examples: Amazon, Netflix)
  •  Product Placement Engines
  •  Targeted Ad Placement Engines
§  Cloudification of Predictive Analytics
  •  Analytics-as-a-Service approach
  •  Big Data support
§  Overall Objectives
  •  Revenue Maximization
  •  Social Penetration and User Engagement
17
Data is Being Stretched                                                INSTANT INTELLIGENCE




                      Intelligent Data
                                 §  Analytics Apps and
                                     Algorithms
                                 §  Discovery, Visibility,
                                     Pattern Extraction




      Big Data                                          Fast Data
                                                   §  Streaming, real-time and
§  Petabytes vs.
                                                       Unpredictable
    Gigabytes
                                                   §  Mobile app proliferation
§  Democratize
    Analytics



                                         §  Non-structured data
                      Flexible Data      §  Developer productivity

 18
Summary                                        INSTANT INTELLIGENCE




§  Cloudification of Analytics is happening




§  Predictive Analytics follow the Big Data
  and Social Web App trends




§  Direct Impact in Monetization from
  Predictive Modeling and Analytics




19

Cetas Predictive Analytics Prezo

  • 1.
    Predictive Analytics Leadsto Successful Recommendations and Direct Revenue Maximization Christos Tryfonas, Ph.D. CTO, Cloud & Big Data Analytics INSTANT INTELLIGENCE Web:  www.cetas.net   Twi)er:  @CetasAnaly/cs   Blog:  www.cetas.net/blog   YouTube:  www.youtube.com/CetasAnaly/cs   © 2009 VMware Inc. All rights reserved
  • 2.
    INSTANT INTELLIGENCE Transform IT + Transform Business + Transform Yourself Leverage Cetas to Transform Real-Time and Predictive Analytics 2
  • 3.
    Infrastructure, Apps andnow Data… INSTANT INTELLIGENCE Build Run Private Public Analyze Manage Simplify Infrastructure Simplify App Platform Simplify Data With Cloud Through PaaS with (IaaS) Analytics as-a-Service 3
  • 4.
    Trend: New DataGrowing at 60% Y/Y INSTANT INTELLIGENCE Exabytes of information stored 20 Zetta by 2015 1 Yotta by 2030 Yes, you are part of the yotta audio   generation… digital  tv   digital  photos   camera  phones,  rfid   medical  imaging,  sensors   satellite  images,  logs,  scanners,  twi)er   cad/cam,  appliances,  machine  data,  digital  movies   Source: The Information Explosion, 2009 4
  • 5.
    Data Growth inthe Enterprise INSTANT INTELLIGENCE 5
  • 6.
    Trend: Value fromData Exceeds Hardware Cost INSTANT INTELLIGENCE §  Value from the intelligence of data analytics now outstrips the cost of hardware •  Hadoop enables the use of 10x lower cost hardware •  Hardware cost halving every 18 months Value Big Iron: $40k/CPU Commodity Cluster: $1k/CPU Cost 6
  • 7.
    The Big DataProblem INSTANT INTELLIGENCE Velocity Volume Variety Value $ 10’s of Billions From Terabytes to Multi-Structured Business of Daily Records Petabytes Insights 7
  • 8.
    Trend: Big DataAnalytics – Driven by Real-World Benefit INSTANT INTELLIGENCE 8
  • 9.
    Big Data UseCases INSTANT INTELLIGENCE 9
  • 10.
    Big Data AnalyticsROI INSTANT INTELLIGENCE Source: Nucleus Research http://www.enterpriseappstoday.com/business-intelligence/the-more-pervasive-business-analytics-roi-big-data.html 10
  • 11.
    Big Data TechnologyStack INSTANT INTELLIGENCE 11
  • 12.
    A Holistic Viewof a Big Data System INSTANT INTELLIGENCE Real-Time Streams Real-Time Processing Analytics Ingestion & Big Query, ETL Real Time Search, Batch Database (HBase, Index Processing Cassandra) (Google, Spunk, Cetas) Unstructured Data (HDFS) 12
  • 13.
    The Unified AnalyticsCloud Platform INSTANT INTELLIGENCE Mahout R Cetas/VMware Analytics Tools Splunk SAS or IBM SPSS Map Reduce Developer Spring PaaS Python Frameworks Cloud Foundry Cassandra HBase HDFS Database/DataStores Greenplum Voldemort Hadoop Data Platform Data PaaS VMware /Cetas vSphere Cloud Infrastructure Private Public 13
  • 14.
    A Unified AnalyticsCloud Significantly Simplifies Infrastructure INSTANT INTELLIGENCE §  Simplify •  Single Hardware Infrastructure •  Faster/Easier provisioning SQL or NoSQL Cluster Big SQL NoSQL Hadoop Analytics Engine Analytics L Cluster Unified Analytics Infrastructure Private Public Hadoop Cluster §  Optimize •  Shared Resources = higher utilization Decision Support Cluster •  Elastic resources = faster on-demand access 14
  • 15.
    Predictive Analytics Objectives INSTANT INTELLIGENCE §  Optimization of one or more target functions •  Example: Increase Customer Engagement §  Vertical Specific •  Increase Customer Satisfaction •  Maximize Customer/User Engagement •  Minimize Churn Rate •  Maximize Revenue 15
  • 16.
    Sample Applications ofPredictive Analytics INSTANT INTELLIGENCE §  eCommerce •  Customer interaction •  Revenue Maximization Statistical / •  Inventory management Regression §  Mobile Models •  Capacity Planning §  Ad/Publishing •  Ad serving Machine •  Customer Retention Management Learning (NN, Bayesian, §  Gaming etc.) •  Ad serving •  Virtual Good Placement •  In-App Purchasing Social Graph §  IT Algorithms •  Capacity Planning •  Alerting and Diagnosis 16
  • 17.
    Taking Predictive Analyticsto logical conclusion INSTANT INTELLIGENCE §  Currently predictive analytics is in Silos •  Data Scientists w/ SAS, SPSS, Custom models •  Localized/Sampled Datasets •  Building Models using Open Source Modeling Tools: Hadoop, Mahout, R §  Actionable Analytics •  Recommendation Engines (Examples: Amazon, Netflix) •  Product Placement Engines •  Targeted Ad Placement Engines §  Cloudification of Predictive Analytics •  Analytics-as-a-Service approach •  Big Data support §  Overall Objectives •  Revenue Maximization •  Social Penetration and User Engagement 17
  • 18.
    Data is BeingStretched INSTANT INTELLIGENCE Intelligent Data §  Analytics Apps and Algorithms §  Discovery, Visibility, Pattern Extraction Big Data Fast Data §  Streaming, real-time and §  Petabytes vs. Unpredictable Gigabytes §  Mobile app proliferation §  Democratize Analytics §  Non-structured data Flexible Data §  Developer productivity 18
  • 19.
    Summary INSTANT INTELLIGENCE §  Cloudification of Analytics is happening §  Predictive Analytics follow the Big Data and Social Web App trends §  Direct Impact in Monetization from Predictive Modeling and Analytics 19