This document discusses using predictive analytics and machine learning for quantitative ROI-driven decision making. It describes how predictive models are trained and scored over multiple business cycles to improve predictions and build trust in the models. Key points covered include aligning business intuition with model predictions, determining the ROI of applying different business decisions to instances, and maturing predictions through experimentation over time.

1. Predictive analytics for
“Quantitative” ROI-driven decision making
Sai Devulapalli
Director, Technology Strategy
Ericsson North America

2. Machines vs. Management
Training
Machines Management
Scoring
Score = 0.65
Gut Feeling !

3. “Crossing the Chasm” in
Predictive analytics
• Align business intuition with predictions
• Is it the data or is it the inference ?
• Quantitative ROI metrics
Trust takes time, i.e. Business Cycles
Trust takes time, i.e. Business Cycles

4. Machines and Management
Score= 0.65
Business
Cycle 1
Training
Scoring
Score= 0.75
Business
Cycle 2
Training
Prediction vs.
Intuition vs. Reality
Scoring
Score= 0.85
Business
Cycle n
Machines Management
Training
Scoring
Prediction vs.
Intuition vs. Reality

5. Micro Decisions:
Few decisions → Millions of instances
For each instance
Upgrade
Replace
Promote
Market
Customers
Equipment
Devices
Network nodes
Products
Observed
Observed
Attributes
Attributes
Instances
Instances
Next business cycle
Decisions
Decisions
Predicted
Predicted
Outcome
Outcome
Business
Business
Decision
Decision
Outcomes after
Outcomes after
Business
Business
Decision
Decision

6. Two Stage Prediction
Score
Score
Outcome
Outcome
Observed
Observed
Attributes
Attributes
Learn
Learn
Outcome
Outcome
Real Outcome on
Real Outcome on
control group
control group
Predicted
Predicted
Outcome
Outcome
Observed
Observed
Attributes
Attributes
bd1
*
**
bd2
bdt
*
**
Business Decision bd11
Business Decision bd
Learn impact of Business Decision bd11
Learn impact of Business Decision bd
Real Outcome after bd11
Real Outcome after bd
Experimental Groups (EG)
EG1
EGt
EG2

7. “Quantitative” ROI-driven decision making
ROI(ia→bdb) : ROI in applying business decision bd b to instance ia
Decreasing order
of ROI
Business Decisions
Budget
ROI(ia1→bdb1)
ROI(ia2→bdb2)
ROI(ia3→bdb3)
ROI(ia3→bdb3)
*
*
*
Instance skipped
Business Milestone. Ex.
•Out of budget
•ROI target reached

8. Determining ROI for a micro-business decision
ROI in applying business decision
ROI in applying business decision
bdb b oninstance iaia: :CC(i(ia →bdb) )
bd on instance
Δ Δ a → bdb
Expected cost savings from
Expected cost savings from
change in outcome
change in outcome
-
Cost of business decision bdb b on
Cost of business decision bd on
instance CCBD(bd) )
instance BD(bdb b
Expected cost of outcome prior
Expected cost of outcome prior
to business decision
to business decision
-
Expected cost of outcome after
Expected cost of outcome after
business decision
business decision
Cost of outcome
Cost of outcome
yykon instance iaia
k on instance
CC(i(i,yk)k)
Y Y a a,y
Over possible
outcomes yk
Σ
X
Probability of
Probability of
outcome yyk
outcome k
for instance iaia prior
for instance prior
to bdb b :P(ia,yk,k,,bdΦ)
to bd: P(ia,y,bdΦ)
Cost of outcome
Cost of outcome
yykon instance iaia
k on instance
CC(i(i,yk)k)
Y Y a a,y
Over possible
outcomes yk
Σ
X
Probability of
Probability of
outcome yyk
outcome k
for instance iaia after
for instance after
bdb b :P(ia,yk,k,,bdb)b)
bd : P(ia,y, bd

9. Maturing predictions over
business cycles
Today
Num. of Instances
Experimentation
Experimentation
Business Intuition
Business Intuition
Predictive
Predictive
Optimization
Optimization
Control Group
Experimental Group 1
Experimental Group 2
Experimental Group 3
Business Cycles and Prediction maturity
1. Pre-paid subscribers Churn analytics
1. Pre-paid subscribers Churn analytics
2. Marketing Campaign management
2. Marketing Campaign management

10. Whitepaper: http://goo.gl/Xw4XFL