SKIM presentation PBIRG 2012: dealing with uncertainty in forecasting

expect great answers Uncertainty? Just deal with it!Jemma Lampkin | Eelke Roos | Gerard Loosschilder For PBIRG | Chicago | May 2012

At the end of this workshop, we hope you are …… even more accepting of uncertainty in yourforecasts, actually turning it into an integratedpart of your scenario thinking.

The purpose of a forecast isto support business planningDetermine …How much you are going tosell.If you will have a positivereturn on your investment.

Your forecast … not a point estimate Performance  Your annual peak sales is $1 Billion Time 4

Your forecast … a range estimate Your annual peak Performance  sales are  100% sure to be $800 million  80% sure to be $1 billion Time   10% sure to be At a likelihood of x% At moment tx $1.5 billion5

The output is a range estimateof likely outcomes Probable revenue Cumulative revenue test treatment range: 90% chance of revenue falling 1200 within this rangeCumulative revenue treatment (million USD) based on Monte Carlo simulation 1000 Maximum cumulative revenue 800 Average cumulative revenue 600 90% likelihood range 400 200 Minimum cumulative revenue 0 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021

Sources of uncertainty can be categorized in two clusters: The accuracy of metrics The likelihood of events • Metrics collected in our • Market conditions that studies may change • Metrics available in the • Competitive actions public domain, and reactions, syndicated data and preempting and trailing with the client That is why we prefer to talk about scenario thinking instead of forecasting, to properly focus the attention on the question “what if?”.7

To deal with uncertainty and risk, we suggest using … Monte Carlo Simulation An alternative way to support scenario thinking8

Monte Carlo Simulation is an extension of your modeling practice Stochastic Δ Input Δ Output Not deterministic Inputs and outputs follow a Uniform if distribution uncertainty 60 is high 50 40 30 20 Normal if 10 0 uncertainty 1 2 3 4 5 6 7 8 9 10 is low9

A normal distribution if uncertainty is low Compliance The input variable of “compliance” 600 Number of simulations at this value (#) 500 assumes a 400 normal distribution with a 300 mean of 50% and 200 a standard 100 deviation of 8%. 0 0% 20% 40% 60% 80% 100% Compliance value (%)Input10

A uniform distribution if uncertainty is high Uptake after the 1st year The input variable of “uptake” 600 Number of simulations at this value (#) 500 assumes a 400 uniform distribution with 300 an equal 200 likelihood of all 100 values between 0 40% and 60% to 1% 11% 21% 31% 41% 51% 61% 71% 81% 91% Uptake value (% of peak share) happen.Input11

The likelihood of events are inserted as discrete variables Launch scenario Efficacy scenario 100% 100% 80% 80% 60% 60% 50% 60% 40% 40% 25% 25% 20% 20% 20% 20% 0% 0% Worst Base Best These events have discrete probabilities of happeningInput12

The likelihood of events are inserted as discrete variables Launch scenario Efficacy scenario 100% 100% 80% 80% 60% 50% 60% 50% 40% 40% 40% 30% 20% 20% 10% 20% 0% 0% Worst Base Best These events have discrete probabilities of happeningInput13

Probability distribution of sales forecast if uncertainties in continuous inputs are high Probability distribution of sales 100% The distribution of Probability of making the sales (%) 90% forecasted sales 80% values shows a 70% 60% gradual decline 50% as a result of 40% higher 30% 20% uncertainties in 10% continuous input 0% 0.5 0.8 1.1 1.4 1.7 2.0 2.3 2.6 2.9 3.2 3.5 3.8 4.1 4.4 4.7 5.0 5.3 5.6 5.9 variables. Sales in billion USDOutput14

Probability distribution of sales forecast if uncertainties in continuous inputs are low Probability distribution of sales 100% The distribution of Probability of making the sales (%) 90% forecasted sales 80% values shows a 70% 60% steep decline as 50% a result of lower 40% uncertainties in 30% 20% continuous input 10% variables. 0% 0.5 0.8 1.1 1.4 1.7 2.0 2.3 2.6 2.9 3.2 3.5 3.8 4.1 4.4 4.7 5.0 5.3 5.6 5.9 Sales in billion USDOutput15

Probability distribution of sales forecast if critical input variables have higher values Probability distribution of sales 100% The distribution of Probability of making the sales (%) 90% forecasted sales 80% values shifts to 70% 60% the right as a 50% result of higher 40% values for the 30% 20% input variables. 10% 0% 0.5 0.8 1.1 1.4 1.7 2.0 2.3 2.6 2.9 3.2 3.5 3.8 4.1 4.4 4.7 5.0 5.3 5.6 5.9 Sales in billion USDOutput16

Probability distribution of sales forecast if strongly impacted by discrete input variables Probability distribution of sales 100% The distribution of Probability of making the sales (%) 90% forecasted sales 80% values assumes a 70% 60% step-wise shape 50% as a result of a 40% higher impact of 30% 20% discrete input 10% variables. 0% 0.5 0.8 1.1 1.4 1.7 2.0 2.3 2.6 2.9 3.2 3.5 3.8 4.1 4.4 4.7 5.0 5.3 5.6 5.9 Sales in billion USDOutput17

Working with uncertainties works best if we also manage our expectations That is why we work with action standards. An action standard is a threshold value that a key performance indicator needs to exceed at an acceptable risk, before we to decide to pursue the initiative. I.e., we want to be 80% sure to make $1 billion or more.18

We met the action standard Probability distribution of Action standard sales 100% We want to be 80% Probability of making the sales (%) 90% sure to make $1 80% 70% billion or more. 60% 50% 40% Result 30% The probability of 20% 10% making $1 billion worth 0% of sales is 84%, so we 2.0 5.9 0.5 0.8 1.1 1.4 1.7 2.3 2.6 2.9 3.2 3.5 3.8 4.1 4.4 4.7 5.0 5.3 5.6 Sales in billion USD have exceeded the action standard.19

Introduction into the business caseHow Ducendi Inc. wants to build a business casefor its in-licensing agreement with Novuspharmaceuticals

Novus is developing an oral type 2 diabetes drug with a novel mode of action Novus pharmaceuticals is a biotechnology company on the rise. In order to raise new funding, Novus has offered the new treatment in an in-license agreement to Ducendi, a big pharmaceutical corporation. Ducendi wants to know how likely it is for Periculum to get a positive ROI.21

Novus claims a high likelihood of success Ducendi is not so sure Upside Downside • Survey sponsored by • A likelihood that efficacy is Novus: 80% of physicians only moderate are positive; 60% are likely • Competitive treatments in to prescribe it clinical development are • Advantages: safety and expected to have tolerability profile, interaction with Periculum risk/benefit profile and • Competitive treatments may Mode of Action be launched sooner than Periculum22

How well do you deal with the uncertainty? Your 5-year revenue will surely be $1.5 billion Your 5-year revenue will have a 80% likelihood of being $1.5 billion Your 5-year revenue will have a 80% likelihood of being $1.5 billion It also has a 99% likelihood of being $300 million and 30% of being $2 billion23

Setting the action standard for this case • What would be the accepted amount of risk you are willing to take? • How would you set the action standard? • Would setting an action standard like this fit with your business practice and resonate with your team?24

Exercise – set the action standard for Ducendi’s $1.5 billion investment in Novus’ Periculum • Senior management has asked you to assess the likelihood of a positive ROI 5 years post-launch • Ducendi has calculated a positive ROI to equal $1.5 billion in 5 years • This investment includes the development, production, launch and maintenance of Periculum How certain do we need to be of 5 year revenue reaching this revenue point? $ 1.5 billion At % $ 2.0 billion At % We will use these numbers in the business case.25

Introduction into the Monte Carlo Simulatorfor scenario thinkingThe return on investment of Periculumlaunched in two major markets

Ducendi wants to forecast the potential in two crucial markets, the United States and Elbonia United States of America Elbonia Strategically important Strategically important established market emerging market • Largest T2D market in the world Big opportunity but … in terms of revenue • Market characterized by high out- • Health insurance provided by the of-pocket expenses both public and private entities • High use of branded generics • Complex payer dynamics • Aggressive low cost local competitors • T2D data available from many • Not many data available. High sources at high precision, quality uncertainty and low quality. and certainty levels Based on qualitative impressions  High risk, low uncertainty  High risk, high uncertainty Accounts for ~70% of revenue Accounts for ~30% of revenue27

With Periculum being launched in 2016, Ducendi wishes to break even in 5 years 30 Let us assume the Million T2D patients 25 size of the patient population is a 20 given at a lower 15 and upper bound. 10 5 0 2016 2017 2018 2019 2020 Year US, Minimum US, Maximum Elbonia, Minimum Elbonia, Maximum28

Ducendi uses conjoint methodology to measure demand for Periculum under various scenarios Ducendi’s conjoint study replicates the following launch scenarios: Efficacy (phase III) of Periculum • Higher than phase II data (best case) • Similar to phase II data (base case) • Lower than phase II data (worst case) Competitive launch • Before Periculum • At the same time as Periculum • After Periculum29

Competition is expected to launch a similar drug. However, who goes first? First-mover advantage: the first mover preempts the follower, and gets a lasting advantage throughout this 5 year period. The first mover advantage is modeled as a likelihood in the scenarios: what’s the likelihood of: P C E.g., Periculum first, 30% competitor second 2015 2016 2017 2018 2019 2020 PC E.g., Periculum and competitor 40% at the same time 2015 2016 2017 2018 2019 2020 C P E.g., Competitor first, 30% Periculum second 2015 2016 2017 2018 2019 202030

Now we need your input!What are the ranges we put in,and what level of uncertainty do we assume?

First, we look at the accuracy of market data: compliance/persistence and uptakeUptake Compliance x PersistenceWhat do you expect the uptake of the new drug to be What do you expect the patient complianceby the physician population? and persistence with the new drug to be?Uptake is influenced by satisfaction with current Compliance is the patient’s adherence to theproducts, awareness/”buzz,” access/price, opportunity, prescribed dose per daycompetition and the quality of the product Shape Persistence is the proportion of patients United States Elbonia (uncertainty) persisting with the prescribed therapy Min: 35 % Min: %2016 Max: 40 % Max: % Uniform (high) United Shape States Elbonia (uncertainty) Min: 65 % Min: %2017 Max: 75 % Max: % 75 Lower % % Min: 95 % Min: % Bound Uniform (high)2018 Max: 100 % Max: % Normal (low) Normal (low) 80 Min: 100 % Min: % Upper % %2019 Bound Max: 100 % Max: % Min: 100 % Min: %2020 Max: 100 % Max: %

Second, we look at the likelihood of events: efficacy and a competitive launch Efficacy Competitive launch Coming out of phase III, what is What is the likelihood of the competitor the likelihood of Periculum to be drug to be launched before or after less, equally, or more efficacious Periculum, or at the same time? than measured in phase II? Higher United States Elbonia ___% (best case) Before 20_ _ _% 60 ___% Similar ___% Same time 50_ _ _% 30 ___% (base case) After 30_ _ _% 10 ___% Lower ___% (worst case)33

Now let us plug in the numbers and … See what happens in the business case34

So, did we make it? Target Actual Revenue % of risk Revenue % of riskTotal $ 1.5 billion At __ % $ 1.5 billion At %Total $ 2.0 billion At % $ 2.0 billion At %Do you want to go back and change a few parametersto see what happens?Set action standard Set market data Set launch data

That is all nice,but my business cannot deal with uncertainty.My business needs to make a decision!So, how can we help the businessmake a decision while dealing with uncertainty?

Eventually, the business needs to make a few decisions to overcome the uncertainty Yes Did we meet or exceed the action standard? No37

First, the business needs to decide if it finds enough reason to continue Continue with Yes the initiative Did we Not meeting the action meet or standard usually exceed the action results in more standard? questions and uncertainty. The business needs to No Now what? decide what to do next.38

If not, the business needs to decide if it is due to the quality and accuracy of the data Continue with Yes the initiative Did we meet or exceed the action Yes standard? Did we have the No best data we could have had? No39

If not, the business needs to decide if it is due to the quality and accuracy of the data Continue with Yes the initiative Deciding that the data were not Did we accurate is the meet or exceed the easiest way out. action Yes Now what? But what if the standard? data were the best Did we we could have? have the No best data we could have had? Invest in more No accurate data40

Last, the business needs to decide what is in its power to meet the action standard Continue with Yes Yes the initiative Can the Did we business Some parameters meet or invest to can be in control exceed the have a of the business, action Yes higher like investments in standard? probability compliance or time Did we of meeting have the to market. the action No best data standard? we could have had? No Invest in more No accurate data41

Last, the business needs to decide what is in its power to meet the action standard Continue with Revise the Yes Yes the initiative business case Can the Did we business meet or invest to exceed the have a action Yes higher standard? probability Did we of meeting have the the action No best data standard? we could Stop the have had? No initiative Invest in more No accurate data42

We hope that by now, you’re even more accepting of uncertainty in your forecasts Turning it into an integrated part of scenario thinking • Working with a Monte Carlo based simulator, thinking in terms of ranges instead of point estimates • Setting action standards in consultation with the business, representative of their appetite to risk43

http://www.skimgroup.com	152
http://www.linkedin.com	2
http://www.docseek.net	1

SKIM presentation PBIRG 2012: dealing with uncertainty in forecasting

by SKIM

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 155

Statistics

SKIM presentation PBIRG 2012: dealing with uncertainty in forecasting Presentation Transcript