The document discusses the importance of cost functions in machine learning models for production forecasting. It begins by defining what a cost function is and how it quantifies the goodness of fit of a model to data. It then discusses alternatives to the typical least squares cost function, including different norm formulations. The document applies these concepts to nonlinear regression problems and forecasting production data from an oil well. It emphasizes that the choice of cost function can significantly impact the regression results.

1. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
IT’S (NOT) HIP TO BE SQUARE — THE IMPORTANCE OF
COST FUNCTIONS IN PRODUCTION FORECASTING
APPLIED DATA ANALYTICS: UPSTREAM
MARCH 18–21, 2019
HOUSTON, TEXAS, USA
DAVID S. FULFORD
DATA ENGINEERING & ANALYTICS — SUBSURFACE ANALYTICS
APACHE CORPORATION

2. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
INTRODUCTION

3. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
INTRODUCTION
Valid Model Data Transform
Robust Regression Is this a Valid Model?

4. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
INTRODUCTION

5. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 What’s a cost function?
 Alternative Norms
 Regression of Non-linear problems
 Applications to Production Forecasting
 Conclusions
OUTLINE

6. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 To fit a model to data, we require an quantification of
“goodness of fit” – a cost function
 Applies to any machine learning (ML) algorithm
 In general, we can write the process of model fitting
as regression
 we desire to map predictor variables to response variables
WHAT’S A COST FUNCTION?
6

7. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 The simplest case of regression would be a linear model
 Holding that the assumptions of a linear model hold true to other models, viz.
 E 𝑒 = 0
 𝑒 are homoscedastic and uncorrelated
 If we have a linear model:
𝐘 = 𝐗𝛽 + 𝒆
 We predict 𝐘 with:
෡𝐘 = map 𝑓 መ𝛽, 𝑥 , 𝐗 + ො𝒆
 Then the cost function is:
J መ𝛽 = 𝐘 − መ𝛽𝐗
𝟐
≡ 𝐘 − map 𝑓 መ𝛽, 𝑥 , 𝐗
𝟐
 And we regress the model by minimizing J መ𝛽
መ𝛽 ← argmin J መ𝛽
WHAT’S A COST FUNCTION?
7
ො𝐳 is an estimator of 𝐳

8. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 Minimization of J( መ𝛽) is equivalent to setting the gradient of J( መ𝛽) to zero:
argmin J መ𝛽 ≡
𝜕J
𝜕 መ𝛽
→ 0
𝜕J
𝜕 መ𝛽
= −2𝐗 𝑌 − map 𝑓 መ𝛽, 𝑥 , 𝐗
 If 𝐗 is:
𝐗 = [1, 1, 1, … , 1]
 Then the gradient becomes:
𝜕J
𝜕 መ𝛽
= −2 𝐘 − መ𝛽 ≔ 0
 And its obvious that a value of መ𝛽 = ത𝑦 satisfies the equation
WHAT’S A COST FUNCTION?
8

9. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 Summarizing:
 The arithmetic mean as a best estimator of a parameter value,
and
 Squared errors as the cost function of choice to regress a model
 … are consequences of the estimator’s linear properties
 መ𝛽 is a fixed linear combination of 𝐘
 e.g. መ𝛽 = 𝐚T
𝐘 for some 𝐚 such as 𝐚 = 𝐗 𝐓
𝐗
−𝟏
𝐗 𝐓
 E መ𝛽 = 𝛽
 Most problems in which we’re interested are not linear!
 e.g. production data
 more on this later…
WHAT’S A COST FUNCTION?
9

10. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 Additionally, means are not robust
ҧ𝑥 =
1
𝑛
෍ 𝐗 =
1
𝑛
෍
𝑖=1
𝑛
𝑥𝑖
 With some manipulation we can show that:
ҧ𝑥 =
𝑛 − 1
𝑛
ҧ𝑥 𝑛−1 +
1
𝑛
𝑥 𝑛
 Indicating that any single value, if large enough, can
dominate ҧ𝑥
WHAT’S A COST FUNCTION?
10

11. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
WHAT’S A COST FUNCTION?
11

12. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 We can write the least squares cost function as the
Euclidean distance between two points:
𝛂 = 𝐘 − map 𝑓 መ𝛽, 𝑥 , 𝐗
𝛂 2 = σ 𝛂 𝟐
1
2
 Generally, we can have any level of distance, which
we call a norm:
𝛂 𝑛 = σ 𝛂 𝑛
1
𝑛
ALTERNATIVE NORMS
12

13. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 L2, L1, and L0 norms:
𝛂 2 = σ 𝛂 𝟐
1
2
𝛂 1 = σ 𝛂
𝛂 0 = ቊ
0 if 𝑥 = 0
1 else
 We can even have a L∞ norm:
𝛂 ∞ = max 𝛂
ALTERNATIVE NORMS
13

14. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 However, not all provide a closed-form analytic
solution for regression
 If we draw the L2 norm between two points, we find
a unique solution
ALTERNATIVE NORMS
14

15. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 Adding some L1 norms, we find multiple non-unique
solutions
ALTERNATIVE NORMS
15
L2
by-flight distance
L1
taxicab distance

16. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
ALTERNATIVE NORMS
16

17. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
ALTERNATIVE NORMS

18. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
ALTERNATIVE NORMS
𝛻 J 𝐻 = ቊ
𝛿 if 𝛼 ≥ 𝛿
𝛼 sign 𝛼 else

19. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
ALTERNATIVE NORMS
J 𝐻 = ൝
𝛿𝛼 − Τ1 2 𝛼2
if 𝛼 ≥ 𝛿
Τ1 2 𝛼2
else

20. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
ALTERNATIVE NORMS

21. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
ALTERNATIVE NORMS

22. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 Use stochastic gradient descent (SGD) to minimize
cost functions:
1. while J መ𝛽 𝑖−1
− J መ𝛽 𝑖
> 𝜀
2. J መ𝛽 = 𝐘 − map 𝑓 መ𝛽, 𝑥 , 𝐗
2
3. መ𝛽 ← መ𝛽 − 𝜂𝛻J መ𝛽
 where:
𝜀 = precision threshold
𝜂 = learn rate
ALTERNATIVE NORMS
22

23. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
ALTERNATIVE NORMS

24. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 We can also regularize based upon norms:
 LASSO Regression → L2 norm of data, L1 norm of መ𝛽
J መ𝛽 = 𝐘 − map 𝑓 መ𝛽, 𝑥 , 𝐗
2
+ 𝜆 መ𝛽
 Ridge Regression → L2 norm of data, L2 norm of መ𝛽
J መ𝛽 = 𝐘 − map 𝑓 መ𝛽, 𝑥 , 𝐗
2
+ 𝜆 መ𝛽2
ALTERNATIVE NORMS
24

25. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
REGRESSION OF NONLINEAR PROBLEMS
25

26. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
REGRESSION OF NONLINEAR PROBLEMS
26

27. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
REGRESSION OF NONLINEAR PROBLEMS
27

28. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 Example well from Eagle Ford
 ≈3.5 years of production history
APPLICATIONS TO PRODUCTION FORECASTING
28

29. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 If we use a hyperbolic model, we believe that:
𝑞 = 𝑞𝑖 1 + 𝐷𝑖 𝑏𝑡 − ൗ1
𝑏
𝑞 ≈
𝑞𝑖 𝐷𝑖 𝑏 − ൗ1
𝑏
𝑡 ൗ1
𝑏
𝑞 ≈
𝛼
𝑡 ൗ1
𝑏
 Meaning, a power-law function is the base functional
relationship and must log-transform the data
APPLICATIONS TO PRODUCTION FORECASTING
29
URTeC 2903036 (Fulford) – A Model-Based Diagnostic Workflow, 2018

30. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 Regression on data... neither is good
APPLICATIONS TO PRODUCTION FORECASTING
30

31. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 Point-by-point cost; note the scale on the colorbars
APPLICATIONS TO PRODUCTION FORECASTING
31

32. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
APPLICATIONS TO PRODUCTION FORECASTING
32

33. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
APPLICATIONS TO PRODUCTION FORECASTING
33

34. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
APPLICATIONS TO PRODUCTION FORECASTING
34
 Is this unique?
 Minimize cost function for 𝑞𝑖 = 200, 2000 with interval
of 100

35. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 Plot the cost function over mesh with:
𝑥 = 𝑞𝑖
𝑦 = 𝐷𝑖
APPLICATIONS TO PRODUCTION FORECASTING
35

36. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 Plot the gradient…
 𝜀 = 1 × 10−40 (precision)
 … yet algorithm still did
not find 𝛻J = 0 for each
forecast in list
APPLICATIONS TO PRODUCTION FORECASTING
36

37. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 Extract the minimum cost from the mesh at each
row/column
 Which is the “correct” forecast?
APPLICATIONS TO PRODUCTION FORECASTING
37

38. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 We’re not limited to pre-defined cost functions… make your own!
 SPE 174784-PA proposed the following:
JF =
1
𝑛
σ 𝑒 − 𝜖min
𝜎𝜖
2
+
1
𝑛
σ 𝑒 −
1
𝑛
σ 𝑒
2
− 𝜀min
𝜎𝜀
2
 Which is more clearly written as:
JF =
E 𝑒 − 𝜖min
𝜎𝜖
2
+ 𝜆 𝜀 Var 𝑒 − 𝜀min
2
 The features of this cost function are:
 L1 cost against data
 L2 cost against best-fit model
 L2 regularization for min. variance (generalizes to noisy data / outliers)
APPLICATIONS TO PRODUCTION FORECASTING
38
SPE 174784-PA (Fulford, Bowie, Berry, Bowen, Turk) – Machine Learning as a Reliable Technology, 2016

39. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 Regularization on variance limits range of 𝐷𝑖 and 𝑏-
parameter
 L2 cost against best-fit penalizes spread through data
APPLICATIONS TO PRODUCTION FORECASTING
39

40. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 The choice of cost function(s) may impact regression
results as much as the choice of predictive model
 Understanding the base expectation of data & model
functional-forms gives insight into appropriate choice of
cost function
 Uncertainty is a fundamental characteristic of modeling
 A best-fit is not the same as a best forecast
 It does not mean only one unique set of model parameters exists!
CONCLUSIONS
40

41. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
41
APPENDIX

42. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
 𝐗 𝐗 𝐓
𝐗
−𝟏
𝐗 𝐓
is the projection matrix of 𝐘 to ෡𝐘
 𝐘 = 𝐗 𝐗 𝐓 𝐗
−𝟏
𝐗 𝐓 𝐘 = 𝐗𝛽
WHAT’S A COST FUNCTION?
42

43. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
DISTRIBUTION OF FORECASTS
 Possible fits of data + uncertainty of future
performance
Actual vs. MCMC Forecasts
Time
Rate
43

44. ADA Upstream | March 18–21, 2019 | Houston, TX, USA
REVISIONS
 How much should I expect to revise forecasts from
month-to-month with this approach?
 On average, zero!
Change in EUR from prior month
Clifford and Torres (2017)
44