This talk was delivered at our meetup in London on August 1, 2019. Recording from the session can be viewed here: https://youtu.be/fxKF616nC1I

1. Background
• Competitive data scientist at H2O.ai
• PhD in ensemble methods at UCL
• Former kaggle #1 – over 150+ competitions

2. H2O.ai Product Suite
Automatic feature engineering,
machine learning and interpretability
• 100% open source – Apache V2 licensed
• Built for data scientists – interface using R, Python or H2O Flow
(interactive notebook interface)
• Enterprise support subscriptions
• Fully automated machine learning
from ingest to deployment
• User licenses on a per seat basis
(annual subscription)
H2O AI open source engine
integration with Spark
Lightning fast machine
learning on GPUs
In-memory, distributed
machine learning algorithms
with H2O Flow GUI

3. Confidential3
Automate
Features Target
Data Quality and
Transformation Modeling Table
Model
Building
Model
Deployment
Data Integration
+
Driverless AI Workflow
visualizations
Machine Learning
Interpretability
FAST!!
Objective
Optimization

4. 0
50
100
150
200
250
12/21/2017 12/31/2017 1/10/2018 1/20/2018 1/30/2018 2/9/2018 2/19/2018
Sales over time
0
10
20
30
40
50
60
70
80
12/21/2017 12/31/2017 1/10/2018 1/20/2018 1/30/2018 2/9/2018 2/19/2018
Sales over time
0
50
100
150
200
250
300
350
400
12/31/2017 1/2/2018 1/4/2018 1/6/2018 1/8/2018 1/10/2018 1/12/2018 1/14/2018
Sales over time
Linear relationshipNonlinear (seasonal) relationship
What is a Time Series Problem?

5. 0
100
200
300
400
500
600
700
800
12/21/2017 12/31/2017 1/10/2018 1/20/2018 1/30/2018 2/9/2018 2/19/2018 3/1/2018 3/11/2018
sales per per day (all groups)
0
100
200
300
400
500
600
700
800
12/21/2017 12/31/2017 1/10/2018 1/20/2018 1/30/2018 2/9/2018 2/19/2018 3/1/2018 3/11/2018
sales by group
group 1 group 2 group 3
time groups sales
01/01/2018 group1 30
01/01/2018 group2 100
01/01/2018 group3 10
02/01/2018 group1 60.2
02/01/2018 group2 200.2
02/01/2018 group3 20.2
03/01/2018 group1 90.3
03/01/2018 group2 300.3
03/01/2018 group3 30.3
04/01/2018 group1 120.4
04/01/2018 group2 400.4
04/01/2018 group3 40.4
Time Groups

6. Modeling Foundation
1 2 3 4 5 6 7 8 9 10 11 12
[Gap]
1 2 3 4 5 6 7 8 9 10 11 12
[Gap] [Gap]
testtrain
tvs train tvs valid test
time:
Gap | Forecast Horizon
invalid lag size
valid lag size
time:

7. • Single time split (most recent training data becomes validation)
Validation Schemas #1
0
10000
20000
30000
40000
50000
60000
70000
2/5/20103/5/20104/5/20105/5/20106/5/20107/5/20108/5/20109/5/201010/5/201011/5/201012/5/20101/5/20112/5/20113/5/20114/5/20115/5/20116/5/20117/5/20118/5/20119/5/201110/5/201111/5/201112/5/20111/5/20122/5/20123/5/20124/5/2012

8. Validation Schemas #2
Rolling window with adjusting training size Rolling window with constant training size
• Multi window validation

9. Date
1/1/2018
2/1/2018
3/1/2018
4/1/2018
5/1/2018
6/1/2018
7/1/2018
8/1/2018
9/1/2018
10/1/2018
Day Month Year Weekday Weeknum IsHoliday
1 1 2018 2 1 1
2 1 2018 3 1 0
3 1 2018 4 1 0
4 1 2018 5 1 0
5 1 2018 6 1 0
6 1 2018 7 1 0
7 1 2018 1 2 0
8 1 2018 2 2 0
9 1 2018 3 2 0
10 1 2018 4 2 0
Feature Engineering: Decomposing the date

10. Feature Engineering: Lags
date target
01/01/2019 10.40
02/01/2019 10.04
03/01/2019 12.22
04/01/2019 12.74
05/01/2019 14.87
06/01/2019 15.43
07/01/2019 16.13
08/01/2019 17.20
09/01/2019 18.96
10/01/2019 19.20
11/01/2019 19.92
12/01/2019 19.31
13/01/2019 20.30
14/01/2019 21.73
15/01/2019 24.64
lag1 lag2 lag3
10.40
10.04 10.40
12.22 10.04 10.40
12.74 12.22 10.04
14.87 12.74 12.22
15.43 14.87 12.74
16.13 15.43 14.87
17.20 16.13 15.43
18.96 17.20 16.13
19.20 18.96 17.20
19.92 19.20 18.96
19.31 19.92 19.20
20.30 19.31 19.92
21.73 20.30 19.31 0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
12/26/2018 1/5/2019 1/15/2019 1/25/2019 2/4/2019 2/14/2019 2/24/2019 3/6/2019
AxisTitle
target
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
12/26/2018 1/5/2019 1/15/2019 1/25/2019 2/4/2019 2/14/2019 2/24/2019 3/6/2019
AxisTitle
target
lag1
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
12/26/2018 1/5/2019 1/15/2019 1/25/2019 2/4/2019 2/14/2019 2/24/2019 3/6/2019
AxisTitle
target
lag1
lag2
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
12/26/2018 1/5/2019 1/15/2019 1/25/2019 2/4/2019 2/14/2019 2/24/2019 3/6/2019
AxisTitle
target
lag1
lag2
lag3

11. Feature Engineering: Windows
date target
01/01/2019 10.40
02/01/2019 10.04
03/01/2019 12.22
04/01/2019 12.74
05/01/2019 14.87
06/01/2019 15.43
07/01/2019 16.13
08/01/2019 17.20
09/01/2019 18.96
10/01/2019 19.20
11/01/2019 19.92
12/01/2019 19.31
13/01/2019 20.30
14/01/2019 21.73
15/01/2019 24.64
lag1 lag2 lag3
10.40
10.04 10.40
12.22 10.04 10.40
12.74 12.22 10.04
14.87 12.74 12.22
15.43 14.87 12.74
16.13 15.43 14.87
17.20 16.13 15.43
18.96 17.20 16.13
19.20 18.96 17.20
19.92 19.20 18.96
19.31 19.92 19.20
20.30 19.31 19.92
21.73 20.30 19.31
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
12/26/2018 1/5/2019 1/15/2019 1/25/2019 2/4/2019 2/14/2019 2/24/2019 3/6/2019
target
lag1
lag2
lag3
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
12/26/2018 1/5/2019 1/15/2019 1/25/2019 2/4/2019 2/14/2019 2/24/2019 3/6/2019
target
MA
12.22 +
10.04 +
10.40 /
(3) =
10.88
MA3
10.88
11.66
13.28
14.35
15.48
16.25
17.43
18.45
19.36
19.48
19.84
20.45
WMA
11.19
12.11
13.72
14.80
15.69
16.55
17.90
18.79
19.52
19.49
19.91
20.85
12.22 x 3 +
10.04 x 2 +
10.40 x 1 /
(3+2+1) =
11.19
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
12/26/2018 1/5/2019 1/15/2019 1/25/2019 2/4/2019 2/14/2019 2/24/2019 3/6/2019
target
MA
WMA
EXPSM
10.92
11.71
13.32
14.39
15.50
16.29
17.48
18.49
19.38
19.48
19.85
20.49 0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
12/26/2018 1/5/2019 1/15/2019 1/25/2019 2/4/2019 2/14/2019 2/24/2019 3/6/2019
target
MA
WMA
EXPSM
For hyper parameter a=0.95
(12.22 x (0.95**1) + 10.04 x (0.95**2) +10.40 x (0.95**3) )
/((0.95**1) + (0.95**2) + (0.95**3) ) = 10.92
STD MAX SKEW
1.17 12.22 1.55
1.43 12.74 -1.48
1.41 14.87 1.47
1.42 15.43 -1.44
0.63 16.13 0.34
0.89 17.20 0.61
1.43 18.96 0.70
1.09 19.20 -1.64
0.50 19.92 1.30
0.39 19.92 1.58
0.50 20.30 -0.68
1.22 21.73 0.52
Other descriptive statistics
Max, min, median, Std, kurtosis, skewness..

12. Feature Engineering: Interractions
date target lag1 lag2 lag3
01/01/2019 10.40
02/01/2019 10.04 10.40
03/01/2019 12.22 10.04 10.40
04/01/2019 12.74 12.22 10.04 10.40
05/01/2019 14.87 12.74 12.22 10.04
06/01/2019 15.43 14.87 12.74 12.22
07/01/2019 16.13 15.43 14.87 12.74
08/01/2019 17.20 16.13 15.43 14.87
09/01/2019 18.96 17.20 16.13 15.43
10/01/2019 19.20 18.96 17.20 16.13
11/01/2019 19.92 19.20 18.96 17.20
12/01/2019 19.31 19.92 19.20 18.96
13/01/2019 20.30 19.31 19.92 19.20
14/01/2019 21.73 20.30 19.31 19.92
15/01/2019 24.64 21.73 20.30 19.31
Diff1=lag1-lag2
diff1
2.18
0.52
2.13
0.56
0.70
1.07
1.75
0.24
0.72
-0.61
1.00
1.43
Diff2=lag1-lag3
diff2
2.70
2.65
2.69
1.26
1.77
2.83
1.99
0.96
0.11
0.38
2.42
4.33
MAdiff= (Diff1+ Diff2)/2
MAdif
2.44
1.59
2.41
0.91
1.24
1.95
1.87
0.60
0.42
-0.12
1.71
2.88
Div1=lag1/lag2
div1
1.22
1.04
1.17
1.04
1.05
1.07
1.10
1.01
1.04
0.97
1.05
1.07
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
12/26/2018 1/5/2019 1/15/2019 1/25/2019 2/4/2019 2/14/2019 2/24/2019 3/6/2019
-12.00
-10.00
-8.00
-6.00
-4.00
-2.00
0.00
2.00
4.00
6.00
8.00
10.00
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
12/26/2018 1/5/2019 1/15/2019 1/25/2019 2/4/2019 2/14/2019 2/24/2019 3/6/2019
-6.00
-4.00
-2.00
0.00
2.00
4.00
6.00
8.00
10.00
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
12/26/2018 1/5/2019 1/15/2019 1/25/2019 2/4/2019 2/14/2019 2/24/2019 3/6/2019
-8.00
-6.00
-4.00
-2.00
0.00
2.00
4.00
6.00
8.00
10.00
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
12/26/2018 1/5/2019 1/15/2019 1/25/2019 2/4/2019 2/14/2019 2/24/2019 3/6/2019

13. Feature Engineering: trends
date target lag1 lag2 lag3
01/01/2019 10.40
02/01/2019 10.04 10.40
03/01/2019 12.22 10.04 10.40
04/01/2019 12.74 12.22 10.04 10.40
05/01/2019 14.87 12.74 12.22 10.04
06/01/2019 15.43 14.87 12.74 12.22
07/01/2019 16.13 15.43 14.87 12.74
08/01/2019 17.20 16.13 15.43 14.87
09/01/2019 18.96 17.20 16.13 15.43
10/01/2019 19.20 18.96 17.20 16.13
11/01/2019 19.92 19.20 18.96 17.20
12/01/2019 19.31 19.92 19.20 18.96
13/01/2019 20.30 19.31 19.92 19.20
14/01/2019 21.73 20.30 19.31 19.92
15/01/2019 24.64 21.73 20.30 19.31
correl
0.78
0.94
0.94
0.95
1.00
0.99
0.99
0.92
0.96
0.14
0.38
0.99
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
1/1/2019 1/2/2019 1/3/2019
R² = 0.6059
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
1/1/2019 1/2/2019 1/3/2019

14. Feature Engineering: Target Transformations
date target
01/01/2019 10.40
02/01/2019 10.04
03/01/2019 12.22
04/01/2019 12.74
05/01/2019 14.87
06/01/2019 15.43
07/01/2019 16.13
08/01/2019 17.20
09/01/2019 18.96
10/01/2019 19.20
11/01/2019 19.92
12/01/2019 19.31
13/01/2019 20.30
14/01/2019 21.73
15/01/2019 24.64
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
12/26/2018 1/5/2019 1/15/2019 1/25/2019 2/4/2019 2/14/2019 2/24/2019 3/6/2019
sqrt
3.22
3.17
3.50
3.57
3.86
3.93
4.02
4.15
4.35
4.38
4.46
4.39
4.51
4.66
4.96
log
2.34
2.31
2.50
2.54
2.70
2.74
2.78
2.85
2.94
2.95
2.99
2.96
3.01
3.08
3.20
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
12/26/2018 1/5/2019 1/15/2019 1/25/2019 2/4/2019 2/14/2019 2/24/2019 3/6/2019
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
12/26/2018 1/5/2019 1/15/2019 1/25/2019 2/4/2019 2/14/2019 2/24/2019 3/6/2019
differ
-0.36
2.18
0.52
2.13
0.56
0.70
1.07
1.75
0.24
0.72
-0.61
1.00
1.43
2.91
-12.00
-10.00
-8.00
-6.00
-4.00
-2.00
0.00
2.00
4.00
6.00
8.00
10.00
12/26/2018 1/5/2019 1/15/2019 1/25/2019 2/4/2019 2/14/2019 2/24/2019 3/6/2019

15. • Ranking based on autocorrelation
• Pre-defined intervals (based on estimated frequency)
Daily data
• [7, 14, 21, …]
• [14, 28, 32, …]
• …
Weekly data
• [2, 4, 6, 8, …]
• [4, 8, 12, 16, …]
• …
…
Candidates for Lag-Sizes

16. • Dropouts
• Random replacement of actual lag-values by „n.a.“
• Align frequency of available lag information between train and validation/test
Regularization of Lag-Features

17. Using top-performing Algorithms

18. Genetic algorithm approach
x1 x2 x3 x4 y
200 cust1 0.01 prod1 32
250 cust1 0.45 prod2 21
50 cust1 0.51 prod3 20
45 cust2 0.79 prod1 18
125 cust2 0.72 prod2 27
400 cust2 0.28 prod3 35
230 cust3 0.68 prod1 37
210 cust3 0.35 prod2 30
500 cust3 0.28 prod3 28
505 cust4 0.63 prod1 29
150 cust4 0.53 prod2 40
170 cust4 0.33 prod3 35
Iteration1/10
X% accuracy
Tuned
Iteration2/10
+ Random
Transformation
lag1(x3,x4)
Z% accuracy
Feature importances
lag1(y,x2) 1
lag1(y,x4) 0.5
lag1(x1,x2) 0.2
lag1(x1,x4) 0.001
Feature importances
lag1(y,x2,x4) 1
lag1(y,x2) 0.5
lag1(y,x4) 0.2
lag1(x1,x2) 0.15
lag1(x3,x4) 0.05
Date
01/01/2019
02/01/2019
03/01/2019
01/01/2019
02/01/2019
03/01/2019
01/01/2019
02/01/2019
03/01/2019
01/01/2019
02/01/2019
03/01/2019

19. MLI for Time Series

20. Bring Your own recipe!
• Bring in your domain knowledge and
achieve even better results.
• Add additional transformers from the
open source git repo :
https://github.com/h2oai/driverlessai-
recipes
• You can contribute too!
• They follow sklearn type of api
• Add models, scorers or transformers

21. The Prophet model
y(t)= g(t) + s(t) + h(t)
g(t) Piecewise linear or logistic
regressor to calculate trend
s(t) models periodic changes (e.g.
weekly/yearly seasonality)
h(t) holiday component