This session was recorded in San Francisco on February 5th, 2019 and can be viewed here: https://youtu.be/0pvvDHfxdZ8 Driverless AI is H2O.ai's latest flagship product for automatic machine learning. It fully automates some of the most challenging and productive tasks in applied data science such as feature engineering, model tuning, model ensembling and model deployment. Driverless AI turns Kaggle-winning grandmaster recipes into production-ready code, and is specifically designed to avoid common mistakes such as under- or overfitting, data leakage or improper model validation, some of the hardest challenges in data science. Avoiding these pitfalls alone can save weeks or more for each model, and is necessary to achieve high modeling accuracy. Driverless AI is now equipped with time-series functionality. Time series helps forecast sales, predict industrial machine failure and more. With the time series capability in Driverless AI, H2O.ai directly addresses some of the most pressing concerns of organizations across industries for use cases such as transactional data in capital markets, in retail to track in-store and online sales, and in manufacturing with sensor data to improve supply chain or predictive maintenance. Bio: Marios Michailidis is a Competitive Data Scientist at H2O.ai. He holds a Bsc in accounting Finance from the University of Macedonia in Greece, an Msc in Risk Management from the University of Southampton and a PhD in machine learning at from UCL . He has worked in both marketing and credit sectors in the UK Market and has led many analytics’ projects with various themes including: acquisition, retention, recommenders, fraud detection, portfolio optimization and more. He is the creator of KazAnova, a freeware GUI for credit scoring and data mining 100% made in Java as well as is the creator of StackNet Meta-Modelling Framework. In his spare time he loves competing on data science challenges and was ranked 1st out of 500,000 members in the popular Kaggle.com data competition platform. He currently ranks 3rd. Bio: A Kaggle Grandmaster and a Data Scientist at H2O.ai, Mathias Müller holds an AI and ML focused diploma (eq. M.Sc.) in computer science from Humboldt University in Berlin. During his studies, he keenly worked on computer vision in the context of bio-inspired visual navigation of autonomous flying quadrocopters. Prior to H2O.ai, he as a machine learning engineer for FSD Fahrzeugsystemdaten GmbH in the automotive sector. His stint with Kaggle was a chance encounter as he stumbled upon the data competition platform while looking for a more ML-focused platform as compared to TopCoder. This is where he entered his first predictive modeling competition and climbed up the ladder to be a Grandmaster. He is an active contributor to XGBoost and is working on Driverless AI with H2O.ai.

1. Time Series in
Driverless AI
#H2OWORLD
Marios Michiadilis & Mathias Müller
Data Scientists | Kaggle Grandmasters
H2O.ai

2. Backgrounds
• Data Scientists
• Former #1 & #4

3. • Some inputdata
• A targetvariable
• An objective(or a successmetric)like RMSE or MAE
• Some allocatedresources(time andhardware)
e.g.salesx1 x2 x3 x4 y
0.14 0.69 0.01 0.71 300
0.22 0.44 0.45 0.69 100
0.12 0.35 0.51 0.23 40
0.22 0.42 0.79 0.60 23
0.93 0.82 0.72 0.50 1900
0.32 0.58 0.28 0.22 231
0.95 0.59 0.68 0.09 700
0.34 0.58 0.35 0.81 423
0.05 0.80 0.28 0.86 222
0.23 0.49 0.63 0.03 190
0.05 0.34 0.53 0.73 890
0.74 0.02 0.33 0.56 1000
Driverless AI Process
- Datavisualization(AutoViz)
- Featureengineering& selection
- AutomatedModeling
- Modelinterpretability (MLI)
- Scoringpipeline(predictions)

4. 0
50
100
150
200
250
12/21/2017 12/31/2017 1/10/2018 1/20/2018 1/30/2018 2/9/2018 2/19/2018
Sales over time
0
10
20
30
40
50
60
70
80
12/21/2017 12/31/2017 1/10/2018 1/20/2018 1/30/2018 2/9/2018 2/19/2018
Sales over time
0
50
100
150
200
250
300
350
400
12/31/2017 1/2/2018 1/4/2018 1/6/2018 1/8/2018 1/10/2018 1/12/2018 1/14/2018
Sales over time
Linear relationshipNonlinear (seasonal) relationship
What is a Time Series Problem?

5. 0
100
200
300
400
500
600
700
800
12/21/2017 12/31/2017 1/10/2018 1/20/2018 1/30/2018 2/9/2018 2/19/2018 3/1/2018 3/11/2018
sales per per day (all groups)
0
100
200
300
400
500
600
700
800
12/21/2017 12/31/2017 1/10/2018 1/20/2018 1/30/2018 2/9/2018 2/19/2018 3/1/2018 3/11/2018
sales by group
group 1 group 2 group 3
time groups sales
01/01/2018 group1 30
01/01/2018 group2 100
01/01/2018 group3 10
02/01/2018 group1 60.2
02/01/2018 group2 200.2
02/01/2018 group3 20.2
03/01/2018 group1 90.3
03/01/2018 group2 300.3
03/01/2018 group3 30.3
04/01/2018 group1 120.4
04/01/2018 group2 400.4
04/01/2018 group3 40.4
Time Groups

6. Modeling Foundation
1 2 3 4 5 6 7 8 9 10 11 12
[Gap]
1 2 3 4 5 6 7 8 9 10 11 12
[Gap] [Gap]
testtrain
tvs train tvs valid test
time:
Gap | Forecast Horizon
invalid lag size
valid lag size
time:

7. • Single time split (most recent training data becomes validation)
Validation Schemas
0
10000
20000
30000
40000
50000
60000
70000

8. Validation Schemas
Rolling window with adjusting training size Rolling window with constant training size
• Multi window validation

9. Validation Schemas
• Random k intervals

10. Date
1/1/2018
2/1/2018
3/1/2018
4/1/2018
5/1/2018
6/1/2018
7/1/2018
8/1/2018
9/1/2018
10/1/2018
Day Month Year Weekday Weeknum IsHoliday
1 1 2018 2 1 1
2 1 2018 3 1 0
3 1 2018 4 1 0
4 1 2018 5 1 0
5 1 2018 6 1 0
6 1 2018 7 1 0
7 1 2018 1 2 0
8 1 2018 2 2 0
9 1 2018 3 2 0
10 1 2018 4 2 0
Feature Engineering

11. Date Sales
1/1/2018 100
2/1/2018 150
3/1/2018 160
4/1/2018 200
5/1/2018 210
6/1/2018 150
7/1/2018 160
8/1/2018 120
9/1/2018 80
10/1/2018 70
Lag1 Lag2
- -
100 -
150 100
160 150
200 160
210 200
150 210
160 150
120 160
80 120
Moving Average
-
100
125
155
180
205
180
155
140
100
Feature Engineering (cont.)
• Lags on subsets of the specified group columns (e.g. {Store, Department} vs. {Department} vs. {Store})
• Exponentially Weighted Moving Averages (EWMA) of n-th order differentiated lags
• Aggregation of lags (mean, std, sums, etc.)
• Interactions of lags (e.g. Lag2 - Lag1)
• Linear regression on lags (taking slope and/or intercept as new features)

12. • Ranking based on autocorrelation
• Pre-defined intervals (based on estimated frequency)
Daily data
• [7, 14, 21, …]
• [14, 28, 32, …]
• …
Weekly data
• [2, 4, 6, 8, …]
• [4, 8, 12, 16, …]
• …
…
Candidates for Lag-Sizes

13. • Lower bound for considered lag sizes
• Dropout
• Random replacement of actual lag-values by „n.a.“
• Align frequency of available lag information between train and validation/test
• Target binning
• Decrease of possible amount of splits GBM can perform
Regularization of Lag-Features

14. MLI for Time Series

15. MLI for Time Series (cont.)

16. Training Holdout Predictions / Backtesting
• Final pipeline will be refitted on various train/valid splits to generate holdout
predictions:
Split
1 1 2 3 4 5 6 7 8 9 10 11 12 13 14
2 1 2 3 4 5 6 7 8 9 10 11 12
3 1 2 3 4 5 6 7 8 9 10
4 1 2 3 4 5 6 7 8
5 1 2 3 4 5 6
x
x
x
training data
validation/holdout data
Training & Validation/Holdout Data
optional training data
time

17. Thank you!
Marios Michiadilis
marios.michiadilis@h2o.ai
in/mariosmichailidis/
@StackNet_
Mathias Müller
mathias.mueller@h2o.ai
/in/muellermat/
@kagglenizer