Gde time series_modeling

Time-Series Modeling
for covid-19 Prediction
Sharmistha Chatterjee
Senior Manager Data Sciences at
Publicis Sapient
Author | Speaker | GDE for ML
@sharmichat82
http://techairesearch.com/
Kaggle's Day Meetup - Surat

Key Takeaways
Time-Series Modeling
Basic Time-Series Modeling
Introduction to Deep Learning based Models
Covid-19 – Univariate and Multi-variate Prediction

Why Time-Series
Applications
● Trends, Seasonality’s, cyclical patterns
and anomalies
● Projections to aid profit planning
● Critical-event analysis (Discrete vs
Continuous)
● Cause-and-effect relationships
Source - https://hbr.org/1971/07/how-to-choose-the-right-forecasting-technique

Why Time-Series Prediction in ML?
Supervised
(e.g. –
ARIMA/SARIMA)
Un-Supervised
traditional ML –
(e.g. time-series
clustering, time-series
segmentation)
Deep-learning uni-
variate/multi-
variate multi-step
Source - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5870978/
Deep-learning uni-
variate/multi-variate
single step
Different Types

Short-term and Long-term Predictions
Source - Simple Demographics Often Identify People Uniquely
Learn stationary distributions over state space not applicable in drifting systems for multi-step
Auto Regressive - Compute current value as a function of a finite past along with some white
noise
Exponentially weights down observations that are farther away in time
Segmentation, clustering, shapelets based feature extraction (USLM)
Deep Learning – Univariate and Multi-variate
Markovian
Exponential
Holt’s Winter
model
Unsupervised
Deep
Learning
Types of Time-Series Modeling
AR/MA/ARIMA
Exponential smoothing to seasonal components in addition to level and trend

Correct Selection of Time-Series Prediction technique
Purpose of Forecast and Dynamics & Components of Forecast

Unsupervised Feature Learning from Time Series
Discriminative Features Shapelet Learning
Metric - Jaccard Score,
Rand Index, Folkes and
Mallow index
Source - https://www.ijcai.org/Proceedings/16/Papers/331.pdf
Time Series of Unequal lengths

Simple Exponential Smoothing & Holt’s Winter Model
Basic Time-Series
Source - https://www.analyticsvidhya.com/blog/2018/02/time-series-forecasting-methods/
Simple Exponential MA
● Double ES- Additive & Multiplicative
Trend, Dampening Factor
● Triple ES - ES- Additive & Multiplicative
Seasonality (with steps), Dampening
Factor
Simple MA
Weighted MA

Holt’s Winter Model
Different Components
Source - https://otexts.com/fpp2/holt-winters.html
Holt’s Winter Model

Basic Time Series Modeling –AR, MA, ARIMA
Manual Model Selection ARIMA
Mechanism of using AR/MA/ARIMA/SARIMA by
self choosing the model orders (p, q, d)
and self tuning it.
● Limitations - Auto-correlation models do not
partitioning the state space and model non-
linear dependencies of exogenous variables
● p: number of lag observations
included in the model
● d: number of times that the raw
observations are differenced also called
the degree of difference.

Basic Understanding
Equation and Objectives
AR Model –
MA Model-
ARIMA Model -
tsa -univariate auto-
regressive time-series
analysis
Statespace -
unobserved state
vectors and irregular
components
vector_ar -
simultaneous modeling
and analyzing multiple
time series

Basic Time Series Modeling –Auto AR, MA, ARIMA
Automatic Model Selection Auto ARIMA
Auto-selection of model
(p, q, d) based on
Accuracy, AIC, BIC
SARIMA – (p, q, d) X (P, Q, D)
P: Seasonal autoregressive order.
D: Seasonal difference order.
Q: Seasonal moving average order.
m: The number of time steps for a
single seasonal period.

Auto Correlation Plots
Differencing to make time-series stationary

VARMAX Models
Multi-variate Time Series
model = sm.tsa.VARMAX(y_train, order=(5, 0), trend='c')
model_result = model.fit(maxiter=1000, disp=False)
model_result.summary()
model_result.plot_diagnostics()
plt.show()
Source - https://towardsdatascience.com/prediction-task-with-multivariate-timeseries-
and-var-model-47003f629f9

Deep Learning
Differences in demographics or other user characteristics.
Differences across online and offline communities, platforms
and contexts
Lexical, syntactic, semantic, and structural differences in the contents
Connections, interactions, or activities obtained from networks and their attributes
Society norms, prejudices, economical status
Seasonal, weekly or observed at a certain time
Time-Series Analysis
● Nonlinear patterns
● Huge training set helps to generate a better
result.
● Saves us time and pain in doing feature
engineering.
Source - https://researcher.watson.ibm.com/researcher/files/us-adhuran/MultistepICDMW.pdf
● Input Features - Total Confirmed Cases, Deaths,
Cured/Discharged/Migrated, oronaenquirycalls,
cumulativepeopleinquarantine, negative,, numcallsstatehelpline,
numicubeds, numisolationbeds, numventilators, , totaltested,
unconfirmed, populationncp2019projection, positive,
testpositivityrate, testspermillion, testsperpositivecase,
testsperthousand, totaln95masks, totalppe,
totalpeoplecurrentlyinquarantine,
totalpeoplereleasedfromquarantine
● Target – No of Active Cases

Frequent Pattern Discovery
and contexts
Multi-Variate Time Series
● Enlarging or Shrinking
● Combine a number of identical
adjacent patterns. without cluttering
is the multi-variable input at time step t
and xn, t ∈ R is the observation of n-th
exogenous time series at time t
Source - https://www.uni-konstanz.de/mmsp/pubsys/publishedFiles/HaMaJa11.pdf

Pattern Discovery Pipeline
and contexts
Multi-Variate Time Series
Source - https://www.uni-konstanz.de/mmsp/pubsys/publishedFiles/HaMaJa11.pdf

Deep Learning
and contexts
Time-Series Analysis
● Multi-variate time-series (of 23 features) with
test data of n samples (=23) (with predicted
output from previous steps i.e. 21+2) for
3 weeks
● Reshaped from (7,7,23), (8,7,23) and (9,7,23)
as (49,23), (56,23) and (63, 23)
Source - https://stackoverflow.com/questions/51344610/how-to-setup-1d-convolution-and-lstm-in-keras
Source - https://stackoverflow.com/questions/53904688/using-keras-to-build-a-

Subsequences re-shaping
and contexts
CNN LSTM (Conv1D) with Time Distributed Layer
Source - https://stackoverflow.com/questions/53904688/using-keras-to-build-a-lstmconv2d-model

Translation in Single-Step and Multi-Step
and contexts
Series Conversion to Supervised Problem
Source - https://github.com/sharmi1206/covid-19-analysis

Translation from Uni-variate to Multi-variate
and contexts
Time-Series (CONV2D + LSTM)
Actual vs Predicted Active Cases
Multi-variate Prediction

What and How
and contexts
Multistep Time-Series Use-Cases
● Predicting parameters ahead, using information from other less
critical parameters measured at every point in time.
● E.g. predict information in the red using the information in the blue
region i.e. time series T1, T2, ..., Td−1 at every instant and Td till time t.
● Measuring deposition rate of nitrogen dioxide on a wafer is
expensive, but temperature, pressure etc. easily measured without
overhead.

Complex Instrumented Domains
and contexts
Multi-Step Time Series-IBM
● With d time series T1, T2, ...,Td , predict Td ahead
as possible from some time instant t, given the
other d − 1 time series & values of Td up until t
● Identify states from predictive standpoint with
thresholds (selected based on domain/cross-
validation)
● Classification technique improve dynamic
range of predictions
● Uncorrelated errors in regression & classification
for overall error reduction
● Regression to predict target
● Pairwise least square coefficients
for every pair of input TS

and contexts
Types of Multi-Step Time Series
● Iterative –
○ Prediction result of one-step forecast is used as the input for forecasting two-
step ahead with the same model.
○ Continues upto prediction horizon.
○ Errors of every prediction are accumulated along the horizon
● Independent –
○ Independent model for each forecasting horizon
○ Dependencies are not extracted
○ High computational complexity
Source - https://pdfs.semanticscholar.org/ad5c/e8b3350c62be221fc9c4ce8934d4e3eaca03.pdf?_ga=2.72997204.659500086.1600000969-

SOMAR, NGMAR, GSOMAR
Vector of each neuron contains the coefficients of an AR model and a mixing
weight
Varied Length Mixture Model for Multi-step
Prediction
341486537.1591885728
● Relational Learning - Consider
observations up to say k steps
behind as relational neighbors
● Combine the various states
based on the closeness of their
pairwise regression coefficients

Where and How
Ensemble Time Series Prediction
Source - https://www.business-science.io/business/2018/12/04/time-series-forecasting.html

ARIMA, Neural Network and Linear Regression
Source - https://link.springer.com/chapter/10.1007/978-3-319-93713-7_55
Source - http://nebula.wsimg.com/5b49ad24a16af2a07990487493272154?AccessKeyId=DFB1BA3CED7E7997D5B1&disposition=0&alloworigin=1

ARIMA, Neural Network and Linear Regression
Source -
https://www.cse.cuhk.edu.hk/lyu/_media/conference/slzhao_sigspatial17.pdf?id=publications%3Aall_by_year&cache=cache

Low variance and Improved Performance
Short-term (daily) + long-term (monthly) models
Preserve/Retain as much dependency
information as possible among the
points within the prediction horizon
Generalized self-organizing probabilistic
mixture of autoregressive models
Ensemble Multi-Step Time-Series
341486537.1591885728
Varied length mixture
Adaptive, horizon-dependent
weighing

Conclusion and Future Work
● Random Walk Model non-stationary time series with long periods of trends up / down
● Seasonal Differencing – not applicable for Covid-19 context
● Impact of Exogenous Variables (variables impacting model without being affected by it)
● Limitations in learning non-linear patterns in ARIMA/SARIMA with/without Exogeneous variables
● Prophet from Facebook composed of a saturating growth model and a piece-wise linear model
Source: https://otexts.com/fpp2/stationarity.html

Follow me on https://techairesearch.com/
Questions?

References
● https://www.ijcai.org/Proceedings/16/Papers/331.pdf
● https://github.com/sharmi1206/covid-19-analysis
● https://machinelearningmastery.com/taxonomy-of-time-series-forecasting-problems/
● https://techairesearch.com/comparative-study-of-best-time-series-models-for-urgent-pandemic-management-2/
● https://techairesearch.com/comparative-study-of-best-time-series-models-for-urgent-pandemic-management-1/
● https://techairesearch.com/basic-understanding-of-arima-sarima-vs-auto-arima-sarima-using-covid-19-prediction/
● Multi-variable LSTM neural network for autoregressive exogenous model : https://arxiv.org/pdf/1806.06384.pdf
● Multi-variate Time Series & VAR Models - https://towardsdatascience.com/prediction-task-with-multivariate-timeseries-
and-var-model-47003f629f9
● https://machinelearningmastery.com/power-transform-time-series-forecast-data-python/
● https://www.aiproblog.com/index.php/2018/11/13/how-to-develop-lstm-models-for-time-series-forecasting/

Gde time series_modeling

Recommended

Recommended

More Related Content

Similar to Gde time series_modeling

Similar to Gde time series_modeling (20)

Recently uploaded

Recently uploaded (20)

Gde time series_modeling

Editor's Notes