Time series in financial domain: A deep learning approach
Agenda:
-----------
3:45pm - 4:00pm: Arrival & Networking
4:00pm - 4:15pm: News & Intro
4:15pm - 5:15pm: Time series in financial domain:
A deep learning approach
5:15pm - 5:30pm: Virtual Snack & Networking
About the main speaker:
---------------------------------
Christophe Pere, Chief Data Scientist (La Capitale Insurance)
"I have a Ph.D. in Astrophysics specializing in atmospheric characterization during which I was able to learn the basics of data mining. I then continued my journey through data science and AI in order to understand the principles of learning. I worked on the evaluation of automotive markets and the prediction of vehicle prices with time series and machine learning models. I then joined an institute specialized in autonomous vehicles, the objective was to be able to recreate the real world from sensor data on vehicles traveling in Europe. I finally moved to Canada in the insurance industry to develop an R&D team and work on research projects focused on NLP, time series, and graphs."
4. Agenda
3:45 - 4:00 Arrival & Networking
4:00 - 4:15 News & Intro
4:15 - 5:15 Time series in financial domain: A
deep learning approach
5:15 - 5:30: Virtual Snack & Networking
4
DSDT Meetup - Sept 29, 2021
5. Monthly cadence, on Wednesdays.
Incredible sessions already planned. Contact us with your expectations & ideas.
ML
Validation
Reinforcement
Learning
Explainable
AI
Rodent brain
& NN
Lorem ipsum
Commodo
April 28
May 26 Aug 25
DSDT meetups in 2021
July 21
Your meetup,
your ideas.
http://bit.ly/DSDTsurvey2021
RNN & Time
Series
Sept 29
6. Sept 28 - Dec 1: The largest Open Innovation
challenge in Canada (cooperathon.ca)
Cooperathon 2021
6
Community & Training Partners
8. DSDT Mtl meetup
Pdipiscing elit
322,722 views
DSDT Meetup
Pdipiscing elit
322,722 views
DSDT Meetup
Pdipiscing elit
322,722 views
DSDT
Pdipiscin
322,722
Virtual Meetups
Until we can do in-person events
again in Montreal…
Past (and future) presentations
available on Slideshare.
on slideshare.net/DSDT_MTL
slideshare.net/DSDT_MTL
10. Time series in
financial domain
A deep learning
approach
10
Christophe Pere
Chief Data Scientist
La Capitale Insurance
Picture
Data Science | Design | Technology
11. Content
11
● Who am I
● Time Series
● RNN Family
● Applications with real data
● Discussion
● Conclusion
● Future
12. Who am I
- French guy
- PhD in Astrophysics
- Researcher since 2016
- Automotive market
- Autonomous vehicle
- NLP
- Synthetic data (imbalanced data)
- Knowledge Representation
- Reasoning
- Passion for AI
- Passion for Quantum
12
14. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
14
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series
15. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
15
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series
Ok, but in practice?
16. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
16
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series
2
Čepulionis Paulius, Lukoševičiūtė Kristina Electrocardiogram time series forecasting and optimization using ant colony optimization algorithm.
Mathematical Models in Engineering, Vol. 2, Issue 1, 2016, p. 69-77
Electrocardiogram2
17. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
17
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series
2
https://ionides.github.io/531w16/final_project/Project13/final_project.html
Stock Market2
18. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
18
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series
2
https://www.w3.org/People/Bos/Nice/tempgraph
Temperatures in Nice city, France2
19. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
19
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series
2
https://climate.nasa.gov/vital-signs/carbon-dioxide/
Carbon Dioxide concentration2
20. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
20
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series
Is that all? Could we use other kind of data with another representation of “time”?
21. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
21
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series
Is that all? Could we use other kind of data with another representation of “time”?
What do we call Time ?
22. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
22
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series
Is that all? Could we use other kind of data with another representation of “time”?
What do we call Time ?
23. Time Series
23
Source: How DNA works, Craig Freudenrich, Ph.D., https://science.howstuffworks.com/life/cellular-microscopic/dna.htm
26. Time Series - Composition
26
Trend
Movement of a series
(high and low values)
over a long period of
time
27. Time Series
27
Stationarity
Constant mean
Constant variance
Covariance independent of time
Source: https://towardsdatascience.com/the-complete-guide-to-time-series-analysis-and-forecasting-70d476bfe775
30. RNN Family
30
Simple RNN
Source: colah’s blog https://colah.github.io/posts/2015-08-Understanding-LSTMs/
Advantage:
- Short-term memory
Hidden state
tanh has a range of -1, 1 allows to keep values coherent
31. RNN Family
31
Simple RNN
Problems:
- Vanishing gradient
- Exploding gradient
- Long memory (only remember the previous output)
Source: colah’s blog https://colah.github.io/posts/2015-08-Understanding-LSTMs/
32. RNN Family
32
Source: colah’s blog https://colah.github.io/posts/2015-08-Understanding-LSTMs/
A little bit more complex...
Cell states
Add
Remove informations
Gates
Excellent resource to know how LSTM works
Long Short-Term Memory
(LSTM)
Forget gate
Input
gate
Output
gate
0 or 1, keep
or forget
33. RNN Family
33
Gated Recurrent Unit
(GRU)
Combines the forget and input gates
into a single “update gate”
Source: colah’s blog https://colah.github.io/posts/2015-08-Understanding-LSTMs/
Reset
gate
Update
gate
34. RNN Family
34
Applications:
- Prediction problems (ex: forecasting)
- Language modelling
- Text Generation
- Machine Translation
- Speech Recognition
- Image description generation
- Video Tagging
- Text summarization
- Call center Analysis
- Face detection
- OCR
- Image Recognition
- Music Generation
- ...
35. RNN Family - Architectures
35
Classical
neural
network
Music
Generation
Sentiment
analysis
Machine
Translation
Name entity
recognition
Andrej Karpathy, 2015. http://karpathy.github.io/2015/05/21/rnn-effectiveness/
40. Application
Forecasting?
How to use the
data?
0, 1, 2, 3, 4, 5, 6, 7, 8, 9 10
1, 2, 3, 4, 5, 6, 7, 8, 9, 10 11
2, 3, 4, 5, 6, 7, 8, 9, 10, 11 12
.
.
.
Process to do for:
- Train
- Test
Window Target
Normalization of data points:
- Subtract the mean computed on the training set
and divide by the standard deviation
Or
- Scale the data between 0 and 1
30 days
41. Application
Forecasting?
A little bit of code
RNN model
One neuron
Save the best model
Will compute the MAE
Visualization of
the loss for each
epoch
Only this part will change
42. Application - Results
- First we need a baseline
- Simple neural network (50 neurons)
- Test MAE: 75.24
2021
Real close value
Predict close value
Computed on the denormalized
data
43. Application - Results
Simple
RNN
Single Stacked Bidirectional
LSTM
Single Stacked Bidirectional
GRU
Single Stacked Bidirectional
Models tested
Variations with
recurrent dropout
have been tested
50. Application - Results
Prediction with
Prophet1
(Facebook)
- additive model
- modular
regression model
1
https://facebook.github.io/prophet/
Recurrent wrong
pattern
51. Application - Results
Prediction with
Prophet1
(Facebook)
- additive model
- modular
regression model
1
https://facebook.github.io/prophet/
Recurrent wrong
pattern
Could we
improve it?
52. Application - Results
Prediction with
Prophet1
(Facebook)
- additive model
- modular
regression model
1
https://facebook.github.io/prophet/
Reducing the
history the
model could
focus on the
extreme part
53. Discussion/Conclusion
The results show that model complexity does not improve performance.
Training a deep learning model is not simple because it requires hyperparameters
optimization.
The choice of the window size has a strong impact on the performance of the
models.
The data history must be chosen with relevance.
Do not forget the more classical statistical models like ARMA, ARIMA, SARIMA...
It is important to always take a baseline.
Memory models (LSTM, GRU) remain the most efficient in single layer or
bidirectional (only considering Deep Learning).
Stock prices are hard to predict, or even impossible if the market fluctuates too
sharply.
54. Conclusion
54
Marco Peixeiro, 2019. The Complete Guide to Time Series Analysis and Forecasting.
https://towardsdatascience.com/the-complete-guide-to-time-series-analysis-and-forecasting-70d476bfe775
55. Future reading
55
→ Comparison between Deep Learning and statistical
approaches
Lara-Benitez & Carranza-Garcia & Riquelme, 2021. An Experimental Review on Deep
Learning Architectures for Time Series Forecasting. ArXiv.
https://arxiv.org/pdf/2103.12057.pdf