Time series in financial domain: A deep learning approach Agenda: ----------- 3:45pm - 4:00pm: Arrival & Networking 4:00pm - 4:15pm: News & Intro 4:15pm - 5:15pm: Time series in financial domain: A deep learning approach 5:15pm - 5:30pm: Virtual Snack & Networking About the main speaker: --------------------------------- Christophe Pere, Chief Data Scientist (La Capitale Insurance) "I have a Ph.D. in Astrophysics specializing in atmospheric characterization during which I was able to learn the basics of data mining. I then continued my journey through data science and AI in order to understand the principles of learning. I worked on the evaluation of automotive markets and the prediction of vehicle prices with time series and machine learning models. I then joined an institute specialized in autonomous vehicles, the objective was to be able to recreate the real world from sensor data on vehicles traveling in Europe. I finally moved to Canada in the insurance industry to develop an R&D team and work on research projects focused on NLP, time series, and graphs."

1. Data Science | Design | Technology
https://www.meetup.com/DSDTmtl
Sept
29
2021

2. 2
Data Science | Design | Technology
https://www.meetup.com/DSDTmtl
Sept
29
(2021)
Please, don't forget to
mute yourself

3. JL Maréchaux
DSDT Co-Organizer
(Google)
Christophe Pere
Chief Data Scientist
La Capitale Insurance
https://www.meetup.com/DSDTmtl

4. Agenda
3:45 - 4:00 Arrival & Networking
4:00 - 4:15 News & Intro
4:15 - 5:15 Time series in financial domain: A
deep learning approach
5:15 - 5:30: Virtual Snack & Networking
4
DSDT Meetup - Sept 29, 2021

5. Monthly cadence, on Wednesdays.
Incredible sessions already planned. Contact us with your expectations & ideas.
ML
Validation
Reinforcement
Learning
Explainable
AI
Rodent brain
& NN
Lorem ipsum
Commodo
April 28
May 26 Aug 25
DSDT meetups in 2021
July 21
Your meetup,
your ideas.
http://bit.ly/DSDTsurvey2021
RNN & Time
Series
Sept 29

6. Sept 28 - Dec 1: The largest Open Innovation
challenge in Canada (cooperathon.ca)
Cooperathon 2021
6
Community & Training Partners

7. <MTL> Connect, October 12-17
7
mtlconnecte.ca

8. DSDT Mtl meetup
Pdipiscing elit
322,722 views
DSDT Meetup
Pdipiscing elit
322,722 views
DSDT Meetup
Pdipiscing elit
322,722 views
DSDT
Pdipiscin
322,722
Virtual Meetups
Until we can do in-person events
again in Montreal…
Past (and future) presentations
available on Slideshare.
on slideshare.net/DSDT_MTL
slideshare.net/DSDT_MTL

9. DSDT Meetups: In-person events
Back to normal soon at Notman House
9

10. Time series in
financial domain
A deep learning
approach
10
Christophe Pere
Chief Data Scientist
La Capitale Insurance
Picture
Data Science | Design | Technology

11. Content
11
● Who am I
● Time Series
● RNN Family
● Applications with real data
● Discussion
● Conclusion
● Future

12. Who am I
- French guy
- PhD in Astrophysics
- Researcher since 2016
- Automotive market
- Autonomous vehicle
- NLP
- Synthetic data (imbalanced data)
- Knowledge Representation
- Reasoning
- Passion for AI
- Passion for Quantum
12

13. 13
When you try to work
on your presentation...

14. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
14
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series

15. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
15
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series
Ok, but in practice?

16. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
16
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series
2
Čepulionis Paulius, Lukoševičiūtė Kristina Electrocardiogram time series forecasting and optimization using ant colony optimization algorithm.
Mathematical Models in Engineering, Vol. 2, Issue 1, 2016, p. 69-77
Electrocardiogram2

17. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
17
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series
2
https://ionides.github.io/531w16/final_project/Project13/final_project.html
Stock Market2

18. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
18
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series
2
https://www.w3.org/People/Bos/Nice/tempgraph
Temperatures in Nice city, France2

19. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
19
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series
2
https://climate.nasa.gov/vital-signs/carbon-dioxide/
Carbon Dioxide concentration2

20. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
20
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series
Is that all? Could we use other kind of data with another representation of “time”?

21. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
21
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series
Is that all? Could we use other kind of data with another representation of “time”?
What do we call Time ?

22. Time Series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a
sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights
of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
22
Wikipedia1
1
https://en.wikipedia.org/wiki/Time_series
Is that all? Could we use other kind of data with another representation of “time”?
What do we call Time ?

23. Time Series
23
Source: How DNA works, Craig Freudenrich, Ph.D., https://science.howstuffworks.com/life/cellular-microscopic/dna.htm

24. Time Series - Composition
24
Autocorrelation

25. Time Series - Composition
25
Seasonality

26. Time Series - Composition
26
Trend
Movement of a series
(high and low values)
over a long period of
time

27. Time Series
27
Stationarity
Constant mean
Constant variance
Covariance independent of time
Source: https://towardsdatascience.com/the-complete-guide-to-time-series-analysis-and-forecasting-70d476bfe775

28. RNN Family
28
Feedforward
Loss
Backpropagation

29. RNN Family
29
Sequence model
- text
- video
- sound
Persist information
- Memory
- Hidden state

30. RNN Family
30
Simple RNN
Source: colah’s blog https://colah.github.io/posts/2015-08-Understanding-LSTMs/
Advantage:
- Short-term memory
Hidden state
tanh has a range of -1, 1 allows to keep values coherent

31. RNN Family
31
Simple RNN
Problems:
- Vanishing gradient
- Exploding gradient
- Long memory (only remember the previous output)
Source: colah’s blog https://colah.github.io/posts/2015-08-Understanding-LSTMs/

32. RNN Family
32
Source: colah’s blog https://colah.github.io/posts/2015-08-Understanding-LSTMs/
A little bit more complex...
Cell states
Add
Remove informations
Gates
Excellent resource to know how LSTM works
Long Short-Term Memory
(LSTM)
Forget gate
Input
gate
Output
gate
0 or 1, keep
or forget

33. RNN Family
33
Gated Recurrent Unit
(GRU)
Combines the forget and input gates
into a single “update gate”
Source: colah’s blog https://colah.github.io/posts/2015-08-Understanding-LSTMs/
Reset
gate
Update
gate

34. RNN Family
34
Applications:
- Prediction problems (ex: forecasting)
- Language modelling
- Text Generation
- Machine Translation
- Speech Recognition
- Image description generation
- Video Tagging
- Text summarization
- Call center Analysis
- Face detection
- OCR
- Image Recognition
- Music Generation
- ...

35. RNN Family - Architectures
35
Classical
neural
network
Music
Generation
Sentiment
analysis
Machine
Translation
Name entity
recognition
Andrej Karpathy, 2015. http://karpathy.github.io/2015/05/21/rnn-effectiveness/

36. Application
TESLA stock prices
Source: yahoo finance
2015-07-09
2021-09-27
Features
+ Date
Min: 28.73$
Max: 883.09$

37. Application
TESLA stock prices
Source: yahoo finance
2015-07-09
2021-09-27
Features
+ Date
Min: 28.73$
Max: 883.09$
Train
Test

38. Application
Forecasting?
How to use the
data?
Tabular
Label

39. Application
Forecasting?
How to use the
data?
Tabular
Label

40. Application
Forecasting?
How to use the
data?
0, 1, 2, 3, 4, 5, 6, 7, 8, 9 10
1, 2, 3, 4, 5, 6, 7, 8, 9, 10 11
2, 3, 4, 5, 6, 7, 8, 9, 10, 11 12
.
.
.
Process to do for:
- Train
- Test
Window Target
Normalization of data points:
- Subtract the mean computed on the training set
and divide by the standard deviation
Or
- Scale the data between 0 and 1
30 days

41. Application
Forecasting?
A little bit of code
RNN model
One neuron
Save the best model
Will compute the MAE
Visualization of
the loss for each
epoch
Only this part will change

42. Application - Results
- First we need a baseline
- Simple neural network (50 neurons)
- Test MAE: 75.24
2021
Real close value
Predict close value
Computed on the denormalized
data

43. Application - Results
Simple
RNN
Single Stacked Bidirectional
LSTM
Single Stacked Bidirectional
GRU
Single Stacked Bidirectional
Models tested
Variations with
recurrent dropout
have been tested

44. Application - Results

45. Application - Results

46. Application - Results

47. Application - Results

48. Application - Results

49. Application - Results

50. Application - Results
Prediction with
Prophet1
(Facebook)
- additive model
- modular
regression model
1
https://facebook.github.io/prophet/
Recurrent wrong
pattern

51. Application - Results
Prediction with
Prophet1
(Facebook)
- additive model
- modular
regression model
1
https://facebook.github.io/prophet/
Recurrent wrong
pattern
Could we
improve it?

52. Application - Results
Prediction with
Prophet1
(Facebook)
- additive model
- modular
regression model
1
https://facebook.github.io/prophet/
Reducing the
history the
model could
focus on the
extreme part

53. Discussion/Conclusion
The results show that model complexity does not improve performance.
Training a deep learning model is not simple because it requires hyperparameters
optimization.
The choice of the window size has a strong impact on the performance of the
models.
The data history must be chosen with relevance.
Do not forget the more classical statistical models like ARMA, ARIMA, SARIMA...
It is important to always take a baseline.
Memory models (LSTM, GRU) remain the most efficient in single layer or
bidirectional (only considering Deep Learning).
Stock prices are hard to predict, or even impossible if the market fluctuates too
sharply.

54. Conclusion
54
Marco Peixeiro, 2019. The Complete Guide to Time Series Analysis and Forecasting.
https://towardsdatascience.com/the-complete-guide-to-time-series-analysis-and-forecasting-70d476bfe775

55. Future reading
55
→ Comparison between Deep Learning and statistical
approaches
Lara-Benitez & Carranza-Garcia & Riquelme, 2021. An Experimental Review on Deep
Learning Architectures for Time Series Forecasting. ArXiv.
https://arxiv.org/pdf/2103.12057.pdf

56. Future
56
- Self-Supervised learning
- Quantum Machine Learning

57. Github
You can access to the notebook, data and presentation here:
- https://github.com/Christophe-pere/DSDT-timeseries-rnn
57

58. Merci / Thank You
@DsdtMtl
Data Science | Design | Technology
(Check for next DSDT meetup at meetup.com/DSDTmtl)
http://bit.ly/dsdtmtl-in

59. RNN Family - Architectures
59
CS 230 - Stanford
https://stanford.edu/~shervine/teachin
g/cs-230/cheatsheet-recurrent-neural-
networks