Dildar Ali wrote a paper on using deep learning approaches like recurrent neural networks (RNNs) and long short-term memory (LSTM) networks to predict financial time series data. The document provides an introduction to time series forecasting in financial markets. It reviews past work applying RNNs and LSTMs to stock price prediction. It also describes issues with standard RNNs like vanishing gradients and explains how LSTMs address this by incorporating memory cells and gates. The paper proposes using RNNs and LSTMs to predict financial data and improve on standard RNN accuracy and stability.

1. Dildar Ali (M.Tech in CSE) Aliah University 1
Deep Learning Approach to Predict Financial Time
Series Data
Dildar Ali
Roll No. CSE182002 of 2018-19
Under the guidance of
Dr. Abhishek Das
Associate Professor & Head
Department of Computer Science and Engineering
Aliah University
Action Area II-A/27, Newtown
Kolkata - 700160
June 27, 2019

2. Dildar Ali (M.Tech in CSE) Aliah University 2
Deep Learning Approach to predict financial time series data
Outline
 Introduction
 Literature Review
 Recurrent Neural Network
 Long Short-Term Memory
 Conclusion & Future work
 References

3. Dildar Ali (M.Tech in CSE) Aliah University 3
Deep Learning Approach to predict financial time series data
Introduction
Time series forecasting has been widely used to determine the
future prices of stock and guide investors to take decisions
Financial markets are highly volatile and generate huge
amounts of data daily
There are various techniques available for the prediction of the
stock values
• Linear models like ARIMA
• Non linear models like ARCH, Deep learning algorithms

4. Dildar Ali (M.Tech in CSE) Aliah University 4
Deep Learning Approach to predict financial time series data
Literature Survey
In 1996, J. Roman & A. Jameel, used back propagation and RNN models
for the prediction of stock index of five different stock markets
In 1998, E. W. Saad & D. V. Prokhrov shows application of time delay,
recurrent and probabilistic neural network models for daily stock prediction
With the introduction of LSTM in 1997, by S. Hochreiter & J. Schmidhuber
the analysis of time dependent data become more efficient. These type of
networks have the capability of holding past information. Later this is used
in stock price prediction by H. Jia in 2016
In 2018, Lin, Chu & Wang uses a neural network with memory capability
Recurrent neural network

5. Dildar Ali (M.Tech in CSE) Aliah University 5
Deep Learning Approach to predict financial time series data
Recurrent Neural Network
st = tanh(W· st−1 + U.xt),
Ot = V. st

6. Dildar Ali (M.Tech in CSE) Aliah University 6
Deep Learning Approach to predict financial time series data
Problems with RNN
When dealing with a time series, it tends to forget old information.
When there is a distant relationship of unknown length, we wish to
have a “memory” to it
Vanishing gradient problem

7. Dildar Ali (M.Tech in CSE) Aliah University 7
Deep Learning Approach to predict financial time series data
Vanishing Gradient Problem
It can be seen that the limits of the
derivative approach zero at both ends.
Multiplication of such small derivatives in
the chain rule causes gradients at later time-
steps to vanish. This implies an inability of
recurrent neural networks to learn long-
term dependencies in sequential inputs.
One solution to this problem is using
rectified linear units (ReLUs) by
Krizhevsky & Sutskever in 2012 defined as
f(x) = max(0, x), instead of the sigmoid or
hyperbolic tangent activation functions.

8. Dildar Ali (M.Tech in CSE) Aliah University 8
Deep Learning Approach to predict financial time series data
LSTM
LSTMs were originally introduced by Hochreiter and Schmidhuber and
further improved by Gers, Schmidhuber, and Cummins [6]. Recurrent
neural networks with LSTMs share the same structure as well as the
learning algorithm (backpropagation through time); the difference is in
the hidden units
ft = σ (uf · xt + wf · Ot-1 + bf ).
it = σ (ui · xt + wi · Ot-1 + bi ).
Bt = tanh(ub · xt + wb · Ot-1 + bb ).
Ct = it · Bt + ft · Ct−1.
ht = σ (uh · xt + wh · Ot-1 + bh ).
ot = σ(ht · tanh(Ct)).

9. Dildar Ali (M.Tech in CSE) Aliah University 9
Deep Learning Approach to predict financial time series data
RNN vs LSTM

10. Dildar Ali (M.Tech in CSE) Aliah University 10
Deep Learning Approach to predict financial time series data
Peephole LSTM
Allows “peeping into the memory”

11. Dildar Ali (M.Tech in CSE) Aliah University 11
Deep Learning Approach to predict financial time series data
Conclusion & Future work
The strategies discussed focuses on Recurrent neural networks, LSTM
now a days used in a wide variety of domains
One of common machine learning problems where neural networks
can be used is time series classification.
To accomplish this task, it is necessary to select a proper architecture
and to optimize hyper parameters of the network
Now for future work we propose a technique to predict financial data
by applying Recurrent neural network(RNN) and Long short-term
memory (LSTM) as LSTM improves the long-term dependence of
traditional RNNs and effectively improves the accuracy and stability.

12. Dildar Ali (M.Tech in CSE) Aliah University 12
Deep Learning Approach to predict financial time series data
References
[1] J. Heaton, N. Polson, and J. Witte, “Deep learning in finance,” arXiv preprint arXiv:1602.06561, 2016.
[2] H. Jia, “Investigation into the effectiveness of long short term memory networks for stock price prediction,”
arXiv preprint arXiv:1603.07893, 2016.
[3] Y. Bengio, I. J. Goodfellow, and A. Courville, “Deep learning,” Nature, vol. 521, pp. 436–444, 2015.
[4] J. Roman and A. Jameel, “Backpropagation and recurrent neural networks in financial analysis of multiple
stock market returns,” in System Sciences, 1996., Proceedings of the Twenty-Ninth Hawaii International
Conference on,, vol. 2. IEEE, 1996, pp. 454–460.
[5] E. W. Saad, D. V. Prokhorov, and D. C. Wunsch, “Comparative study of stock trend prediction using time
delay, recurrent and probabilistic neural networks,” IEEE Transactions on neural networks, vol. 9, no. 6, pp.
1456–1470, 1998.
[6] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, vol. 9, no. 8, pp. 1735–
1780, 1997.
[7] Gers, F. A.; Schmidhuber, J. Recurrent nets that time and count. In Neural Networks, 2000. IJCNN 2000,
Proceedings of the IEEE-INNSENNS International Joint Conference on, volume 3, IEEE, 2000, pp. 189–194
[8] Lin, B.-S., Chu, W.-T., & Wang, C.-M. (2018). Application of Stock Analysis Using Deep Learning. 2018
7th International Congress on Advanced Applied Informatics (IIAI-AAI).

13. Dildar Ali (M.Tech in CSE) Aliah University 13
Deep Learning Approach to predict financial time series data