Prediction of Exchange Rate Using Deep Neural Network

Prediction of Exchange Rate
Using Deep Neural Network
名古屋大学情報科学研究科
武田研究室
林知樹
1

Agenda
1. Background
2. Outline of Deep Learning
3. Proposed method
 The structure of proposed model
 features
4. Experiments
5. Conclusion
2

Background
Earning without working is
a dream for humans.
3
What is the shortest way
to achieve this?
The answer is FX

Background
 FX is money exchange game.
 The shortest way to achieve our dream.
 How to win?
 This is very simple.
4
All you have to do is
only predicting up or down.
When the value is high, sell.
When the value is low, buy.

How to predict
 My Hypothesis is
 Prediction Using Deep Neural Network : DNN
 State-of-the-art machine learning method
5
Future exchange rate
consists of past information.

Deep Neural Network (DNN)
6
Input
layer
Middle
layers
Output
layer
Layered Neural Network has a lot of middle layers
Deep Neural Network ：
In general,
#middle layer > 3
The difference is only #middle layer.

Deep Neural Network (DNN)
 The structure of DNN doesn’t look new
 We can’t train DNN with conventional method.
 Initial parameters : randomization
→ Fall into bad local solution
 Appropriate initialization method appeared.
 Pre-training by RBM or Auto-Encoder
7
We can prevent the
disappearance of gradient.
but
Disappearance of gradient problem

EX. Image recognition
 Before appearance of DNN
 Appearance of DNN
8
Raw data Vector expression
Feature
Extraction
Discriminator
Recognition
result
Raw data
Feature
Extraction Recognition
Deep Learning
Human-made
Training~
Learning comprehensively
from feature extraction to discriminative system

9
Feature
Extraction
Discriminator
Recognition
result
Raw data
Feature
Deep Learning
Human-made
Training~

10
Feature
Extraction
Discriminator
Recognition
result
Raw data
Feature
Deep Learning
Human-made
Training~
Achieved highest score
using only raw data.

Proposed method
 2 kind of approach
1. Direct prediction of the exchange rate
 Like Regression
 Next time value is used as supervised data.
2. Binary option
 2 Class Classification problem
• In next time, the value become high → Class 1
• In next time, the value become low → Class 0
 I used these value as supervised data.
11

Regression by DNN
12
Middle layer
Output layer
xxh )(
)( bhy  Wz
y
z
Regression→no range
Output identity mapping
Output
Real value
Identity function
W

2 Class Classification by DNN
13
]1,0[
)exp(1
1
)(
x
xh


z
2 class → 0 or 1
Output is prob.→ [ 0, 1 ]
Sigmoid function
y
WMiddle layer
Output layer
)( bhy  Wz
Output

Input Features
 We used 10 kind of features as inputs.
 Raw value
 Exchange value
 Top price
 Low price
 Closing value
 Moving Average (9 points)
 Relative Strength Index (RSI)
 Stochastics RSI
 Slow stochastics
 Fast stochastics
 Williams %R
14
Total 10 dim.

DNN input
15
𝐷 dim. feature
・・・
𝐷 dim.
N frame
DNN can deal with high dimension features and many frames.
time
Total (𝑁 + 1) × 𝐷 dim.
Concatenated feature

Flowchart
16
Calculatingfeatures
Rawvalues
10dim.feature
Featureconcatenation
TrainingDNNusingfeatures
Training
phase
Testing
phase
100dim.feature
TrainedDNN
Inputconcatenatedfeature
fortesting
Predictedvalue

Flowchart
17
Calculatingfeatures
Rawvalues
10dim.feature
Training
phase
Testing
phase
100dim.feature
TrainedDNN
fortesting
Predictedvalue
 Exchange value
 Top price
 Low price
 Closing value

Flowchart
18
Calculatingfeatures
Rawvalues
10dim.feature
Training
phase
Testing
phase
100dim.feature
TrainedDNN
fortesting
Predictedvalue
 Raw values
 Moving Average
 RSI
 Stochastics RSI
 Williams %R

Flowchart
19
Calculatingfeatures
Rawvalues
10dim.feature
Training
phase
Testing
phase
100dim.feature
TrainedDNN
fortesting
Predictedvalue
time

Flowchart
20
Calculatingfeatures
Rawvalues
10dim.feature
Testing
phase
100dim.feature
TrainedDNN
fortesting
Predictedvalue
1. Pre-training
 RBM
2. Fine-tuning
 back propagation
Training
phase

Flowchart
21
Calculatingfeatures
Rawvalues
10dim.feature
Training
phase
100dim.feature
TrainedDNN
fortesting
Predictedvalue
Predicted value
100 dim. feature
Test data
Testing
phase

An Exchange rate data
22
 including data from 1991 to 2014.
 Time interval is 1 hour.
Date Time
Exchange
rate
Top
price
Low
price
Closing
value
Total
transaction

Experiment
 Experimental conditions
 DNN training conditions
23
Dataset $-¥ Exchange rate
# data
1991/01/04 ~ 2015/01/5
Total 97362 points
# DNN layer 5 layers
# middle layer node 256 nodes
Pre-training Fine-tuning
Learning rate 0.002 0.00006
Momentum 0.9 0
Batch size 128 128
Epoch 30 50

Dividing dataset
24
Training data ①
Training data ②
Training data ③
Test data ①
Test data ②
Test data ③
 Each test data has 24 points(24 hours).
 In this time, I made from ① to ㉛.

Direct prediction
 Input :
 Features calculated by presence and past signal
 Output :
 the next time closing value
26

Direct prediction result
28
Closed test
Close up

29
Closed test
Prediction could capture characteristics of answer line.

30
Open test
Close up

31
Open test
Predicted signal fluctuates.
There is no information about that
in the next time the value will become up or down.

32
Open test
Predicted signal fluctuates.
There is no information about that
in the next time the value will become up or down.

Binary option
 Input :
 Output :
 In the next time, up(Class 1) or down(Class 0)
33

Binary option result
 Closed test
 Open test
34
)96366/51516([%]46.53Acc.
)744/375([%]40.50Acc.
Using dice is better than this method.

Using dice is better than this method.
Binary option result
 Closed test
 Open test
35
)96366/51516([%]46.53Acc.
)500/252([%]40.50Acc.

Why we couldn’t predict?
 Small fluctuation prevents us from predicting.
36

37
Similar to white noise

38
Similar to white noise

Another approach
 Prediction of trend transition
 Trend transition means
 The value will become up or down
for Moving average in the past 𝑁 hours.
 We can ignore the effect of small fluctuation.
39

Prediction of trend transition
40
 Input :
 Output :
 In the next time, trend will become up(Class 1)
or down(Class 0)

 Closed test
 Open test
41
)97338/81516([%]75.83Acc.
)744/678([%]63.87Acc.

42
Predicted value is [0,1].
The closer to 1 or 0 predicted value is,
the more reliable the prediction is.
We can set the threshold
to make the prediction more reliable.
 Open test (Setting Threshold as 0.8 and 0.2)
)515/487([%]61.94Acc.

43
Predicted value is [0,1].
The closer to 1 or 0 predicted value is,
the more reliable the prediction is.
We can set the threshold
to make the prediction more reliable.
 Open test (Setting Threshold as 0.8 and 0.2)
)515/487([%]61.94Acc.

Conclusion and future works
 Conclusion
 We try to predict exchange rate using DNN.
 3 kind of approach
 Direct prediction
 Binary option
 Trend transition
 We could predict trend transition
with 83%(Closed) and 87%(Open).
 Future problem
 Considering another kind of feature
 Prediction of more long term change
44
Failed…
Failed…
Succeeded!!

Thank you for your attention!
45

Pre training
 RBMによる貪欲学習
 1層目と2層目をRBMとみなして学習
 2層目の出力をサンプリング
 2層目と3層目をRBMとみなして学習
 以下繰り返し
(1)1段目のRBM学習 (2)2段目のRBM学習 (3)3段目のRBM学習

Prediction of Exchange Rate Using Deep Neural Network

More Related Content

What's hot

Similar to Prediction of Exchange Rate Using Deep Neural Network

More from Tomoki Hayashi

Recently uploaded

In this document

Prediction of Exchange Rate Using Deep Neural Network