Successfully reported this slideshow.

# Prediction of Exchange Rate Using Deep Neural Network

Upcoming SlideShare
5.1 K plus proches voisins
×

# Prediction of Exchange Rate Using Deep Neural Network

2015/01/29 RWDA final presentation

2015/01/29 RWDA final presentation

## More Related Content

### Prediction of Exchange Rate Using Deep Neural Network

1. 1. Prediction of Exchange Rate Using Deep Neural Network 名古屋大学 情報科学研究科 武田研究室 林 知樹 1
2. 2. Agenda 1. Background 2. Outline of Deep Learning 3. Proposed method  The structure of proposed model  features 4. Experiments 5. Conclusion 2
3. 3. Background Earning without working is a dream for humans. 3 What is the shortest way to achieve this? The answer is FX
4. 4. Background  FX is money exchange game.  The shortest way to achieve our dream.  How to win?  This is very simple. 4 All you have to do is only predicting up or down. When the value is high, sell. When the value is low, buy.
5. 5. How to predict  My Hypothesis is  Prediction Using Deep Neural Network : DNN  State-of-the-art machine learning method 5 Future exchange rate consists of past information.
6. 6. Deep Neural Network (DNN) 6 Input layer Middle layers Output layer Layered Neural Network has a lot of middle layers Deep Neural Network ： In general, #middle layer > 3 The difference is only #middle layer.
7. 7. Deep Neural Network (DNN)  The structure of DNN doesn’t look new  We can’t train DNN with conventional method.  Initial parameters : randomization → Fall into bad local solution  Appropriate initialization method appeared.  Pre-training by RBM or Auto-Encoder 7 We can prevent the disappearance of gradient. but Disappearance of gradient problem
8. 8. EX. Image recognition  Before appearance of DNN  Appearance of DNN 8 Raw data Vector expression Feature Extraction Discriminator Recognition result Raw data Feature Extraction Recognition Deep Learning Human-made Training~ Learning comprehensively from feature extraction to discriminative system
9. 9. EX. Image recognition  Before appearance of DNN  Appearance of DNN 9 Raw data Vector expression Feature Extraction Discriminator Recognition result Raw data Feature Extraction Recognition Deep Learning Human-made Training~ Learning comprehensively from feature extraction to discriminative system
10. 10. EX. Image recognition  Before appearance of DNN  Appearance of DNN 10 Raw data Vector expression Feature Extraction Discriminator Recognition result Raw data Feature Extraction Recognition Deep Learning Human-made Training~ Learning comprehensively from feature extraction to discriminative system Achieved highest score using only raw data.
11. 11. Proposed method  2 kind of approach 1. Direct prediction of the exchange rate  Like Regression  Next time value is used as supervised data. 2. Binary option  2 Class Classification problem • In next time, the value become high → Class 1 • In next time, the value become low → Class 0  I used these value as supervised data. 11
12. 12. Regression by DNN 12 Middle layer Output layer xxh )( )( bhy  Wz y z Regression→no range Output identity mapping Output Real value Identity function W
13. 13. 2 Class Classification by DNN 13 ]1,0[ )exp(1 1 )( x xh   z 2 class → 0 or 1 Output is prob.→ [ 0, 1 ] Sigmoid function y WMiddle layer Output layer )( bhy  Wz Output
14. 14. Input Features  We used 10 kind of features as inputs.  Raw value  Exchange value  Top price  Low price  Closing value  Moving Average (9 points)  Relative Strength Index (RSI)  Stochastics RSI  Slow stochastics  Fast stochastics  Williams %R 14 Total 10 dim.
15. 15. DNN input 15 𝐷 dim. feature ・・・ 𝐷 dim. N frame DNN can deal with high dimension features and many frames. time Total (𝑁 + 1) × 𝐷 dim. Concatenated feature
16. 16. Flowchart 16 Calculatingfeatures Rawvalues 10dim.feature Featureconcatenation TrainingDNNusingfeatures Training phase Testing phase 100dim.feature TrainedDNN Inputconcatenatedfeature fortesting Predictedvalue
17. 17. Flowchart 17 Calculatingfeatures Rawvalues 10dim.feature Featureconcatenation TrainingDNNusingfeatures Training phase Testing phase 100dim.feature TrainedDNN Inputconcatenatedfeature fortesting Predictedvalue  Exchange value  Top price  Low price  Closing value
18. 18. Flowchart 18 Calculatingfeatures Rawvalues 10dim.feature Featureconcatenation TrainingDNNusingfeatures Training phase Testing phase 100dim.feature TrainedDNN Inputconcatenatedfeature fortesting Predictedvalue  Raw values  Moving Average  RSI  Stochastics RSI  Williams %R
19. 19. Flowchart 19 Calculatingfeatures Rawvalues 10dim.feature Featureconcatenation TrainingDNNusingfeatures Training phase Testing phase 100dim.feature TrainedDNN Inputconcatenatedfeature fortesting Predictedvalue time
20. 20. Flowchart 20 Calculatingfeatures Rawvalues 10dim.feature Featureconcatenation TrainingDNNusingfeatures Testing phase 100dim.feature TrainedDNN Inputconcatenatedfeature fortesting Predictedvalue 1. Pre-training  RBM 2. Fine-tuning  back propagation Training phase
21. 21. Flowchart 21 Calculatingfeatures Rawvalues 10dim.feature Featureconcatenation TrainingDNNusingfeatures Training phase 100dim.feature TrainedDNN Inputconcatenatedfeature fortesting Predictedvalue Predicted value 100 dim. feature Test data Testing phase
22. 22. An Exchange rate data 22  including data from 1991 to 2014.  Time interval is 1 hour. Date Time Exchange rate Top price Low price Closing value Total transaction
23. 23. Experiment  Experimental conditions  DNN training conditions 23 Dataset \$-¥ Exchange rate # data 1991/01/04 ~ 2015/01/5 Total 97362 points # DNN layer 5 layers # middle layer node 256 nodes Pre-training Fine-tuning Learning rate 0.002 0.00006 Momentum 0.9 0 Batch size 128 128 Epoch 30 50
24. 24. Dividing dataset 24 Training data ① Training data ② Training data ③ Test data ① Test data ② Test data ③  Each test data has 24 points(24 hours).  In this time, I made from ① to ㉛.
25. 25. Direct prediction  Input :  Features calculated by presence and past signal  Output :  the next time closing value 26
26. 26. Direct prediction result 27
27. 27. Direct prediction result 28 Closed test Close up
28. 28. Direct prediction result 29 Closed test Prediction could capture characteristics of answer line.
29. 29. Direct prediction result 30 Open test Close up
30. 30. Direct prediction result 31 Open test Predicted signal fluctuates. There is no information about that in the next time the value will become up or down.
31. 31. Direct prediction result 32 Open test Predicted signal fluctuates. There is no information about that in the next time the value will become up or down.
32. 32. Binary option  Input :  Features calculated by presence and past signal  Output :  In the next time, up(Class 1) or down(Class 0) 33
33. 33. Binary option result  Closed test  Open test 34 )96366/51516([%]46.53Acc. )744/375([%]40.50Acc. Using dice is better than this method.
34. 34. Using dice is better than this method. Binary option result  Closed test  Open test 35 )96366/51516([%]46.53Acc. )500/252([%]40.50Acc.
35. 35. Why we couldn’t predict?  Small fluctuation prevents us from predicting. 36
36. 36. Why we couldn’t predict?  Small fluctuation prevents us from predicting. 37 Similar to white noise
37. 37. Why we couldn’t predict?  Small fluctuation prevents us from predicting. 38 Similar to white noise
38. 38. Another approach  Prediction of trend transition  Trend transition means  The value will become up or down for Moving average in the past 𝑁 hours.  We can ignore the effect of small fluctuation. 39
39. 39. Prediction of trend transition 40  Input :  Features calculated by presence and past signal  Output :  In the next time, trend will become up(Class 1) or down(Class 0)
40. 40. Prediction of trend transition  Closed test  Open test 41 )97338/81516([%]75.83Acc. )744/678([%]63.87Acc.
41. 41. 42 Prediction of trend transition Predicted value is [0,1]. The closer to 1 or 0 predicted value is, the more reliable the prediction is. We can set the threshold to make the prediction more reliable.  Open test (Setting Threshold as 0.8 and 0.2) )515/487([%]61.94Acc.
42. 42. 43 Prediction of trend transition Predicted value is [0,1]. The closer to 1 or 0 predicted value is, the more reliable the prediction is. We can set the threshold to make the prediction more reliable.  Open test (Setting Threshold as 0.8 and 0.2) )515/487([%]61.94Acc.
43. 43. Conclusion and future works  Conclusion  We try to predict exchange rate using DNN.  3 kind of approach  Direct prediction  Binary option  Trend transition  We could predict trend transition with 83%(Closed) and 87%(Open).  Future problem  Considering another kind of feature  Prediction of more long term change 44 Failed… Failed… Succeeded!!
44. 44. Thank you for your attention! 45
45. 45. Pre training  RBMによる貪欲学習  1層目と2層目をRBMとみなして学習  2層目の出力をサンプリング  2層目と3層目をRBMとみなして学習  以下繰り返し (1)1段目のRBM学習 (2)2段目のRBM学習 (3)3段目のRBM学習