Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Deep learning in finance
Data science in finance
28 / 07
Sébastien Jehan
CAPITAL FUND MANAGEMENT
 Fundamental research applied to financial
markets
 CFM invests in Science, Technology and Finan...
Artificial Neural Networks
- History of Artificial Neural
Networks
- Recurrent Neural Networks
- Applications in Finance
A...
Current applications to neural
networks
• Medical image processing:
mostly feed forward neural
network
• Robokinetics and ...
10/2013
90-99/100
ON CAPTCHAS
1957: The perceptron
-
d
D0
D1
D2
Input
Layer
Output
Layer
Destinations
1957: The perceptron
D0
D1
D2
Input
Layer
Output
Layer
Destinations
FEED FORWARD
1957: The perceptron
D0
D1
D2
Input
Layer
Output
Layer
Destinations
SINGLE LAYER
Teaching
-
d
D0
D1
D2
Input
Layer
Output
Layer
Theorical
Y0
Y1
Y2
-
-
-
Supervised objective function
Application .NET
Applications
Is A or B
Linear problem
Applications
Is A or B
Not linearly separable
=> no convergence
1986: Multilayers perceptron
input vector
hidden
layers
outputs
1986: Multilayers perceptron
input vector
hidden
layers
outputs
+29 ANS
1986: Multilayers perceptron
input vector
hidden
layers
outputs
Back-
propagate
error signal
Back propagation
Activations
The error:
Update
Weights:
0
1
0
.5
-5 5
Slide credit : Geoffrey H
Back propagation
Activations
The error:
Update
Weights:
0
1
0
.5
-5 5
Slide credit : Geoffrey Hinton
errors
Drawbacks
GRADIENT NATURE => SLOW CONVERGENCE => SLOW LEARNING
Drawbacks
GRADIENT NATURE => SLOW CONVERGENCE => SLOW LEARNING
LEARNING DECREASE WITH HIDDEN LAYERS NUMBERS
Drawbacks
GRADIENT NATURE => SLOW CONVERGENCE => SLOW LEARNING
LEARNING DECREASE WITH HIDDEN LAYERS NUMBERS
CAN FIND ONLY ...
Before 2006: The deapest, the worst
DEEP NETWORKS (LOT OF HIDDEN LAYERS) ARE WORSE THAN WITH ONE
OR TWO LAYERS ! SLOWER AN...
Deep ? How deep ?
DEEP IS >=5 LAYERS
2006: breakthrough
Who cares ?
2013: Director of Facebook
AI research
Google distinguished
researcher
Montreal University
The effect of unsupervised learning
WITHOUT UNSUPERVISED
LEARNING (top quartiles)
WITH UNSUPERVISED
LEARNING
Becomes more non-linear, and this is
good: it prevents the gradient learning to be
transferred to previous layers for loca...
Becomes more non-linear, and this is
good: it prevents the gradient learning to be
Yet we don’t know how.
Just represent t...
Size matters
Connectivity cost
Connectivity cost
Infrastructure costs
• The bad news
In 2012, It took Google 16.000 CPU to
have a single process real time cat face
identif...
Imagenet classification results
2012
2014 Deep Learning GoogleNet 6.66%
2015 Microsoft Research 4.94 %
ai is learned
http:...
Reverse engineering deep learning
results(2012)
SUPERVISED IMAGE CLASSIFICATIONS
+ OTHER TRAININGS
First layer: always Gab...
Reverse engineering deep learning
results(Nov 2014)
SUPERVISED IMAGE CLASSIFICATIONS
+ OTHER TRAININGS
General or Specific...
Static deep learning in finance
+
Person
detection
PMFG
Performance
heat map
… Trade
opportunity
detection
Market States
C...
PMFG: Planar Maximally Filtered Graph of the correlation matrix
Correlation, hierarchies and
Networks in Financial Markets
“market states”
Cluster time
periods by
correlation
matrix
similarity
Heat map 1 year perf, 23/07
1990: RNN
nonstationary I/O mapping, Y(t)f(X(t)), X(t) and Y(t) are
timevarying patterns
1990: RNN
nonstationary I/O mapping, Y(t)f(X(t)), X(t) and Y(t) are
timevarying patterns
The closest thing to computer ...
RNN detailed
Problems in RNN
Time lag > 5 => Difficult to learn
The error either vanishes or
explodes in the learning process
Divergent...
RNN for GARCH(1,1) predictions
IN SAMPLE:
665 observations 01/2011 09/2013
OUT OF SAMPLE:
252 observations 09/2013 09/2014...
A new approach: RNN with wavelet
sigmoid (2D, time and frequency) April
2015
PREDICT “SEMI-CHAOTIC” TIME SERIE
(MACKEY-GLA...
Particle swarm optimization
• Concepts
• Applications: Training a neural network
using PSO instead of backpropagation
PSO principles
Here I am! The best perf. of
my neighbours
Mybest
perf.
x
pg
pi
v
Collective Intelligence: Particles Adjust...
Training neural network with PSO (started in
2006)
Unsupervised training with PSO:
 Not trapped in local minima
 Faster ...
PSO ANN accuracy (July 2014)
Levenberg-Marquadt: second order in BP error evaluation
Credit Approval dataset, UCI
The challenge of
multidimensionality
- Dimensionality reduction
techniques
Dimension reduction techniques
PCA using Neural Networks(2012)
PCA with Gaussian assumption:
Training set: 1/16 compressio...
Radial Basis Function Neural Network with (2D)^2 PCA
(April 2015, Shangai index)
TRAIN
TEST
Radial Basis Function Neural Network with (2D)^2 PCA
(April 2015, Shangai index)
TRAIN
TEST
Exercise: find the issue
Thank you
International workshop on Deep Learning, Lille, France (July
10/11 2015):
https://sites.google.com/site/deeplear...
Deeplearning in finance
Deeplearning in finance
Deeplearning in finance
Deeplearning in finance
Deeplearning in finance
Deeplearning in finance
Deeplearning in finance
Upcoming SlideShare
Loading in …5
×

24

Share

Download to read offline

Deeplearning in finance

Download to read offline

Meetup "Datascience in Finance" presentation
Introduction to deep learning techniques
Applications in finance

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Deeplearning in finance

  1. 1. Deep learning in finance Data science in finance 28 / 07 Sébastien Jehan
  2. 2. CAPITAL FUND MANAGEMENT  Fundamental research applied to financial markets  CFM invests in Science, Technology and Finance  23 years of experience in managing trading systems  A rational process that produces robust Trading Algorithms  Proprietary software able to execute & control large volumes  Opportunities : • Software Engineers mastering C++, Python… • System, Network & Database Administrators • PhDs in theoretical physics, applied mathematics, informatics… PROPRIETARY AND CONFIDENTIAL - NOT FOR REDISTRIBUTION
  3. 3. Artificial Neural Networks - History of Artificial Neural Networks - Recurrent Neural Networks - Applications in Finance AI for the enterprise 2015 $220 millions 2025 $11.1 billions (+56% / year)
  4. 4. Current applications to neural networks • Medical image processing: mostly feed forward neural network • Robokinetics and robovision: smooth moves and object detection • Military: DARPA Synapse, objective 10 billion neurons (86 billions human brain) in 2 liters space for end of 2016. • Email spam detection, Image classification • Text recognition
  5. 5. 10/2013 90-99/100 ON CAPTCHAS
  6. 6. 1957: The perceptron - d D0 D1 D2 Input Layer Output Layer Destinations
  7. 7. 1957: The perceptron D0 D1 D2 Input Layer Output Layer Destinations FEED FORWARD
  8. 8. 1957: The perceptron D0 D1 D2 Input Layer Output Layer Destinations SINGLE LAYER
  9. 9. Teaching - d D0 D1 D2 Input Layer Output Layer Theorical Y0 Y1 Y2 - - - Supervised objective function
  10. 10. Application .NET
  11. 11. Applications Is A or B Linear problem
  12. 12. Applications Is A or B Not linearly separable => no convergence
  13. 13. 1986: Multilayers perceptron input vector hidden layers outputs
  14. 14. 1986: Multilayers perceptron input vector hidden layers outputs +29 ANS
  15. 15. 1986: Multilayers perceptron input vector hidden layers outputs Back- propagate error signal
  16. 16. Back propagation Activations The error: Update Weights: 0 1 0 .5 -5 5 Slide credit : Geoffrey H
  17. 17. Back propagation Activations The error: Update Weights: 0 1 0 .5 -5 5 Slide credit : Geoffrey Hinton errors
  18. 18. Drawbacks GRADIENT NATURE => SLOW CONVERGENCE => SLOW LEARNING
  19. 19. Drawbacks GRADIENT NATURE => SLOW CONVERGENCE => SLOW LEARNING LEARNING DECREASE WITH HIDDEN LAYERS NUMBERS
  20. 20. Drawbacks GRADIENT NATURE => SLOW CONVERGENCE => SLOW LEARNING LEARNING DECREASE WITH HIDDEN LAYERS NUMBERS CAN FIND ONLY LOCAL MINIMA OF ERRORS
  21. 21. Before 2006: The deapest, the worst DEEP NETWORKS (LOT OF HIDDEN LAYERS) ARE WORSE THAN WITH ONE OR TWO LAYERS ! SLOWER AND LESS ACCURATE
  22. 22. Deep ? How deep ? DEEP IS >=5 LAYERS
  23. 23. 2006: breakthrough
  24. 24. Who cares ? 2013: Director of Facebook AI research Google distinguished researcher Montreal University
  25. 25. The effect of unsupervised learning WITHOUT UNSUPERVISED LEARNING (top quartiles) WITH UNSUPERVISED LEARNING
  26. 26. Becomes more non-linear, and this is good: it prevents the gradient learning to be transferred to previous layers for local optima The first layer should react to input changes.
  27. 27. Becomes more non-linear, and this is good: it prevents the gradient learning to be Yet we don’t know how. Just represent the dominant factors of variation of the input.
  28. 28. Size matters
  29. 29. Connectivity cost
  30. 30. Connectivity cost
  31. 31. Infrastructure costs • The bad news In 2012, It took Google 16.000 CPU to have a single process real time cat face identifier http://hexus.net/tech/news/software/41 537-googles-16000-cpu-neural-network- can-identify-cat/ • The good news In 2017, public Beta testing of HP “The machine”, based on Memristors replacing transistors for some parts of the chip. http://insidehpc.com/2015/01/video- memristor-research-at-hp-labs/
  32. 32. Imagenet classification results 2012 2014 Deep Learning GoogleNet 6.66% 2015 Microsoft Research 4.94 % ai is learned http://arxiv.org/pdf/1502.01852v1.pdf
  33. 33. Reverse engineering deep learning results(2012) SUPERVISED IMAGE CLASSIFICATIONS + OTHER TRAININGS First layer: always Gabor Filters like or Color Blob
  34. 34. Reverse engineering deep learning results(Nov 2014) SUPERVISED IMAGE CLASSIFICATIONS + OTHER TRAININGS General or Specific Layer ?? Transfer ANN layers among trained models
  35. 35. Static deep learning in finance + Person detection PMFG Performance heat map … Trade opportunity detection Market States Current markets … Reduce information redundancy for a goal
  36. 36. PMFG: Planar Maximally Filtered Graph of the correlation matrix Correlation, hierarchies and Networks in Financial Markets
  37. 37. “market states” Cluster time periods by correlation matrix similarity
  38. 38. Heat map 1 year perf, 23/07
  39. 39. 1990: RNN nonstationary I/O mapping, Y(t)f(X(t)), X(t) and Y(t) are timevarying patterns
  40. 40. 1990: RNN nonstationary I/O mapping, Y(t)f(X(t)), X(t) and Y(t) are timevarying patterns The closest thing to computer dreams
  41. 41. RNN detailed
  42. 42. Problems in RNN Time lag > 5 => Difficult to learn The error either vanishes or explodes in the learning process Divergent behavior
  43. 43. RNN for GARCH(1,1) predictions IN SAMPLE: 665 observations 01/2011 09/2013 OUT OF SAMPLE: 252 observations 09/2013 09/2014 LAG=1, NO PROBLEM
  44. 44. A new approach: RNN with wavelet sigmoid (2D, time and frequency) April 2015 PREDICT “SEMI-CHAOTIC” TIME SERIE (MACKEY-GLASS), SOLUTION OF Rapidly vanishing property of wavelet function => No divergent behavior
  45. 45. Particle swarm optimization • Concepts • Applications: Training a neural network using PSO instead of backpropagation
  46. 46. PSO principles Here I am! The best perf. of my neighbours Mybest perf. x pg pi v Collective Intelligence: Particles Adjust their positions according to a ``Psychosocial compromise’’ between what an individual is comfortable with, and what society reckons => Solve continuous optimization problems
  47. 47. Training neural network with PSO (started in 2006) Unsupervised training with PSO:  Not trapped in local minima  Faster than back propagation
  48. 48. PSO ANN accuracy (July 2014) Levenberg-Marquadt: second order in BP error evaluation Credit Approval dataset, UCI
  49. 49. The challenge of multidimensionality - Dimensionality reduction techniques
  50. 50. Dimension reduction techniques PCA using Neural Networks(2012) PCA with Gaussian assumption: Training set: 1/16 compression, 4x4 blocks Result
  51. 51. Radial Basis Function Neural Network with (2D)^2 PCA (April 2015, Shangai index) TRAIN TEST
  52. 52. Radial Basis Function Neural Network with (2D)^2 PCA (April 2015, Shangai index) TRAIN TEST Exercise: find the issue
  53. 53. Thank you International workshop on Deep Learning, Lille, France (July 10/11 2015): https://sites.google.com/site/deeplearning2015/accepted- papers
  • keytouch7

    Dec. 17, 2020
  • leongkuilim

    May. 23, 2018
  • JamesFu16

    Mar. 23, 2018
  • TheGU

    Sep. 30, 2017
  • khalala

    Aug. 15, 2017
  • NajeebHZaidiFRM

    Aug. 3, 2017
  • LukaVidoni

    Jun. 25, 2017
  • antreps

    Apr. 11, 2017
  • naveenbali1

    Mar. 17, 2017
  • gprince66

    Oct. 17, 2016
  • ZhunHan

    Oct. 12, 2016
  • HailiDENG

    Sep. 22, 2016
  • PoonsukLohsoonthorn

    Sep. 17, 2016
  • cccheng914

    Sep. 10, 2016
  • BenWang26

    Sep. 6, 2016
  • syuzique

    Aug. 8, 2016
  • toshiogata1

    Jun. 24, 2016
  • erickmiller

    Jun. 2, 2016
  • choeungjin

    Mar. 24, 2016
  • ssuser0b6d80

    Mar. 15, 2016

Meetup "Datascience in Finance" presentation Introduction to deep learning techniques Applications in finance

Views

Total views

14,195

On Slideshare

0

From embeds

0

Number of embeds

130

Actions

Downloads

674

Shares

0

Comments

0

Likes

24

×