NFL, DL, and Financial Sector

NFL, DL, and Financial Sector
2018.05.24
서울대 융합과학부
Applied Data Science Lab.
이원종 부교수, Ph.D., IEEE Fellow

Confidential and Proprietary – SNU ADS Lab.
• Searching for a golden algorithm?
• To beat all in the market?
• Deep Learning?
2
People in Financial Sector

• NFL Theorem
• Feature Engineering
• Power of DL
• Current Status
3
Contents

NFL Theorem

No Free Lunch Theorem
5

• Coin toss game
0 0 1 1 1 0 0 1 1 1 ?
Next – 0 or 1?
• Algorithms
• Alg1:  Random
• Alg2:  If 3 consecutive 1’s predict 1, otherwise random
• Alg3:  If 3 consecutive 1’s predict 0, otherwise random
• …
• Should one algorithm preferred over others?
A Simplified Example
6

0 0 0
0 0 1
0 1 0
0 1 1
1 0 0
1 0 1
1 1 0
1 1 1
• Given the first two outputs, predict the third
• All algorithms perform exactly the same on the average
No If All Are Equally Possible
7

• Domain knowledge can improve performance
• At the cost of generality
The Teaching
8
Figure from https://medium.com/@LeonFedden/the‐no‐free‐lunch‐theorem‐62ae2c3ed10c

• Use of knowledge in machine learning
“Choosing one of NN, RF, SVM”
• Each algorithm has a long history, and thus knowledge has been
formed
• If too many tried over the same data,
statistical learning theory teaches that overfitting will occur…
• The thing is… a prediction algorithm uses only data
It is human’s job to bring knowledge and choose a proper
model
Knowledge + Data
9

• Knowledge in model structure
• NN, RF, SVM, adaBoost, naïve Bayes, etc.
• Knowledge from domain experience
• Knowledge extracted from (extended set of) data itself
• Feature engineering, embedding, autoencoder/transfer‐learning
• Feature engineering
• Heuristically overcoming limitations of models
• Extracting information from data using knowledge
0 0 1 1 1 0 0 1 1 1 0 0 1 1 1 0 1 1 1 0 0 0 1 1 1 0 0 0 1 1 1 0 1 1 1
 Count of ‘1 1 1’
Knowledge
10

Feature Engineering

• Models are limited
• Patterns and rules are very unique for each domain
• If one can extract the right features
• Model does not need to be very smart
• ML’s performance is heavily dependent on how the data is
presented
Feature from Data
12

From Wiki
13

• Done by highly skilled people in each domain
• Often the deciding factor of the performance
• Are you a quant?
What’s your secret sauce?
Regime, factor model, momentum…
Feature Engineering
14

Confidential and Proprietary – SNU ADS Lab. 15
kaggle
Source: http://machinelearningmastery.com/discover‐feature‐engineering‐
how‐to‐engineer‐features‐and‐how‐to‐get‐good‐at‐it/

• Become ‘skilled’
• Try over many data sets
• Repeat
• Deploy
• Monitor if the secret sauces are still working
The Labor of Feature Engineering
16

• Use one’s best knowledge for extracting features (rules)
from the data
• Typically heuristics dominate
• Any smart person can do it
• But, algorithms can be used, too
• With proper training
Domain Experts
17

Hinton’s Slide
1. Convert the raw input vector into a vector o
f feature activations.
Use hand‐written programs based on comm
on‐sense to define the features.
2. Learn how to weight each of the feature act
ivations to get a single scalar quantity.
3. If this quantity is above some threshold, dec
ide that the input vector is a positive examp
le of the target class.
The standard Perceptron
architecture
feature units
decision unit
input units
hand‐coded weights
or programs
learned weights
The standard paradigm for statistical
pattern recognition
Source: G. Hinton, Coursera Class “Neural Networks
for Machine Learning”

Hinton’s Slide
Learning with hidden units
• Networks without hidden units are very limited in the input‐output mappings they can lear
n to model.
• More layers of linear units do not help. Its still linear.
• Fixed output non‐linearities are not enough.
• We need multiple layers of adaptive, non‐linear hidden units.
But how can we train such nets?
• We need an efficient way of adapting all the weights, not just the last layer.
This is hard.
• Learning the weights going into hidden units is equivalent to learning features.
• This is difficult because nobody is telling us directly what the hidden units should do.
Source: G. Hinton, Coursera Class “Neural Networks
for Machine Learning”

Power of DL

• It’s a model (deep version of neural network)
• Unlike other models, so powerful that it can model any
function f(x)
• All of SVM, adaBoost, naïve Bayes plus feature engineering?
Deep Learning
21

NFL and DL
• Considering its millions of parameters, its supposed to
overfit quickly – but it does not…
• Overall effect
 Automatic feature engineering (knowledge)

• Very good at extracting features and encoding them
• Called representation
• Automated feature engineering
• Finally less labor
• Deep learning itself needs knowledge for
• Tuning (hyper‐parameter optimization)
• But less effort, less domain knowledge
• Your secret sauce might be defeated in a few days
• Deciding what constraints to impose
• No guarantee that it will converge to f(x) that is ideal
• CNN, RNN, relational, etc.
• Much higher level of research concepts compared to kernel functions
• No guarantee that it will beat human
• Depends on domain
Deep Learning
23

Representation Learning
24
• Truly random distribution example
– Each pixel 256 possible values
– 3 channels (Red, Green, Blue)
– 1000 pixels
 256 , ,
combinations
• Concept of “Manifold” in learning
– Quite different from statistical noise
problems

• Some high dimensional problems have strong structure
 Deep learning will outperform any other
• Image, language
• Low dimensional and/or statistical noise problems
 No reason for DL to outperform
• What’s your problem like?
• Your data?
Manifold
25

Current Status

• If you wait enough time,
all algorithm engineers
will be working on deep
learning
• As of today, black hole of
research
• (AI winter again?)
Today’s Research
27

• No matter how hard you study, technology changes faster
than you
• Many new tricks
• The new standard
• If a trick is found, look for domains/applications
When So Many Smart People Work on DL
28

• Massive data
• Stock market
• Price is a reflection of all people’s trades
• Many random variables added  Gaussian distribution?
OR
• Can it have a strong structure as in image or language?
Back to Finance
29

• Active investing is not the only problem
• Where should we apply the new tricks?
• My view
• There will be many successful cases
• Often small improvements over what people have already done
well, but big improvements, too
• Will take some time for human (not machine) to figure out where
to apply what trick
• It’s an exciting time
(until the next AI winter?)
Back to Finance
30

NFL, DL, and Financial Sector

More Related Content

More from NAVER Engineering

Recently uploaded

NFL, DL, and Financial Sector