Foster discussion on the practical approach of applying artificial intelligence (AI) in quant trading and avoiding the common pitfalls and the opportunities and future of quant trading. Financial trading is the space in which people have to make decisions under uncertain uncertainty and requires decisions to be explained. However, most AI algorithm (e.g. neural network) are black-box and serves well under only constrained environment (e.g. Go / Atari). Therefore, it is important to gather domain knowledge such that AI (e.g. PGM) could have more cognitive ability to provide a white-box explanation for their trading decisions. If AI could provide a clearer visibility on the reality & present, it can assist human to provide faster, more-accurate and more-informed investment decisions. With a better understanding of the world, we can allocate resources more efficiently and ultimately create a better world to live. Video for the slides: https://www.youtube.com/watch?v=sideoQYAVDM&t=351s

1. Q U A N T T R A D I N G W I T H
A R T I F I C I A L I N T E L L I G E N C E
T H E P R A C T I C A L A P P R O A C H A N D C O M M O N P I T F A L L S
By Roger Lee

2. Develop your own trading
strategy with Python
Avoid common pitfalls
ENHANCE VISIBILITY OF THE REALITY
FOSTER MORE INTELLIGENT USE OF AI IN TRADING
Challenges in AI + Trading

3. Quant Trading Process
• Data Curation
• Feature Analysis (Black box)
• Strategy (White box)
• Backtesting
• Deployment
FUNCTIONDATA PORTFOLIO
MANAGEMENT
OP
OP
ALPHA

4. MACRO FUNDAMENTAL MARKET DATA
• GDP
• INDUSTRY GROWTH RATE
• INDUSTRY STATISTICS
• ASSET / LIABILITIES
• REVENUE / PROFIT
• OPERATIONAL STATS
• STOCK PRICE / YIELDS /
IMPLIED VOL.
• MARKET CAP.
• VOLUME
• DIVIDEND / COUPONS
• MICROSTRUCTURES (BID /
ASK / DEPTH / ORDER TYPE)
Marco
Low Freq.
Mirco
High Freq.
EVENTS ALTERNATIVE DATA
• ANALYST
RECOMMENDATION
• CREDIT RATINGS
• EARNINGS CALL /
GUIDANCE
• NEWS
• NEWS SENTIMENT
• TWITTER
• SATELLITE IMAGES
• GOOGLE TRENDS
DATA

5. FUNCTION
MACHINE LEARNING NLP SPEECH KNOWLEDGE ENGINEERING
• SUPERVISED LEARNING
• UNSUPERVISED LEARNING
• REINFORCEMENT LEARNING
• DEEP LEARNING
• NAME ENTITY RECOGNITION
• SENTIMENT
• TRANSLATION
• SPEECH-TO-TEXT • BAYESIAN LEARNING
• MARKOV

6. PORTFOLIO MANAGEMENT
v In trading, there are 3 parameters for decision making
- Asset
- Time (A lot of people just only focus on the time (1 dimensional) to analyze)
- Position
v Function picks the (Asset, Timing) pair
v Portfolio Management picks (Position)
- Very important for risk management
- Recent news: Optionsellers.com

7. EXAMPLE -
SUPERVISED LEARNING
Apply supervised learning on stock ranking
We use only OHLCV data on GOOGLE (Alphabet) and APPLE stocks

8. STEP 1: CALL ALL RELEVANT LIBRARY
STEP 2: READ DATA FROM PRE-PREPARED CSV (DATA FROM 2014 –
2018, ROUGHLY 1000 SAMPLES)
E
X
A
M
P
L
E
|
S
U
P
E
R
V
I
S
E
D
L
E
A
R
N
I
N
G

9. STEP 3: CALCULATE RETURN AND NORMALIZE DATA
STEP 4: PREPARE TARGET DATA (LABEL GOOG > AAPL AS 1, ELSE 0)
E
X
A
M
P
L
E
|
S
U
P
E
R
V
I
S
E
D
L
E
A
R
N
I
N
G

10. STEP 5: SEPARATE TRAINING AND TEST SAMPLES
STEP 6: APPLY LOGISTIC REGRESSION AND CALCULATE ACCURACY
E
X
A
M
P
L
E
|
S
U
P
E
R
V
I
S
E
D
L
E
A
R
N
I
N
G

11. EXAMPLE -
SUPERVISED LEARNING
Result:
• 44% accuracy on predicting the return ranking of GOOG and AAPL ~200 data
points (with ~800 data as training sets) with 40 lines of code
Not satisfied

12. EXAMPLE -
SUPERVISED LEARNING
Hypothesis:
• People will exit when the achieve a certain return…
• Combine OHLC data (OHLC/4)
• Create a new number called volume-weighted average return
Result:
• With just 4 variables, achieved 65% accuracy
Can we do more?
• Goal is to find features that can identify / predict prices
• We can do it both by human knowledge and Artificial intelligence algorithm

13. EXAMPLE -
UNSUPERVISED LEARNING
“Days Like today”
• Can we cluster similar days historically?
• How do we define similar days?
• How can we cluster high-dimensional data?

14. E
X
A
M
P
L
E
|
U
N
S
U
P
E
R
V
I
S
E
D
L
E
A
R
N
I
N
G
LIST OF 20 FEATURES TO TEST:
• FOCUS ON TECHNICAL ANALYSIS ONLY
• ICHIMOKU
• RSI
• STOCHASTIC
• MACD
• BOLLINGER BAND
• WILLIAM’S R
• PARABOLIC SAR
• CCI
• ETC.

15. E
X
A
M
P
L
E
|
U
N
S
U
P
E
R
V
I
S
E
D
L
E
A
R
N
I
N
G

16. E
X
A
M
P
L
E
|
U
N
S
U
P
E
R
V
I
S
E
D
L
E
A
R
N
I
N
G
1. Normalize Data
2. Svd (20 -> 10)
3. TSNE (10 -> 2D Points)
4. Clustering (Affinity
Propagation)

17. E
X
A
M
P
L
E
|
U
N
S
U
P
E
R
V
I
S
E
D
L
E
A
R
N
I
N
G

18. LIBRARY USES
NUMPY Optimize numerical and matrix operations
PANDAS Pandas dataframe for easy manipulation of timeseries data
Downlad financial data from various sources
MATPLOTLIB Plotting timeseries data on a chart to observe patterns
SCIKIT-LEARN Most ML / AI algorithms (excluding PGM)
SCIPY Scientific / quantitative computing
MULTIPROCESSING Optimize parallel calculations
TENSORFLOW Optimize for neural networks and google’s TPU (tensor processing unit)
PGMPY Basic PGM algorithm
QUANDL / ALPHA-VANTAGE Data sources (free / freemium / premium)
Useful Python Library

19. P I T F A L L S O F
U S I N G A I

20. #1: DATA CURATION
• Biased Data (e.g. look-forward bias / survivorship bias)
• Low-quality Data
• Throw away too much Data
• Common Curation steps
- Stock Prices: should be adjusted for splits and dividends
- Futures / Option Prices: handle rollover situations
- FX prices: OTC Traded so volume from one broker may not be relevant
- Timezone adjustments / lunch hours adjustments
- Holidays / No trading: fill by previous / blank

21. #1: EXAMPLE – LOW QUALITY DATA
• Daily data / Monthly data / Quarterly data / Yearly Data
- Too few data points to work in deep networks (or even shallow networks).
- For Daily data, 10 years (which is already maximum to avoid market regime change issue), just roughly 260*10 = 2,600 data points
- High bias
• Tick data
- Contains too many noises.
- During peak hours when market opens / closes, there is a lot of trading and information flow happening.
- Apart from those, the information contained in the bars outside of these hours are insignificant
- High variance
Solution: Use Turnover Bar / Volume Bar

22. #1: EXAMPLE – THROW AWAY TOO MUCH DATA
• Return = ln(Pn / Pn-1) = ln Pn – ln Pn-1 = yn- yn-1 --- (*), where Pn stands for stock price at t=n
• When you are taking return from a series of price data, you have completely thrown away the Price information (yt) for any t in (1,n) as an
input that might be an important data and assume yn ~ N(μ,σ2)
• Solution: Apply fractional differentiation such that the result timeseries contains information of both price and return
• Fractional Differentiation formula (Easy to prove) :
• When α=1, it is equivalent to the first order derivative (which is formula (*))
• When α=0, it is equivalent to yn (Original log-Price Series)
• When α takes on any number between 0 and 1, it contains both information of return and price.

23. #2: RISK MANAGEMENT
• Neural Network forgets historical black swan events, and most AI cannot provide good
predictions on networks they have not encountered before
• Solution 1: LSTM / Neural network with attention / Memory network
• Solution 2: Knowledge Engineering & Generalisation
• Solution 3: A good risk management system

24. Reference
LSTM Neural Network With Attention Memory Network
Each repeating module will export
information to the next module. The next
module will decide whether to keep it or
throw it when passing to next module.
Attention mechanism distribute attention
of certain data at several time steps to
avoid information being “diluted” after
many time steps.
Controller, reader, writer are trained
consistently to store important events into
the memory.
Photo source: skymind.Ai, deepmind

25. Challenges and Opportunities
Problem Solution
Short-
term
focus
• Not enough structured data
• Too much useless data (noises)
• Turn unstructured data to structured data
(using AI)
• Extract value from information
(domain expert)
Long-
term
focus
• Making decisions based on incomplete
information & under uncertain uncertainty
• General AI algorithm cannot deal with
causation
• Most popular / commercialized / scalable AI
are black-box, there is lack of white-box
• Deal with causation & uncertainty (using AI)
• White-box knowledge engineering
(domain expert)

26. T h a n k y o u !
e m a i l :
c y l e e @ l i v e . h k
l i n k e d i n :
h t t p s : / / w w w . l i n k e d i n . c o m / i n / r o g e r -
l e e - c f a - 8 5 7 1 0 0 4 8 /