Masanori HIRANO, Hiroyasu MATSUSHIMA, Kiyoshi IZUMI, and Hiroki SAKAJI, "Implementation of Real Data for Financial Market Simulation using Clustering, Deep Learning, and Artificial Financial Market," The 23rd International Conference on Principles and Practice of Multi-Agent Systems (PRIMA 2020), Aichi, Nagoya, Japan, Nov. 18-20th, 2020. (Online)

1. Implementation of Real Data
for Financial Market Simulation
using Clustering, Deep Learning, and
Artificial Financial Market
Masanori HIRANO1, Hiroyasu MATSUSHIMA2,
Kiyoshi IZUMI1, and Hiroki SAKAJI1
1 School of Engineering, The University of Tokyo
2 Center for Data Science Education and Research, Shiga University
hirano@g.ecc.u-tokyo.ac.jp
https://mhirano.jp/

2. Motivation
• Instability in Financial Markets
• 2008 financial crisis
• Flush Crush
• Price fluctuation by COVID-19
• Regulations are necessary
• New regulations like Basel III
• Can avoid abovementioned crisis?
• Difficulties in Financial markets
• nonstationary
• Rare phenomena happen frequently
• => Simulation is good solution, but not trustable.
• Find what’s the matter
• Dealing with trustability with actual data
©M.HIRANO & Izumi Lab.
DJIA on May 6, 2010 (Flush crush)
DJIA in 2020 (COVID-19)
11/19/20 2

3. Last year: PRIMA 2019
• We showed the difference between simulation & data
• => simulation model can overpass key features.
©M.HIRANO & Izumi Lab.
Missing feature in
simulation
11/19/20 3

4. Our work
• Proposed a new model built using actual data
• Comparing in Simulation
Traditional model  Our new model w/ data
• Only focus on HFT-MM  Specific trader & strategy
• Target: Tokyo Stock Exchange
• We analyzed a special data
provided by JPX
11/19/20 ©M.HIRANO & Izumi Lab. 4
Tokyo Stock Exchange

5. What’s the HFT-MM?
• High-Frequency-Trader Market-Making strategy
• Market-making strategy:
• (Basically) order near the best price
• Get profit by the spread (1001-999=2)
• Do repeatedly
• Risk-hedge by high-frequency-trade:
• Always have price move risk (Price move >> spread)
• Do action faster & hedge risk by setting off their inventory
• => These features are easy to recognize in data
11/19/20 ©M.HIRANO & Izumi Lab. 5
Sell
Buy

6. Data Extraction
We need HFT-MM ordering data…
©M.HIRANO & Izumi Lab.11/19/20 6

7. Data
• “Order-book reproduction data”
provided by Japan Exchange Group (JPX)
• Containing masked trader information
<- Called “Virtual Server (VS)”
11/19/20 ©M.HIRANO & Izumi Lab. 7
Time Ticker Kind Buy/sell VS Price
11:11:50.702813 A Limit Order sell VS1 2570
11:11:50.703600 B Executed buy VS4 Market Order
11:11:50.704001 A Cancel sell VS1 2570
Sample
Some columns are not shown such as volume

8. Indices for clustering (extracting HFT-MM)
• The logarithm of action per ticker
ActionsPerTicker =
newOrders + changeOrders + (cancelOrders)
(numTicker)
ActionsPerTickerLOG = ln ActionsPerTicker
• Inventory Ratio
InventoryRatioABS
= 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 𝑛𝑛ticker
soldVolume ticker − boughtVolume ticker
soldVolume ticker + boughtVolume ticker
• Executed order ratio
• Cancel order ratio
• Market order ration
• The logarithm of ticker per VS
TickerPerVSLOG = ln
(numTicker)
(numVS)
11/19/20 ©M.HIRANO & Izumi Lab. 8
Many order
Low inventory
Many VS usage
Low executed ratio
High cancel ratio
Low market order ratio

9. Data outline
• Jan. 2015 – mid-Sep. 2015: All 178 business days
• All traders: 2654 traders
• Only HFT: 181 traders <= based on ActionsPerTicker ≥ 1000
• Hierarchical Clustering for HFT-MM
11/19/20 ©M.HIRANO & Izumi Lab. 9

10. Hierarchical Clustering [Uno et al. 18]
• Normalizations for each indices & clustering
• Euclidean distance
• Ward’s method
• 10 clusters
11/19/20 ©M.HIRANO & Izumi Lab. 10
HFT-MM cluster based on indices

11. Data split
• We got ordering data of HFT-MM
• 2015/01-07 => model training of HFT-MM
• 2015/08 => evaluation of simulation
11/19/20 ©M.HIRANO & Izumi Lab. 11

12. Simulation & Models
©M.HIRANO & Izumi Lab.11/19/20 12

13. Simulation outline
• We used “PlhamJ” as a simulation platform.
PlhamJ: Platform for Large-scale and High-frequency Artificial Market (Java version)
11/19/20 ©M.HIRANO & Izumi Lab. 13

14. Simulation setting
• 1,000 stylized traders + 1 traditional HFT-MM trader
vs
• 1,000 stylized traders + 1 ML HFT-MM trader (new)
• Comparison between behaviors of -
• 1 traditional HFT-MM trader in simulation
• 1 ML HFT-MM trader (new) in simulation
• Real data (out of learning data)
©M.HIRANO & Izumi Lab.11/19/20 14

15. Stylized Trader Agents [Chiarella et al. 02]
• Logarithmic return prediction for bid/ask price
𝑟𝑟 =
1
𝑤𝑤𝐹𝐹+𝑤𝑤𝐶𝐶+𝑤𝑤 𝑁𝑁
𝑤𝑤𝐹𝐹 ⋅ 𝐹𝐹 + 𝑤𝑤𝐶𝐶 ⋅ 𝐶𝐶 + 𝑤𝑤𝑁𝑁 ⋅ 𝑁𝑁
• Fundamentals
𝐹𝐹 =
1
mean reversion time
ln
current market price
current fundamental price
• Chartist (trend)
𝐶𝐶 = logarithm averaged return in the past
• Noise 𝑁𝑁 ~ 𝑁𝑁 0, 𝜎𝜎𝑁𝑁
• + margin => decide price
• Every 100 step they make a buy or sell order
11/19/20 ©M.HIRANO & Izumi Lab. 15

16. Traditional HFT-MM Trader [Avellaneda et al. 02]
• Trader’s price interval:
𝛾𝛾𝑖𝑖�𝜎𝜎𝑖𝑖
2
+
2
𝛾𝛾𝑖𝑖
ln 1 +
𝛾𝛾𝑖𝑖
𝑘𝑘
• Trader’s mid-price
𝑝𝑝𝑡𝑡
∗
− 𝛾𝛾𝑖𝑖�𝜎𝜎𝑖𝑖
2
𝑞𝑞𝑡𝑡
𝑖𝑖
• Note:
𝛾𝛾𝑖𝑖: risk-hedge level
�𝜎𝜎𝑖𝑖: SD in price
𝑘𝑘: a parameter for order arrival
𝑝𝑝𝑡𝑡
∗
: fundamental price
𝑞𝑞𝑡𝑡
𝑖𝑖
:inventory
11/19/20 ©M.HIRANO & Izumi Lab. 16
Price
Sell
Buy
Fundamental Price
Trader’s mid-price
Trader’s price interval

17. HFT-MM Machine Learned Model
• Using machine learning for data, we build a model
• Model predict the next action of traders
©M.HIRANO & Izumi Lab.11/19/20 17

18. Results
11/19/20 ©M.HIRANO & Izumi Lab. 18

19. Comparison
11/19/20 ©M.HIRANO & Izumi Lab. 19
Ticks between the best price and ordering of HFT-MM

20. Comparison in KL Divergence
• Our new ML model outperform traditional model
marginally…
• Why so big variance? =>
11/19/20 ©M.HIRANO & Izumi Lab. 20
Distribution of 𝐷𝐷𝐾𝐾𝐾𝐾 of our new model
Q P Mean SD
Actual Traditional 0.730009 0.119884
Actual ML 0.648459 0.957854

21. Comparison w/ omission
• Error case: easy to detect => omit them
• => the omission give us strong results
11/19/20 ©M.HIRANO & Izumi Lab. 21
Q P Mean SD
Actual Traditional 0.730009 0.119884
Actual ML (w/ omissions) 0.186192 0.085099

22. Discussion & Conclusion
• Our new model show the strong result w/ omission
• Reveal the needs & benefits of real data usage
• But, we should deal with non-robustness of ML model
11/19/20 ©M.HIRANO & Izumi Lab. 22
Future work
• More robust ML model
• Model building with data for all trader