2. How to build your own agent
● Modeling
● Training
● Backtesting
● Real-time paper trading
3. How to build the model
Market
Environment
Observation
Agent
Action
Reward
4. How to build the model
● Input : State t (Observation)
● Output : Action (Price Order, Limit Order, long or short, ask or bid)
● Reward
● St, At, Rt, St+1, At+1, Rt+1……
5. How to build the model
● Model architecture
○ FNN
○ CNN
○ RNN(LSTM, GRU)
○ CNN + RNN(LSTM, GRU)
○ UFCNN
6. How to build the model
● Objective Function
● Maximize Total Reward
● Set correct answer using historical data (ex : future>3% buy...)
○ RL way Rt+1*gamma + Rt+2*gamma^2 …
● Reinforcement Learning
7. Open Ai Gym
● https://github.com/openai/gym
● https://gym.openai.com/evaluations/eval_glkKKInTm6GlmcOQRZuhQ
import gym
env = gym.make("Breakout-v0")
observation = env.reset()
for _ in range(1000):
env.render()
action = env.action_space.sample() # your agent here (this takes random actions)
observation, reward, done, info = env.step(action)
8. Trading Gym
● https://github.com/Yvictor/TradingGym
import random
import pandas as pd
import trading_env
df = pd.read_hdf('dataset/SGXTW.h5', 'STW')
env = trading_env.make(obs_data_len=256, step_len=128,
df=df, fee=0.1, max_position=5, deal_col_name='Price',
feature_names=['Price', 'Volume', 'Ask_price','Bid_price',
'Ask_deal_vol','Bid_deal_vol', 'Bid/Ask_deal', 'Updown'])
env.reset()
env.render()
state, reward, done, info = env.step(random.randrange(3))
### random choice action and show the transaction detail
for i in range(500):
state, reward, done, info = env.step(random.randrange(3))
env.render()
if done:
break
env.transaction_details
9. Trading Gym
● Build your own agent
class YourAgent:
def __init__(self):
# build your network and so on
pass
def choice_action(self, state):
## your rule base condition or your max Qvalue action or Policy Gradient action
# action=0 -> do nothing
# action=1 -> buy 1 share
# action=2 -> sell 1 share
## in this testing case we just build a simple random policy
return np.random.randint(3)
10. Trading Gym
● backtesting
agent = YourAgent()
state = env.backtest()
done = False
while not done:
state, reward, done, info = env.step(agent.choice_action(state))
#print(state, reward)
#env.render()
if done:
break
env.transaction_details
11. Trading Gym
● Rule base usage
env = trading_env.make(obs_data_len=10, step_len=1,
df=df, fee=0.1, max_position=5, deal_col_name='Price',
feature_names=['Price', 'MA'],
fluc_div=100.0)
class MaAgent:
def __init__(self):
pass
def choice_action(self, state):
if state[-1][0] > state[-1][1] and state[-2][0] <= state[-2][1]:
return 1
elif state[-1][0] < state[-1][1] and state[-2][0] >= state[-2][1]:
return 2
else:
return 0
# then same as above
12. Real-time paper trading
● Interactive Broker API
○ Cross platform
○ Need to open TWS or IB gateway
○ C++, C#, Java, Python, VB …
○ http://interactivebrokers.github.io/#
13. Real-time paper trading
● Official IB python API
○ Requirements : python 3.1 or higher, TWS 952.x or higher
○ Install API for python https://git.io/vHjng
wget http://interactivebrokers.github.io/downloads/twsapi_macunix.973.02.zip
unzip twsapi_macunix.973.02.zip
cd IBJts/source/pythonclient
python setup.py install
Cd ../../smaples/Python/Testbed/
python Program.py -p 4002
14. Real-time paper trading
● ibapi.client.EClient, ibapi.wrapper.EWrapper
from ibapi.client import EClient
from ibapi.wrapper import EWrapper, iswrapper
class TWSClient(EWrapper, EClient):
def __init__(self):
EClient.__init__(self, wrapper=self)
client = TWSClient()
client.connect("127.0.0.1", 4002, clientId=1)
print("serverVersion:%s connectionTime:%s" % (client.serverVersion(),
client.twsConnectionTime()))
16. Real-time paper trading
● Order
○ Market
○ Market If Touched
○ Market On Close, Open
○ Limit Order
○ Limit if Touched
○ https://interactivebrokers.github.io/tws-api/basic_orders.html#gsc.tab=
0
17. Real-time paper trading
● Market Data
○ Live Market
○ L1
■ Stocks, Futures and others , Frequency : 250 ms
■ US Options Frequency : 100 ms
○ L2
○ Historical Data
○ Real Time Bars