Introduction to trader bots with Python

464 views

Published on

Python is a great tool to implement simple trader bots based on HTTP and JSON. This talk gives an overview on the communication between bot and the exchange platform, what a basic bot loop can look like and how to test it using a scenario mock up.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
464
On SlideShare
0
From Embeds
0
Number of Embeds
24
Actions
Shares
0
Downloads
7
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Introduction to trader bots with Python

  1. 1. Introduction to trader bots with Python Thomas Aglassinger http://roskakori.at @Taglassinger https://github.com/roskakori/talks/tree/master/pygraz/traderbot
  2. 2. Overview ● Outline a simple trading bot ● Example usages for several Python modules (most part of the standard library) ● (Probably) Python beginner friendly
  3. 3. Agenda ● Communication and configuration ● Trading ● Data analysis and debugging ● Testing
  4. 4. Limitations ● You won't get rich (but it's fun nevertheless) ● Code does not work anymore due API update ● Masterexchange is going to shut down soon ● Terms of service
  5. 5. Communication and Configuration
  6. 6. Overview ● Uses https://masterxchange.com/api.php (defunct after 2015-11-15) ● Communicate using HTTPS and JSON ● Public queries available to anyone (e.g. current bids) ● Private queries requiring a personal token bound to you account (e.g. orders)
  7. 7. Query trades ● Query the last 500 trades for maidsafe coins: https://masterxchange.com/api/v2/trades.php?currency=maid ● Result: [{"tradeid":"31242","price":"0.00990000","amount":"0.151 10800","date":"1446399439","market":"msc_btc"}, {"tradeid":"31241","price":"0.00990000","amount":"0.099 89200","date":"1446319270","market":"msc_btc"}, {"tradeid":"31240","price":"0.00562223","amount":"0.037 79028","date":"1446309947","market":"msc_btc"}, ...]
  8. 8. Print the top 3 trades import json import requests def _without_utf8_bom(text): return text[3:] if text.startswith('xefxbbxbf') else text query = requests.get( 'https://masterxchange.com/api/v2/trades.php', headers={'User-Agent': 'demobot/0.1'}, params={'currency': 'maid'} ) print('query.status_code =', query.status_code) if query.status_code < 300: query_text = _without_utf8_bom(query.text) print('query_text = %r...' % query_text[:40]) trades = json.loads(query_text) print('trades =', trades[:3])
  9. 9. Print the top 3 trades - result query.status_code = 200 query_text = '[{"tradeid":"31246","price":"0.00002603"'... trades = [{'market': 'maid_btc', 'date': '1446500342', 'amount': '7000.00000000', 'price': '0.00002603', 'tradeid': '31246'}, {'market': 'maid_btc', 'date': '1446489311', 'amount': '22000.00000000', 'price': '0.00002655', 'tradeid': '31244'}, {'market': 'maid_btc', 'date': '1446462486', 'amount': '1250.00000000', 'price': '0.00002655', 'tradeid': '31243'}]
  10. 10. Configuring the API key ● Private queries require an API key. ● Simple way to manage: configparser ● Store key in a config file ● Read it during startup
  11. 11. Example config file [demobot] api_key = ou1IurT4HQrFfN1ch...
  12. 12. Read the API key from the config import configparser config = configparser.ConfigParser() config.read('demobot.cfg') api_key = config.get('demobot', 'api_key')
  13. 13. Query your balances https://masterxchange.com/api/v2/private/balance s.php?APIkey=ou1IurT4H... {"balances":{"total": {"btc":"9.16311816","msc":"34.63724456","maid":" 43233.50000000"},"available": {"btc":7.16311816,"msc":26.63724456,"maid":426 33.5}},"error_message":"","error_code":0}
  14. 14. Print your balances query = requests.get( 'https://masterxchange.com/api/v2/private/balances.php' , headers={'User-Agent': 'demobot/0.1'}, params={'APIkey': api_key} ) print('query.status_code =', query.status_code) if query.status_code < 300: query_text = _without_utf8_bom(query.text) print('query_text = %r...' % query_text[:40]) balances_result = json.loads(query_text) if balances_result['error_code'] == 0: balances = balances_result['balances'] print('balances =', balances)
  15. 15. Print your balances - result query.status_code = 200 query_text = '{"balances":{"total": {"btc":0,"msc":0,"m'... balances = {'total': {'xcpjenga': 0, 'colzoidy': 0, 'coltdu': 0, 'xcpopcu': 0, 'colgauto': 0, ...}}
  16. 16. Masterexchange error handling 1.Http status < 300? 2.Query result error_code = 0? 3.Process actual data in query result
  17. 17. Wrap error handling in Exceptions class BotError(Exception): pass class HttpsConnection(object): ... def query(self, function_name, payload=None, use_api_key=True): function_url = 'https://masterxchange.com/api/v2/%s.php' % function_name actual_payload = {} if payload is None else dict(payload) if use_api_key: actual_payload['APIkey'] = self._api_key headers = {'User-Agent': 'demobot/0.1'} r = requests.get(function_url, headers=headers, params=actual_payload) if r.status_code >= 300: raise BotError( 'cannot query %s with %s: HTTP error code=%d' % (function_url, actual_payload, r.status_code)) result = json.loads( r.text[3:] if r.text.startswith('xefxbbxbf') else r.text) if use_api_key and (result['error_code'] != 0): raise BotError( 'cannot query %s with %s: %s' % (function_url, actual_payload, result['error_message'])) return result
  18. 18. Trading
  19. 19. Processing monetary values ● Use decimal instead of float http://floating-point-gui.de/ ● Python has a decimal module: https://docs.python.org/3/library/decimal.html ● json and configparser only support float → convert after reading and before writing ● Bitcoin uses 8 digits after decimal separator
  20. 20. Use formatting for decimals >>> from decimal import Decimal >>> print(Decimal('0.00000001')) 1E-8 >>> print('%.8f' % Decimal('0.00000001')) 0.00000001
  21. 21. Modes of operation ● Advise: only suggest to sell or buy → user has to manually initiate transactions ● Lower risk for “stupid” transactions ● Might miss opportunities due slow reaction time ● Helpful when trying out a hopefully improved trading algorithm ● Action: automatically sell and buy on market conditions deemed favorable ● Can react quickly to changes ● Possibility for epic fail on buggy trading algorithms ● Recommendation: reduce risk (but also opportunities) by limiting amount traded per transaction and hour, stop loss limits etc.
  22. 22. Basic bot loop 1.Update own balances 2.Update open orders on the market 3.Apply trading algorithm and decide next action 4.Possibly buy or sell 5.Wait some time 6.Repeat
  23. 23. Trading algorithms On the long run, nothing really works
  24. 24. Some simple trading algorithms ● Spread between 2 different but interchangeable items; e.g. Team Fortress 2's keys and earbuds: http://icrontic.com/article/tf2-black-market-explained ● Delayed correlation between two items; e.g. stocks for Coca Cola and Pepsi: http://www.investopedia.com/university/guide-pairs-trading/pairs-trading-correlation.asp ● Wait for slips from sellers, buy “unusually” cheap items and resell for “normal” price; article about such a bot (violating terms of service): http://diablo3story.blogspot.com.au/2014/07/a-diablo-3-story.html
  25. 25. Data analysis and Debugging
  26. 26. Tracking statistics ● Collect statistics in database ● To debug bot decisions ● To improve trading algorithm ● To monitor market conditions
  27. 27. Sqlite ● Robust and stable ● Efficient for single client use ● Easy to set up ● Included with Python: https://docs.python.org/3/library/sqlite3.html ● Rather creative type system ● “Real” instead of “decimal” ● “int” for timestamp instead of “datetime” type ● Type anarchy concerning comparison ● http://www.sqlite.org/datatype3.html
  28. 28. Create a statistics database def _create_database(self, database_path): _log.info('connect to database %r', database_path) result = sqlite3.connect(database_path) with closing(result.cursor()) as cursor: cursor.execute(""" create table if not exists balances ( action char(4) not null, balance_time int not null, btc real not null, maid real not null, price_per_maid real not null, transferred int not null ) """) cursor.execute(""" create index if not exists idx_balance_time on balances (balance_time) """) result.commit() return result
  29. 29. Insert a statistics row values_to_insert = ( action, int(time.time()), float(self.btc_balance), float(self.maid_balance), float(price_per_maid), int(maid_transferred), ) with closing(self._database.cursor()) as cursor: cursor.execute(""" insert into balances ( action, balance_time, btc, maid, price_per_maid, transferred ) values (?, ?, ?, ?, ?, ?) """, values_to_insert) self._database.commit()
  30. 30. Logging ● More detailed tracking of trading decisions than database ● But no easy structured analysis ● Use logging Module https://docs.python.org/3/library/logging.html ● Use RotatingFileHandler https://docs.python.org/3/library/logging.handler s.html#rotatingfilehandler
  31. 31. Example logging configuration 1/2 # Logging configuration as described in # <https://docs.python.org/3/howto/logging-cookbook.html>. [loggers] keys=root,demobot [handlers] keys=console,file [formatters] keys=default [logger_root] level=DEBUG handlers=console,file [logger_demobot] level=DEBUG handlers=console,file qualname=demobot propagate=0
  32. 32. Example logging configuration 2/2 [handler_console] class=StreamHandler level=INFO formatter=default args=(sys.stderr,) [handler_file] class=RotatingFileHandler level=DEBUG formatter=default args=('/tmp/demobot.log', mode='a', maxBytes=1000000, backupCount=5, encoding='utf-8') [formatter_default] format=%(asctime)s - %(name)s - %(levelname)s - %(message)s datefmt=
  33. 33. Testing
  34. 34. Testing challenges ● Network communication is slow ● Many sites have limits on transactions per second ● Testing actual orders requires money
  35. 35. Mock-up connections with scenarios ● Scenarios are simple text file containing expected queries and JSON results ● The test case makes a decisions that results in a query, parses the scenarios JSON result, and makes the next decision and query ● Scenarios can be maintained by domain experts
  36. 36. Example scenario file # got maid, no open orders # the bot should create a maid-order for 500maid and 0.10000001btc/maid private/balances { "balances": { "total":{"btc":0.9,"maid":500}, "available" {"btc":0,"maid":0} }, "error_message":"", "error_code":0 } private/openedOrders { "open_orders": [], "error_message":"", "error_code":0 } orderbook [ {"market":"maid_btc","type":"sell","amount":"120.00000000","price":"0.20000000","date_created":"1401077847"}, {"market":"maid_btc","type":"buy","amount":"270.00000000","price":"0.10000000","date_created":"1418566454"} ] private/createOrder
  37. 37. Scenario implementation ● Bot constructor gets a connection ● Class HttpConnection → query() returns JSON from Masterexchange ● Class ScenarioConnection → query() checks that function matches next line in scenario file and if so returns next JSON from it Audience feedback: try Gherkin! https://pypi.python.org/pypi/gherkin3
  38. 38. Summary
  39. 39. Summary ● Python has many useful libraries to quickly implement simple trading bots. ● You probably won't get rich. ● It's fun!

×