Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Algorithmic Trading: Statistical Significance

555 views

Published on

Dr. Antony Jackson is lecturer in Financial Economics in the School of Economics at University of East Anglia. He talks about statistical significance in algorithmic trading. Antony is an active researcher of algorithmic trading strategies and finished 2nd in Quantiacs' recent algorithmic trading competition with a $750,000 investment. You can find the example code here: https://github.com/Quantiacs/StatisticalSignificance

Published in: Economy & Finance
  • Be the first to comment

Algorithmic Trading: Statistical Significance

  1. 1. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Algorithmic Trading: Statistical Significance Antony Jackson, CFA University of East Anglia London Quant Club, July 2016 Antony Jackson, CFA London Quant Club 1/ 19
  2. 2. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Overview Economic Significance versus Statistical Significance. Antony Jackson, CFA London Quant Club 2/ 19
  3. 3. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Overview Economic Significance versus Statistical Significance. What is a trading strategy? Binary Signals. Antony Jackson, CFA London Quant Club 2/ 19
  4. 4. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Overview Economic Significance versus Statistical Significance. What is a trading strategy? Binary Signals. 100-day ES momentum example. Antony Jackson, CFA London Quant Club 2/ 19
  5. 5. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Overview Economic Significance versus Statistical Significance. What is a trading strategy? Binary Signals. 100-day ES momentum example. Statistical Significance. Antony Jackson, CFA London Quant Club 2/ 19
  6. 6. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Economic Significance versus Statistical Significance Economic Significance can be thought of as the degree to which a trading strategy satisfies the goals of an investor. Antony Jackson, CFA London Quant Club 3/ 19
  7. 7. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Economic Significance versus Statistical Significance Economic Significance can be thought of as the degree to which a trading strategy satisfies the goals of an investor. What impact does the strategy have on the investor’s portfolio? Do its returns have a low correlation (diversification) and high stand-alone Sharpe Ratio (risk-adjusted returns)? Antony Jackson, CFA London Quant Club 3/ 19
  8. 8. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Economic Significance versus Statistical Significance Economic Significance can be thought of as the degree to which a trading strategy satisfies the goals of an investor. What impact does the strategy have on the investor’s portfolio? Do its returns have a low correlation (diversification) and high stand-alone Sharpe Ratio (risk-adjusted returns)? Economic Significance rule-of-thumb: Antony Jackson, CFA London Quant Club 3/ 19
  9. 9. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Economic Significance versus Statistical Significance Economic Significance can be thought of as the degree to which a trading strategy satisfies the goals of an investor. What impact does the strategy have on the investor’s portfolio? Do its returns have a low correlation (diversification) and high stand-alone Sharpe Ratio (risk-adjusted returns)? Economic Significance rule-of-thumb: Accept strategy into existing portfolio if Antony Jackson, CFA London Quant Club 3/ 19
  10. 10. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Economic Significance versus Statistical Significance Economic Significance can be thought of as the degree to which a trading strategy satisfies the goals of an investor. What impact does the strategy have on the investor’s portfolio? Do its returns have a low correlation (diversification) and high stand-alone Sharpe Ratio (risk-adjusted returns)? Economic Significance rule-of-thumb: Accept strategy into existing portfolio if Stand-alone SR > Existing portfolio SR × correlation Antony Jackson, CFA London Quant Club 3/ 19
  11. 11. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Economic Significance versus Statistical Significance Statistical significance is concerned with how likely the results of our trading strategy could have been generated by pure chance. Antony Jackson, CFA London Quant Club 4/ 19
  12. 12. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Economic Significance versus Statistical Significance Statistical significance is concerned with how likely the results of our trading strategy could have been generated by pure chance. If you have enough monkeys and typewriters, how likely is it that one of them writes the complete works of William Shakespeare? Antony Jackson, CFA London Quant Club 4/ 19
  13. 13. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Economic Significance versus Statistical Significance Statistical significance is concerned with how likely the results of our trading strategy could have been generated by pure chance. If you have enough monkeys and typewriters, how likely is it that one of them writes the complete works of William Shakespeare? It’s possible to have statistical significance without economic significance. For example, you design a strategy with a Sharpe Ratio of 0.05, and you are 99% sure this is significantly larger than zero. Antony Jackson, CFA London Quant Club 4/ 19
  14. 14. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Economic Significance versus Statistical Significance Statistical significance is concerned with how likely the results of our trading strategy could have been generated by pure chance. If you have enough monkeys and typewriters, how likely is it that one of them writes the complete works of William Shakespeare? It’s possible to have statistical significance without economic significance. For example, you design a strategy with a Sharpe Ratio of 0.05, and you are 99% sure this is significantly larger than zero. Equally, we can have economic significance without statistical significance. For example, a strategy with a Sharpe Ratio of 0.40, but where the buy and hold Sharpe Ratio is 0.50. Antony Jackson, CFA London Quant Club 4/ 19
  15. 15. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts What is a Trading Strategy? Binary Signals. I recommend starting algorithmic trading by coding up strategies with simple signals; e.g., 1 for buy and 0 for out of the market. Antony Jackson, CFA London Quant Club 5/ 19
  16. 16. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts What is a Trading Strategy? Binary Signals. I recommend starting algorithmic trading by coding up strategies with simple signals; e.g., 1 for buy and 0 for out of the market. Traditional technical trading systems can be coded up like this. For example, simple moving averages, channel breakouts, filter rules. Antony Jackson, CFA London Quant Club 5/ 19
  17. 17. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts What is a Trading Strategy? Binary Signals. I recommend starting algorithmic trading by coding up strategies with simple signals; e.g., 1 for buy and 0 for out of the market. Traditional technical trading systems can be coded up like this. For example, simple moving averages, channel breakouts, filter rules. Simplifying strategies to a series of ones and zeros helps a lot in establishing the likelihood of a trading strategy being statistically significant, because pure chance strategies can be easily constructed using coin tossing to generate signals. Antony Jackson, CFA London Quant Club 5/ 19
  18. 18. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example There’s a lot of evidence in the academic literature of momentum in asset prices. Antony Jackson, CFA London Quant Club 6/ 19
  19. 19. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example There’s a lot of evidence in the academic literature of momentum in asset prices. Cross-sectional momentum involves buying a portfolio of recent winners and funding the purchase by selling a portfolio of recent losers. Antony Jackson, CFA London Quant Club 6/ 19
  20. 20. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example There’s a lot of evidence in the academic literature of momentum in asset prices. Cross-sectional momentum involves buying a portfolio of recent winners and funding the purchase by selling a portfolio of recent losers. Time-series momentum looks at an asset in isolation. We buy if the asset has gone up, and sell if it has gone down. This is the simple (but effective!) method I’ll consider today. Antony Jackson, CFA London Quant Club 6/ 19
  21. 21. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example Preliminaries: The Matlab scripts for this presentation make use of the Quantiacs Toolbox. Antony Jackson, CFA London Quant Club 7/ 19
  22. 22. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example Preliminaries: The Matlab scripts for this presentation make use of the Quantiacs Toolbox. You will need to open Matlab and place the files from the presentation in the same directory as the Toolbox files such as runts and optimize. Antony Jackson, CFA London Quant Club 7/ 19
  23. 23. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example Preliminaries: The Matlab scripts for this presentation make use of the Quantiacs Toolbox. You will need to open Matlab and place the files from the presentation in the same directory as the Toolbox files such as runts and optimize. The files ESbootstrap.m and ESmomentum.m are Quantiacs trading system files, and the file ESbootstrapCaller.m is a script file used to process the results of random trading strategies. Antony Jackson, CFA London Quant Club 7/ 19
  24. 24. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example The ESbootstrap.m file contains the following code: Antony Jackson, CFA London Quant Club 8/ 19
  25. 25. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example The ESbootstrap.m file contains the following code: function [p, settings] = ESbootstrap(CLOSE, settings) settings.markets = {’F ES’}; settings.lookback = 260; setting.slippage = 0; trial = 1; %#[1:1:1000]# p = 1; % uncomment this to see the performance of buy and hold p = round(rand()); % uncomment this to generate random strategies end Antony Jackson, CFA London Quant Club 8/ 19
  26. 26. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example An important line is Antony Jackson, CFA London Quant Club 9/ 19
  27. 27. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example An important line is settings.slippage = 0; Antony Jackson, CFA London Quant Club 9/ 19
  28. 28. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example An important line is settings.slippage = 0; At this stage we are trying to identify a property of the time series of prices that offers an opportunity, either in terms of identifying a market anomaly or in terms of harvesting a risk premium. Antony Jackson, CFA London Quant Club 9/ 19
  29. 29. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example An important line is settings.slippage = 0; At this stage we are trying to identify a property of the time series of prices that offers an opportunity, either in terms of identifying a market anomaly or in terms of harvesting a risk premium. We don’t want to distort the results at this stage with transaction costs. Antony Jackson, CFA London Quant Club 9/ 19
  30. 30. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example If we uncomment p = 1;, we simulate a buy-and-hold strategy for ES. Antony Jackson, CFA London Quant Club 10/ 19
  31. 31. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example If we uncomment p = 1;, we simulate a buy-and-hold strategy for ES. Enter the command runts(’ESbootstrap’) within the Quantiacs Toolbox, and we obtain the following chart for the buy and hold strategy: Antony Jackson, CFA London Quant Club 10/ 19
  32. 32. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example 1000000 1221663 1492461 1823285 2227440 2721182 3324368 4061258 4961490 996368 4943467 Performance(log) Factsheet of fundEquity in ESbootstrap 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 0 0.2 0.4 0.6 0.8 1 Long/Short Max Drawdown Max Time Off Peak Long Short Your Income Estimation ($ 1Mio Investment, 10% performance fee) total $ per anno $ 324408.45 12747.24 Performance Numbers 0.3474 0.5555 0.0640 0.1843 0.6292 0.1018 3352 Sharpe Ratio Sortino Ratio Performance (prc/y) Volatility (prc/y) Maximum Drawdown MAR Ratio Max Time off peak Antony Jackson, CFA London Quant Club 11/ 19
  33. 33. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example The buy and hold strategy has a Sharpe Ratio of 0.3474. Antony Jackson, CFA London Quant Club 12/ 19
  34. 34. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example The buy and hold strategy has a Sharpe Ratio of 0.3474. Let’s now have a look at the code for a 100-day momentum strategy, which is contained in the file ESmomentum.m Antony Jackson, CFA London Quant Club 12/ 19
  35. 35. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example The buy and hold strategy has a Sharpe Ratio of 0.3474. Let’s now have a look at the code for a 100-day momentum strategy, which is contained in the file ESmomentum.m function [p, settings] = ESmomentum(CLOSE, settings) settings.markets = {’F ES’}; setting.lookback = 260; setting.slippage = 0; p = CLOSE(end) > CLOSE(end - 100); end Antony Jackson, CFA London Quant Club 12/ 19
  36. 36. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example 1000000 1158838 1342905 1556209 1803394 2089841 2421787 2806458 3252230 999574 3250845 Performance(log) Factsheet of fundEquity in ESmomentum 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 0 0.2 0.4 0.6 0.8 1 Long/Short Max Drawdown Max Time Off Peak Long Short Your Income Estimation ($ 1Mio Investment, 10% performance fee) total $ per anno $ 196232.84 7710.73 Performance Numbers 0.3920 0.5974 0.0439 0.1119 0.2584 0.1697 2428 Sharpe Ratio Sortino Ratio Performance (prc/y) Volatility (prc/y) Maximum Drawdown MAR Ratio Max Time off peak Antony Jackson, CFA London Quant Club 13/ 19
  37. 37. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example Not bad. The Sharpe Ratio has been boosted from 0.3474 to 0.3920. Antony Jackson, CFA London Quant Club 14/ 19
  38. 38. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example Not bad. The Sharpe Ratio has been boosted from 0.3474 to 0.3920. So at this stage, we could argue that we have discovered an economically significant trading strategy, because we have boosted the Sharpe Ratio and we’ve come up with a way of staying out of the worst of the financial crisis, without cheating. Antony Jackson, CFA London Quant Club 14/ 19
  39. 39. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts 100-day ES momentum example Not bad. The Sharpe Ratio has been boosted from 0.3474 to 0.3920. So at this stage, we could argue that we have discovered an economically significant trading strategy, because we have boosted the Sharpe Ratio and we’ve come up with a way of staying out of the worst of the financial crisis, without cheating. The next step is to quantify the probability that a “pure chance” strategy could generate a similar (or better) Sharpe ratio. Antony Jackson, CFA London Quant Club 14/ 19
  40. 40. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Statistical Significance In the ESbootstrap.m file, comment out the p = 1; line, and uncomment the p = round(rand()); line. Antony Jackson, CFA London Quant Club 15/ 19
  41. 41. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Statistical Significance In the ESbootstrap.m file, comment out the p = 1; line, and uncomment the p = round(rand()); line. The mysterious %#[1:1:1000]# line is there so that the optimize function in the Quantiacs Toolbox runs a random trading system 1000 times, collecting the Sharpe Ratio each time. Antony Jackson, CFA London Quant Club 15/ 19
  42. 42. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Statistical Significance In the ESbootstrap.m file, comment out the p = 1; line, and uncomment the p = round(rand()); line. The mysterious %#[1:1:1000]# line is there so that the optimize function in the Quantiacs Toolbox runs a random trading system 1000 times, collecting the Sharpe Ratio each time. Essentially, we are going to generate 1000 strategies by tossing a coin every day to decide whether we are long or out of the market. Antony Jackson, CFA London Quant Club 15/ 19
  43. 43. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Statistical Significance The ESbootstrapCaller.m script contains the following code: Antony Jackson, CFA London Quant Club 16/ 19
  44. 44. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Statistical Significance The ESbootstrapCaller.m script contains the following code: % Script to call the ESbootstrap function out = optimize(’ESbootstrap’); hist(out.sharpe) title(’Histogram of Trading Strategy Sharpe Ratios’) Antony Jackson, CFA London Quant Club 16/ 19
  45. 45. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Statistical Significance The ESbootstrapCaller.m script contains the following code: % Script to call the ESbootstrap function out = optimize(’ESbootstrap’); hist(out.sharpe) title(’Histogram of Trading Strategy Sharpe Ratios’) Enter run ESbootstrapCaller at the Matlab command line, and we obtain a histogram something like this: Antony Jackson, CFA London Quant Club 16/ 19
  46. 46. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Statistical Significance −0.2 0 0.2 0.4 0.6 0.8 0 50 100 150 200 250 300 Histogram of Trading Strategy Sharpe Ratios Antony Jackson, CFA London Quant Club 17/ 19
  47. 47. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Statistical Significance Finally we add the following line to the ESbootstrapCaller script to work out the proportion of random strategies that obtain a Sharpe Ratio greater than our “real-life” strategy Sharpe Ratio of 0.3920: Antony Jackson, CFA London Quant Club 18/ 19
  48. 48. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Statistical Significance Finally we add the following line to the ESbootstrapCaller script to work out the proportion of random strategies that obtain a Sharpe Ratio greater than our “real-life” strategy Sharpe Ratio of 0.3920: pValue = sum(out.sharpe(:) > 0.3920)/1000; Antony Jackson, CFA London Quant Club 18/ 19
  49. 49. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Statistical Significance Finally we add the following line to the ESbootstrapCaller script to work out the proportion of random strategies that obtain a Sharpe Ratio greater than our “real-life” strategy Sharpe Ratio of 0.3920: pValue = sum(out.sharpe(:) > 0.3920)/1000; For this particular sequence of coin tosses, we obtain a p-value of 0.1530. Antony Jackson, CFA London Quant Club 18/ 19
  50. 50. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Statistical Significance Finally we add the following line to the ESbootstrapCaller script to work out the proportion of random strategies that obtain a Sharpe Ratio greater than our “real-life” strategy Sharpe Ratio of 0.3920: pValue = sum(out.sharpe(:) > 0.3920)/1000; For this particular sequence of coin tosses, we obtain a p-value of 0.1530. We prefer low p-values, as it’s then less likely that our results could be generated by pure chance. Antony Jackson, CFA London Quant Club 18/ 19
  51. 51. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Closing Thoughts The methods I’ve introduced today can be extended to testing multiple rules on the same time series. Antony Jackson, CFA London Quant Club 19/ 19
  52. 52. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Closing Thoughts The methods I’ve introduced today can be extended to testing multiple rules on the same time series. For instance, if we decide to test both a 100-day and a 250-day momentum rule, we need to take into account that we are now testing multiple strategies, and so the bootstrap methodology needs to be adjusted accordingly. Antony Jackson, CFA London Quant Club 19/ 19
  53. 53. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Closing Thoughts The methods I’ve introduced today can be extended to testing multiple rules on the same time series. For instance, if we decide to test both a 100-day and a 250-day momentum rule, we need to take into account that we are now testing multiple strategies, and so the bootstrap methodology needs to be adjusted accordingly. In this case, for each trial, we would run two random strategies and pick the highest Sharpe Ratio each time. Antony Jackson, CFA London Quant Club 19/ 19
  54. 54. Introduction ES 100-day momentum example Statistical Significance Closing Thoughts Closing Thoughts The methods I’ve introduced today can be extended to testing multiple rules on the same time series. For instance, if we decide to test both a 100-day and a 250-day momentum rule, we need to take into account that we are now testing multiple strategies, and so the bootstrap methodology needs to be adjusted accordingly. In this case, for each trial, we would run two random strategies and pick the highest Sharpe Ratio each time. We balance the increased likelihood of obtaining a good Sharpe Ratio by testing multiple models against the rightward shift in the distribution of random Sharpe Ratios that results from picking the best of multiple random strategies each time. Antony Jackson, CFA London Quant Club 19/ 19

×