1. How to Tell !
if Your Market Timing System will Work
John Elder, Ph.D.
elder@datamininglab.com
@johnelder4
A New Measure of Model Quality
Charlo'esville, VA
Washington, DC
Bal6more, MD
Raleigh, NC
434-973-7673
www.elderresearch.com
2. Outline
• How likely is a result luck (vs. skill)?
• Sharpe measures the quality of returns
• DAPY measures of the quality of a timing model
• Why timing almost always doesn’t work
• Blackjack is good practice
3. 1. You should have received a BDA !
(binary decision artifact) on entry
2. Please flip (and catch) it 10 times
3. Report the #heads
4. Retain your BDA for future executive decision-
making needs
We’re Looking for Skilled Heads Flippers!
4. Find Best Flipper of “Heads” (Active Manager)
709.4
64.2
12.3
3.7
1.5
0.7
0.4
0.2
0.2
0.10.1
1.0
10.0
100.0
1000.0
1 2 3 4 5 6 7 8 9 10
Goal: Minimum #Heads (of 10)
#MonkeystoInterviewforp>.5
• Distribution of active managers vs.
market is ~ random with fat tails
(extremes) and negative mean (fees)
• If 8+ (of 10) heads is acceptable,
expect to interview 13 flippers
• 16 flippers -> 60% probability of 8+
or 95% of 7+
• Multiple targets: What if last 5 are
heads, extraordinary tail history, etc.
• Flippers can now take their record
with them. (Trainer still keeps it too.)
• (And which ones advertise?)
(1-p)m = 0.5
p = 2-10 Σ( )10
k
10
k=goal
is probability of hiring
after history of 10 flips
6. Sharpe Ra6o
(RT - R0) /σT
RT = Trading return (annualized)
R0 = Risk-free return (e.g., T-bills), annualized
σT = standard deviation of trading system (ann.)
Conventional metric by which to score returns:
9. Days Ahead per Year
DAPY = (RT - IT*RB) /ΔB
RT
= Trading return (annualized)
IT
= % of time strategy is invested
RB = Buy & hold return (annualized)
ΔB = average absolute daily return of B&H
A new metric by which to score timing strategies:
14. Can Timing Work?
For most investors, it never has [see next slide]
There are 3 extreme challenges:
1. Skill is hard to distinguish from luck [coin flip ex.]
2. Markets are nearly efficient [huge ensembles]
3. One must constantly complete failed bets!
(which wars with our cognitive bias against losses)
16. More Investment Modeling Challenges
• “Pockets of inefficiency” are trampled if found
• Historical state is hard to recreate accurately in data
[survivor bias, Bloomberg earnings reports]
• The system can change overnight [LTCM+Russia]
• Correlation increases during panics [killing hedges]
17. Best Investment Timing System Analogy is!
Blackjack
Expected losses are (only) -0.5% if playing
perfect Basic Strategy, but can grow to !
gains of 0.5% to 1.5% if able to count cards!
-- but with huge swings (risk)
18. John F. Elder IV
Founder & Chair, Elder Research, Inc.
Dr. John Elder heads the USA’s most experienced data mining consulting
team. Founded in 1995, Elder Research, Inc. has offices in Charlottesville,
VA, Washington DC, Baltimore MD, and Raleigh NC (www.datamininglab.com).
ERI focuses on Federal, commercial, and investment applications of
advanced analytics, including text mining, credit scoring, process
optimization, cross-selling, drug efficacy, market timing, and fraud
detection.
18
John earned Electrical Engineering degrees from Rice University, and a PhD in Systems Engineering
from the University of Virginia, where he’s been an adjunct professor teaching Optimization or Data
Mining. Prior to 22 years at ERI, he spent 5 years in aerospace defense consulting, 4 heading research
at an investment management firm, and 2 in Rice's Computational & Applied Mathematics department.
Dr. Elder has authored innovative data mining tools, is a frequent keynote speaker, and has chaired
International Analytics conferences. John was honored to serve for 5 years on a panel appointed by
President Bush to guide technology for National Security. His book with Bob Nisbet and Gary Miner,
Handbook of Statistical Analysis & Data Mining Applications, won the PROSE award for top book in
Mathematics for 2009. His book with Giovanni Seni, Ensemble Methods in Data Mining, was published in
2010, and his book with colleague Andrew Fast and 4 others on Practical Text Mining won the 2012
PROSE award for Computer Science.
John is grateful to be a follower of Christ and father of 5.
19. Modigliani2
M2 = (RT - RB)*(σB/σT)
RT = Trading return (annualized)
RB = Buy & hold return (annualized)
σT = standard deviation of trading system (ann.)
σB = standard deviation of buy & hold (ann.)
Another metric (equivalent to Sharpe for a given market) !
first matches risk, then compares return:
Franco Modigliani & Leah Modigliani, “Risk-Adjusted Performance”
Journal of Porlolio Management 23, no 2, (Winter 1997): 45-54.
