No-regret learning is a collection of tools designed to give provable performance guarantees in the absence of any statistical or other assumptions on the data (!), and thus stands in stark contrast to most classical modeling approaches. With origins stretching back to the 1950s, the field has yielded a rich body of algorithms and analyses that covers problems ranging from forecasting from expert advice to online convex optimization. Dr. Kearns will survey the field, with special emphasis on applications to quantitative finance problems, including portfolio construction and inventory risk.

1. Trading Without Regret*
Michael Kearns
University of Pennsylvania
Chief Scientist, MANA Partners
QuantCon
April 29, 2017
There ain’t nothing like regret
To remind you’re alive…
Sheryl Crow
That’s how you got me
You taught me to regret
Fiona Apple

2. “No-Regret” Learning
• Have a set of N “signals” or “predictors”
– returns of stocks or funds, portfolios, strategies, alphas, advisors, sentiment, news…
• Each trading period, each signal receives an arbitrary payoff
– no stochastic assumptions; could be generated by omniscient Nature/adversary
– signals could have (in)side information, expertise, specialization, clairvoyance, etc.
– will only assume payoffs are bounded (already present in most stochastic models)
• Algorithm maintains dynamic weighting/portfolio over signals
– receives weighted payoff each period
• Goal: over T periods, payoff close to the best signal in hindsight
– for any sequence of signal payoffs
– “no regret”: (best payoff – algo payoff) grows sublinearly in T (0 per period)
– not competing with optimal unconstrained (switching) policy
• Contrast with boosting criterion
– trying to track best, not to beat best
– “needle in a haystack” vs. “wisdom of crowds”
• Obvious appeal in financial settings
– nonstationarity; adversarial/strategic/game-theoretic behaviors

3. [Arora, Hazan, Kale]
[Blum & Mansour]

4. Remarks
• No-regret algos and analyses have long and rich history
– 1950s: Blackwell approachability
– large and current ML literature
– modern connections to Black-Scholes (non-stochastic derivation), prediction markets
• Strong connections to game theory
– minimax theorem and linear programming
– convergence to Nash and correlated equilibrium
• Demystification #1: “Follow the Leader” has regret ~ # of lead changes
• Demystification #2: log(N) regret term means cannot try “everything”
– e.g. can’t predict sequence of T coin flips by adding all 2T possible signals
• Under stochastic assumptions, often recover (near) optimal solutions

7. [Helmbold, Schapire, Singer, Warmuth]

8. *Unfortunately…
• Large N and “sideways” markets create serious challenges
• Main issue: multiplicative updates lead to portfolio concentration
• Additive loss functions  lack of risk considerations
• Fiddling with learning rate doesn’t help
S&P500, 2005-2011

9. Even Worse…
• Could ask for no regret to best strategy in risk-adjusted metrics:
– Sharpe ratio: μ(returns)/σ(returns)
– mean-variance: μ(returns) - σ(returns)
• Strong negative results:
– no-regret provably impossible
– lower bounds on competitive ratio for any algorithm
• Intuition: volatility introduces switching costs
• Alternative approach:
– measure risk by typical loss (e.g. one standard deviation)
– internalize risk within strategies

10. No-Regret Under Risk Constraints
• Can’t control Sharpe Ratio, but can impose risk limits
• E.g. restrict to portfolios/strategies with daily PNL std at most $X historically
• Leads to elliptical constraint in portfolio space; eccentricity ~ correlations
• Now only compete with strategies:
– obeying risk limits
– making only “local” moves in portfolio space (limit market impact)
• Combine no-regret with pursuit-evasion to recover (theoretical) guarantees
[Dworkin, K., Nevmyvaka]

11. Variations and Generalizations
• Sublinear regret to best and only constant regret to an index
– e.g. track best single stock but never do (much) worse than S&P 500
• Bandit versions
– only receive feedback/payoff of chosen signal/strategy
– e.g. market impact of live trading
• General online convex optimization
– continuous, high-dimensional action space (convex and compact)
– arbitrary convex function each day
– again compete with best fixed action in hindsight

12. Conclusions
• No-regret learning: rich history, powerful theory
• Deviations from additive loss (e.g. risk) present difficulties
• One workaround: endogenize risk
• alpha generation vs. parametric strategy optimization
The Multiplicative Weights Update Method: A Meta-Algorithm and its Applications.
[Arora, Hazan, Kale, 2012]
Learning, Regret Minimization, and Equilibria.
[Blum and Mansour, 2007]
Pursuit-Evasion Without Regret, with an Application to Trading.
[Dworkin, K., Nevmyvaka, 2014]
Regret to the Best vs. Regret to the Average.
[Even-Dar, K., Mansour, Wortman 2007]
Risk-Sensitive Online Learning.
[Even-Dar, K., Wortman 2006]
Contact: mkearns@cis.upenn.edu