Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Mlb replay913pdf

199 views

Published on

Metis Capstone project on when an MLB Manager should challenge an umpire's call

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Mlb replay913pdf

  1. 1. When should an MLB manager challenge? A Strategic Model Michael Laster Metis: September 2016 1
  2. 2. 2
  3. 3. 3
  4. 4. 4
  5. 5. 5
  6. 6. 6
  7. 7. Cespedes throws… 7
  8. 8. Right on the money! 8 but…
  9. 9. 9
  10. 10. Manager Terry Collins challenges the call 10
  11. 11. 11 Background • Starting in 2014, a team can “challenge” an umpire’s call • If video reviewers find the call incorrect, it’s overturned • Otherwise, the team loses the chance to challenge for the rest of the game
  12. 12. Scenario II 12
  13. 13. 13
  14. 14. Now, Should Terry Collins Challenge? Can we make a data-driven decision? 14
  15. 15. Factors to Consider 1) How much of an advantage will a successful challenge give them? Without overturning: 50.0% chance of winning If overturned: 52.2% chance of winning15
  16. 16. Factors to Consider 2) How much would it “cost” the team if the challenge is lost for the rest of the game? Opportunity Cost ≈ .03% / out remaining In our case: 1.37% 16
  17. 17. Upside: 2.2% gain in Win Expectancy Downside: 1.37% loss in Win Expectancy 17
  18. 18. Factors to Consider 3) Probability of Call Being Overturned • Combined data from baseballsavant.mlb.com and retrosheets.org covering around 3,000 replays from April 2014-July 2016 • Approximated set of non-challenged plays •Used SVM and linear regression models to generate probabilistic estimates • Features: challenging team, umpire, type of play, day of week, inning • Best models: .613 and .621 log-loss 18
  19. 19. Probability 19 74.4%
  20. 20. Putting it all Together Upside = (p)(Win Expectancy Added) Downside = (1-p)(Opportunity Cost) 20
  21. 21. Putting it all Together Upside = 1.6% Downside = .35% 21
  22. 22. Challenge it! 22
  23. 23. Conclusions •Challenge calls aggressively! •This hinges on a proper estimate of “opportunity cost” • The consensus is that the cost is very low • “There's not much cost to consider.” -Tom Tango (email correspondence) 23
  24. 24. Check out the code: 24 Github writeup
  25. 25. Thank you! Michael Laster michael.laster@alum.yu.edu LinkedIN: michael-laster 25

×