Why Does a Team Outperform its Run Differential?
•Download as PPTX, PDF•
2 likes•837 views
My presentation for the 2015 SABR Analytics Conference, entitled Why Does a Team Outperform its Run Differential?
Recommended
More Related Content
Recently uploaded
Recently uploaded (20)
Featured
Featured (20)
Why Does a Team Outperform its Run Differential?
- 1. Why Does a Team Outperform its Run Differential? Greg Ackerman Syracuse University Sabermetrics Club
- 2. SU Sabermetrics Club • SABR Student Group Affiliate • Justin Mattingly • Joey Weinberg • Colby Conetta • Ray Garzia • Mallory Miller • Zack Potter • Marcus Shelmidine • Brandon Love • Matt Tanenbaum • Bryan Kilmeade • Justin Moritz • Stephen Marciello • Kyle O’Connor • Willie Kniesner • Michael Rotondo • Matt Russo • Sam Fortier • Matt Filippi • Isaac Nelson • Zack Albright • John Van Ermen • Colton Smith • Chris Karasinski • Zach Tornabene
- 3. Basic Premise • Explain the Difference between Actual Win Percentage and Expected Win Percentage based on Run Differential (Expected Win Percentage based upon Pythagenpat formula from run differential) • X = ((runs scored + runs against)/games)^.285 • If achieve run differential to possibly put team in playoffs – do not want to squander it • If borderline run differential for playoffs – could be difference in attaining playoff spot • Will focus on 3 key factors that may influence teams outperforming or underperforming their run differential • Performance of Bench • Relief Pitching • Pitching Depth • Part II – Add managerial decisions to the model • Pinch Hitters Used • Defensive Substitutions • Relievers Used • Etc.
- 4. Charts • Average of (Actual Win % - Expected Win %) • Standard Deviation of (Actual Win % - Expected Win %) • Variables calculated from www.baseball-reference.com • Team Examples of Difference in Actual Win % - Expected Win % • San Francisco Giants • St. Louis Cardinals • New York Yankees • Toronto Blue Jays • Colorado Rockies
- 5. -0.02 -0.015 -0.01 -0.005 0 0.005 0.01 0.015 0.02 0.025 Average of Actual Win% - Expected Win% - By Team - 2000-2013
- 6. 0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 Standard Deviation of Actual Win% - Exp. Win% - By Team – 2000-2013
- 7. -0.02 -0.01 0 0.01 0.02 0.03 0.04 0.05 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 San Francisco Giants - Difference in Actual and Expected Win Percentage
- 8. -0.04 -0.03 -0.02 -0.01 0 0.01 0.02 0.03 0.04 0.05 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 St. Louis Cardinals - Difference in Actual and Expected Win Percentage
- 9. -0.04 -0.02 0 0.02 0.04 0.06 0.08 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 New York Yankees - Difference in Actual and Expected Win Percentage
- 10. -0.08 -0.06 -0.04 -0.02 0 0.02 0.04 0.06 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 Toronto Blue Jays – Difference in Actual and Expected Win Percentage
- 11. -0.06 -0.05 -0.04 -0.03 -0.02 -0.01 0 0.01 0.02 0.03 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 Colorado Rockies - Difference in Actual and Expected Win Percentage
- 12. Measures of Bench (Hitters) Performance • OPS+ - On-Base Average plus Slugging Percentage – Adjusted for Park and League • HR – Home Runs • SB – Stolen Bases • CS – Caught Stealing • Calculated from Baseball Reference – using only bench players listed for each team – weighted average based upon plate appearances of each player • Ultimately, only included OPS+ • Other variables did not add statistical value to the regression model beyond OPS+
- 13. Measure of Relief/Depth Pitcher Performance • FIP – Fielding Independent Pitching • ERA+ - Earned Run Average adjusted for ballpark • SO/W – strike out to walk ratio • Calculated as a weighted average based upon innings pitched • Calculated for group of “relievers” noted on Baseball-Reference – includes closer and top 4 used relievers • Calculated for group of “depth” pitchers noted on Baseball Reference- includes pitchers not included in “starters” or “relievers” categories
- 14. Regression Model I • After different regression model incarnations – settled upon the following to illustrate results: • (Actual – Expected Win %)i = α0 +β1 (Bench OPS+) + β2 (Relief variable) + β3 (Pitching Depth variable) + εi
- 15. Regression Results – (Actual Win% - Pyth Win%) on Bench and Relief Pitching Performance I II III Intercept 0.0206 (1.4307) Intercept 0.0071 (0.5750) Intercept -0.0102 (-1.0729) OPS+ - Bench 0.00001 (0.1104) OPS+ - Bench 0.00002 (0.1911) OPS+ - Bench 0.00002 (0.1795) FIP – Relief -0.0044** (-1.9130) ERA – Relief -0.0026 (-1.5358) KBB – Relief 0.0014 (1.3735) FIP – Depth -0.0009 (-0.4956) ERA – Depth 0.0001 (0.0756) KBB – Depth 0.0007 (0.7328)
- 16. Results • Variables have expected signs for bench (hitter) performance, relief pitching, and pitching depth • Only statistically significant result is for relief pitching performance • Specifically – FIP • FIP has a negative and significant impact on (Actual Win Percentage – Exp. Win Percentage) • As FIP increases – has negative impact on dependent variable • More likely to underperform run differential • As FIP decreases – has positive impact on dependent variable • More likely to outperform run differential
- 17. Sample Bench OPS+ Relief FIP Depth FIP top 10% Seasons - Outperform Run Diff 81.56304 3.66215 4.691264 Bottom 10% Seasons - Underperform Run Diff 81.63799 3.981814 4.769715 % Differential Between Samples -0.09% -8.03% -1.64%
- 18. 65 70 75 80 85 90 Average of Bench OPS+
- 19. 0 1 2 3 4 5 6 FIP - Relief and Pitching Depth FIP-Relief FIP-Depth
- 20. Managerial Decisions • For Next Step: Added Managerial Decisions to the Data Set • To measure managerial decisions – used the Bill James Handbook • Attempt to measure the impact of various managerial decisions on the ability to outperform (underperform) a team’s run differential
- 21. Managerial Decision Variables • Pinch Hitters Used • Pinch Runners Used • Defensive Substitutions • Relief Pitchers: Innings Pitched • Stolen Bases Attempted • Sacrifices Attempted • Pitch Outs Ordered
- 22. Regression Model II • (Actual – Expected Win %)i = α0 +β1 (Bench OPS+) + β2 (Relief FIP) + β3 (Pitching Depth FIP) + β4 (Pinch Hitters) + β5 (Pinch Runners) + β6 (Defensive Substitutions) + β7 (Relief Innings) + β8 (SB Attempts) + β9 (SAC Attempts) + β10 (Pitch Outs) + εi
- 23. Variable Coefficient Variable Coefficient Intercept 0.0200 (0.9657) Defensive Substitutions 0.0003*** (3.5602) OPS+ - Bench -0.00003 (-0.2786) Relief Innings Pitched -0.00004 (-1.4258) FIP – Relief -0.0023 (-0.9656) Stolen Bases Attempted 0.000007 (0.3458) FIP – Depth 0.0012 (0.5889) Sacrifices Attempted -0.00002 (-0.2871) Pinch Hitters Used -0.000004 (-0.1621) Pitch Outs Ordered -0.0002** (-2.0751) Pinch Runners Used 0.00004 (0.4072)
- 24. Results • Two statistically significant managerial variables: • Defensive Substitutions – (+) – significant at the 1% level • Pitchouts Ordered – (-) – significant at the 5% level • Defensive Substitutions – more defensive substitutions used – greater likelihood to outperform run differential • Part is managerial decision • Part is roster flexibility • Pitchouts Ordered – more pitchouts ordered – greater likelihood to underperform run differential • Part is wasting a pitch • Part is lack of faith in catcher/pitcher • Likely a proxy for risk averse behavior on part of manager
- 25. Results • Relief Pitcher Innings Pitched – (-) but not quite statistically significant (15% level) • When Managerial Statistics included – impact of FIP-Relievers is lessened as well – no longer statistically significant • Tried including one or the other – not quite statistically significant • Appears to still have some marginal effect on ability to outperform/underperform run differential
- 26. Sample Defensive Substitutions Relievers Used Pitch Outs top 10% Seasons - Outperform Run Diff 36.0732 444.0732 17.2927 Bottom 10% Seasons - Underperform Run Diff 26.1892 458.7222 21.5000 % Differential Between Samples 37.74% -0.03% 19.57%
- 28. 0 5 10 15 20 25 30 35 40 45 50 Average of Defensive Substitutions Used
- 29. 0 5 10 15 20 25 30 35 40 Average of Pitchouts Ordered
- 30. Conclusions • Aimed to determine why teams outperform/underperform run differential • Is it just luck? – or are there factors that contribute to its explanation? • Without Manager Data – it appears that Relief Pitcher Performance (measured by FIP) plays an important role • Increase in FIP by Relievers – more likely to underperform • Decrease in FIP by Relievers – more likely to outperform • With Manager Data • Defensive Substitutions – more defensive subs – more likely to outperform • Pitchouts – likely proxy for risk aversion (poor catching performance?) – more pitchouts – more likely to underperform run differential • Starting point of our research – hope to learn more in future – open to different variables/approaches to help determine answers
Editor's Notes
- We are not looking at starting pitching and starting lineup, rather we are looking at performance of bench, relief pitching, and pitching depth.
- Talk about the Yankees, Giants, and Cardinals having high averages and have been outperforming their run differential between 2000 and 2013, while teams like the Blue Jays, Rockies, and
- One interesting thing here is that the Giants have the lowest standard deviation between 2000 and 2013
- 0 is what we would expect, so the Giants have regularly outperformed their run differential
- Last of team group
- The key to this is that we used a weighted average for each team amongst bench hitters only, according to baseball-reference. We are really only going to be focusing on OPS+ because it is the most promising of the bench statistics.
- This is a weighted average of FIP, ERA+, and SO/BBW. We used
- The dependent variable here is actual – expected win percentage. The independent variables are bench OPS+, and a variable amongst FIP, ERA, or strikeout to walk ratio amongst the top five relievers and depth pitching.
- These are all very small numbers because we are dealing with trying to explain a very small number, which is the difference between actual and expected win percentage.
- This makes sense because if you are not pitching well in late innings, you are likely to underperform your run differential, while if you have good Fielding Independent Pitching amongst relievers, they are more likely to outperform their run differential.
- This slide shows the best seasons and worst seasons in terms of outperforming and overperforming run differential. The only place we have a sizeable difference between the top 10% and bottom 10% of seasons is in relief FIP, as there is a 8.03% dropoff.
- This just gives an idea of the average bench OPS+ amongst the team.
- This is the difference between relief and depth FIP.