Making Multimillion-Dollar Baseball Decisions with H2O AutoML, LIME and Shiny (Rome Edition)
•
3 likes•630 views
Moneyball (Rome Edition)
Recommended
Recommended
More Related Content
Similar to Making Multimillion-Dollar Baseball Decisions with H2O AutoML, LIME and Shiny (Rome Edition)
Similar to Making Multimillion-Dollar Baseball Decisions with H2O AutoML, LIME and Shiny (Rome Edition) (20)
More from Sri Ambati
More from Sri Ambati (20)
Recently uploaded
Recently uploaded (20)
Making Multimillion-Dollar Baseball Decisions with H2O AutoML, LIME and Shiny (Rome Edition)
- 2. + + In case you’re wondering… final project result $20Mtrade 2 weeks prior to the season beginning
- 3. About this talk • Quick Overview • Business problem • Solution and result • Details • The “Moneyball” team • Baseball data → ML problem • H2O AutoML, LIME, Shiny You will learn … • How to frame a business problem (e.g. Moneyball) for machine learning. • If we have time … • How to use H2O AutoML with R interface. • How to use LIME to explain H2O models. 3
- 4. About Me 4 • Before H2O • Water Engineer / EngD Researcher / Matlab Fan Boy (wonder why @matlabulous?) • Discovered R, Python, H2O … never look back again • Data Scientist at Virgin Media (UK), Domino Data Lab (US) • At H2O … • Data Scientist / Evangelist / • Sales Engineer / Solution Architect / • Community Manager … The harsh reality of startup life … • H2O SWAG Photographer #AroundTheWorldWithH2Oai Love H2O? Get some stickers! Barcelona Berlin Brussels Paris Milan (2016) Milan (2018)
- 8. Founded 2012, Series C in Nov, 2017 Products • Driverless AI – Automated Machine Learning • H2O Open Source Machine Learning • Sparkling Water Mission Democratize AI. Do Good Team ~100 employees • Distributed Systems Engineers doing Machine Learning • World-class visualization designers Offices Mountain View, London, Prague 8 Company Overview
- 9. H2O Products In-Memory, Distributed Machine Learning Algorithms with H2O Flow GUI H2O AI Open Source Engine Integration with Spark Lightning Fast machine learning on GPUs Automatic feature engineering, machine learning and interpretability Secure multi-tenant H2O clusters
- 10. CONFIDENTIAL CONFIDENTIAL Worldwide Recognition in the H2O.ai Community Financial InsuranceMarketingHW Vendors Retail Advisory & Accounting Healthcare “H2O.ai's reference customers gave it the highest overall score for sales relationship and overall service and support” - Gartner MQ 2018 Open Source Community Paying Customers
- 11. Gartner names H2O as Leader with the most completeness of vision • H2O.ai recognized as a technology leader with most completeness of vision • H2O.ai was recognized for the mindshare, partner network and status as a quasi-industry standard for machine learning and AI. • H2O customers gave the highest overall score among all the vendors for sales relationship and account management, customer support (onboarding, troubleshooting, etc.) and overall service and support.
- 12. Platforms with H2O integration H2O + KNIME Talk at KNIME Summit Mar 2017
- 14. The Real Business Problem in Major League Baseball (MLB) 14 • Existing Forecasts (e.g. ESPN) are usually projections for the next year only. • MLB players usually consider terms for 3 to 5 years when they sign a new contract. • MLB teams need to consider players’ long-term performance (i.e. > 1 year)
- 16. Our Solution 16 • Open data – Lahman Database. • Proprietary data from Ari Kaplan – our real Moneyball guy. • Enriched Lahman data with Ari’s Data • Framed data for machine learning. • Used H2O AutoML to make predictions for next three years.
- 17. From Toy Demo to Real Moneyball • March 19 – AutoML Predictions finalized. Initial presentation in Excel. • March 20 – Version 1 of Shiny app. Ari used the app to validate some players he had in mind and recommended one player to his team. • March 21 – Multimillion-dollar contract finalized. • March 22 – Moneyball presentation at IBM Think 17
- 18. Live Demo 18 Green: Predictions based on Lahman only Orange: Predictions based on AriDB + Lahman
- 19. More Details • Moneyball team • How to frame the Moneyball problem for automatic machine learning • Tools: H2O AutoML, LIME, Shiny … 19
- 20. Ari Kaplan – the Real Moneyball 20 • The real characters in the movie (Billy Beane and Paul DePodesta) did not want to work with Hollywood. • The filmmaker interviewed Ari instead and created the Paul character based on Ari’s real-life story. • Ari happens to work at Aginity so we have a real ”Moneyball” for this project.
- 23. 23
- 25. Lahman Data Framed as a ML problem Player Attributes One of the Targets Past Performance Sliding Windows + Other Stats Training Validation Forecast No data. Used 2017 value. Not perfect (a quick hack).
- 26. Approach One: Learning from Lahman only Lahman: Age, Height, Weight … Historical Performance Stats Home Runs Batting Average … Predictions H2O AutoML: Learn the Pattern Sliding Windows (Stats from previous n years) About 300 Lahman Features 26
- 27. Approach Two: Learning from Lahman & AriDB Lahman: Age, Height, Weight … Historical Performance Stats Home Runs Batting Average … Predictions H2O AutoML: Learn the Pattern Sliding Windows (Stats from previous n years) About 300 Lahman Features + 200 AriDB Features AriDB: Fastball, curveball, slider, velocity … 27
- 32. Supervised Learning • Generalized Linear Models: Binomial, Gaussian, Gamma, Poisson and Tweedie • Naïve Bayes Statistical Analysis Ensembles • Distributed Random Forest: Classification or regression models • Gradient Boosting Machine: Produces an ensemble of decision trees with increasing refined approximations Deep Neural Networks • Deep learning: Create multi-layer feed forward neural networks starting with an input layer followed by multiple layers of nonlinear transformations H2O-3 Algorithms Overview Unsupervised Learning • K-means: Partitions observations into k clusters/groups of the same spatial size. Automatically detect optimal k Clustering Dimensionality Reduction • Principal Component Analysis: Linearly transforms correlated variables to independent components • Generalized Low Rank Models: extend the idea of PCA to handle arbitrary data consisting of numerical, Boolean, categorical, and missing data Anomaly Detection • Autoencoders: Find outliers using a nonlinear dimensionality reduction using deep learning 32
- 33. 33
- 34. 34 AutoML
- 35. Supervised Learning • Generalized Linear Models: Binomial, Gaussian, Gamma, Poisson and Tweedie • Naïve Bayes Statistical Analysis Ensembles • Distributed Random Forest: Classification or regression models • Gradient Boosting Machine: Produces an ensemble of decision trees with increasing refined approximations Deep Neural Networks • Deep learning: Create multi-layer feed forward neural networks starting with an input layer followed by multiple layers of nonlinear transformations H2O-3 Algorithms in AutoML Unsupervised Learning • K-means: Partitions observations into k clusters/groups of the same spatial size. Automatically detect optimal k Clustering Dimensionality Reduction • Principal Component Analysis: Linearly transforms correlated variables to independent components • Generalized Low Rank Models: extend the idea of PCA to handle arbitrary data consisting of numerical, Boolean, categorical, and missing data Anomaly Detection • Autoencoders: Find outliers using a nonlinear dimensionality reduction using deep learning 35
- 36. AutoML Code 36
- 41. 41
- 43. Theory • LIME approximates model locally as logistic or linear model • Repeats process many times • Outputs features that are most important to local models Outcome • Approximate reasoning • Complex models can be interpreted • Neural nets, Random Forest, Ensembles etc. 43 Local Interpretable Model-Agnostic Explanations LIME - How does it work?
- 47. H2O Products In-Memory, Distributed Machine Learning Algorithms with H2O Flow GUI H2O AI Open Source Engine Integration with Spark Lightning Fast machine learning on GPUs Automatic feature engineering, machine learning and interpretability Secure multi-tenant H2O clusters Lightning Fast machine learning on GPUs
- 48. “Confidential and property of H2O.ai. All rights reserved” Supervised Learning • Generalized Linear Models: Binomial, Gaussian, Gamma, Poisson and Tweedie • Naïve Bayes Statistical Analysis Ensembles • Distributed Random Forest: Classification or regression models • Gradient Boosting Machine: Produces an ensemble of decision trees with increasing refined approximations Deep Neural Networks • Deep learning: Create multi-layer feed forward neural networks starting with an input layer followed by multiple layers of nonlinear transformations Algorithms on H2O-3 (CPU) Unsupervised Learning • K-means: Partitions observations into k clusters/groups of the same spatial size. Automatically detect optimal k Clustering Dimensionality Reduction • Principal Component Analysis: Linearly transforms correlated variables to independent components • Generalized Low Rank Models: extend the idea of PCA to handle arbitrary data consisting of numerical, Boolean, categorical, and missing data Anomaly Detection • Autoencoders: Find outliers using a nonlinear dimensionality reduction using deep learning
- 49. “Confidential and property of H2O.ai. All rights reserved” Supervised Learning • Generalized Linear Models: Binomial, Gaussian, Gamma, Poisson and Tweedie • Naïve Bayes Statistical Analysis Ensembles • Distributed Random Forest: Classification or regression models • Gradient Boosting Machine: Produces an ensemble of decision trees with increasing refined approximations Deep Neural Networks • Deep learning: Create multi-layer feed forward neural networks starting with an input layer followed by multiple layers of nonlinear transformations Algorithms on H2O4GPU (more to come) Unsupervised Learning • K-means: Partitions observations into k clusters/groups of the same spatial size. Automatically detect optimal k Clustering Dimensionality Reduction • Principal Component Analysis: Linearly transforms correlated variables to independent components • Generalized Low Rank Models: extend the idea of PCA to handle arbitrary data consisting of numerical, Boolean, categorical, and missing data Anomaly Detection • Autoencoders: Find outliers using a nonlinear dimensionality reduction using deep learning
- 51. 51 blog.h2o.ai
- 52. Coming Up 52 Date City Talk / Workshop 25 June Milan H2O + LIME Workshop 28 June Rome H2O Intro + Use Cases Mid-July London Next London Meetup (T.B.C.) Mid-Oct London H2O World London 13-14 Nov London Big Data LDN Keynote by Sri (CEO) + Meetup
- 53. • Organisers & Sponsors • Simone Scardapane • Gabriele Nocco • Sourcesense 53 Thanks! • Code, Slides & Documents • bit.ly/h2o_meetups • bit.ly/joe_eRum_2018 • docs.h2o.ai • Contact • joe@h2o.ai • @matlabulous • github.com/woobe • Please search/ask questions on Stack Overflow • Use the tag `h2o` (not h2 zero)