Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Using H2O Random Grid Search for
Hyper-parameters Optimization
Jo-fai (Joe) Chow
Data Scientist
joe@h2o.ai
WHO AM I
• Customer Data Scientist at H2O.ai
• Background
o Telecom (Virgin Media)
o Data Science Platform (Domino Data La...
ABOUT H2O
• Company
o Team: 50 (45 shown)
o Founded in 2012, Mountain View,
California.
o Venture capital backed
• Product...
ABOUT THIS TALK
• Story of a baker and a data scientist
o Why you should care
• Hyper-parameters optimization
o Common tec...
STORY OF A BAKER
• Making a cake
o Source
• Ingredients
o Process:
• Mixing
• Baking
• Decorating
o End product
• A nice l...
STORY OF A DATA SCIENTIST
• Making a data product
o Source
• Raw data
o Process:
• Data munging
• Analyzing / Modeling
• R...
BAKER AND DATA SCIENTIST
• What do they have in common?
o Process is important to bakers and data scientists.
Yet, most cu...
WHY YOU SHOULD CARE
• We can use machine/software to automate
some laborious tasks.
• We can spend more time on quality as...
HYPER-PARAMETERS OPTIMISATION
• Overview
o Optimizing an algorithm’s performance.
• e.g. Random Forest, Gradient Boosting ...
HYPER-PARAMETERS OPTIMISATION
• Example of hyper-parameters in H2O
o Random Forest:
• No. of trees, depth of trees, sample...
COMMON TECHNIQUES
• Manual search
o Tuning by hand - inefficient
o Expert opinion (not always reliable)
• Grid search
o Au...
RANDOM GRID SEARCH – DOES IT WORK?
• Random Search for Hyper-Parameter
Optimization
o Journal of Machine Learning Research...
RELATED FEATURE – EARLY STOPPING
• A technique for regularization.
• Avoid over-fitting the training set.
• Useful when co...
H2O RANDOM GRID SEARCH
• Objectives
o Optimize model performance based on evaluation
metric.
o Explore the defined search ...
RANDOM GRID SEARCH (PYTHON API)
RANDOM GRID SEARCH
• Outputs
o Best model based on metric
o A set of hyper-parameters for the best model
• Other APIs
o R,...
OTHER H2O FEATURES
• h2oEnsemble
o Better predictive performance
• Sparkling Water = Spark + H2O
• Plain Old Java Object (...
CONCLUSIONS
• Most people only care about the end product.
• Use H2O random grid search to save time on
hyper-parameters t...
CONCLUSIONS
• H2O Random Grid Search
o An efficient way to tune hyper-parameters
o APIs for Python, R, Java, REST
o Do che...
ACKNOWLEDGEMENTS
• GoDataDriven
• Conference sponsors
• My colleagues at H2O.ai
THANK YOU
• Resources
o Slides + code – github.com/h2oai/h2o-meetups
o Download H2O – www.h2o.ai
o Documentation – www.h2o...
Upcoming SlideShare
Loading in …5
×

Using H2O Random Grid Search for Hyper-parameters Optimization

1,191 views

Published on

My talk at PyData Amsterdam

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Using H2O Random Grid Search for Hyper-parameters Optimization

  1. 1. Using H2O Random Grid Search for Hyper-parameters Optimization Jo-fai (Joe) Chow Data Scientist joe@h2o.ai
  2. 2. WHO AM I • Customer Data Scientist at H2O.ai • Background o Telecom (Virgin Media) o Data Science Platform (Domino Data Lab) o Water Engineering + Machine Learning Research (STREAM Industrial Doctorate Centre)
  3. 3. ABOUT H2O • Company o Team: 50 (45 shown) o Founded in 2012, Mountain View, California. o Venture capital backed • Products o Open-source machine learning platform. o Flow (Web), R, Python, Spark, Hadoop interfaces.
  4. 4. ABOUT THIS TALK • Story of a baker and a data scientist o Why you should care • Hyper-parameters optimization o Common techniques o H2O Python API • Other H2O features for streamlining workflow
  5. 5. STORY OF A BAKER • Making a cake o Source • Ingredients o Process: • Mixing • Baking • Decorating o End product • A nice looking cake Credit: www.dphotographer.co.uk/image/201305/baking_a_cake
  6. 6. STORY OF A DATA SCIENTIST • Making a data product o Source • Raw data o Process: • Data munging • Analyzing / Modeling • Reporting o End product • Apps, graphs or reports Credit: www.simranjindal.com
  7. 7. BAKER AND DATA SCIENTIST • What do they have in common? o Process is important to bakers and data scientists. Yet, most customers do not appreciate the effort. o Most customers only care about raw materials quality and end products.
  8. 8. WHY YOU SHOULD CARE • We can use machine/software to automate some laborious tasks. • We can spend more time on quality assurance and presentation. • This talk is about making one specific task, hyper-parameters tuning, more efficient.
  9. 9. HYPER-PARAMETERS OPTIMISATION • Overview o Optimizing an algorithm’s performance. • e.g. Random Forest, Gradient Boosting Machine (GBM) o Trying different sets of hyper-parameters within a defined search space. o No rules of thumb.
  10. 10. HYPER-PARAMETERS OPTIMISATION • Example of hyper-parameters in H2O o Random Forest: • No. of trees, depth of trees, sample rate … o Gradient Boosting Machine (GBM): • No. of trees, depth of trees, learning rate, sample rate … o Deep Learning: • Activation, hidden layer sizes, L1, L2, dropout ratios …
  11. 11. COMMON TECHNIQUES • Manual search o Tuning by hand - inefficient o Expert opinion (not always reliable) • Grid search o Automated search within a defined space o Computationally expensive • Random grid search o More efficient than manual / grid search o Equal performance in less time
  12. 12. RANDOM GRID SEARCH – DOES IT WORK? • Random Search for Hyper-Parameter Optimization o Journal of Machine Learning Research (2012) o James Bergstra and Yoshua Bengio o “Compared with deep belief networks configured by a thoughtful combination of manual search and grid search, purely random search found statistically equal performance on four of seven data sets, and superior performance on one of seven.”
  13. 13. RELATED FEATURE – EARLY STOPPING • A technique for regularization. • Avoid over-fitting the training set. • Useful when combined with hyper-parameter search: o Additional controls (e.g. time constraint, tolerance)
  14. 14. H2O RANDOM GRID SEARCH • Objectives o Optimize model performance based on evaluation metric. o Explore the defined search space randomly. o Use early-stopping for regularization and additional controls.
  15. 15. RANDOM GRID SEARCH (PYTHON API)
  16. 16. RANDOM GRID SEARCH • Outputs o Best model based on metric o A set of hyper-parameters for the best model • Other APIs o R, REST, Java (see documentation on GitHub)
  17. 17. OTHER H2O FEATURES • h2oEnsemble o Better predictive performance • Sparkling Water = Spark + H2O • Plain Old Java Object (POJO) o Productionize H2O models
  18. 18. CONCLUSIONS • Most people only care about the end product. • Use H2O random grid search to save time on hyper-parameters tuning. • Spend more time on quality assurance and presentation.
  19. 19. CONCLUSIONS • H2O Random Grid Search o An efficient way to tune hyper-parameters o APIs for Python, R, Java, REST o Do check out the code examples on GitHub • Combine with other H2O features o Streamline data science workflow
  20. 20. ACKNOWLEDGEMENTS • GoDataDriven • Conference sponsors • My colleagues at H2O.ai
  21. 21. THANK YOU • Resources o Slides + code – github.com/h2oai/h2o-meetups o Download H2O – www.h2o.ai o Documentation – www.h2o.ai/docs/ o joe@h2o.ai • We are hiring – www.h2o.ai/careers/

×