Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Successfully reported this slideshow.

Like this presentation? Why not share!

- Common Problems in Hyperparameter O... by SigOpt 830 views
- Hyperparameter Optimization 101 by SigOpt 3892 views
- MLConf 2016 SigOpt Talk by Scott Clark by SigOpt 909 views
- Visualizing Threats: Network Visual... by Cambridge Intelli... 3919 views
- Lecture7 xing fei-fei by Tianlu Wang 631 views
- PROTEUS H2020 by Bonaventura Del M... 235 views

1,507 views

Published on

Published in:
Software

No Downloads

Total views

1,507

On SlideShare

0

From Embeds

0

Number of Embeds

73

Shares

0

Downloads

18

Comments

0

Likes

1

No embeds

No notes for slide

- 1. Visualizing Abstract Concepts in Machine Learning PIC Alexandra Johnson ___________ Software Engineer @ SigOpt #MachineLearning #MLViz Visualizing Abstract Concepts in Machine Learning | 1
- 2. Visualizing Abstract Concepts in Machine Learning | 2 What is Machine Learning? Versicolor Setosa Virginica Training Data + Model -> Labels (Classiﬁcation) or Numbers (Regression)
- 3. Why is this so Intimidating? Visualizing Abstract Concepts in Machine Learning | 3 In-brower deep neural net from playground.tensorﬂow.org Hyperparameters = your model's magic numbers Examples: learning rate, ratio of train to test data, number of hidden layers, neurons per hidden layer Hyperparameter values must be set before training
- 4. Solution: Hyperparameter Optimization And four visualization challenges Visualizing Abstract Concepts in Machine Learning | 4
- 5. Values you choose for your hyperparameters have a direct eﬀect on the performance of your model Hard to capture interactions of 20 hyperparameters 20 Dimensional Math is Hard Visualizing Abstract Concepts in Machine Learning | 5
- 6. −15 −10 −5 0 5 0.2 0.4 0.6 0.8 1 log_C Accuracy Visualizing Abstract Concepts in Machine Learning | 6 20 Dimensional Math is Hard First try: graph model performance vs hyperparameter value For every hyperparameter Good for understanding indivudal hyperparameters, bad for understanding interactions
- 7. 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Accuracy Visualizing Abstract Concepts in Machine Learning | 7 20 Dimensional Math is Hard Graph up to 4 dimensions at once: x, y, z axis + color Hard to visualize 4 dimensions at once, imagine 20! Maybe you want to use an algorithm to handle hyperparameter optimization
- 8. Visualizing Abstract Concepts in Machine Learning | 8 Hyperparameter Optimization Strategies are Diﬀerent Grid Search Random Search Bayesian Optimization
- 9. Some Strategies Produce Better Results 0.96 0.97 0.98 0.99 0 5 10 15 20 25 Distribution of Best Found Values over Experiments of 25 Iterations Maximum Accuracy Experiments Visualizing Abstract Concepts in Machine Learning | 9 Experiment = optimizing hyperparameters of your model, results in some maximum performance Some hyperparameter optimization strategies are stochastic, can't just look at one experiment Look at distribution of maximum performance over many experiments optimizing hyperparameters of the same model
- 10. Some Strategies Produce Better Results 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1 0 5 10 15 20 25 Distribution of Best Found Values over Experiments of 25 Iterations Maximum Accuracy Experiments Random Search Grid Search Bayesian Optimization Visualizing Abstract Concepts in Machine Learning | 10 Use the Mann-Whitney U Test to compare distributions of maximum performance
- 11. Some Strategies Produce Better Results, Faster 0 5 10 15 20 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Best Seen Trace Timestep BestSeenAccuracy Visualizing Abstract Concepts in Machine Learning | 11 How much time do you have for optimization? Strategies that reliably produce better results faster can optimize the hyperparameters of your model in less time
- 12. Some Strategies Produce Better Results, Faster 0 5 10 15 20 0.4 0.5 0.6 0.7 0.8 0.9 1 Interquartile Range of Best Seen Traces Timestep BestSeenAccuracy Visualizing Abstract Concepts in Machine Learning | 12 Again, consider a distribution of optimization experiments 25th - 75th percentile of performance our model could acheive if we stopped early
- 13. Some Strategies Produce Better Results, Faster 0 5 10 15 20 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Interquartile Ranges of Best Seen Traces Timestep BestSeenAccuracy Grid Search Random Search Bayesian Optimization Visualizing Abstract Concepts in Machine Learning | 13 Compare the area under the curve of diﬀerent strategies Further reading at sigopt.com/research
- 14. Takeaways Visualizing Abstract Concepts in Machine Learning | 14 Hyperparameter optimization is an invaluable part of any modern machine learning pipeline Concepts like comparing hyperparameter optimization strategies are extremely abstract and diﬃcult to understand Visualizations are in their infancy, but are an important part of explaining these ideas
- 15. Thank You! Visualizing Abstract Concepts in Machine Learning | 14 Email: alexandra@sigopt.com Twitter: @alexandraj777 www.sigopt.com

No public clipboards found for this slide

Be the first to comment