1.
Copyright © GREE, Inc. All Rights Reserved.

2.
Copyright © GREE, Inc. All Rights Reserved.
•
AI
•
•
• Automated Machine Learning (AutoML)
https://y0z.github.io/about/

3.
Copyright © GREE, Inc. All Rights Reserved.
•
!
• !
•
•
Minimize f(x)
subject to x ∈ X
f(x)

4.
Copyright © GREE, Inc. All Rights Reserved.
•
• ! ( )
!
•
• AutoML 1 (Feurer and Hutter, 2019)
• ! !
f(x)
f(x)
f(x) x
” ”

5.
Copyright © GREE, Inc. All Rights Reserved.
•
• !
•
•
•
•
•
•
• Grey-box Bayesian Optimization for AutoML
https://slideslive.com/38916582/keynote-greybox-bayesian-
optimization-for-automl
f(x)

6.
Copyright © GREE, Inc. All Rights Reserved.
•
GP-EI SMAC TPE
•
Population-based methods CMA-ES
•
Nelder–Mead MADS
•
• Google Vizier (Google)
• Optuna (PFN)
• Nevergrad (Facebook)

7.
Copyright © GREE, Inc. All Rights Reserved.
•
GP-EI SMAC TPE
•
Population-based methods CMA-ES
•
Nelder–Mead MADS
•
• Google Vizier (Google)
• Optuna (PFN)
• Nevergrad (Facebook)

8.
Copyright © GREE, Inc. All Rights Reserved.
•
•
(Cohen et al., 2005; Ozaki et al., 2017)
Nelder–Mead
Nelder and Mead, 1965
CNN (Ozaki et al., 2017)

9.
Copyright © GREE, Inc. All Rights Reserved.
Nelder–Mead
reflect, expand, inside contract, outside contract, shrink 5
reflect, expand, inside contract, outside contract shrink

10.
Copyright © GREE, Inc. All Rights Reserved.
• Nelder–Mead
• Nelder–Mead
Nelder–Mead
Accelerating the Nelder–Mead Method with Predictive Parallel Evaluation
Yoshihiko Ozaki, Shuhei Watanabe, and Masaki Onishi
6th ICML Workshop on Automated Machine Learning, Jun 2019.
! !f(x) ∼ GP(m(x), k(x, x′)) g(x)

11.
Copyright © GREE, Inc. All Rights Reserved.
1.
2. Nelder–Mead
3. P
4. 2.
Nelder–Mead
! !f(x) ∼ GP(m(x), k(x, x′)) g(x)

12.
Copyright © GREE, Inc. All Rights Reserved.
•
• 6 (Klein et
al., 2018)
• ! !
• Baseline 1 shrink ( )
• Baseline 2
•
• Baseline 1 49% 2 13%
P = 10 J = 1,2,3,4,5
Nelder–Mead
Method J Average # of eval steps Average # of evaluations
Baseline 1 - 590.27 (±141.42) 614.10 (±142.82)
Baseline 2 - 347.27 (±89.32) 3469.67 (±893.21)
Proposed 1 406.20 (±97.24) 1534.20 (±427.69)
2 314.13 (±72.26) 2307.83 (±558.02)
3 304.97 (±54.57) 2679.13 (±464.80)
4 310.60 (±67.58) 2948.20 (±642.62)
5 301.90 (±58.70) 2942.33 (±567.27)

13.
Copyright © GREE, Inc. All Rights Reserved.
• ! !
• !
P = 10,20,30,40 J = 1,2,3,4,5
P, J
Nelder–Mead

14.
Copyright © GREE, Inc. All Rights Reserved.
•
•
•
• 8 5 KDD AutoML Workshop
• Yoshihiko Ozaki and Masaki Onishi,
“Practical Deep Neural Network Performance Prediction for
Hyperparameter Optimization,”
To appear.
• https://sites.google.com/view/automl2019-workshop/

15.
Copyright © GREE, Inc. All Rights Reserved.