Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

DARTS: Differentiable Architecture Search at 社内論文読み会

DARTS (arXiv: https://arxiv.org/abs/1806.09055)の社内論文読み会資料

  • Login to see the comments

  • Be the first to like this

DARTS: Differentiable Architecture Search at 社内論文読み会

  1. 1. DARTS
 Differentiable Architecture Search Hanxiao Liu, Karen Simonyan, Yiming Yang arXiv: https://arxiv.org/abs/1806.09055 Published as a conference paper at ICLR 2019 Masashi Shibata
  2. 2. NAS
  3. 3. 
 https://tech.mercari.com/entry/2019/05/10/120000
  4. 4. (ex: 3000 GPU days) RL (+ RNN) Evolution algorithms Bayesian optimization
  5. 5. GPU days Cell Weight sharing • → One-shot Neural Architecture Search Discrete domain
  6. 6. Continuous Relaxation 

  7. 7. Continuous Relaxation
  8. 8. • 
 DAG • 
 ( : conv_3x3, max_pool, ) • skip connections multiple branches Directed Acyclic Graph
  9. 9. Discrete domain → Continuous domain Continuous Relaxation Replace after the end of search o(i, j) α softmax o(i, j) α
  10. 10. α(i, j) → bilevel optimization
  11. 11. w α w validation data α α training data w
  12. 12. w α w validation data α α training data w α α w*(α)
  13. 13. w α w validation data α α training data w Hessian
  14. 14. Cell (Block)
  15. 15. Cell (Block) • (skip connections cell ) • repeating building block useful design principle (ex: RNN) Cell (Block) Neural Architecture Search: A Survey 
 (arXiv: https://arxiv.org/abs/1808.05377)
  16. 16. 1 2 3 4 We assume the cell to have two input nodes and a single output node. normal cell normal cell normal cell normal cell reduction cell input tensor preprocess0 preprocess1 Cells located at the 1/3 and 2/3 of the total depth of the network are reduction cells.
  17. 17. • Cell : 8 • Cell Node : 7 • Operation : 8 (zero operation ) • 3×3 and 5×5 separable convolutions • 3×3 and 5×5 dilated separable convolutions • 3×3 max pooling, 3×3 average pooling • identity, and zero. • • CIFAR-10
  18. 18. Continuous Relaxation • SoTA (RobustDARTS, ASNG-NAS)
  19. 19. THANK YOU

×