Successfully reported this slideshow.
Your SlideShare is downloading. ×

Group normalization

Ad

Group Normalization
Fengda Zhu

Ad

Overview
1. BN has memory constraints
2. feature channels have correlatives

Ad

Group Normalization

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Loading in …3
×

Check these out next

1 of 14 Ad
1 of 14 Ad
Advertisement

More Related Content

Advertisement

Group normalization

  1. 1. Group Normalization Fengda Zhu
  2. 2. Overview 1. BN has memory constraints 2. feature channels have correlatives
  3. 3. Group Normalization
  4. 4. Formulation
  5. 5. Implementation
  6. 6. Experiments
  7. 7. Experiments
  8. 8. Experiments
  9. 9. Experiments
  10. 10. Experiments
  11. 11. Experiments
  12. 12. Experiments
  13. 13. Experiments
  14. 14. Discussion 1. BN has been so influential that many state-of- the-art systems and their hyper- parameters have been de- signed for it, which may not be optimal for GN- based models. 2. Sense GN is related to LN and IN, two normalization methods that are particularly successful in training recurrent (RNN/LSTM) or generative (GAN) models. This suggests us to study GN in those areas in the future.

×