Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Group normalization

32 views

Published on

Group normalization

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Group normalization

  1. 1. Group Normalization Fengda Zhu
  2. 2. Overview 1. BN has memory constraints 2. feature channels have correlatives
  3. 3. Group Normalization
  4. 4. Formulation
  5. 5. Implementation
  6. 6. Experiments
  7. 7. Experiments
  8. 8. Experiments
  9. 9. Experiments
  10. 10. Experiments
  11. 11. Experiments
  12. 12. Experiments
  13. 13. Experiments
  14. 14. Discussion 1. BN has been so influential that many state-of- the-art systems and their hyper- parameters have been de- signed for it, which may not be optimal for GN- based models. 2. Sense GN is related to LN and IN, two normalization methods that are particularly successful in training recurrent (RNN/LSTM) or generative (GAN) models. This suggests us to study GN in those areas in the future.

×