Downloaded 61 times
Uploaded byKaty Lee
Overcoming catastrophic forgetting in neural network
EWC is a technique to prevent catastrophic forgetting in neural networks. It works by constraining important parameters learned from previous tasks to stay close to their original values when training on new tasks. The importance of parameters is determined by calculating how much changing each parameter would affect the loss on a previous task. EWC then regularizes the loss function to slow learning more for important parameters and allow less disruption to previous tasks. Experiments on MNIST and Atari demonstrate EWC allows neural networks to continually learn new tasks without forgetting previous ones.
Overcoming catastrophic forgetting in neural network
- 1. Overcoming catastrophic forgetting inneural networks 2017/04/28 Katy@Datalab PNAS, Proceedings of the National Academy of Sciences citation: 4
- 2. Background • Catastrophic forgettingis forgetting key information needed to solve a previous task when training on a new task.
- 3. The kind ofconsolidation that occur in the brain • Continual learning in the neocortex relies on task- specific synaptic consolidation • Synaptic consolidation: connections between neurons are less likely to be overwritten if they have been important in previously learnt tasks.
- 4. • knowledge isdurably encoded by rendering a proportion of synapses less plastic and therefore stable over long timescales.
- 5. Related Work • Ensembleof DNNs: Rusu, Andrei A., et al. "Progressive neural networks." arXiv preprint arXiv: 1606.04671 (2016).
- 6. • Fernando, Chrisantha,et al. "Pathnet: Evolution channels gradient descent in super neural networks." arXiv preprint arXiv:1701.08734 (2017).
- 8. Intuition • They implementan algorithm that performs a similar operation in artificial neural networks by constraining important parameters to stay close to their old values.
- 9. ELASTIC WEIGHT CONSOLIDATION (EWC) •θA* refers to the configuration of θ that performs well at A • slow down the learning for the weights that were important to previous task(s)
- 10. Key Question • Howto determine which weights are most important to task A?
- 11. loss for Bonly regulariser makes sure we don't catastrophically forget about A.
- 12.
- 13. • The biggerthe fish information(negative second-order derivatives), the more pointy of the distribution of the likelihood • likelihood is more sensitive to this parameter
- 14.
- 16. Experiment 2 onAtari Training Schedule
- 17. 10: human level, 1:random agent
- 18. Conclusion • The abilityto learn tasks in succession without forgetting is a core component of biological and artificial intelligence. • Weight uncertainty should inform learning rates. • EWC allows the network to effectively squeeze in more functionality into a network with fixed capacity,