The document discusses continual lifelong learning (CL) in neural networks, emphasizing the challenges of catastrophic forgetting, where new training can overwrite old knowledge. It highlights various strategies and benchmarks for CL, including different neural network architectures and real-world applications in autonomous agents and robots. The paper also touches on future developments needed to enhance CL, such as improving robustness and efficiency.

1. Continual Lifelong Learning
with Neural Networks
April 16, 2019 -Tutorial @ INNSBDDL2019

2. A Practical Example
• 50GB/s streaming
data.
• ~30240TB of data
after only a week.
• Impossible to re-train
the mini-spot brain
from scratch and to
adapt fast.
Mini-spot Robot from Boston Dynamics, 2018

3. Continual Learning (CL)
• Ability to continually acquire, fine-tune, and transfer
new knowledge and skills
• Higher and realistic time-scale where data (and tasks)
become available only during time.
• No access to previously encountered data.
• Constrained computational and memory resources.

4. paper

5. Catastrophic forgetting
• Training a model with new information interferes with
previously learned knowledge
• Abrupt performance decrease or old knowledge
completely overwritten by the new one.

6. Catastrophic forgetting
• Training a model with new information interferes with
previously learned knowledge
• Abrupt performance decrease or old knowledge
completely overwritten by the new one.

7. The Stability-Plasticity Dilemma
Stability-Plasticity Dilemma:
• Remember past concepts
• Learn new concepts
• Generalize
Biggest Problem in Deep Learning:
• Catastrophic Forgetting

8. Biological factors of CL
• Structural Plasticity
• Neurosynaptic adaptation to changes in the environment
• Change of physical structure as the result of learning
• Stability-plasticity balance
• Complementary Learning Systems
• Retaining episodic memories (memorization)
• Extracting statistical structure (generalization)
• Memory replay

9. Biological factors of CL
•

10. Neural Network Architectures

11. ElasticWeights Consolidation (EWC)
Fisher Information
...

12. Growing Networks
Parisi,Tani,Weber,Wermter. Lifelong Learning of Spatio-temporal Representations with Dual-Memory
Recurrent Self-Organization. Frontiers in Neurorobotics 2019.

13. Neurogenesis

14. CL Strategies
Architectural
Regularization Rehearsal
CWR PNN
EWC
SI
LWF
ICARL
AR1
GEM
Pure
Rehearsal
GDM
Exstream

15. Continual Learning
Benchmarks

16. Supervised CL benchmarks
Dataset Strategy
Permuted MNIST EWC, GEM, SI, ...
Rotated MNIST GEM
MNIST Split SI
CIFAR10/100 Split GEM, iCARL, SI, AR1, ...
ILSVRC2012 iCARL
CUB-200 GMD, ...
LomonacoV. and Maltoni D. CORe50: a New Dataset and Benchmark for Continuous Object Recognition. CoRL2017.

17. Sequential CL benchmarks
LomonacoV. and Maltoni D. CORe50: a New Dataset and Benchmark for Continuous Object Recognition. CoRL2017.

18. CRL Environments
Environments Scenarios
Atari Multiple 2D games
DeepMind Lab
Maze Exploration,
Object Picking
Malmo Multiple tasks
OpenAI Gym Multiple 3D tasks
MuJoCo Multiple Joint Stiffness
VizDoom -
Unity 3D -
StarCraft II Curriculum learning

19. Some References for CRL
• Al-Shedivat, Maruan, et al. "Continuous adaptation via meta-learning in
nonstationary and competitive environments." arXiv preprint arXiv:1710.03641
(2017).
• Tessler, Chen, et al. "A deep hierarchical approach to lifelong learning in
minecraft."Thirty-First AAAI Conference on Artificial Intelligence. 2017.
• Kirkpatrick, James, et al. "Overcoming catastrophic forgetting in neural
networks." Proceedings of the national academy of sciences 114.13 (2017):
3521-3526.
• Schwarz, Jonathan, et al. "Progress & compress: A scalable framework for
continual learning." arXiv preprint arXiv:1805.06370 (2018).
• Kaplanis, Christos, Murray Shanahan, and Claudia Clopath. "Continual
reinforcement learning with complex synapses." arXiv preprint arXiv:1802.07239
(2018).

20. LomonacoV. and Maltoni D. CORe50: a New Dataset and Benchmark for Continuous Object Recognition. CoRL2017.
CORe50: aVideo Benchmark for CL and Object
Recognition, Detection and Segmentation

21. # Images 164,866
Format RGB-D
Image size 350x350
128x128
# Categories 10
# Obj. x Cat. 5
# Sessions 11
# img. x Sess. ~300
# Outdoor Sess. 3
Acquisition Sett. Hand held
LomonacoV. and Maltoni D. CORe50: a New Dataset and Benchmark for Continuous Object Recognition. CoRL2017.
CORe50: aVideo Benchmark for CL and Object
Recognition, Detection and Segmentation

22. CORe50 Benchmark
(NI)
(NC)
(NIC)

23. CORe50Website
Dataset, Benchmark, code and additional
information freely available at:
vlomonaco.github.io/core50

24. CRL in 3D non-stationary environment
LomonacoV., Desai K., Maltoni D. and Culurciello, E. Continual Reinforcement Learning in 3D non-stationary
environments. Submitted to ECML-PKDD, 2019.
VIDEO!

25. ConsideredVariations

26. Lifelong Learning
Metrics

27. CL Framework and Metrics
CL Algorithm
N. Díaz-Rodríguez,V. Lomonaco et al. Don't forget, there is more than forgetting: new metrics for Continual Learning.
CLWorkshop, NeurIPS 2018.

28. Continual Learning:
Where to start?

29. ContinualAI non-profit Research Organization
http://continualai.org https://continualai.herokuapp.com/

30. A Gentle Introduction to CL in PyTorch
https://github.com/ContinualAI/colab

31. Limitations and FutureWorks
Limitations
• Young line of research
• Theoretical foundations
• Real-world applications
What’s next?
• Towards Biological Synaptic Plasticity, learning and
memory.
• Robustness, flexibility, and efficiency.

32. CL in autonomous agents & robots
• Progressively acquire, fine-tune, and transfer
knowledge and skills through the interaction with the
environment
• Data are temporally correlated and increasingly more
complex
• Active exploration through intrinsic motivation

33. Rethinking CL for autonomous agents

34. Rethinking CL for autonomous agents

35. Rethinking CL for autonomous agents

36. Rethinking CL for autonomous agents