Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Tutorial inns2019 full

4 views

Published on

Artificial agents interacting in highly dynamic environments are required to continually acquire and fine-tune their knowledge overtime. In contrast to conventional deep neural networks that typically rely on a large batch of annotated training samples, lifelong learning systems must account for situations in which the number of tasks is not known a priori and the data samples become incrementally available over time. Despite recent advances in deep learning, lifelong machine learning has remained a long-standing challenge due to neural networks being prone to catastrophic forgetting, i.e., the learning of new tasks interferes with previously learned ones and leads to abrupt disruptions of performance. Recently proposed deep supervised and reinforcement learning models for addressing catastrophic forgetting suffer from flexibility, robustness, and scalability issues with respect to biological systems. In this tutorial, we will present and discuss well-established and emerging neural network approaches motivated by lifelong learning factors in biological systems such as neurosynaptic plasticity, complementary memory systems, multi-task transfer learning, and intrinsically motivated exploration.

Published in: Science
  • Be the first to comment

  • Be the first to like this

Tutorial inns2019 full

  1. 1. Continual Lifelong Learning with Neural Networks April 16, 2019 -Tutorial @ INNSBDDL2019
  2. 2. A Practical Example • 50GB/s streaming data. • ~30240TB of data after only a week. • Impossible to re-train the mini-spot brain from scratch and to adapt fast. Mini-spot Robot from Boston Dynamics, 2018
  3. 3. Continual Learning (CL) • Ability to continually acquire, fine-tune, and transfer new knowledge and skills • Higher and realistic time-scale where data (and tasks) become available only during time. • No access to previously encountered data. • Constrained computational and memory resources.
  4. 4. paper
  5. 5. Catastrophic forgetting • Training a model with new information interferes with previously learned knowledge • Abrupt performance decrease or old knowledge completely overwritten by the new one.
  6. 6. Catastrophic forgetting • Training a model with new information interferes with previously learned knowledge • Abrupt performance decrease or old knowledge completely overwritten by the new one.
  7. 7. The Stability-Plasticity Dilemma Stability-Plasticity Dilemma: • Remember past concepts • Learn new concepts • Generalize Biggest Problem in Deep Learning: • Catastrophic Forgetting
  8. 8. Biological factors of CL • Structural Plasticity • Neurosynaptic adaptation to changes in the environment • Change of physical structure as the result of learning • Stability-plasticity balance • Complementary Learning Systems • Retaining episodic memories (memorization) • Extracting statistical structure (generalization) • Memory replay
  9. 9. Biological factors of CL •
  10. 10. Neural Network Architectures
  11. 11. ElasticWeights Consolidation (EWC) Fisher Information ...
  12. 12. Growing Networks Parisi,Tani,Weber,Wermter. Lifelong Learning of Spatio-temporal Representations with Dual-Memory Recurrent Self-Organization. Frontiers in Neurorobotics 2019.
  13. 13. Neurogenesis
  14. 14. CL Strategies Architectural Regularization Rehearsal CWR PNN EWC SI LWF ICARL AR1 GEM Pure Rehearsal GDM Exstream
  15. 15. Continual Learning Benchmarks
  16. 16. Supervised CL benchmarks Dataset Strategy Permuted MNIST EWC, GEM, SI, ... Rotated MNIST GEM MNIST Split SI CIFAR10/100 Split GEM, iCARL, SI, AR1, ... ILSVRC2012 iCARL CUB-200 GMD, ... LomonacoV. and Maltoni D. CORe50: a New Dataset and Benchmark for Continuous Object Recognition. CoRL2017.
  17. 17. Sequential CL benchmarks LomonacoV. and Maltoni D. CORe50: a New Dataset and Benchmark for Continuous Object Recognition. CoRL2017.
  18. 18. CRL Environments Environments Scenarios Atari Multiple 2D games DeepMind Lab Maze Exploration, Object Picking Malmo Multiple tasks OpenAI Gym Multiple 3D tasks MuJoCo Multiple Joint Stiffness VizDoom - Unity 3D - StarCraft II Curriculum learning
  19. 19. Some References for CRL • Al-Shedivat, Maruan, et al. "Continuous adaptation via meta-learning in nonstationary and competitive environments." arXiv preprint arXiv:1710.03641 (2017). • Tessler, Chen, et al. "A deep hierarchical approach to lifelong learning in minecraft."Thirty-First AAAI Conference on Artificial Intelligence. 2017. • Kirkpatrick, James, et al. "Overcoming catastrophic forgetting in neural networks." Proceedings of the national academy of sciences 114.13 (2017): 3521-3526. • Schwarz, Jonathan, et al. "Progress & compress: A scalable framework for continual learning." arXiv preprint arXiv:1805.06370 (2018). • Kaplanis, Christos, Murray Shanahan, and Claudia Clopath. "Continual reinforcement learning with complex synapses." arXiv preprint arXiv:1802.07239 (2018).
  20. 20. LomonacoV. and Maltoni D. CORe50: a New Dataset and Benchmark for Continuous Object Recognition. CoRL2017. CORe50: aVideo Benchmark for CL and Object Recognition, Detection and Segmentation
  21. 21. # Images 164,866 Format RGB-D Image size 350x350 128x128 # Categories 10 # Obj. x Cat. 5 # Sessions 11 # img. x Sess. ~300 # Outdoor Sess. 3 Acquisition Sett. Hand held LomonacoV. and Maltoni D. CORe50: a New Dataset and Benchmark for Continuous Object Recognition. CoRL2017. CORe50: aVideo Benchmark for CL and Object Recognition, Detection and Segmentation
  22. 22. CORe50 Benchmark (NI) (NC) (NIC)
  23. 23. CORe50Website Dataset, Benchmark, code and additional information freely available at: vlomonaco.github.io/core50
  24. 24. CRL in 3D non-stationary environment LomonacoV., Desai K., Maltoni D. and Culurciello, E. Continual Reinforcement Learning in 3D non-stationary environments. Submitted to ECML-PKDD, 2019. VIDEO!
  25. 25. ConsideredVariations
  26. 26. Lifelong Learning Metrics
  27. 27. CL Framework and Metrics CL Algorithm N. Díaz-Rodríguez,V. Lomonaco et al. Don't forget, there is more than forgetting: new metrics for Continual Learning. CLWorkshop, NeurIPS 2018.
  28. 28. Continual Learning: Where to start?
  29. 29. ContinualAI non-profit Research Organization http://continualai.org https://continualai.herokuapp.com/
  30. 30. A Gentle Introduction to CL in PyTorch https://github.com/ContinualAI/colab
  31. 31. Limitations and FutureWorks Limitations • Young line of research • Theoretical foundations • Real-world applications What’s next? • Towards Biological Synaptic Plasticity, learning and memory. • Robustness, flexibility, and efficiency.
  32. 32. CL in autonomous agents & robots • Progressively acquire, fine-tune, and transfer knowledge and skills through the interaction with the environment • Data are temporally correlated and increasingly more complex • Active exploration through intrinsic motivation
  33. 33. Rethinking CL for autonomous agents
  34. 34. Rethinking CL for autonomous agents
  35. 35. Rethinking CL for autonomous agents
  36. 36. Rethinking CL for autonomous agents

×