Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Continual Learning: Another Step Towards Truly Intelligent Machines

44 views

Published on

Humans have the extraordinary ability to learn continually from experience. Not only we can apply previously learned knowledge and skills to new situations, we can also use these as the foundation for later learning. One of the grand goals of Artificial Intelligence (AI) is building an artificial continual learning agent that constructs a sophisticated understanding of the world from its own experience through the autonomous incremental development of ever more complex knowledge and skills. However, current AI systems greatly suffer from the exposure to new data or environments which even slightly differ from the ones for which they have been trained for. Moreover, the learning process is usually constrained on fixed datasets within narrow and isolated tasks which may hardly lead to the emergence of more complex and autonomous intelligent behaviors. In essence, continual learning and adaptation capabilities, while more than often thought as fundamental pillars of every intelligent agent, have been mostly left out of the main AI research focus. In this talk, we explore the application of these ideas in the context of Vision with a focus on (deep) continual learning strategies for object recognition running at the edge on highly-constrained hardware devices.

Published in: Science
  • Be the first to comment

Continual Learning: Another Step Towards Truly Intelligent Machines

  1. 1. Continual Learning: Another Step TowardsTruly Intelligent Machines Introduction Meetup @ Numenta 16-09-2019 Vincenzo Lomonaco vincenzo.lomonaco@unibo.it Postdoctoral Researcher @ University of Bologna Supervisor: Davide Maltoni
  2. 2. About me • Post-Doc @ University of Bologna • Research Affiliate @ AI Labs • Teaching Assistant of the courses Machine Learning and Computer Architectures @ UniBo • Author andTechnical reviewer of the online course Deep Learning with R and book R Deep Learning Essentials. • Co-Founder and President of ContinualAI.org • Co-Founder and Board Member of Data Science Bologna and AIforPeople.org
  3. 3. What’s ContinualAI? • ContinualAI is a non-profit research organization and the largest research community on Continual Learning for AI. • It counts more than 550+ members in 17 different time-zones and from top-notch research institutions. • Learn more about ContinualAI at www.continualai.org
  4. 4. ContinualAI Board Members and Advisors
  5. 5. Machine Intelligence @ BioLab Davide Maltoni Vincenzo Lomonaco Lorenzo Pellegrini Gabriele Graffieti
  6. 6. Outline 1. Personal ResearchTrajectory andVision 2. Continual Learning: State-of-the-art 3. Rehearsal-free and Task-agnostic Online Continual Learning 4. CurrentWork and Research Direction
  7. 7. PersonalResearch Trajectory andVision
  8. 8. ResearchTrajectory andVision I meet Davide Maltoni who was working at HTMs from 2011. I read “On Intelligence” and join his quest for understanding intelligence and build it in silicon. MasterThesis Published: “Comparing HTMs and CNNs on Object RecognitionTasks” 2014 Visiting Scholar at Purdue University. Working on Continual Reinforcement / Unsupervised Learning. Visiting Scholar at ENSTA ParisTech. Working on Continual for Robotics and a more comprehensive CL framework definition. 2015 2017 2018 I defend my PhD Dissertation “Continual Learning with Deep Architectures”. Putting everything together. Post-Doc @ UniBo on the same topic. 2019 We abandon HTM (1st Gen.) to work on top of deep learning directly with a focus on Continual Learning. In particular, on Continual Learning from video sequences. 2016 Long-term vision: “Understand the key computational principles of intelligence and build truly intelligent machines.” Main research goal: “Closing the gap between the HTM theory and current AI systems.”
  9. 9. OurWorks with HTMs (1st Gen.) 1. D. Maltoni, Pattern Recognition by HierarchicalTemporal Memory,Technical Report, DEIS - University of Bologna technical report, April 2011. 2. D. Maltoni and E.M. Rehn, Incremental Learning by Message Passing in HierarchicalTemporal Memory in 5thWorkshop on Artificial Neural Networks in Pattern Recognition (ANNPR12), Trento (Italy), pp.24-35, September 2012. 3. E.M. Rehn and D. Maltoni, Incremental Learning by Message Passing in HierarchicalTemporal Memory, Neural Computation, vol.26, no.8, pp.1763-1809, August 2014. 4. D. Maltoni andV. Lomonaco, Semi-supervisedTuning from Temporal Coherence, in International Conference on Pattern Recognition (ICPR16), Cancun, Mexico, December 2016.
  10. 10. Semi-SupervisedTuning from Temporal Coherence (with HTMs)
  11. 11. HTM theory Principles of Intelligence 1. Hierarchical Learning 2. Sequence Learning 3. Continual Learning 4. Sparse Representations 5. Sensory-Motor Integration (Embodiment) 6. Distributed Parallel Modeling (Thousands BrainTheory) 7. … ? Emerging Properties Flexibility Robustness Scalability Efficiency Adaptation Autonomy Generalization Compositionality Reasoning Common Sense ...
  12. 12. Towards “Cortical Learning” Neuroscience Grounding PracticalFunctionality Symbolic AI Kernel Machines Feed-Forward NNs / LSTMs CNNs Deep-CNNs Conv-LSTMs Deep-RL Continual Learning Cortical Learning HTM (1st Gen.) CLA (2nd Gen.) CLA (3rd Gen.) Other Approaches HTM-based Neural Networks Based Bayesian Approaches Analogism-based Approaches Evolutionary Approaches
  13. 13. Continual Learning: State-of-the-art
  14. 14. The Stability-Plasticity Dilemma Stability-Plasticity Dilemma: • Remember past concepts • Learn new concepts • Generalize Biggest Problem in Deep Learning: • Catastrophic Forgetting
  15. 15. The Stability-Plasticity Dilemma
  16. 16. Continual Learning: Approaches T. Lesort,V. Lomonaco et al. Continual Learning for Robotics. pre-print arxiv arXiv:1907.00182 .
  17. 17. CL Framework CL Algorithm Mini-spot Robot from Boston Dynamics, 2018 T. Lesort,V. Lomonaco et al. Continual Learning for Robotics. pre-print arxiv arXiv:1907.00182 .
  18. 18. Rehearsal-free and Task-agnostic Online Continual Learning
  19. 19. 3 Short-term Research Objective for CL 1. Rehearsal-Free: Raw data cannot be stored and re-used for rehearsal. 2. Task Agnostic: No use of supplementary task supervised signal “t”. 3. Online: Bounded computational and memory overheads, efficient, real-time updates (possibly one data instance at a time). T. Lesort,V. Lomonaco et al. Continual Learning for Robotics. pre-print arxiv arXiv:1907.00182 .
  20. 20. Task Agnostic Continual Learning 1. New Instances (NI) 2. New Classes (NC) 3. New Instances and Classes (NIC) Initial Batch Incremental Batches Τ . . .
  21. 21. CORe50Website Dataset, Benchmark, code and additional information freely available at: vlomonaco.github.io/core50 LomonacoV. and Maltoni D. CORe50: a New Dataset and Benchmark for Continuous Object Recognition. CoRL2017.
  22. 22. LomonacoV. and Maltoni D. CORe50: a New Dataset and Benchmark for Continuous Object Recognition. CoRL2017. CORe50: aVideo Benchmark for CL and Object Recognition/Detection
  23. 23. # Images 164,866 Format RGB-D Image size 350x350 128x128 # Categories 10 # Obj. x Cat. 5 # Sessions 11 # img. x Sess. ~300 # Outdoor Sess. 3 Acquisition Sett. Hand held LomonacoV. and Maltoni D. CORe50: a New Dataset and Benchmark for Continuous Object Recognition. CoRL2017. CORe50: aVideo Benchmark for CL and Object Recognition/Detection
  24. 24. Fine-Grained Continual Learning LomonacoV., Maltoni D., Pellegrini L. Fine-Grained Continual Learning. Preprint arxiv arXiv:1907.03799, 2019.
  25. 25. AR-1* Rehearsal-free andTask Agnostic Online Continual Learning Maltoni D. and LomonacoV. Continuous Learning in Single-Incremental-Task Scenarios. Neural Networks Journal, 2019. LomonacoV., Maltoni D., Pellegrini L. Fine-Grained Continual Learning. Preprint arxiv arXiv:1907.03799, 2019.
  26. 26. AR-1*: Overview (with MobileNet-V1) LomonacoV., Maltoni D., Pellegrini L. Fine-Grained Continual Learning. Preprint arxiv arXiv:1907.03799, 2019.
  27. 27. AR-1*: Supervised / Unsupervised Pre-Training Phase ● Supervised or Unsupervised Pre-Training from ImageNet. ● Slowly Fine-tuned or kept fixed. ● future direction: unsupervised co-training from scratch. LomonacoV., Maltoni D., Pellegrini L. Fine-Grained Continual Learning. Preprint arxiv arXiv:1907.03799, 2019.
  28. 28. AR-1*: Regularization Phase LomonacoV., Maltoni D., Pellegrini L. Fine-Grained Continual Learning. Preprint arxiv arXiv:1907.03799, 2019.
  29. 29. AR-1*: Regularization Phase ● Computational Efficient (independent from the number of training batches) ● Just one Fisher matrix (running sum + max clip) ● Importance of Batch ReNormalization LomonacoV., Maltoni D., Pellegrini L. Fine-Grained Continual Learning. Preprint arxiv arXiv:1907.03799, 2019.
  30. 30. AR-1*: Architectural Phase ● CWR*: generalization of CWR+ to handle agnostically NI, NC and NIC settings ● Dual-Memory system for memory consolidation. ● Based on zero-init for new classes, weights consolidation and finetuning for already encountered classes. LomonacoV., Maltoni D., Pellegrini L. Fine-Grained Continual Learning. Preprint arxiv arXiv:1907.03799, 2019.
  31. 31. CORe50 - NICv2 Results ● (0%-92%) -45% avg. memory. ● (0%-94%) -49% avg. compute. ● -20% price in accuracy at the end of last batch. LomonacoV., Maltoni D., Pellegrini L. Fine-Grained Continual Learning. Preprint arxiv arXiv:1907.03799, 2019.
  32. 32. CurrentWork and Research Direction
  33. 33. Real-World Continual Learning on Embedded Systems Pellegrini L., Graffieti G. , LomonacoV. and Maltoni D. Towards Continual Learning on the Edge.To be published.
  34. 34. AR-1*: Closing the Accuracy Gap with Latent Rehearsal Pellegrini L., Graffieti G. , LomonacoV. and Maltoni D. Towards Continual Learning on the Edge.To be published.
  35. 35. AR-1*: Closing the Accuracy Gap with Latent Rehearsal Pellegrini L., Graffieti G. , LomonacoV. and Maltoni D. Towards Continual Learning on the Edge.To be published.
  36. 36. AR-1*: Sparse Representations Pellegrini L., Graffieti G. , LomonacoV. and Maltoni D. Towards Continual Learning on the Edge.To be published. ● Imposing Sparsity of the activactivation does not affect accuracy from ~55% to ~35%. ● It has been shown that sparsity may help the CL process. ● Less memory overhead for latent rehearsal.
  37. 37. FutureWorks and Research Direction 1. Latent Generative Replay 2. Lowering the amount of Supervision (Unsupervised Reinforcement Learning, Active Learning) 3. Infer or make use of the sparse “task signal” (context modulation) 4. Sequence Learning/ Temporal Coherence Integration 5. Improve robustness in real-world embedded applications (Smartphone devices, Nao Robot, …) Maltoni D. and LomonacoV. Semi-SupervisedTuning fromTemporal Coherence. ICPR 2016. LomonacoV., Desai K., Maltoni D. and Culurciello, E. Continual Reinforcement Learning in 3D non-stationary environments. preprint arxiv arXiv:1905.10112, 2019.
  38. 38. AR-1*: Closing the Accuracy Gap with Latent Generative Replay ● ● ● ● ● ● Pellegrini L., Graffieti G. , LomonacoV. and Maltoni D. Towards Continual Learning on the Edge.To be published.
  39. 39. Questions? Introduction Meetup @ Numenta 16-09-2019 Vincenzo Lomonaco vincenzo.lomonaco@unibo.it Postdoctoral Researcher @ University of Bologna Supervisor: Davide Maltoni

×