Humans have the extraordinary ability to learn continually from experience. Not only we can apply previously learned knowledge and skills to new situations, we can also use these as the foundation for later learning. One of the grand goals of Artificial Intelligence (AI) is building an artificial continual learning agent that constructs a sophisticated understanding of the world from its own experience through the autonomous incremental development of ever more complex knowledge and skills. However, current AI systems greatly suffer from the exposure to new data or environments which even slightly differ from the ones for which they have been trained for. Moreover, the learning process is usually constrained on fixed datasets within narrow and isolated tasks which may hardly lead to the emergence of more complex and autonomous intelligent behaviors. In essence, continual learning and adaptation capabilities, while more than often thought as fundamental pillars of every intelligent agent, have been mostly left out of the main AI research focus. In this talk, we explore the application of these ideas in the context of Vision with a focus on (deep) continual learning strategies for object recognition running at the edge on highly-constrained hardware devices.
Artificial agents interacting in highly dynamic environments are required to continually acquire and fine-tune their knowledge overtime. In contrast to conventional deep neural networks that typically rely on a large batch of annotated training samples, lifelong learning systems must account for situations in which the number of tasks is not known a priori and the data samples become incrementally available over time. Despite recent advances in deep learning, lifelong machine learning has remained a long-standing challenge due to neural networks being prone to catastrophic forgetting, i.e., the learning of new tasks interferes with previously learned ones and leads to abrupt disruptions of performance. Recently proposed deep supervised and reinforcement learning models for addressing catastrophic forgetting suffer from flexibility, robustness, and scalability issues with respect to biological systems. In this tutorial, we will present and discuss well-established and emerging neural network approaches motivated by lifelong learning factors in biological systems such as neurosynaptic plasticity, complementary memory systems, multi-task transfer learning, and intrinsically motivated exploration.
Continual Reinforcement Learning in 3D Non-stationary EnvironmentsVincenzo Lomonaco
Dynamic and always-changing environments constitute an hard challenge for current reinforcement learning techniques. Artificial agents, nowadays, are often trained in very static and reproducible conditions in simulation, where the common assumption is that observations can be sampled i.i.d from the environment. However, tackling more complex problems and real-world settings this can be rarely considered the case, with environments often non-stationary and subject to unpredictable, frequent changes. In this talk we discuss about a new open benchmark for learning continually through reinforce in a complex 3D non-stationary object picking task based on VizDoom and subject to several environmental changes. We further propose a number of end-to-end, model-free continual reinforcement learning strategies showing competitive results even without any access to previously encountered environmental conditions or observations.
Continual/Lifelong Learning with Deep ArchitecturesVincenzo Lomonaco
Humans have the extraordinary ability to learn continually from experience. Not only can we apply previously learned knowledge and skills to new situations, we can also use these as the foundation for later learning. One of the grand goals of AI is building an artificial continually learning agent that constructs a sophisticated understanding of the world from its own experience through the autonomous incremental development of ever more complex skills and knowledge.
"Continual Learning" (CL) is indeed a fast emerging topic in AI concerning the ability to efficiently improve the performance of a deep model over time, dealing with a long (and possibly unlimited) sequence of data/tasks. In this workshop, after a brief introduction of the topic, we’ll implement different Continual Learning strategies and assess them on common vision benchmarks. We’ll conclude the workshop with a look at possible real world applications of CL.
Humans have the extraordinary ability to learn continually from experience. Not only we can apply previously learned knowledge and skills to new situations, we can also use these as the foundation for later learning, constantly and efficiently updating our biased understanding of the external world. On the contrary, current AI systems are usually trained offline on huge datasets and later deployed with frozen learning capabilities as they have been shown to suffer from catastrophic forgetting if trained continuously on changing data distributions. A common, practical solution to the problem is to re-train the underlying prediction model from scratch and re-deploy it as a new batch of data becomes available. However, this naive approach is incredibly wasteful in terms of memory and computation other than impossible to sustain over longer timescales and frequent updates. In this talk, we will introduce an efficient continual learning strategy, which can reduce the amount of computation and memory overhead of more than 45% w.r.t. the standard re-train & re-deploy approach, further exploring its real-world application in the context of continual object recognition, running on the edge on highly-constrained hardware platforms such as widely adopted smartphones devices.
https://telecombcn-dl.github.io/2017-dlsl/
Winter School on Deep Learning for Speech and Language. UPC BarcelonaTech ETSETB TelecomBCN.
The aim of this course is to train students in methods of deep learning for speech and language. Recurrent Neural Networks (RNN) will be presented and analyzed in detail to understand the potential of these state of the art tools for time series processing. Engineering tips and scalability issues will be addressed to solve tasks such as machine translation, speech recognition, speech synthesis or question answering. Hands-on sessions will provide development skills so that attendees can become competent in contemporary data analytics tools.
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or text captioning.
Continual Learning is one of the most promising research areas to shift machine learning from solving a single task to something more similar to general intelligence.
Machine learning (and especially deep neural networks research) has shown outstanding results in the past 10 years, bringing us to the deep learning era, where learning models are everywhere and they interact with many aspect of our life.
However, machine learning have an enormous issue, which completely diversity it from biological learning: machine cannot learn continuously.
This is the so called catastrophic forgetting problem, and continual learning is trying to address it, making artificial intelligence able to continually learn for the entire duration of its "life".
https://telecombcn-dl.github.io/2018-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or image captioning.
Artificial agents interacting in highly dynamic environments are required to continually acquire and fine-tune their knowledge overtime. In contrast to conventional deep neural networks that typically rely on a large batch of annotated training samples, lifelong learning systems must account for situations in which the number of tasks is not known a priori and the data samples become incrementally available over time. Despite recent advances in deep learning, lifelong machine learning has remained a long-standing challenge due to neural networks being prone to catastrophic forgetting, i.e., the learning of new tasks interferes with previously learned ones and leads to abrupt disruptions of performance. Recently proposed deep supervised and reinforcement learning models for addressing catastrophic forgetting suffer from flexibility, robustness, and scalability issues with respect to biological systems. In this tutorial, we will present and discuss well-established and emerging neural network approaches motivated by lifelong learning factors in biological systems such as neurosynaptic plasticity, complementary memory systems, multi-task transfer learning, and intrinsically motivated exploration.
Continual Reinforcement Learning in 3D Non-stationary EnvironmentsVincenzo Lomonaco
Dynamic and always-changing environments constitute an hard challenge for current reinforcement learning techniques. Artificial agents, nowadays, are often trained in very static and reproducible conditions in simulation, where the common assumption is that observations can be sampled i.i.d from the environment. However, tackling more complex problems and real-world settings this can be rarely considered the case, with environments often non-stationary and subject to unpredictable, frequent changes. In this talk we discuss about a new open benchmark for learning continually through reinforce in a complex 3D non-stationary object picking task based on VizDoom and subject to several environmental changes. We further propose a number of end-to-end, model-free continual reinforcement learning strategies showing competitive results even without any access to previously encountered environmental conditions or observations.
Continual/Lifelong Learning with Deep ArchitecturesVincenzo Lomonaco
Humans have the extraordinary ability to learn continually from experience. Not only can we apply previously learned knowledge and skills to new situations, we can also use these as the foundation for later learning. One of the grand goals of AI is building an artificial continually learning agent that constructs a sophisticated understanding of the world from its own experience through the autonomous incremental development of ever more complex skills and knowledge.
"Continual Learning" (CL) is indeed a fast emerging topic in AI concerning the ability to efficiently improve the performance of a deep model over time, dealing with a long (and possibly unlimited) sequence of data/tasks. In this workshop, after a brief introduction of the topic, we’ll implement different Continual Learning strategies and assess them on common vision benchmarks. We’ll conclude the workshop with a look at possible real world applications of CL.
Humans have the extraordinary ability to learn continually from experience. Not only we can apply previously learned knowledge and skills to new situations, we can also use these as the foundation for later learning, constantly and efficiently updating our biased understanding of the external world. On the contrary, current AI systems are usually trained offline on huge datasets and later deployed with frozen learning capabilities as they have been shown to suffer from catastrophic forgetting if trained continuously on changing data distributions. A common, practical solution to the problem is to re-train the underlying prediction model from scratch and re-deploy it as a new batch of data becomes available. However, this naive approach is incredibly wasteful in terms of memory and computation other than impossible to sustain over longer timescales and frequent updates. In this talk, we will introduce an efficient continual learning strategy, which can reduce the amount of computation and memory overhead of more than 45% w.r.t. the standard re-train & re-deploy approach, further exploring its real-world application in the context of continual object recognition, running on the edge on highly-constrained hardware platforms such as widely adopted smartphones devices.
https://telecombcn-dl.github.io/2017-dlsl/
Winter School on Deep Learning for Speech and Language. UPC BarcelonaTech ETSETB TelecomBCN.
The aim of this course is to train students in methods of deep learning for speech and language. Recurrent Neural Networks (RNN) will be presented and analyzed in detail to understand the potential of these state of the art tools for time series processing. Engineering tips and scalability issues will be addressed to solve tasks such as machine translation, speech recognition, speech synthesis or question answering. Hands-on sessions will provide development skills so that attendees can become competent in contemporary data analytics tools.
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or text captioning.
Continual Learning is one of the most promising research areas to shift machine learning from solving a single task to something more similar to general intelligence.
Machine learning (and especially deep neural networks research) has shown outstanding results in the past 10 years, bringing us to the deep learning era, where learning models are everywhere and they interact with many aspect of our life.
However, machine learning have an enormous issue, which completely diversity it from biological learning: machine cannot learn continuously.
This is the so called catastrophic forgetting problem, and continual learning is trying to address it, making artificial intelligence able to continually learn for the entire duration of its "life".
https://telecombcn-dl.github.io/2018-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or image captioning.
https://mcv-m6-video.github.io/deepvideo-2020/
Self-supervised techniques define surrogate tasks to train machine learning algorithms without the need of human generated labels. This lecture reviews the state of the art in the field of computer vision, including the baseline techniques based on visual feature learning from ImageNet data.
https://mcv-m6-video.github.io/deepvideo-2020/
Self-supervised audiovisual learning exploits the synchronization between pixels and audio recorded in video files. This lecture reviews the state of the art in deep neural networks trained with this approach, which does not require any manual annotation from humans.
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
https://mcv-m6-video.github.io/deepvideo-2019/
These slides provides an overview of how deep neural networks can be used to solve an object tracking task
https://imatge-upc.github.io/unsupervised-2017-cvprw/
Lin, Xunyu, Victor Campos, Xavier Giro-i-Nieto, Jordi Torres, and Cristian Canton Ferrer. "Disentangling Motion, Foreground and Background Features in Videos." CVPR Workshops 2017. (extended abstract)
This paper introduces an unsupervised framework to extract semantically rich features for video representation. Inspired by how the human visual system groups objects based on motion cues, we propose a deep convolutional neural network that disentangles motion, foreground and background information. The proposed architecture consists of a 3D convolutional feature encoder for blocks of 16 frames, which is trained for reconstruction tasks over the first and last frames of the sequence. A preliminary supervised experiment was conducted to verify the feasibility of proposed method by training the model with a fraction of videos from the UCF-101 dataset taking as ground truth the bounding boxes around the activity regions. Qualitative results indicate that the network can successfully segment foreground and background in videos as well as update the foreground appearance based on disentangled motion features. The benefits of these learned features are shown in a discriminative classification task, where initializing the network with the proposed pretraining method outperforms both random initialization and autoencoder pretraining.
https://mcv-m6-video.github.io/deepvideo-2018/
Overview of deep learning solutions for video processing. Part of a series of slides covering topics like action recognition, action detection, object tracking, object detection, scene segmentation, language and learning from videos.
Prepared for the Master in Computer Vision Barcelona:
http://pagines.uab.cat/mcv/
A full description of the molecular autoencoder for automated exploration of chemical compound space using neural nets and machine learning architectures, developed by the Aspuru-Guzik group at Harvard. Talk given to Prof. Peter W. Chung's research group at the University of Maryland, College Park, August 2017.
Continual Learning with Deep Architectures Workshop @ Computer VISIONers Conf...Vincenzo Lomonaco
Continual Learning (CL) is a fast emerging topic in AI concerning the ability to efficiently improve the performance of a deep model over time, dealing with a long (and possibly unlimited) sequence of data/tasks. In this workshop, after a brief introduction of the subject, we’ll analyze different Continual Learning strategies and assess them on common Vision benchmarks. We’ll conclude the workshop with a look at possible real world application of CL.
Continual/Lifelong Learning with Deep Architectures, Vincenzo LomonacoData Science Milan
Humans have the extraordinary ability to learn continually from experience. Not only can we apply previously learned knowledge and skills to new situations, we can also use these as the foundation for later learning. One of the grand goals of AI is building an artificial continually learning agent that constructs a sophisticated understanding of the world from its own experience through the autonomous incremental development of ever more complex skills and knowledge.
"Continual Learning" (CL) is indeed a fast emerging topic in AI concerning the ability to efficiently improve the performance of a deep model over time, dealing with a long (and possibly unlimited) sequence of data/tasks. In this workshop, after a brief introduction of the topic, we’ll implement different Continual Learning strategies and assess them on common vision benchmarks. We’ll conclude the workshop with a look at possible real world applications of CL.
Vincenzo Lomonaco is a Deep Learning PhD student at the University of Bologna and founder of ContinualAI.org. He is also the PhD students representative at the Department of Computer Science of Engineering (DISI) and teaching assistant of the courses “Machine Learning” and “Computer Architectures” in the same department. Previously, he was a Machine Learning software engineer at IDL in-line Devices and a Master Student at the University of Bologna where he graduated cum laude in 2015 with the dissertation “Deep Learning for Computer Vision: a Comparison Between CNNs and HTMs on Object Recognition Tasks".
In the ever-evolving landscape of technology, programming has emerged as a fundamental skill, empowering individuals to create, innovate, and shape the digital world. Among the diverse array of programming languages, Python stands out as a versatile and beginner-friendly option, particularly for students eager to explore the realm of coding. With its clear syntax, extensive libraries, and active community support, Python offers a gentle yet powerful introduction to the world of programming.
CORe50: a New Dataset and Benchmark for Continuous Object Recognition PosterVincenzo Lomonaco
Continuous/Lifelong learning of high-dimensional data streams is a challenging research problem. In fact, fully retraining models each time new data become available is infeasible, due to computational and storage issues, while na\"ive incremental strategies have been shown to suffer from catastrophic forgetting. In the context of real-world object recognition applications (e.g., robotic vision), where continuous learning is crucial, very few datasets and benchmarks are available to evaluate and compare emerging techniques. In this work we propose a new dataset and benchmark CORe50, specifically designed for continuous object recognition, and introduce baseline approaches for different continuous learning scenarios.
https://mcv-m6-video.github.io/deepvideo-2020/
Self-supervised techniques define surrogate tasks to train machine learning algorithms without the need of human generated labels. This lecture reviews the state of the art in the field of computer vision, including the baseline techniques based on visual feature learning from ImageNet data.
https://mcv-m6-video.github.io/deepvideo-2020/
Self-supervised audiovisual learning exploits the synchronization between pixels and audio recorded in video files. This lecture reviews the state of the art in deep neural networks trained with this approach, which does not require any manual annotation from humans.
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
https://mcv-m6-video.github.io/deepvideo-2019/
These slides provides an overview of how deep neural networks can be used to solve an object tracking task
https://imatge-upc.github.io/unsupervised-2017-cvprw/
Lin, Xunyu, Victor Campos, Xavier Giro-i-Nieto, Jordi Torres, and Cristian Canton Ferrer. "Disentangling Motion, Foreground and Background Features in Videos." CVPR Workshops 2017. (extended abstract)
This paper introduces an unsupervised framework to extract semantically rich features for video representation. Inspired by how the human visual system groups objects based on motion cues, we propose a deep convolutional neural network that disentangles motion, foreground and background information. The proposed architecture consists of a 3D convolutional feature encoder for blocks of 16 frames, which is trained for reconstruction tasks over the first and last frames of the sequence. A preliminary supervised experiment was conducted to verify the feasibility of proposed method by training the model with a fraction of videos from the UCF-101 dataset taking as ground truth the bounding boxes around the activity regions. Qualitative results indicate that the network can successfully segment foreground and background in videos as well as update the foreground appearance based on disentangled motion features. The benefits of these learned features are shown in a discriminative classification task, where initializing the network with the proposed pretraining method outperforms both random initialization and autoencoder pretraining.
https://mcv-m6-video.github.io/deepvideo-2018/
Overview of deep learning solutions for video processing. Part of a series of slides covering topics like action recognition, action detection, object tracking, object detection, scene segmentation, language and learning from videos.
Prepared for the Master in Computer Vision Barcelona:
http://pagines.uab.cat/mcv/
A full description of the molecular autoencoder for automated exploration of chemical compound space using neural nets and machine learning architectures, developed by the Aspuru-Guzik group at Harvard. Talk given to Prof. Peter W. Chung's research group at the University of Maryland, College Park, August 2017.
Continual Learning with Deep Architectures Workshop @ Computer VISIONers Conf...Vincenzo Lomonaco
Continual Learning (CL) is a fast emerging topic in AI concerning the ability to efficiently improve the performance of a deep model over time, dealing with a long (and possibly unlimited) sequence of data/tasks. In this workshop, after a brief introduction of the subject, we’ll analyze different Continual Learning strategies and assess them on common Vision benchmarks. We’ll conclude the workshop with a look at possible real world application of CL.
Continual/Lifelong Learning with Deep Architectures, Vincenzo LomonacoData Science Milan
Humans have the extraordinary ability to learn continually from experience. Not only can we apply previously learned knowledge and skills to new situations, we can also use these as the foundation for later learning. One of the grand goals of AI is building an artificial continually learning agent that constructs a sophisticated understanding of the world from its own experience through the autonomous incremental development of ever more complex skills and knowledge.
"Continual Learning" (CL) is indeed a fast emerging topic in AI concerning the ability to efficiently improve the performance of a deep model over time, dealing with a long (and possibly unlimited) sequence of data/tasks. In this workshop, after a brief introduction of the topic, we’ll implement different Continual Learning strategies and assess them on common vision benchmarks. We’ll conclude the workshop with a look at possible real world applications of CL.
Vincenzo Lomonaco is a Deep Learning PhD student at the University of Bologna and founder of ContinualAI.org. He is also the PhD students representative at the Department of Computer Science of Engineering (DISI) and teaching assistant of the courses “Machine Learning” and “Computer Architectures” in the same department. Previously, he was a Machine Learning software engineer at IDL in-line Devices and a Master Student at the University of Bologna where he graduated cum laude in 2015 with the dissertation “Deep Learning for Computer Vision: a Comparison Between CNNs and HTMs on Object Recognition Tasks".
In the ever-evolving landscape of technology, programming has emerged as a fundamental skill, empowering individuals to create, innovate, and shape the digital world. Among the diverse array of programming languages, Python stands out as a versatile and beginner-friendly option, particularly for students eager to explore the realm of coding. With its clear syntax, extensive libraries, and active community support, Python offers a gentle yet powerful introduction to the world of programming.
CORe50: a New Dataset and Benchmark for Continuous Object Recognition PosterVincenzo Lomonaco
Continuous/Lifelong learning of high-dimensional data streams is a challenging research problem. In fact, fully retraining models each time new data become available is infeasible, due to computational and storage issues, while na\"ive incremental strategies have been shown to suffer from catastrophic forgetting. In the context of real-world object recognition applications (e.g., robotic vision), where continuous learning is crucial, very few datasets and benchmarks are available to evaluate and compare emerging techniques. In this work we propose a new dataset and benchmark CORe50, specifically designed for continuous object recognition, and introduce baseline approaches for different continuous learning scenarios.
Multimedia Information Retrieval: Bytes and pixels meet the challenges of hum...maranlar
Within computer science, "Multimedia" is a field of research that investigates how computers can support people in communication, information finding, and knowledge/opinion building. Multimedia content is defined broadly. It includes not only video, but also images accompanied by text and other information (for example, a geo-location). It can be professionally produced, or generated by users for online sharing. Computer scientists historically have a “love-hate” relationship with multimedia. They “love” it because of the richness of the data sources and the wealth of available data, which leads to interesting problems to tackle with machine learning. They “hate” it because multimedia is a diffuse and moving target: the interpretation of multimedia differs from person to person, and changes over time in the course of its use as a communication medium. This talk gives a view onto ongoing research in the area of multimedia information retrieval algorithms, which help people find multimedia. We look at a series of topics that reveal how pattern recognition, text processing, and crowdsourcing tools are used in multimedia research, and discuss both their limitations and their potential.
https://www.youtube.com/watch?v=5ZUlVlumIQo&list=PLqJzTtkUiq54DDEEZvzisPlSGp_BadhNJ&index=10
Over the last years, deep learning is rapidly advancing with impressive results obtained in several areas including computer vision, machine translation and speech recognition. Deep learning attempts to learn complex function through learning hierarchical representation of data. A deep learning model is composed of non-linear modules that each transforms the representation from lower layer to the higher more abstract one. Very complex functions can be learned using enough composition of the non-linear modules. Furthermore, the need for manual feature engineering can be obviated by learning features themselves through the representation learning. In this talk, we first explain how deep learning architecture in particular and neural networks in general are loosely inspired by mammalian visual cortex and nervous system respectively. We also discuss about the reason for big and successful comeback of neural networks with the deep learning models. Finally, we give a brief introduction of various deep structures and their applications to several domains.
References:
LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. "Deep learning." Nature 521.7553 (2015): 436-444.
Socher, Richard, Yoshua Bengio, and Chris Manning. "Deep learning for NLP." Tutorial at Association of Computational Logistics (ACL), 2012, and North American Chapter of the Association of Computational Linguistics (NAACL) (2013).
Lee, Honglak. "Tutorial on deep learning and applications." NIPS 2010 Workshop on Deep Learning and Unsupervised Feature Learning. 2010.
LeCun, Yann, and M. Ranzato. "Deep learning tutorial." Tutorials in International Conference on Machine Learning (ICML’13). 2013.
Socher, Richard, et al. "Recursive deep models for semantic compositionality over a sentiment treebank." Proceedings of the conference on empirical methods in natural language processing (EMNLP). Vol. 1631. 2013.
https://www.youtube.com/channel/UC9OeZkIwhzfv-_Cb7fCikLQ
https://www.udacity.com/course/deep-learning--ud730
http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/
Deep Learning for Computer Vision: A comparision between Convolutional Neural...Vincenzo Lomonaco
In recent years, Deep Learning techniques have shown to perform well on a large variety of problems both in Computer Vision and Natural Language Processing, reaching and often surpassing the state of the art on many tasks. The rise of deep learning is also revolutionizing the entire field of Machine Learning and Pattern Recognition pushing forward the concepts of automatic feature extraction and unsupervised learning in general.
However, despite the strong success both in science and business, deep learning has its own limitations. It is often questioned if such techniques are only some kind of brute-force statistical approaches and if they can only work in the context of High Performance Computing with tons of data. Another important question is whether they are really biologically inspired, as claimed in certain cases, and if they can scale well in terms of “intelligence”.
The dissertation is focused on trying to answer these key questions in the context of Computer Vision and, in particular, Object Recognition, a task that has been heavily revolutionized by recent advances in the field. Practically speaking, these answers are based on an exhaustive comparison between two, very different, deep learning techniques on the aforementioned task: Convolutional Neural Network (CNN) and Hierarchical Temporal memory (HTM). They stand for two different approaches and points of view within the big hat of deep learning and are the best choices to understand and point out strengths and weaknesses of each of them.
CNN is considered one of the most classic and powerful supervised methods used today in machine learning and pattern recognition, especially in object recognition. CNNs are well received and accepted by the scientific community and are already deployed in large corporation like Google and Facebook for solving face recognition and image auto-tagging problems.
HTM, on the other hand, is known as a new emerging paradigm and a new meanly-unsupervised method, that is more biologically inspired. It tries to gain more insights from the computational neuroscience community in order to incorporate concepts like time, context and attention during the learning process which are typical of the human brain.
In the end, the thesis is supposed to prove that in certain cases, with a lower quantity of data, HTM can outperform CNN.
e-Sim approach at Universidade Aberta: presence, narrative and agency for imm...Leonel Morgado
Slides for the PROF XXI Global Symposium MOOC Lx23. An introduction to the e-Sim (also known as e-SimProgramming) approach used in my software development course at Universidade Aberta - links to research and things to come... including GPT artificial intelligences.
Herramientas 2.0 para el desarrollo de competencias profesionalizadorasIsmael Peña-López
Comuniación en la II Jornada sobre docencia del Derecho y Tecnologías de la Información y la Comunicación, 6 de junio de 2011
Más información http://ictlogy.net/?p=3760
Bayesian networks (BNs) are a powerful tool for graphical representation of the underlying knowledge in the data and reasoning with incomplete or imprecise observations.
BNs have been extended (or generalized) in several ways, as for instance, causal BNs, dynamic BNs, Relational BNs, ...
In this lecture, we will focus on Bayesian network learning.
BN learning can differ with respect to the task : generative model versus discriminative one ? Then, the learning task can also differ w.r.t the nature of the data : complete data, incomplete data, non i.i.d data, number of variables >> number of samples, data stream, presence of prior knowledge ...
Given the diversity of these problems, many approaches have emerged in the literature. I will present a brief panorama of those algorithms and describe our current works in this field, with works about BN structure learning, dynamic BN structure learning and relational BN structure learning.
Hologram Lecturers and Tele-Presence Teachers in the Next DimensionZac Woolfitt
Just because you cannot travel to a university to give a lecture, does not mean you can’t be there ‘in person’. Students can still benefit from your expertise via two potential remote presence educational formats.
1 – Remote Presence Robot
2 – The Lecturer as Hologram.
From a teaching and learning perspective each format has its own strengths and unique affordances. By developing our understanding of the pedagogical potential, we can leverage these distinct elements to enhance learning and create new opportunities for education.
How credible are the as teaching formats of the future? Examining these innovative modes of remote teaching gives us a new position from which to reflect on our traditional face-to-face teaching. Not only do we open our mind to new possibilities, but we gain a deeper understanding of the core-essence of teaching and learning. Current circumstances did not allow us to demonstrate these formats on the stage of the OEB. But there was still room for a lively discussion about the educational possibilities of virtual presence teaching.
Similar to Continual Learning: Another Step Towards Truly Intelligent Machines (20)
The Deep Continual Learning community should move beyond studying forgetting in Class-Incremental Learning Scenarios! In this tutorial we gave at
#CoLLAs2023, me and Antonio Carta try to explain why and how! 👇
Do you agree?
Continual Learning with Deep Architectures - Tutorial ICML 2021Vincenzo Lomonaco
Humans have the extraordinary ability to learn continually from experience. Not only we can apply previously learned knowledge and skills to new situations, we can also use these as the foundation for later learning. One of the grand goals of Artificial Intelligence (AI) is building an artificial “continual learning” agent that constructs a sophisticated understanding of the world from its own experience through the autonomous incremental development of ever more complex knowledge and skills (Parisi, 2019). However, despite early speculations and few pioneering works (Ring, 1998; Thrun, 1998; Carlson, 2010), very little research and effort has been devoted to address this vision. Current AI systems greatly suffer from the exposure to new data or environments which even slightly differ from the ones for which they have been trained for (Goodfellow, 2013). Moreover, the learning process is usually constrained on fixed datasets within narrow and isolated tasks which may hardly lead to the emergence of more complex and autonomous intelligent behaviors. In essence, continual learning and adaptation capabilities, while more than often thought as fundamental pillars of every intelligent agent, have been mostly left out of the main AI research focus.
In this tutorial, we propose to summarize the application of these ideas in light of the more recent advances in machine learning research and in the context of deep architectures for AI (Lomonaco, 2019). Starting from a motivation and a brief history, we link recent Continual Learning advances to previous research endeavours on related topics and we summarize the state-of-the-art in terms of major approaches, benchmarks and key results. In the second part of the tutorial we plan to cover more exploratory studies about Continual Learning with low supervised signals and the relationships with other paradigms such as Unsupervised, Semi-Supervised and Reinforcement Learning. We will also highlight the impact of recent Neuroscience discoveries in the design of original continual learning algorithms as well as their deployment in real-world applications. Finally, we will underline the notion of continual learning as a key technological enabler for Sustainable Machine Learning and its societal impact, as well as recap interesting research questions and directions worth addressing in the future.
Authors: Vincenzo Lomonaco, Irina Rish
Official Website: https://sites.google.com/view/cltutorial-icml2021
Humans have the extraordinary ability to learn continually from experience. Not only we can apply previously learned knowledge and skills to new situations, we can also use these as the foundation for later learning. One of the grand goals of Artificial Intelligence (AI) is building an artificial continual learning agent that constructs a sophisticated understanding of the world from its own experience through the autonomous incremental development of ever more complex knowledge and skills. However, current AI systems greatly suffer from the exposure to new data or environments which even slightly differ from the ones for which they have been trained for. Moreover, the learning process is usually constrained on fixed datasets within narrow and isolated tasks which may hardly lead to the emergence of more complex and autonomous intelligent behaviors. In essence, continual learning and adaptation capabilities, while more than often thought as fundamental pillars of every intelligent agent, have been mostly left out of the main AI research focus. In this talk, we explore the application of these ideas in the context of Robotics with a focus on (deep) continual learning strategies for object recognition running at the edge on highly-constrained hardware devices.
Don't forget, there is more than forgetting: new metrics for Continual Learni...Vincenzo Lomonaco
Continual learning consists of algorithms that learn from a stream of data/tasks continuously and adaptively thought time, enabling the incremental development of ever more complex knowledge and skills. The lack of consensus in evaluating continual learning algorithms and the almost exclusive focus on forgetting motivate us to propose a more comprehensive set of implementation independent metrics accounting for several factors we believe have practical implications worth considering in the deployment of real AI systems that learn continually: accuracy or performance over time, backward and forward knowledge transfer, memory overhead as well as computational efficiency. Drawing inspiration from the standard Multi-Attribute Value Theory (MAVT) we further propose to fuse these metrics into a single score for ranking purposes and we evaluate our proposal with five continual learning strategies on the iCIFAR-100 continual learning benchmark.
Open-Source Frameworks for Deep Learning: an OverviewVincenzo Lomonaco
The rise of deep learning over the last decade has led to profound changes in the landscape of the machine learning software stack both for research and production. In this talk we will provide a comprehensive overview of the open-source deep learning frameworks landscape with both a theoretical and hands-on approach. After a brief introduction and historical contextualization, we will highlight common features and distinctions of their recent developments. Finally, we will take at deeper look into three of the most used deep learning frameworks today: Caffe, Tensorflow, PyTorch; with practical examples and considerations worth reckoning in the choice of such libraries.
CORe50: a New Dataset and Benchmark for Continual Learning and Object Recogni...Vincenzo Lomonaco
Continuous/Lifelong learning of high-dimensional data streams is a challenging research problem. In fact, fully retraining models each time new data become available is infeasible, due to computational and storage issues, while na\"ive incremental strategies have been shown to suffer from catastrophic forgetting. In the context of real-world object recognition applications (e.g., robotic vision), where continuous learning is crucial, very few datasets and benchmarks are available to evaluate and compare emerging techniques. In this work we propose a new dataset and benchmark CORe50, specifically designed for continuous object recognition, and introduce baseline approaches for different continuous learning scenarios.
One of the greatest goals of AI is building an artificial continuous learning agent which can construct a sophisticated understanding about the external world from its own experience through the adaptive, goal-oriented and incremental development of ever more complex skills and knowledge. Yet, Continuous/Lifelong Learning (CL) from high-dimensional streaming data is a challenging research problem far from being solved. In fact, fully retraining deep prediction models each time a new piece of data becomes available is infeasible, due to computational and storage issues, while naïve continuous learning strategies have been shown to suffer from catastrophic forgetting. This talk will cover some of the most common end-to-end continuous learning strategies for gradient-based architectures and the recently proposed AR-1 strategy, which can outperform other state-of-the-art regularization and architectural approaches on the CORe50 benchmark.
Continuous Unsupervised Training of Deep ArchitecturesVincenzo Lomonaco
A number of successful Computer Vision applications have been recently proposed based on Convolutional Networks. However, in most of the cases the system is fully supervised, the training set is fixed and the task completely defined a priori. Even though Transfer Learning approaches proved to be very useful to adapt heavily pre-trained models to ever-changing scenarios, the incremental learning and adaptation capabilities of existing models is still limited and catastrophic forgetting very difficult to control. In this talk we will discuss our experience in the design of deep architectures and algorithms capable of learning objects incrementally both in a supervised and unsupervised way. Finally we will introduce a new dataset and benchmark (CORe50) that we specifically collected to focus on continuous object recognition for Robotic Vision.
Comparing Incremental Learning Strategies for Convolutional Neural NetworksVincenzo Lomonaco
In the last decade, Convolutional Neural Networks (CNNs) have shown to perform incredibly well in many computer vision tasks such as object recognition and object detection, being able to extract meaningful high-level invariant features. However, partly because of their complex training and tricky hyper-parameters tuning, CNNs have been scarcely studied in the context of incremental learning where data are available in consecutive batches and retraining the model from scratch is unfeasible. In this work we compare different incremental learning strategies for CNN based architectures, targeting real-word applications.
If you are interested in this work please cite:
Lomonaco, V., & Maltoni, D. (2016, September). Comparing Incremental Learning Strategies for Convolutional Neural Networks. In IAPR Workshop on Artificial Neural Networks in Pattern Recognition (pp. 175-184). Springer International Publishing.
For further information visit my website: http://www.vincenzolomonaco.com/
Deep Learning for Computer Vision: A comparision between Convolutional Neural...Vincenzo Lomonaco
In recent years, Deep Learning techniques have shown to perform well on a large variety of problems both in Computer Vision and Natural Language Processing, reaching and often surpassing the state of the art on many tasks. The rise of deep learning is also revolutionizing the entire field of Machine Learning and Pattern Recognition pushing forward the concepts of automatic feature extraction and unsupervised learning in general.
However, despite the strong success both in science and business, deep learning has its own limitations. It is often questioned if such techniques are only some kind of brute-force statistical approaches and if they can only work in the context of High Performance Computing with tons of data. Another important question is whether they are really biologically inspired, as claimed in certain cases, and if they can scale well in terms of “intelligence”.
The dissertation is focused on trying to answer these key questions in the context of Computer Vision and, in particular, Object Recognition, a task that has been heavily revolutionized by recent advances in the field. Practically speaking, these answers are based on an exhaustive comparison between two, very different, deep learning techniques on the aforementioned task: Convolutional Neural Network (CNN) and Hierarchical Temporal memory (HTM). They stand for two different approaches and points of view within the big hat of deep learning and are the best choices to understand and point out strengths and weaknesses of each of them.
CNN is considered one of the most classic and powerful supervised methods used today in machine learning and pattern recognition, especially in object recognition. CNNs are well received and accepted by the scientific community and are already deployed in large corporation like Google and Facebook for solving face recognition and image auto-tagging problems.
HTM, on the other hand, is known as a new emerging paradigm and a new meanly-unsupervised method, that is more biologically inspired. It tries to gain more insights from the computational neuroscience community in order to incorporate concepts like time, context and attention during the learning process which are typical of the human brain.
In the end, the thesis is supposed to prove that in certain cases, with a lower quantity of data, HTM can outperform CNN.
In this work we started to develop a novel framework for statically detecting deadlocks in a concurrent Java environment with asynchronous method calls and cooperative scheduling of method activations. Since this language features recursion and dynamic resource creation, dead-
lock detection is extremely complex and state-of-the-art solutions either give imprecise answers or do not scale. The basic component of the framework is a front-end inference algorithm that ex-
tracts abstract behavioral descriptions of methods, called contracts, which retain resource dependency information. This component is integrated with a back-end that analyze contracts and derive deadlock information computing a fixpoint semantics.
Deep Learning libraries and first experiments with TheanoVincenzo Lomonaco
In recent years, neural networks and deep learning techniques have shown to perform well on many
problems in image recognition, speech recognition, natural language processing and many other tasks.
As a result, a large number of libraries, toolkits and frameworks came out in different languages and
with different purposes. In this report, firstly we take a look at these projects and secondly we choose the
framework that best suits our needs: Theano. Eventually, we implement a simple convolutional neural net
using this framework to test both its ease-of-use and efficiency.
Word2vec on the italian language: first experimentsVincenzo Lomonaco
Word2vec model and application by Mikolov et al. have attracted a great amount of attention in recent years. The vector representations of words learned by word2vec models have been proven to be able to carry semantic meanings and are useful in various NLP tasks. In this work I try to reproduce the previously obtained results for the English language and to explore the possibility of doing the same for the Italian language.
The increased availability of biomedical data, particularly in the public domain, offers the opportunity to better understand human health and to develop effective therapeutics for a wide range of unmet medical needs. However, data scientists remain stymied by the fact that data remain hard to find and to productively reuse because data and their metadata i) are wholly inaccessible, ii) are in non-standard or incompatible representations, iii) do not conform to community standards, and iv) have unclear or highly restricted terms and conditions that preclude legitimate reuse. These limitations require a rethink on data can be made machine and AI-ready - the key motivation behind the FAIR Guiding Principles. Concurrently, while recent efforts have explored the use of deep learning to fuse disparate data into predictive models for a wide range of biomedical applications, these models often fail even when the correct answer is already known, and fail to explain individual predictions in terms that data scientists can appreciate. These limitations suggest that new methods to produce practical artificial intelligence are still needed.
In this talk, I will discuss our work in (1) building an integrative knowledge infrastructure to prepare FAIR and "AI-ready" data and services along with (2) neurosymbolic AI methods to improve the quality of predictions and to generate plausible explanations. Attention is given to standards, platforms, and methods to wrangle knowledge into simple, but effective semantic and latent representations, and to make these available into standards-compliant and discoverable interfaces that can be used in model building, validation, and explanation. Our work, and those of others in the field, creates a baseline for building trustworthy and easy to deploy AI models in biomedicine.
Bio
Dr. Michel Dumontier is the Distinguished Professor of Data Science at Maastricht University, founder and executive director of the Institute of Data Science, and co-founder of the FAIR (Findable, Accessible, Interoperable and Reusable) data principles. His research explores socio-technological approaches for responsible discovery science, which includes collaborative multi-modal knowledge graphs, privacy-preserving distributed data mining, and AI methods for drug discovery and personalized medicine. His work is supported through the Dutch National Research Agenda, the Netherlands Organisation for Scientific Research, Horizon Europe, the European Open Science Cloud, the US National Institutes of Health, and a Marie-Curie Innovative Training Network. He is the editor-in-chief for the journal Data Science and is internationally recognized for his contributions in bioinformatics, biomedical informatics, and semantic technologies including ontologies and linked data.
Slide 1: Title Slide
Extrachromosomal Inheritance
Slide 2: Introduction to Extrachromosomal Inheritance
Definition: Extrachromosomal inheritance refers to the transmission of genetic material that is not found within the nucleus.
Key Components: Involves genes located in mitochondria, chloroplasts, and plasmids.
Slide 3: Mitochondrial Inheritance
Mitochondria: Organelles responsible for energy production.
Mitochondrial DNA (mtDNA): Circular DNA molecule found in mitochondria.
Inheritance Pattern: Maternally inherited, meaning it is passed from mothers to all their offspring.
Diseases: Examples include Leber’s hereditary optic neuropathy (LHON) and mitochondrial myopathy.
Slide 4: Chloroplast Inheritance
Chloroplasts: Organelles responsible for photosynthesis in plants.
Chloroplast DNA (cpDNA): Circular DNA molecule found in chloroplasts.
Inheritance Pattern: Often maternally inherited in most plants, but can vary in some species.
Examples: Variegation in plants, where leaf color patterns are determined by chloroplast DNA.
Slide 5: Plasmid Inheritance
Plasmids: Small, circular DNA molecules found in bacteria and some eukaryotes.
Features: Can carry antibiotic resistance genes and can be transferred between cells through processes like conjugation.
Significance: Important in biotechnology for gene cloning and genetic engineering.
Slide 6: Mechanisms of Extrachromosomal Inheritance
Non-Mendelian Patterns: Do not follow Mendel’s laws of inheritance.
Cytoplasmic Segregation: During cell division, organelles like mitochondria and chloroplasts are randomly distributed to daughter cells.
Heteroplasmy: Presence of more than one type of organellar genome within a cell, leading to variation in expression.
Slide 7: Examples of Extrachromosomal Inheritance
Four O’clock Plant (Mirabilis jalapa): Shows variegated leaves due to different cpDNA in leaf cells.
Petite Mutants in Yeast: Result from mutations in mitochondrial DNA affecting respiration.
Slide 8: Importance of Extrachromosomal Inheritance
Evolution: Provides insight into the evolution of eukaryotic cells.
Medicine: Understanding mitochondrial inheritance helps in diagnosing and treating mitochondrial diseases.
Agriculture: Chloroplast inheritance can be used in plant breeding and genetic modification.
Slide 9: Recent Research and Advances
Gene Editing: Techniques like CRISPR-Cas9 are being used to edit mitochondrial and chloroplast DNA.
Therapies: Development of mitochondrial replacement therapy (MRT) for preventing mitochondrial diseases.
Slide 10: Conclusion
Summary: Extrachromosomal inheritance involves the transmission of genetic material outside the nucleus and plays a crucial role in genetics, medicine, and biotechnology.
Future Directions: Continued research and technological advancements hold promise for new treatments and applications.
Slide 11: Questions and Discussion
Invite Audience: Open the floor for any questions or further discussion on the topic.
A brief information about the SCOP protein database used in bioinformatics.
The Structural Classification of Proteins (SCOP) database is a comprehensive and authoritative resource for the structural and evolutionary relationships of proteins. It provides a detailed and curated classification of protein structures, grouping them into families, superfamilies, and folds based on their structural and sequence similarities.
Professional air quality monitoring systems provide immediate, on-site data for analysis, compliance, and decision-making.
Monitor common gases, weather parameters, particulates.
(May 29th, 2024) Advancements in Intravital Microscopy- Insights for Preclini...Scintica Instrumentation
Intravital microscopy (IVM) is a powerful tool utilized to study cellular behavior over time and space in vivo. Much of our understanding of cell biology has been accomplished using various in vitro and ex vivo methods; however, these studies do not necessarily reflect the natural dynamics of biological processes. Unlike traditional cell culture or fixed tissue imaging, IVM allows for the ultra-fast high-resolution imaging of cellular processes over time and space and were studied in its natural environment. Real-time visualization of biological processes in the context of an intact organism helps maintain physiological relevance and provide insights into the progression of disease, response to treatments or developmental processes.
In this webinar we give an overview of advanced applications of the IVM system in preclinical research. IVIM technology is a provider of all-in-one intravital microscopy systems and solutions optimized for in vivo imaging of live animal models at sub-micron resolution. The system’s unique features and user-friendly software enables researchers to probe fast dynamic biological processes such as immune cell tracking, cell-cell interaction as well as vascularization and tumor metastasis with exceptional detail. This webinar will also give an overview of IVM being utilized in drug development, offering a view into the intricate interaction between drugs/nanoparticles and tissues in vivo and allows for the evaluation of therapeutic intervention in a variety of tissues and organs. This interdisciplinary collaboration continues to drive the advancements of novel therapeutic strategies.
THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.Sérgio Sacani
The return of a sample of near-surface atmosphere from Mars would facilitate answers to several first-order science questions surrounding the formation and evolution of the planet. One of the important aspects of terrestrial planet formation in general is the role that primary atmospheres played in influencing the chemistry and structure of the planets and their antecedents. Studies of the martian atmosphere can be used to investigate the role of a primary atmosphere in its history. Atmosphere samples would also inform our understanding of the near-surface chemistry of the planet, and ultimately the prospects for life. High-precision isotopic analyses of constituent gases are needed to address these questions, requiring that the analyses are made on returned samples rather than in situ.
Richard's aventures in two entangled wonderlandsRichard Gill
Since the loophole-free Bell experiments of 2020 and the Nobel prizes in physics of 2022, critics of Bell's work have retreated to the fortress of super-determinism. Now, super-determinism is a derogatory word - it just means "determinism". Palmer, Hance and Hossenfelder argue that quantum mechanics and determinism are not incompatible, using a sophisticated mathematical construction based on a subtle thinning of allowed states and measurements in quantum mechanics, such that what is left appears to make Bell's argument fail, without altering the empirical predictions of quantum mechanics. I think however that it is a smoke screen, and the slogan "lost in math" comes to my mind. I will discuss some other recent disproofs of Bell's theorem using the language of causality based on causal graphs. Causal thinking is also central to law and justice. I will mention surprising connections to my work on serial killer nurse cases, in particular the Dutch case of Lucia de Berk and the current UK case of Lucy Letby.
Continual Learning: Another Step Towards Truly Intelligent Machines
1. Continual Learning: Another Step
TowardsTruly Intelligent Machines
Introduction Meetup @ Numenta
16-09-2019
Vincenzo Lomonaco
vincenzo.lomonaco@unibo.it
Postdoctoral Researcher @ University of Bologna
Supervisor: Davide Maltoni
2. About me
• Post-Doc @ University of Bologna
• Research Affiliate @ AI Labs
• Teaching Assistant of the courses
Machine Learning and Computer
Architectures @ UniBo
• Author andTechnical reviewer of the
online course Deep Learning with R and
book R Deep Learning Essentials.
• Co-Founder and President of
ContinualAI.org
• Co-Founder and Board Member of Data
Science Bologna and AIforPeople.org
3. What’s ContinualAI?
• ContinualAI is a non-profit research organization and
the largest research community on Continual Learning
for AI.
• It counts more than 550+ members in 17 different
time-zones and from top-notch research institutions.
• Learn more about ContinualAI at www.continualai.org
6. Outline
1. Personal ResearchTrajectory andVision
2. Continual Learning: State-of-the-art
3. Rehearsal-free and Task-agnostic
Online Continual Learning
4. CurrentWork and Research Direction
8. ResearchTrajectory andVision
I meet Davide Maltoni
who was working at
HTMs from 2011.
I read “On
Intelligence” and join
his quest for
understanding
intelligence and build it
in silicon.
MasterThesis Published:
“Comparing HTMs and CNNs
on Object RecognitionTasks”
2014
Visiting Scholar at Purdue
University.
Working on Continual
Reinforcement /
Unsupervised Learning.
Visiting Scholar at ENSTA
ParisTech.
Working on Continual for
Robotics and a more
comprehensive CL
framework definition.
2015 2017 2018
I defend my PhD
Dissertation “Continual
Learning with Deep
Architectures”.
Putting everything
together.
Post-Doc @ UniBo
on the same topic.
2019
We abandon HTM (1st Gen.) to
work on top of deep learning
directly with a focus on
Continual Learning.
In particular, on Continual
Learning from video sequences.
2016
Long-term vision: “Understand the key computational
principles of intelligence and build truly intelligent machines.”
Main research goal: “Closing the gap between the HTM
theory and current AI systems.”
9. OurWorks with HTMs (1st Gen.)
1. D. Maltoni, Pattern Recognition by HierarchicalTemporal
Memory,Technical Report, DEIS - University of Bologna technical
report, April 2011.
2. D. Maltoni and E.M. Rehn, Incremental Learning by Message
Passing in HierarchicalTemporal Memory in 5thWorkshop on
Artificial Neural Networks in Pattern Recognition (ANNPR12),
Trento (Italy), pp.24-35, September 2012.
3. E.M. Rehn and D. Maltoni, Incremental Learning by Message
Passing in HierarchicalTemporal Memory, Neural Computation,
vol.26, no.8, pp.1763-1809, August 2014.
4. D. Maltoni andV. Lomonaco, Semi-supervisedTuning from
Temporal Coherence, in International Conference on Pattern
Recognition (ICPR16), Cancun, Mexico, December 2016.
17. CL Framework
CL Algorithm
Mini-spot Robot from Boston Dynamics, 2018
T. Lesort,V. Lomonaco et al. Continual Learning for Robotics. pre-print arxiv arXiv:1907.00182 .
19. 3 Short-term Research Objective for CL
1. Rehearsal-Free: Raw data cannot be stored and re-used
for rehearsal.
2. Task Agnostic: No use of supplementary task supervised
signal “t”.
3. Online: Bounded computational and memory
overheads, efficient, real-time updates (possibly one
data instance at a time).
T. Lesort,V. Lomonaco et al. Continual Learning for Robotics. pre-print arxiv arXiv:1907.00182 .
20. Task Agnostic Continual Learning
1. New Instances (NI)
2. New Classes (NC)
3. New Instances and Classes (NIC)
Initial Batch Incremental Batches
Τ
. . .
21. CORe50Website
Dataset, Benchmark, code and additional
information freely available at:
vlomonaco.github.io/core50
LomonacoV. and Maltoni D. CORe50: a New Dataset and Benchmark for Continuous Object Recognition. CoRL2017.
22. LomonacoV. and Maltoni D. CORe50: a New Dataset and Benchmark for Continuous Object Recognition. CoRL2017.
CORe50: aVideo Benchmark for CL
and Object Recognition/Detection
23. # Images 164,866
Format RGB-D
Image size 350x350
128x128
# Categories 10
# Obj. x Cat. 5
# Sessions 11
# img. x Sess. ~300
# Outdoor Sess. 3
Acquisition Sett. Hand held
LomonacoV. and Maltoni D. CORe50: a New Dataset and Benchmark for Continuous Object Recognition. CoRL2017.
CORe50: aVideo Benchmark for CL
and Object Recognition/Detection
29. AR-1*: Regularization Phase
● Computational
Efficient (independent
from the number of
training batches)
● Just one Fisher matrix
(running sum + max
clip)
● Importance of Batch
ReNormalization
LomonacoV., Maltoni D., Pellegrini L. Fine-Grained Continual Learning. Preprint arxiv arXiv:1907.03799, 2019.
30. AR-1*: Architectural Phase
● CWR*: generalization of
CWR+ to handle
agnostically NI, NC and
NIC settings
● Dual-Memory system for
memory consolidation.
● Based on zero-init for new
classes, weights
consolidation and
finetuning for already
encountered classes.
LomonacoV., Maltoni D., Pellegrini L. Fine-Grained Continual Learning. Preprint arxiv arXiv:1907.03799, 2019.
31. CORe50 - NICv2 Results
● (0%-92%) -45% avg. memory.
● (0%-94%) -49% avg. compute.
● -20% price in accuracy at
the end of last batch.
LomonacoV., Maltoni D., Pellegrini L. Fine-Grained Continual Learning. Preprint arxiv arXiv:1907.03799, 2019.
33. Real-World Continual Learning on
Embedded Systems
Pellegrini L., Graffieti G. , LomonacoV. and Maltoni D. Towards Continual Learning on the Edge.To be published.
34. AR-1*: Closing the Accuracy Gap with
Latent Rehearsal
Pellegrini L., Graffieti G. , LomonacoV. and Maltoni D. Towards Continual Learning on the Edge.To be published.
35. AR-1*: Closing the Accuracy Gap with
Latent Rehearsal
Pellegrini L., Graffieti G. , LomonacoV. and Maltoni D. Towards Continual Learning on the Edge.To be published.
36. AR-1*: Sparse Representations
Pellegrini L., Graffieti G. , LomonacoV. and Maltoni D. Towards Continual Learning on the Edge.To be published.
● Imposing Sparsity of the
activactivation does not
affect accuracy from
~55% to ~35%.
● It has been shown that
sparsity may help the CL
process.
● Less memory overhead for
latent rehearsal.
37. FutureWorks and Research Direction
1. Latent Generative Replay
2. Lowering the amount of Supervision (Unsupervised
Reinforcement Learning, Active Learning)
3. Infer or make use of the sparse “task signal” (context
modulation)
4. Sequence Learning/ Temporal Coherence Integration
5. Improve robustness in real-world embedded
applications (Smartphone devices, Nao Robot, …)
Maltoni D. and LomonacoV. Semi-SupervisedTuning fromTemporal Coherence. ICPR 2016.
LomonacoV., Desai K., Maltoni D. and Culurciello, E. Continual Reinforcement Learning in 3D non-stationary
environments. preprint arxiv arXiv:1905.10112, 2019.
38. AR-1*: Closing the Accuracy Gap with
Latent Generative Replay
●
●
●
●
●
●
Pellegrini L., Graffieti G. , LomonacoV. and Maltoni D. Towards Continual Learning on the Edge.To be published.
39. Questions?
Introduction Meetup @ Numenta
16-09-2019
Vincenzo Lomonaco
vincenzo.lomonaco@unibo.it
Postdoctoral Researcher @ University of Bologna
Supervisor: Davide Maltoni