Continuous/Lifelong learning of high-dimensional data streams is a challenging research problem. In fact, fully retraining models each time new data become available is infeasible, due to computational and storage issues, while na\"ive incremental strategies have been shown to suffer from catastrophic forgetting. In the context of real-world object recognition applications (e.g., robotic vision), where continuous learning is crucial, very few datasets and benchmarks are available to evaluate and compare emerging techniques. In this work we propose a new dataset and benchmark CORe50, specifically designed for continuous object recognition, and introduce baseline approaches for different continuous learning scenarios.

1. CORe50: a New Dataset and Benchmark
for Continuous Object Recognition
Vincenzo Lomonaco & Davide Maltoni
DISI - University of Bologna, Italy
Davide Maltoni and Vincenzo Lomonaco
DISI - University of Bologna
Emails: {vincenzo.lomonaco, davide.maltoni}@unibo.it
Websites: www.vincenzolomonaco.com, http://bias.csr.unibo.it/maltoni/
Contacts
[1] Maltoni Davide and Lomonaco Vincenzo. “Semi-supervised Tuning from Temporal Coherence”. 23rd
International Conference on Pattern Recognition, 2016.
[2] Lomonaco Vincenzo and Maltoni Davide. “Comparing Incremental Learning Strategies for
Convolutional Neural Networks”. ANNPR Workshop. Springer International Publishing, 2016.
References
Continuous/Lifelong learning of high-dimensional data streams is a challenging
research problem. In fact, fully retraining models each time new data become
available is infeasible, due to computational and storage issues, while naïve
incremental strategies have been shown to suffer from catastrophic forgetting.
In this work we propose a new Dataset and Benchmark CORe50, specifically
designed for assessing Continuous Learning strategies and robotic vision
applications.
Biological learning does not require to store perceptual data streams and re-
train the prediction model from scratch as current Deep Learning algorithms.
On the contrary, it excels at processing data continuously and incrementally.
With this work we aimed at providing the Deep Learning and Robotics
communities with a new Dataset and Benchmark for accelerating the process
of closing this gap.
Through our baselines, we have shown that contrasting forgetting is possible in
many ways yet leaving much room for future improvements. Check out our live
website for constantly updated versions!
Why CORe50? Baseline Strategies and Results
Specifically Designed Dataset
3-way Benchmark
Large room for improvements!
https://vlomonaco.github.io/core50/
CORe50
𝐵𝑎𝑡𝑐ℎ0 𝐵𝑎𝑡𝑐ℎ1 𝐵𝑎𝑡𝑐ℎ 𝑛
. . .
Initial Batch Incremental Batches

New Instances (NI)
New training patterns of
the same classes
becomes available
during time.
New Classes (NC)
New training patterns of
different classes
becomes available
during time.
New Instances and
Classes (NIC)
New training patterns of
both known and unknown
classes becomes available
during time.
Common Continuous Learning Context
Three Different Scenarios of Incremental Difficulty
a)
b)
c)
Figure 1. a) The 50 different objects (categories on column) of CORe50. b) Example of 1 sec. recording at 20fps of
object #26 in session #4. c) One frame of the same object (#41) throughout the 11 acquisition sessions. d) Dataset
details and figures.
Baseline Strategies
Cumulative
Training patterns are
accumulated and the
model is re-trained from
scratch for each
incremental batch.
Naive
Stochastic Gradient Descent
(SGD) is continued on each
batch. Previously
encountered training data
are discarded.
CWR
Weights are kept frozen up to
conv5, fc layers removed. For
each batch output units are
trained and then transferred
into a “consolidated”
network.
Results on the Three Scenarios NI, NC and NIC
NI
LWF
Before starting training on the new
batch, surrogate labels for a
“multi-task” loss are computed,
hence enforcing output stability
among incremental batches.
EWC
Regularization technique based on
the penalization of changes in the
weights carrying higher Fisher
Information computed at the end
of the training of each batch.
https://vlomonaco.github.io/core50/
NC
NIC
# Images Format Image
size
# Cat. # Obj. x
Cat.
# Session # img. x
Session
# Outdoor
Session
Acquisition
Setting
164,866 RGB-D 350x350
128x128
10 5 11 ~300 3 Hand held
d)