The document describes Core50, a dataset for continuous object recognition from streaming video. It contains over 164,000 RGB-D images across 10 object categories captured in 11 sessions over time. The dataset aims to facilitate research on learning approaches that can continuously learn from new data distributions without forgetting previously learned knowledge. It addresses challenges of continuous learning including new instances of known classes, new classes, and both new instances and classes.

1. Vincenzo Lomonaco, Davide Maltoni
University of Bologna

2. • Deep Learning holds state-of-the-art performances in many tasks
• Prediction models mainly trained with huge and fixed datasets.

3. • Higher and realistic time-scale where different data distributions
becomes available only during time
• No access to past perception data
• Imperative to build on top of previously learned knowledge and
skills.
• Ideal for Robotics and real-time learning from streaming
perception data.

4. • Continuous Learning needs the presence of multiple (temporal
coherent and unconstrained) views of the same objects taken in
different sessions.

5. # Images 164,866
Format RGB-D
Image size 350x350
128x128
# Categories 10
# Obj. x Cat. 5
# Sessions 11
# img. x Sess. ~300
# Outdoor Sess. 3
Acquisition Sett. Hand held
Categories
Objects

6. Approaches:
• Re-train the entire model from scratch as soon as a new batch of
data is available.
• Update the model with only the new batch of data.
𝐵𝑎𝑡𝑐ℎ0 𝐵𝑎𝑡𝑐ℎ1 𝐵𝑎𝑡𝑐ℎ 𝑛
Initial Batch Incremental Batches

. . .

7. New Instances (NI)
• New training patterns of the same classes.
New Classes (NC)
• New training patterns of different classes.
New Instances and Classes (NIC)
• New training patterns of known and new classes.

8. (NC)

9. (NI)
(NC)
(NIC)

10. Dataset, Benchmark, code and additional
information freely available at:
vlomonaco.github.io/core50
In the poster we also report
very recent results with:
1. Learning without Forgetting
(LwF)
2. ElasticWeight Consolidation
(EWC)