In the last decade, Convolutional Neural Networks (CNNs) have shown to perform incredibly well on face recognition tasks being able to deal with large occlusions, extremely low resolutions, strong illumination variations, etc. However, partly because of their complex training and tricky hyper-parameters tuning, CNNs have been scarcely studied in the context of incremental learning. In this work we compare different incremental learning strategies for CNN-based architectures in the context of face recognition.

1. CNN-Based Incremental Learning
Strategies for Face Recognition
Vincenzo Lomonaco and Davide Maltoni
Biometric System Lab – DISI - University of Bologna
Vincenzo Lomonaco and Davide Maltoni
DISI - University of Bologna
Emails: {vincenzo.lomonaco, davide.maltoni}@unibo.it
Websites: www.vincenzolomonaco.com, http://bias.csr.unibo.it/maltoni/
Contacts
[1] Franco, A., Maio, D., Maltoni, D., Scienze, C.L., Bologna, U.:The Big Brother Database: Evaluating Face
Recognition in Smart Home Environments. Image (Rochester, N.Y.). 142–150 (2009).
[2] Franco, A., Maio, D., Maltoni, D.: Incremental template updating for face recognition in home environments.
Pattern Recognit. 43, 2891–2903 (2010).
[3] Maltoni, D., Lomonaco,V.: Semi-supervised Tuning from Temporal Coherence.Tech. Report. DISI - Univ. of
Bologna. http://arxiv.org/pdf/1511.03163v3.pdf. 1–14 (2015).
References
In the last decade, Convolutional Neural Networks (CNNs) have shown to
perform incredibly well on face recognition tasks being able to deal with large
occlusions, extremely low resolutions, strong illumination variations, etc.
However, partly because of their complex training and tricky hyper-parameters
tuning, CNNs have been scarcely studied in the context of incremental learning.
In this work we compare different incremental learning strategies for CNN-
based architectures in the context of face recognition.
One possible approach to deal with this incremental scenario is to store all the
previously seen past data, and retrain the model from scratch as soon as a
new batch of data is available. However, this solution is often impractical for
many real world systems where memory and computational resources are
subject to stiff constraints.
A different approach to address this issue, is to update the model based only
on the new available batch of data.
The BigBrother dataset (SETB) [1] has been created starting from 2 DVDs made
commercially available at the end of the 2006 edition of the “Big Brother” reality
show produced for the Italian TV.
It consists of 14,675 (70×70) gray-scale images of faces belonging to 7 subject
often characterized by bad lighting, poor focus, occlusions, and non-frontal
pose.
In addition to the typical training and test sets, it provides an additional large
set of images called “updating set” for incremental learning/tuning purposes
and split in 54 days.
Figure 1. The seven subjects of the Big Brother dataset (SETB).
• Forgetting can be a very detrimental issue: hence, when possible (i.e.,
transfer learning from the same domain), it is preferable to use CNN as a
fixed feature extractor to feed an incremental classifier.
• If the features are not optimized (transfer learning from a different domain),
the tuning of low level layers may be preferable and the learning strength
(i.e., learning rate, number of iteration, etc.) can be used to control
forgetting.
• Training a CNN from scratch can be advantageous if the problem patterns
(and feature invariances) are highly specific and a sufficient number of
samples are available.
Figure 2. Accuracy of the different strategies tested on the SETB of the Big Brother dataset.
Figure 3. The impact of the learning rate on forgetting.
Final Acc. % LeNet7 CaffeNet + SVM VGG + SVM CaffeNet + FT VGG + FT
34 Days Split 82.35% 80.10% 96.96% 73.23% 91.39%
Orig. Days
Split
75.33% 75.13% 96.73% 70.23% 89,58%
Cumulative
Days
90.50% 86.79% 97.65% 84.26% 95,51%
Gain +7.03% +4.97% +0.23% +3.00% +1.81%
Loss -8.15% -6.69% -0.69% -11.03% -4.12%
Table 1. Accuracy gain and loss of the 34 Days split with respect of the Original and Cumulative days split.
Abstract Experiments and results
Big Brother dataset
Incremental Learning Strategies
Conclusions
Pre-trained CNN + SVM
Pre-trained CNN + Finetuning
Ad-hoc CNN trained from scratch
SVM
3
2
1