Person Recognition in Personal Photo Album - Computer Vision Research paper presentation in the Semester 7th.

1. Person Recognition in
Personal Photo Album
Fahmeen Mazhar

2. Paper
• 25 September 2015
• Max Planck Institute for Informatics
• Saarbrücken, Germany
• Paper link,
• http://openaccess.thecvf.com/content_iccv_2015/paper
s/Oh_Person_Recognition_in_ICCV_2015_paper.pdf

3. Abstract
• Recognizing persons in everyday photos presents major
challenges for machine vision.
• A convnet based person recognition system is proposed
on which an in-depth analysis of in formativeness of
different body cues is provided, impact of training data,
and the common failure modes of the system.
• The method is simple and is built on open source and
open data, yet it improves the state of the art results on a
large dataset of social media photos (PIPA).

4. Problem statement
• Person recognition in private photo collections is
challenging: people can be shown in all kinds of poses
and activities, from arbitrary viewpoints including back
views, and with diverse clothing.
• This paper presents an in-depth analysis of the problem of
person recognition in photo albums given a few annotated
training images of a person, and a single image at test
time, can we tell if the image contains the same person?
• Person recognition in social media photos sets new
challenges for computer vision, including non-
cooperative subjects (e.g., backward viewpoints, unusual
poses) and great changes in appearance.

6. Solution
• Person recognition in photo albums is hard.
• To tackle this problem, a simple person recognition
framework is built that leverages features from multiple
image regions (head, body, etc.).
• A new recognition scenarios that focus on the time and
appearance gap between training and testing samples.
• An in-depth analysis of the importance of different
features according to time and viewpoint generalizability.

7. • In the process, we verify that simple approach achieves
the state of the art result on the PIPA benchmark,
arguably the largest social media based benchmark for
person recognition to date with diverse poses, viewpoints,
social groups, and events.
• Compared the conference version of the paper , this paper
additionally presents analysis of a face recognizer, new
method naeil2 that combines the conference version
method naeil and DeepID2 to achieve state of the art
results even compared to post-conference works,
discussion of related work since the conference version,
additional analysis including the head viewpoint-wise
breakdown of performance, and results on the open-
world setup.

8. • Recognition tasks (Face clustering, finding important people,
associating names in text to faces in images.
• Recognizing cues (The PIPA dataset was introduced together
with the reference PIPER method. PIPER obtains promising
results.
• It combines three ingredients: a convnet (AlexNet) pretrained
on ImageNet, the DeepFace re-identification convnet (trained
on a large private faces dataset) , and Poselets (trained on
H3D) to obtain robustness to pose variance. In contrast, this
paper considers features based on open data and use the same
AlexNet network for all the image regions considered, thus
providing a direct comparison of contributions from different
image regions.

9. Results
• The recently introduced PIPA dataset (“People In Photo
Albums”) is, to the best of our knowledge, the first
dataset to annotate identities of people with back views.
• The annotators labeled many instances that can be
considered hard even for humans.

11. • Since PIPER uses different training data than naeil
we can expect some complementarity between the two
methods. For experiments, we use the PIPER scores
provided by the authors .
• Note, however, that the PIPER features are unavailable.
By averaging the output scores of the two methods
(PIPER + naeil) gain 1:5 percent points, reaching
88:37%.
• Using a more sophisticated strategy might provide more
gain, but we already see that naeil covers most of the
performance from
PIPER.

12. Future work
• Explore for more effective result on different dataset and
benchmark.
• Try multiple architecture and different feature extraction
techniques.

13. Thank You!