photo detection in personal photo collection

1. Person Recognition in Personal Photo
Collections
Group no -
1.Varad Deshpande (34) Gr no.11810178
2.Prydyumna daware (28) Gr no
3.Sonali jagtap (49) Gr no. 11810918
4.Firdaus Naseem (37) Gr no.

2. What is Face Detection?
Given an image,
tell whether there
is any human face,
if there is, where is
it(or where they are).

3. Face Detection Methods
Face detection methods
Feature-based
Appearance-based
Knowledge based
Template-matching

4. Methods
1.Knowledge-Based:-
The knowledge-based method depends on the set of rules, and it is based on human
knowledge to detect the faces. Ex- A face must have a nose, eyes, and mouth within
certain distances and positions with each other.
2.Feature-Based:-
The feature-based method is to locate faces by extracting structural features of the
face. It is first trained as a classifier and then used to differentiate between facial and
non-facial regions.
4.Appearance-Based:-
The appearance-based method depends on a set of delegate training face images to
find out face models. The appearance-based approach is better than other ways of
performance. In general appearance-based method rely on techniques from
statistical analysis and machine learning to find the relevant characteristics of face
images.

5. Methods
3.Template Matching:-
Template Matching method uses pre-defined or parameterized face
templates to locate or detect the faces by the correlation between the
templates and input images. Ex- a human face can be divided into
eyes, face contour, nose, and mouth. Also, a face model can be built
by edges just by using edge detection method. This approach is
simple to implement, but it is inadequate for face detection.

6. Methodology
 Person recognition in private photo collections is challenging:
people can be shown in all kinds of poses and activities, from
arbitrary viewpoints including back views, and with diverse
clothing
 We explore three other sources: first, body of a person contains
information about their shape and appearance; second, human
attributes such as gender and age help to reduce the search space;
and third, scene context further reduces ambiguities
 Person recognition in photo albums is hard. To handle the diverse
scenarios we need to exploit multiple cues from different body
regions and information sources. For example, the surfer is not
recognised when using only head or head+body cues. However, it is
successfully recognised when the additional attribute cues are
provided.

7. Recognition tasks ;
•There exist multiple tasks related to person recognition Face and
surveillance reidentification is most commonly done via
“verification” (one reference image, one test image).
•Other related tasks are, for instance, face clustering finding
important people or associating names in text to faces in images .

8. Recognition cues ;
•The base cue for person recognition is the appearance of
the face itself. Face normalization (“frontalisation”)
improves robustness to pose, view-point and illumination.
Similarly, pose-independent descriptors can be built for
the body
•
•Multiple other cues have been explored, for example:
attributes classification explicit cloth modelling elative
camera positions , social context . space-time priors and
photo-album priors

9. PIPA dataset
 The recently introduced PIPA dataset (“People In Photo
Albums”) is, to the best of our knowledge, the first dataset
to annotate identities of people with back views.
 The annotators labelled many instances that can be
considered hard even for humans .
 PIPA features 37 107 Flickr personal photo album images
with 63 188 head bounding boxes of 2 356 identities.
 The dataset is partitioned into train, validation, test, and
leftover sets, with rough ratio 45 : 15 : 20 : 20.

10. Cues for recognition
Recognition considered for f - face , h - head , u - upper body , b - full body , s - scene. person
recognition
system is performant yet
simple. At test time, given a
(ground truth) head bound-
ing box, we estimate (based
on the box size) five differ-
ent regions depicted. Each
region is fed into one or
more convnets to obtain
a set of feature vectors.
The vectors are concate-
nated and fed into a linear
SVM,

11. Validation set accuracy of different
cues

14. What is Face
Recognition?
A set of two task:
Face Identification: Given a face image that
belongs to a person in a database, tell whose
image it is.
Face Verification: Given a face image that might
not belong to the database, verify whether it is
from the person it is claimed to be in the
database.

15. Difference between Face Detection and Recognition
Detection – two-class classification
Face vs. Non-face
Recognition – multi-class classification
One person vs. all the others

16. Face Detection + Recognition
> Detection accuracy affects the recognition stage
Key issues:
1. Correct location of key facial features(e.g. the eye
corner)
2. False detection.
3. Missed detection.

17. How the Face Detection Works:-
There are many techniques to detect faces, with the help of these
techniques, we can identify faces with higher accuracy.
1. Firstly the image is imported by providing the location of the
image.
2. Then the picture is transformed from RGB to Grayscale because
it is easy to detect faces in the grayscale.
3. After that, the image manipulation used, in which the resizing,
cropping, blurring and sharpening of the images done if needed.
4. The next step is image segmentation to segments the multiple
objects in a single image so that the classifier can quickly detect
the objects and faces in the picture.

18. Challenges of Image Recognition:
1.Viewpoint Variation
1.Scale Variation
2.Deformation
3.Inter-class Variation

19. Role of Convolutional Neural Networks in Image
Recognition:
Convolutional Neural Networks play a crucial role in
solving the problems stated above. Its basic
principles have taken the inspiration from our visual
cortex.
CNN incorporates changes in its mode of
operations. The inputs of CNN are not fed with the
complete numerical values of the image.

20. Role of Convolutional Neural Networks in Image
Recognition:
> These images are then treated similar to the regular neural
network process. The computer collects patterns with respect to
the image and the results are saved in the matrix format.
> This Matrix is again down sampled (reduced in size) with a
method known as Max-Pooling. It extracts maximum values from
each sub-matrix and results in a matrix of much smaller size.
> These values are representative of the pattern in the image.
This matrix formed is supplied to the neural networks as the input
and the output determines the probability of the classes in an
image.

21. Limitations of CNN:
1. Through complex architectures, it is possible to predict
objects, face in an image with 95% accuracy surpassing the
human capabilities which is 94%.
2. Even with its outstanding capabilities, there are certain
limitations in its utilization.
3. Datasets up to billion parameters require high computation
load, memory usage, and high processing power. Usage of
which requires proper justification.

22. Appearance-Based Methods
summery:
Pros:
1. Use powerful machine learning algorithms
2. Has demonstrated good empirical results
3. Fast and fairly robust
4. Extended to detect faces in different pose and orientation
Cons:
1. Usually needs to search over space and scale
2. Need lots of positive and negative examples
3. Limited view-based approach

23. Thank You