Downloaded 18 times
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 5
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
2. LITERATURE SURVEY
2.1 BACKGROUND
Biometrics based human computer interfaces (HCI) are becoming the most popular
systems for security and access control. Face recognition system is one such biometric
HCI system. It distinguishes among various human faces and identifies a face from a
digital image or a video frame from a video source. This is normally done by comparing
selected facial features of the image with those in a facial database that contain multiple
images of people and their information. The way humans recognize faces is highly
remarkable and efficient irrespective of the changes in the visual input due to lighting
conditions, background, and aging. [1] To build a system that can match humans in this
process is highly desirable yet very difficult to build. A basic face recognition system
may follow a Face Detection System whose function is to identify a face in a given image
and ignore all the other background details. After the face image is extracted it is given as
input to the Face Recognition System, which first extracts basic features of a face that
distinguishes one from the other and then classifiers are used to match images with those
stored in the database to identify a person. Therefore a basic face recognition system
contains the following sub-modules [2].
Face Detection
Feature Extraction
Feature Matching/Classification
In a work proposed by R. Chellappa, P. J. Phillips, A. Rosenfeld entitled ‘Face
Recognition: A Literature Survey’[3] describes face recognition has recently received
significant attention, especially during the past several years, as one of the most
successful applications of image analysis and understanding. This paper provides an up-
to-date critical survey of still and video-based face recognition research. There are two
underlying motivations for them to write this survey paper: the first is to provide an up-
to-date review of the existing literature, and the second is to offer some insights into the
studies of machine recognition of faces. To provide a comprehensive survey, they not
only categorize existing recognition techniques but also present detailed descriptions of
representative methods within each category. In addition, relevant topics such as](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-15-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 6
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
psychophysical studies, system evaluation, and issues of illumination and pose variation
are covered. In the work proposed by Matthew Turk and Alex Pentland in ‘Eigenfaces for
Recognition ‘[4], they have developed a near real-time computer system that can locate
and track a person’s head and then recognize it by comparing the features of this face
with those of known individuals. They treat the face recognition problem as an
intrinsically two dimensional recognition problem. The system functions by projecting
face images onto a feature space that spans the significant variations among known face
images. The significant features are known as ‘eigenfaces’, because they are the eigen
vectors of the set of faces, they do not necessarily correspond to features such as eyes,
ears and noses. The projection operation characterizes an individual face by a weighted
sum of the eigenface features, and so to recognize a face it is necessary only to compare
these weights to those of known individuals. Some particular advantages of the proposed
approach are that it provides for the ability to learn and later recognize new faces in an
unsupervised manner, and that it is easy to implement using a neural network approach.
In the work proposed by Kyungnam Kim entitled ‘Face Recognition using Principal
Component Analysis’ [5] illustrates that the Principal Component Analysis (PCA) is one
of the most successful techniques that have been used in image recognition and
compression. The purpose of PCA is to reduce the large dimensionality of the dataspace
to the smaller intrinsic dimensionality of feature space (independent variables), which are
needed to describe the data economically. The main idea of using PCA for face
recognition is to express the large 1-D vector of pixels constructed from 2-D facial image
into the compact principal components of the feature space. This is called Eigen space
projection. Eigen space is calculated by identifying the eigenvectors of the covariance
matrix derived from a set of facial images (vectors). The paper describes the
mathematical formulation of PCA and a detailed method of face recognition by PCA. In a
work by Baocai Yin Xiaoming Bai, Qin Shi, Yanfeng Sun entitled ‘Enhanced Fisher face
for Face Recognition’ Journal of Information & Computational Science 2: 3 (2005) 591-
595[6] Fisher face is enhanced in this paper for face recognition from one example image
per person. Fisher face requires several training images for each face and can hardly be
applied to applications where only one example image per person is available for
training. They enhance Fisher face by utilizing morphable model to derive multiple](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-16-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 7
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
images of a face from one single image. Region filling and hidden-surface removal
method are used to generate virtual example images. Experimental results on ORL and
UMIST face database show that our method makes impressive performance improvement
compared with conventional Eigenface methods. In another work contributed by P.
Latha, Dr. L. Ganesan, Dr. S. Annadurai, entitled “Face Recognition using Neural
Networks” , Signal Processing: An International Journal (SPIJ) Volume (3) : Issue (5)[7]
explains that Face recognition is one of biometric methods, to identify given face image
using main features of face. In this paper, a neural based algorithm is presented, to detect
frontal views of faces. The dimensionality of face image is reduced by the Principal
component analysis (PCA) and the recognition is done by the Back propagation Neural
Network (BPNN). In this project they have used 200 face images from Yale database and
some performance metrics like Acceptance ratio and Execution time have been
calculated. The Neural based Face recognition approach presented in this paper is robust
and has better performance of more than 90 % acceptance ratio. In the work proposed by
Mayank Agarwal, Nikunj Jain, Mr. Manish Kumar and Himanshu Agrawal, in “Face
Recognition Using Eigen Faces and Artificial Neural Network”, International Journal of
Computer Theory and Engineering, Vol. 2, No. 4, August, 2010[8]. Face is explained to
be a complex multidimensional visual model and developing a computational model for
face recognition is said to be difficult. The paper presents a methodology for face
recognition based on information theory approach of coding and decoding the face
image. Proposed methodology is connection of two stages – Feature extraction using
principle component analysis and recognition using the feed forward back propagation
Neural Network. The algorithm has been tested on 400 images (40 classes). A
recognition score for test lot is calculated by considering almost all the variants of feature
extraction. The proposed methods were tested on Olivetti and Oracle Research
Laboratory (ORL) face database. The test results are said to give a recognition rate of
97.018%.](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-17-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 9
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
In the geometric feature-based approach the primary step is to localize and track a dense
set of facial points. Most geometric feature-based approaches use the active appearance
model (AAM) or its variations, to track a dense set of facial points. The locations of these
facial landmarks are then used in different ways to extract the shape of facial features,
and movement of facial features, as the expression evolves. [9] use AAM with second
order minimization, and a multilayer perceptron, for the recognition of facial expressions.
A recent example of an AAM-based technique for facial expression recognition is
presented in [10], in which different AAM fitting algorithms are compared and evaluated.
Another example of a system that uses geometric features to detect facial expressions is
that by Kotisa et al. [11]. A Kanade-Lucas-Tomasi (KLT) tracker was used, after
manually locating a number of facial points. The geometric displacement of certain
selected candid nodes, defined as the differences of the node coordinates between the first
and the greatest facial expression intensity frames, were used as an input to the novel
multiclass SVM classifier. Sobe et al. [12] also uses geometric features to detect
emotions. In this method, the facial feature points were manually located, and Piecewise
Bezier volume deformation tracking was used to track those manually placed facial
landmarks. They experimented with a large number of m machine learning techniques,
and the best result was obtained with a simple k-nearest neighbor technique. Sung and
Kim [13] introduce Stereo AAM (STAAM), which improves the fitting and tracking of
standard AAMs, by using multiple cameras to model the 3-D shape and rigid motion
parameters. A layered generalized discriminant analysis classifier is then used to combine
3-D shape and registered 2-D appearance, for the recognition of facial expressions. In
[14], a geometric features-based approach for modeling, tracking, and recognizing facial
expressions on a low-dimensional expression manifold was presented. Sandbach et
al. [15] recently presented a method that exploits 3D motion-based features between
frames of 3D facial geometry sequences, for dynamic facial expression recognition.
Onset and offset segment features of the expression are extracted, by using feature
selection methods. These features are then used to train the GentleBoost classifiers, and
build a Hidden Markov Model (HMM), in order to model the full temporal dynamic of
the expressions. Rudovic and Pantic [16] introduce a method for head-pose invariant
facial expression recognition that is based on a set of characteristic facial points,](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-19-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 10
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
extracted using AAMs. A coupled scale Gaussian process regression (CSGPR) model is
used for head-pose normalization.
Researchers have also developed systems for facial expression recognition, by utilizing
the advantages of both geometric-based and appearance-based features. Lyons and
Akamatsu [17] introduced a system for coding facial expressions with Gabor wavelets.
The facial expression images were coded using a multi-orientation, multi-resolution set of
Gabor filters at some fixed geometric positions of the facial landmarks. The similarity
space derived from this representation was compared with one derived from semantic
ratings of the images by human observers. In [18], a comparison between geometric-
based and Gabor-wavelets-based facial expression recognition using multi-layer
perceptorn was presented. Huang et al. [19] presented a boosted component-based facial
expression recognition method by utilizing the spatiotemporal features extracted from
dynamic image sequences, where the spatiotemporal features were extracted from facial
areas centered at 38 detected fiducial interest points.
2.3 PHOTOMETRIC (VIEW BASED)
Photometric is a technique in computer vision for estimating the surface normal of
objects by observing that object under different lighting conditions. It is based on the fact
that the amount of light reflected by a surface is dependent on the orientation of the
surface in relation to the light source and the observer. By measuring the amount of light
reflected into a camera, the space of possible surface orientations is limited. Given
enough light sources from different angles, the surface orientation may be constrained to
a single orientation or even over constrained.
The technique was originally introduced by Woodham in 1980. The special case where
the data is a single image is known as shape from shading, and was analyzed by B. K. P.
Horn in 1989. Photometric stereo has since been generalized to many other situations,
including extended light sources and non-Lambertian surface finishes. Current research
aims to make the method work in the presence of projected shadows, highlights, and non-
uniform lighting. Surface normal define the local metric, using this observation Bronstein
et al. [20] defined a 3D face recognition system based on the reconstructed metric](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-20-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 11
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
without integrating the surface. The metric of the facial surface is known to be robust to
expressions.
Figure 2.3 (a) Facial Surface[20]
The input of a face recognition system is always an image or video stream. The output is
an identification or verification of the subject or subjects that appear in the image or
video.
A face recognition system as a three step process
Figure 2.3 (b) Basic Face Recognition](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-21-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 14
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
Pedestrian Detection
Kinect projects
3D reconstruction
2.5 PCA
PCA is known a Principle Component Analysis – this is a statistical analytical tool that is
used to explore, sort and group data. What PCA does is take a large number of correlated
(interrelated) variables and transform this data into a smaller number of uncorrelated
variables (principal components) while retaining maximal amount of variation, thus
making it easier to operate the data and make predictions. Or as Smith (2002) puts it
“PCA is a way of identifying patterns in data, and expressing the data in such a way as to
highlight their similarities and differences[13]. Since patterns in data can be hard to find
in data of high dimension, where the luxury of graphical representation is not available,
PCA is a powerful tool for analyzing data.”
History of PCA
According to Jolliffe (2002) it is generally accepted that PCA was first described by Karl
Pearson in 1901. In his article ” On lines and planes of closest fit to systems of points in
space,” Pearson (1901) discusses the graphical representation of data and lines that best
represent the data. He concludes that “The best-fitting straight line to a system of points
coincides in direction with the maximum axis of the correlation ellipsoid”. He also states
that the analysis used in his paper can be applied to multiple variables. However, PCA
was not widely used until the development of computers. It is not really feasible to do
PCA by hand when number of variables is greater than four, but it is exactly for larger
amount of variables that PCA is really useful, so the full potential of PCA could not be
used until after the spreading of computers (Jolliffe, 2002). According to Jolliffe (2002)
significant contributions to the development of PCA were made by Hotelling (1933) and
Girshick (1936; 1939) before the expansion in the interest towards PCA. In 1960s. as the
interest in PCA rose, important contributors were Anderson (1963) with a theoretical
discussion, Rao (1964) with numerous new ideas concerning uses, interpretations and](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-24-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 16
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
of variables optimizing a certain algebraic criterion. Shlens 2009) gives an overview how
to perform principal components analysis:
Organize data as an m×n matrix, where m is the number of measurement types and n is
the number of samples
Subtract off the mean for each measurement type
Calculate covariance matrix
Calculate the eigenvectors and eigenvalues of the covariance matrix.
DERIVING PRINCIPAL COMPONENTS
Following is a standard derivation of principal components presented by Jolliffe (2002)
To derive the form of the PCs, consider first α’1x; the vector a1 maximizes
Var [α’x]= α’1∑ α1
It is clear that, as it stands, the maximum will not be achieved for finite α1 so a
normalization constraint must be imposed. The constraint used in the derivation is α1α1 =
1, that is, the sum of squares of elements of α1 equals 1. Other constraints may be more
useful in other circumstances, and can easily be substituted later on. However, the use of
constraints other than α’1α1 = constant in the derivation leads to a more difficult
optimization problem, and it will produce a set of derived variables different from the
principal components.
To maximize α’1∑α subject α’1∑α-λ (α’1α1-1) to α1α1 = 1, the standard approach
is to use the technique of Lagrange multipliers. ∑α1-λα1=0
(∑-λ Ip) α1 =0
Maximize where λ is a Lagrange multiplier. Differentiation with respect to α1 gives or
Where Ip is the (p x p) identity matrix. Thus, ë is an eigenvalue and á1 is the
corresponding eigenvector. To decide which of the
α’1 ∑α1= α’1λ α1= λ ,the eigenvectors gives á’1x with maximum variance, note that
the quantity to be maximized is so ë must be as large as possible. Thus, á1 is the
eigenvector corresponding to the largest eigenvalue, and var [α’1x] = α’1∑α1 = λ1, the](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-26-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 17
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
largest eigenvalue. In general, the kth PC of x is a’kx and, where ëk is the kth largest
eigenvalue of
var [α’kx] =λk for k=1, 2, 3,……,p.
And ák is the corresponding eigenvector.
Shlens (2009) derives an algebraic solution to PCA based on an important property of
eigenvector decomposition. Once again, the data set is X, an
m×n matrix, where m is the number of measurement types and n is the number of
samples. The goal is summarized as follows: Find some orthonormal matrix P in Y = PX
such that CY=1/n (YYT) is a diagonal matrix. The rows of P are the principal
components of X. The rewriting CY in terms of the unknown variable.](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-27-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 18
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
2.6 ADVANTAGES AND DISADVANTAGES OF PCA
IMPORTANCE OF PCA
Principal component analysis (PCA) is a standard tool in modern data analysis - in
diverse fields from neuroscience to computer graphics - because it is a simple, non-
parametric method for extracting relevant information from confusing data sets. With
minimal effort PCA provides a roadmap for how to reduce a complex data set to a lower
dimension to reveal the sometimes hidden, simplified structures that often underlie it.
(Shlens, 2009) Importance of PCA is manifested by its use in so many different fields of
science and life. PCA is very much used in neuro-science, for example. Another fields of
use are pattern recognition and image compression, therefore PCA is suited for use in
facial recognition software for example, as well as for recognition and storing of other
biometric data. Many IT related fields also use PCA, even artificial intelligence research.
According to Jolliffe (2002) PCA is also used in research of agriculture, biology,
chemistry, climatology, demography, ecology, food research, genetics, geology,
meteorology, oceanography, psychology, quality control, etc. But in this paper we are
going to focus more on uses in finance and economy. PCA has been used in economics
and finance to study changes in stock markets, commodity markets, economic growth,
exchange rates, etc. Earlier studies were done in economics, but stock markets were also
under research already in 1960s. Lessard (1973) claims that “principal component or
factor analysis have been used in several recent empirical studies (Farrar [1962], King
[1967], and Feeney and Hester [1967]) concerned with the existence of general
movements in the returns from common stocks.” PCA has mostly been used to compare
different stock markets in search for diversification opportunities, especially in earlier
studies like the ones by Makridakis (1974) and by Phillipatos et al. (1983).
BENEFITS OF PCA
PCA is a special case of Factor Analysis that is highly useful in the analysis of many time
series and the search for patterns of movement common to several series (true factor
analysis makes different assumptions about the underlying structure and solves](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-28-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 22
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
2.8 PCA VERSUS 2DPCA
In the PCA-based face-recognition technique, the 2D face-image matrices must be
previously transformed into 1D image vectors. The resulting image vectors usually lead
to a high dimensional image vector space in which it’s difficult to evaluate the covariance
matrix accurately due to its large size and relatively few training samples. Fortunately,
we can calculate the eigenvectors (eigenfaces) efficiently using single value
decomposition (SVD) techniques, which avoid the process of generating the covariance
matrix. Unlike conventional PCA, 2DPCA is based on 2D matrices rather than 1D
vectors. That is, the image matrix doesn’t need to be previously transformed into a
vector. Instead, an image covariance matrix can be constructed directly using the original
image matrices. In contrast to PCA’s covariance matrix, the image covariance matrix’s
size using 2DPCA is much smaller. As a result, 2DPCA has two important advantages
over PCA. First, it’s easier to evaluate the covariance matrix accurately. Second, less
time is required to determine the corresponding eigenvectors. We extract the features
from the 2DPCA matrix using the optimal projection vector axes. The vector’s size is
given by the image’s size and the number of coefficients. If the image size is 112 × 92,
for example, then the number of coefficients is 112 × d. Researchers have demonstrated
experimentally that should be set to no less than 5 to satisfy accuracy.4 This leads us to a
numerous coefficients.
Figure 2.8 Examples from the faces dataset[12]](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-32-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 24
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
2.10 HAAR FEATURE ALGORITHMS
Detecting human facial features, such as the mouth, eyes, and nose require that Haar
classifier cascades first be trained. In order to train the classifiers, this gentle AdaBoost
algorithm and Haar feature algorithms must be implemented. Fortunately, Intel developed
an open source library devoted to easing the implementation of computer vision related
programs called Open Computer Vision Library (OpenCV). The OpenCV library is
designed to be used in conjunction with applications that pertain to the field of HCI,
robotics, biometrics, image processing, and other areas where visualization is important
and includes an implementation of Haar classifier detection and training [8]. To train the
classifiers, two set of images are needed. One set contains an image or scene that does not
contain the object, in this case a facial feature, which is going to be detected. This set of
images is referred to as the negative images. The other set of images, the positive images,
contain one or more instances of the object. The location of the objects within the
positive images is specified by: image name, the upper left pixel and the height, and
width of the object [21]. For training facial features 5,000 negative images with at least a
mega-pixel resolution were used for training. These images consisted of everyday
objects, like paperclips, and of natural scenery, like photographs of forests and
mountains. In order to produce the most robust facial feature detection possible, the
original positive set of images needs to be representative of the variance between
different people, including, race, gender, and age. A good source foor these images is
National Institute of Standards and Technology’s (NIST) Facial Recognition Technology
(FERET) database. This database contains over 10,000 images of over 1,000 people
under different lighting conditions, poses, and angles. In training each facial feature,
1,500 images were used. These images were taken at angles ranging from zero to forty
five degrees from a frontal view. This provides the needed variance required to allow
detection if the head is turned slightly [22]. Three separate classifiers were trained, one
for the eyes, one for the nose, and one for the mouth. Once the classifiers were trained,
they were used to detect the facial features within another set of images from the FERET
database.](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-34-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 25
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
The accuracy of the classifier was then computed as shown in Table 1. With the
exception of the mouth classifier, the classifiers have a high rate of detection. However,
as implied by [23], the false positive rate is also quite high.
2.11 REGIONALIZED DETECTION
Since it is not possible to reduce the false positive rate of the classifier without reducing
the positive hit rate, a method besides modifying the classifier training attribute is needed
to increase accuracy [24]. The method proposed to is to limit the region of the image that
is analyzed for the facial features. By reducing the area analyzed, accuracy will increase
since less area exists to produce false positives. It also increases efficiency since fewer
features need to be computed and the area of the integral images is smaller. In order to
regionalize the image, one must first determine the likely area where a facial feature
might exist. The simplest method is to perform facial detection on the image first. The
area containing the face will also contain facial features.
However, the facial feature cascades often detect other facial features as illustrated in
Figure 2.8. The best method to eliminate extra feature detection is to further regionalize
the area for facial feature detection. It can be assumed that the eyes will be located near
the top of the head, the nose will be located in the center area and the mouth will be
located near the bottom. The upper 5/8 of the face is analyzed for the eyes. This area
eliminates all other facial features while still allowing a wide variance in the tilt angle.
The center of the face, an area that is 5/8 by 5/8 of the face, was used to for detection of
the nose. This area eliminates all but the upper lip of the mouth and lower eyelid. The
lower half of the facial image was used to detect the mouth. Since the facial detector used
sometimes eliminates the lower lip the facial image was extended by an eighth for mouth
detection only.](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-35-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 26
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
2.12 COMPUTATIONS AND EXAMPLES
Principal Component Analysis, or simply PCA, is a statistical procedure concerned with
elucidating the covari-ance structure of a set of variables. In particular it allows us to
identify the principal directions in which the data varies.
For example, in figure 2.12.a, suppose that the triangles represent a two variable data set
which we have measured in the X-Y coordinate system. The principal direction in which
the data varies is shown by the U axis and the second most important direction is the V
axis orthogonal to it. If we place the U V axis system at the mean of the data it gives us a
compact representation. If we transform each (X; Y ) coordinate into its corresponding
(U; V ) value, the data is de-correlated, meaning that the co-variance between the U and
V variables is zero. For a given set of data, principal component analysis finds the axis
system defined by the principal directions of variance (ie the U V axis system in figure
2.12.a). The directions U and V are called the principal components.
Figure 2.12(a) PCA for Data Representation[3]](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-36-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 27
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
Figure 2.12(b) PCA for Dimension Reduction[3]
If the variation in a data set is caused by some natural property, or is caused by random
experimental error, then we may expect it to be normally distributed. In this case we
show the nominal extent of the normal distribution by a hyper-ellipse (the two
dimensional ellipse in the example). The hyper ellipse encloses data points that are
thought of as belonging to a class. It is drawn at a distance beyond which the probability
of a point belonging to the class is low, and can be thought of as a class boundary.
If the variation in the data is caused by some other relationship then PCA gives us a way
of reducing the dimensionality of a data set. Consider two variables that are nearly related
linearly as shown in figure 2.12.b. As in figure 2.12.a the principal direction in which the
data varies is shown by the U axis, and the secondary direction by the V axis. However in
this case all the V coordinates are all very close to zero. We may assume, for example,
that they are only non-zero because of experimental noise. Thus in the U V axis system
we can represent the data set by one variable U and discard V. Thus we have reduced the
dimensionality of the problem by 1.
COMPUTING THE PRINCIPAL COMPONENTS
In computational terms the principal components are found by calculating the
eigenvectors and eigenvalues of the data covariance matrix. This process is equivalent to
finding the axis system in which the co-variance matrix is diagonal. The eigenvector with](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-37-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 30
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
PCA IN PRACTICE
Figure 2.12 (c) The PCA Transformation[3]
Figure 2.12.c gives a geometric illustration of the process in two dimensions. Using all
the data points we find the mean values of the variables (µx1; µx2) and the covariance
matrix ∑ which is a 2 2 matrix in this case. If we calculate the eigenvectors of the co-
variance matrix we get the direction vectors indicated by Ø1 and Ø 2. Putting the two
eigenvectors as columns in the matrix = [Ø 1; Ø 2] we create a transformation matrix
which takes our data points from the [x1; x2] axis system to the axis [Ø 1; Ø 2] system with
the equation:
p = (px µx)
Where px is any point in the [x1; x2] axis system, µx = (µx1; µx2) is the data mean, and p is
the coordinate of the point in the [Ø 1; Ø 2] axis system.
PCA Based Face Recognition
Face recognition is one example where principal component analysis has been
extensively used, primarily for reducing the number of variables. Let us consider the 2D](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-40-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 31
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
case where we have an input image, and wish to compare this with a set of data base
images to find the best match. We assume that the images are all of the same resolution
and are all equivalently framed. (ie the faces appear at the same place and same scale in
the images). Each pixel can be considered a variable thus we have a very high
dimensional problem which can be simplified by PCA.
Formally, in image recognition an input image with n pixels can be treated as a point in
an n-dimensional space called the image space. The individual ordinates of this point
represent the intensity values of each pixel of the image and form a row vector: px = (i1;
i2; i3; :::in) This vector is formed by concatenating each row of image pixels, so for a
modest sized image, say 128 by 128 resolution it will dimension 16384. For example:
Becomes the vector:
[150; 152; 151; 131; 133; 72; 144; 171; 67]16K
Clearly this number of variables is far more than is required for the problem. Most of the
image pixels will be highly correlated. For example if the background pixels are all equal
then adjacent background pixels are exactly correlated. Thus we need to consider how to
achieve a reduction in the number of the variables.
DIMENSION REDUCTION
Let’s consider an application where we have N images each with n pixels. We can write
our entire data set as an N n data matrix D. Each row of D represents one image of our
data set. For example we may have:](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-41-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 33
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
It is in effect an axis system in which we can represent our data in a compact form.
We can achieve size reduction by choosing to represent our data we fewer dimensions.
Normally we choose to use the set of m (m <= n) eigenvectors of ∑ which have the m
largest eigenvalues. Typically for face recognition system m will be quite small (around
20-50 in number). We can compose these in an n x m matrix
Øpca = [Ø1; Ø2; Ø3 _ _ _ _m] which performs the PCA projection.
For any given image px = (i1; i2; i3; :::in)
We can find a corresponding point in the PCA space by computing
The m-dimension vector pØ is all we need to represent the image. We have achieved a
massive reduction in data size since typically n will be at least 16K and m as small as 20.
We can store all our data base images in the PCA space and can easily search the data
base to find the closest match to a test image. We can also reconstruct any image with the
inverse transform:
It can be shown that choosing the m eigenvectors of ∑ that have the largest eigenvalues
minimizes the mean square reconstruction error over all choices of m orthonormal bases.
Clearly we would like to be as small as possible compatible with accurate recognition and
reconstruction, but this problem is data dependent. We can make a decision on the basis
of the amount of the total variance accounted for by the m principal components that we
have chosen. This can be assessed by looking at the eigenvalues.](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-43-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 35
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
If we multiply both sides of this equation by UT we get:
Adding brackets to this equation:
Makes it clear to see that UTØ` are the eigenvectors of UTU, which is what we wanted to
find. Since UUT has dimension N x N and is a considerably smaller matrix than UTU, we
have solved the much smaller eigenvector problem, and can calculate the N-1
eigenvectors of UTU with one further matrix multiply. UUT has at most N - 1
eigenvectors, but, as noted above, all other eigenvectors of UTU have zero eigenvalues.
The resulting eigenvectors are not orthonormal, and so should be normalized. The
method of PCA is sometimes also called the Karhunen-Lowe transform, and occasionally
the Householder transform. It is a specific application of the general mathematical
technique called single value decomposition in which an n x m matrix is projected to a
diagonal form. It is closely related to the technique called single value decomposition
which is a more general diagonalization method that applies to matrices that are not
square.
CORRESPONDENCE IN PCA
In 2D face images the pixels are not usually in correspondence. That is to say a given
pixel [xi; yi] may be part of the cheek in one image, part of the hair in another and so on.
This causes a lot of problems in face recognition and also in reconstructing faces. This
means that any linear combination of eigenfaces does not represent a true face but a
composition of face parts. A true face is only created when the eigenfaces are added
together in exactly the right proportions to re-create one of the original face images of the
training set.
However, if we represent a face in 3D, then it is possible to establish a correspondence
between each point on the surface map. We can organize the data in such a way that each
anatomical location (for example the tip of the nose) has the same index in each different](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-45-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 36
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
face data set. Figure shows two different faces whose surface points are in
correspondence.
Figure 2.12.d 3D surfaces in correspondence [7]
PCA can be carried out on 3D surface maps of faces (and other anatomical structures).
The variables are the individual co-ordinates of the surface points. A data set is of the
form:
[x1; y1; z1; x2; y2; z2; x3; xN ; yN ; zN ]
The face maps in figure have about 5,000 surface points, hence the number of variables is
15,000.
If the points of each subject are in correspondence, and the different subjects are aligned
as closely as possible in 3D before calculating the PCA, then any reasonable combination
of the eigenvectors will represent a valid face. This means that PCA on 3D surface maps
has the potential for constructing and manipulating faces that are different from any
example in the training set. This has many of applications in film and media. The set of
eigenvectors representing surface data in this manner is called an active shape model](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-46-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 37
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
Texture can be mapped onto a 3D surface map to give it a realistic face appearance. The
texture values (like pixel values in 2D) can also be included as variables in PCA. Models
of this sort are called active appearance models.
2.13 RESULTS
The first step in facial feature detection is detecting the face. This requires analyzing the
entire image. The second step is using the isolated face(s) to detect each feature.
The result is shown in Figure2.12.d. Since each the portion of the image used to detect a
feature is much smaller than that of the whole image, detection of all three facial features
takes less time on average than detecting the face itself. Using a 1.2GHz AMD processor
to analyze a 320 by 240 image, a frame rate of 3 frames per second was achieved. Since a
frame rate of 5 frames per second was achieved in facial detection only by [25] using a
much faster processor, regionalization provides a tremendous increase in efficiency in
facial feature detection.
Regionalization also greatly increased the accuracy of the detection. All false positives
were eliminated, giving a detection rate of around 95% for the eyes and nose. The mouth
detection has a lower rate due to the minimum size required for detection. By changing
the height and width parameter to more accurately represent the dimensions of the mouth
and retraining the classifier the accuracy should increase the accuracy to that of the other
features.
2.14 ARTIFICIAL NEURAL NETWORKS FOR FACE DETECTION
In the recent years, different architectures and models of Artificial neural networks
(ANN) were used for face detection and recognition. ANN can be used in face detection
and recognition because these models can simulate the way neurons work in the human
brain. This is the main reason for its role in face recognition. Artificial neural networks
(ANN) were used largely in the recent years in the fields of image processing
(compression, recognition and encryption) and pattern recognition. Many literature
researches used different ANN architecture and models for face detection and recognition](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-47-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 38
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
to achieve better compression performance according to: compression ratio (CR);
reconstructed image quality such as Peak Signal to Noise Ratio (PSNR); and mean square
error (MSE). Few literature surveys that give overview about researches related to face
detection based on ANN. Therefore, this research includes survey of literature studies
related to face detection systems and approaches which were based on ANN.
RETINAL CONNECTED NEURAL NETWORK (RCNN)
Rowley, Baluja and Kanade (1996) [27] presented face detection system based on a
retinal connected neural network (RCNN) that examine small windows of an image to
decide whether each window contains a face. Figure 3 shows this approach. The system
arbitrates between many networks to improve performance over one network. They used
a bootstrap algorithm as training progresses for training networks to add false detections
into the training set. This eliminates the difficult task of manually selecting non-face
training examples, which must be chosen to span the entire space of non-face images.
First, a pre-processing step, adapted from [28], is applied to a window of the image. The
window is then passed through a neural network, which decides whether the window
contains a face. They used three training sets of images. Test SetA collected at CMU:
consists of 42 scanned photographs, newspaper pictures, images collected from WWW,
and TV pictures (169 frontal views of faces, and require ANN to examine 22,053,124
20×20 pixel windows). Test SetB consists of 23 images containing 155 faces (9,678,084
windows). Test SetC is similar to Test SetA, but contains images with more complex
backgrounds and without any faces to measure the false detection rate: contains 65
images, 183 faces, and 51,368,003 window. The detection ratio of this approach equal](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-48-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 39
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
79.6% of faces over two large test sets and small number of false positives.
Figure 2.14.a Retinal Connected Neural Network (Rcnn)
ROTATION INVARIANT NEURAL NETWORK (RINN)
Rowley, Baluja and Kanade (1997) [29] presented a neural network-based face detection
system. Unlike similar systems which are limited to detecting upright, frontal faces, this
system detects faces at any degree of rotation in the image plane. Figure 4 shows the
RINN approach. The system employs multiple networks; the first is a “router” network
which processes each input window to determine its orientation and then uses this
information to prepare the window for one or more detector networks. We present the
training methods for both types of networks. We also perform sensitivity analysis on the
networks, and present empirical results on a large test set. Finally, we present preliminary
results for detecting faces which are rotated out of the image plane, such as profiles and
semi-profiles [29].](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-49-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 40
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
Figure 2.14.b Rotation Invariant Neural Network (Rinn)
PRINCIPAL COMPONENT ANALYSIS WITH ANN (PCA & ANN)
Jeffrey Norris (1999) [30] used using principal component analysis (PCA) with class
specific linear projection to detect and recognized faces in a real-time video stream.
Figure 5 shows PCA & ANN for face detection. The system sends commands to an
automatic sliding door, speech synthesizer, and touchscreen through a multi-client door
control server. Matlab, C, and Java were used for developing system. The system steps to
search for a face in an image:
1. Select every 20×20 region of input image.
2. Use intensity values of its pixels as 400 inputs to ANN.
3. Feed values forward through ANN, and
4. If the value is above 0.5, the region represents a face.
5. Repeat steps (1..4) several times, each time on a resized version of the original
input
6. image to search for faces at different scales [30].](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-50-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 45
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
faces. These approaches are more appropriate for our specific case since they make use of
a template (e.g., the shape of an object). Among the early deformable template models is
the Active Contour Model by Kass et al. [47] in which a correlation structure between
shape markers is used to constrain local changes. Cootes et al. [48] proposed a
generalized extension, namely Active Shape Models (ASM), where deformation
variability is learned using a training set. Active Appearance Models (AAM) were later
proposed in [49] and they are closely related to the simultaneous formulation of Active
Blobs [50] and Morphable Models [51]. AAM can be seen as an extension of ASM which
includes the appearance information of an object.
EMOTION RECOGNITION RESEARCH
Ekman and Friesen [52] developed the Facial Action Coding System (FACS) to code
facial expressions where movements on the face are described by a set of action units
(AUs). Each AU has some related muscular basis. This system of coding facial
expressions is done manually by following a set of prescribed rules. The inputs are still
images of facial expressions, often at the peak of the expression. This process is very
time-consuming. Ekman’s work inspired many researchers to analyze facial expressions
by means of image and video processing. By tracking facial features and measuring the
amount of facial movement, they attempt to categorize different facial expressions.
Recent work on facial expression analysis and recognition has used these “basic
expressions” or a subset of them. The two recent surveys in the area [53, 54] provide an
in depth review of many of the research done in automatic facial expression recognition
in recent years.](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-55-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 51
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
Figure 3.2.c Face Detection and Facial expression classification[12]
Once User approves or accept the faces with proper name, system records changes. This
changes will be recorded into XML file. XML will maintain fully qualified file name
(name with location) and the face value. Typical structure of XML is like below
<?xml version="1.0" encoding="utf-8"?>
<Photo>
<TagedPhoto>
<Photo FileName="photo - 1.jpg">](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-61-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 54
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
3.4 SIMULATION
INTRODUCTION
Simulation can be defined to show the eventual real behavior of the selected system
model .It is used for performance optimization on the basis of creating a model of the
system in order to gain insight into their functioning. We can predict the estimation and
assumption of the real system by using simulation results.
SIMULATOR
Photograph Database Management Using Face Recognition Algorithm is proposed to
overcome management of Photographs with faces without duplicating. It provides search
mechanism to query photographs from facial name. This is turning point as search will
not try any standard face recognition mechanism but it will look into index prepared
during copying of image. This makes system quite fast.
EGMU CV FACE RECOGNIZER AND EIGEN CLASSIFIER
FaceRecognizer is a global constructor that allows Eigen, Fisher, and LBPH classifiers to
be used together. The class combines common method calls between the classifiers. The
constructor for each classifier type is as follows:
FaceRecognizer recognizer = new EigenFaceRecognizer(num_components, threshold);
THE EIGEN CLASSIFIER
The Eigen recognizer takes two variables. The 1st, is the number of components kept for
this Principal Component Analysis. There’s no rule how many components that should be
kept for good reconstruction capabilities. It is based on input data, so experiment with the
number. OpenCV documentation suggests keeping 80 components should almost always
be sufficient. The 2nd variable is designed to be a prediction threshold[7].
FaceRecognizer recognizer = new EigenFaceRecognizer(80, double.PositiveInfinity);
PRINCIPLE COMPONENT ANALYSIS (PCA)](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-64-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 55
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
The EigenFaceRecognizer applies PCA. The FisherFaceRecognizer applies Linear
Discriminant Analysis derived by R.A. The EigenFaceRecognizer class applies PCA on
each image, the results of which will be an array of Eigen values that a Neural Network
can be trained to recognize[7]. PCA is a commonly used method of object recognition as
its results, when used properly can be fairly accurate and resilient to noise. The method of
which PCA is applied can vary at different stages so what will be demonstrated is a clear
method for PCA application that can be followed. It is up for individuals to experiment in
finding the best method for producing accurate results from PCA.
To perform PCA several steps are undertaken:
Stage 1: Subtract the Mean of the data from each variable (our adjusted data)
Stage 2: Calculate and form a covariance Matrix
Stage 3: Calculate Eigenvectors and Eigenvalues from the covariance Matrix
Stage 4: Chose a Feature Vector (a fancy name for a matrix of vectors)
Stage 5: Multiply the transposed Feature Vectors by the transposed adjusted data
STAGE 1: Mean Subtraction
This data is fairly simple and makes the calculation of our covariance matrix a little
simpler now this is not the subtraction of the overall mean from each of our values as for
covariance we need at least two dimensions of data. It is in fact the subtraction of the
mean of each row from each element in that row.
(Alternatively the mean of each column from each element in the column however this
would adjust the way we calculate the covariance matrix)](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-65-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 81
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
6. FUTURE ENHANCEMENT
A new technique for image feature extraction and representation—two-dimensional
principal component analysis (2DPCA)—was developed. 2DPCA has many advantages
over conventional PCA (Eigenfaces).
In the first place, since 2DPCA is based on the image matrix, it is simpler and more
straightforward to use for image feature extraction. Second, 2DPCA is better than PCA in
terms of recognition accuracy in all experiments. Although this trend seems to be
consistent for different databases and conditions, in some experiments the differences in
performance were not statistically significant. Third, 2DPCA is computationally more
efficient than PCA and it can improve the speed of image feature extraction significantly.
However, it should be pointed out that 2DPCA-based image representation was not as
efficient as PCA in terms of storage requirements, since 2DPCA requires more
coefficients for image representation than PCA. Why does 2DPCA outperform PCA in
face recognition? In our opinion, the underlying reason is that 2DPCA is more suitable
for small sample size problems (like face recognition) since its image covariance matrix
is quite small. Image representation and recognition based on PCA (or 2DPCA) is
statistically dependent on the evaluation of the covariance matrix (although for PCA the
explicit construction of the covariance matrix can be avoided)[32]. The obvious
advantage of 2DPCA A over PCA is that the former evaluates the covariance matrix
more accurately. Finally, there are still some aspects of 2DPCA that deserve further
study. When a small number of the principal components of PCA are used to represent an
image, the mean square error (MSE) between the approximation and the original pattern
is minimal. In addition, 2DPCA needs more coefficients for image representation than
PCA. Although, as a feasible alternative to deal with this problem is to use PCA after
2DPCA for further dimensional reduction, it is still unclear how the dimension of 2DPCA
could be reduced directly.](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-91-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 82
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
7. REFERENCES
[1] Shilpi Soni, Raj Kumar Sahu,” Face Recognition Based on Two Dimensional
Principal Component Analysis (2DPCA) and Result Comparison with Different
Classifiers”, International Journal of Advanced Research in Computer and
Communication Engineering Vol. 2, Issue 10, October 2013
[2] Vo Dinh Minh Nhat and Sungyoung Lee. “Improvement On PCA and 2DPCA
algorithms for face recognition “CIVR 2005, LNCS 3568, Pp: 568-577
[3] D.Q. Zhang, and H.Zhou” (2D) 2PCA, Two-directional two-dimensional PCA for
efficient face representation and recognition for Nero computing”
[4] Zhao W., Chellappa R, Phillips and Rosenfeld, “Face Recognition: A Literature
Survey”, ACM Computing Surveys, Vol. 35, No. 4, December 2003, pp. 399–458.
[5] Patil A.M., Kolhe S.R. and Patil P.M, “2D Face Recognition Techniques: A Survey,
International Journal of Machine Intelligence,”, ISSN: 0975–2927, Volume 2, Issue 1,
2010, pp-74-8
[6] G. H. Dunteman. “Principal Components Analysis”. Sage Publications, 1989.
[7] Turk M. And Pentland A.” Eigenfaces for recognition”, J. Cogn. Neurosci. 3, 72–86,
1991
[8] S. J. Lee, S. B. Yung, J. W. Kwon, and S. H. Hong, “Face Detection and Recognition
Using PCA”, pp. 84-87, IEEE TENCON,1999.
[9] Kohonen, T., "Self-organization and associative memory”, Berlin: Springer- Verlag,,
1989.
[10] Kohonen, T., and Lehtio, P., "Storage and processing of information in distributed
associative memory systems", 1981.
[11] Fischler, M. A.and Elschlager, R. A., "The representation and matching of pictorial
structures", IEEE Trans. on Computers, c-22.1, 1973.
[12] Yuille, A.L., Cohen, D. S., and Hallinan, P. W., "Feature extraction from faces
using deformable templates", Proc. of CVPR, 1989.
[13] Kanade, T., "Picture processing system by computer complex and recognition of
human faces", Dept. of Information Science, Kyoto University, 1973.](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-92-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 83
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
[14] B.Moghaddam, and A. Pentland, “Probabilistic Visual Learning for Object
Representation”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.
19, No. 7, July 1997
[15] K. C. Chung, S. C. Kee, and S. R. Kim, “Face Recognition using Principal
Component Analysis of Gabor Filter Responses”, p.53- 57, IEEE, 1999.
[16] Hossein Sahoolizadeh, B. Zargham Heidari, and C. Hamid Dehghani, “A New Face
Recognition Method using PCA, LDA and Neural Network”, World Academy of Science,
Engineering and Technology 41008
[17] Kyungim Baek, Bruce A. Draper, J. Ross Beveridge, Kai , “PCA vs. ICA: A
comparison on the FERET data set”.
[18] B. Moghaddam, and A. Pentland, “An Automatic System for Model-Based Coding of
Faces”, pp. 362- 370, IEEE, 1995.
[19] Fleming, M., and Cottrell, G., "Categorization of faces using unsupervised feature
extraction", Proc. of IJCNN, Vol. 90(2), 1990.
[20] T. Yahagi and H. Takano, “Face Recognition using neural networks with multiple
combinations of categories,” International Journal of Electronics Information and
Communication Engineering., vol.J77-D-II, no.11, pp.2151-2159, 1994.
[21] Patrick J. Grother; George W. Quinn; P J. Phillips,”Report on the Evaluation of 2D
Still-Image Face Recognition Algorithms”, NIST Interagency/Internal Report (NISTIR) –
7709, June 17, 2010.
[22] Patrick J. Grother, George W. Quinn and P. Jonathon Phillips,” Report on the
Evaluation of 2D Still-Image Face Recognition Algorithms”, NIST Interagency Report
7709.
[23] Patrik Kamencay, Robert Hudec, Miroslav Benco and Martina Zachariasova, “2D-
3D Face Recognition Method Based on a Modified CCA-PCA Algorithm “, International
Journal of Advanced Robotic System, DOI: 10.5772/58251
[24] Phillip Ian Wilson, Dr. John Fernandez “Facial Feature Detection Using HAAR
Classifiers”, Texas A&M University – Corpus
[25] Matthew Turk and Alex Pentland,”Eigenfaces for Massachusetts Institute of
Technology Recognition, vision and Modeling Group “, The Media Laboratory](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-93-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 84
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
[26] Paulo Menezes, Jos´e Carlos Barreto and Jorge Dias,” Face Tracking Based On
Haar-Like Features And Eigenfaces”,ISR-University of Coimbra.
[27] Barreto, Jos´e, Paulo Menezes and Jorge Dias,” Human-robot interaction based on
Haar-like features and eigenfaces”, International Conference on Robotics and
Automation,(2004).
[28] Freund, Yoav and Robert E.Schapire (1996),”Experiments with a new boosting
algorithm.”
[29] Kalman, R.E. (1960),”A new approach to linear filtering and prediction problems”,
Journal of Basic Engineering (82series D), 35–45.
[30] Lienhart, Rainer and Jochen Maydt (2002).” An extended set of haar-like features
for rapid object detection. “In: IEEE ICIP 2002, Vol. 1, pp 900-903.
[31] Moghaddam, B. and A.P. Pent and A.P. Pentland (1995). “Probabilistic visual
learning for object representation”, Technical Report 326, Media Laboratory,
Massachusetts Institute of Technology.
[32] Mohan, Anuj, Constantine Papageorgiou and Tomaso Poggio (2001).,”Example-
based object detection in images by components”, IEEE Transactions on Pattern Analysis
and Machine Intelligence 23(4), 349–361.
[33] Oren, M., C.Papageorgiou, P.Sinha, E.Osuna and T.Poggio (1997),” Pedestrian
detection using wavelet templates”.
[34] Rainer Lienhart, Alexander Kuranov and Vadim Pisarevsky (2002),”Empirical
analysis of detection cascades of boosted classifiers for rapid object detection”, MRL
Technical Report, Intel Labs.
[35] Rowley, Henry A., Shumeet Baluja and Takeo Kanade (1998),”Neural network-
based face Detection”, IEEE Transactions on Pattern Analysis and Machine Intelligence
20(1), 23– 38.
[36] Schneiderman, H. and T. Kanade (2000),” A statistical method for 3D object
detection applied to faces and cars”, In International Conference on Computer Vision.
[37] Sung, K. and T. Poggio (1998),“Example-based learning for view based face
detection.” In IEEE Patt. Anal. Mach. Intell. 20(1), 39–51.
[38] Turk, M.A. and A.P. Pentland (1991),”Face recognition using eigenfaces”, In Proc.
of IEEE Conference on Computer Vision and Pattern Recognition pp. 586 – 591.](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-94-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 85
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
[39] Viola, Paul and Michael Jones (2001),”Rapid object detection using boosted cascade
of simple features”, IEEE Conf.Pattern Recognition 2001.
[40] Yang, Ming-Hsuan (2002),”Detecting faces images: A survey”. IEEE Transations on
Pattern Analysis and Machine Intelligence 24(1), 34–58.
[41] Sung, K. and T. Poggio (1998).” Example-based learning for view based face
detection.”
[42] Henry Rowley, Baluja S. & Kanade T. (1999) “Neural Network-Based Face
Detection, Computer Vision and Pattern Recognition”, Neural Network-Based Face
Detection, Pitts-burgh, Carnegie Mellon University, PhD thesis.
[43] KahKay Sung & Tomaso Poggio (1994) Example Based Learning For View Based
Human Face Detection, Massachusetts Institute of Technology Artificial Intelligence
Laboratory and Center For Biological And Computational Learning, Memo 1521, CBCL
Paper 112, MIT, December.
[44] Henry A. Rowley, Shumeet Baluja &Takeo Kanade. (1997) Rotation Invariant
Neural Network-Based Face Detection, December, CMU-CS-97-201
[45] Jeffrey S. Norris (1999) Face Detection and Recognition in Office Environments,
thesis, Dept. of Electrical Eng. and CS, Master of Eng in Electrical Eng., Massachusetts
Institute of Technology.
[46] Hazem M. El-Bakry (2002), Face Detection Using Neural Networks and Image
Decomposition Lecture Notes in Computer Science Vol. 22, pp:205-215.
[47]. M. Kass, A. Witkin, and D. Terzopoulos. Snakes: Active contour models. IJCV,
1(4):321–331, 1987
[48]. T. Cootes, C. Taylor, D. Cooper, and J. Graham. Active shape models - Their
training and application. CCVIU, 61(1):38–59, 1995.
[49]. T. Cootes, G. Edwards, and C. Taylor. Active appearance models. PAMI,
23(6):681–685, 2001.
[50]. S. Sclaroff and J. Isidoro. Active blobs. In ICCV, 1998.
[51]. M. Jones and T. Poggio. Multidimensional morphable models. In ICCV, pages 683–
688, 1998.
[52]. P. Ekman and W.V. Friesen. Facial Action Coding System: Investigator’s Guide.
Consulting Psychologists Press, 1978.](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-95-2048.jpg)
![PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM 86
DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI
[53]. B. Fasel and J. Luettin. Automatic facial expression analysis: A survey. Pattern
Recognition, 36:259–275, 2003.
[54]. M. Pantic and L.J.M. Rothkrantz. Automatic analysis of facial expressions: The
state of the art. IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(12):1424–
1445, 2000.](https://image.slidesharecdn.com/2009ht12895-report-160710200025/75/Photograph-Database-management-report-96-2048.jpg)
More Related Content
Similar to Photograph Database management report
Photograph Database management report
- 1. Photograph Database ManagementUsing Face Recognition Algorithm BITS ZG628T: Dissertation by Amrit Ranjan 2009HT12895 Dissertation work carried out at Microsoft GD, Bangalore BIRLA INSTITUTE OF TECHNOLOGY & SCIENCE PILANI (RAJASTHAN) April 2016
- 2. Photograph Database ManagementUsing Face Recognition Algorithm BITS ZG628T: Dissertation by Amrit Ranjan ID No. 2009HT12895 Dissertation work carried out at Microsoft GD, Bangalore Submitted in partial fulfillment of M.Tech. Software Systems degree programme Under the Supervision of Anuj Gupta, Consultant, Microsoft GD, Bangalore BIRLA INSTITUTE OF TECHNOLOGY & SCIENCE PILANI (RAJASTHAN) April, 2016
- 4. ABSTRACT This thesis proposedthe face recognition method using common and most sophisticated face recognition algorithm to maintain and manage image or photograph database. Here Photograph Image means Image with human faces. In this thesis, image’s faces are extracted and classified and indexed. Indexed values are later used for creating logical database of images. This indexed values will be used to search and display related images. This is also features about the usage of the indexed values to identify or search one face with combinations of other faces and it will help police to investigate in more details. In this thesis a feature is extracted using principal component analysis and then classification by creation neural network. Recently, the PCA (Principal Component Analysis) has been extensively employed for face recognition algorithms. It is one of the most popular representation methods for a face image. It not only reduces the dimensionality of the image, but also retains some of the variations in the image data. We run our algorithm for face recognition application using principal component analysis. The system produces promising results for face verification and face recognition. Demonstrate the recognition accuracy for given number of input pattern. Our approach treats face recognition problem as intrinsically(inherent) two- dimensional (2-D) recognition problem rather than requiring recovery of three- dimensional(3-D) geometry, taking advantage of fact that faces are normally upright and thus may be described by a small set of 2-D characteristic views. The system functions by projecting face images onto a feature space that spans the significant variations among known face images. The significant features are known as “eigenfaces”, because they are the eigenvectors (principal components) of the set of faces. Computer model of face recognition in particular, are interesting because they can contribute not only to theoretical insights but also to the practical applications. Computer that recognizes faces could be applied to a wide variety of problems like criminal identification, security system, image and film processing and human-computer interaction. Keywords- Face recognition, PCA, Eigenfaces, Eigenvectors, Machine Learning.
- 5. ACKNOWLEDGEMENT The satisfaction thatI feel at the successful completion of the Dissertation work titled “PHOTOGRAPH DATABASE MANAGEMENT USING FACE RECOGNITION ALGORITHM” would be incomplete if I do not mention the names of the people whose valuable guidance and encouragement has made this project work a success. It is a great pleasure to express my gratitude and respect to all those who inspired and helped me in completing this project. I would like to express my immense gratitude to Principal and H.O.D of the Department of Computer Science, BITS Pilani, for their valuable assistance and Co- operation. It is also pleasure to express my gratitude to all Teaching and Non- Teaching staff members of Department of Computer Science, BITS Pilani for their encouragement and providing valuable requirements. With a deep sense of indebtedness, I convey my heartiest thanks to my parents who have taken effort and given us such an opportunity. Their love and blessing has given us to acquire knowledge and gain experience in our life. As well as my friends who have helped for this accomplished task. Amrit Ranjan
- 6. TABLE OF CONTENTS SLNo. Topic Page No. 1 INTRODUCTION………………………………………………………........................... 1.1 PURPOSE…………………………………………………….……………………... 1.2 SCOPE…………………………………………………..………………………....... 1.3 EXISTING SYSTEM……………….…………………………………..……...…… 1.4 LIMITATIONS OF THE EXISTING SYSTEM……………………..…..…...……. 1.5 PROPOSED SYSTEM AND ITS ADVANTAGES ………………………....……... 1 2 3 3 3 4 2 LITERATURE SURVEY………………………………………………………..………. 5 2.1 BACKGROUND………………………………………………………….............. 5 2.2 FACE RECOGNITION TECHNOLOGY OVERVIEW………………………... 8 2.3 PHOTOMETRIC (VIEW BASED) ……………………………………………... 10 2.4 EGMU CV …………………………………………………………………….… 14 2.5 PCA ………………………………………………………..…………………..… 14 2.6 ADVANTAGES AND DISADVANTAGES OF PCA ….………………..…..… 18 2.7 BIOMETRIC FACIAL RECOGNITION REVENUE ……………….………… 21 2.8 PCA VERSUS 2DPCA.……………………………………………………….…. 22 2.9 CONCLUSION…………………………………………………………............... 23 2.10 HAAR FEATURE ALGORITHMS …….…......................................................... 24 2.11 REGIONALIZED DETECTION INTERNET…………………….….……......... 25 2.12 COMPUTATIONS AND EXAMPLES………………………………………...... 26 2.13 RESULTS………………………………………………………………………… 37 2.14 ARTIFICIAL NEURAL NETWORKS FOR FACE DETECTION……………… 37
- 7. 2.15 MACHINE LEARNING……………………………………………………………41 2.16 FUTURE PLANS………………………………..…………………......................... 46 3 OBJECTIVES AND METHODOLOGY……………………………............................... 47 3.1 FACE RECOGNITION ………………………………............................................ 47 3.2 FACE INDEXING ……………………………………..……………………........... 49 3.3 FACE SEARCH …………………............................................................................ 53 3.4 SIMULATION……………………………………………………………………… 54 3.5 METHODOLOGY …………………………………………..………………...… 67 4 ANALYSIS AND INFERENCE…………………………………..……………..……….. 73 4.1 SIMULATION RESULTS AND ANALYSIS ………………..………………........ 73 4.2 SNAPSHOTS…………………………………………………………...................... 74 5 CONCLUSION………………………………………………..………………………….. 80 6 FUTURE ENHANCEMENT…………………………………………………………....... 81 7 REFERENCES…………………………………………………………............................. 82
- 8. LIST OF FIGURES Fig.No Fig. Name Page No. 2.3 a. Facial Surface b. Basic Face Recognition 11 12 2.8 Examples from the faces dataset 22 2.12 a. PCA for Data Representation b. PCA for Dimension Reduction c. The PCA Transformation d. 3D surfaces in correspondence 26 27 30 36 2.14 a. Retinal Connected Neural Network (Rcnn) b. Rotation Invariant Neural Network (Rinn) c. Principal Component Analysis With Ann (Pca & Ann) 39 40 41 3.2 a. Face Indexing b. Face Detection and Training sets c. Face Detection and Facial expression classification 49 50 51 3.3 Face Search 53 3.5 a.SCRUM Process b.SCRUM Roles 68 71
- 9. LIST OF TABLES TableNo. Title Page No. 3.5 a. System Configuration b. Index c. Search 72 72 72
- 10. ABBREVIATIONS PCA Principle ComponentAnalysis OCR FRVT Optical Character Recognition Face Recognition Vendor Test FRT Facial Recognition Technology HCI Human Computer Interfaces LFA Local Feature Analysis STA Surface Texture Analysis SVD Single Value Decomposition 2DPCA 2-Dimentional Principle Component Analysis
- 11. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 1 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 1. INTRODUCTION Over the last ten years or so, face recognition has become a popular area of research in computer vision and one of the most successful applications of image analysis and understanding. It not only helps computer science researchers in security and authentication of a person, which is a crucial part of any industry but neuroscientists and psychologists also. There are many techniques has evolved and tested for face detection and recognition. In present era one of dramatic changes happened is related to easy availability of handy smart devices with camera. Digital images are revolutionary in many context. If we do comparison with reel based camera to digital camera system, storage are not concern at all. We can click as many photographs and can store in memory itself. Adding more and more images makes it difficult to find photograph which we are looking for. This thesis work is carried out to logically index photographs with the faces and to address the identified above issue. This depends on already proven and tested face detection and recognition algorithm. There are many techniques used for these purpose one of them is face recognition. Face recognition is an effective means of authenticating a person the advantage of this approach is that, it enables us to detect changes in the face pattern of an individual to an appreciable extent the recognition system can tolerate local variations in the face expressions of an individual. Hence face recognition can be used as a key factor in crime detection mainly to identify criminals there are several approaches to face recognition of which principal component analysis (PCA) and neural networks have been incorporated in our project face recognition as many applicable areas. The system consist of a database of a set of facial patterns for each individual. The characteristic features called ‘eigen faces’ are extracted from the storage images using which the system is trained for subsequent recognition of new images. Principal component analysis (PCA) is one of the popular methods used for feature extraction and data representation. It not only reduces the dimensionality of the image, but also retains some of the variations in the image data and provides a compact representation of a face image. The key idea of the PCA method is to transform the face images into a small set of characteristics feature images, called Eigen faces, which are the principal components of the initial training set of the face image.
- 12. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 2 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 1.1 PURPOSE We live in digital world. Managing growing photographs are becoming bottleneck, especially to identify who belongs to which photographs. This project will help to manage and index faces in each photographs. Indexing helps to search images faster. The proposed face recognition system overcomes certain limitations of the existing face recognition system. Existing system is capable of searching images from training faces but it comes with overhead of time constraint. Searching faces in hundreds and thousands of photographs is bottleneck in context of time. Proposed system is based on extracting the dominating features of a set of human faces stored in the database and performing mathematical operations on the values corresponding to them. Hence when a new image is fed into the system for recognition the main features are extracted and computed to find the distance between the input image and the stored images. Thus, some variations in the new face image to be recognized can be tolerated. When the new image of a person differs from the images of that person stored in the database, the system will be able to recognize the new face and identify who the person is. The recognize faces are presented to the user and allow them to map with the face belongs to. The proposed system is better mainly due to the use of facial features rather than the entire face. Its advantages are in terms of: • Face search is more accurate and fast. Two and more faces could be combined to do unique searches. • Recognition accuracy and better discriminatory power Computational cost because smaller images (main features) require less processing to train the PCA. • Because of the use of dominant features and hence can be used as an effective means of authentication 1.2 SCOPE The scope of this project is to feasible study, plan, design, build and implemented “Photograph Database Management Using Face Recognition Algorithm” through which photographs and faces will be managed and indexed. This project will allow user to select face name and interface to search faces with various combinations.
- 13. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 3 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 1.3 EXISTING SYSTEM • Face recognition biometrics is the science of programming a computer to recognize a human face. When a person is enrolled in a face recognition system, a video camera takes a series of snapshots of the face and then represents it by a unique holistic code. • When someone has their face verified by the computer, it captures their current appearance and compares it with the facial codes already stored in the system. • The faces match, the person receives authorization; otherwise, the person will not be identified. The existing face recognition system identifies only static face images that almost exactly match with one of the images stored in the database. • When the current image captured almost exactly matches with one of the images stored then the person is identified and granted access. • When the current image of a person is considerably different, say, in terms of facial expression from the images of that person which are already stored in the database the system does not recognize the person and hence access will be denied. 1.4 LIMITATIONS OF THE EXISTING SYSTEM The existing or traditional face recognition system has some limitations which can be overcome by adopting new methods of face recognition: • Though existing system able to identify and recognize face from photograph, but it takes reasonable amount of time to do facial search in each photograph. And it will become impossible with high volume of imagesphotographs. • The existing system cannot tolerate variations in the new face image. It requires the new image to be almost exactly matching with one of the images in the database which will otherwise result in denial of access for the individual. • The performance level of the existing system is not appreciable.
- 14. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 4 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 1.5 PROPOSED SYSTEM AND ITS ADVANTAGES • Face search is more accurate and fast. Two and more faces could be combined to do unique searches. • Recognition accuracy and better discriminatory power Computational cost because smaller images (main features) require less processing to train the PCA. • Because of the use of dominant features and hence can be used as an effective means of authentication.
- 15. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 5 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 2. LITERATURE SURVEY 2.1 BACKGROUND Biometrics based human computer interfaces (HCI) are becoming the most popular systems for security and access control. Face recognition system is one such biometric HCI system. It distinguishes among various human faces and identifies a face from a digital image or a video frame from a video source. This is normally done by comparing selected facial features of the image with those in a facial database that contain multiple images of people and their information. The way humans recognize faces is highly remarkable and efficient irrespective of the changes in the visual input due to lighting conditions, background, and aging. [1] To build a system that can match humans in this process is highly desirable yet very difficult to build. A basic face recognition system may follow a Face Detection System whose function is to identify a face in a given image and ignore all the other background details. After the face image is extracted it is given as input to the Face Recognition System, which first extracts basic features of a face that distinguishes one from the other and then classifiers are used to match images with those stored in the database to identify a person. Therefore a basic face recognition system contains the following sub-modules [2]. Face Detection Feature Extraction Feature Matching/Classification In a work proposed by R. Chellappa, P. J. Phillips, A. Rosenfeld entitled ‘Face Recognition: A Literature Survey’[3] describes face recognition has recently received significant attention, especially during the past several years, as one of the most successful applications of image analysis and understanding. This paper provides an up- to-date critical survey of still and video-based face recognition research. There are two underlying motivations for them to write this survey paper: the first is to provide an up- to-date review of the existing literature, and the second is to offer some insights into the studies of machine recognition of faces. To provide a comprehensive survey, they not only categorize existing recognition techniques but also present detailed descriptions of representative methods within each category. In addition, relevant topics such as
- 16. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 6 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI psychophysical studies, system evaluation, and issues of illumination and pose variation are covered. In the work proposed by Matthew Turk and Alex Pentland in ‘Eigenfaces for Recognition ‘[4], they have developed a near real-time computer system that can locate and track a person’s head and then recognize it by comparing the features of this face with those of known individuals. They treat the face recognition problem as an intrinsically two dimensional recognition problem. The system functions by projecting face images onto a feature space that spans the significant variations among known face images. The significant features are known as ‘eigenfaces’, because they are the eigen vectors of the set of faces, they do not necessarily correspond to features such as eyes, ears and noses. The projection operation characterizes an individual face by a weighted sum of the eigenface features, and so to recognize a face it is necessary only to compare these weights to those of known individuals. Some particular advantages of the proposed approach are that it provides for the ability to learn and later recognize new faces in an unsupervised manner, and that it is easy to implement using a neural network approach. In the work proposed by Kyungnam Kim entitled ‘Face Recognition using Principal Component Analysis’ [5] illustrates that the Principal Component Analysis (PCA) is one of the most successful techniques that have been used in image recognition and compression. The purpose of PCA is to reduce the large dimensionality of the dataspace to the smaller intrinsic dimensionality of feature space (independent variables), which are needed to describe the data economically. The main idea of using PCA for face recognition is to express the large 1-D vector of pixels constructed from 2-D facial image into the compact principal components of the feature space. This is called Eigen space projection. Eigen space is calculated by identifying the eigenvectors of the covariance matrix derived from a set of facial images (vectors). The paper describes the mathematical formulation of PCA and a detailed method of face recognition by PCA. In a work by Baocai Yin Xiaoming Bai, Qin Shi, Yanfeng Sun entitled ‘Enhanced Fisher face for Face Recognition’ Journal of Information & Computational Science 2: 3 (2005) 591- 595[6] Fisher face is enhanced in this paper for face recognition from one example image per person. Fisher face requires several training images for each face and can hardly be applied to applications where only one example image per person is available for training. They enhance Fisher face by utilizing morphable model to derive multiple
- 17. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 7 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI images of a face from one single image. Region filling and hidden-surface removal method are used to generate virtual example images. Experimental results on ORL and UMIST face database show that our method makes impressive performance improvement compared with conventional Eigenface methods. In another work contributed by P. Latha, Dr. L. Ganesan, Dr. S. Annadurai, entitled “Face Recognition using Neural Networks” , Signal Processing: An International Journal (SPIJ) Volume (3) : Issue (5)[7] explains that Face recognition is one of biometric methods, to identify given face image using main features of face. In this paper, a neural based algorithm is presented, to detect frontal views of faces. The dimensionality of face image is reduced by the Principal component analysis (PCA) and the recognition is done by the Back propagation Neural Network (BPNN). In this project they have used 200 face images from Yale database and some performance metrics like Acceptance ratio and Execution time have been calculated. The Neural based Face recognition approach presented in this paper is robust and has better performance of more than 90 % acceptance ratio. In the work proposed by Mayank Agarwal, Nikunj Jain, Mr. Manish Kumar and Himanshu Agrawal, in “Face Recognition Using Eigen Faces and Artificial Neural Network”, International Journal of Computer Theory and Engineering, Vol. 2, No. 4, August, 2010[8]. Face is explained to be a complex multidimensional visual model and developing a computational model for face recognition is said to be difficult. The paper presents a methodology for face recognition based on information theory approach of coding and decoding the face image. Proposed methodology is connection of two stages – Feature extraction using principle component analysis and recognition using the feed forward back propagation Neural Network. The algorithm has been tested on 400 images (40 classes). A recognition score for test lot is calculated by considering almost all the variants of feature extraction. The proposed methods were tested on Olivetti and Oracle Research Laboratory (ORL) face database. The test results are said to give a recognition rate of 97.018%.
- 18. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 8 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 2.2 FACE RECOGNITION TECHNOLOGY Facial recognition technology (FRT) has emerged as an attractive solution to address many contemporary needs for identification and the verification of identity claims. It brings together the promise of other biometric systems, which attempt to tie identity to individually distinctive features of the body, and the more familiar functionality of visual surveillance systems. This report develops a socio-political analysis that bridges the technical and social-scientific literatures on FRT and addresses the unique challenges and concerns that attend its development, evaluation, and specific operational uses, contexts, and goals. It highlights the potential and limitations of the technology, noting those tasks for which it seems ready for deployment, those areas where performance obstacles may be overcome by future technological developments or sound operating procedures, and still other issues which appear intractable. Its concern with efficacy extends to ethical considerations. Every face has numerous, distinguishable landmarks, the different peaks and valleys that make up facial features. FaceIt defines these landmarks as nodal points. Each human face has approximately 80 nodal points. Some of these measured by the software are: Distance between the eyes Width of the nose Depth of the eye sockets The shape of the cheekbones The length of the jaw line. There are two predominant approaches to the face recognition problem: Geometric (feature based) and Photometric (view based). Geometric (feature based)
- 19. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 9 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI In the geometric feature-based approach the primary step is to localize and track a dense set of facial points. Most geometric feature-based approaches use the active appearance model (AAM) or its variations, to track a dense set of facial points. The locations of these facial landmarks are then used in different ways to extract the shape of facial features, and movement of facial features, as the expression evolves. [9] use AAM with second order minimization, and a multilayer perceptron, for the recognition of facial expressions. A recent example of an AAM-based technique for facial expression recognition is presented in [10], in which different AAM fitting algorithms are compared and evaluated. Another example of a system that uses geometric features to detect facial expressions is that by Kotisa et al. [11]. A Kanade-Lucas-Tomasi (KLT) tracker was used, after manually locating a number of facial points. The geometric displacement of certain selected candid nodes, defined as the differences of the node coordinates between the first and the greatest facial expression intensity frames, were used as an input to the novel multiclass SVM classifier. Sobe et al. [12] also uses geometric features to detect emotions. In this method, the facial feature points were manually located, and Piecewise Bezier volume deformation tracking was used to track those manually placed facial landmarks. They experimented with a large number of m machine learning techniques, and the best result was obtained with a simple k-nearest neighbor technique. Sung and Kim [13] introduce Stereo AAM (STAAM), which improves the fitting and tracking of standard AAMs, by using multiple cameras to model the 3-D shape and rigid motion parameters. A layered generalized discriminant analysis classifier is then used to combine 3-D shape and registered 2-D appearance, for the recognition of facial expressions. In [14], a geometric features-based approach for modeling, tracking, and recognizing facial expressions on a low-dimensional expression manifold was presented. Sandbach et al. [15] recently presented a method that exploits 3D motion-based features between frames of 3D facial geometry sequences, for dynamic facial expression recognition. Onset and offset segment features of the expression are extracted, by using feature selection methods. These features are then used to train the GentleBoost classifiers, and build a Hidden Markov Model (HMM), in order to model the full temporal dynamic of the expressions. Rudovic and Pantic [16] introduce a method for head-pose invariant facial expression recognition that is based on a set of characteristic facial points,
- 20. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 10 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI extracted using AAMs. A coupled scale Gaussian process regression (CSGPR) model is used for head-pose normalization. Researchers have also developed systems for facial expression recognition, by utilizing the advantages of both geometric-based and appearance-based features. Lyons and Akamatsu [17] introduced a system for coding facial expressions with Gabor wavelets. The facial expression images were coded using a multi-orientation, multi-resolution set of Gabor filters at some fixed geometric positions of the facial landmarks. The similarity space derived from this representation was compared with one derived from semantic ratings of the images by human observers. In [18], a comparison between geometric- based and Gabor-wavelets-based facial expression recognition using multi-layer perceptorn was presented. Huang et al. [19] presented a boosted component-based facial expression recognition method by utilizing the spatiotemporal features extracted from dynamic image sequences, where the spatiotemporal features were extracted from facial areas centered at 38 detected fiducial interest points. 2.3 PHOTOMETRIC (VIEW BASED) Photometric is a technique in computer vision for estimating the surface normal of objects by observing that object under different lighting conditions. It is based on the fact that the amount of light reflected by a surface is dependent on the orientation of the surface in relation to the light source and the observer. By measuring the amount of light reflected into a camera, the space of possible surface orientations is limited. Given enough light sources from different angles, the surface orientation may be constrained to a single orientation or even over constrained. The technique was originally introduced by Woodham in 1980. The special case where the data is a single image is known as shape from shading, and was analyzed by B. K. P. Horn in 1989. Photometric stereo has since been generalized to many other situations, including extended light sources and non-Lambertian surface finishes. Current research aims to make the method work in the presence of projected shadows, highlights, and non- uniform lighting. Surface normal define the local metric, using this observation Bronstein et al. [20] defined a 3D face recognition system based on the reconstructed metric
- 21. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 11 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI without integrating the surface. The metric of the facial surface is known to be robust to expressions. Figure 2.3 (a) Facial Surface[20] The input of a face recognition system is always an image or video stream. The output is an identification or verification of the subject or subjects that appear in the image or video. A face recognition system as a three step process Figure 2.3 (b) Basic Face Recognition
- 22. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 12 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Face detection is defined as the process of extracting faces from scenes. So, the system positively identifies a certain image region as a face. This procedure has many applications like face tracking, pose estimation or compression. The next step -feature extraction- involves obtaining relevant facial features from the data. These features could be certain face regions, variations, angles or measures, which can be human relevant (e.g. eyes spacing) or not. This phase has other applications like facial feature tracking or emotion recognition. Finally, the system does recognize the face. In an identification task, the system would report an identity from a database. This phase involves a comparison method, a classification algorithm and an accuracy measure. This phase uses methods common to many other areas which also do some classification process -sound engineering, data mining et al. These phases can be merged, or new ones could be added. Therefore, we could find many different engineering approaches to a face recognition problem. Face detection and recognition could be performed in tandem, or proceed to an expression analysis before normalizing the face. Difference between verification and Identification. Verification: Face verification is a 1:1 match that compares a face image against a template face images, whose identity is being claimed. Identification: Face identification is a 1: N problem that compares a query face image against all image templates in a face database to determine the identity of the query face. At last a third scenario, the watch list, has been proposed in Face Recognition Vendor Test (FRVT2002) (Phillips et al., 2002). The test individual may or may not be in the system database. The query face image is compared against all the face images in the database, computing a score for each one. All these scores are numerically ranked so that the highest score is first, and if a similarity score is higher than a given threshold, an alarm is raised. In last decade, major advances occurred in face recognition, with many systems capable of achieving recognition rates greater than 90%. However real-world scenarios remain a challenge, because face acquisition process can undergo to a wide range of variations.
- 23. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 13 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI There are five key factors that can significantly affect system face recognition performances: • Illumination variations due to skin reflectance properties and due to the internal camera control. Several 2D methods do well in recognition tasks only under moderate illumination variation, while performances noticeably drop when both illumination and pose changes occur. • Pose changes affect the authentication process, because they introduce projective deformations and self-occlusion. Even if methods dealing with up to 32 head rotation exists, they do not solve the problem considering that security cameras can create viewing angles that are outside of this range when positioned. On the contrary, with exception of extreme expressions such as scream, the algorithms are relatively robust to facial expression. • Another important factor is the time delay, because the face changes over time, in a nonlinear way over long periods. In general this problem is harder to solve with respect to the others and not much has been done especially for age variations. • At last, occlusions can dramatically affect face recognition performances, in particular if they located on the upper-side of the face, as documented in literature. In order to assess how well proposed methods work when dealing with one or a combination of these variation, several face images databases have been built. 2.4 EGMU CV Emgu CV is a cross platform .Net wrapper to the Intel OpenCV image processing library. Allowing OpenCV functions to be called from .NET compatible languages such as C#, VB, VC++, IronPython etc. The wrapper can be compiled in Mono and run on Linux / Mac OS X. EGMU CV helps us to perform following tasks Optical Character Recognition (OCR) Face Detection
- 24. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 14 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Pedestrian Detection Kinect projects 3D reconstruction 2.5 PCA PCA is known a Principle Component Analysis – this is a statistical analytical tool that is used to explore, sort and group data. What PCA does is take a large number of correlated (interrelated) variables and transform this data into a smaller number of uncorrelated variables (principal components) while retaining maximal amount of variation, thus making it easier to operate the data and make predictions. Or as Smith (2002) puts it “PCA is a way of identifying patterns in data, and expressing the data in such a way as to highlight their similarities and differences[13]. Since patterns in data can be hard to find in data of high dimension, where the luxury of graphical representation is not available, PCA is a powerful tool for analyzing data.” History of PCA According to Jolliffe (2002) it is generally accepted that PCA was first described by Karl Pearson in 1901. In his article ” On lines and planes of closest fit to systems of points in space,” Pearson (1901) discusses the graphical representation of data and lines that best represent the data. He concludes that “The best-fitting straight line to a system of points coincides in direction with the maximum axis of the correlation ellipsoid”. He also states that the analysis used in his paper can be applied to multiple variables. However, PCA was not widely used until the development of computers. It is not really feasible to do PCA by hand when number of variables is greater than four, but it is exactly for larger amount of variables that PCA is really useful, so the full potential of PCA could not be used until after the spreading of computers (Jolliffe, 2002). According to Jolliffe (2002) significant contributions to the development of PCA were made by Hotelling (1933) and Girshick (1936; 1939) before the expansion in the interest towards PCA. In 1960s. as the interest in PCA rose, important contributors were Anderson (1963) with a theoretical discussion, Rao (1964) with numerous new ideas concerning uses, interpretations and
- 25. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 15 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI extensions of PCA, Gower (1966) with discussion about links between PCA and other statistical techniques and Jeffers (1967) with a practical application in two case studies. BASIC ASSUMPTIONS According to Shlens (2009) there are three basic assumptions behind PCA that need to be considered when calculating and interpreting principal components: • Linearity - Linearity frames the problem as a change of basis. Several areas of research have explored how extending these notions to nonlinear regimes. • Large variances have important structure - This assumption also encompasses the belief that the data has a high SNR. Hence, principal components with larger associated variances represent interesting structure, while those with lower variances represent noise. Note that this is a strong, and sometimes, incorrect assumption. • The principal components are orthogonal - This assumption provides an intuitive simplification that makes PCA soluble with linear algebra decomposition techniques. IMPORTANT CONCEPTS Principal component - a linear combination of the original variables (1st principal component explains most of the variation n the data, 2nd PC explains most of the rest of the variance and so on) Eigenvectors - the coefficients of the original variables used to construct factors Eigenvalue - a corresponding scalar value for each eigenvector of a linear transformation. CALCULATING PRINCIPAL COMPONENTS Jolliffe (2002) states that principal components (PCs) can be found using purely mathematical arguments – they are given by an orthogonal linear transformation of a set
- 26. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 16 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI of variables optimizing a certain algebraic criterion. Shlens 2009) gives an overview how to perform principal components analysis: Organize data as an m×n matrix, where m is the number of measurement types and n is the number of samples Subtract off the mean for each measurement type Calculate covariance matrix Calculate the eigenvectors and eigenvalues of the covariance matrix. DERIVING PRINCIPAL COMPONENTS Following is a standard derivation of principal components presented by Jolliffe (2002) To derive the form of the PCs, consider first α’1x; the vector a1 maximizes Var [α’x]= α’1∑ α1 It is clear that, as it stands, the maximum will not be achieved for finite α1 so a normalization constraint must be imposed. The constraint used in the derivation is α1α1 = 1, that is, the sum of squares of elements of α1 equals 1. Other constraints may be more useful in other circumstances, and can easily be substituted later on. However, the use of constraints other than α’1α1 = constant in the derivation leads to a more difficult optimization problem, and it will produce a set of derived variables different from the principal components. To maximize α’1∑α subject α’1∑α-λ (α’1α1-1) to α1α1 = 1, the standard approach is to use the technique of Lagrange multipliers. ∑α1-λα1=0 (∑-λ Ip) α1 =0 Maximize where λ is a Lagrange multiplier. Differentiation with respect to α1 gives or Where Ip is the (p x p) identity matrix. Thus, ë is an eigenvalue and á1 is the corresponding eigenvector. To decide which of the α’1 ∑α1= α’1λ α1= λ ,the eigenvectors gives á’1x with maximum variance, note that the quantity to be maximized is so ë must be as large as possible. Thus, á1 is the eigenvector corresponding to the largest eigenvalue, and var [α’1x] = α’1∑α1 = λ1, the
- 27. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 17 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI largest eigenvalue. In general, the kth PC of x is a’kx and, where ëk is the kth largest eigenvalue of var [α’kx] =λk for k=1, 2, 3,……,p. And ák is the corresponding eigenvector. Shlens (2009) derives an algebraic solution to PCA based on an important property of eigenvector decomposition. Once again, the data set is X, an m×n matrix, where m is the number of measurement types and n is the number of samples. The goal is summarized as follows: Find some orthonormal matrix P in Y = PX such that CY=1/n (YYT) is a diagonal matrix. The rows of P are the principal components of X. The rewriting CY in terms of the unknown variable.
- 28. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 18 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 2.6 ADVANTAGES AND DISADVANTAGES OF PCA IMPORTANCE OF PCA Principal component analysis (PCA) is a standard tool in modern data analysis - in diverse fields from neuroscience to computer graphics - because it is a simple, non- parametric method for extracting relevant information from confusing data sets. With minimal effort PCA provides a roadmap for how to reduce a complex data set to a lower dimension to reveal the sometimes hidden, simplified structures that often underlie it. (Shlens, 2009) Importance of PCA is manifested by its use in so many different fields of science and life. PCA is very much used in neuro-science, for example. Another fields of use are pattern recognition and image compression, therefore PCA is suited for use in facial recognition software for example, as well as for recognition and storing of other biometric data. Many IT related fields also use PCA, even artificial intelligence research. According to Jolliffe (2002) PCA is also used in research of agriculture, biology, chemistry, climatology, demography, ecology, food research, genetics, geology, meteorology, oceanography, psychology, quality control, etc. But in this paper we are going to focus more on uses in finance and economy. PCA has been used in economics and finance to study changes in stock markets, commodity markets, economic growth, exchange rates, etc. Earlier studies were done in economics, but stock markets were also under research already in 1960s. Lessard (1973) claims that “principal component or factor analysis have been used in several recent empirical studies (Farrar [1962], King [1967], and Feeney and Hester [1967]) concerned with the existence of general movements in the returns from common stocks.” PCA has mostly been used to compare different stock markets in search for diversification opportunities, especially in earlier studies like the ones by Makridakis (1974) and by Phillipatos et al. (1983). BENEFITS OF PCA PCA is a special case of Factor Analysis that is highly useful in the analysis of many time series and the search for patterns of movement common to several series (true factor analysis makes different assumptions about the underlying structure and solves
- 29. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 19 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI eigenvectors of a slightly different matrix). This approach is superior to many of the bivariate statistical techniques used earlier, in that it explores the interrelationships among a set of variables caused by common "factors," mostly economic in nature. (Philippatos, Christofi, & Christofi, 1983) PCA is a way of identifying patterns in data, and expressing the data in such a way as to highlight their similarities and differences. A primary benefit of PCA arises from quantifying the importance of each dimension for describing the variability of a data set(Shlens, 2009). PCA can also be used to compress the data, by reducing the number of dimensions, without much loss of information. When using principal component analysis to analyze a data set, it is usually possible to explain a large percentage of the total variance with only a few components. Principal components are selected so that each successive one explains a maximum of the remaining variance, the first component is selected to explain the maximum proportion of the total variance, the second to explain the maximum of the remaining variance, etc. Therefore, the principal component solution is a particularly appropriate test for the existence of a strong market factor PCA is completely nonparametric: any data set can be plugged in and an answer comes out, requiring no parameters to tweak and no regard for how the data was recorded. From one perspective, the fact that PCA is non-parametric (or plug- and-play) can be considered a positive feature because the answer is unique and independent of the user. Limitations of PCA Limitations in PCA occur mainly due to the previously mentioned main assumptions and the data at hand. PCA is not a statistical method from the viewpoint that there is no probability distribution specified for the observations. Therefore it is important to keep in mind that PCA best serves to represent data in simpler, reduced form. It is often difficult, if not impossible, to discover the true economic interpretation of PCs since the new variables are linear combinations of the original variables. In addition, for PCA to work Zi=(xi-μx)/ϭx Exactly, one should use standardized data so that the mean is zero and the unbiased estimate of variance is unity: Where zi =ith standardized variable. This is because it is often the case that the scales of the original variables are not comparable and that (those)
- 30. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 20 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI variable (variables) with high absolute variance will dominate the first principal component. There is one major drawback to standardization, however. Standardizing means that PCA results will come out with respect to standardized variables. This makes the interpretation and further applications of PCA results even more difficult. (Malava, 2006) The mission when using PCA is often to get rid of correlation and interdependence of variables. PCA succeeds in getting rid of second order dependences, but it has trouble with higher-order dependencies. This problem might be solved by using kernel PCA or independent component analysis.
- 31. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 21 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 2.7 BIOMETRIC FACIAL RECOGNITION The image may not always be verified or identified in facial recognition alone. Identix® has created a new product to help with precision. The development of FaceIt®Argus uses skin biometrics, the uniqueness of skin texture, to yield even more accurate results. The process, called Surface Texture Analysis, works much the same way facial recognition does. A picture is taken of a patch of skin, called a skin print. That patch is then broken up into smaller blocks. Using algorithms to turn the patch into a mathematical, measurable space, the system will then distinguish any lines, pores and the actual skin texture. It can identify differences between identical twins, which is not yet possible using facial recognition software alone. According to Identix, by combining facial recognition with surface texture analysis, accurate identification can increase by 20 to 25 percent. Working of facial recognition (Surface texture analysis model) FaceIt currently uses three different templates to confirm or identify the subject: Vector, local feature analysis and surface texture analysis. • The vector template is very small and is used for rapid searching over the entire database primarily for one-to-many searching. • The Local Feature Analysis (LFA) template performs a secondary search of ordered matches following the vector template. • The Surface Texture Analysis (STA) is the largest of the three. It performs a final pass after the LFA template search, relying on the skin features in the image, which contains the most detailed information. By combining all three templates, as an advantage over other systems. It is relatively insensitive to changes in expression, including blinking, frowning or smiling and has the ability to compensate for mustache or beard growth and the appearance of eyeglasses. The system is also uniform with respect to race and gender.
- 32. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 22 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 2.8 PCA VERSUS 2DPCA In the PCA-based face-recognition technique, the 2D face-image matrices must be previously transformed into 1D image vectors. The resulting image vectors usually lead to a high dimensional image vector space in which it’s difficult to evaluate the covariance matrix accurately due to its large size and relatively few training samples. Fortunately, we can calculate the eigenvectors (eigenfaces) efficiently using single value decomposition (SVD) techniques, which avoid the process of generating the covariance matrix. Unlike conventional PCA, 2DPCA is based on 2D matrices rather than 1D vectors. That is, the image matrix doesn’t need to be previously transformed into a vector. Instead, an image covariance matrix can be constructed directly using the original image matrices. In contrast to PCA’s covariance matrix, the image covariance matrix’s size using 2DPCA is much smaller. As a result, 2DPCA has two important advantages over PCA. First, it’s easier to evaluate the covariance matrix accurately. Second, less time is required to determine the corresponding eigenvectors. We extract the features from the 2DPCA matrix using the optimal projection vector axes. The vector’s size is given by the image’s size and the number of coefficients. If the image size is 112 × 92, for example, then the number of coefficients is 112 × d. Researchers have demonstrated experimentally that should be set to no less than 5 to satisfy accuracy.4 This leads us to a numerous coefficients. Figure 2.8 Examples from the faces dataset[12]
- 33. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 23 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 2.9 CONCLUSION A new technique for image feature extraction and representation—two-dimensional principal component analysis (2DPCA)—was developed. 2DPCA has many advantages over conventional PCA (Eigenfaces). In the first place, since 2DPCA is based on the image matrix, it is simpler and more straightforward to use for image feature extraction. Second, 2DPCA is better than PCA in terms of recognition accuracy in all experiments. Although this trend seems to be consistent for different databases and conditions, in some experiments the differences in performance were not statistically significant. Third, 2DPCA is computationally more efficient than PCA and it can improve the speed of image feature extraction significantly. However, it should be pointed out that 2DPCA-based image representation was not as efficient as PCA in terms of storage requirements, since 2DPCA requires more coefficients for image representation than PCA. Why does 2DPCA outperform PCA in face recognition? In our opinion, the underlying reason is that 2DPCA is more suitable for small sample size problems (like face recognition) since its image covariance matrix is quite small. Image representation and recognition based on PCA (or 2DPCA) is statistically dependent on the evaluation of the covariance matrix (although for PCA the explicit construction of the covariance matrix can be avoided). The obvious advantage of 2DPCA A over PCA is that the former evaluates the covariance matrix more accurately. Finally, there are still some aspects of 2DPCA that deserve further study. When a small number of the principal components of PCA are used to represent an image, the mean square error (MSE) between the approximation and the original pattern is minimal. Does 2DPCA have a similar property? In addition, 2DPCA needs more coefficients for image representation than PCA. Although, as a feasible alternative to deal with this problem is to use PCA after 2DPCA for further dimensional reduction, it is still unclear how the dimension of 2DPCA could be reduced directly.
- 34. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 24 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 2.10 HAAR FEATURE ALGORITHMS Detecting human facial features, such as the mouth, eyes, and nose require that Haar classifier cascades first be trained. In order to train the classifiers, this gentle AdaBoost algorithm and Haar feature algorithms must be implemented. Fortunately, Intel developed an open source library devoted to easing the implementation of computer vision related programs called Open Computer Vision Library (OpenCV). The OpenCV library is designed to be used in conjunction with applications that pertain to the field of HCI, robotics, biometrics, image processing, and other areas where visualization is important and includes an implementation of Haar classifier detection and training [8]. To train the classifiers, two set of images are needed. One set contains an image or scene that does not contain the object, in this case a facial feature, which is going to be detected. This set of images is referred to as the negative images. The other set of images, the positive images, contain one or more instances of the object. The location of the objects within the positive images is specified by: image name, the upper left pixel and the height, and width of the object [21]. For training facial features 5,000 negative images with at least a mega-pixel resolution were used for training. These images consisted of everyday objects, like paperclips, and of natural scenery, like photographs of forests and mountains. In order to produce the most robust facial feature detection possible, the original positive set of images needs to be representative of the variance between different people, including, race, gender, and age. A good source foor these images is National Institute of Standards and Technology’s (NIST) Facial Recognition Technology (FERET) database. This database contains over 10,000 images of over 1,000 people under different lighting conditions, poses, and angles. In training each facial feature, 1,500 images were used. These images were taken at angles ranging from zero to forty five degrees from a frontal view. This provides the needed variance required to allow detection if the head is turned slightly [22]. Three separate classifiers were trained, one for the eyes, one for the nose, and one for the mouth. Once the classifiers were trained, they were used to detect the facial features within another set of images from the FERET database.
- 35. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 25 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI The accuracy of the classifier was then computed as shown in Table 1. With the exception of the mouth classifier, the classifiers have a high rate of detection. However, as implied by [23], the false positive rate is also quite high. 2.11 REGIONALIZED DETECTION Since it is not possible to reduce the false positive rate of the classifier without reducing the positive hit rate, a method besides modifying the classifier training attribute is needed to increase accuracy [24]. The method proposed to is to limit the region of the image that is analyzed for the facial features. By reducing the area analyzed, accuracy will increase since less area exists to produce false positives. It also increases efficiency since fewer features need to be computed and the area of the integral images is smaller. In order to regionalize the image, one must first determine the likely area where a facial feature might exist. The simplest method is to perform facial detection on the image first. The area containing the face will also contain facial features. However, the facial feature cascades often detect other facial features as illustrated in Figure 2.8. The best method to eliminate extra feature detection is to further regionalize the area for facial feature detection. It can be assumed that the eyes will be located near the top of the head, the nose will be located in the center area and the mouth will be located near the bottom. The upper 5/8 of the face is analyzed for the eyes. This area eliminates all other facial features while still allowing a wide variance in the tilt angle. The center of the face, an area that is 5/8 by 5/8 of the face, was used to for detection of the nose. This area eliminates all but the upper lip of the mouth and lower eyelid. The lower half of the facial image was used to detect the mouth. Since the facial detector used sometimes eliminates the lower lip the facial image was extended by an eighth for mouth detection only.
- 36. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 26 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 2.12 COMPUTATIONS AND EXAMPLES Principal Component Analysis, or simply PCA, is a statistical procedure concerned with elucidating the covari-ance structure of a set of variables. In particular it allows us to identify the principal directions in which the data varies. For example, in figure 2.12.a, suppose that the triangles represent a two variable data set which we have measured in the X-Y coordinate system. The principal direction in which the data varies is shown by the U axis and the second most important direction is the V axis orthogonal to it. If we place the U V axis system at the mean of the data it gives us a compact representation. If we transform each (X; Y ) coordinate into its corresponding (U; V ) value, the data is de-correlated, meaning that the co-variance between the U and V variables is zero. For a given set of data, principal component analysis finds the axis system defined by the principal directions of variance (ie the U V axis system in figure 2.12.a). The directions U and V are called the principal components. Figure 2.12(a) PCA for Data Representation[3]
- 37. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 27 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Figure 2.12(b) PCA for Dimension Reduction[3] If the variation in a data set is caused by some natural property, or is caused by random experimental error, then we may expect it to be normally distributed. In this case we show the nominal extent of the normal distribution by a hyper-ellipse (the two dimensional ellipse in the example). The hyper ellipse encloses data points that are thought of as belonging to a class. It is drawn at a distance beyond which the probability of a point belonging to the class is low, and can be thought of as a class boundary. If the variation in the data is caused by some other relationship then PCA gives us a way of reducing the dimensionality of a data set. Consider two variables that are nearly related linearly as shown in figure 2.12.b. As in figure 2.12.a the principal direction in which the data varies is shown by the U axis, and the secondary direction by the V axis. However in this case all the V coordinates are all very close to zero. We may assume, for example, that they are only non-zero because of experimental noise. Thus in the U V axis system we can represent the data set by one variable U and discard V. Thus we have reduced the dimensionality of the problem by 1. COMPUTING THE PRINCIPAL COMPONENTS In computational terms the principal components are found by calculating the eigenvectors and eigenvalues of the data covariance matrix. This process is equivalent to finding the axis system in which the co-variance matrix is diagonal. The eigenvector with
- 38. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 28 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI the largest eigenvalue is the direction of greatest variation, the one with the second largest eigenvalue is the (orthogonal) direction with the next highest variation and so on. To see how the computation is done we will give a brief review on eigenvectors/eigenvalues. Let A be an n x n matrix. The eigenvalues of A are defined as the roots of: Where I is the n x n identity matrix. This equation is called the characteristic equation (or characteristic polynomial) and has n roots. Let λ be an eigenvalue of A. Then there exists a vector x such that: The vector x is called an eigenvector of A associated with the eigenvalue λ. Notice that there is no unique solution for x in the above equation. It is a direction vector only and can be scaled to any magnitude. To find a numerical solution for x we need to set one of its elements to an arbitrary value, say 1, which gives us a set of simultaneous equations to solve for the other other elements. If there is no solution we repeat the process with another element. Ordinarily we normalise the final values so that x has length one, that is x xT = 1. Suppose we have a 3 x 3 matrix A with eigenvectors x1, x2, x3, and eigenvalues λ1, λ2, λ3 so. Ax1 = λ1x1 Ax2 = λ2x2 Ax3 = λ3x3 Putting the eigenvectors as the columns of a matrix gives:
- 39. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 29 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Writing: Gives us the matrix equation: We normalized the eigenvectors to unit magnitude, and they are orthogonal, so: Which means that: And: Now let us consider how this applies to the covariance matrix in the PCA process. Let ∑ be an n x n covariance Matrix. There is an orthogonal n x n matrix whose columns are eigenvectors of ∑ and a diagonal matrix Whose diagonal elements are the eigenvalues of ∑, such that
- 40. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 30 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI PCA IN PRACTICE Figure 2.12 (c) The PCA Transformation[3] Figure 2.12.c gives a geometric illustration of the process in two dimensions. Using all the data points we find the mean values of the variables (µx1; µx2) and the covariance matrix ∑ which is a 2 2 matrix in this case. If we calculate the eigenvectors of the co- variance matrix we get the direction vectors indicated by Ø1 and Ø 2. Putting the two eigenvectors as columns in the matrix = [Ø 1; Ø 2] we create a transformation matrix which takes our data points from the [x1; x2] axis system to the axis [Ø 1; Ø 2] system with the equation: p = (px µx) Where px is any point in the [x1; x2] axis system, µx = (µx1; µx2) is the data mean, and p is the coordinate of the point in the [Ø 1; Ø 2] axis system. PCA Based Face Recognition Face recognition is one example where principal component analysis has been extensively used, primarily for reducing the number of variables. Let us consider the 2D
- 41. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 31 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI case where we have an input image, and wish to compare this with a set of data base images to find the best match. We assume that the images are all of the same resolution and are all equivalently framed. (ie the faces appear at the same place and same scale in the images). Each pixel can be considered a variable thus we have a very high dimensional problem which can be simplified by PCA. Formally, in image recognition an input image with n pixels can be treated as a point in an n-dimensional space called the image space. The individual ordinates of this point represent the intensity values of each pixel of the image and form a row vector: px = (i1; i2; i3; :::in) This vector is formed by concatenating each row of image pixels, so for a modest sized image, say 128 by 128 resolution it will dimension 16384. For example: Becomes the vector: [150; 152; 151; 131; 133; 72; 144; 171; 67]16K Clearly this number of variables is far more than is required for the problem. Most of the image pixels will be highly correlated. For example if the background pixels are all equal then adjacent background pixels are exactly correlated. Thus we need to consider how to achieve a reduction in the number of the variables. DIMENSION REDUCTION Let’s consider an application where we have N images each with n pixels. We can write our entire data set as an N n data matrix D. Each row of D represents one image of our data set. For example we may have:
- 42. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 32 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI The first step in PCA is to move the origin to mean of the data. In this application we achieve this by finding a mean image by averaging the columns of D. We then subtract the mean image µ from each image of the data set (ie each row of D) to create the mean centered data vector which we write as U. Let us suppose that the mean centered image is: Then we have that: It is very easy to compute the covariance matrix from the mean centered data matrix. It is just and has dimension n x n. We now calculate the eigenvectors and eigenvalues of ∑ using the standard technique outlined above. That is to say we solve for Ø and V that satisfy: If we normalize the eigenvectors, then the system of vectors Ø forms an orthonormal basis, that is to say:
- 43. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 33 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI It is in effect an axis system in which we can represent our data in a compact form. We can achieve size reduction by choosing to represent our data we fewer dimensions. Normally we choose to use the set of m (m <= n) eigenvectors of ∑ which have the m largest eigenvalues. Typically for face recognition system m will be quite small (around 20-50 in number). We can compose these in an n x m matrix Øpca = [Ø1; Ø2; Ø3 _ _ _ _m] which performs the PCA projection. For any given image px = (i1; i2; i3; :::in) We can find a corresponding point in the PCA space by computing The m-dimension vector pØ is all we need to represent the image. We have achieved a massive reduction in data size since typically n will be at least 16K and m as small as 20. We can store all our data base images in the PCA space and can easily search the data base to find the closest match to a test image. We can also reconstruct any image with the inverse transform: It can be shown that choosing the m eigenvectors of ∑ that have the largest eigenvalues minimizes the mean square reconstruction error over all choices of m orthonormal bases. Clearly we would like to be as small as possible compatible with accurate recognition and reconstruction, but this problem is data dependent. We can make a decision on the basis of the amount of the total variance accounted for by the m principal components that we have chosen. This can be assessed by looking at the eigenvalues.
- 44. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 34 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Let the sum of all the n eigenvalues be written (The ∑ denoting summation in this case, not co-variance.) We can express the percentage of the variance accounted for by the ith eigenvector as: We can then choose m to meet a heuristic condition that will be application dependent. For example, we could ensure that we account for a minimum percentage of the total variance, say 95%, by making 95. Alternatively we could remove all eigenvectors whose eigenvalues account for less than 1% of the total variance. Unfortunately there is no definitive rule for deciding how to choose the number of eigenvectors to retain. FEW SAMPLES AND MANY VARIABLES Typically in face recognition we have a very large number of variables n (16K) but far fewer images N. This means that although the dimension of the covariance matrix is n x n its rank can be at most N x 1, and consequently the number of non-zero eigenvalues will be limited to at most N x 1. The rank may well be less if the data points are not linearly independent. In the extreme case of the data points all being co-linear, there is just one eigenvector with a non-zero eigenvalue, which is in the direction of the line joining the points, and the rank of the covariance matrix will be 1. Since the number of variables n is very large, the calculation of the eigenvectors of ∑ is problematic. We can save considerable amounts of computation time in finding these N - 1 non-zero eigenvalues by employing a trick first published by Kohonen and Lowe. We noted above that the covariance matrix could be written in terms of the mean centred data matrix as: And has dimension n x n. However, the matrix UUT has the much lower dimension of N x N. Suppose we calculate the eigenvectors of UUT. These satisfy the equation:
- 45. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 35 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI If we multiply both sides of this equation by UT we get: Adding brackets to this equation: Makes it clear to see that UTØ` are the eigenvectors of UTU, which is what we wanted to find. Since UUT has dimension N x N and is a considerably smaller matrix than UTU, we have solved the much smaller eigenvector problem, and can calculate the N-1 eigenvectors of UTU with one further matrix multiply. UUT has at most N - 1 eigenvectors, but, as noted above, all other eigenvectors of UTU have zero eigenvalues. The resulting eigenvectors are not orthonormal, and so should be normalized. The method of PCA is sometimes also called the Karhunen-Lowe transform, and occasionally the Householder transform. It is a specific application of the general mathematical technique called single value decomposition in which an n x m matrix is projected to a diagonal form. It is closely related to the technique called single value decomposition which is a more general diagonalization method that applies to matrices that are not square. CORRESPONDENCE IN PCA In 2D face images the pixels are not usually in correspondence. That is to say a given pixel [xi; yi] may be part of the cheek in one image, part of the hair in another and so on. This causes a lot of problems in face recognition and also in reconstructing faces. This means that any linear combination of eigenfaces does not represent a true face but a composition of face parts. A true face is only created when the eigenfaces are added together in exactly the right proportions to re-create one of the original face images of the training set. However, if we represent a face in 3D, then it is possible to establish a correspondence between each point on the surface map. We can organize the data in such a way that each anatomical location (for example the tip of the nose) has the same index in each different
- 46. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 36 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI face data set. Figure shows two different faces whose surface points are in correspondence. Figure 2.12.d 3D surfaces in correspondence [7] PCA can be carried out on 3D surface maps of faces (and other anatomical structures). The variables are the individual co-ordinates of the surface points. A data set is of the form: [x1; y1; z1; x2; y2; z2; x3; xN ; yN ; zN ] The face maps in figure have about 5,000 surface points, hence the number of variables is 15,000. If the points of each subject are in correspondence, and the different subjects are aligned as closely as possible in 3D before calculating the PCA, then any reasonable combination of the eigenvectors will represent a valid face. This means that PCA on 3D surface maps has the potential for constructing and manipulating faces that are different from any example in the training set. This has many of applications in film and media. The set of eigenvectors representing surface data in this manner is called an active shape model
- 47. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 37 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Texture can be mapped onto a 3D surface map to give it a realistic face appearance. The texture values (like pixel values in 2D) can also be included as variables in PCA. Models of this sort are called active appearance models. 2.13 RESULTS The first step in facial feature detection is detecting the face. This requires analyzing the entire image. The second step is using the isolated face(s) to detect each feature. The result is shown in Figure2.12.d. Since each the portion of the image used to detect a feature is much smaller than that of the whole image, detection of all three facial features takes less time on average than detecting the face itself. Using a 1.2GHz AMD processor to analyze a 320 by 240 image, a frame rate of 3 frames per second was achieved. Since a frame rate of 5 frames per second was achieved in facial detection only by [25] using a much faster processor, regionalization provides a tremendous increase in efficiency in facial feature detection. Regionalization also greatly increased the accuracy of the detection. All false positives were eliminated, giving a detection rate of around 95% for the eyes and nose. The mouth detection has a lower rate due to the minimum size required for detection. By changing the height and width parameter to more accurately represent the dimensions of the mouth and retraining the classifier the accuracy should increase the accuracy to that of the other features. 2.14 ARTIFICIAL NEURAL NETWORKS FOR FACE DETECTION In the recent years, different architectures and models of Artificial neural networks (ANN) were used for face detection and recognition. ANN can be used in face detection and recognition because these models can simulate the way neurons work in the human brain. This is the main reason for its role in face recognition. Artificial neural networks (ANN) were used largely in the recent years in the fields of image processing (compression, recognition and encryption) and pattern recognition. Many literature researches used different ANN architecture and models for face detection and recognition
- 48. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 38 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI to achieve better compression performance according to: compression ratio (CR); reconstructed image quality such as Peak Signal to Noise Ratio (PSNR); and mean square error (MSE). Few literature surveys that give overview about researches related to face detection based on ANN. Therefore, this research includes survey of literature studies related to face detection systems and approaches which were based on ANN. RETINAL CONNECTED NEURAL NETWORK (RCNN) Rowley, Baluja and Kanade (1996) [27] presented face detection system based on a retinal connected neural network (RCNN) that examine small windows of an image to decide whether each window contains a face. Figure 3 shows this approach. The system arbitrates between many networks to improve performance over one network. They used a bootstrap algorithm as training progresses for training networks to add false detections into the training set. This eliminates the difficult task of manually selecting non-face training examples, which must be chosen to span the entire space of non-face images. First, a pre-processing step, adapted from [28], is applied to a window of the image. The window is then passed through a neural network, which decides whether the window contains a face. They used three training sets of images. Test SetA collected at CMU: consists of 42 scanned photographs, newspaper pictures, images collected from WWW, and TV pictures (169 frontal views of faces, and require ANN to examine 22,053,124 20×20 pixel windows). Test SetB consists of 23 images containing 155 faces (9,678,084 windows). Test SetC is similar to Test SetA, but contains images with more complex backgrounds and without any faces to measure the false detection rate: contains 65 images, 183 faces, and 51,368,003 window. The detection ratio of this approach equal
- 49. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 39 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 79.6% of faces over two large test sets and small number of false positives. Figure 2.14.a Retinal Connected Neural Network (Rcnn) ROTATION INVARIANT NEURAL NETWORK (RINN) Rowley, Baluja and Kanade (1997) [29] presented a neural network-based face detection system. Unlike similar systems which are limited to detecting upright, frontal faces, this system detects faces at any degree of rotation in the image plane. Figure 4 shows the RINN approach. The system employs multiple networks; the first is a “router” network which processes each input window to determine its orientation and then uses this information to prepare the window for one or more detector networks. We present the training methods for both types of networks. We also perform sensitivity analysis on the networks, and present empirical results on a large test set. Finally, we present preliminary results for detecting faces which are rotated out of the image plane, such as profiles and semi-profiles [29].
- 50. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 40 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Figure 2.14.b Rotation Invariant Neural Network (Rinn) PRINCIPAL COMPONENT ANALYSIS WITH ANN (PCA & ANN) Jeffrey Norris (1999) [30] used using principal component analysis (PCA) with class specific linear projection to detect and recognized faces in a real-time video stream. Figure 5 shows PCA & ANN for face detection. The system sends commands to an automatic sliding door, speech synthesizer, and touchscreen through a multi-client door control server. Matlab, C, and Java were used for developing system. The system steps to search for a face in an image: 1. Select every 20×20 region of input image. 2. Use intensity values of its pixels as 400 inputs to ANN. 3. Feed values forward through ANN, and 4. If the value is above 0.5, the region represents a face. 5. Repeat steps (1..4) several times, each time on a resized version of the original input 6. image to search for faces at different scales [30].
- 51. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 41 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Figure 2.14.b Principal Component Analysis With Ann (Pca & Ann) 2.15 MACHINE LEARNING Two definitions of Machine Learning are offered. Arthur Samuel described it as: "the field of study that gives computers the ability to learn without being explicitly programmed." This is an older, informal definition. Tom Mitchell provides a more modern definition: "A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E." Example: playing checkers. • E = the experience of playing many games of checkers • T = the task of playing checkers. • P = the probability that the program will win the next game. Supervised Learning In supervised learning, we are given a data set and already know what our correct output should look like, having the idea that there is a relationship between the input and the output. Supervised learning problems are categorized into "regression" and "classification" problems. In a regression problem, we are trying to predict results within a continuous output, meaning that we are trying to map input variables to some continuous function. In a classification problem, we are instead trying to predict results in a discrete output. In other words, we are trying to map input variables into discrete categories.
- 52. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 42 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Example 1: Given data about the size of houses on the real estate market, try to predict their price. Price as a function of size is a continuous output, so this is a regression problem. We could turn this example into a classification problem by instead making our output about whether the house "sells for more or less than the asking price." Here we are classifying the houses based on price into two discrete categories. Example 2: (a)Regression - Given a picture of Male/Female, We have to predict his/her age on the basis of given picture. (b)Classification - Given a picture of Male/Female, We have to predict Whether He/She is of Highschool,College, Graduate age. Another Example for Classification - Banks have to decide whether or not to give a loan to someone on the basis of his credit history. Unsupervised Learning Unsupervised learning, on the other hand, allows us to approach problems with little or no idea what our results should look like. We can derive structure from data where we don't necessarily know the effect of the variables. We can derive this structure by clustering the data based on relationships among the variables in the data. With unsupervised learning there is no feedback based on the prediction results, i.e., there is no teacher to correct you. It’s not just about clustering. For example, associative memory is unsupervised learning. Example: Clustering: Take a collection of 1000 essays written on the US Economy, and find a way to automatically group these essays into a small number that are somehow similar or
- 53. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 43 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI related by different variables, such as word frequency, sentence length, page count, and so on. Another, non-Clustering example, is the "Cocktail Party Algorithm" which can find structure in messy data- like identifying individual voices and music from a mesh of sounds at a cocktail party. MACHINE LEARNING TECHNIQUES FOR FACE ANALYSIS In recent years there has been a growing interest in improving all aspects of the interaction between humans and computers with the clear goal of achieving a natural interaction, similar to the way human-human interaction takes place. The most expressive way humans display emotions is through facial expressions. Humans detect and interpret faces and facial expressions in a scene with little or no effort. Still, development of an automated system that accomplishes this task is rather difficult. There are several related problems: detection of an image segment as a face, extraction of the facial expression information, and classification of the expression (e.g.,in emotion categories).A system that performs these operations accurately and in real time would be a major step forward in achieving a human-like interaction between the man and machine. In this chapter, we present several machine learning algorithms applied to face analysis and stress the importance of learning the structure of Bayesian network classifiers when they are applied to face and facial expression analysis. Information systems are ubiquitous in all human endeavors including scientific, medical, military, transportation, and consumer. Individual users use them for learning, searching for information (including data mining), doing research (including visual computing), and authoring. Multiple users (groups of users, and groups of groups of users) use them for communication and collaboration. And either single or multiple users use them for entertainment. An information system consists of two components: Computer (data/knowledge base, and information processing engine), and humans. It is the intelligent interaction between the two that we are addressing in this chapter. Automatic face analysis has attracted increasing interest in the research community mainly due to its many useful applications. A system involving such an analysis assumes
- 54. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 44 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI that the face can be accurately detected and tracked, the facial features can be precisely identified, and that the facial expressions, if any, can be precisely classified and interpreted. For doing this, in the following, we present in detail the three essential components of our automatic system for human-computer interaction: face detection, facial feature detection, and facial emotion recognition. FACE DETECTION Images containing face are essential to intelligent vision-based human-computer interaction. The rapidly expanding research in face processing is based on the premise that information about user’s identity, state, and intend can be extracted from images and that computers can react accordingly, e.g., by observing a person’s facial expression. Given an arbitrary image, the goal of face detection is to automatically locate a human face in an image or video, if it is present. Face detection in a general setting is a challenging problem for various reasons. The first set of reasons are inherent: there are many types of faces, with different colors, texture, sizes, etc. In addition, the face is a non-rigid object which can change its appearance. The second set of reasons are environmental: changing lighting, rotations, translations, and scales of the faces in natural images. To solve the problem of face detection, two main approaches can be taken. The first is a model based approach, where a description of what is a human face is used for detection. The second is an appearance based approach, where we learn what faces are directly from their appearance in images. FACIAL FEATURE DETECTION Various approaches to facial feature detection exist in the literature. Although many of the methods have been shown to achieve good results, they mainly focus on finding the location of some facial features (e.g., eyes and mouth corners) in restricted environments (e.g., constant lighting, simple background, etc.). Since we want to obtain a complex and accurate system of feature annotation, these methods are not suitable for us. In recent years’ deformable model-based approaches for image interpretation have been proven very successful, especially in images containing objects with large variability such as
- 55. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 45 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI faces. These approaches are more appropriate for our specific case since they make use of a template (e.g., the shape of an object). Among the early deformable template models is the Active Contour Model by Kass et al. [47] in which a correlation structure between shape markers is used to constrain local changes. Cootes et al. [48] proposed a generalized extension, namely Active Shape Models (ASM), where deformation variability is learned using a training set. Active Appearance Models (AAM) were later proposed in [49] and they are closely related to the simultaneous formulation of Active Blobs [50] and Morphable Models [51]. AAM can be seen as an extension of ASM which includes the appearance information of an object. EMOTION RECOGNITION RESEARCH Ekman and Friesen [52] developed the Facial Action Coding System (FACS) to code facial expressions where movements on the face are described by a set of action units (AUs). Each AU has some related muscular basis. This system of coding facial expressions is done manually by following a set of prescribed rules. The inputs are still images of facial expressions, often at the peak of the expression. This process is very time-consuming. Ekman’s work inspired many researchers to analyze facial expressions by means of image and video processing. By tracking facial features and measuring the amount of facial movement, they attempt to categorize different facial expressions. Recent work on facial expression analysis and recognition has used these “basic expressions” or a subset of them. The two recent surveys in the area [53, 54] provide an in depth review of many of the research done in automatic facial expression recognition in recent years.
- 56. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 46 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 2.16 FUTURE PLANS With the successful detection of facial features, the next goal is to research the ability for more precise details, like individual points, of the facial features to be gathered. These points will be use to differentiate general human emotions, like happiness and sadness. Recognition of human emotion would require detection and analysis of the various elements of a human face, like the brow and the mouth, to determine an individual’s current expression. The expression can then be compared to what is considered to be the basic signs of an emotion. This research will be used in the field human-computer interaction to analyze the emotions one exhibits while interacting with a user interface.
- 57. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 47 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 3. OBJECTIVES AND METHODOLOGY 3.1 FACE RECOGNITION Principal components analysis (PCA) is used for two objectives: 1. Reducing the number of variables comprising a dataset while retaining the variability in the data. 2. Identifying hidden patterns in the data, and classifying them according to how much of the information, stored in the data, they account for. When mining a dataset comprised of numerous variables, (used interchangeably with the term dimensions hereinafter), it is likely that subsets of variables are highly correlated with each other. Given a high correlation between two or more variables it can be concluded that these variables are quite redundant thus share the same driving principle in defining the outcome of interest. In order to demonstrate this argument let us consider a basic example. Suppose we have measured 2 parametric properties (i.e. properties represented by numerical quantities) of a planar shape, which are the length and the width of the shape, that determine a certain outcome of interest. From examining our observations we have noticed that these two properties seem to be positively correlated. Hence, we can replace them with a single new variable which is the area of the shape that still captures most of the information about the shape supplied by its length and width. In multivariate datasets, dimension reduction by PCA enables us to analyze our data in a visible 2-dimensional (2D) or 3D space, with only a mere loss of information. The computational approach for defining correlation between a pair of variables measures to what extent their values co-vary. The term covariance is undoubtedly associatively prompted immediately; not in vain, as covariance and correlation is nearly the same thing. The term in equation 1 is used to compute the covariance between a pair of variables ( X1 ,X2 ) K K . (Actually, equation 1 computes the estimator for the covariance, as the values of the X variables are only a sample of the entire population). The principal component analysis for the example above took a large set of data and identified an optimal new basis in which to re-express the data. This mirrors the general
- 58. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 48 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI aim of the PCA method: can we obtain another basis that is a linear combination of the original basis and that re-expresses the data optimally? There are some ambiguous terms in this statement, which we shall address shortly, however for now let us frame the problem in the following way. Assume that we start with a data set that is represented in terms of an m × n matrix, X where the n columns are the samples (e.g. observations) and the m rows are the variables. We wish to linearly transform this matrix, X into another matrix, Y, also of dimension m × n, so that for some m × m matrix, P, Y = PX (1) This equation represents a change of basis. If we consider the rows of P to be the row vectors p1, p2, . . . , pm, and the columns of X t to be the column vectors x1, x2, . . . , xn, then (3) can be interpreted in the following way. PX = Px1 Px2 . . . Pxn = p1.x1 p1.x2 · · · p1.xn p2.x1 p2.x2 · · · p2.xn . . . . =Y . . . . . . . . pm.x1 pm.x2 …… pm.xn Note that pi , xj ∈ R m, and so pi .xj is just the standard Euclidean inner (dot) product. This tells us that the original data, X is being projected on to the columns of P. Thus, the rows of P, {p1, p2, . . . , pm} are a new basis for representing the columns of X. The rows of P will later become our principal component directions.
- 59. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 49 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 3.2 FACE INDEXING Faces from each photographs will be index on basis of historic data with proper user’s input. Indexed faces will be used as training faces to recognize faces in subsequent photographs. User System Physical Storage Logical Storage Set Listener Copy Photo Notify Image Added Detect Faces Notify user to Name face Map faces Index Image with face name Set Listener Figure 3.2 (a) Face Indexing Each captured faces will be added training database for future reference. An XML will be maintained which records imagesphotographs with facial name. There is provision to add file listener to the directory which triggers system when any photographs will be copied. Identified faces will be first recognized from system using face recognition mechanism and will be later presented to the user for final selection. User intervention is required to minimize errors and indexing unidentified faces.
- 60. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 50 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Figure 3.2.b Face Detection and Training sets
- 61. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 51 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Figure 3.2.c Face Detection and Facial expression classification[12] Once User approves or accept the faces with proper name, system records changes. This changes will be recorded into XML file. XML will maintain fully qualified file name (name with location) and the face value. Typical structure of XML is like below <?xml version="1.0" encoding="utf-8"?> <Photo> <TagedPhoto> <Photo FileName="photo - 1.jpg">
- 62. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 52 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI <FaceName>Face 1</FaceName> <FaceName>Face 2</FaceName> <FaceName>Face 3</FaceName> <FaceName>Face 4</FaceName> </Photo> <Photo FileName="photo - 2.jpg"> <FaceName>Face 1</FaceName> <FaceName>Face 3</FaceName> <FaceName>Face 5</FaceName> </Photo> </TagedPhoto> <UnTagedPhoto> </UnTagedPhoto> </Photo>
- 63. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 53 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 3.3 FACE SEARCH Indexed faces can be easily search without applying any standard face recognition algorithm. This could be possible as system had already indexed images in XML. User can select the one or more faces and request system to search related photographs. System applies complex XPATH logic to filter out related photographs on basis of faces selected by users. User System Logical Storage Physical Storage Search Request Search UI Show Search Faces Image details Load Image Notify User Figure 3.3 Face Search Typical XPATH could be FaceName=”Face1” and FaceName=”Face2” This query will be processed as XPATH and related selected node will be presented to the user.
- 64. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 54 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 3.4 SIMULATION INTRODUCTION Simulation can be defined to show the eventual real behavior of the selected system model .It is used for performance optimization on the basis of creating a model of the system in order to gain insight into their functioning. We can predict the estimation and assumption of the real system by using simulation results. SIMULATOR Photograph Database Management Using Face Recognition Algorithm is proposed to overcome management of Photographs with faces without duplicating. It provides search mechanism to query photographs from facial name. This is turning point as search will not try any standard face recognition mechanism but it will look into index prepared during copying of image. This makes system quite fast. EGMU CV FACE RECOGNIZER AND EIGEN CLASSIFIER FaceRecognizer is a global constructor that allows Eigen, Fisher, and LBPH classifiers to be used together. The class combines common method calls between the classifiers. The constructor for each classifier type is as follows: FaceRecognizer recognizer = new EigenFaceRecognizer(num_components, threshold); THE EIGEN CLASSIFIER The Eigen recognizer takes two variables. The 1st, is the number of components kept for this Principal Component Analysis. There’s no rule how many components that should be kept for good reconstruction capabilities. It is based on input data, so experiment with the number. OpenCV documentation suggests keeping 80 components should almost always be sufficient. The 2nd variable is designed to be a prediction threshold[7]. FaceRecognizer recognizer = new EigenFaceRecognizer(80, double.PositiveInfinity); PRINCIPLE COMPONENT ANALYSIS (PCA)
- 65. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 55 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI The EigenFaceRecognizer applies PCA. The FisherFaceRecognizer applies Linear Discriminant Analysis derived by R.A. The EigenFaceRecognizer class applies PCA on each image, the results of which will be an array of Eigen values that a Neural Network can be trained to recognize[7]. PCA is a commonly used method of object recognition as its results, when used properly can be fairly accurate and resilient to noise. The method of which PCA is applied can vary at different stages so what will be demonstrated is a clear method for PCA application that can be followed. It is up for individuals to experiment in finding the best method for producing accurate results from PCA. To perform PCA several steps are undertaken: Stage 1: Subtract the Mean of the data from each variable (our adjusted data) Stage 2: Calculate and form a covariance Matrix Stage 3: Calculate Eigenvectors and Eigenvalues from the covariance Matrix Stage 4: Chose a Feature Vector (a fancy name for a matrix of vectors) Stage 5: Multiply the transposed Feature Vectors by the transposed adjusted data STAGE 1: Mean Subtraction This data is fairly simple and makes the calculation of our covariance matrix a little simpler now this is not the subtraction of the overall mean from each of our values as for covariance we need at least two dimensions of data. It is in fact the subtraction of the mean of each row from each element in that row. (Alternatively the mean of each column from each element in the column however this would adjust the way we calculate the covariance matrix)
- 66. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 56 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Stage 2: Covariance Matrix The basic Covariance equation for two dimensional data is: Which is similar to the formula for variance however, the change of x is in respect to the change in y rather than solely the change of x in respect to x. In this equation x represents the pixel value and ̄x is the mean of all x values, and n the total number of values. The covariance matrix that is formed of the image data represents how much the dimensions vary from the mean with respect to each other. The definition of a covariance matrix is: Now the easiest way to explain this is but an example the easiest of which is a 3x3 matrix.
- 67. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 57 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Now with larger matrices this can become more complicated and the use of computational algorithms essential. Stage 3: Eigenvectors and Eigenvalues Eigenvalues are a product of multiplying matrices however they are as special case. Eigenvalues are found by multiples of the covariance matrix by a vector in two dimensional space (i.e. a Eigenvector). This makes the covariance matrix the equivalent of a transformation matrix. It is easier to show in a example: Eigenvectors can be scaled so ½ or x2 of the vector will still produce the same type of results. A vector is a direction and all you will be doing is changing the scale not the direction. Eigenvectors are usually scalled to have a length of 1:
- 68. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 58 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Thankfully finding these special Eigenvectors is done for you and will not be explained however there are several tutorials available on the web to explain the computation. The Eigenvalue is closely related to the Eigenvector used and is the value of which the original vector was scaled in the example the Eigenvalue is 4. Stage 4: Feature Vectors Now a usually the results of Eigenvalues and Eigenvectors are not as clean as in the example above. In most cases the results provided are scaled to a length of 1. So here are some example values calculated using Matlab: Once Eigenvectors are found from the covariance matrix, the next step is to order them by Eigenvalue, highest to lowest. This gives you the components in order of significance. Here the data can be compressed and the weaker vectors are removed producing a lossy compression method, the data lost is deemed to be insignificant. Stage 5: Transposition
- 69. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 59 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI The final stage in PCA is to take the transpose of the feature vector matrix and multiply it on the left of the transposed adjusted data set (the adjusted data set is from Stage 1 where the mean was subtracted from the data). The EigenFaceRecognizer class performs all of this and then feeds the transposed data as a training set into a Neural Network. When it is passed an image to recognize it performs PCA and compares the generated Eigenvalues and Eigenvectors to the ones from the training set the Neural Network then produces a match if one has been found or a negative match if no match is found. The is a little more to it than this however the use of Neural Networks is a complex subject to cover and is not the object of this article. USER INTERFACES This project is developed on .NET framework and is windows based (Desktop) application. C# is used as programming language. Microsoft Visual Studio .NET is used to design interfaces and handle build. System stores photographs in disk itself and add listener to track any changes. The technologies used to develop this application are:- C#.NET: C# is an elegant and type-safe object-oriented language that enables developers to build a variety of secure and robust applications that run on the .NET Framework. You can use C# to create Windows client applications, XML Web services, distributed components, client-server applications, database applications, and much, much more. Visual C# provides an advanced code editor, convenient user interface designers, integrated debugger, and many other tools to make it easier to develop applications based on the C# language and the .NET Framework. Backend: There is no standard database management tools and software is used. However XML is used to index photographs with faces and system build query as XPATH and perform search. XML stands for eXtensible Markup Language. It is a markup language much like HTML, but there are several differences between them: • HTML includes a collection of predefined tags that you can use right away in editing your HTML files, e.g. <font>and <h1>, but XML tags are not predefined. To use XML,
- 70. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 60 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI users have to define their own tags for their specific application before using them. For example, to describe a note <note>, <to>, <from> ,<heading> and <body> <note> < to >Tove</to> <from> Jani</from> <heading>Reminder</heading> < body> Don’t forget the party! </body> </note> With XML, one can define whatever tags needed, which together compose a user-defined markup language similar to HTML, and then use the language to describe data. Specifically XML uses Document Type Definition (DTD) or an XML Schema to define tags. In this sense, XML is viewed as a meta-language since it can be used to define and describe A markup language instead of concrete data directly. That is also why it is called extensible. • XML was designed to describe data while HTML was designed for displaying data. If you remember, HTML tags control the way data is presented to browser users, color, font size, spacing, etc. Differently XML aims to deal with the logic meaning of data, or semantics. In the above example, the text wrapped in <from> and </from> is the name Of the sender of the note. This enables the fulfillment of the task of finding all the notes written by a specific person. So XML was designed to describe data and to focus on What data is while HTML was designed to display data and to focus on how data looks? Actually XML and HTML can complement each other. For example, we use XML files to store data on a web server machine and when a request arrives, a servlet runs to retrieve data in XML, compose a HTML file, and finally output it to the client. This way you can concentrate on using HTML for data layout and display, and be sure that changes in the underlying data will not require any changes to your HTML.Besides XML’s role of storing data, with XML, data can be exchanged between incompatible systems. In the real world, computer systems and databases contain data in incompatible formats. One of the most time-consuming challenges for developers has been to exchange data between such
- 71. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 61 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI systems over the Internet. Converting the data to XML can greatly reduce this complexity and create data that can be read by many different types of applications, since XML data is stored in plain text format, which is software- and hardware-independent. The best description of XML may be this: XML is a cross-platform, software and hardware Independent tool for transmitting information. Since the creation of XML, it has been amazing to see how quickly the XML standard has been developed and how quickly a large number of software vendors have adopted the standard. It is strongly believed by the IT community at large that XML will be as important to the future of the Web as HTML has been to the foundation of the Web and that XML will be the most common tool for all data manipulation and data transmission. Figure 3.4 XML XML is a meta markup language for text documents / textual data XML allows to define languages (“applications“) to represent text documents / textual data
- 72. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 62 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Advantages of Using XML Truly Portable Data Easily readable by human users Very expressive (semantics near data) Very flexible and customizable (no finite tag set) Easy to use from programs (libs available) Easy to convert into other representations (XML transformation languages) Many additional standards and tools Widely used and supported OPEN CV: OpenCV (Open Source Computer Vision) is a library of programming functions mainly aimed at real-time computer vision, developed by Intel Russia research center in Nizhny Novgorod, and now supported by Willow Garage and Itseez. It is free for use under the open-source BSD license. The library is cross-platform. It focuses mainly on real-time image processing. If the library finds Intel's Integrated Performance Primitives on the system, it will use these proprietary optimized routines to accelerate itself. EGMU CV: Emgu CV is a cross platform .Net wrapper to the OpenCV image processing library. Allowing OpenCV functions to be called from .NET compatible languages such as C#, VB, VC++, Iron Python etc. The wrapper can be compiled by Visual Studio, Xamarin Studio and Unity, it can run on Windows, Linux, Mac OS X, iOS, Android and Windows Phone.
- 73. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 63 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI SOFTWARE INTERFACE Any windows based operating system. C# for developing code. .NET Framework 4.5.1 for runtime environment. Microsoft .NET FRAMEWORK The Microsoft .NET Framework is a software component that is a part of Microsoft Windows operating systems. It has a large library of pre-coded solutions to common programming problems and manages the execution of programs written specifically for the framework. The .NET Framework is a key Microsoft offering and is intended to be used by most new applications created for the windows platform. The pre-coded solutions that form the framework's Base Class Library cover a large range of programming needs in a number of areas, including user interface, data access, database connectivity, cryptography, web application development, numeric algorithms, and network communications. The class library is used by programmers, who combine it with their own code to produce applications. Programs written for the .NET Framework execute in a software environment that manages the program's runtime requirements. Also part of the .NET Framework, this runtime environment is known as the Common Language Runtime (CLR). The CLR provides the appearance of an application virtual machine so that programmers need not consider the capabilities of the specific CPU that will execute the program. The CLR also provides other important services such as security, memory management, and exception handling. The class library and the CLR together compose the .NET Framework. The .NET Framework is included with Windows Server 2008 and Windows Vista. The current version of the framework can also be installed on Windows XP and the Windows Server 2003 family of operating systems.
- 74. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 64 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Principle Design Features Interoperability The interaction between new and older applications is commonly required, the .NET Framework provides means to access functionality that is implemented in programs that execute outside the .NET environment. Access to COM components is provided in the system runtime. Interop Services and System Enterprise Services namespaces of the framework and access to other functionality is provided using the P/Invoke feature. Common Runtime Engine Programming languages on the .NET Framework compile into an intermediate language known as the Common Intermediate Language or CIL (formerly known as Microsoft Intermediate Language, or MSIL). In Microsoft’s implementation this intermediate language is not interpreted but rather compiled in a manner known as just-in-time compilation (JIT) into native code. The combination of these concepts is called the Common Language Infrastructure (CLI). Microsoft’s implementation of the CLI is known as Common Language Runtime (CLR). Language Independence The .NET Framework introduces a Common Type System, or CTS. The CTS specification defines all possible Data type and programming constructs supported by the CLR and how they may or may not interact with each other. Because of this feature, the .NET Framework supports the exchange of instances of types between programs written in any of the .NET languages. Base Class Library The Base Class Library (BCL), part of the Framework Class Library (FCL), is a library of functionality available to all languages using the .NET Framework. The BCL provides classes which encapsulate a number of common functions, including file reading and writing, graphic rendering, database interaction and XML document manipulation. Simplified Deployment Installation of computer software must be carefully managed to ensure that is does not interfere with previously installed software, and that it conforms to increasingly stringent
- 75. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 65 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI security requirements. The .NET framework includes design features and tools that help address these requirements. Security The design is meant to address some of the vulnerabilities, such as buffer overflows, that have been exploited by malicious software. Additionally, .NET provides a common security model for all applications. Portability The design of the .NET Framework allows for it to be platform agnostic, and thus be cross platform compatible. That is, a program written to use the framework should run without change on any type of system for which the framework is implemented. MEMORY CONSTRAINTS At least 512 MB RAM and 5 MB space on hard disk will be required for running the application. Ideally disk space should me more enough to accommodate and allowing copying photographs. OPERATIONS This system includes the following modules: System Settings and Configurations System should be configured to know where images should be copied and where to copy Training faces. Face Indexer System watches any changes in designated directory (where photographs will be copied) and on basis of changes, system detect and recognize faces. This faces will be presented to the user and later saved in XML database. Face Search System provides interface to user to build query and search into photographs database. THESIS FUNCTIONS This thesis can be accessed by anyone who logged in to the system. The major functions of this project is listed below Configure system for firs time usage.
- 76. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 66 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Add listener to the Photograph directory to track any changes. Listen changes and apply face detection algorithm to identify faces in photographs. Try to identify or recognize faces and prompt user to finally accept. Index faces and saves into xml file. Provide means (UI) to search faces from photographs. CONSTRAINTS Listeners should be running prior to any changes in Photograph directory. User should manually override unrecognized of false faces. The future version of this project will have more sophisticated way of handling query to photograph and should able to save the query for future use. EXTERNAL INTERFACES The requirements work product produced during system or application development that formally specifies the interfaces to all external systems and applications.
- 77. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 67 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 3.5 METHODOLOGY Scrum software development methodology is used to develop simulator. Scrum is an iterative and incremental agile software development methodology for managing product development. Agile development provides opportunities to assess the direction throughout the development lifecycle. This is achieved through regular cadences of work, known as Sprints or iterations, at the end of which teams must present a potentially shippable product increment. By focusing on the repetition of abbreviated work cycles as well as the functional product they yield, agile methodology is described as “iterative” and “incremental.” In waterfall, development teams only have one chance to get each aspect of a project right. In an agile paradigm, every aspect of development requirements, design, etc. is continually revisited. When a team stops and re-evaluates the direction of a project every two weeks, there’s time to steer it in another direction. This “inspect-and-adapt” approach to development reduces development costs and time to market. Because teams can develop software at the same time they’re gathering requirements, “analysis paralysis” is less likely to impede a team from making progress. And because a team’s work cycle is limited to two weeks, stakeholders have recurring opportunities to calibrate releases for success in the real world. Agile development helps companies build the right product. Instead of committing to market a piece of software that hasn’t been written yet, agile empowers teams to continuously re-plan their release to optimize its value throughout development, allowing them to be as competitive as possible in the marketplace. Agile development preserves a product’s critical market relevance and ensures a team’s work wind up on a shelf, never released. Scrum is a lightweight process framework for agile development, and the most widely-used one. A “process framework” is a particular set of practices that must be followed in order for a process to be consistent with the framework. (For example, the Scrum process framework requires the use of development cycles called Sprints, the XP framework requires pair programming, and so forth.) “Lightweight” means that the overhead of the process is kept as small as possible, to maximize the amount of productive time available for getting useful work done. A Scrum process is distinguished from other agile processes by specific concepts and practices, divided into the three categories of Roles, Artifacts, and Time Boxes. These
- 78. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 68 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI and other terms used in Scrum are defined below. Scrum is most often used to manage complex software and product development, using iterative and incremental practices. Scrum significantly increases productivity and reduces time to benefits relative to classic “waterfall” processes. Scrum processes enable organizations to adjust smoothly to rapidly-changing requirements, and produce a product that meets evolving business goals. An agile Scrum process benefits the organization by helping it to Increase the quality of the deliverables Cope better with change (and expect the changes) Provide better estimates while spending less time creating them Be more in control of the project schedule and state As a result, Scrum projects achieve higher customer satisfaction rates. Figure 3.5.a. SCRUM Process Agile development refers to any development process that is aligned with the concepts of the Agile Manifesto. The Manifesto was developed by a group fourteen leading figures in
- 79. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 69 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI the software industry, and reflects their experience of what approaches do and do not work for software development. A DEEPER LOOK INTO SCRUM The benefits are different for different people. Benefits to Customer Customers find that the vendor is more responsive to development requests. High-value features are developed and delivered more quickly with short cycles, than with the longer cycles favored by classic “waterfall” processes. Benefits of Scrum Benefits to Vendors Vendors reduce wastage by focusing development effort on high-value features, and reduce time-to-market relative to waterfall processes due to decreased overhead and increased efficiency. Improved customer satisfaction translates to better customer retention and more positive customer references. Benefits to Development Teams Team members enjoy development work, and like to see their work used and valued. Scrum benefits Team members by reducing non-productive work (e.g., writing specifications or other artifacts that no one uses), and giving them more time to do the work they enjoy. Team members also know their work is valued, because requirements are chosen to maximize value to customers. Benefits to Product Managers Product Managers, who typically fill the Product Owner role, are responsible for making customers happy by ensuring that development work is aligned with customer needs. Scrum makes this alignment easier by providing frequent opportunities to re-prioritize work, to ensure maximum delivery of value. Benefits to Project Managers Project Managers (and others) who fill the ScrumMaster role find that planning and tracking are easier and more concrete, compared to waterfall processes. The focus on task-level tracking, the use of Burndown Charts to display daily progress, and the Daily Scrum meetings, all together give the Project Manager tremendous awareness about the state of the project at all times. This awareness is key to monitoring the project, and to catching and addressing issues quickly.
- 80. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 70 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Benefits to PMOs and C-Level Executives Scrum provides high visibility into the state of a development project, on a daily basis. External stakeholders, such as C-Level executives and personnel in the Project Management Office, can use this visibility to plan more affectively, and adjust their strategies based on more hard information and less speculation. Scrum differs from traditional “waterfall” approaches to project management in many ways, but is based on sound project-management principles. REQUIREMENTS Scrum does not define just what form requirements are to take, but simply says that they are gathered into the Product Backlog, and referred to generically as “Product Backlog Items,” or “PBIs” for short. Given the time-boxed nature of a Sprint, we can also infer that each set should require significantly less time to implement than the duration of the Sprint. Most Scrum projects borrow the “XP” (Extreme Programming) practice of describing a feature request as a “User Story,” although a minority uses the older concept of a “Use Case.” ROLES The three roles defined in Scrum are the Scrum Master, the Product Owner, and the Team (which consists of Team members). The people who fulfill these roles work together closely, on a daily basis, to ensure the smooth flow of information and the quick resolution of issues. SCRUM MASTER The ScrumMaster (sometimes written “Scrum Master,” although the official term has no space after “Scrum”) is the keeper of the process. He is responsible for making the process run smoothly, for removing obstacles that impact productivity, and for organizing and facilitating the critical meetings.
- 81. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 71 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Figure 3.5.b. SCRUM Roles The ScrumMasters responsibilities include Removing the barriers between the development Team and the Product Owner so that the Product Owner directly drives development. Teach the Product Owner how to maximize return on investment (ROI), and meet his/her objectives through Scrum. Improve the lives of the development Team by facilitating creativity and empowerment. Improve the productivity of the development Team in any way possible. Improve the engineering practices and tools so that each increment of functionality is potentially shippable. Keep information about the Team’s progress up to date and visible to all parties. In practical terms, the Scrum Master needs to understand Scrum well enough to train and mentor the other roles, and educate and assist other stakeholders who are involved in the process.It should maintain a constant awareness of the status of the project (its progress
- 82. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 72 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI to date) relative to the expected progress, investigate and facilitate resolution of any roadblocks that hold back progress, and generally be flexible enough to identify and deal with any issues that arise, in any way that is required. He must protect the Team from disturbance from other people by acting as the interface between the two. The ScrumMaster does not assign tasks to Team members, as task assignment is a Team responsibility. His general approach towards the Team is to encourage and facilitate their decision-making and problem-solving capabilities, so that they can work with increasing efficiency and decreasing need for supervision. His goal is to have a team that is not only empowered to make important decisions, but does so well and routinely. USER INTERFACE System Configuration Table 3.5 a.System Configuration Field Name Read Only Remarks Photograph Directory NO Directory where photographs are intended to copy. Training Face Directory YES Directory where training faces from photographs copied by system. INDEXER Indexer will populate all recognized faces and input and dropdown boxes. Table 3.5.b. Index Field Name Remarks Face Image Detected face Face name Input field to get name of face Face name list List of faces already identified and present in database Search Table.3.5.c. Search Field Name Remarks List of face name Detected face which user will select to search in database.
- 83. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 73 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 4. ANALYSIS AND INTERFACES 4.1 SIMULATION RESULTS AND ANALYSIS As already discussed, proposed system uses listener patternmechanism to track changes in photograph directory. System need to know the directory where to attach listener. During first-time access, system setting is mandatory to perform. User has to start listener to activate listener service on the photograph directory. Once listener starts, any changes in photograph directory will be tracked by the system. If user performs changes like copying new photograph to the directory, system automatically try to detect and recognize faces. To make final decision on the naming of faces system required user’s acceptance. For first-time as there is no training images, user has to input the name of faces. Once user accepts the changes, system indexes the photograph with face name. User can search photographs anytime basis of facial name. User can select any many face name and system will look for all photographs matching to the criteria.
- 84. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 74 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 4.2 SNAPSHOT Figure 4.3.a System Configuration System Configuration is very first step. User configures photograph directory where photographs will be copied. System internally configures listener on photograph directory. Training Face Directory maintains training faces and will be hidden from the user. System internally keeps identified faces upto ten. Photograph directory should be empty.
- 85. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 75 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Figure 4.3.b System Configuration with values Photograph directory should be empty. System creates hidden Training Face Directory. System also creates xml file to track all changes and record the indexes of the face and photograph.
- 86. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 76 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Figure 4.3.c Post System Configuration Once Configuration is verified and saved, system locks the configuration. Configuration could not be modified. User can start Listener on configured photograph directory. Listener basically track changes in photograph directory. If any changes like copy, delete or rename happens notification comes to system for further processing.
- 87. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 77 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Figure 4.3.d Screen to provide face name As soon as user copies photograph into configureddesignated directory, system automatically detect the faces and notify user to name the faces. Named faces will be saved as training faces. System internally add index to the face and photograph. Any Unknown/Not face selection will be discarded.
- 88. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 78 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Figure 4.3.e Photograph search User can search photograph(s) on basis of face name which was indexed during copying process. List will be populated automatically. If some photographs has been copied later, Update list button will fetch latest face names. Show button pulls photographs related to the face names selected from the list.
- 89. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 79 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI Figure 4.3.f Photograph search result On basis of search criteria, system fetches photographs and render as thumbnail. However user can click Open button to load photograph in explorer.
- 90. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 80 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 5. CONCLUSION It gives a contribution for the development of new human-machine interfaces for mobile robots and autonomous systems, based on computer vision techniques. An approach for real-time face recognition and tracking which can be very useful for human-robot interaction systems. In a human robot interaction environment this system starts with a very fast real-time learning process and then allows the robot to follow the person and to be sure it is always interacting with the right one under a wide range of conditions including: illumination, scale, pose, and camera variation. The face tracking system works as a preprocessing stage to the face recognition system, which allows it to concentrate the face recognition task in a sub-window previously classified as face. This abruptly reduces the computation time. The introduction of a position predictive stage would also reduce the face search area driving to the creation of a robust automatic tracking and real-time recognition system. This system can also be used in security applications since it has the ability of tracking a set of known people.
- 91. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 81 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 6. FUTURE ENHANCEMENT A new technique for image feature extraction and representation—two-dimensional principal component analysis (2DPCA)—was developed. 2DPCA has many advantages over conventional PCA (Eigenfaces). In the first place, since 2DPCA is based on the image matrix, it is simpler and more straightforward to use for image feature extraction. Second, 2DPCA is better than PCA in terms of recognition accuracy in all experiments. Although this trend seems to be consistent for different databases and conditions, in some experiments the differences in performance were not statistically significant. Third, 2DPCA is computationally more efficient than PCA and it can improve the speed of image feature extraction significantly. However, it should be pointed out that 2DPCA-based image representation was not as efficient as PCA in terms of storage requirements, since 2DPCA requires more coefficients for image representation than PCA. Why does 2DPCA outperform PCA in face recognition? In our opinion, the underlying reason is that 2DPCA is more suitable for small sample size problems (like face recognition) since its image covariance matrix is quite small. Image representation and recognition based on PCA (or 2DPCA) is statistically dependent on the evaluation of the covariance matrix (although for PCA the explicit construction of the covariance matrix can be avoided)[32]. The obvious advantage of 2DPCA A over PCA is that the former evaluates the covariance matrix more accurately. Finally, there are still some aspects of 2DPCA that deserve further study. When a small number of the principal components of PCA are used to represent an image, the mean square error (MSE) between the approximation and the original pattern is minimal. In addition, 2DPCA needs more coefficients for image representation than PCA. Although, as a feasible alternative to deal with this problem is to use PCA after 2DPCA for further dimensional reduction, it is still unclear how the dimension of 2DPCA could be reduced directly.
- 92. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 82 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI 7. REFERENCES [1] Shilpi Soni, Raj Kumar Sahu,” Face Recognition Based on Two Dimensional Principal Component Analysis (2DPCA) and Result Comparison with Different Classifiers”, International Journal of Advanced Research in Computer and Communication Engineering Vol. 2, Issue 10, October 2013 [2] Vo Dinh Minh Nhat and Sungyoung Lee. “Improvement On PCA and 2DPCA algorithms for face recognition “CIVR 2005, LNCS 3568, Pp: 568-577 [3] D.Q. Zhang, and H.Zhou” (2D) 2PCA, Two-directional two-dimensional PCA for efficient face representation and recognition for Nero computing” [4] Zhao W., Chellappa R, Phillips and Rosenfeld, “Face Recognition: A Literature Survey”, ACM Computing Surveys, Vol. 35, No. 4, December 2003, pp. 399–458. [5] Patil A.M., Kolhe S.R. and Patil P.M, “2D Face Recognition Techniques: A Survey, International Journal of Machine Intelligence,”, ISSN: 0975–2927, Volume 2, Issue 1, 2010, pp-74-8 [6] G. H. Dunteman. “Principal Components Analysis”. Sage Publications, 1989. [7] Turk M. And Pentland A.” Eigenfaces for recognition”, J. Cogn. Neurosci. 3, 72–86, 1991 [8] S. J. Lee, S. B. Yung, J. W. Kwon, and S. H. Hong, “Face Detection and Recognition Using PCA”, pp. 84-87, IEEE TENCON,1999. [9] Kohonen, T., "Self-organization and associative memory”, Berlin: Springer- Verlag,, 1989. [10] Kohonen, T., and Lehtio, P., "Storage and processing of information in distributed associative memory systems", 1981. [11] Fischler, M. A.and Elschlager, R. A., "The representation and matching of pictorial structures", IEEE Trans. on Computers, c-22.1, 1973. [12] Yuille, A.L., Cohen, D. S., and Hallinan, P. W., "Feature extraction from faces using deformable templates", Proc. of CVPR, 1989. [13] Kanade, T., "Picture processing system by computer complex and recognition of human faces", Dept. of Information Science, Kyoto University, 1973.
- 93. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 83 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI [14] B.Moghaddam, and A. Pentland, “Probabilistic Visual Learning for Object Representation”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 19, No. 7, July 1997 [15] K. C. Chung, S. C. Kee, and S. R. Kim, “Face Recognition using Principal Component Analysis of Gabor Filter Responses”, p.53- 57, IEEE, 1999. [16] Hossein Sahoolizadeh, B. Zargham Heidari, and C. Hamid Dehghani, “A New Face Recognition Method using PCA, LDA and Neural Network”, World Academy of Science, Engineering and Technology 41008 [17] Kyungim Baek, Bruce A. Draper, J. Ross Beveridge, Kai , “PCA vs. ICA: A comparison on the FERET data set”. [18] B. Moghaddam, and A. Pentland, “An Automatic System for Model-Based Coding of Faces”, pp. 362- 370, IEEE, 1995. [19] Fleming, M., and Cottrell, G., "Categorization of faces using unsupervised feature extraction", Proc. of IJCNN, Vol. 90(2), 1990. [20] T. Yahagi and H. Takano, “Face Recognition using neural networks with multiple combinations of categories,” International Journal of Electronics Information and Communication Engineering., vol.J77-D-II, no.11, pp.2151-2159, 1994. [21] Patrick J. Grother; George W. Quinn; P J. Phillips,”Report on the Evaluation of 2D Still-Image Face Recognition Algorithms”, NIST Interagency/Internal Report (NISTIR) – 7709, June 17, 2010. [22] Patrick J. Grother, George W. Quinn and P. Jonathon Phillips,” Report on the Evaluation of 2D Still-Image Face Recognition Algorithms”, NIST Interagency Report 7709. [23] Patrik Kamencay, Robert Hudec, Miroslav Benco and Martina Zachariasova, “2D- 3D Face Recognition Method Based on a Modified CCA-PCA Algorithm “, International Journal of Advanced Robotic System, DOI: 10.5772/58251 [24] Phillip Ian Wilson, Dr. John Fernandez “Facial Feature Detection Using HAAR Classifiers”, Texas A&M University – Corpus [25] Matthew Turk and Alex Pentland,”Eigenfaces for Massachusetts Institute of Technology Recognition, vision and Modeling Group “, The Media Laboratory
- 94. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 84 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI [26] Paulo Menezes, Jos´e Carlos Barreto and Jorge Dias,” Face Tracking Based On Haar-Like Features And Eigenfaces”,ISR-University of Coimbra. [27] Barreto, Jos´e, Paulo Menezes and Jorge Dias,” Human-robot interaction based on Haar-like features and eigenfaces”, International Conference on Robotics and Automation,(2004). [28] Freund, Yoav and Robert E.Schapire (1996),”Experiments with a new boosting algorithm.” [29] Kalman, R.E. (1960),”A new approach to linear filtering and prediction problems”, Journal of Basic Engineering (82series D), 35–45. [30] Lienhart, Rainer and Jochen Maydt (2002).” An extended set of haar-like features for rapid object detection. “In: IEEE ICIP 2002, Vol. 1, pp 900-903. [31] Moghaddam, B. and A.P. Pent and A.P. Pentland (1995). “Probabilistic visual learning for object representation”, Technical Report 326, Media Laboratory, Massachusetts Institute of Technology. [32] Mohan, Anuj, Constantine Papageorgiou and Tomaso Poggio (2001).,”Example- based object detection in images by components”, IEEE Transactions on Pattern Analysis and Machine Intelligence 23(4), 349–361. [33] Oren, M., C.Papageorgiou, P.Sinha, E.Osuna and T.Poggio (1997),” Pedestrian detection using wavelet templates”. [34] Rainer Lienhart, Alexander Kuranov and Vadim Pisarevsky (2002),”Empirical analysis of detection cascades of boosted classifiers for rapid object detection”, MRL Technical Report, Intel Labs. [35] Rowley, Henry A., Shumeet Baluja and Takeo Kanade (1998),”Neural network- based face Detection”, IEEE Transactions on Pattern Analysis and Machine Intelligence 20(1), 23– 38. [36] Schneiderman, H. and T. Kanade (2000),” A statistical method for 3D object detection applied to faces and cars”, In International Conference on Computer Vision. [37] Sung, K. and T. Poggio (1998),“Example-based learning for view based face detection.” In IEEE Patt. Anal. Mach. Intell. 20(1), 39–51. [38] Turk, M.A. and A.P. Pentland (1991),”Face recognition using eigenfaces”, In Proc. of IEEE Conference on Computer Vision and Pattern Recognition pp. 586 – 591.
- 95. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 85 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI [39] Viola, Paul and Michael Jones (2001),”Rapid object detection using boosted cascade of simple features”, IEEE Conf.Pattern Recognition 2001. [40] Yang, Ming-Hsuan (2002),”Detecting faces images: A survey”. IEEE Transations on Pattern Analysis and Machine Intelligence 24(1), 34–58. [41] Sung, K. and T. Poggio (1998).” Example-based learning for view based face detection.” [42] Henry Rowley, Baluja S. & Kanade T. (1999) “Neural Network-Based Face Detection, Computer Vision and Pattern Recognition”, Neural Network-Based Face Detection, Pitts-burgh, Carnegie Mellon University, PhD thesis. [43] KahKay Sung & Tomaso Poggio (1994) Example Based Learning For View Based Human Face Detection, Massachusetts Institute of Technology Artificial Intelligence Laboratory and Center For Biological And Computational Learning, Memo 1521, CBCL Paper 112, MIT, December. [44] Henry A. Rowley, Shumeet Baluja &Takeo Kanade. (1997) Rotation Invariant Neural Network-Based Face Detection, December, CMU-CS-97-201 [45] Jeffrey S. Norris (1999) Face Detection and Recognition in Office Environments, thesis, Dept. of Electrical Eng. and CS, Master of Eng in Electrical Eng., Massachusetts Institute of Technology. [46] Hazem M. El-Bakry (2002), Face Detection Using Neural Networks and Image Decomposition Lecture Notes in Computer Science Vol. 22, pp:205-215. [47]. M. Kass, A. Witkin, and D. Terzopoulos. Snakes: Active contour models. IJCV, 1(4):321–331, 1987 [48]. T. Cootes, C. Taylor, D. Cooper, and J. Graham. Active shape models - Their training and application. CCVIU, 61(1):38–59, 1995. [49]. T. Cootes, G. Edwards, and C. Taylor. Active appearance models. PAMI, 23(6):681–685, 2001. [50]. S. Sclaroff and J. Isidoro. Active blobs. In ICCV, 1998. [51]. M. Jones and T. Poggio. Multidimensional morphable models. In ICCV, pages 683– 688, 1998. [52]. P. Ekman and W.V. Friesen. Facial Action Coding System: Investigator’s Guide. Consulting Psychologists Press, 1978.
- 96. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 86 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI [53]. B. Fasel and J. Luettin. Automatic facial expression analysis: A survey. Pattern Recognition, 36:259–275, 2003. [54]. M. Pantic and L.J.M. Rothkrantz. Automatic analysis of facial expressions: The state of the art. IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(12):1424– 1445, 2000.
- 97. PHOTOGRAPH DATABASE MANAGEMENTUSING FACE RECOGNITION ALGORITHM 87 DEPARTMENT OF COMPUTER SCIENCE, BITS PILANI