Use of facial landmarks for evaluating sentiment analysis on human faces

1. Introduction
Facial Landmarks is the localization and detection of core features on the face. Facial
Landmarks involve the focus of an area of interest on the human face. Its application finds
various usage but not limited to these areas, such as video surveillance, computer vision,
human-computer interaction. It revolves across core subject areas in face detection, face
expression detection, face tracking. It is a subset of the shape prediction; shape prediction
is used to localize distinguished features in a shape. A shape prediction attempts to localize
key points of interest along with the shape [1]. Facial landmarks are used to localize and
represent the salient regions in the face such as eyebrows, eyes, nose, mouth, and jawline.
Despite, the fact that face landmark technology has found immense several applications
across different fields, it still remains one of the most challenging tasks to perform.
Challenges such as poor image quality, image orientation, pose variation, lighting condition
on images, a variation of face shapes and sizes. In video-based face recognition, several
issues take place such as illumination variation, different pose orientation. In different
algorithms, images of faces to be trained are stored in a database and compared to the input
facial image There is a need to make recognition reliable under an uncontrolled
environment where the images captured are likely to reduce quality and other useful details
due to various lighting conditions imperatives for the face recognition algorithm to function
properly. The variation of the output will immensely depend on this. Compromise viewing
condition makes this a challenging area. Facial Landmark is used in various computer
vision techniques such as face part extraction, face alignment, pose estimations, face
swapping, blink detection. The facial landmark algorithm is very specific to the kind of
problem and cannot be guaranteed to work unless it is applied and results are obtained.
Sequel to this, we have an implementation face analyzing algorithm to categorize human
face as happy or sad. Facial Landmark can be achieved by face detection and feature
extraction, it is done with the use of Histogram of Oriented Gradients (HOG) and Linear
Support Vector Machine (SVM) Algorithm. In this paper, our approach tends to leverage
on a compositional learning strategy of combining both algorithms. We use learning
machine learning for face detection and deep learning for feature extractions. In other
words, we use HOG + Linear SVM for feature detection and extraction of distinguishable
features with CNN.
Related Work
In the last decade, a great deal of work has been done on this topic, Barnabas et al [5]
developed a three-stage detection strategy that hierarchically integrates face- and facial
landmark candidates as marked by individual filters. The image is first scanned using multi-
resolution eye pair detectors’ to mark possible candidate face regions, which are then
further processed by full-face detectors to find the best matching candidate. Facial
landmarks are found using specific eye, nose and mouth filters applied to regions of
interests meeting average geometric constraints derived from a large set of face images.
The specific subsystem for face- and facial landmark detection employs filter banks and

2. registers memory activations as the center of the retinal sampling grid move over a selected
region in the image. Consisting of four stages retinal sampling, feature extraction, visual
filter banks, and holistic detection/classification. The drawback of this method can be found
in the time-consuming scanning algorithm used to prime the facial landmarks, with purely
data-driven attention mechanisms, the inaccurate results yielded for a compromised image
quality input
Several algorithms can be used to combat facial landmarks among these are histogram of
oriented gradients, local binary pattern histogram, eigenfaces, Fisher faces. In the histogram
of oriented gradients (HOG), every pixel of the captured image is compared with other
pixels in darkness. Based on that, an accurate image is obtained. It is unaffected by any
kind of lighting problems. Hence, it's a less complicated and more efficient technique for
face detection, we tend to combine the machine learning and deep learning approach to
achieve this result.
Histogram of Oriented Gradients (HOG) is commonly implored as a shape predictor form
which by detecting objects or human face in computer vision and image processing. The
object search is based on the detection technique applied for the small images defined by
the sliding detector window that probes region by region of the original input image and its
scaled versions [2]. Histogram of Oriented Gradients is generally used along with Support
Vector Machine (SVM) classifiers. Which handles the detecting the bounding box is used
to convert the 2-dimensional representation of the image into spatial coordinates. The
techniques counts occurrences of gradient orientation in localized portions of an image.
The simplest method to do that is to use vertical and horizontal operators [3]:
𝐺_𝑥 (𝑦, 𝑥) = 𝑌(𝑦, 𝑥 + 1)– 𝑌(𝑦, 𝑥 − 1); 𝐺_𝑦 (𝑦, 𝑥) = 𝑌(𝑦 + 1, 𝑥)– 𝑌(𝑦 − 1, 𝑥) (1)
𝑌( 𝑦, 𝑥) - pixel intensity at coordinates 𝑥 and 𝑦
𝐺𝑥(𝑦, 𝑥) - horizontal gradient
𝐺𝑦(𝑦, 𝑥) - vertical gradient
Magnitude and phase of the gradient are determined as:
𝐺(𝑦, 𝑥) = √ 𝐺_𝑥(𝑥, 𝑦)2 + 𝐺_𝑦(𝑥, 𝑦)2 (2)
𝜃( 𝑦, 𝑥) = 𝑎𝑟𝑐𝑡𝑎𝑛(𝐺𝑥(𝑦, 𝑥)) (3)
HOG is a good performer of human facial detection, however, it is very sensitive to image
rotation. Because the HOG shows occurrences of specific gradient orientation, the histogram
can be considerably changed by image rotation. Therefore, HOG is not suitable as feature
vectors for classification of objects which are often detected as rotated images or texture
images, hence the need to augment HOG with Linear SVM.

3. References
[1] Adrian Rosebrock “Facial landmarks with dlib, OpenCV, and Python,”
https://www.pyimagesearch.com/2017/04/03/facial-landmarks-dlib-opencv-python/
[2] Nicolas Delbiaggio, A Comparison of Facial Recognition’s Algorithms.
[3] R.Angeline, Kavithvajen.K, Toshita Balaji, Malavika Saji, Sushmitha.S.R, “CNN
Integrated With HOG For Efficient Face Recognition”, International Journal of Recent
Technology and Engineering (IJRTE) ISSN: 2277-3878, Volume-7, Issue-6, March 2019.
[4] Nadia Jmour; Sehla Zayen; Afef Abdelkrim, “Convolutional Neural networks for image
classification”, 2018 International Conference on Advanced Systems and Electric
Technologies (IC_ASET), 22-25 March 2018.
[5] Barnabas Takacs and Harry Wechsleri; “Detection Of Faces And Facial Landmarks Using
Iconic Filter Banks”, Department of Computer Science, George Mason University, Fairfax,
VA 22030, U.S.A