The document discusses optimizing a face recognition model for processing images from multiple IP cameras with low memory usage and fast response times. It proposes using the LBPH face recognition algorithm with a database structure to match faces from the camera stream to trained images. Tests were able to recognize faces from a wireless camera with 95% accuracy using this approach on Google Cloud servers. Future work could involve object recognition, surveillance applications, and using deep learning models.

1. Smart Face Recognition
Analysis Report​- ​Optimization, Response Time & Efficiency
Introduction: Face Recognition using opencv python to detect face & recognize all
facial features of particular person can be achieved with more than 90% efficiency in
opencv-3.1.0. The results depends on trained model & the database of grayscale
images supplied.
Purpose of Analysis: The objective is to find the best face recognition model/algorithm
for less memory utilization & low processing from servers possible i.e, millions of
images can be processed much faster by ​segmentation, ​multi-threading or parallel
processing. Also, explore & test best computational models like knn(k Nearest
Neighbour), cnn(Computational Neural Network), dnn(Deep Neural Network).
Hypothesis: ​From the official documentation of Computer Vision​[1]​, the currently
available algorithms for opencv2 & later versions are:
● Eigenfaces (see​ ​createEigenFaceRecognizer()​)
● Fisherfaces (see​ ​createFisherFaceRecognizer()​)
● Local Binary Patterns Histograms (see​ ​createLBPHFaceRecognizer()​)
We choose LBPHFaceRecognizer() for our project as this algorithm can recognize at
different angles, color modes, ambiance(lights) for all trained datasets. The basic idea
of Local Binary Patterns is to summarize the local structure in an image by comparing
1 Smart Face Recognition System

2. each pixel with its neighborhood. Take a pixel as center and threshold its neighbors
against. If the intensity of the center pixel is greater-equal its neighbor, then denote it
with 1 and 0 if not. You’ll end up with a binary number for each pixel, just like 11001111.
So with 8 surrounding pixels you’ll end up with 2^8 possible combinations, called ​Local
Binary Patterns or sometimes referred to as ​LBP codes​. The first LBP operator
described in literature actually used a fixed 3 x 3 neighborhood just like this:
By definition the LBP operator is robust against monotonic gray scale transformations.
We can easily verify this by looking at the LBP image of an artificially modified image
(so you see what an LBP image looks like!):
Watch this video for better idea about feature extraction: ​https://vimeo.com/12774628
Problem Statement: ​Required a face recognition model that can capture image from
wireless camera and if the user’s face exists in database then allow him/her to enter
else display option to train than face & put it to existing database by the permission of
admin else trigger an alarm for unauthorized entry. This whole thing must be further
2 Smart Face Recognition System

3. implemented to android application & the face recognition model can be processed via
backend servers. Further, optimize the whole process and increase the response time
Methodology: ​One possible approach/solution to the above problem can be as follows.
● Create database for facerec with possible structure as table
facerec{id,name,dob,time}
● Get image from camera over wireless network(​IP & PORT​) only if motion of
human is detected then capture face.
● Get image sent over network then process it:
○ Get ID from facerec_database & if ID of image received is same then
process image for that ID dataset available from trained model
○ Get the prediction & confidence of result.
● Start the server and return the results in ​json​ format (ID, result).
○ ID: image ID from the database
○ result: true/false
● Get the json result in android application & display the name corresponding to
that image ID from ​facerec​ database.
Flow Chart: ​All possible flow charts of face recognition models
3 Smart Face Recognition System

4. Deploy in cloud-servers:
1. Using Google Cloud: VM instances (​https://console.cloud.google.com​)
VM Instance- Ubuntu 16.4 4Gb RAM, 375 GB Storage, 3 Years Usage
Cost- ​45.60$ per month
Benefits: All kinds of servers can be installed LAMP, Apache Tomcat, WAMP etc.
4 Smart Face Recognition System

5. Test Results: ​success when passed images for each trained model in the
facerec_app. Using Mozilla Browser for ssh into VM instance.
2. Using Python Hosting: Heroku (​https://www.heroku.com/pricing​)
Cost upto: ​500$ per month
5 Smart Face Recognition System

6. Deploy in Raspberry Pi 3:
● Hardware Details: ​1Gb RAM, 1.2GHz ARM Cortex Processor, 16Gb Storage,
64-bit Quad Core Processor( increases efficiency by 33%)
● Cost: Rs.4499 (​RPi​) + Rs. 2349 (​Pi Cam​) = Rs. 6848/-
● Opencv build using cmake requires: ​1h 12m​ to compile
● Problems: 24x7 Running Mode, Heating of Chip, May hang
● Requires testing for 24x7.
● Benefits: Small & portable, All modules built-in same chip.
Problems with other ARM Processor: Required interfacing of all modules like WiFi
module, Memory etc again embedded person required.
6 Smart Face Recognition System

7. Multiple Camera Tests: ​Connected USBCam & WebCam
Using opencv multiple streaming is achieved
IP Camera: ​Video stream can be captured from multiple IP Cameras over WiFi network
& further can be converted into frames to numpy array for processing.
After mapping the HTTP video stream of IP Camera to video driver ​/dev/video*
videoCapture(1) will work for that mapped device ID & all opencv processing is possible
with the same.
7 Smart Face Recognition System

8. Problems/Parameters for IP Camera:
● Bandwidth required to process multiple cameras: Video CODEC (Compression),
Streaming Mode, Frame Rate, Resolution, Video Quality.
● Network topology
● Resource calculation for servers to process & capture streams of ip camera from
http​ to​ /dev/video*
Using IP Camera Simulation Software to get above parameters for 100+ cameras.
IP Video System Design Tools by Vivotek: To monitor the network & check for all
parameters. ​http://www.vivotek.com/ip-video-system-design-tool/#downloads
8 Smart Face Recognition System

9. 100 Viviotek cc8130 cameras 30 days storage required 1,01,868.6 GB &
Bandwidth= 314.11Mbit/s
Using this tool we can estimate everything & plan a efficient systems for cctv monitoring
& processing wireless through servers.
9 Smart Face Recognition System

10. WebCam Streaming: using opencv
https://github.com/benhowell/examples/tree/master/WebcamStreamingOpenCV
Face Detection using IP Camera connected to Netgear Router is tested and found
some frame lags and error detection. Now, training of data sets should be done with
same situation, lightning conditions & camera.
Python Client Server Connection: ​To deploy in My Attendance App
● Create php web service to upload image and store to database.
● Get images in python.
10 Smart Face Recognition System

11. Results:
● Faces can be trained in real time into datasets using LBPH model with response
time of 0.03 ~ 0.05 secs from Ubuntu 16.04, 4GB RAM intel i5 Processor.
● Recognition of faces is done from wireless camera using web server remotely
with success rate of 95% approx.
● Success on Cloud Servers (Google VM Instance: Ubuntu 16.4, 4GB RAM, 10GB
Storage)
Future Scope:
● Object Recognition: It can be used to check terrorist activities like human with
weapons, knife, face-masks at ATM etc.
● Camera Surveillance in traffic, hotels, institutions & companies etc.
● All those big companies like Facebook, Google, Microsoft are now relying on
their own algorithms & using Tensorflow and Deep Neural networks in A.I to
predict their datasets and for the same openface is utilized.
● Also, with the use of Google Cloud for Vision API object recognition is very easy
in tensorflow & python. (​https://cloud.google.com/vision/​)
● Emotion Detection like mood of person happy, sad, crying faces.
References:
1. http://docs.opencv.org/2.4/modules/contrib/doc/facerec/facerec_tutorial.ht
ml#face-recognition-with-opencv
2. http://www.pyimagesearch.com/2016/10/24/ubuntu-16-04-how-to-install-opencv/
3. https://medium.com/@ageitgey/machine-learning-is-fun-part-4-modern-face-reco
gnition-with-deep-learning-c3cffc121d78
4. http://cmusatyalab.github.io/openface/
5. https://multi-link.net/much-bandwidth-need-ip-video-surveillance-application/
GitHub: ​https://github.com/vsltech/smartfacerecognition
Project Details:
http://www.vslcreations.com/2017/12/smart-face-recognition-system-using.html
11 Smart Face Recognition System