AI(Artificial Intelligence) is the future in software, and already industry is advancing its steps in terms of machine learning and its application for mobiles as well. Here is my view on what is currently been implemented and what may come in future.

1. MACHINE LEARNING -
MOBILE VISION
-Manikanta Garikipati

2. WHAT COMES IN FUTURE
➤ Deeper personalisation - Personalised recommendations and less
annoying ads
➤ Neural networks running on mobile devices - Mobile may start
running machine learning tasks locally Ex: face detection, speech and other
innovations.
➤ Mobile experience automation - Automate work of different
connected apps ex: IFTT
➤ NLP will begin to make sense - In a state that it can barely beat rule
based engines. That’s an exaggeration it’s still in its infancy.
➤ Real time speech translation - If this technology continues to develop, it
could significantly improve the quality of international communication or
even eradicate language barriers.

3. ➤ Face Detection
➤ Natural Language Processing
➤ Barcode Detection
➤ Text Detection (OCR)
What is machine learning?
Machine learning is the idea that there are generic algorithms that can
tell you something interesting about a set of data without you having to
write any custom code specific to the problem. Instead of writing code,
you feed data to the generic algorithm and it builds its own logic based
on the data.
For example:

4. ➤ Supervised and unsupervised learning
➤ Neural Networks
➤ Stateless vs State-full model
➤ Deeplearning
➤ Convolution(small neural networks, maxpooling)
➤ Precision(true positives vs all positive guess) and recall(true positives vs
dataset)
ML Basics

5. FACE DETECTION
➤ Process of locating human faces in visual media(digital images or video).
➤ Once a face is detected it can be
searched for landmarks such as eyes,
nose etc..
➤ A landmark is a point of interest
within a face. The left eye, right eye,
nose base are examples of
landmarks…
➤ We can see its common usage in
cameras(they recognise faces to make
sure all of them are in focus it takes
the picture)
➤ Social networking sites like Facebook
uses it!!

6. HOW TO DETECT FACES?
Face Recognition is a series of several related problems:
➤ First, look at a frame and find all faces in it.
➤ Second, focus on each face and able to understand that even if a face is
turned in a weird direction or in bad lighting, it is still the same person.
As a human, your brain is wired to do all of this automatically and instantly. In
fact, humans are too good at recognising faces and end up seeing faces in
everyday objects:

7. STEP 1: FINDING ALL THE FACES
➤ The first step in the pipeline is going to be finding faces in a photograph
before we try to tell them apart.
➤ More reliable solutions exist these days to recognise a face in a given frame.
We are going to look at HOG(Histogram of Oriented Gradients) solution
Mobile cameras using face detection

8. ➤ We’ll look at every single pixel in our image one at a time. For every single
pixel, we want to look at the pixels that directly surrounding it:

9. ➤ Our goal is to figure out how dark the current pixel is compared to the
pixels directly surrounding it. Then we want to draw an arrow showing in
which direction the image is getting darker:
➤ If you repeat that process for every single pixel in the image, you end up
with every pixel being replaced by an arrow. These arrows are called
gradients and they show the flow from light to dark across the entire image:

10. ➤ We’ll break up the image into small squares of 16x16 pixels each. In each
square, we’ll count up how many gradients point in each major direction
(how many point up, point up-right, point right, etc…). Then we’ll replace
that square in the image with the arrow directions that were the strongest.
➤ The end result is we turn the original image into a very simple
representation that captures the basic structure of a face in a simple way:

11. ➤ To find faces in this HOG image, all we have to do is find the part of our
image that looks the most similar to a known HOG pattern that was
extracted from a bunch of other training faces:

12. STEP 2: POSING AND PROJECTING FACES
➤ To account for this we will warp picture so that eyes and lips are always
in sample place in image. This makes face comparison a bit easier.
➤ To do this, we are going to use an algorithm called face landmark
estimation.
➤ There are lots of ways to do this, but we are going to use the approach
invented in 2014 by Vahid Kazemi and Josephine Sullivan
➤ Now we have to deal with the problem that faces turned different
directions look totally different to a computer

13. ➤ The basic idea is we will come up with 68 specific points (called landmarks) that exist
on every face — the top of the chin, the outside edge of each eye, the inner edge of each
eyebrow, etc. Then we will train a machine learning algorithm to be able to find these 68
specific points on any face
The 68 landmarks we will locate on every face locating the 68 face landmarks on our test image
➤ You can also use this same technique to implement your own version of Snapchat’s
real-time 3d face filters!

14. ➤ Now that we know were the eyes and mouth are, we’ll simply rotate, scale and shear the
image so that the eyes and mouth are centered as best as possible. We won’t do any fancy
3d warps because that would introduce distortions into the image. We are only going to
use basic image transformations like rotation and scale that preserve parallel lines (called
affine transformations):
➤ Code to find landmarks: https://gist.github.com/ageitgey/ae340db3e493530d5e1f9c15292e5c74
➤ Now no matter how the face is turned, we are able to center the eyes and mouth are in
roughly the same position in the image.

15. Thank You!
Questions?

16. Further Reading
➤ https://github.com/cmusatyalab/openface
➤ https://developers.google.com/vision/face-detection-concepts
References
➤ https://cmusatyalab.github.io/openface/
➤ https://medium.com/@ageitgey/machine-learning-is-fun-part-4-modern-
face-recognition-with-deep-learning-c3cffc121d78#.red05avch