For the full video of this presentation, please visit: https://www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/may-2017-embedded-vision-summit-ghazali For more information about embedded vision, please visit: http://www.embedded-vision.com Adham Ghazali, co-founder and CEO of Imagry, presents the "Edge Intelligence: Visual Reinforcement Learning for Mobile Devices" tutorial at the May 2017 Embedded Vision Summit. Real-life visual data encompasses a tremendous amount of information and presents a huge challenge for the design and development of a perceptual engine. Smart machines equipped with visual understanding technology will always be presented and challenged with new data. In this talk, Ghazali presents algorithm methods to allow learning in the end device to enable it to understand new data. The main challenges to learning in the end device stem from the lack of computing power, lack of access to sufficient data samples and the need for involvement of human experts. Imagry addresses these challenges by combining a binary weights representation of deep neural networks and reinforcement learning. In particular, Ghazali explores and introduces a self-expanding cost function and the incorporation of external memory to enable DNNs to adapt to new data.

1. Copyright © 2017 IMAGRY 1
Adham Ghazali
May 2017
Edge Intelligence: Visual Reinforcement
Learning for Mobile Devices

2. Copyright © 2017 IMAGRY 2
Difficult Imagery
Object Variability
New Objects
Real-life Vision
Problems with supervised learning

3. Copyright © 2017 IMAGRY 3
Visual Reconstruction
Association with
Language
Adaptation
Real-life Vision
Human inspired solution Real Life
Life in the eyes
of the Dataset
Dataset

4. Copyright © 2017 IMAGRY 4
Supervised Learning
Organization of
knowledge
Template for recognition
Real-life Vision
Root in Psychology- Jean Piaget

5. Copyright © 2017 IMAGRY 5
Using an existing schema to deal
with a new object or situation.
Dealing with new
Information
Adaptation
Real-life Vision
Root in Psychology- Jean Piaget

6. Copyright © 2017 IMAGRY 6
å
Templates are not easy to collect
Understanding new information
Learning new information
Shortage of samples
Challenges for AI
6.

7. Copyright © 2017 IMAGRY 7
Proposed Solution: 2 Stages
Auto Expanding Engine Stages:
Stage 1: Curriculum Training for adding new classes
Stage 2: Inverse Reinforcement Learning (IRL)
Why:
-Not obvious reward function; lack of environment for action
Phase 1: Server
Phase 2: Edge Device

8. Copyright © 2017 IMAGRY 8
Auto Expanding Engine
E0
Data(D0)
Initialization
E1
D1
E0
E3
D2
E1
Modifications:
1- Adding new Class (Stage 1)
2- Update of parameters (Stage 2)
3- Removal of parameters
4- Update Memory

9. Copyright © 2017 IMAGRY 9
Stage 1: Curriculum Training
Modifications:
1- Adding new Class
This Engine is capable of adding memory to the top level of the network
automatically by distinguishing whether a particular class is new or old.
Algorithm:
- Forward Pass Set D containing (D0, D1..etc)
- If D!= D0: Allocate weight and activation layers for D1
- Using Standard Back Propagation, train E0 with D1
E0
Data(D0)
Initialization
E1
D1
E0
D2
E1

10. Copyright © 2017 IMAGRY 10
Curriculum Training (Stage 1)—Results
Results
- Random Selection of new classes
- Automatic Identification of new class
- 4000 classes
Time Acc (top5)
One training
session
6 days 75%
Curriculum
Training
6 days 74%
One training session: All classes are
trained simultaneously from the start of the
session.
Curriculum Training: Classes are added
incrementally to the output layer based on
Automatically Identifying the classes.

11. Copyright © 2017 IMAGRY 11
Stage 2: Reinforcement Training
EN1Data(D) EN2 Data(D)
2
0
1
4
3
Phase 1: Training on labeled data
Engine 1 and 2 are initially trained
separately using different starting
conditions and different initial sets
using method in stage 1.
(D) is a large labeled dataset
0
1
2
4
3
Modifications:
1- Adding new Class
2- Update of parameters

12. Copyright © 2017 IMAGRY 12
Stage 2: Reinforcement Training
Interchanging teacher student networks (Server)
EN1Data(8) EN2 Data(8)
2
9
8
4
Phase 1: Training on labeled
data
Feedback Network: Is consensus
between the engines; in this phase the
engines are trained to expand based on
existing dataset (D); Simultaneously a one
layer Feedback NN is trained to provide
feedback based on the following labels:
1. E1 and E2 produce correct
classification results
2. Either E1 or E2 is incorrect
3. E1 and E2 produce incorrect
classification results
The inputs to the FB network are
interchanged.
0
1
4
3
Feedback

13. Copyright © 2017 IMAGRY 13
Stage 2: Reinforcement Training
(Embedded Engine)
EN1Data(S) EN2 Data(S)
2
0
1
4
3
Phase 2: Training on large
unlabeled data
- E1 and E2 are both trained on
an unlabeled large dataset to
correctly classify unlabeled
input.
- The auto expanding
mechanism is triggered at
confidence levels returned by
the Feedback Network.
0
1
2
4
3
Auto
expanding
Feedback

14. Copyright © 2017 IMAGRY 14
Evaluation and Experimental Results
Stage 2 Labeled Set
Separation
Tr. # of samples= 50
Labeled Set
Separation
Tr. # of samples= 400
Unlabeled Set
Separation
# of Samples= 5
Unlabeled Set Separation
# of Samples= 1
E1 69.7% 82.6% 17.7% 10.1%
E2 71.3% 85.8% 18.1% 10.7%
Phase 2 with F/B 75.5% 87.4% 25.6% 12.3%
Dataset:
D: 4000 classes from ImageNet (labeled set)
S: 5000 classes from ImageNet (unlabeled set*)
Total Number of classes= 9000
Evaluation Metrics (Levels of separation):
𝑺𝒆𝒑𝒆𝒓𝒂𝒕𝒊𝒐𝒏 =
𝑵𝒖𝒎𝒃𝒆𝒓 𝒐𝒇 𝑰𝒅𝒆𝒏𝒕𝒊𝒇𝒊𝒆𝒅 𝒄𝒍𝒂𝒔𝒔𝒆𝒔 𝒂𝒔 𝑵𝒆𝒘 𝒌𝒏𝒐𝒘𝒍𝒆𝒅𝒈𝒆
𝑻𝒐𝒕𝒂𝒍 𝒏𝒖𝒎𝒃𝒆𝒓 𝒐𝒇 𝒄𝒍𝒂𝒔𝒔𝒆𝒔
*The labels of S are not used in training the engine only their count is used to determine the
total number of classes

15. Copyright © 2017 IMAGRY 15
Use Cases of Auto Expanding Engines (AEE)
Defining alternate schemes and policies for autonomous driving
- Understanding new situations (e.g. emergencies).
• AEE provides a structured framework for Autonomous driving switching
between different mechanisms
- Automatic update of driving policy (urban/highway/new city)
- Understanding new objects: unidentified obstacles!

16. Copyright © 2017 IMAGRY 16
Use Cases of Auto Expanding Engines (AEE)
Identification of new diseases in medical data
AEE engines provide a framework for medical image understanding where it
can be used to:
- Identify rare cases in medical imagery
-Update response policy for medical practitioners

17. Copyright © 2017 IMAGRY 17
Conclusion and Discussion
The results suggest that training on a labeled data set and then extending the
engine with unlabeled dataset training can provide significant understanding
of new information.
Feedback mechanism improves the results significantly.
Number of Samples: In some applications the number of samples is 1.
Future Work: Study in depth the relationship between the number of samples
with the number of engines. Test on more specific datasets with fine grained
classes and provide in depth results for use cases specific scenario.

18. Copyright © 2017 IMAGRY 18
Thank You
mail
adham@imagry.co
phone
Adham +972-542-005870
online
www.imagry.co
LinkedIn / Imagry
Facebook/ Imagry
Twitter/ Imagry

19. Copyright © 2017 IMAGRY 19
References
1. Abbeel, Pieter. Inverse Reinforcement Learning.
https://people.eecs.berkeley.edu/~pabbeel/cs287-
fa12/slides/inverseRL.pdf.
2. Ng, Andrew. Algorithms for Inverse Reinforcement Learning.
http://ai.stanford.edu/~ang/papers/icml00-irl.pdf.
3. Santoro, Adam. One-shot Learning with Memory-Augmented Neural
Networks. https://arxiv.org/pdf/1605.06065.pdf.
4. Vinyals, Oriol. Matching Networks for One Shot Learning.
https://arxiv.org/pdf/1606.04080.pdf.