This program can detect presenter's motions. In this demo, introducing the slides (postures of pointing at slides) is defined as good motion. Playing with hairs is defined as bad motion.

1. Motion Detection
Zezheng(Andy) Long
Machine Learning/AI Engineer
Chih Lai, Ph.D.
Supervisor, AI Professor at University of St. Thomas
© Copyright 2019 by Zezheng Long

2. • Collect RGB Image Data
• Create Motion Proposal from Gray-Scale Frames
• Convert Motion Proposal to Binary Motion Map (BMM)
• Create Color Motion Map (CMM) by Stacking 8 BMMs
• Retrain CNN Transfer Learning Model to Classify
Motions
• Real-Time Motion Detection Demo
© Copyright 2019 by Zezheng Long
Contents

3. Collect RGB Image Data
Resolution Rate: 640 x 480
Camera FPS: 30fps
© Copyright 2019 by Zezheng Long

4. Motion Proposal
Difference Between 2 Frames
© Copyright 2019 by Zezheng Long
Why Do I Transfer RGB to Gray-Scale?
• Get Rid of Color Effect
• Less Noise
Original Motion Proposal Original Motion Proposal x 50 (easy to visualize the noise)

5. Binary Motion Map (BMM)
Original Motion Proposal Original Motion Proposal x 50 (easy to visualize the noise)
BMM
Motion Proposal > 40 = 1
Motion Proposal <=40 = 0
© Copyright 2019 by Zezheng Long
What Does BMM Do?
• Capture Spatial Motion Information

6. Color Motion Map (CMM)
BMM 1 BMM 2 BMM 3 BMM 4 BMM 5 BMM 6 BMM 7 BMM 8
CMM 1
© Copyright 2019 by Zezheng Long
• Stack 8 BMMs, and Assign Colors
to Motions Based on Time
• See the Color-Map Bar
Less
Recent
BMM
More
Recent
BMM
What Does CMM do?
• Capture Spatial + Temporal
Motion Information

7. CNN Transfer Learning
• Transfer Learning Model: AlexNet
• Dataset: 2966 CMMs
• Good Motion: 1483 CMMs
• Bad Motion: 1483 CMMs
• Testing Result:
• Recall: 98.7%
• precision: 99.7%
• Accuracy: 99.2%
© Copyright 2019 by Zezheng Long

8. Real-Time Motion Detection Demo
•Exciting Part!!!
© Copyright 2019 by Zezheng Long

9. Further Study
• Use Case:
• Observe Patients / AI Family Doctor
• Public Security / Surveillance
• … …
• Multiple People
• 3-D camera
• Instance Segmentation
© Copyright 2019 by Zezheng Long

10. Questions
• Why NOT Use CNN AutoEncoder + LSTM?
• … …?
© Copyright 2019 by Zezheng Long