Slides used on my session "Let’s code a drone to follow faces syncing everything with Azure IoT" for the CodeGen Verona on 2021 02 13.

1. 1
TOPIC
Bruno Capuano
Innovation Lead @Avanade
@elbruno | http://elbruno.com
Let's hack a drone,
hack the
camera and use AI!

2. @elbruno
elbruno
Bruno Capuano
Bruno Capuano

3. 3
Optional knowledge
Python understanding
Cognitive Services
Face Recognition concepts
Docker and Containers Concepts
No TCP / UDP

4. 4
DJI Tello
https://www.amazon.com/Tello-CP-PT-00000252-01-Quadcopter-Drone/dp/B07BDHJJTH

5. Intel Processor,
Professional processing
yields high-quality
footage.
Flight Time: 13 minutes Image Transmission
Distance: 100 meters
HD Transmission: 720p
Smart Switching: 2
Antennas
Auto Takeoff/Landing
DJI Tello
Lift off or land with a
single tap.
Low Battery Protection Alerts go off when
your battery gets low.
Fail-safe Protection
Land safely, even if you
lose connection.
Vision Positioning
System
Smart technology that
facilitates precise
hovering.

6. DJI Tello - Drone Programming
Mobile Scratch Programming
Move-Based Control
Programming Simulator
SDK
DIY Accessories
https://www.ryzerobotics.com/tello

7. DJI Tello - Tello SDK 2.0 User Guide.pdf
https://dl-cdn.ryzerobotics.com/downloads/Tello/Tello%20SDK%202.0%20User%20Guide.pdf

8. DJI Tello - Tello SDK 2.0 User Guide.pdf
https://dl-cdn.ryzerobotics.com/downloads/Tello/Tello%20SDK%202.0%20User%20Guide.pdf

9. 9
Connect to Drone and Share
Network
Cable
Wireless
Connection
Lynsys Velop Node Gaming / Work PC DJI Tello Drone

10. 10
Let’s connect
MakeMagicHappen();
https://www.avanade.com/AI

11. DJI Tello - Tello SDK 2.0 User Guide.pdf
https://dl-cdn.ryzerobotics.com/downloads/Tello/Tello%20SDK%202.0%20User%20Guide.pdf

12. SDK Description
2 main actions
Send Commands
Read Commands
Flow
1st Command >> “command”
2nd Command >> “battery?”

13. 13
Demo: Hello World !!!
or get the battery charge %
MakeMagicHappen();
https://www.avanade.com/AI

14. Drone Camera Feed
Stream available via UDP
udp://192.168.10.1:11111
Didn’t work with VLC 
Cool tool: FFmpeg
.ffplay.exe -i
udp://192.168.10.1:11111
https://elbruno.com/2020/02/17/coding4fun-how-to-control-your-drone-with-20-lines-of-code-7-n/

15. 15
Demo: Camera OpenCV
MakeMagicHappen();
https://www.avanade.com/AI

16. State-of-the-art face detection demo
(Courtesy Boris Babenko)

17. 17
Face Detection / Recognition
FACE
DETECTION
DETECT FACIAL
FEATURES
FACE
RECOGNITION
OBJECT
RECOGNITION

18. 18
Face Detection / Recognition
FACE
DETECTION
DETECT FACIAL
FEATURES
FACE
RECOGNITION
OBJECT
RECOGNITION

19. 19
Viola/Jones (2001) Object Detection Framework
The algorithm has four stages:
• Haar Feature Selection
• Creating an Integral Image
• Adaboost Training
• Cascading Classifiers
Haar Features
- The eye region is darker than the upper-
cheeks.
- The nose bridge region is brighter than the
eyes
Supported by OpenCV !
https://en.wikipedia.org/wiki/Viola%E2%80%93Jones_object_detection_framework

20. 20
Viola/Jones (2001) Object Detection Framework
https://en.wikipedia.org/wiki/Viola%E2%80%93Jones_object_detection_framework

21. 21
Demo: Face Detection using
HaarCascades and OpenCV
MakeMagicHappen();
https://www.avanade.com/AI

22. 22
Prepare Your Data Build & Train Run
Build your own custom machine learning models

23. 23

24. 24
Demo: Object Recognition using
Custom Vision Models
MakeMagicHappen();
https://www.avanade.com/AI

25. 25
Introduction to the
drone. Features and
how-to of the SDK
Drone SDK
Analyze the drone camera
feed using a custom object
recognition model created
with http://customvision.ai
CustomVision.AI
with Containers
How to use OpenCV
to access the Drone
Camera feed via UDP
feed.
Drone Camera
Use Haar Cascades
files to perform face
detection using
OpenCV from the
drone camera feed.
Face Detection
with OpenCV
Quick Recap

26. 26
Resources
DJI Tello Drone,
https://www.amazon.com/Tello-CP-PT-00000252-01-Quadcopter-Drone/dp/B07BDHJJTH
Ryze Tello SDK, https://www.ryzerobotics.com/tello
My Blog – 20 lines to control your drone,
https://elbruno.com/2020/02/17/coding4fun-how-to-control-your-drone-with-20-lines-of-code-7-n/
Viola Jones, Object Detection Framework,
https://en.wikipedia.org/wiki/Viola%E2%80%93Jones_object_detection_framework
Cognitive Services Custom Vision, https://customvision.ai
Damian Tello, Python DJI SDK, https://github.com/damiafuentes/DJITelloPy

27. Resources
EASY WAY TO INSTALL OPENCV
AND TENSORFLOW WITH
ANACONDA
HTTPS://ELBRUNO.COM/2020/
05/12/PYTHON-EASY-WAY-TO-
INSTALL-OPENCV-AND-
TENSORFLOW-WITH-
ANACONDA/
PIXELLIB IS A LIBRARY FOR
PERFORMING SEGMENTATION
OF IMAGES
HTTPS://GITHUB.COM/AYOOLA
OLAFENWA/PIXELLIB
WINDOWS TERMINAL AND
ANACONDA. AND HOW TO
START AN ANACONDA VIRTUAL
ENVIRONMENT
HTTPS://ELBRUNO.COM/2020/
05/13/WINDOWS10-
WINDOWSTERMINAL-AND-
ANACONDA-BONUS-START-AN-
ANACONDA-VIRTUAL-
ENVIRONMENT/
FLIP WHEN A FACE IS
DETECTED
HTTPS://ELBRUNO.COM/2020/
03/18/CODING4FUN-HOW-TO-
CONTROL-YOUR-DRONE-WITH-
20-LINES-OF-CODE-12-N/
INSTANCE SEGMENTATION
HTTPS://ELBRUNO.COM/2020/
05/21/CODING4FUN-HOW-TO-
CONTROL-YOUR-DRONE-WITH-
20-LINES-OF-CODE-20-N/

28. @cloudgen_verona
#CodeGen2021