The document describes a learning-based algorithm for drone trail following with obstacle avoidance. The algorithm uses deep learning approaches including convolutional neural networks (CNN) and deep reinforcement learning. CNN is used for trail following to classify the drone's direction, and deep reinforcement learning with a dual deep Q network is used for obstacle avoidance. The algorithm integrates these approaches, using monocular camera images and estimated depth images to determine if an obstacle is present and either execute the trail following or obstacle avoidance module. Simulation and real-world experiments demonstrated the algorithm's ability to safely follow trails while avoiding obstacles.

1. CONTACT
Autonomous Systems Laboratory
Mechanical Engineering
5st Engineering Building Room 810
Web. https://sites.google.com/site/aslunist/
Learning-Based Algorithm
for Drone Trail following with
Obstacle Avoidance
Minjae Jung, Jinwoo Oh, Minwoo Kim and Hyondong
Oh*

2. Autonomous Systems Laboratory
Introduction
1/19
• Following the trail in unknown environments is important for autonomous missions such as rescue and
surveillance
• Obstacle avoidance while following the trail is necessary for navigating the environment safely
• Deep learning algorithm such as supervised learning and deep reinforcement learning gives UAV
the robustness and the flexibility
! !

3. Autonomous Systems Laboratory
Related works: Trail following 2/19
• Convolutional neural network(CNN) is used to recognizing the UAV’s direction on the trail and
classifying which direction that the UAV must go from the monocular camera image
• Yaw rate(rad/s) of UAV is determined by the output from CNN’s
• Dataset for training is made by human with camera
• Giusti, A., Guzzi, J., Ciresan, D. C., He, F. L., Rodríguez, J. P., Fontana, F., and Scaramuzza, D., “A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots,” IEEE Robotics and Automation
Letters, Vol. 1, No. 2, 2016, pp. 661~667.
• Smolyanskiy, N., Kamenev, A., Smith, J., and Birchfield, S., “Toward low-flying autonomous MAV trail navigation using deep neural networks for environmental awareness,” arXiv preprint, 2017, arXiv:1705.02550.

4. Autonomous Systems Laboratory
Related works: Obstacle avoidance
3/19
• Obstacle avoidance with the monocular camera using reinforcement learning selects the UAV’s linear
velocity and yaw rate as the action from the image input
• Since training should be done in simulation for a lot of trial-and-error process, the gap between the real
world and simulation is matter
• Using canonical images(estimated depth image or segmented image) from deep neural network can
overcome the Sim2Real problem
• 김민우, 김종윤, 정민재, 오현동. (2020). 단안 카메라와 심층강화학습을 이용한 소형 무인기 실내 충돌 회피 시스템. 제어로봇시스템학회 논문지, 26(6), 399-411.

5. Autonomous Systems Laboratory
Proposed Algorithm
4/19
Front
Camera
Downward
Camera
Obstacle avoidance
Depth
Estimation
Trail Following
Direction Classification
Action
Selection
Obstacle
Recognition
True
False
Linear velocity(m/s): 𝑉
Yaw rate(rad/s): ሶ𝜓

6. Autonomous Systems Laboratory
Proposed Algorithm
5/19
Front
Camera
Downward
Camera
Obstacle avoidance
Depth
Estimation
Trail Following
Direction Classification
Action
Selection
Obstacle
Recognition
True
False
Linear velocity(m/s): 𝑉
Yaw rate(rad/s): ሶ𝜓

7. Autonomous Systems Laboratory
Trail following
• Determine the directions to follow the bike trails through CNN
• Dataset : the images looking at {left of the trail, front of the trail, right of the trail, out of the trail}
• Label : {Turn right, Go straight, Turn left, Non-trail}
6/19
Dataset
Label Go Straight Turn right Turn left Non-Trail

8. Autonomous Systems Laboratory
Trail following
• Yaw rate command is determined by the difference of Turn right and Turn left output from CNN’s
• Recovery mode for non trail situation is considered
𝑇𝑙𝑒𝑓𝑡
𝑇𝑐𝑒𝑛𝑡𝑒𝑟
𝑇𝑟𝑖𝑔ℎ𝑡
ሶψ: yaw rate(rad/s), 𝑉: linear velocity(m/s)
𝑘: trail following gain
ሶ𝜓1,2,3,4: last yaw rate command before non trail
𝑣 𝑡𝑓: desired velocity(m/s)
൞
ሶψ = k 𝑇𝑙𝑒𝑓𝑡 − 𝑇𝑟𝑖𝑔ℎ𝑡 , 𝑉 = 𝑣𝑡𝑓 𝑇𝑟𝑎𝑖𝑙
ሶψ = −
ሶ𝜓1 + ሶ𝜓2 + ሶ𝜓3 + ሶ𝜓4
4
, 𝑉 = 0 𝑁𝑜𝑛 𝑇𝑟𝑎𝑖𝑙
𝑁𝑜𝑛 − 𝑇𝑟𝑎𝑖𝑙
7/19

9. Autonomous Systems Laboratory
Proposed Algorithm
8/19
Front
Camera
Downward
Camera
Obstacle avoidance
Depth
Estimation
Trail Following
Direction Classification
Action
Selection
Obstacle
Recognition
True
False
Linear velocity(m/s): 𝑉
Yaw rate(rad/s): ሶ𝜓

10. Autonomous Systems Laboratory
Depth estimation
9/19
Godard, Clément, et al. "Digging into self-supervised monocular depth estimation." Proceedings of the IEEE international conference on computer vision. 2019.
Laina, Iro, et al. "Deeper depth prediction with fully convolutional residual networks." 2016 Fourth international conference on 3D vision (3DV). IEEE, 2016.
Depth network
(Monodepth2)
Raw RGB image Estimated depth image
• Depth estimation from single RGB image can be done by Fully Convolutional Neural network(FCN)
• FCN type neural networks (Monodedpth2, FCRN, etc.) predict depth as 0 to 1 value for every pixel
• Monodepth2 algorithm was trained from KITTI dataset which is good to use in outdoor environment

11. Autonomous Systems Laboratory
Obstacle avoidance
0
1
2
3
4
5
6
7
Linear velocity(m/s) Yaw rate(rad/s)
𝑣 𝑜𝑎 0.0
𝑣 𝑜𝑎/2 0.0
0.0 0.0
Last 𝑣 𝑜𝑎 0.25
Last 𝑣 𝑜𝑎 -0.25
Last 𝑣 𝑜𝑎 0.0
Last 𝑣 𝑜𝑎 -0.1
Last 𝑣 𝑜𝑎 0.1
𝑣 𝑜𝑎: desired velocity
• Dueling double deep Q network(D3QN) algorithm is used to obstacle avoidance.
• Discretized action is converted to linear velocity and yaw rate command.
• Desired velocity is the parameter that user can determine with flexibility.
• Algorithm is trained in Gazebo simulation and tested in real world environment.
10/1
9

12. Autonomous Systems Laboratory
Proposed Algorithm
11/1
9
Front
Camera
Downward
Camera
Obstacle avoidance
Depth
Estimation
Trail Following
Direction Classification
Action
Selection
Obstacle
Recognition
True
False
Linear velocity(m/s): 𝑉
Yaw rate(rad/s): ሶ𝜓

13. Autonomous Systems Laboratory
Obstacle recognition(1/2): Construct obstacle map
Estimated depth image
Threshold: 𝛼
Pixel value: 𝑥 ∈ ℝ, (0,1)
ቊ
0, 𝑥 < 𝛼
1, 𝑥 ≥ 𝛼
𝛼: 0.5 𝛼: 0.6 𝛼: 0.7
Binary image
Depth image binarization
• Obstacle map is constructed from depth image by adjusting the
threshold parameter 𝜶
• All pixels with values less than 𝜶 on the screen will be 0
• The parameter 𝜶 determine the sensitivity of detecting the obstacle
0 1
12/19

14. Autonomous Systems Laboratory
Obstacle recognition(2/2): Area based recognition
Binary image
Threshold: 𝛽
Area ratio: y =
𝑜𝑏𝑠𝑡𝑎𝑐𝑙𝑒 𝑝𝑖𝑥𝑒𝑙𝑠
𝑇𝑜𝑡𝑎𝑙 𝑝𝑖𝑥𝑒𝑙𝑠
• Obstacle will be recognized by the ratio of the obstacle’s area that occupying the obstacle map
• If the ratio of the number of obstacle pixel in the entire scene is over the threshold 𝜷, the obstacle will be
recognized and will be ignored otherwise
• The result of obstacle recognition is used to execute trail following module or obstacle avoidance
module
Obstacle recognition 640x480
16400
Area ratio:
16400
640∗480
≈ 0.053
Threshold: 𝛽 = 0.05
ቊ
𝑜𝑏𝑠𝑡𝑎𝑐𝑙𝑒, 𝑦 ≥ 𝛽
𝑛𝑜 𝑜𝑏𝑠𝑡𝑎𝑐𝑙𝑒, 𝑦 < 𝛽
Obstacle detected
True
Example
13/19

15. Autonomous Systems Laboratory
Proposed Algorithm
14/1
9
Front
Camera
Downward
Camera
Obstacle avoidance
Depth
Estimation
Trail Following
Direction Classification
Action
Selection
Obstacle
Recognition
True
False
Linear velocity(m/s): 𝑉
Yaw rate(rad/s): ሶ𝜓Obstacle detected: True
640x480
640x480
640x190
640x480
640x480

16. Autonomous Systems Laboratory
Simulation
Trail following Obstacle avoidance Proposed algorithm
• Px4 SITL(Software in the loop)is used for simulation
• Gazebo is used for simulator with ROS
• We used Iris quadrotor model and two monocular cameras for trail
following and obstacle avoidance
• Proposed algorithm in the complex environment was done successfully
in the simulation
15/19
Obstacle
Avoidance
Trail
Following

17. Autonomous Systems Laboratory
Experiment setup: UAV
Frame Tarot Ironman 680 Pro
Flight Controller Pixhawk 4 with GPS
Firmware PX4 1.11.0
Motor
P5010(300KV) Brushless Motor
x 6
ESC Xrotor Pro 40A ESC x 6
Feature HD3000 Widecam F100
Max Digital Video
Resolution
1280 x 720 1920 x 1080
Type CMOS 12MP CMOS
Horizontal FOV about 53(deg) 120(deg)
Computer Interface USB 2.0 USB 2.0
Features
NVIDIA
Jetson Xavier
CPU
8-core NVIDIA Carmel 64-bit ARMv8.2 @
2265MHz
GPU 512-Core NVIDIA Volta 1377MhZ
RAM
16GB 256-bit LPDDR4x @ 2133MHz |
137GB/s
Power Consumption
(only processor)
10W / 15W / 30W profiles
16/19

18. Autonomous Systems Laboratory
Experiment Video
Experiment parameters
Linear velocity 0.35 (m/s)
Altitude 1.5 (m)
Threshold 𝛼 0.55
Threshold 𝛽 0.05
17/19

19. Autonomous Systems Laboratory
Experiment Result
18/19

20. Autonomous Systems Laboratory
Conclusion
• We proposed an integrated algorithm for trail following with obstacle avoidance
• Each algorithm is based on deep learning approaches with monocular camera images as an
input
• Detect the obstacle with estimated depth image and determine what algorithm to execute
• Real experiment was done by on board processing
In this presentation,
Future work
• Train trail following algorithm for more general environments
• Consider more flexible approach to detect obstacles
• Add object (ex. human) tracking algorithm with trail following
19/19

21. THANK YOU