lane detection, deep learning, autonomous driving, CNN, RNN, LSTM, GRU, lane localization, lane fitting, ego lane, end-to-end, vanishing point, segmentation, FCN, regression, classification

1. Camera-Based Road Lane
Detection by Deep Learning III
Yu Huang
Yu.huang07@gmail.com
Sunnyvale, California

2. Outline
• CurveLane-NAS: Unifying Lane-Sensitive Architecture Search and Adaptive Point
Blending
• Heatmap-based Vanishing Point boosts Lane Detection
• LDNet: End-to-End Lane Detection Approach using a Dynamic Vision Sensor
• RONELD: Robust Neural Network Output Enhancement for Active Lane Detection
• Keep your Eyes on the Lane: Real-time Attention-guided Lane Detection
• End-to-end Lane Shape Prediction with Transformers
• End-to-End Deep Learning of Lane Detection and Path Prediction for Real-Time
Autonomous Driving

3. CurveLane-NAS: Unifying Lane-Sensitive
Architecture Search and Adaptive Point Blending
• Lane-sensitive architecture search framework, CurveLane-NAS;
• Three parts:
• a) feature fusion search module, for multi-level hierarchy features to
fuse local and global context;
• b) flexible backbone search module, semantic feature extractor;
• c) adaptive point blending module, search for a multi-level post
processing refinement strategy to combine multi-scale head prediction.
• CurveLanes：around 150K images, 90% for curved lanes (~ 135K
images).

4. CurveLane-NAS: Unifying Lane-Sensitive
Architecture Search and Adaptive Point Blending

5. CurveLane-NAS: Unifying Lane-Sensitive
Architecture Search and Adaptive Point Blending
(a) dense prediction based (SCNN), (b) proposal based (Line-CNN), (c) CurveLane-NAS (holistic search)

6. CurveLane-NAS: Unifying Lane-Sensitive
Architecture Search and Adaptive Point Blending
The pipeline of NAS structure for lane detection

7. CurveLane-NAS: Unifying Lane-Sensitive
Architecture Search and Adaptive Point Blending
Point Blending Search Module

8. CurveLane-NAS: Unifying Lane-Sensitive
Architecture Search and Adaptive Point Blending

9. CurveLane-NAS: Unifying Lane-Sensitive
Architecture Search and Adaptive Point Blending

10. Heatmap-based Vanishing Point boosts Lane Detection
• vanishing point as structure info;
• ERFNet for segmentation;
• Heatmap regression for VP prediction.

11. Heatmap-based Vanishing Point boosts Lane Detection

12. Heatmap-based Vanishing Point boosts Lane Detection

13. Heatmap-based Vanishing Point boosts Lane Detection

14. LDNet: End-to-End Lane Detection Approach
using a Dynamic Vision Sensor
• Lane Detection using dynamic vision sensor (LDNet)
Comparison of Normal Camera and Event Camera

15. LDNet: End-to-End Lane Detection Approach
using a Dynamic Vision Sensor

16. LDNet: End-to-End Lane Detection Approach
using a Dynamic Vision Sensor

17. LDNet: End-to-End Lane Detection Approach
using a Dynamic Vision Sensor

18. RONELD: Robust Neural Network Output
Enhancement for Active Lane Detection
• Real-time robust neural network output enhancement for active
lane detection (RONELD)

19. RONELD: Robust Neural Network Output
Enhancement for Active Lane Detection

20. RONELD: Robust Neural Network Output
Enhancement for Active Lane Detection

21. Keep your Eyes on the Lane: Real-time
Attention-guided Lane Detection
LaneATT: Anchor-based lane detection, similar to Line-CNN.

22. Keep your Eyes on the Lane: Real-time
Attention-guided Lane Detection

23. Keep your Eyes on the Lane: Real-time
Attention-guided Lane Detection

24. End-to-end Lane Shape Prediction with Transformers
• Directly estimate the lane shape model with Transformers (contextual info).

25. End-to-end Lane Shape Prediction with Transformers
The backbone: a reduced transformer network, several feed-forward networks (FFNs) for
parameter predictions, and Hungarian bipartite matching Loss.

26. End-to-end Lane Shape Prediction with Transformers
The encoder-decoder

27. End-to-end Lane Shape Prediction with Transformers
TuSimple
FVL Transfer

28. End-to-End Deep Learning of Lane Detection and
Path Prediction for Real-Time Autonomous Driving
• Three-task convolutional
neural network (3TCNN);
• Include a bounding box
regression branch, a Hu
moments regression branch
and a classification branch;
• Lateral offset-path prediction.
TORCS

29. End-to-End Deep Learning of Lane Detection and
Path Prediction for Real-Time Autonomous Driving

30. End-to-End Deep Learning of Lane Detection and
Path Prediction for Real-Time Autonomous Driving

31. Robust Lane Detection via Expanded Self Attention
• HESA (Horizontal Expanded Self Attention) and VESA (Vertical Expanded Self Attention).

32. Robust Lane Detection via Expanded Self Attention
ESA Encoder-Decoder

33. Robust Lane Detection via Expanded Self Attention

34. Robust Lane Detection via Expanded Self Attention