More Related Content
What's hot
What's hot (18)
Similar to Autonomous driving function on driving video
Similar to Autonomous driving function on driving video (20)
Recently uploaded
Recently uploaded (20)
Autonomous driving function on driving video
- 1. Autonomous driving function On driving video
- 2. 1 Outline 2 Feature 3 Result
- 3. Outline
- 5. 5 01 Outline Concept Maximizing Autonomous driving function On driving video Using computer vision & deep learning
- 6. 6 01 Outline Detail Features 1 Lane 2 Vehicle 3 Traffic Light
- 7. 7 01 Outline Detail Features Traffic light Detection Lane Detection Vehicle Detection Open CV Image Undistortion Image Binarization Perspective Transform & ROI Measuring & Projecting
- 8. 8 01 Outline Detail Features Traffic light Detection Lane Detection Vehicle Detection Tensorflow pretrained model ssd_mobilenet ssd_inception v2 faster_rcnn_resnet50 faster_rcnn_resnet101 faster_rcnn_inception v2 rfcn_resnet50
- 9. 9 01 Outline Detail Features Traffic light Detection Lane Detection Vehicle Detection Transfer Learning Data collection Re - training Exporting
- 11. Feature
- 12. 12 02 Feature 1 Lane Detection Undistortion - Compute the camera calibration matrix and distortion coefficients given a set of chessboard images - Apply a distortion correction to raw images
- 13. 13 02 Feature 1 Lane Detection Binarization - A color transformation to HSL and select S(saturation) channel - Apply Sobel X filter
- 14. 14 02 Feature 1 Lane Detection Wrapping - Apply a perspective transform to rectify binary image ("birds-eye view") - Apply ROI (Region of Interest)
- 15. 15 02 Feature 1 Lane Detection Measuring & Projecting - Determine curvature of the lane (Use Radius of Curvature) - Warping the detected lane boundaries back onto the original image - Output visual display of the lane boundaries
- 16. 16 02 Feature 2 Vehicle Detection Using Tensorflow Object detection API Model zoo Model name Speed ( ms ) Acc (mAP) Model size Ssd_mobilenet_v2 31 22 201m Ssd_inception_v2 42 24 295m Faster_rcnn_inception_v2 58 28 167m Faster_rcnn_resnet50 89 30 405m Faster_rcnn_resnet101 106 32 624m Rfcn_resnet101 92 30 685m
- 17. 17 02 Feature 2 Vehicle Detection Coco dataset Instance person bicycle car motorcycle airplane bus train truck boat trafficlight firehydrant stopsign parkingmeter bench bird cat dog horse sheep cow elephant bear zebra giraffe backpack umbrella handbag tie suitcase frisbee skis snowboard sportsball kite baseballbat baseballglove skateboard surfboard tennisracket bottle wineglass cup fork knife spoon bowl banana apple sandwich orange broccoli carrot hotdog pizza donut cake chair couch pottedplant bed diningtable toilet tv laptop mouse remote keyboard cellphone microwave oven toaster sink refrigerator book clock vase scissors teddybear hairdrier toothbrush person bicycle car motorcycle airplane bus train truck boat trafficlight firehydrant stopsign parkingmeter bench bird cat dog
- 18. 18 02 Feature 2 Vehicle Detection result Ssd_mobilenet_v2 Ssd_inception_v2 Faster_rcnn_inception_v2 Faster_rcnn_resnet50 Faster_rcnn_resnet101 Rfcn_resnet101 * Prediction Environment : gtx 1080
- 19. 19 * Prediction Environment : gtx 1080 02 Feature 2 Vehicle Detection result Model name frame / 1 sec sec / 1 frame Ssd_mobilenet_v2 20.4675 0.0701 Ssd_inception_v2 14.2496 0.0488 Faster_rcnn_inception_v2 12.2945 0.0813 Faster_rcnn_resnet50 10.0331 0.0996 Faster_rcnn_resnet101 8.0481 0.1242 Rfcn_resnet101 8.4848 0.1178
- 20. 20 02 Feature 3 Light Traffic Detection • Video source - Blackbox video : Albatross gold 3 ( 1920 * 1080 ) bundang - suwon - Youtube video : 1280 * 720 busan , pohang , jeju , masan Data Collecting • Class - Red , Green , RedLeft , GreenLeft • Restrictions - Sunny - Day
- 21. 21 02 Feature 3 Light Traffic Detection • Dataset count Data Collecting Result Class Train Validate red 458 51 green 600 67 redleft 92 11 greenleft 98 11
- 22. 22 02 Feature 3 Light Traffic Detection Model name Training step Training time Model size Ssd_mobilenet_v2 100,000 21 hour 40min 201m Ssd_inception_v2 200,000 25 hour 295m Faster_rcnn_inception_v2 200,000 7 hour 40min 167m Faster_rcnn_resnet50 200,000 12 hour 40min 405m Faster_rcnn_resnet101 200,000 20 hour 624m Rfcn_resnet101 200,000 18 hour 5min 685m * Prediction Environment : gtx 1080 Transfer learning
- 23. 23 02 Feature 3 Light Traffic Detection result Transfer learning result Ssd_mobilenet_v2 Ssd_inception_v2 Faster_rcnn_inception_v2 Faster_rcnn_resnet50 Faster_rcnn_resnet101 Rfcn_resnet101
- 24. 24 02 Feature 3 Light Traffic Detection result Prediction time Model name Model size frame / 1 sec sec / 1 frame Ssd_mobilenet_v2 201m 34.4649 0.0290 Ssd_inception_v2 295m 30.4438 0.0328 Faster_rcnn_inception_v2 167m 16.8890 0.0592 Faster_rcnn_resnet50 405m 8.3088 0.1203 Faster_rcnn_resnet101 624m 7.1389 0.1400 Rfcn_resnet101 685m 12.0749 0.0828 * Precdiction Environment : gtx 1080
- 25. 25 02 Feature 4 Process pipeline Input Video Vehicle Detection Total frame count iteration Frame image read Lane Detection Traffic light Detection Image merge & write output Output Video
- 26. 26 03 Result • Format : mp4 • Resolution : 1280 * 720 ( HD ) • Total frame : 1217 • Frame / sec : 16.86 • Data size : 17.2 MB • Data length : 1 min 12 sec 0 Test video summary
- 27. 27 03 Result 1 Prediction summary Model Prediction time frame / 1 sec sec / 1 frame Ssd_mobilenet_v2 10 min 2.025 0.49 Ssd_inception_v2 10 min 40 sec 1.905 0.52 Faster_rcnn_inception_v2 11 min 40 sec 1.737 0.57 Faster_rcnn_resnet50 13 min 7 sec 1.545 0.64 Faster_rcnn_resnet101 14 min 1.447 0.69 Rfcn_resnet101 12 min 50 sec 1.588 0.62 * Precdiction Environment : gtx 1080 * Model : Vehicle + Traffic light ( transfer learning )
- 28. 28 03 Result 2 Prediction video Ssd_mobilenet_v2 Ssd_inception_v2 Faster_rcnn_inception_v2 Faster_rcnn_resnet50 Faster_rcnn_resnet101 Rfcn_resnet101
- 29. 29 03 Result 3 Our Best output! Faster rcnn resnet101 coco
- 30. Thank you!!