“Developing an Efficient Automotive Augmented Reality Solution Using Teacher-student Learning and Sprints,” a Presentation from STRADVISION
•
0 likes•18 views
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/07/developing-an-efficient-automotive-augmented-reality-solution-using-teacher-student-learning-and-sprints-a-presentation-from-stradvision/ Jack Sim, CTO of STRADVISION, presents the “Developing an Efficient Automotive Augmented Reality Solution Using Teacher-student Learning and Sprints” tutorial at the May 2023 Embedded Vision Summit. ImmersiView is a deep learning–based augmented reality solution for automotive safety. It uses a head-up display to draw a driver’s attention to important objects. The development of such solutions often resembles research more than development, with long development cycles and unpredictable accuracy and inference speed advances. In this talk, Sim presents an efficient development process for projects involving multiple deep learning tasks. This process decouples task dependencies through teacher-student learning and concurrently improves accuracy and speed via sprints. In each sprint, STRADVISION trains teacher networks for each task, focusing only on improving accuracy. In the same sprint, a unified student network learns all tasks from the most accurate teacher networks. To optimize accuracy and speed, STRADVISION applies neural architecture search to the student network in the initial sprints and then fixes the architecture. This development process enabled STRADVISION to create the ImmersiView prototype in three months, followed by monthly releases.
Recommended
Recommended
More Related Content
Similar to “Developing an Efficient Automotive Augmented Reality Solution Using Teacher-student Learning and Sprints,” a Presentation from STRADVISION
Similar to “Developing an Efficient Automotive Augmented Reality Solution Using Teacher-student Learning and Sprints,” a Presentation from STRADVISION (20)
More from Edge AI and Vision Alliance
More from Edge AI and Vision Alliance (20)
Recently uploaded
Recently uploaded (20)
“Developing an Efficient Automotive Augmented Reality Solution Using Teacher-student Learning and Sprints,” a Presentation from STRADVISION
- 1. © 2023 STRADVISION 1 Developing an Efficient Automotive Augmented Reality Solution Using Teacher-Student Learning and Sprints Jack Sim CTO STRADVISION
- 2. © 2023 STRADVISION 2 What We Do Vison perception for active safety driving features Sensors Sense Plan Act SVNet is part of the perception stack in ADAS, which provides key information for next stages such as planning and control
- 3. © 2023 STRADVISION 3 SVNet, Cutting Edge AI Technology Industry Recognized Network 470+ patents with deep neural network ASPICE2) certification Maturity Multiple commercial projects with automotive OEMs for mass- production of vehicle models High Efficiency Light weight and compact DNN1) Algo Less computing resource required Strong Performance Expand and refine target algorithm through the state-of- the-art learning techniques Deep learning-based vision perception algorithm + own know-how in automotive applications DNN1) Deep Neural Network ASPICE2) Automotive Software Process Improvement and Capability dEtermination
- 4. © 2023 STRADVISION 4 Product Strategy Vision Perception Software - SVNet SoCs Deep Learning Technologies Development Tool / Framework ADAS / AD1) Application Active Safety Functions Semi-autonomous Driving Parking Autonomous Vehicle Mono Front Vision … 360-degree Vision Multi-channel Cameras Surround Vision Front Vision for Augmented Reality Supervised Learning Semi-supervised Learning Unsupervised Learning CUSTOMER … AI Toolkits – Data selection, data labeling, AI algorithm training, platform implementation and SW evaluation Self-learning AI Platform Infotainment 2023 PARTNER 1) ADAS: Advanced Driver Assistance Systems / AD: Autonomous Driving Versatile software for flexible and customized applications in the automotive industry
- 5. © 2023 STRADVISION 5 EntryVision BasicVision CoreVision UltraVision BasicAgent AdvancedAgent ValetAgent 1.ProDriver 2.ParkAgent 3.ImmersiView 4.CompliKit SVNet Product Lineup
- 6. © 2023 STRADVISION 6 Product Maturity – Design Wins with 13 OEMs 13 million vehicles in 50+ models on their way to the road with SVNet1) software embedded Infotainment ADAS3) (Level 2&3) AV2) (Level 4) Augmented Reality Navigation SCHEDULED SOP SVNet1): STRADVISION Deep Learning Network AV2): Autonomous vehicles ADAS3): Advanced Drivers’ Assistance Systems SOP YEAR 2019 2020 2021 2022 2023~
- 7. © 2023 STRADVISION 7 7 © 2023 STRADVISION Key Techniques of SVNet
- 8. © 2023 STRADVISION 8 Release Model Model Deployment SW Integration QA Testing Deep Learning Projects Prep Data Data Acquisition Data Sampling Data Labeling Train Data Logits Loss Training Train Model Test Model Test Data Logits Accuracy Training SV SPRINT MODEL
- 9. © 2023 STRADVISION 9 Challenges in Deep Learning Projects • Long development cycles • Unpredictable outcomes in each cycle (e.g., accuracy and speed) Inference speed Accuracy KPI 1 KPI 2 SV SPRINT 01 SV SPRINT 02 SV SPRINT 03
- 10. © 2023 STRADVISION 10 Even More Challenges: Multi-Task DL Projects Accuracy 01 KPI 1 • Looooonger development cycles • Harder to meet KPIs in limited time KPI 2 KPI 3 SV SPRINT 01 SV SPRINT 02 SV SPRINT 03 GOAL
- 11. © 2023 STRADVISION 11 Naïve Approach to the Multi-Task DL Projects • Train individual networks on separate datasets • Shorten development cycles • Causing resource allocation issues • Inference speed in Task 1 vs. Task 2 SP 01 SP 02 SP 03 Inference speed Accuracy KPI 1 SP 01 SP 03 SP 02 Inference speed Accuracy KPI 2 Latency Sprint KPI 3 SP 01 SP 02 SP 03
- 12. © 2023 STRADVISION 12 12 © 2023 STRADVISION Proposed Development Process
- 13. © 2023 STRADVISION 13 Multiple Teacher Networks Proposed Development Process Labeled Data Task1 Logits Loss Teacher Net – Task 01 Multiple Teacher - Unified Student learning Unlabeled Data Task1 Logits Task2 Logits Task3 Logits Distillation Loss Soft Labels Unified Student Networks Task1 Logits Task2 Logits Task3 Logits Labeled Data Task1 Logits Loss Teacher Net – Task 02 Labeled Data Task1 Logits Loss Teacher Net – Task 03 Updated during training Fixed during training
- 14. © 2023 STRADVISION 14 Multi-Task DL (Video section)
- 15. © 2023 STRADVISION 15 Teacher-Student Learning Semi-supervised learning Teacher Networks Unlabeled Data Logits Distillation Loss Soft Labels Student Networks Logits Labeled Data Loss Better teachers, better students.
- 16. © 2023 STRADVISION 16 Decoupling Dependencies Between Tasks Between Sprints Between KPIs For teacher networks, focus on improving accuracy only. Multi-Task Labeled Data Task 01 Logits Task 02 Logits Task 03 Logits Loss Task 01 Labeled Data Task 01 Logits Loss Task 02 Labeled Data Task 02 Logits Loss Task 03 Labeled Data Task 03 Logits Loss Sprint 01 Sprint 02 Sprint 03 Tea 01 Tea 02 Tea 03 Tea 01 Tea 02 Tea 03 Student F SP 01 SP 02 SP 03 Inference speed Accuracy KPI 1 F
- 17. © 2023 STRADVISION 17 Benefits of Using Teacher-Student Learning • Can develop independently of other networks. • Focus solely on improving accuracy. • Keep only the most accurate models. • Not dependent on target hardware. • Can reuse for other projects. • Can use off-the-shelf DNNs for the initial phase. • Use large-scale unlabeled datasets. • Can use partially labeled datasets. • Improve accuracy for free with more accurate teacher networks. Teacher network Student network
- 18. © 2023 STRADVISION 18 Teacher Network Training 1. Feed more data into deeper and wider models. 2. Apply pre-training if needed. 3.Apply data augmentation if needed. To improve accuracy
- 19. © 2023 STRADVISION 19 Unified Student Network Training To meet KPIs for inference speed 1. Design your network. 2. Measure its inference speed before you start training. 3. Start training once the speed KPI is met. To improve accuracy 1. Use more accurate teacher networks. 2. Use better losses, leveraging cross-task relationships. To accelerate training 1. Precompute and store soft labels to SSD. 2. Load the labels from SSD during training.
- 20. © 2023 STRADVISION 20 Benefits of The Proposed Process Pipelined w/T-S learning Network Type Target HW KPIs Task Teacher Cloud Accuracy A B C Student Embedded Speed Accuracy A,B,C - Embedded Speed Accuracy A,B,C Time Period 01 Time Period 02 Time Period 03 Time Period 04 Time Period 05 Time Period 06 Tea A 01 Tea A 02 Tea A 03 Tea A 04 Tea A 05 Tea A 06 Tea B 01 Tea B 02 Tea B 03 Tea B 04 Tea B 05 Tea B 06 Tea C 01 Tea C 02 Tea C 03 Tea C 04 Tea C 05 Tea C 06 - Student 01 Student 02 Student 03 Student 04 Student 05 Net 01 Net 02 Non-pipelined w/o T-S learning Model Release Rapid prototyping & Continuous updating
- 21. © 2023 STRADVISION 21 Conclusions • Efficient SW development process for multiple DL tasks • using teacher-student learning. • Concurrently improves accuracy and speed through sprints. • Reduced product development time by up to 50%.
- 22. © 2023 STRADVISION 22 22 © 2023 STRADVISION STRADVISION
- 23. © 2023 STRADVISION 23 STRADVISION Linked In @stradvision Youtube @stradvision Webpage https://stradvision.com
- 24. © 2023 STRADVISION 24 References 24 [1] Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. "Distilling the knowledge in a neural network." arXiv preprint, 2015. https://arxiv.org/abs/1503.02531 [2] Gou, Jianping, et al. "Knowledge distillation: A survey." IJCV, 2021. https://arxiv.org/abs/2006.05525 [3] Wang, Lin, and Kuk-Jin Yoon. "Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks." IEEE TPAMI, 2021. https://arxiv.org/abs/2004.05937