“Flexible Machine Learning Solutions with Lattice FPGAs,” a Presentation from Lattice Semiconductor
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2021/08/flexible-machine-learning-solutions-with-lattice-fpgas-a-presentation-from-lattice-semiconductor/ Sreepada Hegade, Senior Manager for ML Software and Solutions at Lattice Semiconductor, presents the “Flexible Machine Learning Solutions with Lattice FPGAs” tutorial at the May 2021 Embedded Vision Summit. The ability to perform neural network inference in resource-constrained devices is fueling the growth of machine learning at the edge. But application solutions require more than just inference—they also incorporate aggregation and pre-processing of input data, and post-processing of inference results. In addition, new neural network topologies are emerging rapidly. This diversity of functionality and quick evolution of topologies means that processing engines must have the flexibility to execute different types of workloads. I/O flexibility is also key, to enable system developers to choose the best sensor and connectivity options for their applications. In this talk, Hegade explores how the configurable nature of Lattice FPGAs and the soft cores implemented on them allow for quick adoption of emerging neural network topologies, efficient execution of pre- and post-processing functions, and flexible I/O interfacing. He also shows how his company optimizes network topologies and its compiler to get the best out of FPGAs.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to “Flexible Machine Learning Solutions with Lattice FPGAs,” a Presentation from Lattice Semiconductor
Similar to “Flexible Machine Learning Solutions with Lattice FPGAs,” a Presentation from Lattice Semiconductor (20)
More from Edge AI and Vision Alliance
More from Edge AI and Vision Alliance (20)
Recently uploaded
Recently uploaded (20)
“Flexible Machine Learning Solutions with Lattice FPGAs,” a Presentation from Lattice Semiconductor
- 1. © 2021 Lattice Semiconductor Sreepada V Hegade Sr. Manager – ML Software and Solutions Flexible ML Solution with Lattice FPGA
- 2. © 2021 Lattice Semiconductor • Review Need for Flexibility in ML • Flexible Hardware and Engines • Flexibility with Cost • Discussion on Trade-Off • Framework to Enable Flexibility • Conclusion 2 Agenda
- 3. © 2021 Lattice Semiconductor Flexibility At Every Step • Interface Flexibility • Different Type of Sensors interface with required IPs to process sensor data • Implementation Flexibility • Proven library of networks, easy to use workflow • Flexibility with Cost, Performance and Power • Scalable solution 3 Sensor Data Aggregation and Augmentation To Inference Post Processing Inference with Optimal Network and Engine Configuration 1 2 3
- 4. © 2021 Lattice Semiconductor 4 HARDWARE PLATFORMS IP CORES SOFTWARE TOOLS REFERENCE DESIGNS / DEMOS CNN Compact Accelerator CNN Accelerator UPduino Himax Shield – iCE40 UltraPlus FPGA ECP5 EVDK – ECP5 FPGA CUSTOM DESIGN SERVICES Smart Car Smart Home Smart City Smart Factory Neural Network Compiler Neural Network Accelerators Face Detection Speed Sign Detection Key Phrase Detection Face Tracking Object Counting Human Presence Detection Hand Gesture Detection 1 mW, 5.5 mm2 1 W, 100 mm2 CNN Plus Accelerator CrossLink-NX EVDK – CrossLink-NX 50-200 mW 16 mm2 Our Solution
- 5. © 2021 Lattice Semiconductor 5 5mm2 16mm2 Flexible Interface
- 6. © 2021 Lattice Semiconductor 6 • High Performance at low power • Configurable Engine (on-chip memory, number of cores) • Can support up to 60 fps QVGA and 30fps VGA resolution image • Supports TensorFlow on bare metal RISC-V + coprocessor • Good for background inference jobs like predictive maintenance • Shares computation resources AHBL Interconnect RISC-V CPU I$ D$ Data Ram UART AHBL2APB Bridge PDM Accelerator FIFO DMA EtherControl (Main) QSPI Memory Control + prefetch buffer M S0 M M S S S FIFO Ethernet IRQ0 IRQ2 S M IRQ1 APB Interconnect S GPIO ISR Ram S S1 Configurable ML Engines
- 7. © 2021 Lattice Semiconductor 7 38FPS 320x240 48FPS 224x224 64FPS 160x160 48FPS 320x240 60FPS 224x224 72FPS 160x160 0 10 20 30 40 50 60 70 80 0 50 100 150 200 250 300 Frames Per Sec On-Chip Memory (128KB -> 256KB) 32FPS 224x224 (VGG) 60FPS 224x224 (Mobile Net) 48FPS 224x224 (VGG) 0 10 20 30 40 50 60 70 0 20 40 60 80 100 Frames Per Sec DSP Blocks (48 -> 84 MUL9 Units) Solution Cost Flexibility
- 8. © 2021 Lattice Semiconductor 8 38FPS 320x240 (QVGA) 48 FP 320x240 (QVGA) 0 10 20 30 40 50 60 0 50 100 150 200 250 300 350 Frames Per Sec Runtime Size with Parameters (KB) ~20% performance with 2% Accuracy loss Trade-Off With Optimization
- 9. © 2021 Lattice Semiconductor 9 Model Zoo Data Capture / Label Model Selection Transfer Learning Compile Runtime (inference) Model Visualize (Dashboard) Optimize Train Confi gure Evaluate Framework to Enable Flexibility
- 10. © 2021 Lattice Semiconductor 10 Framework Demo
- 11. © 2021 Lattice Semiconductor 11 • Flexibility is NOT just about operations supported by engine ▪ It is required at every step • Finding right trade-off is key to success • Performance and Power optimized engine with trade-off options • Lattice solution provides flexibility and framework to achieve right trade-off • Many possibilities, One framework Conclusion