Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

"Machine Learning at the Edge in Smart Factories Using TI Sitara Processors," a Presentation from Texas Instruments

40 views

Published on

For the full video of this presentation, please visit:
https://www.embedded-vision.com/platinum-members/texas-instruments/embedded-vision-training/videos/pages/may-2019-embedded-vision-summit

For more information about embedded vision, please visit:
http://www.embedded-vision.com

Manisha Agrawal, Software Applications Engineer at Texas Instruments, presents the "Machine Learning at the Edge in Smart Factories Using TI Sitara Processors" tutorial at the May 2019 Embedded Vision Summit.

Whether it’s called “Industry 4.0,” “industrial internet of things” (IIOT) or “smart factories,” a fundamental shift is underway in manufacturing: factories are becoming smarter. This is enabled by networks of connected devices forming systems that collect, monitor, exchange and analyze data. Machine learning, including deep neural network algorithms such as convolution neural networks (CNNs), are enabling smart robots and machines to autonomously complete tasks with precision, accuracy and speed.

The Texas Instruments (TI) Sitara line of processors is helping to enable vision-based deep learning inference at the edge in factory automation products. TI’s AM57x class processors with specialized neural network accelerators and integrated industrial peripherals provide the processing and connectivity needed to enable smart factory vision applications to reduce production costs, improve quality and create safer work environments. This presentation covers TI’s deep learning solution on AM57x processors for smart factories.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

"Machine Learning at the Edge in Smart Factories Using TI Sitara Processors," a Presentation from Texas Instruments

  1. 1. © 2019 Texas Instruments Inc Machine Learning at the Edge in Smart Factories Using TI Sitara Processors Manisha Agrawal Texas Instruments Incorporated May 2019
  2. 2. © 2019 Texas Instruments Inc Agenda • Smart factory overview • Sitara processors for smart factories • Sitara machine learning overview • Getting started with Sitara machine learning • Third-party ecosystem 2
  3. 3. © 2019 Texas Instruments Inc Smart factory overview
  4. 4. © 2019 Texas Instruments Inc Industry 4.0 / IIOT / Smart Factory 4 Smart factory overview Industry 4.0, the Industrial Internet of Things (IIoT) or Smart Factory are all terms describing a network of devices connected via communications technologies to form systems that monitor, collect, exchange and analyze data. Based on four design principles – Interconnection: Machines, devices, sensors, and people to connect and communicate with each other via the internet of things (IoT) or the Internet of People (IoP). Information transparency: the systems create a virtual copy of the physical world through sensor data in order to contextualize information Technical assistance: Ability of the systems to support humans in making decisions and solving problems and the ability to assist humans with tasks that are too difficult or unsafe for humans. Decentralized decisions: The ability of cyber-physical systems to make simple decisions on their own and become as autonomous as possible
  5. 5. © 2019 Texas Instruments Inc Industry 4.0 / IIOT / Smart Factory Key technology considerations for smart factories: • Industrial communication protocols • Real time control of cyber physical system • Security and functional safety requirements • Machine learning • Enhanced reliability • Device longevity 5 Smart factory overview Industry 4.0, the Industrial Internet of Things (IIoT) or Smart Factory are all terms describing a network of devices connected via communications technologies to form systems that monitor, collect, exchange and analyze data.
  6. 6. © 2019 Texas Instruments Inc Sitara processors for smart factories
  7. 7. © 2019 Texas Instruments Inc Sitara processors enabling smart factories 7 • Industrial robots for automated sorting • Vision computer & optical Inspection • Automated Guided Vehicles (AGVs) • Smart HMI (teaching operators) • Optimize equipment settings • Predictive Maintenance (PdM) • Classify and recognize specific sounds/audio patterns • Track/identify/count people and objects • Extracting useful information gathered from the data aggregator Sitara processors for smart factories
  8. 8. © 2019 Texas Instruments Inc Sitara processors: Key differentiation for smart factories • Support for multiple industrial communication protocols on one platform • Real-time control and processing with incorporated PRU-ICSS and C66x DSP • Accelerated machine learning and multimedia capabilities with integrated hardware accelerators • Advanced integration for enhanced functionality • Enables machine learning inference at the edge for many vision applications • Predictive Maintenance at the edge at the field, controller or operator level devices • CSI-2 and parallel interface for camera connection • Support for PCIe, USB3, SATA, etc. connectivity standards 8 Sitara processors for smart factories Highly integrated processors supporting industrial peripherals and connectivity needed for smart factories products Highly integrated processors for smart factory applications
  9. 9. © 2019 Texas Instruments Inc Sitara Processors: Key differentiation for smart factories • Enhanced Reliability • Extended temperature & high voltage I/O • Low Failure in Time (FIT) • High POH (Power On Hours) • Enabling lower system power • Low Latency • Scalable family of devices • Software scalability across the entire portfolio ‒ Reuse software across platforms with TI’s processor SDK • TI.com support • Technical documentation, including reference designs, on machine learning with Sitara processors • Technical support by subject matter experts on e2e.ti.com 9 Sitara processors for smart factories
  10. 10. © 2019 Texas Instruments Inc Sitara Arm processor overview 10 Sitara processors for smart factories Layout Compatible AM43x •1x Cortex-A9, 1 GHz •Secure Boot, Tamper, QSPI •Camera, 2x PRU-ICSS, GBE, 3D •17x17mm AM335x •1x Cortex-A8, 1 GHz •1x PRU-ICSS, 3D, GbE Switch •DDR3, CAN •13x13, 15x15mm AM572x • 2x Cortex- A15, 1.5 GHz • 2x C66x DSP, 750 MHz • 4x Machine Learning Accelerators • 2x 3D, 4x M4, 2x PRU-ICSS • 23x23mm HD AM571x • 1x Cortex-A15, 1.5 GHz • 1x C66x DSP, 750 MHz • 3D, 4x M4, 2x PRU-ICSS • 23x23mm HD AM574x • 2x Cortex-A15, 1.5 GHz • 2x C66x DSP, 750 MHz • 2x Machine Learning Accelerators • 2x 3D, 4x M4, 2x PRU-ICSS • 23x23mm, ECC HD AM570x •1x Cortex-A15, 1.0 GHz •1x C66x DSP, 750 MHz •3D, 4x M4, 2x PRU-ICSS •17x17mm HD Performance&Integration AM654x • 4x A53, up to 1.1 GHz • 2x R5F MCU/Safety Processor • 32-bit DDR3/4, PCIe, USB3, 3D • 3x PRU-ICSS (Gb) , ECC • 23x23mm • Highly integrated solution • Brings Deep Learning to the edge • Dedicated machine learning accelerators (Embedded Vision Engine) on AM5729 and AM5749 device for high performance and low power • AM5729 is the highest performing device • Up to 120 GOPS
  11. 11. © 2019 Texas Instruments Inc Sitara machine learning overview
  12. 12. © 2019 Texas Instruments Inc Sitara machine learning 12 Sitara machine learning Sitara machine learning solutions bring machine learning to the edge by enabling deep learning inference on all Sitara devices. Sitara machine learning solutions consists of: • TI Deep Learning (TIDL) software framework • Available on AM57x device only, runs on C66x DSP cores and/or on Embedded Vision Engine (EVE) subsystems. • ArmNN: Runs on all Sitara devices.
  13. 13. © 2019 Texas Instruments Inc TI Deep Learning (TIDL) software framework on AM57x 13 Sitara machine learning • TIDL software framework consists of: – Converter tool to convert TensorFlow, Caffe and ONNX formats to TIDL network format – Quantize weight with 4-12 bit accuracy – Dynamic quantization of activation layer outputs with 8-bit accuracy – Accumulation is done with 40-bit accuracy – C++ based TIDL API on Arm/Linux – TIDL Library to offload processing on C66x DSP cores and EVE subsystem • Ease of use – Easily integrate TIDL APIs into other frameworks such as OpenCV – Provides a common host abstraction for user applications across multiple compute engines (EVEs and C66x DSPs) TIDL software framework enables machine learning inference on Sitara AM57x processors by leveraging integrated EVE and/or C66x DSP hardware accelerators to run CNN models specifically. Format Conversion (desktop / Arm Linux) Inference – embedded deployment (Sitara AM57x) TIDL-InferenceTIDL-translator TIDL Import TIDL API OpenCL TIDL Library
  14. 14. © 2019 Texas Instruments Inc TI Deep Learning (TIDL) software framework • Provides several TI network models: • JacintoNet11 for image classification • JDetNet for classification and detection • JSeg21 for pixel level semantic segmentation • TI network models leverages the following for embedded performance optimization: • Complexity reductions • Sparse convolutions • fixed point dynamic quantization • These tools result in minor decrease in accuracy but result in major computational and bandwidth reductions 14 Sitara machine learning Classification + Localization Object detection + classification Semantic Segmentation
  15. 15. © 2019 Texas Instruments Inc Performance of validated network models on AM57x 15 Sitara machine learning Network Model Type of Model ROI size Performance in ROIs/Sec AM5729 (2x Arm A15 @1.5GHz, 4x EVE @650MHz, 2x C66x DSP @750MHz) AM5749 (2x Arm A15 @1.5GHz, 2x EVE @650MHz, 2x C66x DSP @750MHz) AM572x/AM574x (2x Arm A15@1.5GHz, 2x C66x DSP @750MHz) MobileNet_v1 Image Classification 224x224 7.3 5.4 2.6 1.6 SqueezeNet_v1 Image Classification 227x227 19.7 9.6 4.6 5.2 InceptionNet_v1 Image Classification 224x224 6.3 3.2 1.4 1.6 JacinctoNet11 Embedded Image Classification 224x224 64.6 33.4 16.6 NA JSeg21 Embedded Semantic Segmentation 1024x512 11.8 3.4 1.8 NA JDetNet Embedded Object detection 768x320 22.6 11.2 NA NA *A15 is available for a real-time control loop or communication protocol or run additional NN models Utilizing 4xEVE+1xDSP* Utilizing 2xEVE+1xDSP* Utilizing DSP* Utilizing ARM** • Power efficient embedded solution at the edge, < 5W **Arm benchmarking done using Arm NN.
  16. 16. © 2019 Texas Instruments Inc Getting started with Sitara machine learning
  17. 17. © 2019 Texas Instruments Inc beagleboard.zorg/ai Getting started with Sitara machine learning Features: 4 EVEs with built-in AM5729 processor in addition to the following development support: ⚫ Zero-download training workshops ⚫ Tool for everyday automation ⚫ Definitive AM5279 reference design 17 Available from 3Q, 2019 BeagleBone AI (AM5729 SoC)TMDSIDK574 AM574x Industrial Development Kit (IDK) Board Features: 2 EVEs, 4 Ethernet ports with concurrent operation including 2 from PRU-ICSS DDR3 with ECC Profibus connection, EtherCAT and RS485 Headers On- board eMMC Mini PCIe, USB3, and HDMI connectors http://www.ti.com/tool/SITARA-MACHINE-LEARNING
  18. 18. © 2019 Texas Instruments Inc Third-party ecosystem
  19. 19. © 2019 Texas Instruments Inc Third-party ecosystem • Cross Compass: Cross Compass has a strong reputation in the manufacturing industry, provide AI consulting services, and develops and provides algorithms and neural networks. • Path Partner: Provides services for development, optimization and integration of AI solutions on TI platforms • D3 Engineering: Provides engineering services, software development and platforms on AI solutions with TI technology. • Ittiam Systems: Provides services for high- performance computer vision, machine learning and analytics 19 www.cross-compass.com www.d3engineering.com www.pathpartnertech.com www.ittiam.com
  20. 20. © 2019 Texas Instruments Inc Resources Sitara Arm processor overview http://www.ti.com/processors/sitara -arm/overview.html Sitara machine learning http://www.ti.com/tool/SITARA- MACHINE-LEARNING Software development Kit http://software-dl.ti.com/processor-sdk- linux/esd/AM57X/latest/index_FDS.html TI Design on TI Deep Learning http://www.ti.com/tool/TIDEP-01004 TI Deep Learning document http://software-dl.ti.com/processor- sdk- linux/esd/docs/latest/linux/Foundati onal_Components_TIDL.html Embedded Vision Summit “Sitara Machine learning for smart factories” (Wednesday, May 22 from 11:20 – 11:50 am) 20
  21. 21. © 2019 Texas Instruments Inc Backup
  22. 22. © 2019 Texas Instruments Inc Sitara processors enable machine learning inference at the edge 22 Data transmission cost Network latency Security Reliability System power Network Bandwidth Network connectivity Data uploaded to cloud Prediction result returned Machine learning inferenceSensor data Action The Cloud Inference on the cloud Sitara machine learning addresses following concerns from cloud processing ActionSensor data Machine learning inference Local hardware Inference at the edge
  23. 23. © 2019 Texas Instruments Inc TIDL software framework reference design TIDEP-01004 Design: available now • Highly integrated solution bringing deep learning inference to the edge. • Guide through dense and sparse model training, importing and deploying the model for inference run on AM57x SoC • API to run multiple (same or different) networks in parallel on different EVE subsystem and DSP cores. • Example trained networks ready to run Benefits • TIDEP-01004 • Design Guide • Design Files: Schematics, BOM, Gerbers, Software, etc. • Device Datasheets: ‒ AM5749 IDK ‒ AM5749 SoC • This TI Design demonstrates TI Deep Learning (TIDL) inference running on an AM5749 IDK. • Shows running TIDL inference both on the EVE subsystems and on the C66x DSP cores • Includes benchmarks of several popular networks on AM57x • Walk-through of deep learning development flow including training for sparse network, import and inference deployment Features • Machine Vision: Vision Computer, Code Readers • Automated Machinery: Automated Sorting Equipment, Optical Inspection • Logistics Robots: Logistics Robots CPU board • EPOS: ATMs, Currency Counter • Many others… Applications Tools & Resources

×