Download to read offline
More Related Content
Similar to Nvidia Toolkits- DeepStream.pptx
Nvidia Toolkits- DeepStream.pptx
- 1.
- 2. NVIDIA DEEPSTREAM SDK Buildand deploy AI-powered Intelligent Video Analytics apps and services. DeepStream offers a multi-platform scalable framework with TLS security to deploy on the edge and connect to any cloud.
- 3. NVIDIA DEEPSTREAM SDK •With DeepStream SDK you can apply AI to streaming video and can simultaneously optimize video decode/encode, image scaling and conversion and edge-to-cloud connectivity for complete end-to-end performance optimization • Developers can build seamless streaming pipelines for AI-based video, audio and image analytics using DeepStream. • DeepStream offers an IoT integration interface with Kafka, MQTT and AMQP and turnkey integration with AWS IoT and Microsoft Azure IoT. • By using DeepStream, you can deploy at scale and manage containerized apps with Kubernetes and Helm Charts.
- 4. DEEPSTREAM SDK PLUG-INS •H.264and H.265 video decoding •Stream aggregation and batching •TensorRT-based inferencing for detection, classification and segmentation •Object tracking reference implementation •On-screen display API for highlighting objects and text overlay •Frame rendering from multi-source into a 2D grid array •Accelerated X11/EGL-based rendering •Filtering based on Region of Interest (ROI) •JPEG decoding •Scaling, format conversion, and rotation •Dewarping for 360-degree camera input •Metadata generation and encoding •Messaging to cloud •Audio/Video Template Plug-In
- 5. NVIDIA DEEPSTREAM SDK •Python bindings • The DeepStream Python application uses the Gst-Python API action to construct the pipeline and use probe functions to access data at various points in the pipeline. • Prerequisites • Ubuntu 18.04 • DeepStream SDK 5.1 or later • Python 3.6 • Gst Python v1.14.5 1.Clone the deepstream_python_apps repo under <DeepStream 5.1 ROOT>/sources: git clone https://github.com/NVIDIA-AI-IOT/deepstream_python_apps 2.This will create the following directory: <DeepStream 5.1 ROOT>/sources/deepstream_python_apps Sample Application Source Details
- 6. APPLICATION ARCHITECTURE • Theimage below shows the architecture of the NVIDIA® DeepStream reference application.
- 7. APPLICATION ARCHITECTURE The listof GStreamer plugins implemented by NVIDIA and provided as a part of DeepStream SDK include: • The Stream Muxer plugin (Gst-nvstreammux) to form a batch of buffers from multiple input sources. • The NVIDIA TensorRT™ based plugin (Gst-nvinfer) for primary and secondary (attribute classification of primary objects) detection and classification respectively. • The OpenCV based tracker plugin (Gst-nvtracker) for object tracking with unique ID. • The Multi Stream Tiler plugin (Gst-nvmultistreamtiler) for forming 2D array of frames. • The Onscreen Display (OSD) plugin (Gst-nvdsosd) to draw shaded boxes, rectangles and text on the composited frame using the generated metadata. • The Message Converter (Gst-nvmsgconv) and Message Broker (Gst-nvmsgbroker) plugins in combination to send analytics data to a server in the Cloud. Configuration Groups • The application configuration is divided into groups of configurations for each component and application-specific component.
- 8. REMOTELY MANAGING ANDCONTROLLING THE APPLICATION • supports bi-directional communication to send and receive cloud-to- device messages • This is particularly important for various use cases, such as triggering the application to record an important event, changing operating parameters and app configurations, over-the-air (OTA) updates, or requesting system logs and other vital information.
- 9. REMOTELY MANAGING ANDCONTROLLING THE APPLICATION • For Device-to-cloud messaging: The Gstnvmsgbroker plugin by default calls the lower-level adapter library with the appropriate protocol. • You can choose between Kafka, AMQP, MQTT, or Azure IoT, or you can even create a custom adapter. • For cloud-to-edge messaging: The supported protocol in DeepStream 5.0 is Kafka using the new low-level msgbroker library, interacting directly with the DeepStream application. • DeepStream also supports OTA updates of AI models while the application is running.
- 10. SMART VIDEO RECORDING •There is often a need to have event-based video recording. Instead of continuously recording the content, smart recording can save valuable disk space. • Smart record only records the event when specific rules or conditions are met. The trigger to signal the record can come locally from the app, from some service running at the edge or from the cloud. provide faster searchability. • When an event must be recorded, it’s useful to start saving the clip before the trigger. • The smart recording module keeps a video cache so that the recorded video not only has frames after the event is generated.
- 11. SMART VIDEO RECORDING •Recording can be triggered by JSON messages received from the cloud. The message format is as follows: The source code for deepstream-test5 can be found in the following directory: $DEEPSTREAM_DIR/sources/apps/sample_apps/deepstream-test5/ The implementation of the smart record event manager can be found in the following file: $DEEPSTREAM_DIR/sources/apps/apps-common/src/deepstream_source_bin.c
- 12. OTA MODEL UPDATE •With DeepStream 5.0, you can now update a model while the application is running. • This means that models can be updated with zero-downtime. This is important for mission-critical applications that can’t accept any latency. • This feature can also be useful when models need to be swapped continuously.
- 13. SECURE COMMUNICATION • WithDeepStream 5.0, the Kafka adapter supports secure communication using TLS-based encryption, thereby ensuring the confidentiality of data. • TLS (Transport Layer Security) is a successor to SSL. • DeepStream 5.0 supports two forms of client authentication: two-way TLS authentication based on SSL certificates and SASL/Plain authentication based on a username/password mechanism.
- 14.