Robotics & AI User Group - Computer Vision - Azure Kinect
•Download as PPTX, PDF•
0 likes•121 views
Slide deck from the first Robotics & Artificial Intelligence Virtual User Group, with theme "Computer Vision". Includes a presentation of Azure Kinect. https://www.meetup.com/robotics-artificial-intelligence-meetup-group/
Recommended
More Related Content
What's hot
Similar to Robotics & AI User Group - Computer Vision - Azure Kinect
Similar to Robotics & AI User Group - Computer Vision - Azure Kinect (20)
More from Stefano Tempesta
More from Stefano Tempesta (20)
Recently uploaded
Recently uploaded (20)
Robotics & AI User Group - Computer Vision - Azure Kinect
- 2. EVENTS
- 3. INTRODUCTIONS CEO @ JouleHub CTO @ Connecting Software, Microsoft Regional Director & MVP Alessandro Graps STEFANOTEMPESTA
- 5. COMPUTER VISION | 29 OCT 2020 SmartHelmet Azure Kinect
- 7. AZURE KINECT
- 10. RETAIL
- 12. HEALTH & FITNESS
- 13. WHAT’S INSIDE 1 MP depth sensor with wide and narrow field- of-view (FOV) options that help you optimise for your application 7-microphone array for far-field speech and sound capture 12 MP RGB video camera for an additional colour stream that’s aligned to the depth stream Accelerometer and gyroscope (IMU) for sensor orientation and spatial tracking External sync pins to easily synchronise sensor streams from multiple Kinect devices 1 2 3 4 5
- 14. AZURE KINECT DK
- 15. COLOUR CAMERA + AZURE COGNITIVE SERVICES Computer Vision Image classification Object detection Text extraction (OCR) Face Recognition Face detection Face identification Face matching
- 16. DEPTH CAMERA
- 17. BODY TRACKING
Editor's Notes
- Azure Kinect is a cutting-edge spatial computing developer kit with sophisticated computer vision and speech models, advanced AI sensors and a range of powerful SDKs that can be connected to Azure cognitive services. Using Azure Kinect, manufacturing, retail, healthcare and media enterprises are leveraging spatial data and context to enhance operational safety, increase performance, improve outcomes and revolutionise the customer experience.
- Deliver improved safety and performance Use advanced body tracking to monitor and analyse worker movement and behaviour. Design safer, more ergonomic workstations that can mitigate long-term injury risk while improving productivity through task analysis and process optimisation. Pair real-time depth sensor data with cognitive services for AI-driven part identification and real-time anomaly detection. Monitor assembly line workers to catch part or process issues straight away for a just-in-time fix.
- Gain new environmental understanding Use depth sensing to automate depalletisation and palletisation processes. Streamline repetitive tasks such as stripping down a pallet to improve productivity while optimising storage space in smart ways. Mobilise robot workers with spatial data. Automate new tasks by bringing robots into human spaces, from picking online orders and tracking shelf inventory to maintaining the spatial data of digital twin models.
- Transform experiences and accelerate innovation Use dimensional analysis to maximise transport storage space and optimise delivery logistics. Move from pre-defined delivery routes to real-time routes based on inventory analysis and subsequent last-mile efficiency. Create a connected retail experience by mapping and monitoring retail stores in real time. Gain actionable customer insights and create alerts leveraging spatial data.
- Reimagine communication and engagement Add computer vision and speech capabilities to smart signage. Transform smart signage into interactive, personalised kiosks that can enhance brand engagement. Use depth sensing to capture high-quality volumetric video for use in interactive 3D environments. Leverage 3D volumetric video for cutting-edge immersive storytelling, delivered through interactive XR experiences.
- Develop safer solutions and strengthen engagement Use computer vision to track and monitor exercise movements and overall form. Create a smart gym experience that can enhance results, or an accessible and effective virtual rehabilitation solution. Pair real-time depth sensor data with AI-driven insights. Prevent and mitigate potential patient accidents and injuries in care environments with predictive alerts.
- The Sensor SDK has the following features that work once installed and run on the Azure Kinect DK: Depth camera access and mode control (a passive IR mode, plus wide and narrow field-of-view depth modes) RGB camera access and control (for example, exposure and white balance) Motion sensor (gyroscope and accelerometer) access Synchronized Depth-RGB camera streaming with configurable delay between cameras External device synchronization control with configurable delay offset between devices Camera frame meta-data access for image resolution, timestamp, etc. Device calibration data access
- Azure Kinect DK is a developer kit with advanced AI sensors that provide sophisticated computer vision and speech models. Kinect contains a depth sensor, spatial microphone array with a video camera, and orientation sensor as an all in-one small device with multiple modes, options, and software development kits (SDKs). The Azure Kinect DK development environment consists of the following multiple SDKs: Sensor SDK for low-level sensor and device access. Body Tracking SDK for tracking bodies in 3D. Speech Cognitive Services SDK for enabling microphone access and Azure cloud-based speech services. In addition, Cognitive Vision services can be used with the device RGB camera.
- The Azure Kinect DK depth camera implements the Amplitude Modulated Continuous Wave (AMCW) Time-of-Flight (ToF) principle. The camera casts modulated illumination in the near-IR (NIR) spectrum onto the scene. It then records an indirect measurement of the time it takes the light to travel from the camera to the scene and back. These measurements are processed to generate a depth map. A depth map is a set of Z-coordinate values for every pixel of the image, measured in units of millimeters. Along with a depth map, we also obtain a so-called clean IR reading. The value of pixels in the clean IR reading is proportional to the amount of light returned from the scene. The image looks similar to a regular IR image. The figure below shows an example depth map (left) and a corresponding clean IR image (right).
- Azure Kinect body tracking can track multiple human bodies at the same time. Joint coordinates The position and orientation of each joint form its own joint coordinate system. All joint coordinate systems are absolute coordinate systems relative to the depth camera 3D coordinate system. Joint hierarchy A skeleton includes 32 joints with the joint hierarchy flowing from the center of the body to the extremities. Each connection (bone) links the parent joint with a child joint. The figure illustrates the joint locations and connection relative to the human body.