“How Do We Enable Edge ML Everywhere? Data, Reliability and Silicon Flexibility,” a Presentation from Edge Impulse

•

0 likes•91 views

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2022/06/how-do-we-enable-edge-ml-everywhere-data-reliability-and-silicon-flexibility-a-presentation-from-edge-impulse/ Zach Shelby, Co-founder and CEO of Edge Impulse, presents the “How Do We Enable Edge ML Everywhere? Data, Reliability and Silicon Flexibility” tutorial at the May 2022 Embedded Vision Summit. In this talk, Shelby reveals insights from the company’s recent global edge ML developer survey, which identified key barriers to machine learning adoption, and shares the company’s vision for how the industry can overcome these obstacles. Unsurprisingly, the first critical obstacle identified by the survey is data. But the issue isn’t simply a lack of massive datasets, as is often assumed. On the contrary, the biggest opportunities in ML will be enabled by highly custom, industry-specific and even user-specific data. We need to master data lifecycle and active learning techniques that enable developers to move quickly from “zero to dataset.” The real and perceived inability of today’s ML algorithms to reach the ultra-high accuracy needed in industrial systems is another key barrier. New techniques for explainable ML, better testing, sensor fusion and model fusion will increasingly allow developers to achieve industrial-grade reliability. Finally, in order to accelerate ML adoption in embedded products, we must recognize that most developers can’t immediately upgrade their systems to use the latest chips — a problem that is compounded by today’s chip shortages. To enable ML everywhere, we have to find ways to deploy ML on today’s silicon, while ensuring a smooth transition to new devices with AI acceleration in the future.

Technology

How Do We Enable Edge ML
Everywhere? Data,
Reliability, and Silicon
Flexibility
Zach Shelby
Co-founder and CEO
Edge Impulse

Visual inspection system to monitor worker safety and flag delays
on the production line in real-time.
Advantech increases manufacturing productivity by 15%
● A reported 15% overall increase in production line
efficiency
● Faster detection of idle time raises assembly-line
productivity
● Managers free up time to focus on production planning
and operations
2
© 2022 Edge Impulse

State of the edge ML 2022
3
© 2022 Edge Impulse

What are the barriers to edge ML scale?
4
© 2022 Edge Impulse

Big data, big model is a problem
6
Dataset Test Deploy
Model
• State of the art: Monolithic batch, from big data to big model
• Developed for ML research, unsuitable for most edge ML applications
© 2022 Edge Impulse

Data-Centric ML – from zero to hero
7
Test Deploy
Anomaly
• Auto-clustering, using feature analysis to help experts quickly label data
• Auto-labeling, using pre-trained expert models to suggest labels
• Active learning, using inference to drive the data collection process
Model
Dataset
Auto-clustering
Auto-labeling
Active Learning
© 2022 Edge Impulse

Auto-clustering in action
8
© 2022 Edge Impulse

Sensor fusion for industrial-grade ML
Image
Processing
Sensor
Audio
Image
ML Classifier
© 2022 Edge Impulse 10

Sensor fusion for industrial-grade ML
Image
Processing
Sensor
Audio
Image
ML Classifier
DSP Features
Post-
processing
ML Classifier
Anomaly
Detector
© 2022 Edge Impulse 11

Sensor fusion for industrial-grade ML
Image
Processing
Sensor
Audio
Image
ML Classifier
DSP Features
Multi-modal
ML Classifier
Anomaly
Detector
Post-
processing
Audio Features
© 2022 Edge Impulse 12

• Model training validation != performance
• Requires testing on the entire algorithm
with real-world data for realistic
performance
• Understand the impact of post-processing
while accounting for device constraints and
latency
• Choose the ideal balance between false
activations and false rejections
• Leverage genetic algorithms to design
optimal post-processing configuration
Calibrating performance at scale
© 2022 Edge Impulse

Hardware is sexy again!
15
Sensor
Audio
Image
Video
MCU + FPU CPU GPU
High-end MCU
Low-end MCU
RA MCU
PSoC 6
© 2022 Edge Impulse

Hardware is sexy again!
16
MCU + FPU CPU GPU
High-end MCU
Low-end MCU
Sensor
Audio
Image
Video
Wakeup NPU
NDP100 RA MCU
PSoC 6
© 2022 Edge Impulse

Hardware is sexy again!
17
MCU + FPU CPU GPU
MCU + NPU
Low-end MCU
Sensor
Audio
Image
Video
Wakeup NPU
NDP100
NDP200
Akida
M55/U55
Katana
RA MCU
PSoC 6
© 2022 Edge Impulse
xG24

Hardware is sexy again!
18
MCU + FPU CPU + NPU NPU + GPU
MCU + NPU
Low-end MCU
Sensor
Audio
Image
Video
TD4VA
Wakeup NPU
NDP200
RZ MPU
NDP100 RA MCU
PSoC 6
© 2022 Edge Impulse
Akida
M55/U55
Katana
xG24

• The power of hardware profiling
• Digital twin of ML on hardware
• We are combining hardware profiling with
hyperparameter search – EON Tuner
• Hardware-aware AutoML across data, pre-
processing and ML blocks
Hardware profiling & tuning
19
© 2022 Edge Impulse

FOMO: Faster objects, more objects
● Object detection on constrained compute
● 20x performance improvement
● Scales down to 100k RAM to enable MCUs
● Better at detecting smaller and more numerous
objects
● Capable of segmentation and counting objects
Cortex-M4 Cortex-M7 Cortex-A Nvidia
FOMO 2 fps 15-30 fps 60+ fps 150+ fps
SSD NA NA 3 fps 20 fps
© 2022 Edge Impulse
20

Resources
Increasing Manufacturing Efficiency
https://casestudies.edgeimpulse.com/industrial-iot-with-advantech
Constrained object detection: FOMO Blog
docs.edgeimpulse.com/docs/tutorials/fomo-object-detection-for-
constrained-devices
Request a demo
edgeimpulse.com/schedule-a-demo
21
© 2022 Edge Impulse

Don’t miss these talks!
FOMO: Real-time Object Detection on Microcontrollers
Jan Jongboom, Co-founder and CTO, Edge Impulse
• Date: Wednesday, May 18
• Start Time: 10:50 am
Deep Dive: Develop and Deploy Advanced Edge Computer Vision—Fast!
Jenny Plunkett and Shawn Hymel, Snr. DevRel engineers
• Date: Thursday, May 19
• Time: 9 am – 12:00 pm
22
© 2022 Edge Impulse

Similar to “How Do We Enable Edge ML Everywhere? Data, Reliability and Silicon Flexibility,” a Presentation from Edge Impulse

HiPEAC-CSW 2022_Pedro Trancoso presentation

VEDLIoT Project

Download our special report, IoT Tech for the Manager: http://bit.ly/report1-slideshare Hey IT, Meet OT as presented at the IoT Inc Business' fifteenth Meetup. See: http://www.iot-inc.com/hey-it-meet-ot-meetup/ In our fifteenth Meetup we have Hima Mukkamala, Head of Engineering at Predix, GE Digital presenting “Hey IT, Meet OT”. Presentation Abstract Software has been the domain of information technology, but it is quickly becoming key to operations technology as well. Operating smart, networked machines from wind turbines to jet engines requires an intricate understanding of both the machines and the data and information that flows through them. The combination of these two disciplines is bringing new efficiencies and capabilities that do more—faster and cheaper. The key is leveraging connectivity, data, and mobility to optimize efficiency and deliver new services to customers. Join Hima Mukkamala of GE Digital to hear how software technology can help companies bridge the divide between IT and OT and how IT can help industrial companies build, deploy, and manage Industrial Internet applications that bring game-changing efficiencies to businesses.

Hey IT, Meet OT with Hima Mukkamala

gogo6

In the design of electronics and semiconductors, challenges are compounded by the integration of AI, multi-core, real-time software, network, connectivity, diagnostics, and security. Performance limits, battery life, and cost are adoption barriers. It is extremely important to have tools and processes that deliver efficiency throughout the design cycle. Continuous verification from planning to development addresses the multi-discipline needs of hardware, software, and networks. This unique approach accelerates the design phase, defines the test efforts, and finds defects during specification. Architecture modeling is required to meet timing deadlines, generate the lowest power consumption, and attain the highest Quality-of-Service. optimize the electronic design system and designing of custom components.

Introduction to architecture exploration

Deepak Shankar

Trends towards the merge of HPC + Big Data systems

Igor José F. Freitas

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2021/07/the-data-driven-engineering-revolution-a-presentation-from-edge-impulse/ Zach Shelby, Co-founder and CEO of Edge Impulse, presents the “Data-Driven Engineering Revolution” tutorial at the May 2021 Embedded Vision Summit. In this talk, IoT industry pioneer and Edge Impulse co-founder Zach Shelby shares insights about how machine learning is revolutionizing embedded engineering. Advances in silicon and deep learning are enabling embedded machine learning (TinyML) to be deployed where data is born, from industrial sensor data to audio and video. Shelby explains the new paradigm of data-driven engineering with ML, showing how developers are using data instead of code to drive algorithm innovation. To support widespread deployment, ML workloads need to run on embedded computing targets from MCUs to GPUs, with MLOps processes to support efficient development and deployment. Industrial, logistics and health markets are particularly ripe to deploy this data-driven approach, and Shelby highlights several exciting case studies.

“The Data-Driven Engineering Revolution,” a Presentation from Edge Impulse

Edge AI and Vision Alliance

Neo4j: The path to success with Graph Database and Graph Data Science

Neo4j

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2020/11/smarter-manufacturing-with-intels-deep-learning-based-machine-vision-a-presentation-from-intel/ For more information about edge AI and computer vision, please visit: https://www.edge-ai-vision.com Tara K. Thimmanaik, Solutions Architect at Intel, presents the “Smarter Manufacturing with Intel’s Deep Learning-Based Machine Vision” tutorial at the September 2020 Embedded Vision Summit. As demand for smarter and more efficient manufacturing is growing, IoT technologies⁠—including sensors, edge devices, gateways, servers and the cloud⁠—are being used throughout the factory to compute deep learning analytics workloads at the appropriate location. Efficient data-driven manufacturing can help to reduce labor costs, increase quality and maximize profit. The biggest hindrance to achieving these outcomes is the difficulty in extracting data from vendor-locked and proprietary systems for analytics downstream. In this presentation, Thimmanaik covers Intel’s approach to developing open, flexible and scalable solutions, including: • Intel’s technologies such as OpenVINO, Movidius Vision Processor Units, Edge Insights Software (EIS) and deep learning algorithms • How Intel’s offerings come together in the industrial marketplace with partnerships forged to address the constraints of manufacturing infrastructure • Real-world examples highlighting defect detection in textile printing (where 90% accuracy at 50 fps was achieved) and smartphone screen production (where false negatives were only 0.6%)

“Smarter Manufacturing with Intel’s Deep Learning-Based Machine Vision,” a Pr...

Edge AI and Vision Alliance

byteLAKE's CFD Suite (AI-accelerated CFD) (2024-02)

byteLAKE

Cisco Connect Ottawa 2018 dna assurance shortest path to network innocence

Cisco Canada

In this video from Switzerland HPC Conference, Martin Hilgeman from Dell presents: HPC Workload Efficiency and the Challenges for System Builders. "With all the advances in massively parallel and multi-core computing with CPUs and accelerators it is often overlooked whether the computational work is being done in an efficient manner. This efficiency is largely being determined at the application level and therefore puts the responsibility of sustaining a certain performance trajectory into the hands of the user. It is observed that the adoption rate of new hardware capabilities is decreasing and lead to a feeling of diminishing returns. This presentation shows the well-known laws of parallel performance from the perspective of a system builder. It also covers through the use of real case studies, examples of how to program for energy efficient parallel application performance." Watch the video: http://wp.me/p3RLHQ-gIS Learn more: http://dell.com and http://www.hpcadvisorycouncil.com/events/2017/swiss-workshop/agenda.php Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

Trends in Systems and How to Get Efficient Performance

inside-BigData.com

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/06/deploy-your-embedded-vision-solution-on-any-processor-using-edge-impulse-a-presentation-from-edge-impulse/ Amir Sherman, Global Semiconductor Business Development Director at Edge Impulse, presents the “Deploy Your Embedded Vision Solution on Any Processor Using Edge Impulse” tutorial at the May 2023 Embedded Vision Summit. Vision-based product developers have a vast array of processors to choose from. Unfortunately, each processor has its own unique tool chain. This makes it difficult for developers to evaluate processor options and target different processors for different product needs. Fortunately, there’s Edge Impulse Studio—a universal development solution that is compatible with any vision-based processor. For example, when using Arm processors, developers can target Arm Cortex-M MCUs, the latest Cortex-M55, the Ethos U55 microNPU, and Cortex-A CPUs—along with the GPUs that are often used with them. Edge Impulse Studio also supports a wide range of specialized accelerators, such as the Renesas DRP-AI engine and neuromorphic processor cores. In this session, Sherman shows the key features and benefits of Edge Impulse Studio and demonstrate how it enables developers to quickly evaluate processor platforms and easily deploy vision applications on a wide range of embedded hardware.

“Deploy Your Embedded Vision Solution on Any Processor Using Edge Impulse,” A...

Edge AI and Vision Alliance

Graph-Based Network Topology Analysis for Telecom Operators

Neo4j

Traceability, reproducibility and scalability with hundreds of AI services

LauraCalem

MAJiK Systems are the creators of a cloud-based factory and manufacturing monitoring platform. Their Visual Factory solution helps their customers monitor, analyze, and optimize operations by providing real-time plant floor analytics. They aim to provide insights into root cause analysis of downtime, quality issues and other production slowdowns. Clients have seen reduction in equipment downtime of 35-45% and reductions of waste and scrap of 7-10%. Discover how MAJiK uses a time series data platform to continuously improve overall equipment effectiveness (OEE) within factories. Join this webinar as Jared Evans dives into: MAJiK's approach to data-driven Industry 4.0 monitoring Industrial IoT best practices - including when to use specific protocols (Modbus, OPC-UA, MQTT, Ethernet/IP, or S7 TIA) How data from Programmable Logic Controllers (PLCs) can be integrated into InfluxDB - which helps process engineers monitor the telemetry of any automated process or production method

How to Digitize Industrial Manufacturing with Azure IoT Edge, InfluxDB, and M...

InfluxData

s2c-success-story-ingenic.pdf

S2C Limited

Nervana and the Future of Computing

Intel Nervana

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2022/06/optimization-techniques-with-intels-openvino-to-enhance-performance-on-your-existing-hardware-a-presentation-from-intel/ Nico Galoppo, Principal Engineer (substituting for Ansley Dunn, Product Marketing Manager), and Ryan Loney, Technical Product Manager, both of Intel, present the “Optimization Techniques with Intel’s OpenVINO to Enhance Performance on Your Existing Hardware” tutorial at the May 2022 Embedded Vision Summit. Whether you’re using TensorFlow, PyTorch or another framework, Galoppo and Loney show you optimization techniques to enhance performance on your existing hardware. With the OpenVINO Toolkit, built on the foundation of OneAPI, developers can utilize their own AI model or leverage one of the hundreds of pre-trained models available across vision and audio use cases. In this presentation, you’ll learn how the Neural Network Compression Framework provides optimal model training templates for performance boosts while preserving accuracy, and how the Model Optimizer reduces complexity and makes model conversion faster. Other areas explored by Galoppo and Loney include auto device discovery to enable automatic load balancing and how to optimize for latency or throughput based on your workload.

“Optimization Techniques with Intel’s OpenVINO to Enhance Performance on Your...

Edge AI and Vision Alliance

Reproducibility of ML system is increasingly important topic in ML community. Reproducibility ensures conclusiveness of the model performance, provides an understanding how ML system works and reduces unnecessary errors when the system is deployed into production. With increasing AI regulation, it will soon become a requirement for many ML applications. In this talk, we will explore different aspects of reproducibility such as reproducibility of the dataset, data processing, ML model, its randomness and hyperparameters, code and SW environment, as well as concepts and practical tools such as data versioning, feature, metadata and artifact store, model registry and containerization that together ensure reproducibility of our experiments.

[DSC Europe 22] Reproducibility and Versioning of ML Systems - Spela Poklukar

DataScienceConferenc1

Ibm symp14 referentin_barbara koch_power_8 launch bk

IBM Switzerland

The number of internet-connected devices is growing exponentially, enabling an increasing number of edge applications in environments such as smart cities, retail, and industry 4.0. These intelligent solutions often require processing large amounts of data, running models to enable image recognition, predictive analytics, autonomous systems, and more. Increasing system workloads and data processing capacity at the edge is essential to minimize latency, improve responsiveness, and reduce network traffic back to data centers. Purpose-built systems such as Supermicro’s short-depth, multi-node SuperEdge, powered by 3rd Gen Intel® Xeon® Scalable processors, increase compute and I/O density at the edge and enable businesses to further accelerate innovation. Join this webinar to discover new insights in edge-to-cloud infrastructures and learn how Supermicro SuperEdge multi-node solutions leverage data center scale, performance, and efficiency for 5G, IoT, and Edge applications.

Accelerating Innovation from Edge to Cloud

Rebekah Rodriguez

Similar to “How Do We Enable Edge ML Everywhere? Data, Reliability and Silicon Flexibility,” a Presentation from Edge Impulse (20)

HiPEAC-CSW 2022_Pedro Trancoso presentation

Hey IT, Meet OT with Hima Mukkamala

Introduction to architecture exploration

Trends towards the merge of HPC + Big Data systems

“The Data-Driven Engineering Revolution,” a Presentation from Edge Impulse

Neo4j: The path to success with Graph Database and Graph Data Science

“Smarter Manufacturing with Intel’s Deep Learning-Based Machine Vision,” a Pr...

byteLAKE's CFD Suite (AI-accelerated CFD) (2024-02)

Cisco Connect Ottawa 2018 dna assurance shortest path to network innocence

Trends in Systems and How to Get Efficient Performance

“Deploy Your Embedded Vision Solution on Any Processor Using Edge Impulse,” A...

Graph-Based Network Topology Analysis for Telecom Operators

Traceability, reproducibility and scalability with hundreds of AI services

How to Digitize Industrial Manufacturing with Azure IoT Edge, InfluxDB, and M...

s2c-success-story-ingenic.pdf

Nervana and the Future of Computing

“Optimization Techniques with Intel’s OpenVINO to Enhance Performance on Your...

[DSC Europe 22] Reproducibility and Versioning of ML Systems - Spela Poklukar

Ibm symp14 referentin_barbara koch_power_8 launch bk

Accelerating Innovation from Edge to Cloud

More from Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/11/learning-compact-dnn-models-for-embedded-vision-a-presentation-from-the-university-of-maryland-at-college-park/ Shuvra Bhattacharyya, Professor at the University of Maryland at College Park, presents the “Learning Compact DNN Models for Embedded Vision” tutorial at the May 2023 Embedded Vision Summit. In this talk, Bhattacharyya explores methods to transform large deep neural network (DNN) models into effective compact models. The transformation process that he focuses on—from large to compact DNN form—is referred to as pruning. Pruning involves the removal of neurons or parameters from a neural network. When performed strategically, pruning can lead to significant reductions in computational complexity without significant degradation in accuracy. It is sometimes even possible to increase accuracy through pruning. Pruning provides a general approach for facilitating real-time inference in resource-constrained embedded computer vision systems. Bhattacharyya provides an overview of important aspects to consider when applying or developing a DNN pruning method and presents details on a recently introduced pruning method called NeuroGRS. NeuroGRS considers structures and trained weights jointly throughout the pruning process and can result in significantly more compact models compared to other pruning methods.

“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/11/introduction-to-computer-vision-with-cnns-a-presentation-from-mohammad-haghighat/ Independent consultant Mohammad Haghighat presents the “Introduction to Computer Vision with Convolutional Neural Networks” tutorial at the May 2023 Embedded Vision Summit. This presentation covers the basics of computer vision using convolutional neural networks. Haghighat begins by introducing some important conventional computer vision techniques and then transition to explaining the basics of machine learning and convolutional neural networks (CNNs) and showing how CNNs are used in visual perception. Haghighat illustrates the building blocks and computational elements of neural networks through examples. This session provides an overview of how modern computer vision algorithms are designed, trained and used in real-world applications.

“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/11/selecting-tools-for-developing-monitoring-and-maintaining-ml-models-a-presentation-from-yummly/ Parshad Patel, Data Scientist at Yummly, presents the “Selecting Tools for Developing, Monitoring and Maintaining ML Models” tutorial at the May 2023 Embedded Vision Summit. With the boom in tools for developing, monitoring and maintaining ML models, data science teams have many options to choose from. Proprietary tools provided by cloud service providers are enticing, but teams may fear being locked in—and may worry that these tools are too costly or missing important features when compared with alternatives from specialized providers. Fortunately, most proprietary, fee-based tools have an open-source component that can be integrated into a home-grown solution at low cost. This can be a good starting point, enabling teams to get started with modern tools without making big investments and leaving the door open to evolve tool selection over time. In this talk, Patel presents a step-by-step process for creating an MLOps tool set that enables you to deliver maximum value as a data scientist. He shares how Yummly built pipelines for model development and put them into production using open-source projects.

“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/11/building-accelerated-gstreamer-applications-for-video-and-audio-ai-a-presentation-from-wave-spectrum/ Abdo Babukr, Accelerated Computing Consultant at Wave Spectrum, presents the “Building Accelerated GStreamer Applications for Video and Audio AI,” tutorial at the May 2023 Embedded Vision Summit. GStreamer is a popular open-source framework for creating streaming media applications. Developers often use GStreamer to streamline the development of computer vision and audio perception applications. Since perceptual algorithms are often quite demanding in terms of processing performance, in many cases developers need to find ways to accelerate key GStreamer building blocks, taking advantage of specialized features of their target processor or co-processor. In this talk, Babukr introduces GStreamer and shows how to use it to build computer vision and audio perception applications. He also shows how to create efficient, high-performance GStreamer applications that utilize specialized hardware features.

“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/11/understanding-selecting-and-optimizing-object-detectors-for-edge-applications-a-presentation-from-walmart-global-tech/ Md Nasir Uddin Laskar, Staff Machine Learning Engineer at Walmart Global Tech, presents the “Understanding, Selecting and Optimizing Object Detectors for Edge Applications” tutorial at the May 2023 Embedded Vision Summit. Object detectors count objects in a scene and determine their precise locations, while also labeling them. Object detection plays a crucial role in many vision applications, from autonomous driving to smart appliances. In many of these applications, it’s necessary or desirable to implement object detection at the edge. In this presentation, Laskar explores the evolution of object detection algorithms, from traditional approaches to deep learning-based methods and transformer-based architectures. He delves into widely used approaches for object detection, such as two-stage R-CNNs and one-stage YOLO algorithms, and examines their strengths and weaknesses. And he provides guidance on how to evaluate and select an object detector for an edge application.

“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/introduction-to-modern-lidar-for-machine-perception-a-presentation-from-the-university-of-ottawa/ Robert Laganière, Professor at the University of Ottawa and CEO of Sensor Cortek, presents the “Introduction to Modern LiDAR for Machine Perception” tutorial at the May 2023 Embedded Vision Summit. In this presentation, Laganière provides an introduction to light detection and ranging (LiDAR) technology. He explains how LiDAR sensors work and their main advantages and disadvantages. He also introduces different approaches to LiDAR, including scanning and flash LiDAR. Laganière explores the types of data produced by LiDAR sensors and explains how this data can be processed using deep neural networks. He also examines the synergy between LiDAR and cameras, and the concept of pseudo-LiDAR for detection.

“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/11/vision-language-representations-for-robotics-a-presentation-from-the-university-of-pennsylvania/ Dinesh Jayaraman, Assistant Professor at the University of Pennsylvania, presents the “Vision-language Representations for Robotics” tutorial at the May 2023 Embedded Vision Summit. In what format can an AI system best present what it “sees” in a visual scene to help robots accomplish tasks? This question has been a long-standing challenge for computer scientists and robotics engineers. In this presentation, Jayaraman provides insights into cutting-edge techniques being used to help robots better understand their surroundings, learn new skills with minimal guidance and become more capable of performing complex tasks. Jayaraman discusses recent advances in unsupervised representation learning and explains how these approaches can be used to build visual representations that are appropriate for a controller that decides how the robot should act. In particular, he presents insights from his research group’s recent work on how to represent the constituent objects and entities in a visual scene, and how to combine vision and language in a way that permits effectively translating language-based task descriptions into images depicting the robot’s goals.

“Vision-language Representations for Robotics,” a Presentation from the Unive...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/adas-and-av-sensors-whats-winning-and-why-a-presentation-from-techinsights/ Ian Riches, Vice President of the Global Automotive Practice at TechInsights, presents the “ADAS and AV Sensors: What’s Winning and Why?” tutorial at the May 2023 Embedded Vision Summit. It’s clear that the number of sensors per vehicle—and the sophistication of these sensors—is growing rapidly, largely thanks to increased adoption of advanced safety and driver assistance features. In this presentation, Riches explores likely future demand for automotive radars, cameras and LiDARs. Riches examines which vehicle features will drive demand out to 2030, how vehicle architecture change is impacting the market and what sorts of compute platforms these sensors will be connected to. Finally, he shares his firm’s vision of what the landscape could look like far beyond 2030, considering scenarios out to 2050 for automated driving and the resulting sensor demand.

“ADAS and AV Sensors: What’s Winning and Why?,” a Presentation from TechInsights

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/computer-vision-in-sports-scalable-solutions-for-downmarkets-a-presentation-from-sportlogiq/ Mehrsan Javan, Co-founder and CTO of Sportlogiq, presents the “Computer Vision in Sports: Scalable Solutions for Downmarket Leagues” tutorial at the May 2023 Embedded Vision Summit. Sports analytics is about observing, understanding and describing the game in an intelligent manner. In practice, this requires a fully automated, robust end-to-end pipeline, spanning from visual input, to player and group activities, to player and team evaluation to planning. Despite major advancements in computer vision and machine learning, today sports analytics solutions are limited to top leagues and are not widely available for downmarket leagues and youth sports. In this talk, Javan explains how his company has developed scalable and robust computer vision solutions to democratize sport analytics and offer pro-league-level insights to leagues with modest resources, including youth leagues. He highlights key challenges—such as the requirement for low-cost, low-latency processing and the need for robustness despite variations in venues. He discusses the approaches Sportlogiq tried and how it ultimately overcame these challenges, including the use of transformers and fusion of multiple type of data streams to maximize accuracy.

“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/detecting-data-drift-in-image-classification-neural-networks-a-presentation-from-southern-illinois-university/ Spyros Tragoudas, Professor and School Director at Southern Illinois University Carbondale, presents the “Detecting Data Drift in Image Classification Neural Networks” tutorial at the May 2023 Embedded Vision Summit. An unforeseen change in the input data is called “drift,” and may impact the accuracy of machine learning models. In this talk, Tragoudas presents a novel scheme for diagnosing data drift in the input streams of image classification neural networks. His proposed method for drift detection and quantification uses a threshold dictionary for the prediction probabilities of each class in the neural network model. The method is applicable to any drift type in images such as noise and weather effects, among others. Tragoudas shares experimental results on various data sets, drift types and neural network models to show that his proposed method estimates the drift magnitude with high accuracy, especially when the level of drift significantly impacts the model’s performance.

“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/deep-neural-network-training-diagnosing-problems-and-implementing-solutions-a-presentation-from-sensor-cortek/ Fahed Hassanat, Chief Operating Officer and Head of Engineering at Sensor Cortek, presents the “Deep Neural Network Training: Diagnosing Problems and Implementing Solutions” tutorial at the May 2023 Embedded Vision Summit. In this presentation, Hassanat delves into some of the most common problems that arise when training deep neural networks. He provides a brief overview of essential training metrics, including accuracy, precision, false positives, false negatives and F1 score. Hassanat then explores training challenges that arise from problems with hyperparameters, inappropriately sized models, inadequate models, poor-quality datasets, imbalances within training datasets and mismatches between training and testing datasets. To help detect and diagnose training problems, he also covers techniques such as understanding performance curves, recognizing overfitting and underfitting, analyzing confusion matrices and identifying class interaction issues.

“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/ai-start-ups-the-perils-of-fishing-for-whales-war-stories-from-the-entrepreneurial-front-lines-a-presentation-from-seechange-technologies/ Tim Hartley, Vice President of Product for SeeChange Technologies, presents the “AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepreneurial Front Lines)” tutorial at the May 2023 Embedded Vision Summit. You have a killer idea that will change the world. You’ve thought through product-market fit and differentiation. You have seed funding and a world-beating team. Best of all, you’ve caught the attention of major players in your industry. You’ve reached peak “start-up”—that point of limitless possibility—when you go to bed with the same level of energy and enthusiasm you had when you woke. And then the first proof of concept starts… In this talk, Hartley lays out some of the pitfalls that await those building the next big thing. Using real examples, he shares some of the dos and don’ts, particularly when dealing with that big potential first customer. Hartley discusses the importance of end-to-end design, ensuring your product solves real-world problems. He explores how far the big companies will tell you to jump—and then jump again—for free. And, most importantly, how to build long-term partnerships with major corporations without relying on over-promising sales pitches.

“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/a-computer-vision-system-for-autonomous-satellite-maneuvering-a-presentation-from-scout-space/ Andrew Harris, Spacecraft Systems Engineer at SCOUT Space, presents the “Developing a Computer Vision System for Autonomous Satellite Maneuvering” tutorial at the May 2023 Embedded Vision Summit. Computer vision systems for mobile autonomous machines experience a wide variety of real-world conditions and inputs that can be challenging to capture accurately in training datasets. Few autonomous systems experience more challenging conditions than those in orbit. In this talk, Harris describes how SCOUT Space has designed and trained satellite vision systems using dynamic and physically informed synthetic image datasets. Harris describes how his company generates synthetic data for this challenging environment and how it leverages new real-world data to improve our datasets. In particular, he explains how these synthetic datasets account for and can replicate real sources of noise and error in the orbital environment, and how his company supplements them with in-space data from the first SCOUT-Vision system, which has been in orbit since 2021.

“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/bias-in-computer-vision-its-bigger-than-facial-recognition-a-presentation-from-santa-clara-university/ Susan Kennedy, Assistant Professor of Philosophy at Santa Clara University, presents the “Bias in Computer Vision—It’s Bigger Than Facial Recognition!” tutorial at the May 2023 Embedded Vision Summit. As AI is increasingly integrated into various industries, concerns about its potential to reproduce or exacerbate bias have become widespread. While the use of AI holds the promise of reducing bias, it can also have unintended consequences, particularly in high-stakes computer vision applications such as facial recognition. However, even seemingly low-stakes computer vision applications such as identifying potholes and damaged roads can also present ethical challenges related to bias. This talk explores how bias in computer vision often poses an ethical challenge, regardless of the stakes involved. Kennedy discusses the limitations of technical solutions aimed at mitigating bias, and why “bias-free” AI may not be achievable. Instead, she focuses on the importance of adopting a “bias-aware” approach to responsible AI design and explores strategies that can be employed to achieve this.

“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/sensor-fusion-techniques-for-accurate-perception-of-objects-in-the-environment-a-presentation-from-sanborn-map-company/ Baharak Soltanian, Vice President of Research and Development for the Sanborn Map Company, presents the “Sensor Fusion Techniques for Accurate Perception of Objects in the Environment” tutorial at the May 2023 Embedded Vision Summit. Increasingly, perceptual AI is being used to enable devices and systems to obtain accurate estimates of object locations, speeds and trajectories. In demanding applications, this is often best done using a heterogeneous combination of sensors (e.g., vision, radar, LiDAR). In this talk, Soltanian introduces techniques for combining data from multiple sensors to obtain accurate information about objects in the environment. Soltanian briefly introduces the roles played by Kalman filters, particle filters, Bayesian networks and neural networks in this type of fusion. She then examines alternative fusion architectures, such as centralized and decentralized approaches, to better understand the trade-offs associated with different approaches to sensor fusion as used to enhance the ability of machines to understand their environment.

“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/updating-the-edge-ml-development-process-a-presentation-from-samsara/ Jim Steele, Vice President of Embedded Software at Samsara, presents the “Updating the Edge ML Development Process” tutorial at the May 2023 Embedded Vision Summit. Samsara (NYSE:IOT) is focused on digitizing the world of operations. The company helps customers across many industries—including food and beverage, utilities and energy, field services and government—get information about their physical operations into the cloud, so they can operate more safely, efficiently and sustainably. Samsara’s sensors collect billions of data points per day and on-device processing is instrumental to its success. The company is constantly developing, improving and deploying ML models at the edge. Samsara has found that the traditional development process—where ML scientists create models and hand them off to firmware engineers for embedded implementation—is slow and often produces difficult-to-resolve differences between the original model and the embedded implementation. In this talk, Steele presents an alternative development process that his company has adopted with good results. In this process, firmware engineers develop a general framework that ML scientists use to design, develop and deploy their models. This enables quick iterations and fewer confounding bugs.

“Updating the Edge ML Development Process,” a Presentation from Samsara

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/combating-bias-in-production-computer-vision-systems-a-presentation-from-red-cell-partners/ Alex Thaman, Chief Architect at Red Cell Partners, presents the “Combating Bias in Production Computer Vision Systems” tutorial at the May 2023 Embedded Vision Summit. Bias is a critical challenge in predictive and generative AI that involves images of humans. People have a variety of body shapes, skin tones and other features that can be challenging to represent completely in training data. Without attention to bias risks, ML systems have the potential to treat people unfairly, and even to make humans more likely to do so. In this talk, Thaman examines the ways in which bias can arise in visual AI systems. He shares techniques for detecting bias and strategies for minimizing it in production AI systems.

“Combating Bias in Production Computer Vision Systems,” a Presentation from R...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/developing-an-embedded-vision-ai-powered-fitness-system-a-presentation-from-peloton-interactive/ Sanjay Nichani, Vice President for Artificial Intelligence and Computer Vision at Peloton Interactive, presents the “Developing an Embedded Vision AI-powered Fitness System” tutorial at the May 2023 Embedded Vision Summit. The Guide is Peloton’s first strength-training product that runs on a physical device and also the first that uses AI technology. It turns any TV into an interactive personal training studio. Members have access to Peloton’s instructors, who lead classes that use dumbbells and body weight. The Guide uses an innovative combination of hardware, software and user interface; it is one of only a handful of consumer products in this market that use real-time embedded computer vision, and Peloton’s members love it! In this talk, Nichani shares insights into how the computer vision technology behind the Guide was built. He focuses on data, models and processes. The AI and Computer Vision team at Peloton worked through numerous obstacles by using synthetic data, advanced algorithms and field-testing iteration to create an affordable AI-powered device that enables at-home fitness for everyone.

“Developing an Embedded Vision AI-powered Fitness System,” a Presentation fro...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/navigating-the-evolving-venture-capital-landscape-for-edge-ai-start-ups-a-presentation-from-hpe-pathfinder/ Todd Poole, Director of Venture Capital Investments for HPE Pathfinder Ventures, presents the “Navigating the Evolving Venture Capital Landscape for Edge AI Start-ups” tutorial at the May 2023 Embedded Vision Summit. The venture capital environment has undergone a huge shift in the past year. In this talk, Poole explains how the VC landscape has changed and the implications for start-ups. He examines the impacts on valuations, VC’s willingness to fund, appetite for risk and leverage in deal structuring and terms negotiation. Pool explores how AI and ML hardware and software companies have been affected differently from the broader start-up ecosystem and considers the distinct challenges faced by start-ups in the edge AI sector. He offers guidance for founders, soon-to-be-founders and start-up employees on how to navigate this dynamic environment and strategize effectively in response to changing market conditions.

“Navigating the Evolving Venture Capital Landscape for Edge AI Start-ups,” a ...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/advanced-presence-sensing-what-it-means-for-the-smart-home-a-presentation-from-omdia/ Jack Narcotta, Principal Analyst for the Smart Home at Omdia, presents the “Advanced Presence Sensing: What It Means for the Smart Home” tutorial at the May 2023 Embedded Vision Summit. Eventually, homes will become highly autonomous—powered by ubiquitous, connected, intelligent devices (sometimes referred to as ambient computing)—but this remains a distant vision. In the meantime, advanced presence sensing innovations from component manufacturers, software developers and smart home device companies are enabling meaningful improvements in convenience, efficiency and safety. Omdia expects advanced presence detection to be adopted rapidly in the coming years, and to become a key factor in how consumers select devices and service providers. In this talk, Narcotta provides an overview of key enabling technologies, market appetite, consumer expectations and long-term outlook for advanced presence sensing in the home.

“Advanced Presence Sensing: What It Means for the Smart Home,” a Presentation...

Edge AI and Vision Alliance

More from Edge AI and Vision Alliance (20)