SlideShare a Scribd company logo
Doing More with Less:
Optimizing Image Quality
and Stereo Depth at the
Edge
Tarik Loukili & Travis Davis
Automation & Autonomy
John Deere
Overview
• John Deere Background
• Image Quality Considerations
• Stereo Method Comparisons
• Q&A
© 2023 John Deere 2
Technology at John Deere
5
© 2023 John Deere
Low
Light
Color
Temp
Exposure
Control
Stuck
Material
Dust
Flare/
Glare
Motion
Blur
Image
Quality
Noise
[1]
[2]
© 2023 John Deere 6
What Makes A Good Image For
Computer Vision & Machine Learning?
• 120 ft boom
• 96 nozzles
• 34 cameras
• 5 controllers
• @ 12.5 mph
Configuration
© 2023 John Deere 7
See & Spray Select – Use Case
8
vs
Captured Expected
𝒎𝟏 𝒎𝟐 𝒎𝟑
𝒎𝟒 𝒎𝟓 𝒎𝟔
𝒎𝟕 𝒎𝟖 𝒎𝟗
+
𝒄𝟏
𝒄𝟐
𝒄𝟑
CCM
White
Balance
Bayer
Interpolation
CCM Gamma
Color
Conversion
© 2023 John Deere
Color Correction
© 2023 John Deere 9
[3]
Ambient Light Inconsistency
• The change of color temperature through the day will cause the image to appear differently at different
times of the day, hence make it challenging to develop an algorithm that deal with the color inconsistency.
© 2023 John Deere 10
[4]
Stuck Material On The Lens
• Stuck Material maybe hard to
detect especially when dealing with
translucent material.
• Not all stuck material
necessarily impact the image
processing.
Stuck Material On The Lens
© 2023 John Deere 11
• Computing a Stuck Material mask
Make it easier to estimate the
impact on the viewing ROI
• The impact may be different based
on application
© 2023 John Deere 12
Atlas Algolux [6] Visionary.AI
[5]
Dealing With Shadows
• Tunning the ISP settings may be
challenging or impossible to get
the optimum setting to deal
with issues such as shadows.
Left Image
Right Image
Disparity
Stereo Vision
© 2023 John Deere 13
(274, 133) (274, 133)
(242, 133)
(224, 133)
Left Image Right Image
Disparity: dx = xl-xr = 274-242 = 32
Distance: D = fB/dx = (focal length x baseline) /
disparity
[7]
© 2023 John Deere 14
Stereo Correspondence
[7]
© 2023 John Deere 15
Stereo Disparity Image
Various Baselines and Imagers
Various Image & Video Processors (FPGA or GPU)
Stereo At John Deere
© 2023 John Deere 16
OpenCV StereoBM [8]
Further
Away
Closer
Invalid (Dark
Blue)
Block Matching Example – Minimal Filtering
© 2023 John Deere 17
OpenCV StereoSGBM [9]
Further
Away
Closer
Invalid (Dark
Blue)
© 2023 John Deere 18
Semi-Global Block Matching
iRaftStereo_RVC [10]
Further
Away
Closer
© 2023 John Deere 19
Neural Net Based Stereo Approach
(Machine Learning – RAFT)
Alternate BM
Further
Away
Closer
Invalid (Dark
Blue)
© 2023 John Deere 20
Alternate Block Matching Example
Block Matching AI Matching (RAFT)
Pixel by Pixel Disparity Differences Disparity Differences > 2
Reference Image
21
Stereo Method Differences
RAFT Stereo (Different
Color)
OpenCV BM Alternate BM
Reference
© 2023 John Deere 22
Additional Examples with
Various Stereo Comparisons
• “Alternate BM” most efficient method & meets requirements for several John Deere applications.
• ML approach has higher precision capability, but greater chance of error for low confidence areas.
• Many use cases are already doing a Computer Vision or Machine Learning Algorithm On Image(s)
Conclusions & Recommendations
Strive to do More with Less
• Start with the Worst Case & Strive for the Best Case
(Data, Application Requirements, & Compute)
• Characterize Your Image Conditions for Outdoor
Environments
• Higher Computation Stereo Has More Resolution &
Accuracy Potential, But Also Potential for Error
• Higher Computation Stereo Improves Edges – Do
you use Depth for your Edges?
© 2023 John Deere 23
Image Quality References:
1. https://www.adobe.com/uk/creativecloud/photography/discover/color-changer.html
2. https://www.photoreview.com.au/tips/shooting/iso-and-image-quality/
3. ttps://www.lumistrips.com/lumistrips-blog/color-temperature-explained/
4. www.lensrentals.com
5. https://algolux.com/newsroom/algolux-brings-atlas-to-the-cloud-to-democratize-camera-isp-
optimization-for-computer-vision/
6. https://www.visionary.ai/
Stereo References:
7. Stereo Reference Example Image from https://vision.middlebury.edu/stereo/data/scenes2003/
8. OpenCV Stereo Block Matching from opencv/stereobm.cpp
9. OpenCV Semi Global Block Matching from opencv/stereosgbm.cpp
10. RAFT Stereo from princeton-vl/RAFT-Stereo
Special “Thanks” for Content & Contributions
Zach Bonefas, Nick Butts, Ruveen Perera, & Vincenzo Macri
© 2023 John Deere 24
References

More Related Content

Similar to "Optimizing Image Quality and Stereo Depth at the Edge," a Presentation from John Deere

"Introduction to Optics for Embedded Vision," a Presentation from Jessica Geh...
"Introduction to Optics for Embedded Vision," a Presentation from Jessica Geh..."Introduction to Optics for Embedded Vision," a Presentation from Jessica Geh...
"Introduction to Optics for Embedded Vision," a Presentation from Jessica Geh...
Edge AI and Vision Alliance
 
“Building an Autonomous Detect-and-Avoid System for Commercial Drones,” a Pre...
“Building an Autonomous Detect-and-Avoid System for Commercial Drones,” a Pre...“Building an Autonomous Detect-and-Avoid System for Commercial Drones,” a Pre...
“Building an Autonomous Detect-and-Avoid System for Commercial Drones,” a Pre...
Edge AI and Vision Alliance
 
Ahmed Elmalla ECE Bonding Vision System proposal by Python & Computer science...
Ahmed Elmalla ECE Bonding Vision System proposal by Python & Computer science...Ahmed Elmalla ECE Bonding Vision System proposal by Python & Computer science...
Ahmed Elmalla ECE Bonding Vision System proposal by Python & Computer science...
Ahmed Elmalla
 
Croma_key_Report.pdf
Croma_key_Report.pdfCroma_key_Report.pdf
Croma_key_Report.pdf
GauriHadgekar
 
Digital to Physical: 3D Printing for Diverse Sectors
Digital to Physical: 3D Printing for Diverse SectorsDigital to Physical: 3D Printing for Diverse Sectors
Digital to Physical: 3D Printing for Diverse SectorsWMG, University of Warwick
 
Unmanned Aircraft System (UAS) 3D Product Comparisons to Airborne LiDAR
Unmanned Aircraft System (UAS) 3D Product Comparisons to Airborne LiDARUnmanned Aircraft System (UAS) 3D Product Comparisons to Airborne LiDAR
Unmanned Aircraft System (UAS) 3D Product Comparisons to Airborne LiDAR
Merrick & Company
 
Unmanned Aerial Systems (UAS) Data Quality and Accuracy Realities
Unmanned Aerial Systems (UAS) Data Quality and Accuracy RealitiesUnmanned Aerial Systems (UAS) Data Quality and Accuracy Realities
Unmanned Aerial Systems (UAS) Data Quality and Accuracy Realities
UAS Colorado
 
“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...
“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...
“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...
Edge AI and Vision Alliance
 
Designing and Specifying Display Systems
Designing and Specifying Display SystemsDesigning and Specifying Display Systems
Designing and Specifying Display SystemsrAVe [PUBS]
 
"Computer-vision-based 360-degree Video Systems: Architectures, Algorithms an...
"Computer-vision-based 360-degree Video Systems: Architectures, Algorithms an..."Computer-vision-based 360-degree Video Systems: Architectures, Algorithms an...
"Computer-vision-based 360-degree Video Systems: Architectures, Algorithms an...
Edge AI and Vision Alliance
 
“Toward the Era of AI Everywhere,” a Presentation from DEEPX
“Toward the Era of AI Everywhere,” a Presentation from DEEPX“Toward the Era of AI Everywhere,” a Presentation from DEEPX
“Toward the Era of AI Everywhere,” a Presentation from DEEPX
Edge AI and Vision Alliance
 
Amcrest ProHD
Amcrest ProHDAmcrest ProHD
Amcrest ProHD
carquarters01
 
"Introduction to Optics for Embedded Vision," a Presentation from Edmund Optics
"Introduction to Optics for Embedded Vision," a Presentation from Edmund Optics"Introduction to Optics for Embedded Vision," a Presentation from Edmund Optics
"Introduction to Optics for Embedded Vision," a Presentation from Edmund Optics
Edge AI and Vision Alliance
 
bro_geoslam-introduction.pdf
bro_geoslam-introduction.pdfbro_geoslam-introduction.pdf
bro_geoslam-introduction.pdf
YoussefZghari
 
DrTAD BLENDER CYCLES Tutorial – Glass Material BSDF + Reducing Noise
DrTAD BLENDER CYCLES Tutorial – Glass Material BSDF + Reducing NoiseDrTAD BLENDER CYCLES Tutorial – Glass Material BSDF + Reducing Noise
DrTAD BLENDER CYCLES Tutorial – Glass Material BSDF + Reducing Noise
FIDE Master Tihomir Dovramadjiev PhD
 
"Selecting the Right Imager for Your Embedded Vision Application," a Presenta...
"Selecting the Right Imager for Your Embedded Vision Application," a Presenta..."Selecting the Right Imager for Your Embedded Vision Application," a Presenta...
"Selecting the Right Imager for Your Embedded Vision Application," a Presenta...
Edge AI and Vision Alliance
 
“Five Things You Might Overlook on Your Next Vision-enabled Product Design,” ...
“Five Things You Might Overlook on Your Next Vision-enabled Product Design,” ...“Five Things You Might Overlook on Your Next Vision-enabled Product Design,” ...
“Five Things You Might Overlook on Your Next Vision-enabled Product Design,” ...
Edge AI and Vision Alliance
 
Real-time Screen-space Geometry Draping for 3D Digital Terrain Models
Real-time Screen-space Geometry Draping for 3D Digital Terrain ModelsReal-time Screen-space Geometry Draping for 3D Digital Terrain Models
Real-time Screen-space Geometry Draping for 3D Digital Terrain Models
Matthias Trapp
 
Dynamic Resolution Techniques for Intel® Processor Graphics | SIGGRAPH 2018 T...
Dynamic Resolution Techniques for Intel® Processor Graphics | SIGGRAPH 2018 T...Dynamic Resolution Techniques for Intel® Processor Graphics | SIGGRAPH 2018 T...
Dynamic Resolution Techniques for Intel® Processor Graphics | SIGGRAPH 2018 T...
Intel® Software
 
2018 intro machine_vision-webinar (1)
2018 intro machine_vision-webinar (1)2018 intro machine_vision-webinar (1)
2018 intro machine_vision-webinar (1)
طه محمد
 

Similar to "Optimizing Image Quality and Stereo Depth at the Edge," a Presentation from John Deere (20)

"Introduction to Optics for Embedded Vision," a Presentation from Jessica Geh...
"Introduction to Optics for Embedded Vision," a Presentation from Jessica Geh..."Introduction to Optics for Embedded Vision," a Presentation from Jessica Geh...
"Introduction to Optics for Embedded Vision," a Presentation from Jessica Geh...
 
“Building an Autonomous Detect-and-Avoid System for Commercial Drones,” a Pre...
“Building an Autonomous Detect-and-Avoid System for Commercial Drones,” a Pre...“Building an Autonomous Detect-and-Avoid System for Commercial Drones,” a Pre...
“Building an Autonomous Detect-and-Avoid System for Commercial Drones,” a Pre...
 
Ahmed Elmalla ECE Bonding Vision System proposal by Python & Computer science...
Ahmed Elmalla ECE Bonding Vision System proposal by Python & Computer science...Ahmed Elmalla ECE Bonding Vision System proposal by Python & Computer science...
Ahmed Elmalla ECE Bonding Vision System proposal by Python & Computer science...
 
Croma_key_Report.pdf
Croma_key_Report.pdfCroma_key_Report.pdf
Croma_key_Report.pdf
 
Digital to Physical: 3D Printing for Diverse Sectors
Digital to Physical: 3D Printing for Diverse SectorsDigital to Physical: 3D Printing for Diverse Sectors
Digital to Physical: 3D Printing for Diverse Sectors
 
Unmanned Aircraft System (UAS) 3D Product Comparisons to Airborne LiDAR
Unmanned Aircraft System (UAS) 3D Product Comparisons to Airborne LiDARUnmanned Aircraft System (UAS) 3D Product Comparisons to Airborne LiDAR
Unmanned Aircraft System (UAS) 3D Product Comparisons to Airborne LiDAR
 
Unmanned Aerial Systems (UAS) Data Quality and Accuracy Realities
Unmanned Aerial Systems (UAS) Data Quality and Accuracy RealitiesUnmanned Aerial Systems (UAS) Data Quality and Accuracy Realities
Unmanned Aerial Systems (UAS) Data Quality and Accuracy Realities
 
“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...
“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...
“Machine Learning for the Real World: What is Acceptable Accuracy, and How Ca...
 
Designing and Specifying Display Systems
Designing and Specifying Display SystemsDesigning and Specifying Display Systems
Designing and Specifying Display Systems
 
"Computer-vision-based 360-degree Video Systems: Architectures, Algorithms an...
"Computer-vision-based 360-degree Video Systems: Architectures, Algorithms an..."Computer-vision-based 360-degree Video Systems: Architectures, Algorithms an...
"Computer-vision-based 360-degree Video Systems: Architectures, Algorithms an...
 
“Toward the Era of AI Everywhere,” a Presentation from DEEPX
“Toward the Era of AI Everywhere,” a Presentation from DEEPX“Toward the Era of AI Everywhere,” a Presentation from DEEPX
“Toward the Era of AI Everywhere,” a Presentation from DEEPX
 
Amcrest ProHD
Amcrest ProHDAmcrest ProHD
Amcrest ProHD
 
"Introduction to Optics for Embedded Vision," a Presentation from Edmund Optics
"Introduction to Optics for Embedded Vision," a Presentation from Edmund Optics"Introduction to Optics for Embedded Vision," a Presentation from Edmund Optics
"Introduction to Optics for Embedded Vision," a Presentation from Edmund Optics
 
bro_geoslam-introduction.pdf
bro_geoslam-introduction.pdfbro_geoslam-introduction.pdf
bro_geoslam-introduction.pdf
 
DrTAD BLENDER CYCLES Tutorial – Glass Material BSDF + Reducing Noise
DrTAD BLENDER CYCLES Tutorial – Glass Material BSDF + Reducing NoiseDrTAD BLENDER CYCLES Tutorial – Glass Material BSDF + Reducing Noise
DrTAD BLENDER CYCLES Tutorial – Glass Material BSDF + Reducing Noise
 
"Selecting the Right Imager for Your Embedded Vision Application," a Presenta...
"Selecting the Right Imager for Your Embedded Vision Application," a Presenta..."Selecting the Right Imager for Your Embedded Vision Application," a Presenta...
"Selecting the Right Imager for Your Embedded Vision Application," a Presenta...
 
“Five Things You Might Overlook on Your Next Vision-enabled Product Design,” ...
“Five Things You Might Overlook on Your Next Vision-enabled Product Design,” ...“Five Things You Might Overlook on Your Next Vision-enabled Product Design,” ...
“Five Things You Might Overlook on Your Next Vision-enabled Product Design,” ...
 
Real-time Screen-space Geometry Draping for 3D Digital Terrain Models
Real-time Screen-space Geometry Draping for 3D Digital Terrain ModelsReal-time Screen-space Geometry Draping for 3D Digital Terrain Models
Real-time Screen-space Geometry Draping for 3D Digital Terrain Models
 
Dynamic Resolution Techniques for Intel® Processor Graphics | SIGGRAPH 2018 T...
Dynamic Resolution Techniques for Intel® Processor Graphics | SIGGRAPH 2018 T...Dynamic Resolution Techniques for Intel® Processor Graphics | SIGGRAPH 2018 T...
Dynamic Resolution Techniques for Intel® Processor Graphics | SIGGRAPH 2018 T...
 
2018 intro machine_vision-webinar (1)
2018 intro machine_vision-webinar (1)2018 intro machine_vision-webinar (1)
2018 intro machine_vision-webinar (1)
 

More from Edge AI and Vision Alliance

“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
"OpenCV for High-performance, Low-power Vision Applications on Snapdragon," a...
"OpenCV for High-performance, Low-power Vision Applications on Snapdragon," a..."OpenCV for High-performance, Low-power Vision Applications on Snapdragon," a...
"OpenCV for High-performance, Low-power Vision Applications on Snapdragon," a...
Edge AI and Vision Alliance
 
“Deploying Large Models on the Edge: Success Stories and Challenges,” a Prese...
“Deploying Large Models on the Edge: Success Stories and Challenges,” a Prese...“Deploying Large Models on the Edge: Success Stories and Challenges,” a Prese...
“Deploying Large Models on the Edge: Success Stories and Challenges,” a Prese...
Edge AI and Vision Alliance
 
“Scaling Vision-based Edge AI Solutions: From Prototype to Global Deployment,...
“Scaling Vision-based Edge AI Solutions: From Prototype to Global Deployment,...“Scaling Vision-based Edge AI Solutions: From Prototype to Global Deployment,...
“Scaling Vision-based Edge AI Solutions: From Prototype to Global Deployment,...
Edge AI and Vision Alliance
 
“What’s Next in On-device Generative AI,” a Presentation from Qualcomm
“What’s Next in On-device Generative AI,” a Presentation from Qualcomm“What’s Next in On-device Generative AI,” a Presentation from Qualcomm
“What’s Next in On-device Generative AI,” a Presentation from Qualcomm
Edge AI and Vision Alliance
 
“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...
“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...
“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...
Edge AI and Vision Alliance
 
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...
Edge AI and Vision Alliance
 
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...
Edge AI and Vision Alliance
 
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...
Edge AI and Vision Alliance
 
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
Edge AI and Vision Alliance
 
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...
Edge AI and Vision Alliance
 
“Vision-language Representations for Robotics,” a Presentation from the Unive...
“Vision-language Representations for Robotics,” a Presentation from the Unive...“Vision-language Representations for Robotics,” a Presentation from the Unive...
“Vision-language Representations for Robotics,” a Presentation from the Unive...
Edge AI and Vision Alliance
 
“ADAS and AV Sensors: What’s Winning and Why?,” a Presentation from TechInsights
“ADAS and AV Sensors: What’s Winning and Why?,” a Presentation from TechInsights“ADAS and AV Sensors: What’s Winning and Why?,” a Presentation from TechInsights
“ADAS and AV Sensors: What’s Winning and Why?,” a Presentation from TechInsights
Edge AI and Vision Alliance
 
“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...
“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...
“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...
Edge AI and Vision Alliance
 
“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...
“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...
“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...
Edge AI and Vision Alliance
 
“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...
“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...
“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...
Edge AI and Vision Alliance
 
“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...
“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...
“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...
Edge AI and Vision Alliance
 
“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...
“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...
“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...
Edge AI and Vision Alliance
 
“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...
“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...
“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...
Edge AI and Vision Alliance
 
“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...
“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...
“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...
Edge AI and Vision Alliance
 

More from Edge AI and Vision Alliance (20)

“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
"OpenCV for High-performance, Low-power Vision Applications on Snapdragon," a...
"OpenCV for High-performance, Low-power Vision Applications on Snapdragon," a..."OpenCV for High-performance, Low-power Vision Applications on Snapdragon," a...
"OpenCV for High-performance, Low-power Vision Applications on Snapdragon," a...
 
“Deploying Large Models on the Edge: Success Stories and Challenges,” a Prese...
“Deploying Large Models on the Edge: Success Stories and Challenges,” a Prese...“Deploying Large Models on the Edge: Success Stories and Challenges,” a Prese...
“Deploying Large Models on the Edge: Success Stories and Challenges,” a Prese...
 
“Scaling Vision-based Edge AI Solutions: From Prototype to Global Deployment,...
“Scaling Vision-based Edge AI Solutions: From Prototype to Global Deployment,...“Scaling Vision-based Edge AI Solutions: From Prototype to Global Deployment,...
“Scaling Vision-based Edge AI Solutions: From Prototype to Global Deployment,...
 
“What’s Next in On-device Generative AI,” a Presentation from Qualcomm
“What’s Next in On-device Generative AI,” a Presentation from Qualcomm“What’s Next in On-device Generative AI,” a Presentation from Qualcomm
“What’s Next in On-device Generative AI,” a Presentation from Qualcomm
 
“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...
“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...
“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...
 
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...
 
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...
 
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...
 
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
 
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...
 
“Vision-language Representations for Robotics,” a Presentation from the Unive...
“Vision-language Representations for Robotics,” a Presentation from the Unive...“Vision-language Representations for Robotics,” a Presentation from the Unive...
“Vision-language Representations for Robotics,” a Presentation from the Unive...
 
“ADAS and AV Sensors: What’s Winning and Why?,” a Presentation from TechInsights
“ADAS and AV Sensors: What’s Winning and Why?,” a Presentation from TechInsights“ADAS and AV Sensors: What’s Winning and Why?,” a Presentation from TechInsights
“ADAS and AV Sensors: What’s Winning and Why?,” a Presentation from TechInsights
 
“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...
“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...
“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...
 
“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...
“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...
“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...
 
“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...
“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...
“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...
 
“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...
“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...
“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...
 
“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...
“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...
“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...
 
“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...
“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...
“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...
 
“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...
“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...
“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...
 

Recently uploaded

By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024
Pierluigi Pugliese
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
UiPathCommunity
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
DianaGray10
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
sonjaschweigert1
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
Product School
 
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfSAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
Peter Spielvogel
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionGenerative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Aggregage
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Paige Cruz
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
OnBoard
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
91mobiles
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
BookNet Canada
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Product School
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
Ralf Eggert
 

Recently uploaded (20)

By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
 
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
 
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfSAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionGenerative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
 

"Optimizing Image Quality and Stereo Depth at the Edge," a Presentation from John Deere

  • 1. Doing More with Less: Optimizing Image Quality and Stereo Depth at the Edge Tarik Loukili & Travis Davis Automation & Autonomy John Deere
  • 2. Overview • John Deere Background • Image Quality Considerations • Stereo Method Comparisons • Q&A © 2023 John Deere 2
  • 3. Technology at John Deere 5 © 2023 John Deere
  • 5. • 120 ft boom • 96 nozzles • 34 cameras • 5 controllers • @ 12.5 mph Configuration © 2023 John Deere 7 See & Spray Select – Use Case
  • 6. 8 vs Captured Expected 𝒎𝟏 𝒎𝟐 𝒎𝟑 𝒎𝟒 𝒎𝟓 𝒎𝟔 𝒎𝟕 𝒎𝟖 𝒎𝟗 + 𝒄𝟏 𝒄𝟐 𝒄𝟑 CCM White Balance Bayer Interpolation CCM Gamma Color Conversion © 2023 John Deere Color Correction
  • 7. © 2023 John Deere 9 [3] Ambient Light Inconsistency • The change of color temperature through the day will cause the image to appear differently at different times of the day, hence make it challenging to develop an algorithm that deal with the color inconsistency.
  • 8. © 2023 John Deere 10 [4] Stuck Material On The Lens • Stuck Material maybe hard to detect especially when dealing with translucent material. • Not all stuck material necessarily impact the image processing.
  • 9. Stuck Material On The Lens © 2023 John Deere 11 • Computing a Stuck Material mask Make it easier to estimate the impact on the viewing ROI • The impact may be different based on application
  • 10. © 2023 John Deere 12 Atlas Algolux [6] Visionary.AI [5] Dealing With Shadows • Tunning the ISP settings may be challenging or impossible to get the optimum setting to deal with issues such as shadows.
  • 11. Left Image Right Image Disparity Stereo Vision © 2023 John Deere 13
  • 12. (274, 133) (274, 133) (242, 133) (224, 133) Left Image Right Image Disparity: dx = xl-xr = 274-242 = 32 Distance: D = fB/dx = (focal length x baseline) / disparity [7] © 2023 John Deere 14 Stereo Correspondence
  • 13. [7] © 2023 John Deere 15 Stereo Disparity Image
  • 14. Various Baselines and Imagers Various Image & Video Processors (FPGA or GPU) Stereo At John Deere © 2023 John Deere 16
  • 15. OpenCV StereoBM [8] Further Away Closer Invalid (Dark Blue) Block Matching Example – Minimal Filtering © 2023 John Deere 17
  • 16. OpenCV StereoSGBM [9] Further Away Closer Invalid (Dark Blue) © 2023 John Deere 18 Semi-Global Block Matching
  • 17. iRaftStereo_RVC [10] Further Away Closer © 2023 John Deere 19 Neural Net Based Stereo Approach (Machine Learning – RAFT)
  • 18. Alternate BM Further Away Closer Invalid (Dark Blue) © 2023 John Deere 20 Alternate Block Matching Example
  • 19. Block Matching AI Matching (RAFT) Pixel by Pixel Disparity Differences Disparity Differences > 2 Reference Image 21 Stereo Method Differences
  • 20. RAFT Stereo (Different Color) OpenCV BM Alternate BM Reference © 2023 John Deere 22 Additional Examples with Various Stereo Comparisons • “Alternate BM” most efficient method & meets requirements for several John Deere applications. • ML approach has higher precision capability, but greater chance of error for low confidence areas. • Many use cases are already doing a Computer Vision or Machine Learning Algorithm On Image(s)
  • 21. Conclusions & Recommendations Strive to do More with Less • Start with the Worst Case & Strive for the Best Case (Data, Application Requirements, & Compute) • Characterize Your Image Conditions for Outdoor Environments • Higher Computation Stereo Has More Resolution & Accuracy Potential, But Also Potential for Error • Higher Computation Stereo Improves Edges – Do you use Depth for your Edges? © 2023 John Deere 23
  • 22. Image Quality References: 1. https://www.adobe.com/uk/creativecloud/photography/discover/color-changer.html 2. https://www.photoreview.com.au/tips/shooting/iso-and-image-quality/ 3. ttps://www.lumistrips.com/lumistrips-blog/color-temperature-explained/ 4. www.lensrentals.com 5. https://algolux.com/newsroom/algolux-brings-atlas-to-the-cloud-to-democratize-camera-isp- optimization-for-computer-vision/ 6. https://www.visionary.ai/ Stereo References: 7. Stereo Reference Example Image from https://vision.middlebury.edu/stereo/data/scenes2003/ 8. OpenCV Stereo Block Matching from opencv/stereobm.cpp 9. OpenCV Semi Global Block Matching from opencv/stereosgbm.cpp 10. RAFT Stereo from princeton-vl/RAFT-Stereo Special “Thanks” for Content & Contributions Zach Bonefas, Nick Butts, Ruveen Perera, & Vincenzo Macri © 2023 John Deere 24 References