Infrared sensors and LIDAR for ADAS and autonomous vehicles

www.osram-os.com
Infrared sensors for ADAS and beyond –
LIDAR / Infrared camera
Rajeev Thakur| 4th October 2016| Novi
Light is OSRAM

IR for ADAS | OS IR NA MK | R.Thakur
TU Automotive – ADAS & Autonomous | 10/04/2016
2
Content
Page
1. OSRAM Overview 03
2. Sensing challenges 06
3. LIDAR 12
4. Infrared Camera 19
5. Sensor Fusion 21
6. Collaboration & Competition in the self driving car business 22

3
Global Market Leader in LED & Laser
LIDAR – Infrared Lasers - AEB
Consumer
Industry
General Lighting
Laser front light
Xenon front light Laser front light
OLED rear light
Matrix LED light
Automotive Lighting
Source: OSRAM, excluding LAMPS
1) at the end of the fiscal year
2) countries where OSRAM had operations at the end of the fiscal year
Employees1) : 20,300
Worldwide Presence2) : >120 countries
Revenue1): 3,571.9 m €

4
Key Automotive Trends
Exterior , Interior & IR
Safety
Design
Visualization &
Connectivity
Comfort &
Safety
Key Automotive Trends
ExteriorInterior
• µAFS
• High Luminance
LEDs
Dynamic
Lighting
Projection
HuD
Full Digital Cluster
LED Applications New LED
Development
• Display Portfolio
• HuD Portfolio
BLU Displays
High ResolutionADB/Matrix
ProjectionUltra slim HL
LIDAR / ADB
Gesture
Wireless
Connectivity
Driver Monitoring
Night Vision

5
OSRAM Infrared & Laser Automotive Applications
Existing Applications / New Applications
Rain Light
Tunnel Sensors
Ambient light sensors for dimming
and illumination
• Dashboard
• Car radio
• Displays
Immobilizer
Steering wheel
angle sensor
Blue Lasers for
Headlamps
Driver monitoring
Gesture Recognition
IRED based Night vision
Blind spot detection
Lane departure warning
Family Entertainment
System
LIDAR sensing
AEB & ADAS
Laser HuD

6
Sensing Needs for Vehicle Environment
Continue
Brake
Steer to safety Prepare for crash
Steering is
best option Cannot avoid
crash
Braking is
best option
NO
YES
Current Sensing Range
Upper limit
(For Large Objects)
RADAR : 50 - 250m
Camera : 50 - 70m
LIDAR : 50 – 200m
Braking Distance / Minimum Sensing Range
(Assumptions : Dry Road, µ = 0.7, 1 sec reaction time)
@100mph (161kph/44.7m/s) : > 190 meters
@74mph (119kph,33 m/s) : > 112 meters
@45mph (72.4kph/20.1 m/s) : > 50 meters
@25mph (40kph/11.1 m/s) : > 20 meters
If the closing speed is less than ~ 45mph , current sensing technology can mitigate
collision to large objects under normal daylight dry conditions (distance < 70 meters)
Challenge 1 : Sensing Range
Is projected vehicle
trajectory safe for next
XX meters?
Calculate Time
to Crash

7
• Who decides ?
• Ignore objects at own risk ..
Challenge 2 : Angular Sensing Resolution
• Standard object list for detection does not exist (ignore / standardize with risk)
• LIDAR is capable of < 0.5° resolution at > 100 m (with small form factor)
• RADAR size for 0.5° resolution not practical (~ 0.5 m for 76 GHz RADAR)
• Camera range needs to improve & image quality in lowlight (or infrared)
1 Bosch Multi Purpose Camera (MPC2) , 1280 x 960 pixels, 50° HFOV, 28° VFOV 2 Velodyne VLP16 (0.1° – 0.4°) 3 RADAR equation
What objects should be detected to avoid collision ?
Typical Angular Resolution
1 Camera : 25 pixels / °
2 LIDAR : 0.3°
3 RADAR : 2.6° (76 GHz, 10 cm aperture)
1.5m 0.25m 0.4m 0.2m 0.1m 0.2m
tire
piece
potholedog Resolution Size (m)
1° @ 100 m = > 1.7 m
1° @ 200 m = > 3.4 m
0.1° @ 200 m = > 0.4 m

8
Challenge 3 : Field of View
Winding Roads
Need Wide FOV
Traffic Lights & Overhead Signs Need High FOV Up & Down Ramps Need High FOV
FOV – Field Of View

9
Challenge 4 : Computational challenges
• Time Needed = Sensing time + Reaction Time + Safety Margin
• Sensing Time per Sensor = (Points/Frame x # of Frames in Buffer x
compute time/point)
• Finer resolution => More data points => more time (or faster
computation)
• Redundancy / sensor fusion needed prior to reaction
• Reaction Time = (Human delay) + latency in steering or braking
system
• Safety Margin : To accommodate environment conditions (road /
temperature) , sensing and computational delays and tolerances
1 Can we unify monocular detectors for autonomous driving by using the pixel-wise semantic segmentation of CNNs? : Department of Electronics, UAH. Alcala de ´
Henares, Madrid, Spain ; IEEE Workshop in June 2016 on Intelligent Vehicles
1
Sensor / Processor / environment / algorithm .. affects
computational time and accuracy

10
Other Challenges
• Form factor – small and compatible to vehicle styling & materials
• Increasing noise from surrounding vehicle RADAR/LIDAR ..
• Dealing with satellite signal / GPS loss in real time
• Harsh environment – Snow/rain/dust/dirt/shock and vibrations
• Power / EMC / ESD / ..
• Service
• Cost
• ..
• Tremendous innovation currently in sensing field
• OSRAM working with multiple startups / Tier1 and OEM’s

11
RADAR / Camera / LIDAR Comparison
Sensor Typical
Range
Horizontal
FOV
Vertical
FOV
2020
Price Range
Comments
24 GHz
RADAR
60 m 1 56° 1 ~ ± 20° < $100
USA Bandwidth 100 -250 MHz 2
Robust for Rain/snow ;
People Detection / Angular Resolution
77 GHz
RADAR
200 m 1 18° 1 ~ ± 5° < $100
USA Bandwidth 600 MHz 2
Robust for Rain/snow ;
People Detection / Angular Resolution
Front Mono
Camera
50 m 1 36° 1 ~ ± 14 ° < $100
Versatile Sensor (Applications)
Limited depth perception ; affected by rain / fog
Needs illumination (Visible/IR)
LIDAR
(Flash)
75 m 140° ~ ± 5° < $100
Concerns for Rain/Snow;
Good reflection off people w/ angular resolution
Range & S/N limited by eye safety
LIDAR
(Scanning)
200 m 360° ~ ± 14° < $500
Concerns for Rain/Snow;
Typically higher price for angular resolution
Range & S/N limited by eye safety
1 : Vehicle-to-Vehicle Communications: readiness of V2V technology for application – DOT HS 812014 ; Table V-7
2 : Millimeter Wave Receiver concepts for 77 GHz automotive radar in silicon Germanium Technology – D.Kissenger (SpringerBrief’s 2012)
• False positives  Nuisance to consumer  Turns feature off (if possible)
• False negatives  did not meet spec / expectations
• Optimum combination of sensors will be a learning process
• Sensor fusion …can be done at best on common subset in field of view

12
Flash LIDAR – Design Overview
start
stop
Δt
Laser
Photodiode
Array
Emitter
Lens
Receiving
Lens
Target
FOV
Working principle : Laser beam spread into field of view and received on photodiode array. Range
determined by eye safe laser power , resolution determined by number of photodiode pixels
Why use : Mature low cost sensor that can be integrated into headlamp / Tail lamp / behind windshield / ..
Range : ~ 30 - 60 meters @ 24 HFOV
Resolution : 3 deg or less
Wavelength : 905 nm has proven sufficient for short range
Laser : OSRAM lasers with peak power 75 – 120W , with & without drivers , bare die to SMT w/ < 5ns pulse
width (2019 SOP) , also with multiple emitters in one SMT package
Photodiode : OSRAM PD array concepts of various sizes planned for SOP 2018
Why not as popular as RADAR yet in NAFTA?: 2019 NCAP upgrade will incentivize market , more room
for creativity lower cost than RADAR ; market waiting for low cost scanning LIDAR …
R – Distance
C – speed of light
Δt – time between start - stop of pulse
HFOV – Horizontal Field Of View
SOP – Start of Production
PD – Photo Diode

13
LIDAR Head Lamp Integration – LeddarTech Concept
Leddartech Video link

14
LIDAR Tail Lamp Integration – LeddarTech Concept

15
Phantom Intelligence : Guardian Flash LIDAR
The Guardian
BY PHANTOM INTELLIGENCE
Fully customizable
2x8 Pixels (1x16 also available)
Field of View 9°x36° – Customizable up to 2°x120°
Range limited to 30 meters (for cost optimization)
Connectivity: USB, CAN, GPIO
Programmable alarms/triggers
Power Consumption less than 3 Watts
Laser Output of 70 Watts
Eye Safe (Class 1M)
Price: ~ 100$ in 10k units volume production
Engineering Samples  December 2016
AWL Video link
YOU CAN AFFORD THE SAFEST JOURNEY !

16
LIDAR – Low Cost Concept (Reference Design)
• 30m range
• ~1cm accuracy
• 16 pixel array
• 24°H x 6°V FOV : 2 x 8 array (3° x 3°per pixel)
• Arrangement of pixels and field of view can be customized in future products.
• Multiple targets in each pixel can be resolved
• Targets ~1m apart (range) can be separated
• Differentiating through performance, small size,
scalability, and low power consumption
• No moving (scanning) parts
• Sun blinding can affect no more than a single pixel
• Estimated BOM ~ $25 @ High Volume
• Functional sample Q1 2017
• Target SOP 2019
Distance (m) Area (m²)
1 0.003
2 0.011
5 0.069
10 0.274
20 1.097
30 2.469
Field Of View Per Pixel
FOV – Horizontal Field Of View
SOP – Start of Production
BOM – Bill of Material (For Hard Ware)

17
Scanning LIDAR Technologies
Mechanical - Velodyne
Principle: Matched pair of laser &
detectors rotating with a motor at 5
to 20 Hz
Range : 200 m (VLP 32A)
Resolution : 0.1 - 0.4° (VLP 16)
Vertical FOV: 28° (VLP 32A)
Price Target : < $500 ~ 2020
Pro : Proven technology
Con: Mechanical integration / price
Principle: Laser scanned with
OPA (& received on SPAD array )
Range : > 150 m
Resolution : 0.1°
FOV: 120° (HFOV & VFOV ; S3)
Price Target : <$100 ~ 2020
Pro : small size (1” x 1.5” , S3-Qi)
Con: OPA scanning is relatively
new technology
1 Velodyne.com 2 Quanergy.com 3 Innoluce.com
OPA - Quanergy MEMS – Innoluce
Principle: Laser scanned with 1D
MEMS Mirror (& received on APD
array )
Range : > 200 m
Resolution : < 0.5°
HFOV: 80°
VFOV: 16°
Price Target : <$100 ~ 2020
Pro : MEMS scanning is proven
Con: Working demo not shown
yet…
MEMS – Micro Electro Mechanical Systems
APD – Avalanche Photo Diode
OPA – Optical Phase Array
SPAD – Single Photon Avalanche Diodes
FOV – Field Of View

18
MEMS LIDAR – Innoluce – OSRAM : Concept Proof
Principle: 1 25W OSRAM laser scanned with 1 Innoluce MEMS Mirror and received on an APD array
Range : ~ 60 m
Resolution : 0.1° Horizontal and 0.2° Vertical
HFOV: 10°
VFOV: 3°
Next Steps :
• Show progressively improved reference design demos in next few months
• Targets : >200 m/car ; > 60m/Ped ; 80° HFOV ; 16° VFOV ; < 0.5° Resolution (High power multiple emitter lasers)
High resolution concept proof
MEMS LIDAR Video Link

19
Infrared Camera - Interior
Mature applications transitioning to mainstream
• Why IR Camera : Works in day & night without visible illumination
• Moving to Mainstream : Driver monitoring (Drowsy/Distracted)
• Catching speed : Gesture recognition
• Mobile to Automotive : Iris recognition
• Technology frontiers: NIR sensitivity (15 – 35%), > 2Mp Global shutter
, increasing IRED o/p & efficiency
• Concern/Tradeoffs: Privacy Vs App. value , Redglow (850  940 nm)
• Future applications : Optimum airbag deployment, Mood lighting ..
Driver Monitoring
1 Deltaid.com
Gesture Control
Iris Recognition
1

20
Infrared Camera - Exterior
Forward Camera
Surround View
Camera
Rearview
Camera
Cars have to be autonomous at night also …Cameras need to work with IR also ..
• Why IR Exterior Camera : Need to see adjacent lanes at night w/o visible light
• What’s the problem : Visible cameras block IR for better image / use of color information
• Options : Use mechanical or SW filter to switch between IR & visible spectrums
• Challenges : Modify camera / Illumination / SW for wider FOV and range
• Things to watch out for : Laser beam headlamps (Dynamic range of oncoming camera) /
Use of matrix beam lighting (adaptive beams ..)
IR emitters in headlamp

21
Sensor Fusion – Fuse Information of high quality which overlap
• Objective of Sensor Fusion : Determine environment around vehicle trajectory
with enough resolution, confidence and speed - to navigate efficiently.
Object_list RADAR Camera LIDAR Sensor Fusion
Car@150m
Don’t See it
(Noise)
Not_Classified@100
m & low light
Evaluate TTC &
brake if unresolved ?
@50m Person on bicycle
Not classified Don’t see it (Noise) Brake or ignore ?
Potholes & stuff
What can be safely
ignored ?
• Object Identification & Classification in range & FOV of interest must be comparable.
• LIDAR + Camera fusion potentially better (Due to angular resolution)
• Camera improvements : Range (~ 70 m); speed (30 – 60 Hz) & Low light sensitivity

22
Collaboration & Competition - Self Driving Cars
• Why collaborate :
• Need NHTSA support  Regulations / Testing / Infrastructure / ..
• Combine R&D resources & strengths
• Be / be with a technology leader to gain market share
• Why Compete?
• Branding / Technology Leadership (Intangible $ Value)
• ADAS technology has shown real market value ($1500 - $3000/car)
• Prepare for future market changes (Self driving cars occupy significant share)
• What more could/should be done ?
• Use Silicon Valley playbook more – open source development
• Example : Provide raw data from all sensors in a drive ; show me object
identification / classification & tracking .. (Buy the best solution..)
• Make sensor requirements and roadmap open
• Small startups have very creative solutions & fast development
• Why be more open ?
• 1 year after a new gadget is shown  3 more appear next year (Benefit/Cost)
• Will enable faster development of SDC technology for community & save lives !

www.osram-os.com
Thank you !
Contact :
Rajeev.Thakur@osram-os.com

Infrared sensors and LIDAR for ADAS and autonomous vehicles

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to Infrared sensors and LIDAR for ADAS and autonomous vehicles

Similar to Infrared sensors and LIDAR for ADAS and autonomous vehicles (20)

Infrared sensors and LIDAR for ADAS and autonomous vehicles