TII_Thierry_LESTABLE_WCNC_2022_v10_Short.pdf

tii.ae
A.I & Telecom for Road Safety:
Unlocking the Potential of Sustainable Smart Cities (SSC)
Thierry LESTABLE, Ph.D
Executive Director Digital Science Research Center (DSRC)
Acting Executive Director, Artificial Intelligence Cross-Center Unit
Source: Nvidia WCNC 2022, Austin, US

TII – Technology Innovation Institute
Global Status Report on Road Safety - 2018
VRU:
Vulnerable Road Users
50 Millions injured

A.I for Road Safety → 2030
« Accelerating the United Nations
Sustainable Development Goals (SDG)»
Road traffic
deaths and
injuries to be
reduced by at
least 50 %
from 2021 to
2030
6G to address UN’s SDG
➔ Contribute to the Quality & opportunities of human Life

4
Alignment of Planets
6G Timeline

5
6G KPIs - Purpose Built System
6G KPIs Let’s handle Mobility as
Keystone of Sustainable
Cognitive City

tii.ae
WCNC 2022, Austin, US
ADAS, DMS, OMS
Safer Vehicles & Road Users
ADAS: Advanced Driver Assistance System, DMS: Driver Monitoring System, OMS: Occupant Monitoring System

A.C.E.S : 4 Mega-Trends Taking over the Automotive Industry

Software Defined Vehicles – OEM Market trends
Big Tech’s Trojan’s Horse
$4.5 Billion
(ADAS lead)
Brand new SW Business Unit (2021)
How to change the Mindset?

Automotive Grade HW platforms: Centralized & Powerful
SA8155AP
SA8195AP
STB « on-wheels » iMX9
GPU/NPU
IVI (In-Vehicle Infotainment)

10
Beyond Moore’s Law thanks to Specialized Processing (GPU, NPU)
NPU and GPU are ‘Must have’ – no way back
1000 x
By 2025
Moore’s Law is stalling.
Introduction of new architecture with
GPU, then NPU enables to keep
impressive computing power growth rate.
Moore’s Law stalled.

11
Native A.I + Open Source Everything
Delivering RISC-V Purpose built Architecture
ARM’s Next Gen architecture
& Mobility
For sake of efficiency (Processing
power, Energy consumption, Cost)
main Vertical Markets would require a
specialized ‘Killer Reference Design’
best suited to address their own
strategic Key Use cases.
Those architectures will emerge as
optimized (a.k.a purpose built)
combination of CPU/GPU/NPU.
RISC-V reduces investment barrier for silicon, therefore a new ‘generation’/Ecosystem will emerge, Designing &
delivering such ‘Killer’ architecture for prioritized verticals on the mid term, whilst disruption can then be achieved on long
term via Neuromorphic chipsets.
Native A.I :: Embedded in all system components ➔
obtain & evaluate massive amount of real-time
information. / Federated learning

12
Neuromorphic ‘Brain-like’ architectures – 6G companions?

Autonomous Vehicle (AV): The Ultimate Sensor Fusion
Source: Waymo
« Space
Odissey »/ « Moonshot »
with some Direct Short
Terms & Affordable
Enhancements for our
Daily Life!

LiDAR + Video Camera Fusion + A.I
80% of all car accidents involve Distraction (rationale for
SAFE act in the USA).
25% of fatal accidents involve distraction

Two (2) Distinctive Paths towards Autonomous Vehicle
Source: Yole 2019
Automotive camera module industry has reached
$3Billion in 2018. Sales are expected to reach
$5.7B in revenue by 2024.
The average number of « Cameras per end-
product » will exceed 3 in 2024, for Automotive
Market & Mobile phones.
Last mile delivery
Robot Taxi
Shuttle

Virtuous Circle of Connected Fleets
Crashes
-91%
Cost reduction
Police fines
-50%
Fines reduction
Insurance
-20%
Insurance
premium
Energy
-10%
Consumption
AfterMarket (A/M) example for Telematics

A.I based Vision: from Innovation Labs to R&D Implementation
From Simulation To implementation
Mobile phone
Snapdragon
Benchmarking with competition
Driver Monitoring (Drowsiness, Distraction…)
Mobile platform
Snapdragon based
Passive ADAS
In-depth
understanding
of Key
technologies is
critical:
Credibility for
an efficient
Design
(Choice of HW
& SW
platforms +
integration).

Paving the way to Digital Twin
Digital Twin: stepping stone to Metaverse
Vulnerable Road Users (VRU)

DMS/ADAS Stakeholders : ‘Tip of the Iceberg’
May 2021
Acquisition of AFFECTIVA: $73 Million
Feb. 2021
Sale completed: €100 Million
July 28th 2021
IPO Tel Aviv Stock Exchange (TASE)
DMS/OMS ADAS

Complete In-car SDK sensing solutions
Driver Sense (DMS) Cabin Sense (OMS)
- Drowsiness
- Distraction
- Smoking
- Phone
- Drinking
- Seat belt
- Demographics
- Driver intention
- Profile Cabin Personalization
/ recognition of driver
- Occupancy
- Personalization
- Demographics
- Posture
- Objects
OMS: Occupant Monitoring System
DMS: Driver Monitoring System

Driver Attention Alert (DAA) :: building blocks
Drowsiness Distraction
Green point = Target to be followed by Driver
== ‘Cat Laser Chaser’ ! Being able to track accurately the
eye gaze can:
(1) improve efficiency for detection
of lack of attention,
(2) gather insights about driver
behaviour in context,
(3) trigger new services

Mobile Phone: Texting/Calling & Drinking
Mobile Phone: Texting, Calling Drinking
The main target cluster : young adults, since road crashes are #1 cause of death for this
category, and it’s proven that distraction due to Mobile phone usage amongst them is
dramatic.

DMS/OMS :: Seat Belt detection

Child presence Detection :: using OMS building blocks
Child Seat Detection
Child Detection (via Age estimation)

On the road to Identifying Impaired Drivers….
Health Monitoring: Heart Rate (H.R) & Breath Rate (B.R)
This might require
additional sensors to
complement the
camera.

DMS for Face Detection
Liveness (Anti-spoofing)
Mainly used for Commercial Transport of hazardous
Goods == brings somehow security of usage, and thus
safety.
E.g. only qualified/experienced & rested Drivers can be
authorized to drive vehicles.
Then this could be used also for Car/Environment
personalized configuration (Comfort, Multimedia).

Vision based ADAS
Road Safety Improvement:
• FCW (Forward Collision Warning)
• LDW (Lane Departure Warning)
• PCW (Pedestrian Collision Warning )
• HMW (Headway Monitoring Warning)
• FVSA (Front Vehicle Start Alarm)
PCW
FCW
LDW
In-cabin integration &
HMI is critical.
Avoid Distraction &
False Alarms.

Vulnerable Road Users (VRU): 54% of Road Deaths !
It’s time to focus (more) on Human!
Pedestrian
Cyclist
Pedestrian
Source: courtesy of Humanising Autonomy

VRU* :: Prediction of pedestrian Crossing intention
Alphapose
YOLOv3
DeepSORT
High interest to assess feasibility & TRL
(Technology Readiness Level) for further
integration into real world product ==
Differentiating Factor w.r.t Vulnerable Road
Users (VRU)
SUP’COM Students crossing the street, again & again…
*VRU : Vulnerable Road User

VRU :: Monocular camera based (not LiDar!)
Source: courtesy of Humanising Autonomy
Paving the Way to Autonomous Vehicles
ROI
Distance estimate

Traffic Sign Recognition (TSR) with A/M Products (overview)
For an After Market (A/M) Product, within the scope of TSR, mostly 2 types are of real interest:
1) STOP/YIELD & NO ENTRY: to generate ALARM
2) SPEED LIMITS: Intelligent Speed Assistance (ISA)
TSR: Traffic Sign Recognition

Augmented Reality (AR)
AR brought to Head-Up Displays
Source: courtesy of Wayray
- navigation + car status information
- surrounding environment, and conditions,
- LBS (Location Based services),
- C-V2X could warn of potentially ragged edges,
approaching accidents can be identified,
- road signs and route trackers to be magnified.
Road Safety improvement, since the Driver doesn’t need to get his eyes off the road…
However UI still require some good design to avoid Driver distraction by feeding too many
information at a time…

Video-Telematics cloud platform: DMS & ADAS
Cloud Telematics platform integrating Video (DMS & ADAS) is available.
Besides Telematics (position, acceleration, speed, mileage, diagnostic, …),
short videos triggered by warnings and alarms from DMS/ADAS are stored
in the platform.
Key for Driver Coaching,
insurance claims...
+ ’Pay as you Drive’ Models
Field
Qualification
are
still
necessary

tii.ae
Regulation trends overview
Towards Road Safety Improvement

EU, USA
Driver
Drowsiness
& ISA
Driver
(advanced)
Distraction
2023 2025
Sudden
Sickness
GSR v2
Driver under
influence*
(impaired)
Child
presence
Detection
face recognition, Genre classification, occupant
counting, object detection & classification,
Driver Gaze Zone tracking (for 3D control),
Health Monitoring, Body tracking
2026
Market demands for Next Gen Digital Cockpit (2026)
USA (2021): bipartisan $1.2 Trillion Infrastructure Bill
GSR: Global Safety Regulation, NCAP: New Car Assessment Programme
ISA: Intelligent Speed Assistance
features
Minimum
set
Extended
set
Lack of Ambition, still very slow adoption: Will OTA change/accelerate this?
Once True Software Defined Vehicle reached

UK regulations : Direct Vision Standard (DVS) & FORS
Camera sensors & Radar means
more opportunity for A.I based
warning system to support the
Driver to take care of its
surroundings, especially with
VRUs.
DVS: Direct Vision Standard
FORS: Fleet Operator Recognition Scheme
Safety Permit for Heavy Good
Vehicle (HGV) to access
Greater London

tii.ae
Smart/Cognitive City
Road Infrastructure & Traffic Management

The Top 10 Smart City Use Cases
2020 survey
“ 55% of the world’s population lives in urban areas and this percentage is expected to rise to 68%
by 2050. Additionally, freight volumes are expected to grow by 40% during this time period”
traffic congestion cost U.S.
commuters $305 billion in 2017 due
to wasted fuel, lost time and the
increased cost of transporting goods
through congested areas
Source: INRIX

Smart Traffic Analysis
Stereo-vision OTUS3D

Urban Traffic Management :: A.I + 5G & C-V2X
Cross-Working Group Work Item VRU: 2020
A.I combined with 5G & C-V2X is paving the way to disrupting road
traffic Management services, saving and changing lives of citizens.
Heterogeneous Interconnected devices, with Data privacy concerns
make it a perfect playground for Federated Learning & A.I on Edge !

C-V2X services (very brief) Overview: Keystone of nextGen Mobility
Emergency First Responder priority
Smart Traffic Light
Blind curve: Hazard warning

Road Side Unit (RSU) – C-V2X + LiDAR + Video & A.I
Road Side Units (RSU) deployment jointly with
LiDAR & Cameras to capture activities & intention
of Vehicles + Pedestrian to improve safety, then
prioritized emergency vehicles, and finally
smooth/efficient traffic.
Smart City :: Security & Safety
Edge Computing (MEC) will also play a Key role within the IT infrastructure,
to secure Low Latency (RTT), gathering/fusion of sensors data & processing power.
MEC: Multi-Access Edge Computing

Traffic Management :: Classifying & counting – Flow control
Fixed Camera Flying Camera onboard Drone
6G Non-
Terrestrial
Networks
(NTN)
Reconfigurable
Intelligent Surface
(RIS)

Autonomous Vehicle (A.V) Teleoperation Dispatch Center
A “Must have” for Liability purpose? …
Still in the infancy. Quite some room for improvements in terms
of User/Technician Immersive experience! Needs interaction
mechanisms to maintain perceptual illusions.
Possible thanks to high resolution imaging and sensing, and
5G/6G wireless networks (URLLC)
URLLC
Multisensory Telepresence & Immersion
URLLC: Ultra-Reliable Low Latency Communication

Traffic Management :: Carpooling & Tunnel Watch
Carpooling :: High Occupancy Vehicle (HOV)
- Lateral & frontal detection for occupants counting
- Vehicle class recognition
- License plate matching
- GDPR compliant (Privacy: blurring of personnal
data)
Tunnel Watch :: Automatic Incident Detection (AID)
- High Accuracy of Incident Detection
- Time saving for first responders
- vehicle categories & counting per Lane, for real-time
traffic informations
- ➔ Safety improved within such complex environment
Video Semantic Segmentation

Prioritizing Emergency Vehicles flow :: A.I+C-V2X+(NG)eCall
A.I Jointly with C-V2X and (NG)e-Call system, will save lives, by
minimizing time for intervention (first responders),
whilst also increasing global security of cities.

Urban investments in road safety :: How to Prioritize?
NYC – TOTAL Collisions NYC – VRU (Cyclists)
Collisions
Acquisition of relevant and quality Data from Road Traffic
Crashes (RTC) must be a Keystone for Sustainable Urban
Planning, and still unmet challenge, in spite of some progress
thanks to open Data policies in some Major cities worldwide.
Machine Learning + Geolocation
applied to Social Media
Crowdsourced Crash Report
(City of Nairoby)
Milusheva S. et al.

Improving Road Safety via Infrastructure Hazards detection (e.g. Pothole)
Sensor Fusion : Video + Accelerometer (IMU) + GNSS
Potholes are amongst the primary
Road Infrastructure Hazards
jeopardizing safety of drivers
(especially Motorbikes, Cycles).
Thanks to latest Leading Edge
technologies such as Computer
Vision, fusioned with Inertial sensors,
it is now possible to detect and
monitor such hazards, and set-up
efficient repair campaigns via
predictive Maintenance.

Potholes Detection & Mapping: Smart City Cloud Platform Extension
Prioritization & Optimization of Urban investment / Road Maintenance,
whilst cross-referencing with Road Traffic Crashes (RTC) Risk Management.

Motorway Maintenance: Cost efficiency
“The costs of road maintenance in the Netherlands add up to 1.6
Billion euros on an annual basis”
A.I
Driving footage
“a team of specialized technicians to go frame by frame
through the footage in order to recognize anomalies.
These defects are then manually categorized before they
are reported for maintenance”
The past: Costly & Lengthy
Now Semantic segmentation
Cost efficient Maintenance

tii.ae
Micro-Mobility
Smarter & Safer e-Scooters, e-Bikes

Micro-Mobility is Key to Sustainable Smart Cities
Faster growth than ride-hailing !
Rebound after Covid-19 crisis.
Reduce
Carbon
Gaz
emission

A.I Taking Fleet e-Scooter safety ( and Business) to the next level (1/2)
• Lane recognition – Is the
scooter riding on the Road,
the Sidewalk or the Bike
lane?
• Pedestrian detection
• Helmet detection
• Camera assisted parking
(using QR code recognition
in designated spaces that
don’t have any GPS signal –
e.g. underground car parks
or ‘urban canyons’)
• Smart City Insights
(Surface conditions,
obstructions, near misses
etc)
Source: Luna & Voi
Sidewalk detection/Protection Parking assistance
Main challenge for business growth:
Sidewalk Protection

A.I Taking Fleet e-Scooter safety ( and Business) to the next level (2/2)

What about Consumer e-Scooters?
Model XI Crash Assist A.I
ADAS Camera sensor, making difference between: stop-light, stop-sign, person, car, or inanimate oject
➔ Audible & visual warnings!!.

Qualcomm & Spoke: Bringing C-V2X to e-Bikes

Boréal Bikes - Holoscene X (Developer Edition)

Intelligent Helmets : A.I + A.R
A.R: Augmented Reality
Far beyond « Standard Helmets », however fueling
Innovation towards more Safety.
Potential Concerns with Distraction….likely to require quite
some Feedback Experience from the Field / Real
World…before any commercial launch.

tii.ae
Let’s make a Difference for a Better World !!

TII_Thierry_LESTABLE_WCNC_2022_v10_Short.pdf

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to TII_Thierry_LESTABLE_WCNC_2022_v10_Short.pdf

Similar to TII_Thierry_LESTABLE_WCNC_2022_v10_Short.pdf (20)

More from Thierry Lestable

More from Thierry Lestable (20)

Recently uploaded

Recently uploaded (20)

TII_Thierry_LESTABLE_WCNC_2022_v10_Short.pdf