Professor Michael Milford's (Queensland University of Technology) presentation at Mumbrella's Automotive Marketing Summit

www.roboticvision.orgroboticvision.orgARC CENTRE OF EXCELLENCE FOR ROBOTIC VISION
Hands On With The Self-Driving Car
Past and Present Funding SupportPast and Present Collaborators
Professor Michael Milford
Australian Research Council Future Fellow |
Microsoft Research Faculty Fellow | Chief
Investigator, Australian Centre for Robotic Vision

Overview
• Introductions
• Key Motivations – Why Self-driving Cars?
• Self-driving Cars: A Technology Holy Grail with History
• Current State of Play – Major Tech Players
• The Technology Behind Self-driving Cars
• Perception and Sensing
• To Map or Not to Map: That is the Question
• Infrastructure Reliance
• The AI Technology Driving Self-driving Cars and What It Can Do
• Flying Cars…
• Unsolved Challenges in Self-driving Cars – The Last 1%
• Disruption Highlights
• Key Takeaways

Publication references appear in red
For More Details…

Where this Information Comes From

Why?

Why?
1.3 million
deaths / year
http://asirt.org/initiatives/informing-road-
users/road-safety-facts/road-crash-statistics

History

NavLab 1, CMU (1986)
https://www.youtube.com/watch?v=ntIczNQKfjQ

NavLab 5, CMU (1996)
https://www.youtube.com/watch?v=bdQ5rsVgPuk

DARPA Grand Challenge, 2004-2005
https://www.youtube.com/watch?v=M2AcMnfzpN

DARPA Urban Grand Challenge, 2007

Current State of Play

Autonomy Levels
https://www.theverge.com/2016/9/28/13076948/self-driving-car-poll-autonomy-kelley-blue-book
Where the technology
is currently

Conventional Startup Example: NuTonomy
https://www.youtube.com/watch?v=iP_lAjIfZwU
• Partnerships for vehicles, ride
sharing market, government
• Controlled ride sharing fleet
• Formal rigorous approach
• People don’t own cars

Conventional Startup Example: Cruise Automation
• Rapidly scaling
• Software and testing-intensive
approach

AI-based Startup Example: DriveAI
https://www.youtube.com/watch?v=GMvgtPN2IBU
• Machine learning and deep
learning as the answer
• Small fleets
• AI-background

Car Maker Corporate Example: Toyota & Toyota Research Institute
https://www.youtube.com/watch?v=5s6HbrOYads
• Long term outlook, cultural
context
• Vehicle making legacy
• Dual assistive (Guardian) and
fully autonomous projects
(Chauffeur)
• Private car ownership still
feasible

Tech Accessibility Implications
• Toyota’s Guardian program
(and other equivalents) have
marginal if any benefits in
terms of accessibility and
mobility
• Safety vigilance issue is still
not resolved

Tech Company Corporate Example: Waymo/Google
https://www.youtube.com/watch?v=uHbMt6WDhQ8
• Long history
• New builder of vehicles
• Existing tech and consumer
infrastructure
Watch for Nathaniel later in
this presentation…

A Note About Reliability
Company X in 2016: 0.0002 driver takeovers per mile
Our car is more reliable than a human driver in a limited range of
circumstances, using our own interpretation of “takeover”
BUT
If forced to be autonomous without intervention, it will fail
repeatedly and sometimes catastrophically in a small percentage
of corner cases
Translation

DARPA Grand Challenge 2005
26-year-old hacker George Hotz
https://www.youtube.com/watch?v=YuKAmsMg2ZE
Google hits a bus (slowly)
https://www.youtube.com/watch?v=I9T6LkNm-5w
Graphics Card Company NVIDIA
https://www.youtube.com/watch?v=qhUvQiKec2U

An Incredible Race
Dr Will Maddern
Automotive Flagship Lead, Oxford
University (formerly QUT)
Dr Brett Browning
VP Robotics
(Formerly of UQ)

An Irrational Environment Fosters Irrational Behaviour

http://www.prweb.com/releases/VSI/SegmentsAutonomousVehicle/prweb13472308.htm

http://medicalfuturist.com/wp-content/uploads/2016/08/artificial-intelligence-map6.jpg

The Technology Behind
Self-driving Cars

Sensing & Perception

www.roboticvision.orgroboticvision.orgARC CENTRE OF EXCELLENCE FOR ROBOTIC VISIONhttps://tctechcrunch2011.files.wordpress.com/2016/12/uber-atg-volvo.jpg?w=2234&h=1694

GPS
https://www.youtube.com/watch?v=C5s1ucmiACU
• Provides position, good for
journey planning
• Satellite-based, not 100%
reliable (tunnels, urban
canyons)
• “Autonomy-enabling” still not
universally available (drop
outs, accuracy, latency)
• Almost universal assumption
that it can’t be used as
primary source of positioning
information

RADAR
http://www.dailymail.co.uk/sciencetech/article-2288472/Volvo-unveils-scanner-automatically-slam-brakes-detects-cyclist-wobbling-cars-path.html
• Can see through fog, smoke,
some rain
• Coarse resolution, no fine
detail
• Primarily used for collision
safety with people, vehicles
and the environment
• Moderate to expensive price

LIDAR
• 3D scan of range to all line of
sight objects in all directions
• Medium resolution, high
range accuracy
• Disrupted by heavy rain,
particulates
• Good for detecting vehicles,
pedestrians, mapping and
positioning
• Expensive
In-car photo by Michael Milford. Google Self-driving Car, Velodyne
Here’s Nathaniel
back at Google

VISION (CAMERAS)
• Richest source of information
• High resolution
• Ranging to objects difficult
• Affected by environmental conditions
(day-night, weather, atmospheric
conditions)
• Relatively cheap

Does Sensing Solve
Everything?

EXAMPLE – TESLA CRASH
“Neither Autopilot nor the driver noticed
the white side of the tractor trailer
against a brightly lit sky, so the brake
was not applied”
https://electrek.co/2016/07/01/understanding-fatal-tesla-accident-autopilot-nhtsa-probe/

Visual Sensing Challenge

The Last 1% is Incredibly Difficult to Solve
“This wouldn’t have
happened with a long
range sensor”
https://electrek.co/2016/07/01/understanding-fatal-tesla-accident-autopilot-nhtsa-probe/

The tanker truck has to reliably predict, well ahead of time, that the
yellow truck coming around the bend is not a threat
“False Positives” are just as, if not more dangerous than “missed”
detections when the system is already highly reliable

To Map or Not to Map: That is
the Question
https://www.gizmodo.com.au/2015/05/how-to-teach-an-autonomous-car-to-drive/

Most Companies Assume & Exploit Existing 3D Maps
Advantages
• Facilitates positioning and navigation
technology
• Prior knowledge of road rules,
turning lanes, intersection behaviour
• Prior knowledge about obscured
obstacles, signs
• Place-based learning
Disadvantages
• Expensive to acquire and maintain
• May enable 99% performance
quickly but not 100% - a “crutch”
• We have a proof of concept in
humans that they’re not needed –
some companies are pursuing this
approach

Sensible “Cheating” - Assistive Infrastructure

Flying Autonomous Cars
Power
Price
Fail-safe
Regulation
Infrastructure
Laws of Physics suck
If physics is solved,
easier in many ways
than ground-based
autonomous vehicles

The AI Technology Driving
Self-driving Cars

https://raw.githubusercontent.com/qingkaikong/blog/master/38_ANN_part1/figures/figure1_ANN_history.jpg

The “Comeback”
What Changed in the Past Decade?
1. Scale of data available
2. Compute available
https://lh3.googleusercontent.com/4fMZAd6hp-Zp2olrW3szbpkm8-zPZ3hPOFHIjnrlvA5wBLgwHaAiz6Wm5m2bfxBucQ=h900
https://gigaom.com/wp-content/uploads/sites/1/2010/11/tesla1.jpg

http://www.extremetech.com/wp-content/uploads/2015/07/NeuralNetwork.png
Datain
Recognition/description
/understandingout

What Can the AI Tech Do?

What Deep Learning Can Do
With enough attention, current learning-based
technologies match or surpass many
constrained tasks that humans currently do, if
sufficient quantities of representative training
data can be obtained and ideal performance
can be defined.

http://cs.stanford.edu/people/karpathy/deepimagesent/
Google Deepmind via Tech Insider
Google Deepmind
Michael
Milford
Guillaume Lample, Devendra Singh Chaplot
A Neural Algorithm of Artistic Style: https://arxiv.org/abs/1508.06576

“I’m a luddite – give me some
ammunition to throw back at the
hype mongers”
or
“I don’t have budget to do any of
that so I need a way out”

www.roboticvision.orgroboticvision.orgARC CENTRE OF EXCELLENCE FOR ROBOTIC VISIONSzegedy et al, “Intriguing properties of neural networks”, https://arxiv.org/abs/1312.6199
+ =
+ =
+ =
+ =
+ =
+ =
Current Limitations – Understanding Machine Learning
Incredible Resources Being Brought to Bear
on Solving These Problems

Unsolved Challenges in Self-
driving Cars – The Last 1%

Adverse Conditions
• Not an issue with centrally controlled
ride sharing fleets that can be turned
off

Extreme Corner Cases
https://www.youtube.com/watch?v=GihuUYmoMXU
• An issue for all technology versions

Human Interaction
https://www.youtube.com/watch?v=VG68SKoG7vE
• An issue for all deployments in mixed
environments (excepting niches like
airports, dedicated lanes or roads…)

Disruption Examples

Endless Disruption & Opportunity

Key Takeaways

Onboard intelligence
3 Primary Technological Scenarios
Connected vehicles
Infrastructure-reliant
Requires
occasional
human
intervention
No human
control
……

Inner-city Controlled Ride Sharing Fleets are Feasible
Relatively Low Speeds
Can Afford to “Cheat”
by Modifying
Infrastructure
Ride-sharing & tech-
saavy wealthy
customer base
High density
High utilization
Ease of transition,
supported by
human-driver fleets
Can Be Turned off
During Challenging
Weather

Beyond That, A Range of Scenarios with Highly
Varying Implications for all Stakeholders

My Recommendations
Avoid the temptation to think in unchanging
absolutes about the technology e.g. “never
be like a human”, “solves everything”
See through the hype but don’t dismiss
everything out of hand
Keep an informed, regularly updated
awareness of the range of scenarios that
could play out

We’re Always Here to Talk

Professor Michael Milford's (Queensland University of Technology) presentation at Mumbrella's Automotive Marketing Summit

More Related Content

Similar to Professor Michael Milford's (Queensland University of Technology) presentation at Mumbrella's Automotive Marketing Summit

More from Ruperta Daher

Recently uploaded

Professor Michael Milford's (Queensland University of Technology) presentation at Mumbrella's Automotive Marketing Summit