An estimated 64% of all travel today is made within urban environments. By 2050, the total amount of urban kilometres travelled worldwide is expected to triple, with traffic congestion potentially bringing major cities to a standstill. In Singapore, a small island with a population of 5.4 million, there are approximately 1 million cars on the roads. At the same time, roads take up 12% of land space. With the limited land space in Singapore, it is unrealistic to further increase the number of vehicles or add more roads. To address these challenges, the Singapore government plans to implement an intelligent and adaptable transport system which uses data to empower commuters and adjusts to their needs. Sensor networks are being deployed that collect data from busy areas such as traffic junctions, bus stops and taxi queues, then relay it back to the relevant agencies for analysis through data analytics and real-world applications. Besides transportation systems powered by big data analytics, driverless vehicles are also a major focus so far for the Singapore government. More than six kilometres of public roads have been opened this year for AV trials, currently in use for trials with a small fleet of public self-driving taxis. Various stakeholders are aiming for full-scale commercial autonomous taxi service in 2018 in Singapore. In this presentation, Dr. Justin will address various aspects of AV technologies, including latest technical developments, opportunities and challenges related to AVs, safety and liability issues, and commercialisation aspects. For further information, visit our website at ma2017.mymagic.my. Facebook - Facebook.com/magic.cyberjaya Twitter - Twitter.com/MagicCyberjaya Instagram - Instagram.com/magic_cyberjaya/ LinkedIn - my.linkedin.com/in/magiccyberjaya YouTube - https://www.youtube.com/channel/UCIT_ihmWh5f3MCobvEwWMaA

1. Hands Off the Wheel: Driverless Cars
The Future is Amongst Us
Prof. Justin Dauwels
Nanyang Technological University, Singapore
jdauwels@ntu.edu.sg
www.dauwels.com

2. §
Car
that
drives
itself.
§ Perceives
the
environment
and
moves
where
safe
and
desirable.
§ No
human
supervision
is
required.
§ Everyone
in
AV
is
a
passenger,
or
it
can
travel
with
no
occupants
at
all.
2

3. ¡ Introduction
to
Autonomous
Vehicle
¡ Economic
Aspects
¡ Regulatory
Aspects
¡ Basic
AV
Architecture
¡ Scene
Perception
for
AVs
¡ Safety
of
AVs
¡ Hype
or
Future
Mobility
3

4. 4
Original
Video
Source
:
https://www.youtube.com/watch?v=ftouPdU1-­‐Bo
Dough Aamoth, Tech Editor at TIME Magazine

5. According
to
National
Highway
Traffic
Safety
Administration
(NHTSA,2013),
automated
vehicles
are
classified
in
five
levels.
5
Level
of
AV
Level
0
(No
automation)
Level
2
(combined
function
automation)
e.g.,
braking
and
steering
Level
1
(function-­‐specific
automation)
e.g.,
braking
Level
3
(limited
self-­‐driving
automation)
Human
intervention
Level
4
(full
self-­‐driving
automation)

6.
6
AV
Maker
Car
Manufacturer
Tesla,
GM,
Ford,
VW-­‐Audi,
Volvo,
Nissan,
Toyota,
Daimler-­‐AG
Technology
and
Idea
Innovator
Google,
Uber,
Apple,
Baidu,
nuTonomy
Component
Manufacturer
Bosch,
Delphi,
Continental

7.
7
2018
2019
2020
2023
2021
2030
nuTonomy
‘s
self-­‐driving
taxi
services
in
Singapore
Baidu
Tesla
Delphi and
MobilEye to
provide off-
the-shelf self-
driving
system
BMW
iNext
and
Ford’s
AV
Uber

8. ¡ Timeline
–
by
Boston
Consulting
Group
(BCG)
¡ People’s
acceptance
survey
of
AV
§ By
KPMG
(accounting
firm)
§ By
BCG
(1500
participants)
8
2025
2035
AV
on
the
market
10%
market
share
is
AV
Usual
travel
time
50%
less
travel
time
6/10
willingness
8/10
willingness
Very
likely
to
buy
partially
AV
within
5
years
Very
likely
to
buy
fully
AV
within
10
years
55%
44%

9. 9
q
Benefits
ü Reduced
driver
stress
and
costs
ü Mobility
for
non-­‐drivers.
ü Increased
safety.
ü Increased
road
capacity,
reduced
costs.
ü More
efficient
parking.
q
Challenges
ü Requires
additional
vehicle
equipment,
services
and
maintenance.
ü May
introduce
new
risks,
such
as
system
failures
and
hacking,
less
safe
under
certain
conditions.

10. 10
https://www.youtube.com/watch?v=iHzzSao6ypE

11. ¡ Accelerate
and
brake
§ AV
will
brake
and
accelerate
more
gradually
§ Fuel
saving
15-­‐20%
§ Reduction
of
CO2
emission
of
20
-­‐100
millions
tons
per
year
(McKinsey)
¡ Platooning
§ Reduce
wind
resistance
§ Use
road
space
more
efficiently
11

12. ¡ Introduction
to
Autonomous
Vehicle
¡ Economic
Aspects
¡ Regulatory
Aspects
¡ Basic
AV
Architecture
¡ Scene
Perception
for
AVs
¡ Safety
of
AVs
¡ Hype
or
Future
Mobility
12

13. ¡ Reduction
in
jobs
§ Truck
driver
§ Taxi
driver/chauffeur
¡ Jobs
that
change
§ Car
service
company
§ Car
design
company
§ Insurance
company
§ Retail
industry
¡ New
jobs
§ HD
map
industry
§ Driverless
car
software
company
¡ Creative
destruction
§ Whether
the
new
job
is
better
than
the
old
one:
secure,
interesting
and
well
paid
§ Length
of
the
period
of
displacement:
how
long
will
the
displaced
workers
be
unemployed?
§ Worst
case:
income
inequality
13

14. ¡ Transfer
of
car
ownership:
§ Less
individual
car
ownership
§ More
public
transportation
(AVs
for
mobility
on
demand)
¡ Shift
in
automotive
industry
due
to
AVs:
§ Corporate
marriage:
software
company
+
car
company
§ Partnerships:
Google
+
Ford,
Microsoft
+
Volvo,
Lyft
+
GM
§ Evolution
paradigm:
Microsoft
(software)
or
Apple
(hardware)?
▪ Microsoft:
good
for
software
company
▪ Apple:
good
for
car
company
14
Individual
Public

15. ¡ Parking
§ Parking
tickets:
$600
million/year
for
NYC
§ High
construction
cost:
Disney
Hall,2188
parking
lots,
$110
million,
$50k
per
parking
lot
¡ Traffic
Accidents
and
illegal
driving
§ Hospital:
1
million
days
in
hospital
in
US
§ Organ
transplantation:
20%
for
victims
of
fatal
car
accident
§ Jail:
14%
of
jail
population
due
to
illegal
driving
§ Fine:
$6
billion/year
for
US
speeding
tickets
15

16. ¡ Introduction
to
Autonomous
Vehicle
¡ Economic
Aspects
¡ Regulatory
Aspects
¡ Basic
AV
Architecture
¡ Scene
Perception
for
AVs
¡ Safety
of
AVs
¡ Hype
or
Future
Mobility
16

17. ¡ In
conventional
car
accident,
blame
is
distributed
between
the
driver
involved
and
the
vehicle
manufacturer.
¡ In
fully
automated
car
responsibility
for
avoiding
accidents
shifts
completely
to
vehicle
and
its
accident
avoiding
system.
¡ Multiple
liable
parties
including
the
§ vehicle
manufacturer
§ manufacturer
of
component
of
system
§ software
engineer
who
programmed
the
code
for
AV
§ road
designer
of
intelligent
road
system
that
used
to
help
guide
AVs.
17

18. ¡ Tesla
Model
S
-­‐
May
7
2016
18
The
first
known
fatal
accident
took
place
in
Florida
on
7
May
2016
while
a
Tesla
Model
S
electric
car
was
in
Autopilot
mode.
The
driver
was
killed
in
a
crash
with
a
large
18-­‐
wheel
tractor-­‐trailer.
According
to
the
NHTSA,
the
crash
occurred
when
the
tractor-­‐
trailer
made
a
left
turn
in
front
of
the
Tesla
at
an
intersection
on
a
non-­‐controlled
access
highway,
and
the
car
failed
to
apply
the
brakes.
According
to
Tesla
Motors,
“neither
autopilot
nor
the
driver
noticed
the
white
side
of
the
tractor-­‐trailer
against
a
brightly
lit
sky,
so
the
brake
was
not
applied”.
Credit
:
http://cleantechnica.com/2016/07/02/tesla-­‐model-­‐s-­‐
autopilot-­‐crash-­‐gets-­‐bit-­‐scary-­‐negligent/

19. ¡ 33
states
introduced
legislation
related
to
autonomous
vehicles
in
2017,
up
from
20
states
in
2016,
16
states
in
2015,
12
states
in
2014,
9
states
and
D.C.
in
2013,
and
6
states
in
2012.
¡ Since
2012,
at
least
33
states
and
D.C.
have
considered
legislation
related
to
autonomous
vehicles.
19
On
Sep.
20,2016
NHTSA
issued
updated
guidance
for
the
safe
development
of
highly
autonomous
vehicles
(HAVs).
The
policy
update
is
broken
down
into
four
parts:
• vehicle
performance
guidelines,
• model
state
policy,
• NHTSA’s
current
regulatory
tools
and
• possible
new
regulatory
actions
Source
:
http://www.ncsl.org/research/transportation/autonomous-­‐vehicles-­‐legislation.aspx
https://www.transportation.gov/AV
http://www.ncsl.org/research/transportation/autonomous-­‐vehicles-­‐self-­‐driving-­‐vehicles-­‐enacted-­‐legislation.aspx

20. § Some
European
countries,
like
the
UK
and
Germany,
have
adjusted
national
laws
to
allow
testing
of
driverless
cars
and
adopted
strategies
to
make
the
new
technology
commercially
available
within
the
next
few
years.
§ In
Singapore
the
Committee
on
Autonomous
Road
Transport
for
Singapore
(CARTS)
has
been
set
up
to
chart
the
strategic
direction
for
AV-­‐enabled
land
mobility
concepts.
To
support
the
visioning
work
of
CARTS,
LTA
signed
a
Memorandum
of
Understanding
with
Singapore’s
lead
R&D
agency,
A*STAR,
to
set
up
the
Singapore
Autonomous
Vehicle
Initiative
(SAVI),
which
will
explore
the
technological
possibilities
that
AVs
can
create
for
Singapore.
§ The
Japanese
government
plans
to
draw
up
law
to
govern
use
of
driverless
cars.
The
National
Police
Agency,
meanwhile,
will
consider
who
should
take
responsibility
if
a
car
without
a
driver
or
a
steering
wheel
causes
an
accident.
It
also
aims
to
set
guidelines
within
the
fiscal
year
in
order
to
allow
manufacturers
to
road-­‐test
driverless
cars
on
highways.
20
Source
:
http://www.mot.gov.sg/Transport-­‐Matters/Motoring/Driverless-­‐vehicles-­‐-­‐A-­‐vision-­‐for-­‐Singapore-­‐s-­‐transport/

21. ¡ Introduction
to
Autonomous
Vehicle
¡ Economic
Aspects
¡ Regulatory
Aspects
¡ Basic
AV
Architecture
¡ Scene
Perception
for
AVs
¡ Safety
of
AVs
¡ Hype
or
Future
Mobility
21

22. Sensors
Internal
(proprioception)
Odometer
Wheel
“Optical”
Mouse
Pose
Gyro
Compass
Tilt
Sensors
Sensors
Sensors
External
(perception)
Acoustic
Sonar
Audio
Optical
Lidar
Camera
Radio
Frequency
Radar
Communication
(data)
Direct
V2V
V2I
Cellular
Maps
Traffic
Media
Satellite
GPS
Software
Updates
22

23. 23

24. ¡ Working
principle
§ Gathers
light
through
a
lens
in
the
form
of
photons.
Each
photon
carries
a
certain
amount
of
energy.
§ As
the
photons
stream
through
the
camera’s
lens,
they
land
on
a
silicon
wafer
that’s
made
up
of
grid
of
tiny
individual
photoreceptor
cells.
§ Each
photoreceptor
absorbs
its
share
of
photons
and
translates
the
photons
into
electrons,
which
are
stored
as
electrical
charges.
§ The
electrical
charges
is
then
transformed
into
“pixel”
¡ 3D
digital
camera
image
§ Multiple/stereo
camera:
two
or
more
camera
to
capture
the
scene
from
slightly
different
viewing
angles
§ Structured-­‐light
cameras:
camera-­‐projector
combo
that
augments
image
data
with
depth
information
¡ Weakness:
dirt,
night
time,
rain
24

25. ¡ RADAR:
radio
detection
and
ranging
¡ Working
principle:
§ RADAR
device
sends
out
a
series
of
electromagnetic
wave
and
radiate
outward
§ Keeps
track
of
the
reflected
electromagnetic
waves
¡ Advantages
§ Perspective:
“see”
through
fog,
rain,
dust,
sand…
§ Detect
speed:
Doppler
effect
¡ Weakness:
Poor
resolution
¡ SONARS:
ultrasonic
sensors
¡ Working
principle:
“sound
navigation
radar”,
using
sound
waves
instead
of
electromagnetic
waves
¡ Weakness:
It
can
only
detect
object
at
closer
range
25

26. ¡ LIDAR:
light
detection
and
ranging,
also
called
“laser
radar”
¡ Working
principle:
§ “Spray
paints”
its
surroundings
with
intense
beams
of
pulsed
light
§ Measures
how
long
it
takes
for
each
of
those
beams
to
bounce
back
§ Calculates
a
three
dimensional
digital
model
of
its
nearby
physical
environment
¡ Software:
“Point
Cloud”
¡ Difference
with
digital
photo
§ No
colour
information
§ Temporal
depiction:
a
spinning
LIDAR
sensor
continually
refreshes
the
digital
model
it
generates
¡ Weakness:
expensive
26

27. ¡ High
definition
digital
map:
a
detailed
and
precise
model
of
a
region’s
most
important
surface
features
§ Compensation
for
the
inability
of
GPS
data:
tunnel,
and
skyscraper
§ Real-­‐time
digital
map:
the
car’s
operating
system
will
calculate
its
current
location
by
relying
on
visual
cues
in
the
flow
of
real-­‐time
sensor
data
that
depicts
the
nearby
environment
§ HD
map
offers
its
user
a
pictorial
depiction
of
the
region
§ Millions
of
stored
entries
of
topographical
details.
27

28. ¡ Introduction
to
Autonomous
Vehicle
¡ Economic
Aspects
¡ Regulatory
Aspects
¡ Basic
AV
Architecture
¡ Scene
Perception
for
AVs
¡ Safety
of
AVs
¡ Hype
or
Future
Mobility
28

29. ¡ Lab
Vision
§ Develop
collaborative
and
autonomous
unmanned
systems
that
integrate
seamlessly
and
safely
in
extreme
and
complex
environments
§ Two
initiatives:
▪ APART:
Airport
Precision
Air-­‐side
Robotics
Technology
▪ CRISP:
Crisis
Response
Intelligence
Support
Program
§ Develop
critical
TRL
4-­‐6
DUAL-­‐USE
intelligent
and
unmanned
enablers
to
sustain
competitive
advantage
¡ Rationale
of
Lab
§ De-­‐risk
up-­‐stream
TRL
1-­‐3
technologies
▪ Mission
planning
&
control,
sensor
technologies,
A.I.
,
machine
vision,
exoskeleton,
V2X,
meshed
network
etc.
§ Transit
critical
TRL
4-­‐6
technologies
for
commercialization
§ Develop
IP
pipeline
to
sustain
continuous
innovation
¡ Manpower:
21
NTU
PIs,
25
STE
staff,
37
PhD
students,
25
research
fellows,
20
research
associates,
and
10
project
officers.
¡ Management:
Prof.
Wang
Danwei
(Co-­‐Director),
Assoc.
Prof.
Justin
Dauwels
(Dy
Director),
Mr.
Paul
Tan
(Co-­‐Director)
29

30. ¡ Industry
is
investigating
autonomous
vehicles:
§ Google,
Tesla,
Nutonomy,
Navya,
many
car
manufacturers.
§ Heavily
reliant
on
pre-­‐programmed
route
data.
¡ Perception
in
difficult
&
dynamic
environments
§ Limited
sensors?
§ Night?
§ Rain/dust
(lidar
is
known
to
degrade)?
§ No
GPS?
¡ Aim:
create
flexible
perception
frameworks
§ Make
full
use
of
multi-­‐sensor
information
§ Foundation
in
statistical
signal
processing
theory
§ Computationally
efficient
algorithms
for
real-­‐time
implementation
30
Google vehicle
Tesla vehicle

31. ¡ Five
main
goals:
31
§ 3D
tracking
of
pedestrians/vehicles.
§ Intention
prediction
of
road
users.
§ Terrain
classification
and
mapping.
§ Multi-­‐sensor
localization
(SLAM).
§ Road
structure
inference.

32. ¡ Classification
of
road,
trees,
sky,
and
water
puddles.
¡ Key
issues:
§ Complex
terrain
§ Ambiguous
terrain:
Similarity
between
dirty
road
and
muddy
water
puddle
¡ Novelty:
§ Fusion
of
RGB
+
lidar
§ New
architecture
for
late
fusion
§ Water
puddle
detection
32

33. Pixel
wise
evaluation.
33
¡ Deep
learning
for
RGB
[1].
¡ New
scheme
to
include
lidar.
¡ Results
better
or
comparable
with
prior
studies
[2,3].
¡ Sensor
fusion
generalizes
better
to
different
environments.
[1] J. Long et al. (2015). Fully convolutional networks for semantic segmentation. CVPR
[2] M. Häselich et al. (2011). Terrain Classification with Markov Random Fields on fused Camera and 3D Laser Range Data. In ECMR
[3] G. Zhao et al. (2014). Fusion of 3D-LIDAR and camera data for scene parsing. JVCI
Evaluation on
All the Frames
Road Vegetation Puddle Per-Frame
Time
Image Only 92.5 94.6 74.3 70 ms
Fusion 93.0 94.7 74.8 90 ms
Fusion +
Specialized
Puddle Detector
93.1 94.7 79.4 110 ms

34. ¡ Vehicles
in
water
puddle
may
need
towing.
§ Vehicle
should
try
to
avoid
puddles.
¡ A
novel
two-­‐stage
architecture
improves
the
performance.
¡ First
to
use
deep
learning
for
outdoor
water
puddle
detection.
34
Image Groundtruth Detection
[1] A. Rankin et al. (2011). Daytime water detection based on sky reflections. ICRA
[2] A. Rankin, et al. (2006). Daytime water detection and localization for unmanned ground vehicle autonomous navigation. Army Science Conference.

35. ¡ Given
tracked
objects,
use
visual
features
and
contextual
information
to
predict
future
actions.
§ Vision:
Include
full
contextual
information,
in
addition
to
visual
cues.
35
[1] C. Keller et al. "Will the pedestrian cross? Probabilistic path prediction based on learned motion features." Joint Pattern Recognition Symposium 2011.
[2] A. Schulz et al.. "Pedestrian intention recognition using latent-dynamic conditional random fields." IEEE Intelligent Vehicles Symposium 2015.

36. ¡ Novelty:
§ Use
of
efficient
features/inference
models,
e.g.:
▪ Latent
dynamic
conditional
random
fields.
▪ CNN
based
visual
features.
§ General
enough
to
allow
subtle
visual
cues
to
be
learnt.
¡ Evaluation:
§ Daimler
dataset
[1].
§ Plots
show
results
averaged
over
all
test
sequences.
§ Faster
than
baseline
(0.2
Hz
vs
50
Hz)
[2,3]
36
[1] C. Keller et al. “A New Benchmark for Stereo-based Pedestrian Detection” IEEE Intelligent Vehicles Symposium 2011.
[2] C. Keller et al. "Will the pedestrian cross? Probabilistic path prediction based on learned motion features." Joint Pattern Recognition Symposium 2011.
[3] A. Schulz et al.. "Pedestrian intention recognition using latent-dynamic conditional random fields." IEEE Intelligent Vehicles Symposium 2015.
Baseline
Proposed

37. ¡ Introduction
to
Autonomous
Vehicle
¡ Economic
Aspects
¡ Regulatory
Aspects
¡ Basic
AV
Architecture
¡ Scene
Perception
for
AVs
¡ Safety
of
AVs
¡ Hype
or
Future
Mobility
37

38. ¡ “Humansafe”
level:
A
car
that
can
drive
twice
as
many
accident-­‐free
miles
as
the
average
human
could
be
advertised
as
having
a
humansafe
rating
of
2.0
§ Different
humansafe
rating
required:
e.g.
10
for
school
bus,
5
for
cargo-­‐only
vehicle
§ MDBF:
mean
distance
between
failure
¡ Accident
Response
§ Rule
book:
open,
transparent
and
verifiable
after
an
accident
§ Black
box:
similarly
as
in
aviation,
driverless
car
data
should
be
made
available
to
insurance
investigators
and
law
enforcement
officials.
38

39. ¡ Trolley
Problem
§ Rules
defined
by
programmer
ahead
of
time
§ More
rational
and
rapid
risk/benefits
calculation
§ 360-­‐degree
sensory
perception
39

40. 40
n Centre
of
Excellence
for
Testing
and
Research
of
Autonomous
Vehicles
-­‐
NTU
(CETRAN)
launched
by
the
Land
Transport
Authority
(LTA)
and
JTC
in
Aug
2016,
in
partnership
with
NTU.
n 1.8-­‐ha
CETRAN
Test
Circuit
with
a
simulated
road
environment
for
the
testing
AVs
prior
to
their
deployment
on
public
roads.
n Testing
in
a
computer-­‐simulated
environment
representative
of
Singapore’s
traffic
conditions,
to
complement
the
tests
performed
in
the
test
circuit.
n Goal:
n Conduct
research
towards
standards
and
test
procedures
to
ensure
n safety
and
n security
(including
cybersecurity)
of
autonomous
vehicles
to
enable
deployment
on
Singapore
Public
Roads.

41. 41
Rain
Simulator
Rain
generator
to
test
AV
performance
in
tropical
rain
condi4ons
Bus
bay
Singapore
style
bus
bay
with
yellow
“give”
way
box
to
test
give
way
rules
in
rela4on
to
public
transport
Carpark
Gantry
Car
park
gantry
to
test
entry
to
and
exit
from
HDB
estates
to
simulate
passenger
pickup
from
HDB
Flats
Roundabout
Test
of
give
way
rules
on
roundabouts
Raised
pedestrian
crossing
Test
of
speed
hum
detec4on
and
zebra
crossing
detec4on
on
crossings
as
seen
in
HDB
estates
S-­‐Course
Test
AV
maneuvering
capability
in
4ght
spaces
Carpark
Carpark
space
–
to
test
behavior
of
extended
wait
for
a
passenger
at
a
HDB
pickup
point
The
straight
Asses
performance
on
straight
and
empty
sec4on
of
road
–
ensure
vehicle
does
not
show
“road
hogging”
tendencies
Un-­‐signaled
intersec=on
Test
management
of
an
intersec4on
without
traffic
light
but
with
yellow
box
Turning
lane
Handling
of
mul4-­‐
lane
intersec4on
and
selec4on
of
correct
lane.
Slope
Test
of
slope
and
handling
of
reduced
visibility
on
crest
Small
speed
hump
Detec4on
of
and
handling
of
small
speed
hump
Signaled
intersec=on
Handling
of
signaled
intersec4on
and
zebra
crossings
-­‐
assessment
of
correct
priori4za4on
of
these
intersec4ons
Bus
lane
Correct
handling
of
Bus
Lane
as
seen
in
Singapore.
Workshop
Workshop
to
prepare
vehicles
and
test
equipment
for
use
on
circuit
Smart
Mobility
Network
Extension
of
NTU
Smart
Mobility
Network
to
test
vehicle
to
infrastructure
communica4ons
and
to
support
test
equipment
and
high
accuracy
posi4oning.

42. 42
Proposed CETRAN AV Test
Circuit
CETRAN AV Test
Circuit model in IPG
CarMaker

43. Test
Case
Test
Case
Test
Case
Scenario
Manager
CarMaker
Vehicle
Dynamics
Object
Generation
Test
Case
ROS
Communication
Data
formatting
Clock
master
Autoware
Motion
Planning
Vehicle
Control
Host
Mission
Drive
Database
Scenarios
Challanging
Scenarios
Obj.
Map
Test
Config
Analysis
Filter
for
Use
Case
Test
Scenario
Functional
Safety
VISSIM
Traffic
Simulator

44. 44
Scenario: AV reacting to a Jaywalker

45. ¡ Introduction
to
Autonomous
Vehicle
¡ Economic
Aspects
¡ Regulatory
Aspects
¡ Basic
AV
Architecture
¡ Scene
Perception
for
AVs
¡ Safety
of
AVs
¡ Hype
or
Future
Mobility
45

46. ¡ Autonomous
Vehicle
current
state
§ Peak
of
inflated
expectations
§ 10+
years
to
mainstream
adoption
¡ Hurdles
to
mainstream
§ Regulatory:
governments
need
to
be
comfortable
with
the
rules
put
in
place
before
cars
are
released
to
general
public
-­‐
Mike
Ramsey,
research
director
at
Gartner
§ Opportunity:
Until
clear
leaders
and
standards
begin
to
emerge,
tech
innovators
are
all
vying
for
a
seat
at
the
table
-­‐
Mike
Ramsey,
research
director
at
Gartner
46
Source: http://www.gartner.com/newsroom/id/3784363
http://www.gartner.com/smarterwithgartner/the-road-to-connected-autonomous-cars/

47. ¡ Zero
Principle:
a
test
to
access
the
long-­‐term
potential
of
new
technology
§ Emerging
technology
common
trait:
their
introduction
dramatically
reduces
one
or
more
costs
to
nearly
zero
§ Steam
engine:
dramatically
reduced
the
cost
of
keeping
industrial
machinery
running.
§ Computer:
dramatically
reduce
the
cost
of
numerical
calculation
47

48. ¡ Zero
harm
§ Accidents
rate
will
dramatically
drop
§ Reduction
of
cost
of
traffic
related
hospital
bills
($18
billions)
and
wages
($33
billions)
¡ Zero
skill
§ People
no
longer
needs
to
learn
how
to
drive
§ Reduction
of
a
major
cost
of
transportation:
salary
¡ Zero
time
§ Reduction
of
indirect
cost
time
spent
driving
§ The
opportunity
cost
of
time
will
be
replaced
by
productive
work
or
enjoyable
time
¡ Zero
size
§ Human-­‐driven
cars
are
large
and
bulky
as
a
result
of
safety-­‐related
design
§ Driverless
car
can
be
smaller
and
more
lightweight
–
delivery
vehicle
will
be
only
as
large
as
the
object
they’re
delivery
48

49. 49

50. PhD
student:
Liu
Letao
50

51. Transport analytics AV technologies & simulations Transportation OR
Banishree
Ghosh
Vishnu
Prasad
Payyada
Songwei
Wu
Dr.
Hang
Yu
Nikola
Mitrovic
Tayyab
Muhammad
Asif
Selena
Jiang
Apratim
Choudhury
Prakash
Khunti
Pallavi
Mitra
Liu
Letao
Satyajit
Neogi
Xie
Chen
Nishant
Sinha
Dr.
Michael
Hoy
Dr.
Dang
Kang
Dr.
Changyun
Weng
Dr.
Yimin
Zhao
Dr.
Soumya
Dasgupta
Dr.
Tomasz
Maszczyk
Anatoliy
Prokhorchuk
Ramesh
Pandi
Ramasamy
Dr.
Kaveh
Azizian
Dr.
Sarat
Chandra
Dr.
Twinkle
Tripathy
Prof.
Patrick
Jaillet
(MIT)
Dr.
Ulrich
Fastenrath
(BMW)
Dr.
Emily
Low
(STE)
Dr.
F.
Klanner
(BMW)
Prof.
J.
Yuan
(NTU)
Dr.
N.
de
Boer
(NTU)
Dr.
Y-­‐H.
Eng
(SMART)
Wei-­‐K.
Leong
(SMART)
Zelin
Li
(SMART/MIT)
Dr.
Yu
Shen
(SMART)
Profs.
Daniela
Rus
and
Jinhua
Zhao
(MIT)
Prof.
Patrick
Jaillet
(MIT)
Government Industry
§ Lab
Members
51
§ Collaborators
§ Funding
support

52. THANK
YOU
Prof. Justin Dauwels
Nanyang Technological University, Singapore
jdauwels@ntu.edu.sg
www.dauwels.com

53. 53

54. ¡ Fuse
together
lidar,
camera,
GPS
and
GIS
data.
¡ Main
use
case
semi-­‐automatic
fitting
tool
to
make
offline
maps.
54

55. 55

56. 56
§ Develop
an
integrated
(Mobility
+
Network
+
Application)
simulator
to
test
V2X
protocols
and
applications
§ V2X
is
an
abbreviation
for
Vehicle-­‐to-­‐everything
communication
§ “Everything”
includes
other
vehicles
and
Road-­‐Side
Units
(RSU)
§ Purpose
of
V2X
is
to
create
a
network
of
communicating
vehicles
§ This
vehicular
network
can
be
leveraged
to
realize
multiple
applications
§ Traffic
congestion
detection
and
avoidance
§ Optimal
Speed
advisory
communication
§ Cooperative
Adaptive
Cruise
control
§ Collision
avoidance
§ Platooning
of
commercial
vehicles

57. § Road-­‐tests
of
V2X
protocols/applications
would
require
significant
resources
§ Costly
both
in
terms
of
hardware
and
administration
§ Safety
applications
(collision
avoidance,
brake
ahead,
etc.)
will
be
difficult
to
test
§ Existing
simulator
integration
environments
not
very
flexible
for
use
and
do
not
provide
much
support
for
the
simulators
that
we
aim
to
integrate
[1]
57
[1]: Choudhury, Apratim, et al. "An Integrated Simulation Environment for Testing V2X Protocols and Applications." Procedia Computer Science 80 (2016).

58. 58
Mobility
Compone
nt
(VISSIM)
Road Network
Calibrated
Traffic Model
Vehicle
routing
behavior
model
Driver
Behavior
model
Traffic Signal
logic
Applicatio
n
Compone
nt
(MATLAB)
Data
Exchange
Code
V2X
Application
algorithm
Traffic model
calibration and
validation algorithm
Network
Compone
nt
(NS3)
Propagation
Models
V2X
communicatio
n protocol
Packet
Delivery
Status
Delay in
packet
reception
Mobility model
Choudhury, Apratim, et al. "An Integrated Simulation Environment for Testing V2X Protocols and Applications." Procedia Computer Science 80 (2016).

59. 59
Simulation
of
the
Green
Light
Optimized
Speed
Advisory
(GLOSA)
§ The
GLOSA
application[1,2]
was
simulated
on
the
following
traffic
corridor
model
§ Basic
GLOSA
algorithm
§ Vehicles
come
within
communication
range
of
traffic
lights/RSU
§ Traffic
signal
is
constantly
broadcasting
remaining
phase
time
at
regular
intervals
§ Vehicles
receive
the
broadcast
and
calculate
the
speed
for
crossing
intersection
without
stopping
[1]: Stevanovic, Aleksandar, Jelka Stevanovic, and Cameron Kergaye. "Impact of signal phasing information accuracy on green light optimized speed advisory
systems." Proc. 92nd Annual Meeting of the Transportation Research Board (TRB), Washington DC, USA. 2013.
[2]: Katsaros, Konstantinos, et al. "Performance study of a Green Light Optimized Speed Advisory (GLOSA) application using an integrated cooperative ITS
simulation platform." 2011 7th International Wireless Communications and Mobile Computing Conference. IEEE, 2011.

60. 60
GLOSA
Simulation
Demo
Video

61. ¡ People
feel
lonely
§ Family
structure,
work
demands,
isolating
effect
of
television
and
internet,
false
friendships
formed
on
social
media
§ Third
space:
a
place
neither
work
nor
home,
but
to
hang
out
¡ Take
the
pod
–
meet
people
§ “Meet
people
option”:
passengers
could
choose
to
meet
people
of
the
same
age,
or
with
similar
patterns
of
web
browsing
and
Facebook
“likes”
when
they
are
taking
the
driverless
car
§ New
private
driverless
car
entertainment:
sex,
drugs,
alcohol
61

62.
¡ Cost:
§ high
human
labour
and
time
§ Cost
reduction:
▪ Digital
cameras
and
deep-­‐learning
software
continue
to
improve,
the
balance
of
reliance
on
a
car’s
operation
system
will
shift
from
stored
map
data
to
real-­‐time
scene
recognition
▪ Automation:
once
cars
are
capable
of
driving
themselves
around,
they
can
automatically
update
and
enhance
the
level
of
detail
in
their
own
on-­‐board
HD
digital
maps
¡ Potential
client:
§ City
motor-­‐vehicle
department:
to
keep
track
of
the
surface
condition
of
local
streets
and
monitor
the
erosion
of
lane
markers
§ Insurance
company
and
tech
company:
the
more
detailed
a
car’s
built-­‐in
digital
map,
the
safer
the
car,
the
greater
its
market
value
62

63. ¡ Commercial
delivery
§ Path
planning:
“order
of
operations”
–
complex
system
research
§ Deep
learning
with
HD
digital
maps:
information
of
the
exact
location,
status
and
growth
rate
of
every
detail
on
the
road
surface
will
be
optimized
§ More
convenient
delivery
system:
lower
the
transportation
barrier
for
small
business
and
online
shopping
¡ Taxi
§ Point
of
interest
discount:
passengers
who
agree
to
visit
the
point
of
interest
along
their
way
to
the
final
destination
will
get
discount
§ Data
share
discount:
due
to
the
existence
of
digital
recording
inside
the
driverless
car,
passengers
who
agree
to
share
their
own
data
will
get
discount
63

64. ¡ Bottom
up
object
detections
from
lidar,
camera,
radar.
§ Using
state-­‐of-­‐art
detectors
for
each
modality.
¡ Sensor
fusion
based
on
data
association
and
filtering.
§ Key
issue:
Many
3D
tracking
approaches
focus
on
well
separated
objects
[1-­‐3].
§ Key
novelty:
Improved
handling
of
ambiguous
scenarios,
using
model
based
clustering
step.
64
[1] H. Cho et al. "A multi-sensor fusion system for moving object detection and tracking in urban driving environments." ICRA 2014
[2] A. Petrovskaya, et al. "Model based vehicle detection and tracking for autonomous urban driving." Autonomous Robots 2009
[3] A. Vatavu et al. "Stereovision-based multiple object tracking in traffic scenarios using free-form obstacle delimiters and particle filters." IEEE Transactions on Intelligent Transportation Systems 2015

65. ¡ Single
vehicle
tracking
(offroad)
§ Focus
on
accurate
position
estimation.
§ Results
comparable
to
other
approaches
[1-­‐2].
65
[1] N. Wojke et al. "Moving vehicle detection and tracking in unstructured environments." ICRA 2012
[2] Y. Yeo et al. "A perception system for obstacle detection and tracking in rural, unstructured environment.“ FUSION 2014

66. ¡ Multiple
vehicle
tracking
66

67. 67

68. ¡ ECU:
engine
control
unit
¡ ABS:
antilock
braking
system
¡ TCU:
transmission
control
unit
¡ CAN:
controller
area
network
§ Working
principle:
ferries
data
back
and
forth
at
a
rate
of
approximately
1
Mbps
§ Key
challenges:
bandwidth
and
reliability
68

69. ¡ Waymo:
§ Company
Overview:
an
autonomous
car
development
company
spun
out
of
Google’s
parent
company,
Alphabet
Inc.
§ Technology
Advancement:
by
far
the
most
sophisticated
self-­‐driving
system,
it
simulated
over
a
billion
miles
of
driving,
its
car
have
had
the
most
self-­‐driving
experience
on
real
streets
(over
3
million
miles
in
multiple
cities)
¡ Other
Companies'
Move
§ General
Motor:
invested
a
billion
to
Cruise
Automation
§ Ford:
invested
a
billion
to
Argo
AI
§ Intel:
spent
15.3
billion
to
purchase
Mobileye
69
Source: https://spectrum.ieee.org/cars-that-think/transportation/self-driving/google-has-spent-over-11-billion-on-selfdriving-tech

70. ¡ Audi
A8
Capability
§ First
SAE
level
3
autonomous
vehicle
in
the
world
§ It
now
only
works
on
roads
with
proper
dividers,
easily
identified
lane
markings,
no
cross
traffic,
no
pedestrians,
no
merging
traffic
and
only
at
speeds
up
to
60km/h
(37mph)
¡ Partially
Autonomous
or
Fully
Autonomous
§ Google:
direct
SAE
level
5
(fully
autonomous)
system
for
commercial
release
§ Audi
A8:
first
with
SAE
level
3
system,
it
will
then
gradually
evolve
to
SAE
level
5
§ Partially
autonomous
drawback:
sometimes
the
system
will
handback
the
driving
task
to
the
human
driver,
which
might
be
dangerous
if
the
driver
is
sleeping
or
playing
games,
despite
Audi
A8
has
a
strong
alerting
system
¡ Law
Issue
§ Most
of
the
world:
they
use
the
cars
rules
announced
by
the
UN
committee
§ Exception:
US
and
China
will
make
and
use
their
own
convention
70
Source: https://www.slashgear.com/2019-audi-a8-level-3-autonomy-first-drive-chasing-the-perfect-jam-11499082/
Video: - test drive: https://www.youtube.com/watch?v=WsiUwq_M8lE&feature=youtu.be
- handback to human driver condition: https://www.youtube.com/watch?v=xDBqAYmGjyA&feature=youtu.be

71. ¡ GPS:
global
positioning
system
¡ Working
principle
§ Each
satellite
emits
its
own
unique
signature
beep
§ Beeps
stream
into
the
GPS
receiver
§ By
calculating
the
time
lapse
between
beeps,
a
GPS
receiver
is
able
to
calculate
its
won
exact
location
using
a
mathematical
process
known
as
“triangulation”
§ A
total
of
four
satellites
are
needed
to
pinpoint
exactly
where
the
receiver
is
¡ Weakness:
tunnel,
skyscrapers
–
urban
canyon
effect
71

72. ¡ IMU:
inertial
measurement
unit
¡ Components:
odometer,
accelerometer,
gyroscope,
compass
¡ Working
Principle:
dead
reckoning
§ Odometer
will
count
the
number
of
wheel
revolutions
from
tis
last
known
location
§ When
the
car
increases
its
speed,
slows
down
or
suddenly
changes
direction,
its
accelerometer
varies
§ Compass
will
provide
insight
of
the
direction
the
car
is
driving
§ Gyroscope
is
a
spinning
wheel
to
measure
the
pose,
the
direction
the
car’
nose
is
pointed
and
the
degree
its
body
is
tilted
¡ Weakness:
IMU
can’t
work
without
a
GPS
for
long
without
gradually
drifting
off
course
72

73. ¡ Driverless
License
§ Standard:
AV
passes
a
certain
minimal
safety
record
§ Current
situation:
California,
Nevada,
Michigan
and
Florida
provide
driverless
license
§ No
“human
in
the
loop”:
humans
don’t
drive
well
when
they
believe
a
capable
computer
is
handling
things
for
them
73

74. ¡ Crime
§ Robojacking:
walking
in
front
of
the
driverless
car
while
it
stopped
at
an
intersection
▪ Human
life
priority
:
the
driverless
car
will
be
programmed
to
spare
the
lives
of
a
human
whenever
possible
▪ Target:
lucrative,
high
value
cargo
or
passenger
▪ No
“manual
override”:
there
will
be
no
way
to
speed
away
to
safety,
bizarre
AI
nightmare
74

75. ¡ AVs
will
involve
large
amounts
of
connected
data;
smart
phones
and
tablets
–
hackers
can
access
personal
data
such
as
typical
journeys,
or
where
a
person
is
at
a
particular
time.
¡ Malicious
interference
with
AVs
could
have
serious
safety
implications.
§ Large-­‐scale
immobilization
of
AVs.
§ Misdirection
of
AVs.
75
Picture
Reference
:
http://image.slidesharecdn.com/mt5009autonomousvehiclesgroup5-­‐160413055228/95/autonomous-­‐vehicles-­‐
technologies-­‐economics-­‐and-­‐opportunities-­‐28-­‐638.jpg?cb=1460529672

76. 76
D. Kang, A. Goyal, M. Hoy, J. Yuan and J. Dauwels (2016) Efficient Terrian Classification and Mapping with LIDAR and Visual Information Autonomous Robots and Multirobot Systems (ARMS).

77. ¡ Sophisticated
predictive
traffic
analytics
&
route
design
§ Machine
learning:
study
the
factors
that
cause
traffic
congestion
§ Factors:
accidents,
road
construction,
sport
games,
social
events,
weather…
¡ Car
ownership
§ Improved
urban
transportation
efficiency
will
reduce
private
vehicle
ownership
§ The
reduced
number
of
vehicles
will
reduce
the
traffic
congestion
and
hence
improve
the
transportation
efficiency
77

78. ¡ Segway:
two-­‐wheeled
go-­‐cart
§ Prediction
▪ Steve
Jobs
predicted
the
Segway
would
be
“as
big
as
PC”
▪ Venture
capitalist
John
Doerr
pondered
whether
Segway
could
be
“maybe
bigger
than
the
Internet”
§ Current
situation:
Segway
serves
as
a
niche
transportation
solution,
enabling
tourists,
warehouse
workers
and
mail
delivery
personnel
to
roll
short
distances
78