SlideShare a Scribd company logo
1 of 43
Autonomous
Vehicles Webinar
The intersection of robotics and
artificial intelligence
Streaming live via Hangouts
8pm CT - August 28th, 2016
Undergraduate student at University
of Illinois at Urbana - Champaign,
Class of 2017
B.S. Mechanical Engineering, Minor
in Electrical Engineering
Previous: PwC, Cummins, UIUC RA
Overview
I. What is an AV?
II. Technology
A. AI + Robotics = AVs
B. “Self-Driving Stack”
1. Sensing
2. Processing
3. Actuation
III. Up Next
What is an autonomous vehicle (AV) ?
Within the context of this discussion are focusing of
roadway motor vehicles.
AVs at their simplest would be a car with cruise-
control capability. At its most complex is an entirely
driverless vehicle.
Much like everything else in tech, there is a lot of
contention on how the classification should be
structured. What is ‘full autonomy’, etc? Thankfully,
the U.S. Dept. of Transportation developed an official
tiering with very clear distinctions.
Autonomous vehicles (AVs) are vehicles that are
capable movement with limited or no outside
instruction or intervention.
Autonomy, per the U.S. Dept. of Transportation:
SOURCE:
http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Departmen
t+of+Transportation+Releases+Policy+on+Automated+Vehicle+Develo
pment
Tier 1
Automation at this level involves one or more specific control functions. Examples include
electronic stability control or pre-charged brakes, where the vehicle automatically assists
with braking to enable the driver to regain control of the vehicle or stop faster than possible
by acting alone.
Tier 2
This level involves automation of at least two primary control functions designed to work in
unison to relieve the driver of control of those functions. An example of combined functions
enabling a Level 2 system is adaptive cruise control in combination with lane centering.
Tier 3
Vehicles at this level of automation enable the driver to cede full control of all safety-critical
functions under certain traffic or environmental conditions and in those conditions to rely
heavily on the vehicle to monitor for changes in those conditions requiring transition back to
driver control. The driver is expected to be available for occasional control, but with
sufficiently comfortable transition time.
Tier 4
The vehicle is designed to perform all safety-critical driving functions and monitor roadway
conditions entirely. The driver could provide destination input and is not expected to be
available for control at any time during the trip. This includes unoccupied vehicles.
AI + robotics = AVs
The intersection of artificial intelligence and robotics
An intelligent system that is
capable of taking
information/data and acting upon
that data, capable of learning how
to draw further insight
Study of design and control of
mechanical systems. On a closed-
loop, these systems are capable
of controlling themselves using
sensory information
● Modern machine learning
and AI techniques are
capable of this for specific
tasks (AlphaGo, Image
Classification)
● These similar techniques,
especially Deep Learning,
could be applied to vehicles
to teach it them drive given
high volumes of data
● Robotics is a well
understood field of study
with decades of research
and progress
● Has been applied to planes,
cars, etc, but in an extremely
limited fashion
● Autonomy cannot be “hard-
coded”, must be “learned”
AI
Robotics
The intersection of artificial intelligence and robotics: where the magic happens
Autonomous vehicles have always been a scientific dream. Planes have been capable of auto-pilot,
“self-flying” features for decades. How is it taking so long to happen on cars? Well, existing
infrastructures and roads cannot support rule-based robotic systems. There are too many possible
scenarios that could occur when driving, rules for robotic vehicles cannot be “hard-coded”.
True autonomy requires artificial intelligence. Intelligence that resembles the human capability to
decipher 3D space changing in time. With decades of advances in machine learning and artificial
intelligence we are nearing a time when machines are better at understanding roads than we are.
Technology
Deep-Dive
There is a lot going on under the hood, let’s try to simplify it
Pose Graph
LIDAR
Graph SLAM
1 Sensing
Processing2
Actuation3
The “Self-Driving
Stack”
The architecture of autonomy
Commands are sent to
Control Unit which tells
engine/motor to speed
up or slow down. An
analogous process
occurs for vehicle
steering.
Sensor data is passed on ro
algorithms and is processed
locally (GPUs) or over a
distributed network (the Cloud)
Autonomous Vehicle Architecture
0100011010101010
0010110101000101
Video Camera (still images
processing, pixels)
LIDAR (light-radar, point clouds)
Specific sensors (e.g. red light
detection, pedestrian detection)
1
Sensing Processing
2 3
Actuation
Autonomous Vehicle Architecture
Electromechanical
Actuation
Sensing
Sensing
Processing/Computation
1
1
2
3
1
Sensing
Technology Deep-
Dive:
Sensing
1 Sensing
Processing2
Actuation3
LIDAR, video cameras, and radar/sonic sensors are most
commonly used for gathering vehicle environment data
Video Camera (still images
processing, pixels)
LIDAR (light-radar, point clouds)
Specific sensors (e.g. red light
detection, stop signs)
Sensing
● “Light radar” - LIDAR
● Generates point clouds that are 3D representations
of the driving environment
● Seen as the high-resolution input data that is
integral to SLAM + RRT techniques
● Simple video cameras input feeds of still images
that can be processed for lanes, obstacles,
pedestrians, etc
● Cheap and effective, now being heavily
implemented as the choice data for deep learning
● Case-specific sensors are heavily leveraged to
provide insight in areas that LiDAR and cameras
cannot handle in a general way
● Ex) a specific camera pointed at where stoplights
are - feed directly into a specific algorithm for
sensing red, yellow, and green colors
A deep-dive on LIDAR Sensing
● LIDAR has quickly become a go-to
sensor for autonomous applications.
Velodyne is an industry leader with
relatively cheap, easy to calibrate units
● LIDAR units send out pulses of light and
measure the time to return, which can
be used to compute the distance of an
object
● A rotating LIDAR sensor gathering
distances of objects at different angles
can gather enough points of data to
construct a “point cloud”
● It is evident how useful point clouds are,
similar effect as the human eye, 3D
representation of space in real time
Researchers at MIT in collaboration with DARPA have been
able to fabricate and implement a solid-state LIDAR chip:
“Our lidar chips promise to be orders of magnitude smaller, lighter, and cheaper than lidar
systems available on the market today. They also have the potential to be much more robust
because of the lack of moving parts, with a non-mechanical beam steering 1,000 times faster
than what is currently achieved in mechanical lidar systems.”
“At the moment, our on-chip lidar system can detect objects at ranges of up to 2 meters, though
we hope to achieve a 10-meter range within a year. The minimum range is around 5
centimeters. We have demonstrated centimeter longitudinal resolution and expect 3-cm lateral
resolution at 2 meters. There is a clear development path towards lidar on a chip technology
that can reach 100 meters, with the possibility of going even farther.”
Massive size and price reduction of LIDAR sensors could
fundamentally change approach to autonomous vehicles,
drones, prosthetics, etc.
“MIT and DARPA pack LIDAR sensor onto single chip”
IEEE Spectrum, Aug 4 2016
A new, cheaper, solid state LIDAR emerging Sensing
SOURCE:
http://spectrum.ieee.org/tech-
talk/semiconductors/optoelectronics/mit-lidar-on-a-chip
The sensing stage needs to gather lots of data from different
sources in order to fully understand the environment
Video Camera (still images
processing, pixels)
LIDAR (light-radar, point clouds)
Specific sensors (e.g. red light
detection, stop signs)
Sensing
Technology Deep-
Dive: Processing
1 Sensing
Processing2
Actuation3
z
The Processing Stack Processing
● CPUs, GPUs, SoCs on
board
● Large amounts of flash
memory
● “Cloud” compute
● Powerful endpoints,
limited only by speed of
data communication
Computational Methods
Local
Distributed
RRT*, SLAM, Kinematics
End-to-End, DNN, CNN
Motion Planning / Mapping
Machine Learning / Deep
Learning
Intersections, Left-turn
Rule-based systems
LIDAR point
cloud data
Video Camera
Feed
Computational Muscle
Input Data
Output Commands
Computational Methods
Motion Planning Artificial Intelligence (ML/Deep Learning)
Motion Planning - Algorithm 1: SLAM Processing
What is the world around me (mapping)
● Sense from various positions
● Integrate measurements to produce map
Where in I am in the world (localization)
● Sense
● Relate sensor reading to a world model (a priori maps)
● Compute (probabilistic) location relative to model
**above points taken from CMU paper cited below
Depicted to the right is a Kalman Filter being applied to
position measurements and sensory information that in turn
generates a Gaussian distribution of the possible positions
Simultaneous localization and mapping (SLAM)
SOURCE:
http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-
Kalman-Mapping_howie.pdf
Motion Planning - Algorithm 1: SLAM Processing
SLAM Walkthrough
SOURCE: http://ocw.mit.edu/courses/aeronautics-and-
astronautics/16-412j-cognitive-robotics-spring-
2005/projects/1aslam_blas_repo.pdf
Motion Planning - Algorithm 1: SLAM Processing
SLAM Walkthrough 1 2 3 4 5 6 7
Robot
Landmark
SOURCE: http://ocw.mit.edu/courses/aeronautics-and-
astronautics/16-412j-cognitive-robotics-spring-
2005/projects/1aslam_blas_repo.pdf
Motion Planning - Algorithm 1: SLAM Processing
SLAM Walkthrough 1 2 3 4 5 6 7
Robot
Landmark
SOURCE: http://ocw.mit.edu/courses/aeronautics-and-
astronautics/16-412j-cognitive-robotics-spring-
2005/projects/1aslam_blas_repo.pdf
Motion Planning - Algorithm 1: SLAM Processing
SLAM Walkthrough 1 2 3 4 5 6 7
Robot
Landmark
SOURCE: http://ocw.mit.edu/courses/aeronautics-and-
astronautics/16-412j-cognitive-robotics-spring-
2005/projects/1aslam_blas_repo.pdf
Motion Planning - Algorithm 1: SLAM Processing
SLAM Walkthrough 1 2 3 4 5 6 7
Robot
Landmark
SOURCE: http://ocw.mit.edu/courses/aeronautics-and-
astronautics/16-412j-cognitive-robotics-spring-
2005/projects/1aslam_blas_repo.pdf
Motion Planning - Algorithm 1: SLAM Processing
SLAM Walkthrough 1 2 3 4 5 6 7
SOURCE: http://ocw.mit.edu/courses/aeronautics-and-
astronautics/16-412j-cognitive-robotics-spring-
2005/projects/1aslam_blas_repo.pdf
Motion Planning - Algorithm 1: SLAM Processing
SLAM Walkthrough 1 2 3 4 5 6 7
SOURCE: http://ocw.mit.edu/courses/aeronautics-and-
astronautics/16-412j-cognitive-robotics-spring-
2005/projects/1aslam_blas_repo.pdf
Motion Planning - Algorithm 1: SLAM Processing
SLAM Walkthrough 1 2 3 4 5 6 7
Location Likelihood Distribution
SOURCE: http://ocw.mit.edu/courses/aeronautics-and-
astronautics/16-412j-cognitive-robotics-spring-
2005/projects/1aslam_blas_repo.pdf
Motion Planning - Algorithm 2: RRTs Processing
● Rapidly-exploring Random Trees (RRTs) are a set of
exploratory algorithms that are useful for trajectory
planning
● With a set of polygonal obstacles, an RRT can generate
a possible path from the starting configuration to the
ending (goal) configuration
● Sample paths are then input to a controller/model
representation of the vehicle dynamics and the
predicted trajectory of the vehicle is computed (x)
● The runtime of these algorithms can vary since
accuracy is based on samples taken
Once a probabilistic localization is realized, a probabilistic
path can be generated using RRTs
SOURCE:
http://acl.mit.edu/papers/KuwataTCST09.pdf
http://www.staff.science.uu.nl/~gerae101/pdf/compare.pdf
Motion Planning - SLAM + RRTs = advanced guesswork Processing
● In order to obtain a higher-resolution probabilistic
model of the ideal trajectory more samples need to be
taken and more computations performed, hence the
need for massive compute power!
● It is understandable that a car driving 60mph would
have issues performing this depth of computation in a
rapidly changing environment
For more in-depth understanding of algorithmic robotics
motion planning works check out SLAM for Dummies
A probabilistic path generated from probabilistic input poses
issues for vehicles moving at high speeds
SOURCE:
http://workshops.acin.tuwien.ac.at/clutter2014/papers/ric201
4_submission_9.pdf
http://acl.mit.edu/papers/KuwataTCST09.pdf
**white spots
represent sampled
points used to
generate RRT
Artificial Intelligence (ML/Deep Learning) Processing
● Newly emerging methodologies all revolve around deep
learning via neural nets
○ RNNs, CNNs, GANs, Autoencoding, etc
● Two main forces driving adoption of these methods:
○ Cheaper and more powerful local and cloud
computing (GPUs)
○ Open-source deep learning platforms
(TensorFlow)
These deep learning methodologies are injecting intelligence
into vehicles, feeding them massive amounts of data, and
letting them learn
Please check out this Deep Learning Playground for a better
visualization of the concept
Artificial Intelligence Methods
Feature extraction
performed by a
CNN on video from
a forward facing
camera. Model
was able to
determine what
were road edges
with relative
accuracy (via
NVIDIA)
Lane centering generator that predicts
path of vehicles based on video input
from front facing camera (via
Comma.ai)
Artificial Intelligence (ML/Deep Learning) Processing
Important Academic Papers Regarding Deep Learning Processing
● NVIDIA - “End to End Learning for Self-Driving Cars”
Video input from a forward facing camera is trained against steering wheel position and deep
learning networks are capable of detecting important road features with limited additional
nudging in the right direction
● Comma.ai - "Learning A Driving Simulator"
Using video input with no additional training metadata (IMU, wheel angle) auto-encoded video
was generated, predicting many frames into the future while maintaining road features
● Radford et al. (Facebook AI) - "Unsupervised Representational Learning w/ Deep GANs"
Seminal work on deep learning auto-encoding that allowed Comma.ai breakthrough and
similar work i.e. “Autoencoding Blade Runner”
● NYU & Facebook AI - “Deep Multi-Scale Video Prediction Beyond Mean Square Error”
Implications of these papers indicate deep learning is a highly promising solution for AVs
Computational Muscle
CPUs GPUs SoCs (Onboard) Distributed Computing (Cloud)
Computational muscle limited to local compute, for now Processing
● Current self-driving solutions are all implemented with
local compute due to the need for simplicity, focusing
on software first
● Utilizing GPUs and special SoCs to perform simple
operations (i.e. with pixels and point clouds) at
massive scale in parallel
● New TPUs (tensor processing units) are being
designed specifically for the purpose of machine
learning and AI, as well as new platforms emerging
specifically for AVs
● A distributed network offering massive computational
muscle would be ideal, but does not offer immediate
simplicity due to latency, security, reliability, ...
● Movement toward an “AWS for AVs” is a huge
opportunity, many companies are actively working on
Two paradigms currently, local compute (CPUs, SoCs, GPUs)
and distributed computation over a network (Cloud)
Google’s new TPU that
powered AlphaGo
Technology Deep-
Dive: Actuation
1 Sensing
Processing2
Actuation3
Actuation stage is primarily based on field of controls and
electromechanical systems
Actuation
● The control unit is circuit hardware that manages
electromechanical systems within a car
● Large amount of low-level controls have been
standardized into protocols like CAN
● Most well-studied and understood portion of the
self-driving technology stack, high feasibility
relative to other parts of the “stack”
● Companies like Delphi and Bosch are large players
in this space and have invested decades of time
and research into vehicle controls
● Innovation in this space is much more iterative,
positioning incumbents to dominate the controls
hardware/software for AVs
The processing stage sends commands via bus like CAN
or similar architectures to engine control unit/modules
Up Next
High level trends, “Self-Driving Stack” trends, general comments
● Costs of sensors is falling
through the floor
● No “best sensor” yet,
converging toward LIDAR
and video camera,
dependent on processing
approaches
● Accuracy limits, distance
limits, latency of data
feed (LIDAR especially)
are improving
exponentially with cost
Sensing
● Models vs. Neural vs.
Mixed, no “best practice”
● Local compute only
implementation yet, will
transition toward “Cloud”
same way as software
● Mapping is important but
AI vector bank is the new
data network effect
● V2V, V2I communication
cannot be relied upon
Processing
● Actuation / controls is out
in front of the rest of the
tech, not a limiting factor
● Mission critical safety and
reliability needs to be
investigated more
heavily, beyond “Six
Sigma”
● Incumbents well
positioned
● Security has not been
investigated thoroughly,
will emerge as a large
space later on
Actuation / Controls
My Thoughts
1
Data Network Effects for AI systems are
the sole most important factor to long-
term success. Advantage Uber and Tesla.
LIDAR and GPU companies will become
important OEMs and provide it as a
service to Big Auto, only non-commodity
hardware that matters to enable “AV”.
2
The inherently difficult problems are
software related, Big Auto not positioned
to “win” at software. Defer to startups
with ex-researchers.
3
- Otto (recently acquired by Uber for ~$600M)
- Zoox
- $200M fundraise with not evening a landing page, talk
about stealthy! Team consists of “fathers of AVs”
- Comma.ai
- Attempting to offer autonomy enablement to vehicle
mfgs.
- Drive.ai
- Software for AVs, not much info, rockstar team with
very deep background
- Peloton Tech
- More immediate use case for semi-autonomy with
platooning. Strategic investors, UPS venture arm is a
positive signal.
- NuTonomy
- Released functioning product in Singapore, great
team
Companies to pay
attention to
Thank You

More Related Content

What's hot

Autonomous Driving
Autonomous DrivingAutonomous Driving
Autonomous DrivingUsman Hashmi
 
Gps tracking system
Gps tracking system Gps tracking system
Gps tracking system Sumit Kumar
 
Autonomous cars
Autonomous carsAutonomous cars
Autonomous carsAmal Jose
 
Google driverless car
Google driverless carGoogle driverless car
Google driverless carpiyupatel99
 
Self driving cars -
Self driving cars - Self driving cars -
Self driving cars - Hany G. Amer
 
Google Driverless Cars
Google Driverless Cars Google Driverless Cars
Google Driverless Cars Gopikrishna VM
 
Computer Vision for autonomous driving
Computer Vision for autonomous drivingComputer Vision for autonomous driving
Computer Vision for autonomous drivingBill Liu
 
Connected & Driverless vehicles: the road to Safe & Secure mobility?
Connected & Driverless vehicles: the road to Safe & Secure mobility?Connected & Driverless vehicles: the road to Safe & Secure mobility?
Connected & Driverless vehicles: the road to Safe & Secure mobility?Bill Harpley
 
V2V- Vehicle to Vehicle Communication
V2V- Vehicle to Vehicle CommunicationV2V- Vehicle to Vehicle Communication
V2V- Vehicle to Vehicle CommunicationSrijanKumar18
 
Automatic vehicle accident detection and messaging system using gsm and gps m...
Automatic vehicle accident detection and messaging system using gsm and gps m...Automatic vehicle accident detection and messaging system using gsm and gps m...
Automatic vehicle accident detection and messaging system using gsm and gps m...mahesh_rman
 
GNSS Jamming Detection, Localization and Mitigation
GNSS Jamming Detection, Localization and MitigationGNSS Jamming Detection, Localization and Mitigation
GNSS Jamming Detection, Localization and MitigationMarco Lisi
 

What's hot (20)

Autonomous cars
Autonomous carsAutonomous cars
Autonomous cars
 
Waymo Driverless car
Waymo Driverless carWaymo Driverless car
Waymo Driverless car
 
Autonomous Driving
Autonomous DrivingAutonomous Driving
Autonomous Driving
 
Autonomous car
Autonomous carAutonomous car
Autonomous car
 
Gps tracking system
Gps tracking system Gps tracking system
Gps tracking system
 
Autonomous cars
Autonomous carsAutonomous cars
Autonomous cars
 
Google driverless car
Google driverless carGoogle driverless car
Google driverless car
 
Self driving cars -
Self driving cars - Self driving cars -
Self driving cars -
 
Autonomous cars by ihazn
Autonomous cars by ihaznAutonomous cars by ihazn
Autonomous cars by ihazn
 
Smart parking - Happiestminds !
Smart parking - Happiestminds !Smart parking - Happiestminds !
Smart parking - Happiestminds !
 
Self driving car
Self driving carSelf driving car
Self driving car
 
Google Driverless Cars
Google Driverless Cars Google Driverless Cars
Google Driverless Cars
 
ADAS.ppt.pptx
ADAS.ppt.pptxADAS.ppt.pptx
ADAS.ppt.pptx
 
Automated vehicle
Automated vehicleAutomated vehicle
Automated vehicle
 
Computer Vision for autonomous driving
Computer Vision for autonomous drivingComputer Vision for autonomous driving
Computer Vision for autonomous driving
 
Autonomous car original
Autonomous car originalAutonomous car original
Autonomous car original
 
Connected & Driverless vehicles: the road to Safe & Secure mobility?
Connected & Driverless vehicles: the road to Safe & Secure mobility?Connected & Driverless vehicles: the road to Safe & Secure mobility?
Connected & Driverless vehicles: the road to Safe & Secure mobility?
 
V2V- Vehicle to Vehicle Communication
V2V- Vehicle to Vehicle CommunicationV2V- Vehicle to Vehicle Communication
V2V- Vehicle to Vehicle Communication
 
Automatic vehicle accident detection and messaging system using gsm and gps m...
Automatic vehicle accident detection and messaging system using gsm and gps m...Automatic vehicle accident detection and messaging system using gsm and gps m...
Automatic vehicle accident detection and messaging system using gsm and gps m...
 
GNSS Jamming Detection, Localization and Mitigation
GNSS Jamming Detection, Localization and MitigationGNSS Jamming Detection, Localization and Mitigation
GNSS Jamming Detection, Localization and Mitigation
 

Viewers also liked

Probing the atmosphere - new radar & lidar technologies for remote sensing of...
Probing the atmosphere - new radar & lidar technologies for remote sensing of...Probing the atmosphere - new radar & lidar technologies for remote sensing of...
Probing the atmosphere - new radar & lidar technologies for remote sensing of...CLIC Innovation Ltd
 
Value for a Connected Vehicle (IoT)
Value for a Connected Vehicle (IoT)Value for a Connected Vehicle (IoT)
Value for a Connected Vehicle (IoT)Brian Loomis
 
TU Automotive Osram Presentation Final
TU Automotive  Osram Presentation  FinalTU Automotive  Osram Presentation  Final
TU Automotive Osram Presentation FinalRajeev Thakur, P.E.
 
Sensors and Data Management for Autonomous Vehicles report 2015 by Yole Devel...
Sensors and Data Management for Autonomous Vehicles report 2015 by Yole Devel...Sensors and Data Management for Autonomous Vehicles report 2015 by Yole Devel...
Sensors and Data Management for Autonomous Vehicles report 2015 by Yole Devel...Yole Developpement
 
The Race to 2021: The State of Autonomous Vehicles and a "Who's Who" of Indus...
The Race to 2021: The State of Autonomous Vehicles and a "Who's Who" of Indus...The Race to 2021: The State of Autonomous Vehicles and a "Who's Who" of Indus...
The Race to 2021: The State of Autonomous Vehicles and a "Who's Who" of Indus...Altimeter, a Prophet Company
 
Autonomous driving revolution- trends, challenges and machine learning 
Autonomous driving revolution- trends, challenges and machine learning  Autonomous driving revolution- trends, challenges and machine learning 
Autonomous driving revolution- trends, challenges and machine learning  Junli Gu
 
Autonomous Vehicles
Autonomous VehiclesAutonomous Vehicles
Autonomous VehiclesIoT613
 
Artificial Intelligence and the Data Center
Artificial Intelligence and the Data CenterArtificial Intelligence and the Data Center
Artificial Intelligence and the Data Centersflaig
 
Codes and standards
Codes and standardsCodes and standards
Codes and standardssflaig
 
Study HERE SBD - How autonomous vehicles could relieve or worsen traffic cong...
Study HERE SBD - How autonomous vehicles could relieve or worsen traffic cong...Study HERE SBD - How autonomous vehicles could relieve or worsen traffic cong...
Study HERE SBD - How autonomous vehicles could relieve or worsen traffic cong...Ludovic Privat
 
Steffen Rendle, Research Scientist, Google at MLconf SF
Steffen Rendle, Research Scientist, Google at MLconf SFSteffen Rendle, Research Scientist, Google at MLconf SF
Steffen Rendle, Research Scientist, Google at MLconf SFMLconf
 
Cruise control devices
Cruise control devicesCruise control devices
Cruise control devicesgireeshlal
 
Junli Gu at AI Frontiers: Autonomous Driving Revolution
Junli Gu at AI Frontiers: Autonomous Driving RevolutionJunli Gu at AI Frontiers: Autonomous Driving Revolution
Junli Gu at AI Frontiers: Autonomous Driving RevolutionAI Frontiers
 
TensorFlow in Context
TensorFlow in ContextTensorFlow in Context
TensorFlow in ContextAltoros
 

Viewers also liked (20)

Probing the atmosphere - new radar & lidar technologies for remote sensing of...
Probing the atmosphere - new radar & lidar technologies for remote sensing of...Probing the atmosphere - new radar & lidar technologies for remote sensing of...
Probing the atmosphere - new radar & lidar technologies for remote sensing of...
 
Value for a Connected Vehicle (IoT)
Value for a Connected Vehicle (IoT)Value for a Connected Vehicle (IoT)
Value for a Connected Vehicle (IoT)
 
TU Automotive Osram Presentation Final
TU Automotive  Osram Presentation  FinalTU Automotive  Osram Presentation  Final
TU Automotive Osram Presentation Final
 
Autonomous Car
Autonomous CarAutonomous Car
Autonomous Car
 
Autonomous Vehicles
Autonomous VehiclesAutonomous Vehicles
Autonomous Vehicles
 
LIDAR
LIDARLIDAR
LIDAR
 
Sensors and Data Management for Autonomous Vehicles report 2015 by Yole Devel...
Sensors and Data Management for Autonomous Vehicles report 2015 by Yole Devel...Sensors and Data Management for Autonomous Vehicles report 2015 by Yole Devel...
Sensors and Data Management for Autonomous Vehicles report 2015 by Yole Devel...
 
The Race to 2021: The State of Autonomous Vehicles and a "Who's Who" of Indus...
The Race to 2021: The State of Autonomous Vehicles and a "Who's Who" of Indus...The Race to 2021: The State of Autonomous Vehicles and a "Who's Who" of Indus...
The Race to 2021: The State of Autonomous Vehicles and a "Who's Who" of Indus...
 
Autonomous driving revolution- trends, challenges and machine learning 
Autonomous driving revolution- trends, challenges and machine learning  Autonomous driving revolution- trends, challenges and machine learning 
Autonomous driving revolution- trends, challenges and machine learning 
 
Autonomous Vehicles
Autonomous VehiclesAutonomous Vehicles
Autonomous Vehicles
 
2014 MATC Spring Lecture Series: Chris Schwarz
2014 MATC Spring Lecture Series: Chris Schwarz2014 MATC Spring Lecture Series: Chris Schwarz
2014 MATC Spring Lecture Series: Chris Schwarz
 
Artificial Intelligence and the Data Center
Artificial Intelligence and the Data CenterArtificial Intelligence and the Data Center
Artificial Intelligence and the Data Center
 
Codes and standards
Codes and standardsCodes and standards
Codes and standards
 
Study HERE SBD - How autonomous vehicles could relieve or worsen traffic cong...
Study HERE SBD - How autonomous vehicles could relieve or worsen traffic cong...Study HERE SBD - How autonomous vehicles could relieve or worsen traffic cong...
Study HERE SBD - How autonomous vehicles could relieve or worsen traffic cong...
 
Steffen Rendle, Research Scientist, Google at MLconf SF
Steffen Rendle, Research Scientist, Google at MLconf SFSteffen Rendle, Research Scientist, Google at MLconf SF
Steffen Rendle, Research Scientist, Google at MLconf SF
 
Cruise control devices
Cruise control devicesCruise control devices
Cruise control devices
 
Bayesian risk assessment of autonomous vehicles
Bayesian risk assessment of autonomous vehiclesBayesian risk assessment of autonomous vehicles
Bayesian risk assessment of autonomous vehicles
 
Junli Gu at AI Frontiers: Autonomous Driving Revolution
Junli Gu at AI Frontiers: Autonomous Driving RevolutionJunli Gu at AI Frontiers: Autonomous Driving Revolution
Junli Gu at AI Frontiers: Autonomous Driving Revolution
 
The Revolution of Deep Learning
The Revolution of Deep LearningThe Revolution of Deep Learning
The Revolution of Deep Learning
 
TensorFlow in Context
TensorFlow in ContextTensorFlow in Context
TensorFlow in Context
 

Similar to Autonomous Vehicles: the Intersection of Robotics and Artificial Intelligence

Perceiving and understanding the world for ADAS and Autonomous _Peter Labazie...
Perceiving and understanding the world for ADAS and Autonomous _Peter Labazie...Perceiving and understanding the world for ADAS and Autonomous _Peter Labazie...
Perceiving and understanding the world for ADAS and Autonomous _Peter Labazie...Peter Labaziewicz
 
How I built my robot with ROS and deep learning
How I built my robot with ROS and deep learningHow I built my robot with ROS and deep learning
How I built my robot with ROS and deep learningMeetupDataScienceRoma
 
Autonomous Vehicle Hardware word (1).pdf
Autonomous Vehicle Hardware word (1).pdfAutonomous Vehicle Hardware word (1).pdf
Autonomous Vehicle Hardware word (1).pdfDorleControls
 
Driverless Vehicles: Future Outlook on Intelligent Transportation
Driverless Vehicles: Future Outlook on Intelligent TransportationDriverless Vehicles: Future Outlook on Intelligent Transportation
Driverless Vehicles: Future Outlook on Intelligent TransportationIJERA Editor
 
Worknet smart pole overview
Worknet smart pole overviewWorknet smart pole overview
Worknet smart pole overviewMike Maziarka
 
Autonomous Path Following Robot For Industrial Applications
Autonomous Path Following Robot For Industrial ApplicationsAutonomous Path Following Robot For Industrial Applications
Autonomous Path Following Robot For Industrial Applications338DEEPRAJANILPAGARE
 
ROAD The Road Event Awareness Dataset for Autonomous Driving.docx
ROAD The Road Event Awareness Dataset for Autonomous Driving.docxROAD The Road Event Awareness Dataset for Autonomous Driving.docx
ROAD The Road Event Awareness Dataset for Autonomous Driving.docxShakas Technologies
 
A Smart Approach for Traffic Management
A Smart Approach for Traffic Management	A Smart Approach for Traffic Management
A Smart Approach for Traffic Management journal ijrtem
 
An autonomous driverless car
An autonomous driverless carAn autonomous driverless car
An autonomous driverless carAlexander Decker
 
IRJET- Working of Autonomous Vehicles
IRJET-  	  Working of Autonomous VehiclesIRJET-  	  Working of Autonomous Vehicles
IRJET- Working of Autonomous VehiclesIRJET Journal
 
Jorge Sebastiao "Using AI for Smart traffic Management"
Jorge Sebastiao "Using AI for Smart traffic Management"Jorge Sebastiao "Using AI for Smart traffic Management"
Jorge Sebastiao "Using AI for Smart traffic Management"Lviv Startup Club
 
Self-Driving Cars With Convolutional Neural Networks (CNN.pptx
Self-Driving Cars With Convolutional Neural Networks (CNN.pptxSelf-Driving Cars With Convolutional Neural Networks (CNN.pptx
Self-Driving Cars With Convolutional Neural Networks (CNN.pptxssuserf79e761
 
Autonomus_CarsFinal ok.pptx
Autonomus_CarsFinal ok.pptxAutonomus_CarsFinal ok.pptx
Autonomus_CarsFinal ok.pptxalphaalpha17
 
How AI is Disrupting Traffic Management in Smart City
How AI is DisruptingTraffic Management in Smart CityHow AI is DisruptingTraffic Management in Smart City
How AI is Disrupting Traffic Management in Smart CityJorge Sebastiao
 
13 9246 it implementation of cloud connected (edit ari)
13 9246 it implementation of cloud connected (edit ari)13 9246 it implementation of cloud connected (edit ari)
13 9246 it implementation of cloud connected (edit ari)IAESIJEECS
 
IRJET- IoT based Motion Control System of a Robotic car
IRJET-  	  IoT based Motion Control System of a Robotic carIRJET-  	  IoT based Motion Control System of a Robotic car
IRJET- IoT based Motion Control System of a Robotic carIRJET Journal
 

Similar to Autonomous Vehicles: the Intersection of Robotics and Artificial Intelligence (20)

Smart car
Smart carSmart car
Smart car
 
Perceiving and understanding the world for ADAS and Autonomous _Peter Labazie...
Perceiving and understanding the world for ADAS and Autonomous _Peter Labazie...Perceiving and understanding the world for ADAS and Autonomous _Peter Labazie...
Perceiving and understanding the world for ADAS and Autonomous _Peter Labazie...
 
How I built my robot with ROS and deep learning
How I built my robot with ROS and deep learningHow I built my robot with ROS and deep learning
How I built my robot with ROS and deep learning
 
Autonomous Vehicle Hardware word (1).pdf
Autonomous Vehicle Hardware word (1).pdfAutonomous Vehicle Hardware word (1).pdf
Autonomous Vehicle Hardware word (1).pdf
 
Driverless Vehicles: Future Outlook on Intelligent Transportation
Driverless Vehicles: Future Outlook on Intelligent TransportationDriverless Vehicles: Future Outlook on Intelligent Transportation
Driverless Vehicles: Future Outlook on Intelligent Transportation
 
MEETUP Maggio - Team Automata
MEETUP Maggio - Team AutomataMEETUP Maggio - Team Automata
MEETUP Maggio - Team Automata
 
Worknet smart pole overview
Worknet smart pole overviewWorknet smart pole overview
Worknet smart pole overview
 
Autonomous Path Following Robot For Industrial Applications
Autonomous Path Following Robot For Industrial ApplicationsAutonomous Path Following Robot For Industrial Applications
Autonomous Path Following Robot For Industrial Applications
 
ROAD The Road Event Awareness Dataset for Autonomous Driving.docx
ROAD The Road Event Awareness Dataset for Autonomous Driving.docxROAD The Road Event Awareness Dataset for Autonomous Driving.docx
ROAD The Road Event Awareness Dataset for Autonomous Driving.docx
 
A Smart Approach for Traffic Management
A Smart Approach for Traffic Management	A Smart Approach for Traffic Management
A Smart Approach for Traffic Management
 
An autonomous driverless car
An autonomous driverless carAn autonomous driverless car
An autonomous driverless car
 
IRJET- Working of Autonomous Vehicles
IRJET-  	  Working of Autonomous VehiclesIRJET-  	  Working of Autonomous Vehicles
IRJET- Working of Autonomous Vehicles
 
Jorge Sebastiao "Using AI for Smart traffic Management"
Jorge Sebastiao "Using AI for Smart traffic Management"Jorge Sebastiao "Using AI for Smart traffic Management"
Jorge Sebastiao "Using AI for Smart traffic Management"
 
Self-Driving Cars With Convolutional Neural Networks (CNN.pptx
Self-Driving Cars With Convolutional Neural Networks (CNN.pptxSelf-Driving Cars With Convolutional Neural Networks (CNN.pptx
Self-Driving Cars With Convolutional Neural Networks (CNN.pptx
 
vinay 155.pptx
vinay 155.pptxvinay 155.pptx
vinay 155.pptx
 
Autonomus_CarsFinal ok.pptx
Autonomus_CarsFinal ok.pptxAutonomus_CarsFinal ok.pptx
Autonomus_CarsFinal ok.pptx
 
How AI is Disrupting Traffic Management in Smart City
How AI is DisruptingTraffic Management in Smart CityHow AI is DisruptingTraffic Management in Smart City
How AI is Disrupting Traffic Management in Smart City
 
13 9246 it implementation of cloud connected (edit ari)
13 9246 it implementation of cloud connected (edit ari)13 9246 it implementation of cloud connected (edit ari)
13 9246 it implementation of cloud connected (edit ari)
 
IRJET- IoT based Motion Control System of a Robotic car
IRJET-  	  IoT based Motion Control System of a Robotic carIRJET-  	  IoT based Motion Control System of a Robotic car
IRJET- IoT based Motion Control System of a Robotic car
 
final_report
final_reportfinal_report
final_report
 

Recently uploaded

What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dashnarutouzumaki53779
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...AliaaTarek5
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 

Recently uploaded (20)

What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dash
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 

Autonomous Vehicles: the Intersection of Robotics and Artificial Intelligence

  • 1. Autonomous Vehicles Webinar The intersection of robotics and artificial intelligence Streaming live via Hangouts 8pm CT - August 28th, 2016 Undergraduate student at University of Illinois at Urbana - Champaign, Class of 2017 B.S. Mechanical Engineering, Minor in Electrical Engineering Previous: PwC, Cummins, UIUC RA
  • 2. Overview I. What is an AV? II. Technology A. AI + Robotics = AVs B. “Self-Driving Stack” 1. Sensing 2. Processing 3. Actuation III. Up Next
  • 3. What is an autonomous vehicle (AV) ? Within the context of this discussion are focusing of roadway motor vehicles. AVs at their simplest would be a car with cruise- control capability. At its most complex is an entirely driverless vehicle. Much like everything else in tech, there is a lot of contention on how the classification should be structured. What is ‘full autonomy’, etc? Thankfully, the U.S. Dept. of Transportation developed an official tiering with very clear distinctions. Autonomous vehicles (AVs) are vehicles that are capable movement with limited or no outside instruction or intervention.
  • 4. Autonomy, per the U.S. Dept. of Transportation: SOURCE: http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Departmen t+of+Transportation+Releases+Policy+on+Automated+Vehicle+Develo pment Tier 1 Automation at this level involves one or more specific control functions. Examples include electronic stability control or pre-charged brakes, where the vehicle automatically assists with braking to enable the driver to regain control of the vehicle or stop faster than possible by acting alone. Tier 2 This level involves automation of at least two primary control functions designed to work in unison to relieve the driver of control of those functions. An example of combined functions enabling a Level 2 system is adaptive cruise control in combination with lane centering. Tier 3 Vehicles at this level of automation enable the driver to cede full control of all safety-critical functions under certain traffic or environmental conditions and in those conditions to rely heavily on the vehicle to monitor for changes in those conditions requiring transition back to driver control. The driver is expected to be available for occasional control, but with sufficiently comfortable transition time. Tier 4 The vehicle is designed to perform all safety-critical driving functions and monitor roadway conditions entirely. The driver could provide destination input and is not expected to be available for control at any time during the trip. This includes unoccupied vehicles.
  • 6. The intersection of artificial intelligence and robotics An intelligent system that is capable of taking information/data and acting upon that data, capable of learning how to draw further insight Study of design and control of mechanical systems. On a closed- loop, these systems are capable of controlling themselves using sensory information ● Modern machine learning and AI techniques are capable of this for specific tasks (AlphaGo, Image Classification) ● These similar techniques, especially Deep Learning, could be applied to vehicles to teach it them drive given high volumes of data ● Robotics is a well understood field of study with decades of research and progress ● Has been applied to planes, cars, etc, but in an extremely limited fashion ● Autonomy cannot be “hard- coded”, must be “learned” AI Robotics
  • 7. The intersection of artificial intelligence and robotics: where the magic happens Autonomous vehicles have always been a scientific dream. Planes have been capable of auto-pilot, “self-flying” features for decades. How is it taking so long to happen on cars? Well, existing infrastructures and roads cannot support rule-based robotic systems. There are too many possible scenarios that could occur when driving, rules for robotic vehicles cannot be “hard-coded”. True autonomy requires artificial intelligence. Intelligence that resembles the human capability to decipher 3D space changing in time. With decades of advances in machine learning and artificial intelligence we are nearing a time when machines are better at understanding roads than we are.
  • 9. There is a lot going on under the hood, let’s try to simplify it Pose Graph LIDAR Graph SLAM
  • 11. Commands are sent to Control Unit which tells engine/motor to speed up or slow down. An analogous process occurs for vehicle steering. Sensor data is passed on ro algorithms and is processed locally (GPUs) or over a distributed network (the Cloud) Autonomous Vehicle Architecture 0100011010101010 0010110101000101 Video Camera (still images processing, pixels) LIDAR (light-radar, point clouds) Specific sensors (e.g. red light detection, pedestrian detection) 1 Sensing Processing 2 3 Actuation
  • 14. LIDAR, video cameras, and radar/sonic sensors are most commonly used for gathering vehicle environment data Video Camera (still images processing, pixels) LIDAR (light-radar, point clouds) Specific sensors (e.g. red light detection, stop signs) Sensing ● “Light radar” - LIDAR ● Generates point clouds that are 3D representations of the driving environment ● Seen as the high-resolution input data that is integral to SLAM + RRT techniques ● Simple video cameras input feeds of still images that can be processed for lanes, obstacles, pedestrians, etc ● Cheap and effective, now being heavily implemented as the choice data for deep learning ● Case-specific sensors are heavily leveraged to provide insight in areas that LiDAR and cameras cannot handle in a general way ● Ex) a specific camera pointed at where stoplights are - feed directly into a specific algorithm for sensing red, yellow, and green colors
  • 15. A deep-dive on LIDAR Sensing ● LIDAR has quickly become a go-to sensor for autonomous applications. Velodyne is an industry leader with relatively cheap, easy to calibrate units ● LIDAR units send out pulses of light and measure the time to return, which can be used to compute the distance of an object ● A rotating LIDAR sensor gathering distances of objects at different angles can gather enough points of data to construct a “point cloud” ● It is evident how useful point clouds are, similar effect as the human eye, 3D representation of space in real time
  • 16. Researchers at MIT in collaboration with DARPA have been able to fabricate and implement a solid-state LIDAR chip: “Our lidar chips promise to be orders of magnitude smaller, lighter, and cheaper than lidar systems available on the market today. They also have the potential to be much more robust because of the lack of moving parts, with a non-mechanical beam steering 1,000 times faster than what is currently achieved in mechanical lidar systems.” “At the moment, our on-chip lidar system can detect objects at ranges of up to 2 meters, though we hope to achieve a 10-meter range within a year. The minimum range is around 5 centimeters. We have demonstrated centimeter longitudinal resolution and expect 3-cm lateral resolution at 2 meters. There is a clear development path towards lidar on a chip technology that can reach 100 meters, with the possibility of going even farther.” Massive size and price reduction of LIDAR sensors could fundamentally change approach to autonomous vehicles, drones, prosthetics, etc. “MIT and DARPA pack LIDAR sensor onto single chip” IEEE Spectrum, Aug 4 2016 A new, cheaper, solid state LIDAR emerging Sensing SOURCE: http://spectrum.ieee.org/tech- talk/semiconductors/optoelectronics/mit-lidar-on-a-chip
  • 17. The sensing stage needs to gather lots of data from different sources in order to fully understand the environment Video Camera (still images processing, pixels) LIDAR (light-radar, point clouds) Specific sensors (e.g. red light detection, stop signs) Sensing
  • 18. Technology Deep- Dive: Processing 1 Sensing Processing2 Actuation3
  • 19. z The Processing Stack Processing ● CPUs, GPUs, SoCs on board ● Large amounts of flash memory ● “Cloud” compute ● Powerful endpoints, limited only by speed of data communication Computational Methods Local Distributed RRT*, SLAM, Kinematics End-to-End, DNN, CNN Motion Planning / Mapping Machine Learning / Deep Learning Intersections, Left-turn Rule-based systems LIDAR point cloud data Video Camera Feed Computational Muscle Input Data Output Commands
  • 20. Computational Methods Motion Planning Artificial Intelligence (ML/Deep Learning)
  • 21. Motion Planning - Algorithm 1: SLAM Processing What is the world around me (mapping) ● Sense from various positions ● Integrate measurements to produce map Where in I am in the world (localization) ● Sense ● Relate sensor reading to a world model (a priori maps) ● Compute (probabilistic) location relative to model **above points taken from CMU paper cited below Depicted to the right is a Kalman Filter being applied to position measurements and sensory information that in turn generates a Gaussian distribution of the possible positions Simultaneous localization and mapping (SLAM) SOURCE: http://www.cs.cmu.edu/~motionplanning/lecture/Chap8- Kalman-Mapping_howie.pdf
  • 22. Motion Planning - Algorithm 1: SLAM Processing SLAM Walkthrough SOURCE: http://ocw.mit.edu/courses/aeronautics-and- astronautics/16-412j-cognitive-robotics-spring- 2005/projects/1aslam_blas_repo.pdf
  • 23. Motion Planning - Algorithm 1: SLAM Processing SLAM Walkthrough 1 2 3 4 5 6 7 Robot Landmark SOURCE: http://ocw.mit.edu/courses/aeronautics-and- astronautics/16-412j-cognitive-robotics-spring- 2005/projects/1aslam_blas_repo.pdf
  • 24. Motion Planning - Algorithm 1: SLAM Processing SLAM Walkthrough 1 2 3 4 5 6 7 Robot Landmark SOURCE: http://ocw.mit.edu/courses/aeronautics-and- astronautics/16-412j-cognitive-robotics-spring- 2005/projects/1aslam_blas_repo.pdf
  • 25. Motion Planning - Algorithm 1: SLAM Processing SLAM Walkthrough 1 2 3 4 5 6 7 Robot Landmark SOURCE: http://ocw.mit.edu/courses/aeronautics-and- astronautics/16-412j-cognitive-robotics-spring- 2005/projects/1aslam_blas_repo.pdf
  • 26. Motion Planning - Algorithm 1: SLAM Processing SLAM Walkthrough 1 2 3 4 5 6 7 Robot Landmark SOURCE: http://ocw.mit.edu/courses/aeronautics-and- astronautics/16-412j-cognitive-robotics-spring- 2005/projects/1aslam_blas_repo.pdf
  • 27. Motion Planning - Algorithm 1: SLAM Processing SLAM Walkthrough 1 2 3 4 5 6 7 SOURCE: http://ocw.mit.edu/courses/aeronautics-and- astronautics/16-412j-cognitive-robotics-spring- 2005/projects/1aslam_blas_repo.pdf
  • 28. Motion Planning - Algorithm 1: SLAM Processing SLAM Walkthrough 1 2 3 4 5 6 7 SOURCE: http://ocw.mit.edu/courses/aeronautics-and- astronautics/16-412j-cognitive-robotics-spring- 2005/projects/1aslam_blas_repo.pdf
  • 29. Motion Planning - Algorithm 1: SLAM Processing SLAM Walkthrough 1 2 3 4 5 6 7 Location Likelihood Distribution SOURCE: http://ocw.mit.edu/courses/aeronautics-and- astronautics/16-412j-cognitive-robotics-spring- 2005/projects/1aslam_blas_repo.pdf
  • 30. Motion Planning - Algorithm 2: RRTs Processing ● Rapidly-exploring Random Trees (RRTs) are a set of exploratory algorithms that are useful for trajectory planning ● With a set of polygonal obstacles, an RRT can generate a possible path from the starting configuration to the ending (goal) configuration ● Sample paths are then input to a controller/model representation of the vehicle dynamics and the predicted trajectory of the vehicle is computed (x) ● The runtime of these algorithms can vary since accuracy is based on samples taken Once a probabilistic localization is realized, a probabilistic path can be generated using RRTs SOURCE: http://acl.mit.edu/papers/KuwataTCST09.pdf http://www.staff.science.uu.nl/~gerae101/pdf/compare.pdf
  • 31. Motion Planning - SLAM + RRTs = advanced guesswork Processing ● In order to obtain a higher-resolution probabilistic model of the ideal trajectory more samples need to be taken and more computations performed, hence the need for massive compute power! ● It is understandable that a car driving 60mph would have issues performing this depth of computation in a rapidly changing environment For more in-depth understanding of algorithmic robotics motion planning works check out SLAM for Dummies A probabilistic path generated from probabilistic input poses issues for vehicles moving at high speeds SOURCE: http://workshops.acin.tuwien.ac.at/clutter2014/papers/ric201 4_submission_9.pdf http://acl.mit.edu/papers/KuwataTCST09.pdf **white spots represent sampled points used to generate RRT
  • 32. Artificial Intelligence (ML/Deep Learning) Processing ● Newly emerging methodologies all revolve around deep learning via neural nets ○ RNNs, CNNs, GANs, Autoencoding, etc ● Two main forces driving adoption of these methods: ○ Cheaper and more powerful local and cloud computing (GPUs) ○ Open-source deep learning platforms (TensorFlow) These deep learning methodologies are injecting intelligence into vehicles, feeding them massive amounts of data, and letting them learn Please check out this Deep Learning Playground for a better visualization of the concept Artificial Intelligence Methods Feature extraction performed by a CNN on video from a forward facing camera. Model was able to determine what were road edges with relative accuracy (via NVIDIA) Lane centering generator that predicts path of vehicles based on video input from front facing camera (via Comma.ai)
  • 33. Artificial Intelligence (ML/Deep Learning) Processing
  • 34. Important Academic Papers Regarding Deep Learning Processing ● NVIDIA - “End to End Learning for Self-Driving Cars” Video input from a forward facing camera is trained against steering wheel position and deep learning networks are capable of detecting important road features with limited additional nudging in the right direction ● Comma.ai - "Learning A Driving Simulator" Using video input with no additional training metadata (IMU, wheel angle) auto-encoded video was generated, predicting many frames into the future while maintaining road features ● Radford et al. (Facebook AI) - "Unsupervised Representational Learning w/ Deep GANs" Seminal work on deep learning auto-encoding that allowed Comma.ai breakthrough and similar work i.e. “Autoencoding Blade Runner” ● NYU & Facebook AI - “Deep Multi-Scale Video Prediction Beyond Mean Square Error” Implications of these papers indicate deep learning is a highly promising solution for AVs
  • 35. Computational Muscle CPUs GPUs SoCs (Onboard) Distributed Computing (Cloud)
  • 36. Computational muscle limited to local compute, for now Processing ● Current self-driving solutions are all implemented with local compute due to the need for simplicity, focusing on software first ● Utilizing GPUs and special SoCs to perform simple operations (i.e. with pixels and point clouds) at massive scale in parallel ● New TPUs (tensor processing units) are being designed specifically for the purpose of machine learning and AI, as well as new platforms emerging specifically for AVs ● A distributed network offering massive computational muscle would be ideal, but does not offer immediate simplicity due to latency, security, reliability, ... ● Movement toward an “AWS for AVs” is a huge opportunity, many companies are actively working on Two paradigms currently, local compute (CPUs, SoCs, GPUs) and distributed computation over a network (Cloud) Google’s new TPU that powered AlphaGo
  • 37. Technology Deep- Dive: Actuation 1 Sensing Processing2 Actuation3
  • 38. Actuation stage is primarily based on field of controls and electromechanical systems Actuation ● The control unit is circuit hardware that manages electromechanical systems within a car ● Large amount of low-level controls have been standardized into protocols like CAN ● Most well-studied and understood portion of the self-driving technology stack, high feasibility relative to other parts of the “stack” ● Companies like Delphi and Bosch are large players in this space and have invested decades of time and research into vehicle controls ● Innovation in this space is much more iterative, positioning incumbents to dominate the controls hardware/software for AVs The processing stage sends commands via bus like CAN or similar architectures to engine control unit/modules
  • 40. High level trends, “Self-Driving Stack” trends, general comments ● Costs of sensors is falling through the floor ● No “best sensor” yet, converging toward LIDAR and video camera, dependent on processing approaches ● Accuracy limits, distance limits, latency of data feed (LIDAR especially) are improving exponentially with cost Sensing ● Models vs. Neural vs. Mixed, no “best practice” ● Local compute only implementation yet, will transition toward “Cloud” same way as software ● Mapping is important but AI vector bank is the new data network effect ● V2V, V2I communication cannot be relied upon Processing ● Actuation / controls is out in front of the rest of the tech, not a limiting factor ● Mission critical safety and reliability needs to be investigated more heavily, beyond “Six Sigma” ● Incumbents well positioned ● Security has not been investigated thoroughly, will emerge as a large space later on Actuation / Controls
  • 41. My Thoughts 1 Data Network Effects for AI systems are the sole most important factor to long- term success. Advantage Uber and Tesla. LIDAR and GPU companies will become important OEMs and provide it as a service to Big Auto, only non-commodity hardware that matters to enable “AV”. 2 The inherently difficult problems are software related, Big Auto not positioned to “win” at software. Defer to startups with ex-researchers. 3
  • 42. - Otto (recently acquired by Uber for ~$600M) - Zoox - $200M fundraise with not evening a landing page, talk about stealthy! Team consists of “fathers of AVs” - Comma.ai - Attempting to offer autonomy enablement to vehicle mfgs. - Drive.ai - Software for AVs, not much info, rockstar team with very deep background - Peloton Tech - More immediate use case for semi-autonomy with platooning. Strategic investors, UPS venture arm is a positive signal. - NuTonomy - Released functioning product in Singapore, great team Companies to pay attention to

Editor's Notes

  1. Want to review the A16Z presentation, like to use that type of timeline!
  2. Want to review the A16Z presentation, like to use that type of timeline! DOT Link: http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Department+of+Transportation+Releases+Policy+on+Automated+Vehicle+Development
  3. Want to review the A16Z presentation, like to use that type of timeline! DOT Link: http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Department+of+Transportation+Releases+Policy+on+Automated+Vehicle+Development
  4. SOURCE: https://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwichvLwtbjOAhVI5mMKHXbIBSsQjRwIBw&url=https%3A%2F%2Fwww.google.com%2Fselfdrivingcar%2Fpaint%2F&psig=AFQjCNF8HMvdtSGahQ0YlSaBAy-MxdS94A&ust=1470972244533229
  5. Want to review the A16Z presentation, like to use that type of timeline! DOT Link: http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Department+of+Transportation+Releases+Policy+on+Automated+Vehicle+Development
  6. Want to review the A16Z presentation, like to use that type of timeline! DOT Link: http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Department+of+Transportation+Releases+Policy+on+Automated+Vehicle+Development
  7. Want to review the A16Z presentation, like to use that type of timeline! DOT Link: http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Department+of+Transportation+Releases+Policy+on+Automated+Vehicle+Development
  8. Want to review the A16Z presentation, like to use that type of timeline! DOT Link: http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Department+of+Transportation+Releases+Policy+on+Automated+Vehicle+Development
  9. Want to review the A16Z presentation, like to use that type of timeline! DOT Link: http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Department+of+Transportation+Releases+Policy+on+Automated+Vehicle+Development
  10. Want to review the A16Z presentation, like to use that type of timeline! DOT Link: http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Department+of+Transportation+Releases+Policy+on+Automated+Vehicle+Development
  11. Want to review the A16Z presentation, like to use that type of timeline! DOT Link: http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Department+of+Transportation+Releases+Policy+on+Automated+Vehicle+Development http://velodynelidar.com/lidar/hdlpressroom/pdf/Articles/A%20Perception-Driven%20Autonomous%20Urban%20Vehicle.pdf
  12. Want to review the A16Z presentation, like to use that type of timeline! DOT Link: http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Department+of+Transportation+Releases+Policy+on+Automated+Vehicle+Development
  13. Want to review the A16Z presentation, like to use that type of timeline! DOT Link: http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Department+of+Transportation+Releases+Policy+on+Automated+Vehicle+Development
  14. Want to review the A16Z presentation, like to use that type of timeline! DOT Link: http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Department+of+Transportation+Releases+Policy+on+Automated+Vehicle+Development
  15. Want to review the A16Z presentation, like to use that type of timeline! DOT Link: http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Department+of+Transportation+Releases+Policy+on+Automated+Vehicle+Development
  16. http://www.forbes.com/sites/tomcoughlin/2016/04/08/rolling-memory-in-autonomous-cars/2/#2121c8fc5645 http://www.nvidia.com/object/drive-px.html
  17. http://www.forbes.com/sites/tomcoughlin/2016/04/08/rolling-memory-in-autonomous-cars/2/#2121c8fc5645 http://www.nvidia.com/object/drive-px.html http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf
  18. http://www.forbes.com/sites/tomcoughlin/2016/04/08/rolling-memory-in-autonomous-cars/2/#2121c8fc5645 http://www.nvidia.com/object/drive-px.html http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf
  19. http://www.forbes.com/sites/tomcoughlin/2016/04/08/rolling-memory-in-autonomous-cars/2/#2121c8fc5645 http://www.nvidia.com/object/drive-px.html http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf
  20. http://www.forbes.com/sites/tomcoughlin/2016/04/08/rolling-memory-in-autonomous-cars/2/#2121c8fc5645 http://www.nvidia.com/object/drive-px.html http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf
  21. http://www.forbes.com/sites/tomcoughlin/2016/04/08/rolling-memory-in-autonomous-cars/2/#2121c8fc5645 http://www.nvidia.com/object/drive-px.html http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf
  22. http://www.forbes.com/sites/tomcoughlin/2016/04/08/rolling-memory-in-autonomous-cars/2/#2121c8fc5645 http://www.nvidia.com/object/drive-px.html http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf
  23. http://www.forbes.com/sites/tomcoughlin/2016/04/08/rolling-memory-in-autonomous-cars/2/#2121c8fc5645 http://www.nvidia.com/object/drive-px.html http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf
  24. http://www.forbes.com/sites/tomcoughlin/2016/04/08/rolling-memory-in-autonomous-cars/2/#2121c8fc5645 http://www.nvidia.com/object/drive-px.html http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf
  25. http://www.forbes.com/sites/tomcoughlin/2016/04/08/rolling-memory-in-autonomous-cars/2/#2121c8fc5645 http://www.nvidia.com/object/drive-px.html http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf
  26. http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf http://acl.mit.edu/papers/KuwataTCST09.pdf http://www.staff.science.uu.nl/~gerae101/pdf/compare.pdf
  27. http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf http://acl.mit.edu/papers/KuwataTCST09.pdf http://www.staff.science.uu.nl/~gerae101/pdf/compare.pdf
  28. http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf http://acl.mit.edu/papers/KuwataTCST09.pdf http://www.staff.science.uu.nl/~gerae101/pdf/compare.pdf
  29. http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf http://acl.mit.edu/papers/KuwataTCST09.pdf http://www.staff.science.uu.nl/~gerae101/pdf/compare.pdf
  30. http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf http://acl.mit.edu/papers/KuwataTCST09.pdf http://www.staff.science.uu.nl/~gerae101/pdf/compare.pdf
  31. http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf http://acl.mit.edu/papers/KuwataTCST09.pdf http://www.staff.science.uu.nl/~gerae101/pdf/compare.pdf
  32. Want to review the A16Z presentation, like to use that type of timeline! DOT Link: http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Department+of+Transportation+Releases+Policy+on+Automated+Vehicle+Development
  33. http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf http://acl.mit.edu/papers/KuwataTCST09.pdf http://www.staff.science.uu.nl/~gerae101/pdf/compare.pdf
  34. http://www.cs.cmu.edu/~motionplanning/lecture/Chap8-Kalman-Mapping_howie.pdf http://acl.mit.edu/papers/KuwataTCST09.pdf http://www.staff.science.uu.nl/~gerae101/pdf/compare.pdf
  35. Want to review the A16Z presentation, like to use that type of timeline! DOT Link: http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Department+of+Transportation+Releases+Policy+on+Automated+Vehicle+Development
  36. Want to review the A16Z presentation, like to use that type of timeline! DOT Link: http://www.nhtsa.gov/About+NHTSA/Press+Releases/U.S.+Department+of+Transportation+Releases+Policy+on+Automated+Vehicle+Development
  37. Want to review the A16Z presentation, like to use that type of timeline!