Forrest Iandola discusses his journey from wanting to start a truck company as a child to co-founding DeepScale, a company that provides deep learning solutions for the automotive industry. He shares how he transitioned from studying mechanical engineering to computer science and deep learning research. Iandola describes DeepScale's founding in 2015 to address challenges automakers faced in training neural networks on large datasets and deploying them on embedded systems with limited computing power. DeepScale takes a full-stack approach across data, models, infrastructure and applications to deliver trained neural networks and software to automakers.

1. deepscale.ai
F O R R E S T
I A N D O L A
Co-­‐founder
and
CEO,
DeepScale
How
to
become
a
full-­‐stack
deep
learning
engineer

2. OUTLINE
• Part
1:
My
adventures
in
entrepreneurship
and
deep
learning
• How
I
became
a
full-­‐stack
DL
engineer
• DeepScale's
founding
story
• Part
2:
Full-­‐stack
deep
learning
engineering
• What
is
the
full
stack?
• How
to
organize
a
full-­‐stack
DL
team
• How
to
become
a
full-­‐stack
DL
engineer

3. PART
1
My
adventures
in
entrepreneurship
and
deep
learning

4. For
as
long
as
I
can
remember,
I
have
been
obsessed
with
cars
and
trucks

5. Fo's
Truck
Company

6. Illinois
Math
and
Science
Academy
(2005)

7. Mechanical
Engineering
Building
at
University
of
Illinois
(2008)

8. Siebel
Center
for
Computer
Science
at
University
of
Illinois
(2008)

9. What
my
classmates
planned
to
do
after
finishing
their
undergraduate
degree
in
CS
at
University
of
Illinois
(2012)
Entry-­‐level
jobs
at
Big
Companies
Starting
Companies
…and
countless
smartphone
app
startups
Grad
School

10. Starting
my
PhD
in
EECS
at
UC
Berkeley
(2012)
Kurt
Keutzer

11. Our
full-­‐stack
approach
FOR
DEEP
NEURAL
NETWORKS
v0.1
(2016)
Train
rapidly
using
multiprocessor
scaling
Collect
and
annotate
adequate
training
data
Develop
small
and
efficient
deep
neural
network
architectures
Create
efficient
implementations
for
embedded
hardware
F IREC AF F E
[ 1] T H E
D E E P S C A L E
C A R
SQ U EEZEN ET
[ 2]
B O DA
[ 3]
[1]
F.N.
Iandola,
K.
Ashraf,
M.W.
Moskewicz,
and
K.
Keutzer.
FireCaffe:
near-­‐linear
acceleration
of
deep
neural
network
training
on
compute
clusters.
CVPR,
2016.
[2]
F.N.
Iandola,
M.
Moskewicz,
K.
Ashraf,
S.
Han,
W.
Dally,
K.
Keutzer.
SqueezeNet:
AlexNet-­‐level
accuracy
with
50x
fewer
parameters
and
<1MB
model
size.
arXiv,
2016.
[3]
M.W.
Moskewicz,
F.N.
Iandola,
K.
Keutzer.
Boda-­‐RTC:
Productive
Generation
of
Portable,
Efficient
Code
for
Convolutional
Neural
Networks
on
Mobile
Computing
Platforms.
WiMob,
2016.

12. DeepScaleTHE
BEGINNING
(2015)
50x
speedup

13. DeepScaleHOW
OUR
BUSINESS
MODEL
CRYSTALLIZED
(2016)
First
we
learned…
• Automakers
have
gathered
lots
of
data
for
training
computer
vision
models
– for
use
in
autonomous
driving
• Training
deep
neural
nets
on
these
data
volumes
is
really
slow
(months
or
even
years)
• DeepScale's
distributed
training
system
can
help
After
more
discussions,
we
learned…
• Automakers
would
prefer
to
just
buy
the
"right"
deep
neural
networks,
already
trained
• For
mass-­‐production,
automakers
have
really small
processors
(100x
less
computation
than
full-­‐size
NVIDIA
GPUs)

14. Our
full-­‐stack
approach
FOR
DEEP
NEURAL
NETWORKS
v0.1
(2016)
Train
rapidly
using
multiprocessor
scaling
Collect
and
annotate
adequate
training
data
Develop
small
and
efficient
deep
neural
network
architectures
Create
efficient
implementations
for
embedded
hardware
F IREC AF F E
[ 1] T H E
D E E P S C A L E
C A R
SQ U EEZEN ET
[ 2]
B O DA
[ 3]
[1]
F.N.
Iandola,
K.
Ashraf,
M.W.
Moskewicz,
and
K.
Keutzer.
FireCaffe:
near-­‐linear
acceleration
of
deep
neural
network
training
on
compute
clusters.
CVPR,
2016.
[2]
F.N.
Iandola,
M.
Moskewicz,
K.
Ashraf,
S.
Han,
W.
Dally,
K.
Keutzer.
SqueezeNet:
AlexNet-­‐level
accuracy
with
50x
fewer
parameters
and
<1MB
model
size.
arXiv,
2016.
[3]
M.W.
Moskewicz,
F.N.
Iandola,
K.
Keutzer.
Boda-­‐RTC:
Productive
Generation
of
Portable,
Efficient
Code
for
Convolutional
Neural
Networks
on
Mobile
Computing
Platforms.
WiMob,
2016.

15. DeepScale
PERCEPTION
SYSTEMS
FOR
AUTOMATED
VEHICLES
http://deepscale.ai

16. My
story
in
one
slide
1993:
Planned
to
start
"Forrest's
Truck
Company"
when
I
grow
up
2008:
Went
to
college
at
University
of
Illinois
with
the
plan
to
study
Mechanical
Engineering
and
then
go
into
automotive
industry
2009:
Switched
to
computer
science;
gave
up
my
dreams
of
working
in
automotive
2012:
Considered
starting
a
startup,
but
choose
grad
school
instead
2013:
Started
doing
research
in
deep
learning
2015:
Started
DeepScale
2016:
Graduated
with
PhD
in
EECS
2016:
Focused
DeepScale
entirely
on
the
automotive
industry
2018:
Today,
DeepScale
is
on
track
to
supply
lifesaving
deep
learning
software
to
automakers
You
can't
connect
the
dots
looking
forward;
you
can
only
connect
them
looking
backwards.
So,
you
have
to
trust
that
the
dots
will
somehow
connect
in
your
future."
– Steve
Jobs

17. PART
2
The
playbook
for
full-­‐stack
deep
learning

18. Data
Infrastructure
Models
Applications
The
full
system
stack
FOR
DEEP
NEURAL
NETWORKS
v1.0 (2018)

19. Data
Data
Collection Data
Annotation Task
Definition
Accuracy
MetricsData
Storage Simulation

20. Models
Design
Space
ExplorationNew
Layer
Types
New
Objective
Functions
DNN
Model
Structure
Preprocessing
&
Featurization
Quantization
&
Compression

21. Infrastructure
DNN
Frameworks
Visualization
Tools
Deploy
on
cloud
&
embedded
hardware
Efficient
Computational
Kernels
Train
on
large-­‐scale
hardware
Distributed
Communication

22. Applications
GADGETSDATACENTERS
SOCIAL
MEDIA
ANALYSIS
WEB
INDEXING
GOVERNMENT
INTELLIGENCE
DRONES
SELF-­‐DRIVING
CARSSMARTPHONES
OTHER
ROBOTICS

23. Traditional
Workflow
for
DL
Research
and
Engineering
Models Infrastructure
(FYI,
these
are
supposed
to
be
silos)
Applications
Data

24. Breaking
Down
Silos
in
DL
Research
and
Engineering
Data
Models
Infrastructure
Applications

25. How
full-­‐stack
DL
teams
produce
full-­‐stack
engineers
Data
Models
Infrastructure
Applications

27. So,
how
do
I
get
started?

28. First
go
deep
Bring
your
skills
to
a
full-­‐stack
Deep
Learning
team
• I
bet
you're
already
pretty
good
some
piece
of
the
stack
• So,
start
by
applying
your
existing
skills to
an
DL-­‐related
project
• Work
with
a
group
who
has
complementary
skills
• Or,
choose
a
project
that
plays
to
your
strengths
Data
Models
Infrastructure
Applications

29. Then
go
wide
Becoming
a
full-­‐stack
Deep
Learning
engineer
• At
this
point,
you've
built
experience
applying
your
skills in
the
DL
field
• Now,
challenge
yourself
to
learn
new
skills
• Lots
of
places
to
learn
• If
you
worked
with
a
group,
you
probably
already
learned
from
your
team
members
• If
you're
in
a
company,
hopefully
you
work
with
people
who
have
complementary
skills
Data
Models
Infrastructure
Applications

30. Conclusions
• Full-­‐stack
Deep
Learning
spans
several
disciplines
• For
entrepreneurs
/
managers:
if
you
want
to
do
deep
learning
at
scale,
you
may
need
to
build
a
full-­‐stack
team
• For
engineers:
there
are
lots
of
ways
to
get
into
deep
learning
and
ultimately
to
be
come
a
full-­‐stack
DL
engineer