AI in Production
Crunch, Budapest
October 20, 2017
Hello
Hello
Hello
Hello
I am a fool
We’re here to talk
about reality.
Reality Bytes
Everything Else
Models
What you see What you get
1. Infrastructure
Reality Bytes
1. Infrastructure
2. Product
Reality Bytes
WORK
Infrastructure
Infrastructure
“Fanny pack”
0.8
Document
Model
Training Data
Infrastructure
Training Data
“Fanny pack”
Document
Yes / No
Training Data
Is this item a fanny pack?
Is this item relevant to the
query “fanny pack”?
On a scale of 1-10 how
“fanny pack” is this thing?
Which of these is the most
“fanny pack”?
Crowd Sourcing
Fanny Que?
Crowd Sourcing
So...umm...Is this data
any good?
Traffic to the
Rescue!
Training Data
Search results for “fanny pack”
SKIN WINS
Traffic FTW
Transfer
Learning!
Training Data
Wolf or Husky?
Transfer Learning
Does it work for
our data?
Transfer Learning
When it works
1. Embeddings
2. Super General Tasks
Training Data REGEX
Serving
Inferences
Infrastructure
Serving
Search Engine Ranking Model
Model
Search
Engine
Serving
1. RPC
Serving
1. RPC
a. Service Wrapper?
b. Monitoring/Deployment?
c. New Models?
d. Expertise?
Serving
1. RPC
40 Results
800ms/inference
32 seconds
Serving
1. RPC
2. Rewrite Search Engine
Model
Modern ML applications are comprised of an
increasingly diverse mix of libraries and systems...Even if
each of these libraries is optimized in isolation, real
pipelines combine multiple libraries, so production use at
scale usually requires a software engineering team to
rewrite the whole application in low-level code”
words = []
for word in title.split():
words.append(stripNonAlphaNumeric(
word.strip()
))
val words = title
.trim
.split(" ")
.map(w => w.trim)
.map(w => stripNonAlphaNumeric(w))
.map(w => stem(w))
Training
Inference
Serving
1. RPC
2. Rewrite
3. Cache
Search
Engine
Cache Model
Serving
A new hope?
Product
Top 3 Reasons AI
Products Fail
Product
Product
Top 3 Reasons AI Products Fail
1. Measuring stuff is hard.
Nightly Retraining
Model
Day 1
Top Result for “shorts”
Control Experiment
Clicks for “shorts”
Control
Control
Model
Experiment
Model
Experiment
Top Result for “shorts”
Day 1
Day 20
ExperimentControl
Clicks for “shorts”
Control
Control
Model
Experiment
Model
Experiment
1. Think about your objective.
2. Isolate.
3. Change one thing: the model.
4. Check your instrumentation.
Measurement is hard
Product
Top 3 Reasons AI Products Fail
1. Measuring stuff is hard.
2. Wrong Venue/Treatment/Approach.
The Taste Test
Choose an item you like
Items matching your taste
Venue
Home Page
Objective
↑ Registrations
Product
Top 3 Reasons AI Products Fail
1. Measuring stuff is hard.
2. Wrong Venue/Treatment/Approach.
3. Models First. Users Last.
Models First
Cool Model: word2vec!
bluenavy
Models First
Cool Model → Query Expansion!
(navy or blue) totenavy tote
Search Results for “indian sea”
Precision Problems
Garbage results
Recall Problems
Not enough results
f you wanted to make this one statement
as well.
Or another one.
Search results for “wedding dress”
“1930s”“1920s”
UsersModel
Search results for “jewellery”
UsersModel
Everything Else
Models
What you see What you get
AI Product✨
Denoument
Denoument
Things you should care about
1. Humans
2. Measurement
3. Training Data
4. Serving Inferences
5. Models
gio@relatedworks.io

AI in Production