Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
AI in Production
Crunch, Budapest
October 20, 2017
Hello
Hello
Hello
Hello
I am a fool
We’re here to talk
about reality.
Reality Bytes
Everything Else
Models
What you see What you get
1. Infrastructure
Reality Bytes
1. Infrastructure
2. Product
Reality Bytes
WORK
Infrastructure
Infrastructure
“Fanny pack”
0.8
Document
Model
Training Data
Infrastructure
Training Data
“Fanny pack”
Document
Yes / No
Training Data
Is this item a fanny pack?
Is this item relevant to the
query “fanny pack”?
On a scale of 1-10 how
“fanny pack” is this th...
Fanny Que?
Crowd Sourcing
So...umm...Is this data
any good?
Traffic to the
Rescue!
Training Data
Search results for “fanny pack”
SKIN WINS
Traffic FTW
Transfer
Learning!
Training Data
Wolf or Husky?
Transfer Learning
Does it work for
our data?
Transfer Learning
When it works
1. Embeddings
2. Super General Tasks
Training Data REGEX
Serving
Inferences
Infrastructure
Serving
Search Engine Ranking Model
Model
Search
Engine
Serving
1. RPC
Serving
1. RPC
a. Service Wrapper?
b. Monitoring/Deployment?
c. New Models?
d. Expertise?
Serving
1. RPC
40 Results
800ms/inference
32 seconds
Serving
1. RPC
2. Rewrite Search Engine
Model
Modern ML applications are comprised of an
increasingly diverse mix of libraries and systems...Even if
each of these libra...
words = []
for word in title.split():
words.append(stripNonAlphaNumeric(
word.strip()
))
val words = title
.trim
.split(" ...
Serving
1. RPC
2. Rewrite
3. Cache
Search
Engine
Cache Model
Serving
A new hope?
Product
Top 3 Reasons AI
Products Fail
Product
Product
Top 3 Reasons AI Products Fail
1. Measuring stuff is hard.
Nightly Retraining
Model
Day 1
Top Result for “shorts”
Control Experiment
Clicks for “shorts”
Control
Control
Model
Experiment
Model
Experiment
Top Result for “shorts”
Day 1
Day 20
ExperimentControl
Clicks for “shorts”
Control
Control
Model
Experiment
Model
Experiment
1. Think about your objective.
2. Isolate.
3. Change one thing: the model.
4. Check your instrumentation.
Measurement is h...
Product
Top 3 Reasons AI Products Fail
1. Measuring stuff is hard.
2. Wrong Venue/Treatment/Approach.
The Taste Test
Choose an item you like
Items matching your taste
Venue
Home Page
Objective
↑ Registrations
Product
Top 3 Reasons AI Products Fail
1. Measuring stuff is hard.
2. Wrong Venue/Treatment/Approach.
3. Models First. Use...
Models First
Cool Model: word2vec!
bluenavy
Models First
Cool Model → Query Expansion!
(navy or blue) totenavy tote
Search Results for “indian sea”
Precision Problems
Garbage results
Recall Problems
Not enough results
f you wanted to make this one statement
as well.
Or another one.
Search results for “wedding dress”
“1930s”“1920s”
UsersModel
Search results for “jewellery”
UsersModel
Everything Else
Models
What you see What you get
AI Product✨
Denoument
Denoument
Things you should care about
1. Humans
2. Measurement
3. Training Data
4. Serving Inferences
5. Models
gio@relatedworks.io
AI in Production
AI in Production
AI in Production
AI in Production
AI in Production
Upcoming SlideShare
Loading in …5
×

AI in Production

This is a talk I gave about all the unsexy things you have to deal with when putting AI in production.

  • Be the first to comment

  • Be the first to like this

AI in Production

  1. 1. AI in Production Crunch, Budapest October 20, 2017
  2. 2. Hello
  3. 3. Hello
  4. 4. Hello
  5. 5. Hello
  6. 6. I am a fool
  7. 7. We’re here to talk about reality.
  8. 8. Reality Bytes Everything Else Models What you see What you get
  9. 9. 1. Infrastructure Reality Bytes
  10. 10. 1. Infrastructure 2. Product Reality Bytes
  11. 11. WORK
  12. 12. Infrastructure
  13. 13. Infrastructure “Fanny pack” 0.8 Document Model
  14. 14. Training Data Infrastructure
  15. 15. Training Data “Fanny pack” Document Yes / No
  16. 16. Training Data
  17. 17. Is this item a fanny pack? Is this item relevant to the query “fanny pack”? On a scale of 1-10 how “fanny pack” is this thing? Which of these is the most “fanny pack”? Crowd Sourcing
  18. 18. Fanny Que?
  19. 19. Crowd Sourcing So...umm...Is this data any good?
  20. 20. Traffic to the Rescue! Training Data
  21. 21. Search results for “fanny pack”
  22. 22. SKIN WINS
  23. 23. Traffic FTW
  24. 24. Transfer Learning! Training Data
  25. 25. Wolf or Husky?
  26. 26. Transfer Learning Does it work for our data?
  27. 27. Transfer Learning When it works 1. Embeddings 2. Super General Tasks
  28. 28. Training Data REGEX
  29. 29. Serving Inferences Infrastructure
  30. 30. Serving Search Engine Ranking Model
  31. 31. Model Search Engine Serving 1. RPC
  32. 32. Serving 1. RPC a. Service Wrapper? b. Monitoring/Deployment? c. New Models? d. Expertise?
  33. 33. Serving 1. RPC 40 Results 800ms/inference 32 seconds
  34. 34. Serving 1. RPC 2. Rewrite Search Engine Model
  35. 35. Modern ML applications are comprised of an increasingly diverse mix of libraries and systems...Even if each of these libraries is optimized in isolation, real pipelines combine multiple libraries, so production use at scale usually requires a software engineering team to rewrite the whole application in low-level code”
  36. 36. words = [] for word in title.split(): words.append(stripNonAlphaNumeric( word.strip() )) val words = title .trim .split(" ") .map(w => w.trim) .map(w => stripNonAlphaNumeric(w)) .map(w => stem(w)) Training Inference
  37. 37. Serving 1. RPC 2. Rewrite 3. Cache Search Engine Cache Model
  38. 38. Serving A new hope?
  39. 39. Product
  40. 40. Top 3 Reasons AI Products Fail Product
  41. 41. Product Top 3 Reasons AI Products Fail 1. Measuring stuff is hard.
  42. 42. Nightly Retraining Model
  43. 43. Day 1 Top Result for “shorts” Control Experiment
  44. 44. Clicks for “shorts” Control Control Model Experiment Model Experiment
  45. 45. Top Result for “shorts” Day 1 Day 20 ExperimentControl
  46. 46. Clicks for “shorts” Control Control Model Experiment Model Experiment
  47. 47. 1. Think about your objective. 2. Isolate. 3. Change one thing: the model. 4. Check your instrumentation. Measurement is hard
  48. 48. Product Top 3 Reasons AI Products Fail 1. Measuring stuff is hard. 2. Wrong Venue/Treatment/Approach.
  49. 49. The Taste Test
  50. 50. Choose an item you like
  51. 51. Items matching your taste
  52. 52. Venue Home Page Objective ↑ Registrations
  53. 53. Product Top 3 Reasons AI Products Fail 1. Measuring stuff is hard. 2. Wrong Venue/Treatment/Approach. 3. Models First. Users Last.
  54. 54. Models First Cool Model: word2vec! bluenavy
  55. 55. Models First Cool Model → Query Expansion! (navy or blue) totenavy tote
  56. 56. Search Results for “indian sea”
  57. 57. Precision Problems Garbage results Recall Problems Not enough results
  58. 58. f you wanted to make this one statement as well. Or another one. Search results for “wedding dress”
  59. 59. “1930s”“1920s”
  60. 60. UsersModel
  61. 61. Search results for “jewellery”
  62. 62. UsersModel
  63. 63. Everything Else Models What you see What you get
  64. 64. AI Product✨
  65. 65. Denoument
  66. 66. Denoument Things you should care about 1. Humans 2. Measurement 3. Training Data 4. Serving Inferences 5. Models
  67. 67. gio@relatedworks.io

×