Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

AI in Production

45 views

Published on

This is a talk I gave about all the unsexy things you have to deal with when putting AI in production.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

AI in Production

  1. 1. AI in Production Crunch, Budapest October 20, 2017
  2. 2. Hello
  3. 3. Hello
  4. 4. Hello
  5. 5. Hello
  6. 6. I am a fool
  7. 7. We’re here to talk about reality.
  8. 8. Reality Bytes Everything Else Models What you see What you get
  9. 9. 1. Infrastructure Reality Bytes
  10. 10. 1. Infrastructure 2. Product Reality Bytes
  11. 11. WORK
  12. 12. Infrastructure
  13. 13. Infrastructure “Fanny pack” 0.8 Document Model
  14. 14. Training Data Infrastructure
  15. 15. Training Data “Fanny pack” Document Yes / No
  16. 16. Training Data
  17. 17. Is this item a fanny pack? Is this item relevant to the query “fanny pack”? On a scale of 1-10 how “fanny pack” is this thing? Which of these is the most “fanny pack”? Crowd Sourcing
  18. 18. Fanny Que?
  19. 19. Crowd Sourcing So...umm...Is this data any good?
  20. 20. Traffic to the Rescue! Training Data
  21. 21. Search results for “fanny pack”
  22. 22. SKIN WINS
  23. 23. Traffic FTW
  24. 24. Transfer Learning! Training Data
  25. 25. Wolf or Husky?
  26. 26. Transfer Learning Does it work for our data?
  27. 27. Transfer Learning When it works 1. Embeddings 2. Super General Tasks
  28. 28. Training Data REGEX
  29. 29. Serving Inferences Infrastructure
  30. 30. Serving Search Engine Ranking Model
  31. 31. Model Search Engine Serving 1. RPC
  32. 32. Serving 1. RPC a. Service Wrapper? b. Monitoring/Deployment? c. New Models? d. Expertise?
  33. 33. Serving 1. RPC 40 Results 800ms/inference 32 seconds
  34. 34. Serving 1. RPC 2. Rewrite Search Engine Model
  35. 35. Modern ML applications are comprised of an increasingly diverse mix of libraries and systems...Even if each of these libraries is optimized in isolation, real pipelines combine multiple libraries, so production use at scale usually requires a software engineering team to rewrite the whole application in low-level code”
  36. 36. words = [] for word in title.split(): words.append(stripNonAlphaNumeric( word.strip() )) val words = title .trim .split(" ") .map(w => w.trim) .map(w => stripNonAlphaNumeric(w)) .map(w => stem(w)) Training Inference
  37. 37. Serving 1. RPC 2. Rewrite 3. Cache Search Engine Cache Model
  38. 38. Serving A new hope?
  39. 39. Product
  40. 40. Top 3 Reasons AI Products Fail Product
  41. 41. Product Top 3 Reasons AI Products Fail 1. Measuring stuff is hard.
  42. 42. Nightly Retraining Model
  43. 43. Day 1 Top Result for “shorts” Control Experiment
  44. 44. Clicks for “shorts” Control Control Model Experiment Model Experiment
  45. 45. Top Result for “shorts” Day 1 Day 20 ExperimentControl
  46. 46. Clicks for “shorts” Control Control Model Experiment Model Experiment
  47. 47. 1. Think about your objective. 2. Isolate. 3. Change one thing: the model. 4. Check your instrumentation. Measurement is hard
  48. 48. Product Top 3 Reasons AI Products Fail 1. Measuring stuff is hard. 2. Wrong Venue/Treatment/Approach.
  49. 49. The Taste Test
  50. 50. Choose an item you like
  51. 51. Items matching your taste
  52. 52. Venue Home Page Objective ↑ Registrations
  53. 53. Product Top 3 Reasons AI Products Fail 1. Measuring stuff is hard. 2. Wrong Venue/Treatment/Approach. 3. Models First. Users Last.
  54. 54. Models First Cool Model: word2vec! bluenavy
  55. 55. Models First Cool Model → Query Expansion! (navy or blue) totenavy tote
  56. 56. Search Results for “indian sea”
  57. 57. Precision Problems Garbage results Recall Problems Not enough results
  58. 58. f you wanted to make this one statement as well. Or another one. Search results for “wedding dress”
  59. 59. “1930s”“1920s”
  60. 60. UsersModel
  61. 61. Search results for “jewellery”
  62. 62. UsersModel
  63. 63. Everything Else Models What you see What you get
  64. 64. AI Product✨
  65. 65. Denoument
  66. 66. Denoument Things you should care about 1. Humans 2. Measurement 3. Training Data 4. Serving Inferences 5. Models
  67. 67. gio@relatedworks.io

×