Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Velox: Models in Action

4,254 views

Published on

Presentation of Velox project at AMPCamp 5 (http://ampcamp.berkeley.edu/5/). More info about Velox at https://amplab.cs.berkeley.edu/projects/velox/.

Published in: Software

Velox: Models in Action

  1. 1. VELOX: MODELS IN ACTION Presented by Dan Crankshaw crankshaw@cs.berkeley.edu Henry Milner, Joseph Gonzalez, Peter Bailis, Haoyuan Li, Tomer Kaftan, Zhao Zhang, Ali Ghodsi, Michael Franklin, Michael Jordan, and Ion Stoica https://amplab.cs.berkeley.edu/projects/velox/
  2. 2. MODELS AT REST Data Well Studied Train Observe Predictions Model Predict
  3. 3. Data Training Feedback Open Challenges Predictions Model Serving
  4. 4. Velox Model Management System Data Training Feedback Open Challenges Predictions Model Serving
  5. 5. Catify: Music for Cats
  6. 6. Catify: Music for Cats Apache Web Server Node.js App Server MongoDB
  7. 7. MODELING TASK Rating Songs
  8. 8. MODELING TASK Ratings Songs Prediction
  9. 9. Data Training Feedback Predictions Model Serving
  10. 10. Catify: Music for Cats Tachyon + HDFS Pipeline CatID Song Score 1 16 2.1 1 14 3.7 3 273 4.2 4 14 1.9
  11. 11. Catify: Music for Cats Tachyon + HDFS Pipeline CatID Song Score 1 16 2.1 1 14 3.7 3 273 4.2 4 14 1.9
  12. 12. Catify: Music for Cats Tachyon + HDFS Pipeline CatID Song Score 1 16 2.1 1 14 3.7 3 273 4.2 4 14 1.9
  13. 13. Catify: Music for Cats Pipeline Tachyon + HDFS Apache Web Server Node.js App Server MongoDB
  14. 14. Data Training Feedback Predictions Model Serving
  15. 15. Catify: Music for Cats Pipeline Tachyon + HDFS Apache Web Server Node.js App Server MongoDB
  16. 16. Catify: Music for Cats Tachyon + HDFS NGINX Node.js App Server MongoDB Materialize all predictions Pipeline
  17. 17. Catify: Music for Cats Songs Users O(users + songs)
  18. 18. Songs Catify: Music for Cats Users O(users * songs)
  19. 19. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB
  20. 20. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB Training Data
  21. 21. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB Training Data New Model
  22. 22. What’s wrong?
  23. 23. What’s wrong? 1. Built from scratch for each application
  24. 24. What’s wrong? 1. Built from scratch for each application 2. Different systems
  25. 25. What’s wrong? 1. Built from scratch for each application 2. Different systems 3. Space inefficient
  26. 26. What’s wrong? 1. Built from scratch for each application 2. Different systems 3. Space inefficient 4. Stale predictions
  27. 27. What’s wrong? 1. Built from scratch for each application 2. Different systems 3. Space inefficient 4. Stale predictions 5. The T-Swift effect Sample Bias
  28. 28. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB Training Data New Model
  29. 29. Pipeline Tachyon + HDFS The Missing Piece Web Application Velox
  30. 30. Data Training Feedback Predictions Model Serving
  31. 31. Tachyon + HDFS The Missing Piece Velox Prediction Service Model Manager Web Application Pipeline
  32. 32. BENEFITS
  33. 33. BENEFITS 1. Low-latency and scalable predictions as a service
  34. 34. BENEFITS 1. Low-latency and scalable predictions as a service 2. Integrated approach leads to fresher, better predictions
  35. 35. BENEFITS 1. Low-latency and scalable predictions as a service 2. Integrated approach leads to fresher, better predictions 3. Easy translation to production predictions
  36. 36. BENEFITS 1. Low-latency and scalable predictions as a service 2. Integrated approach leads to fresher, better predictions 3. Easy translation to production predictions 4. Eases operational pain
  37. 37. PERSONALIZED MODELING
  38. 38. PERSONALIZED MODELING
  39. 39. PERSONALIZED MODELING Rating = wu · f(x; ✓)
  40. 40. PERSONALIZED MODELING Shared Basis Feature Models Rating = wu · f(x; ✓)
  41. 41. PERSONALIZED MODELING Shared Basis Feature Models Personalized User Model Rating = wu · f(x; ✓)
  42. 42. PERSONALIZED MODELING Change slowly Shared Basis Feature Models Personalized User Model wu · f(x; ✓) Rating =
  43. 43. PERSONALIZED MODELING Highly dynamic Change slowly Shared Basis Feature Models Personalized User Model wu · f(x; ✓) Rating =
  44. 44. Data Training Feedback Predictions Model Serving
  45. 45. VELOX Pipeline Tachyon + HDFS Velox Prediction Service Model Manager Web Application Predictions as a service
  46. 46. VELOX Pipeline Tachyon + HDFS Velox Prediction Service Model Manager Web Application Predictions as a service
  47. 47. PREDICTION API GET /velox/catify/predict?userid=22&song=27632 GET /velox/catify/predict_top_k?userid=22&k=100
  48. 48. PREDICTION API GET /velox/catify/predict?userid=22&song=27632 GET /velox/catify/predict_top_k?userid=22&k=100
  49. 49. PREDICTION API GET /velox/catify/predict?userid=22&song=27632 GET /velox/catify/predict_top_k?userid=22&k=100
  50. 50. PREDICTIONS def predict( u: UUID, x: Context ) wu · f(x; ✓)
  51. 51. PREDICTIONS def predict( u: UUID, x: Context ) Look up user weight wu · f(x; ✓)
  52. 52. PREDICTIONS def predict( u: UUID, x: Context ) Compute Features Look up user weight wu · f(x; ✓)
  53. 53. LOW-LATENCY PREDICTIONS Partition 0 Velox Tachyon Partition 1 Velox Tachyon Partition 2 Velox Tachyon Partition users
  54. 54. PREDICTIONS def predict( u: UUID, x: Context ) Compute Features Look up user weight wu · f(x; ✓)
  55. 55. LOW-LATENCY PREDICTIONS Velox Tachyon Feature Cache
  56. 56. LOW-LATENCY PREDICTIONS Velox Tachyon Features shared between users Feature Cache
  57. 57. Data Training Feedback Predictions Model Serving
  58. 58. Data Training Feedback Predictions Model Serving
  59. 59. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB
  60. 60. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB Training Data
  61. 61. SIMPLE EXPLORATION Rating Songs Prediction
  62. 62. SIMPLE EXPLORATION Rating Songs Epsilon-greedy Prediction
  63. 63. SIMPLE EXPLORATION Rating Songs Epsilon-greedy Prediction
  64. 64. ACTIVE LEARNING Rating Songs Prediction
  65. 65. ACTIVE LEARNING: LinUCB Rating Songs Uncertainty Prediction Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010). A contextual-bandit approach to personalized news article recommendation. WWW '10: Proceedings of the 19th international conference on World wide web, New York, New York, USA: ACM. doi:10.1145/1772690.1772758
  66. 66. ACTIVE LEARNING: LinUCB Rating Songs Look at upper confidence bound Uncertainty Prediction Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010). A contextual-bandit approach to personalized news article recommendation. WWW '10: Proceedings of the 19th international conference on World wide web, New York, New York, USA: ACM. doi:10.1145/1772690.1772758
  67. 67. ACTIVE LEARNING: LinUCB Rating Songs Look at upper confidence bound Uncertainty Prediction Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010). A contextual-bandit approach to personalized news article recommendation. WWW '10: Proceedings of the 19th international conference on World wide web, New York, New York, USA: ACM. doi:10.1145/1772690.1772758
  68. 68. Data Training Feedback Predictions Model Serving
  69. 69. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB Velox Prediction Service Model Manager
  70. 70. Data Training Feedback Mgmt. Predictions Model Serving
  71. 71. Realtime Learning Data Training Feedback Mgmt. Predictions Model Serving
  72. 72. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB Training Data New Model
  73. 73. USER-FACING API GET /velox/catify/predict?userid=22&song=27632 GET /velox/catify/predict_top_k?userid=22&k=100
  74. 74. USER-FACING API GET /velox/catify/predict?userid=22&song=27632 GET /velox/catify/predict_top_k?userid=22&k=100 POST /velox/catify/observe?userid=22&song=27632?score=3.7
  75. 75. ONLINE UPDATES def observe(u: UUID, x: Context, y: Score) wu · f(x; ✓)
  76. 76. ONLINE UPDATES def observe(u: UUID, x: Context, y: Score) Update wu with new training point wu · f(x; ✓)
  77. 77. ONLINE UPDATES def observe(u: UUID, x: Context, y: Score) Basis functions stay fixed Update wu with new training point wu · f(x; ✓)
  78. 78. Realtime Learning Data Training Feedback Mgmt. Predictions Model Serving
  79. 79. Realtime Learning + Offline Retraining Data Training Feedback Mgmt. Predictions Model Serving
  80. 80. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB Velox Prediction Service Model Manager
  81. 81. Velox Model Management System Data Feedback Spark Predictions Model Serving
  82. 82. The future of research in scalable learning systems will be in the integration of the learning lifecycle: Data Training Feedback Predictions Model Serving
  83. 83. SUMMARY
  84. 84. SUMMARY •Model training and predictions rely on ad-hoc, manual processes spread across multiple systems
  85. 85. SUMMARY •Model training and predictions rely on ad-hoc, manual processes spread across multiple systems •The Velox system automatically maintains multiple models while providing low latency, scalable, and personalized predictions
  86. 86. SUMMARY •Model training and predictions rely on ad-hoc, manual processes spread across multiple systems •The Velox system automatically maintains multiple models while providing low latency, scalable, and personalized predictions •Velox is part of BDAS, is coming soon…
  87. 87. SUMMARY •Model training and predictions rely on ad-hoc, manual processes spread across multiple systems •The Velox system automatically maintains multiple models while providing low latency, scalable, and personalized predictions •Velox is part of BDAS, is coming soon… •https://amplab.cs.berkeley.edu/projects/velox/
  88. 88. BACKUP MATERIAL
  89. 89. RETRAIN OFFLINE def retrainOffline(sc: SparkContext, trainingData: RDD) wu · f(x; ✓)
  90. 90. RETRAIN OFFLINE def retrainOffline(sc: SparkContext, trainingData: RDD) Retrain feature functions Use Spark for batch retrain wu · f(x; ✓)

×