Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Velox: Models in Action
Similar to Velox: Models in Action (20)
Recently uploaded
Recently uploaded (20)
Velox: Models in Action
- 1. VELOX: MODELS IN ACTION Presented by Dan Crankshaw crankshaw@cs.berkeley.edu Henry Milner, Joseph Gonzalez, Peter Bailis, Haoyuan Li, Tomer Kaftan, Zhao Zhang, Ali Ghodsi, Michael Franklin, Michael Jordan, and Ion Stoica https://amplab.cs.berkeley.edu/projects/velox/
- 2. MODELS AT REST Data Well Studied Train Observe Predictions Model Predict
- 3. Data Training Feedback Open Challenges Predictions Model Serving
- 4. Velox Model Management System Data Training Feedback Open Challenges Predictions Model Serving
- 5. Catify: Music for Cats
- 6. Catify: Music for Cats Apache Web Server Node.js App Server MongoDB
- 7. MODELING TASK Rating Songs
- 8. MODELING TASK Ratings Songs Prediction
- 9. Data Training Feedback Predictions Model Serving
- 10. Catify: Music for Cats Tachyon + HDFS Pipeline CatID Song Score 1 16 2.1 1 14 3.7 3 273 4.2 4 14 1.9
- 11. Catify: Music for Cats Tachyon + HDFS Pipeline CatID Song Score 1 16 2.1 1 14 3.7 3 273 4.2 4 14 1.9
- 12. Catify: Music for Cats Tachyon + HDFS Pipeline CatID Song Score 1 16 2.1 1 14 3.7 3 273 4.2 4 14 1.9
- 13. Catify: Music for Cats Pipeline Tachyon + HDFS Apache Web Server Node.js App Server MongoDB
- 14. Data Training Feedback Predictions Model Serving
- 15. Catify: Music for Cats Pipeline Tachyon + HDFS Apache Web Server Node.js App Server MongoDB
- 16. Catify: Music for Cats Tachyon + HDFS NGINX Node.js App Server MongoDB Materialize all predictions Pipeline
- 17. Catify: Music for Cats Songs Users O(users + songs)
- 18. Songs Catify: Music for Cats Users O(users * songs)
- 19. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB
- 20. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB Training Data
- 21. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB Training Data New Model
- 22. What’s wrong?
- 23. What’s wrong? 1. Built from scratch for each application
- 24. What’s wrong? 1. Built from scratch for each application 2. Different systems
- 25. What’s wrong? 1. Built from scratch for each application 2. Different systems 3. Space inefficient
- 26. What’s wrong? 1. Built from scratch for each application 2. Different systems 3. Space inefficient 4. Stale predictions
- 27. What’s wrong? 1. Built from scratch for each application 2. Different systems 3. Space inefficient 4. Stale predictions 5. The T-Swift effect Sample Bias
- 28. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB Training Data New Model
- 29. Pipeline Tachyon + HDFS The Missing Piece Web Application Velox
- 30. Data Training Feedback Predictions Model Serving
- 31. Tachyon + HDFS The Missing Piece Velox Prediction Service Model Manager Web Application Pipeline
- 32. BENEFITS
- 33. BENEFITS 1. Low-latency and scalable predictions as a service
- 34. BENEFITS 1. Low-latency and scalable predictions as a service 2. Integrated approach leads to fresher, better predictions
- 35. BENEFITS 1. Low-latency and scalable predictions as a service 2. Integrated approach leads to fresher, better predictions 3. Easy translation to production predictions
- 36. BENEFITS 1. Low-latency and scalable predictions as a service 2. Integrated approach leads to fresher, better predictions 3. Easy translation to production predictions 4. Eases operational pain
- 39. PERSONALIZED MODELING Rating = wu · f(x; ✓)
- 40. PERSONALIZED MODELING Shared Basis Feature Models Rating = wu · f(x; ✓)
- 41. PERSONALIZED MODELING Shared Basis Feature Models Personalized User Model Rating = wu · f(x; ✓)
- 42. PERSONALIZED MODELING Change slowly Shared Basis Feature Models Personalized User Model wu · f(x; ✓) Rating =
- 43. PERSONALIZED MODELING Highly dynamic Change slowly Shared Basis Feature Models Personalized User Model wu · f(x; ✓) Rating =
- 44. Data Training Feedback Predictions Model Serving
- 45. VELOX Pipeline Tachyon + HDFS Velox Prediction Service Model Manager Web Application Predictions as a service
- 46. VELOX Pipeline Tachyon + HDFS Velox Prediction Service Model Manager Web Application Predictions as a service
- 47. PREDICTION API GET /velox/catify/predict?userid=22&song=27632 GET /velox/catify/predict_top_k?userid=22&k=100
- 48. PREDICTION API GET /velox/catify/predict?userid=22&song=27632 GET /velox/catify/predict_top_k?userid=22&k=100
- 49. PREDICTION API GET /velox/catify/predict?userid=22&song=27632 GET /velox/catify/predict_top_k?userid=22&k=100
- 50. PREDICTIONS def predict( u: UUID, x: Context ) wu · f(x; ✓)
- 51. PREDICTIONS def predict( u: UUID, x: Context ) Look up user weight wu · f(x; ✓)
- 52. PREDICTIONS def predict( u: UUID, x: Context ) Compute Features Look up user weight wu · f(x; ✓)
- 53. LOW-LATENCY PREDICTIONS Partition 0 Velox Tachyon Partition 1 Velox Tachyon Partition 2 Velox Tachyon Partition users
- 54. PREDICTIONS def predict( u: UUID, x: Context ) Compute Features Look up user weight wu · f(x; ✓)
- 55. LOW-LATENCY PREDICTIONS Velox Tachyon Feature Cache
- 56. LOW-LATENCY PREDICTIONS Velox Tachyon Features shared between users Feature Cache
- 57. Data Training Feedback Predictions Model Serving
- 58. Data Training Feedback Predictions Model Serving
- 59. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB
- 60. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB Training Data
- 61. SIMPLE EXPLORATION Rating Songs Prediction
- 62. SIMPLE EXPLORATION Rating Songs Epsilon-greedy Prediction
- 63. SIMPLE EXPLORATION Rating Songs Epsilon-greedy Prediction
- 64. ACTIVE LEARNING Rating Songs Prediction
- 65. ACTIVE LEARNING: LinUCB Rating Songs Uncertainty Prediction Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010). A contextual-bandit approach to personalized news article recommendation. WWW '10: Proceedings of the 19th international conference on World wide web, New York, New York, USA: ACM. doi:10.1145/1772690.1772758
- 66. ACTIVE LEARNING: LinUCB Rating Songs Look at upper confidence bound Uncertainty Prediction Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010). A contextual-bandit approach to personalized news article recommendation. WWW '10: Proceedings of the 19th international conference on World wide web, New York, New York, USA: ACM. doi:10.1145/1772690.1772758
- 67. ACTIVE LEARNING: LinUCB Rating Songs Look at upper confidence bound Uncertainty Prediction Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010). A contextual-bandit approach to personalized news article recommendation. WWW '10: Proceedings of the 19th international conference on World wide web, New York, New York, USA: ACM. doi:10.1145/1772690.1772758
- 68. Data Training Feedback Predictions Model Serving
- 69. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB Velox Prediction Service Model Manager
- 70. Data Training Feedback Mgmt. Predictions Model Serving
- 71. Realtime Learning Data Training Feedback Mgmt. Predictions Model Serving
- 72. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB Training Data New Model
- 73. USER-FACING API GET /velox/catify/predict?userid=22&song=27632 GET /velox/catify/predict_top_k?userid=22&k=100
- 74. USER-FACING API GET /velox/catify/predict?userid=22&song=27632 GET /velox/catify/predict_top_k?userid=22&k=100 POST /velox/catify/observe?userid=22&song=27632?score=3.7
- 75. ONLINE UPDATES def observe(u: UUID, x: Context, y: Score) wu · f(x; ✓)
- 76. ONLINE UPDATES def observe(u: UUID, x: Context, y: Score) Update wu with new training point wu · f(x; ✓)
- 77. ONLINE UPDATES def observe(u: UUID, x: Context, y: Score) Basis functions stay fixed Update wu with new training point wu · f(x; ✓)
- 78. Realtime Learning Data Training Feedback Mgmt. Predictions Model Serving
- 79. Realtime Learning + Offline Retraining Data Training Feedback Mgmt. Predictions Model Serving
- 80. Catify: Music for Cats Pipeline Tachyon + HDFS NGINX Node.js App Server MongoDB Velox Prediction Service Model Manager
- 81. Velox Model Management System Data Feedback Spark Predictions Model Serving
- 82. The future of research in scalable learning systems will be in the integration of the learning lifecycle: Data Training Feedback Predictions Model Serving
- 83. SUMMARY
- 84. SUMMARY •Model training and predictions rely on ad-hoc, manual processes spread across multiple systems
- 85. SUMMARY •Model training and predictions rely on ad-hoc, manual processes spread across multiple systems •The Velox system automatically maintains multiple models while providing low latency, scalable, and personalized predictions
- 86. SUMMARY •Model training and predictions rely on ad-hoc, manual processes spread across multiple systems •The Velox system automatically maintains multiple models while providing low latency, scalable, and personalized predictions •Velox is part of BDAS, is coming soon…
- 87. SUMMARY •Model training and predictions rely on ad-hoc, manual processes spread across multiple systems •The Velox system automatically maintains multiple models while providing low latency, scalable, and personalized predictions •Velox is part of BDAS, is coming soon… •https://amplab.cs.berkeley.edu/projects/velox/
- 88. BACKUP MATERIAL
- 89. RETRAIN OFFLINE def retrainOffline(sc: SparkContext, trainingData: RDD) wu · f(x; ✓)
- 90. RETRAIN OFFLINE def retrainOffline(sc: SparkContext, trainingData: RDD) Retrain feature functions Use Spark for batch retrain wu · f(x; ✓)