Download to read offline
Uploaded byAvner Algom
How in memory technology will impact machine deep learning services (redis labs case study)
This document discusses how in-memory technology can impact machine and deep learning services using Redis Labs as a case study. It describes how Redis can provide a simple, extensible, and high performance platform for serving machine learning models. Serving complex models at scale is challenging due to their size, lack of standardization, and high costs. Redis-ML module allows predictive models to be stored and evaluated directly in Redis, reducing infrastructure needs by 97% for an ad serving use case compared to a homegrown solution. Co-locating streams, data, and machine learning engines in an in-memory database like Redis can reduce data movement, messages, and latency compared to traditional machine learning pipelines.
More Related Content
Similar to How in memory technology will impact machine deep learning services (redis labs case study)
How in memory technology will impact machine deep learning services (redis labs case study)
- 1. How in-memory technologywill impact Machine/Deep Learning services (Redis Labs case study) DECEMBER 2017
- 2. Who We Are Opensource. The leading in-memory database platform, supporting any high performance operational, analytics or hybrid use case. The open source home and commercial provider of Redis Enterprise (Redise) technology, platform, products & services. 2
- 3. Why Redis ? SimplicityExtensibilityPerformance NoSQL Benchmark 1 Redis Data Structures 2 3 Redis Modules 3 Lists Hashes Bitmaps Strings Bit field Streams Hyperloglog Sorted Sets Sets Geospatial Indexes
- 4. The New Standardfor Application Serving 4 100 msec - The new standard for E2E application response time, under any load 50 msec - Average roundtrip internet latency 50 msec - Required round trip app response time (includes processing & multi-DB access) 1 msec - required DB response time DatabaseApp Servers
- 5. The Machine /Deep Learning (ML/DL) World (1) Training (2) Creating a model (3) Serving the model
- 6. The Machine /Deep Learning (ML/DL) World (1) Training (2) Creating a model (3) Serving the model ?
- 7. Challenges in ServingML Models – Models are becoming biggers and complex – Not standardized – Can be very complex to deploy – Do not scale well, speed and size – Reliable services are hard to do – Can be very expensive
- 8. Accurate Model isImportant
- 9. Accurate Model isImportant Otherwise….
- 10. Random Forest Model •A collection of decision trees • Supports classification & regression • Splitter Node can be • Categorical (e.g. day == “Sunday”) • Numerical (e.g. age < 43) – Decision is taken by the majority of decision trees
- 11. Titanic Survival Predictoron a Decision Tree YES Sex = Male ? Age < 9.5? *Sibps > 2.5? Survived Died SurvivedDied *Sibps = siblings + spouses NO
- 12. Titanic Survival Predictoron a Random Forest YES Sex = Male ? Age < 9.5? *Sibps > 2.5? Survived Died SurvivedDied NO YES Country= US? State = CA? Height> 1.60m? Survived Died SurvivedDied NO YES Weight< 80kg? I.Q<100? Eye color =blue? Survived Died SurvivedDied NO Tree #1 Tree #2 Tree #3
- 13. Would John SurviveThe Titanic • John’s features: {male, 34, married + 2, US, CA, 1.78m, 78kg, 110iq, blue eyes} • Tree#1 – Survived • Tree#2 – Failed • Tree#3 – Survived • Random forest decision - Survived
- 14. Real World Challenge •Ads serving company • Need to serve 20,000 ads/sec @ 50msec data-center latency • Runs 1k campaigns 1K random forest • Each forest has 15K trees • On average each tree has 7 levels (depth)
- 15. Accurate Models areExpensive to Serve ! Item Calculation Total Random forest ops/sec 20K (ads/sec) x 1K (forests) x 15K(trees) x 7 x 0.5 (levels) 1.05 trillion Max ops/sec on the strongest AWS instance vcore 2.6Ghz x 0.9 (OS overhead) x 0.1 (10 lines of code per ops) x 0.1 (Java overhead) 23.4 million # of vcores needed 2.1 trillion / 23.4 million 44,872 # of c4.8xlarge instances needed 44,872 / 36 1,247 Total cost reserved instances 1,247 x 9213 ~$11.5M/yr
- 16. Redis-ML: a RedisModule for Serving ML models Supported Models Tree Ensembles Linear Regression Logistic Regression Matrix + Vector Operations K-Means Neural Networks
- 17. Redis-ML: Predictive ModelServing Engine • Predictive models as native Redis types • Perform evaluation directly in Redis • Store training output as “hot model” Spark Training Data loaded into Spark Model is saved in Redis-ML Redis-ML Serving Client Client App Client App Client App Any Training Platform
- 18. Ads Serving Use-Casew/ and w/o Redise + ML Homegrown 1,247 x c4.8xlarge 35 x c4.8xlarge Cut computing infrastructure by 97%
- 19. Ads Serving Use-Casew/ and w/o Redise + ML msec msec x2,000 Faster
- 20. Why Serving ML/DLat the Data Layer ?
- 21. Traditional ML Pipeline Streams Engine Acat or a dog? 1
- 22. Traditional ML Pipeline Streams Engine Acat or a dog? Ingestion 1 2 Pre-processing
- 23. Traditional ML Pipeline Streams Engine Acat or a dog? Ingestion Custom App 1 2 1 Pre-processing Notification
- 24. Traditional ML Pipeline Streams Engine Acat or a dog? Ingestion Custom App 1 2 1 DBMS 2 Pre-processing Notification
- 25. Traditional ML Pipeline Streams Engine Acat or a dog? Ingestion Custom App 1 2 1 DBMS 2 43 Data+Metadata ML/DL Engine Pre-processing Notification
- 26. Traditional ML Pipeline Streams Engine Acat or a dog? Ingestion Custom App 1 2 1 DBMS 2 4 Data+Metadata 3 ML/DL Engine Pre-processing Notification 3
- 27. Traditional ML Pipeline Streams Engine Acat or a dog? Ingestion Custom App 1 2 1 DBMS 2 4 Data+Metadata 3 ML/DL Engine 4x data-movement O(3)x messages Pre-processing Notification 3
- 28. Traditional ML Pipeline Streams Engine Acat or a dog? Ingestion Custom App 1 2 1 DBMS 2 4 Data+Metadata 3 ML/DL Engine > 100msec !!! Pre-processing Notification 3
- 29. What if Streams+MLruns on your in-memory database ? A cat or a dog? 1 Redis Enterprise +
- 30. What if Streams+MLruns on your in-memory database ? A cat or a dog? Custom App 1 1 Redis Enterprise + Notification
- 31. What if Streams+MLruns on your in-memory database ? A cat or a dog? Custom App 1 1 Redis Enterprise + Notification 2 One API call
- 32. What if Streams+MLruns on your in-memory database ? A cat or a dog? Custom App 1 1 Redis Enterprise + Notification 2 One API call 3
- 33. What if Streams+MLruns on your in-memory database ? A cat or a dog? Custom App 1 1 Redis Enterprise + Notification 2 One API call 3 1x data-movement 3x messages
- 34. What if Streams+MLruns on your in-memory database ? A cat or a dog? Custom App 1 1 Redis Enterprise + Notification 2 One API call 3 Single digit latency (<10msec)
- 35.