Download to read offline
Uploaded byDaniel Takabayashi, MSc
Marvin Platform – Potencializando equipes de Machine Learning
The document discusses the development of Marvin, an artificial intelligence platform designed to simplify the creation of AI applications for data science teams at B2W Digital. It outlines the importance of a data-driven culture, the architecture of Marvin, and features that enable tasks such as model training and deployment. Additionally, it showcases a case study on a risk analysis model highlighting the benefits Marvin brings to project efficiency and collaboration.
More Related Content
Similar to Marvin Platform – Potencializando equipes de Machine Learning
Marvin Platform – Potencializando equipes de Machine Learning
- 1. Daniel Takabayashi Lucas Bonatto Fromexploratory models to production BIG DATA WEEK SÃO PAULO 2017
- 2.
- 3.
- 4. B2W Digital: e-commerceleader in LatAm Source: 2016 Results from ri.b2w.digital Total GMV (R$) 12,458 MM Market share (%) 26,2% The Digital Platform that connects People, Businesses,Products and Services.
- 5. Outline • Context • Data-drivenculture • Artificial Intelligence • Domains of knowledge • Problem Statement • Marvin • Main components • Architecture • DASFE pattern • General features • Case • Roadmap
- 7. Context: data-driven culture Singlesource of truth Data dictionary Broad data access Data literacy Decision making Why is it important to be data-driven?
- 8. Context: data-driven culture Optimizedecision-making
- 9. Context: Artificial Intelligence MachineLearning NLP Computer Vision • Buy Box optimization • Forecast demand • Fraud detection • Adspend optimization • Feature extraction from product description • Product category classification • Image matching to find associated products
- 10. Context: domains ofknowledge
- 11. Context: model lifecycle ModelsManagement Support & Feedback Model ServingExploration & Development
- 13. Problem statement How canwe abstract the complexity in the creation of an AI application? Building AI projects is not a simple task. One is required to have advanced knowledge in different domains.
- 14.
- 15. Marvin Artificial IntelligencePlatform Empowers data science teams to deliver AI applications, simplifying the process of exploitation and modeling.
- 16. Marvin: main components ENGINEEXECUTOR ENGINE Data acquisitor Prediction preparator Predictor Trainingpreparator Feedback Trainer TOOLBOX Evaluator
- 17.
- 18.
- 19. Marvin: DASFE pattern BatchData Acquisition & Cleaning Training Preparation Model Training Model Evaluation
- 20. Marvin: DASFE pattern OnlinePrediction Preparation Model Prediction Batch Data Acquisition & Cleaning Training Preparation Model Training Model Evaluation
- 21. Marvin: DASFE pattern OnlinePrediction Feedback Online Prediction Preparation Model Prediction Batch Data Acquisition & Cleaning Training Preparation Model Training Model Evaluation
- 22.
- 23. Marvin: general features •Training pipeline REST interface • Experiment and artifacts versioning • Engine project scaffold generator • Data sampling and import CLI • Engine test framework (unit, functional, dryrun) • Toolbox: Python support • Artifacts persistence layer: HDFS support • Remote provisioning and deployment
- 25. Case: risk analysismodel • XGBoost in python • Dataset: 1,0 M of orders • Training pipeline: 15 min • REST HTTP predictions: 15 ms • Load test: 100 rps w/ 15 ms mrt
- 26. Case: how marvinhelped the team? “... Jupyter notebook integration with Spark through Marvin’s toolbox lib was very helpful during prototyping phase...” “... the data importation utility speeds up data collection and sampling... ” “... it was easy to do feature engineering, feature selection and model choice using the DASFE model... ” “... we automated the training and deployment phase without having a dev/ops in our team...”
- 28. Marvin: roadmap • Adminmodule • Toolbox: Java and Scala support • Feedback server • Artifacts persistence layer: S3 and local FS support • Remote provisioning and deployment: Azure, AWS and GCP • Customized notebook kernel • Automate feature engineering • Hyper parameters support • ML for no-data scientists • …
- 29. Artificial Intelligence Platform Forkme on GitHub.com/marvin-ai and feel free to contribute! Thank you! @ GitHub.com/marvin-ai twitter.com/_marvin_ai marvin-ai@googlegroups.com