Download to read offline
Uploaded byAlexis Perrier
AWS Machine Learning Big Data NYC
The document discusses AWS Machine Learning, highlighting its predictive analytics capabilities and limitations, including the focus on supervised learning and the absence of deep learning support. It also covers the AWS ML project workflow, data transformation techniques, and potential improvements such as the lack of cross-validation and limited feature engineering. Overall, it emphasizes the need for domain expertise while noting the time-saving benefits of using AWS for machine learning tasks.
![[판교에서 만나는 아마존웹서비스] Obama for America를 통해서 본 AWS에서의 데이터 분석](https://cdn.slidesharecdn.com/ss_thumbnails/ofa-and-redshift-pangyo-131203232750-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
AWS Machine Learning Big Data NYC
- 2.
- 3. ▸ AWS MachineLearning : predictive analytics ▸ Simple, efficient but somewhat limited ▸ Auto-ML on AWS marketplace AWS MACHINE LEARNING PLAN
- 4. DATA SCIENCE TAKETIME AND RESOURCES
- 5. 10 FALLACIES OFDATA SCIENCE Shane Brennan https://medium.com/towards-data-science/the-ten-fallacies-of-data-science-9b2af78a1862 1. Exists 2. Accessible 3. Consistent 4. Relevant 5. Understandable 6. Processable 7. Reproducibility 8. Compliance and security 9. Results are understood 10.Expected outcomes Data scientist ROI Access Massage Train Models ProductionPresentation The plan
- 6. “AWS WANTS TOPUT MACHINE LEARNING IN REACH OF ANY DEVELOPER“ April 2015 - Techcrunch MACHINE LEARNING AS A SERVICE
- 7.
- 8. AWS MACHINE LEARNING WHATIT DOES ▸ Supervised Predictive Analytics ▸ On structured data and text, ▸ Outcome as function of variables, ground truth known on subset WHAT IT DOES NOT ▸ Unsupervised learning ▸ Reinforcement learning ▸ Deep learning
- 9. WORK FLOW -AWS ML PROJECT Create a datasource from S3, RDS or Redshift Transform the data with recipes (opt) Train a Model Evaluate Create endpoints 3 1 2 4 5
- 10. DATASOURCE AWS extracts theschema AWS analyses the data Provides simple visualization Offers default transformation S3 Redshift RDS (CLI - SDK only) 3 1 2 4
- 11. TITANIC DATASET -DEFAULT SCHEMA
- 12. TITANIC DATASET -SIMPLE ANALYSIS
- 13. SCHEMA, RECIPES ANDFEATURES ▸ From the data, AWS suggests the optimal transformations - recipe ▸ 7 transformations are available ▸ Text: N-gram, Orthogonal Sparse Bigram, Lowercase, Punctuation ▸ TF-IDF by default, no stop words, no POS, Lemma, … ▸ Categorical: Cartesian product ▸ Numeric: Normalization, Quantile Binning ▸ QB: non linearities in continuous, numeric to categorical ▸ Recipe is downloadable
- 14. TITANIC DATASET -DATA TRANSFORMATION
- 15. TITANIC DATASET -DATA TRANSFORMATION
- 16. TRAIN YOUR MODEL STOCHASTICGRADIENT DESCENT One model to rule them all ▸ Simple ▸ Regularization ▸ Epochs ▸ Shuffling ▸ That’s it!
- 17. STOCHASTIC GRADIENT DESCENT- TUNING Epochs Shuffling Regularization
- 18. Stochastic Gradient inscikit-learn
- 19. CONVERGENCE - POWERFULQUANTILE BINNING Accuracy Accuracy
- 20.
- 21.
- 22. TEXT END POINT -STREAMING
- 23.
- 25. STRONG POINTS ▸ Powerfulmodeling: SGD + quantile binning ▸ AWS ecosystem ▸ Multiple sources (S3, RDS, Redshift) ▸ Simple to setup and use ▸ Great for benchmarking ▸ No need for production code! ▸ CLI - SDKs (python, …)
- 26. ROOM FOR IMPROVEMENTS ‣No cross validation! ‣ Can’t export your trained models* ‣ No scripting (*) Stealing Machine Learning Models via prediction APIs http://www.cs.unc.edu/~reiter/papers/2016/USENIX.pdf ▸ Limited data visualization ▸ Limited feature engineering ▸ SGD model only: no forests, SVMs, Bayes, … ▸ No deep learning (EC2)
- 27. STILL A NEEDFOR DOMAIN EXPERTISE AND FEATURE ENGINEERING GREAT TIME SAVER BUT
- 28.
- 29. Modeling is simplified Butfeature engineering still needs attention 3 1 2 Feature engineering Include all available datasets Feature importance
- 30. ‣ Feature importance ‣Composite features surfacing ‣ Complementary dataset integration AUTOMATIC FEATURE ENGINEERING Smart (Not naive) Bayesian Optimization
- 31. ON AWS MARKETPLACE EC2INSTANCE
- 32. DATA EXPLORATION -FEATURE SELECTION
- 33.
- 34.
- 35. + ‣ Reduced TTM+ TCO ‣ Less resources ‣ Powerful benchmarking ‣ Fast iterations ‣ Holistic data integration
- 36. Jerry Hargrove @awsgeekhttps://www.awsgeek.com/posts/amazon-machine-learning-summary
- 37.