Download as PDF, PPTX
More Related Content
![The Evolution of Metadata: LinkedIn's Story [Strata NYC 2019]](https://cdn.slidesharecdn.com/ss_thumbnails/metadatajourneylinkedinstratapublic-190930045839-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Making Data Scientists Productive in Azure
![[第35回 Machine Learning 15minutes!] Microsoft AI Updates](https://cdn.slidesharecdn.com/ss_thumbnails/20190427ml15minmicrosoftai-190427060706-thumbnail.jpg?width=640&height=640&fit=bounds)
Making Data Scientists Productive in Azure
- 1. MAKING DATA SCIENTISTS PRODUCTIVEIN AZURE Valdas Maksimavičius
- 2. Quiz Microsoft Machine LearningServer Machine Learning for .NET Azure Machine Learning Service Azure Machine Learning Studio Azure Databricks Data Science Virtual Machine SQL Server Machine Learning Services Azure Cognitive Services
- 3. Inspiration for thetalk One thing about Microsoft - they have multiple ways to solve the same problem
- 6. So what doyou mean by saying “Making Data Scientists Productive in Azure”?
- 10. 6 Data Sciencestories
- 11. Tom • Full stack softwaredeveloper • .Net, Node.js, Vue, React Scan faces to decide what advert to serve
- 12. Azure Cognitive Services Whatis it? Azure services with pre-built AI and ML models What can you do with it? Add intelligent features to your apps
- 13. Azure Cognitive Services- Overview • Vision (e.g. face / scene / object recognition, video analysis) • Speech (e.g. speaker recognition, speech-to-text) • Language (e.g. translations, phrase extraction, QnA maker) • Decision (e.g. content moderation, anomaly detection) • Search (e.g. Bing search)
- 16.
- 17. Azure Cognitive Services- Summary Key benefits: • Minimal development effort • Easy integration via HTTP REST • Built-in support with other Azure services • Containers support
- 18. Azure Cognitive Services- Summary Key benefits: • Minimal development effort • Easy integration via HTTP REST • Built-in support with other Azure services • Containers support Considerations: • Limited customization allowed • Limited support for Non-English languages
- 19. ML.NET What is it? Anopen source and cross-platform ML framework What can you do with it? Create custom ML models using C# or F# without leaving the .NET ecosystem
- 22. ML.NET - Summary Keybenefits: • Powers products like Microsoft Defender, Outlook, Bing, PowerBI • Seamlessly integrates ML into .NET apps • AutoML functionality • Leverage TensorFlow or ONNX Considerations: • Limited support for other ML libraries
- 23. Mark • Business Analyst • Basics ofstatistical analysis Create a sales lead list
- 24. Azure Machine LearningStudio What is it? Drag-and-drop visual interface for ML What can you do with it? Build, experiment, and deploy models using pre-configured algorithms
- 28. Deploy as web services • Batchexecution • Request / Response
- 30. Azure Machine LearningStudio - Summary Key benefits: • Interactive visual interface • Built-in Jupyter Notebooks for data exploration • Direct deployment of trained models as web services • Built-in support for other Azure services
- 31. Azure Machine LearningStudio - Summary Key benefits: • Interactive visual interface • Built-in Jupyter Notebooks for data exploration • Direct deployment of trained models as web services • Built-in support for other Azure services Considerations: • Limited scalability (the maximum size of a training dataset is 10 GB) • Online only • Limited support for custom Python/R code
- 32. Lucy • Machine Learning Engineer • Python,Scikit-learn, Keras, TensorFlow Estimate damage (repair cost) in auto insurance
- 34. Azure Machine LearningStudio Service What is it? Managed cloud service for ML What can you do with it? Train, deploy and manage models in Azure using Python and CLI
- 35. Azure Machine LearningService - Overview • Python SDK • Data preparation • Compute targets • Experiment tracking • Deployment targets
- 36. Azure Machine LearningService - Compute Targets
- 37. Azure Machine LearningService - Compute Targets • Your local computer
- 38. Azure Machine LearningService - Compute Targets • Your local computer • Linux VM in Azure • Azure Batch AI Cluster • Azure Databricks • Azure Container Instance • Apache Spark for HDInsight
- 39. Azure Machine LearningService - Compute Targets
- 41. Azure Machine LearningService - Experiment Tracking
- 42. Azure Machine LearningService - Experiment Tracking
- 43. Azure Machine LearningService - Deployment Targets
- 45. Azure Machine LearningService - Summary Key benefits: • Central management of scripts and run history • Run model training scripts locally, and then scale out to the cloud • Deployment and management of models to the cloud or edge devices • Start development locally (offline)
- 46. Azure Machine LearningService - Summary Key benefits: • Central management of scripts and run history • Run model training scripts locally, and then scale out to the cloud • Deployment and management of models to the cloud or edge devices • Start development locally (offline) Considerations: • Python only • Requires some familiarity with the model management model
- 47. Bradley • Data Scientist /Engineer • Apache Spark / SQL / Python / Scala • Wants to spend more time outdoors than exploring beta version tools Build an enterprise data lake and data science environment
- 48. Azure Databricks What isit? Spark-based analytics platform What can you do with it? Build and deploy models and data workflows
- 49. Azure Databricks -Overview Collaborative Workspace • Notebooks • User access • Git integration
- 51. Azure Databricks -Overview Collaborative Workspace • Notebooks • User access • Git integration Databricks Runtime • Apache Spark • Rest APIs • Libraries
- 53. Azure Databricks -Overview Collaborative Workspace • Notebooks • User access • Git integration Deploy Jobs & Workflows • Job scheduler • Notifications & logs • Multi-stage pipelines Databricks Runtime • Apache Spark • Rest APIs • Libraries
- 54. Azure Databricks -Overview Collaborative Workspace • Notebooks • User access • Git integration Deploy Jobs & Workflows • Job scheduler • Notifications & logs • Multi-stage pipelines Databricks Runtime • Apache Spark • Rest APIs • Libraries Security • Single sign-on (SSO) • Access control list (ACL) • Secrets via Key Vault
- 56. Azure Databricks -Summary Key benefits: • Probably the most mature development environment for ML on the Azure platform • Nicely integrated with other Azure services
- 57. Azure Databricks -Summary Key benefits: • Probably the most mature development environment for ML on the Azure platform • Nicely integrated with other Azure services Considerations: • Online only
- 58. Joshua • Data Scientist • Research anddevelopment “I need a sandbox to learn and evaluate new tools”
- 59. Data Science VirtualMachine What is it? A virtual machine with pre-installed data science tools What can you do with it? Develop ML solutions in a pre-configured environment
- 61. Azure Data ScienceVirtual Machine - Summary Key benefits: • Probably the most complete development environment for ML on the Azure platform • Reduced time to install, manage, and troubleshoot data science tools and frameworks • Virtual machine options include highly scalable GPU images • A dedicated geospatial with ArcGIS distribution Considerations: • Online only • You need to take care of VM management
- 62. I asked forCloud and they said NO
- 63. Rick • Specializes in R • Notallowed to push data to Azure Create personalized treatment based on individual health data
- 64. Microsoft Machine LearningService Server What is it? Cross-platform standalone server for predictive analysis What can you do with it? Build and deploy models with R and Python
- 65. Microsoft Machine LearningServer - Overview • A new name for Microsoft R Server • Install on Windows / Linux / Hadoop cluster • Deploy models as web services packaged as container images • Satisfy security and compliance needs of any enterprise
- 67. Microsoft Machine LearningServer - Summary Key benefits: • Built on a legacy of Microsoft R Server and Revolution R Enterprise • Advanced security options • Deploy R and Python models as web services
- 68. Microsoft Machine LearningServer - Summary Key benefits: • Built on a legacy of Microsoft R Server and Revolution R Enterprise • Advanced security options • Deploy R and Python models as web services Considerations: • You need to deploy and manage Machine Learning Server in your enterprise
- 69. SQL Server MachineLearning Services What is it? A built-in SQL Server feature to support machine learning What can you do with it? Execute Python and R scripts with relational data
- 71. SQL Server MachineLearning Services - Summary Key benefits: • Run your scripts where the data resides and eliminate transfer of the data across the network to another server • Encapsulate predictive logic in a database function
- 72. SQL Server MachineLearning Services - Summary Key benefits: • Run your scripts where the data resides and eliminate transfer of the data across the network to another server • Encapsulate predictive logic in a database function Considerations: • Assumes a SQL Server database as the data tier for your application • Limited scalability • Long list of known issues
- 73. Quiz Azure Cognitive Services MachineLearning for .NET Azure Machine Learning Studio Azure Machine Learning Service Azure Databricks Data Science Virtual Machine Microsoft Machine Learning Server SQL Server Machine Learning Services
- 74.
- 75. Azure Machine LearningService - Deployment Targets
- 76. Azure Machine LearningService - Deployment Targets Native support: • Azure Container Instance • Azure Kubernetes Service • Azure IoT Edge Requires rework: • Linux VMs • Other cloud providers
- 77. Azure Machine LearningService - Deployment Targets Native support: • Azure Container Instance • Azure Kubernetes Service • Azure IoT Edge
- 80. Azure Machine LearningService - Overview • Python SDK • Data preparation • Compute targets • Experiment tracking • Deployment targets
- 81. Welcome to Vilnius Microsoft Data Platform Meetup Lookingfor enthusiasts to share their stories
- 82. Me in 2015 “Withjust a few clicks, you can have a Hadoop cluster up and running”
- 83.
- 84. Azure Cognitive Services- Similar services
- 85.
- 89.
- 90. Welcome to Vilnius DataPlatform Meetup • Making Data Scientists Productive in Azure by Valdas Maksimavičius • Building Churn Prediction Model Using Azure Databricks, Sklearn and MLflow by Tomas Lukas Komar • Snacks & Networking Agenda: