Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

AI for an intelligent cloud and intelligent edge: Discover, deploy, and manage with Azure ML services


Published on

Discover, manage, deploy, monitor – rinse and repeat.  In this session we show how Azure Machine Learning can be used to create the right AI model for your challenge and then easily customize it using your development tools while relying on Azure ML to optimize them to run in hardware accelerated environments for the cloud and the edge using FPGAs and Neural Network accelerators.  We then show you how to deploy the model to highly scalable web services and nimble edge applications that Azure can manage and monitor for you.  Finally, we illustrate how you can leverage the model telemetry to retrain and improve your content.

Published in: Technology
  • Unlock Her Legs - How to Turn a Girl On In 10 Minutes or Less... 
    Are you sure you want to  Yes  No
    Your message goes here
    Are you sure you want to  Yes  No
    Your message goes here
  • Earn a 6-Figure Side-Income Online... Signup for the free training HERE ♣♣♣
    Are you sure you want to  Yes  No
    Your message goes here
    Are you sure you want to  Yes  No
    Your message goes here
  • Real Ways To Make Money, Most online opportunities are nothing but total scams! ■■■
    Are you sure you want to  Yes  No
    Your message goes here

AI for an intelligent cloud and intelligent edge: Discover, deploy, and manage with Azure ML services

  1. 1. AI for Intelligent Cloud and Intelligent Edge: Discover, deploy, and manage with Azure ML Services James Serra Microsoft Technical Architect, Data & AI Blog:
  2. 2. About Me  Microsoft, Big Data Evangelist  In IT for 30 years, worked on many BI and DW projects  Worked as desktop/web/database developer, DBA, BI and DW architect and developer, MDM architect, PDW/APS developer  Been perm employee, contractor, consultant, business owner  Presenter at PASS Business Analytics Conference, PASS Summit, Enterprise Data World conference  Certifications: MCSE: Data Platform, Business Intelligence; MS: Architecting Microsoft Azure Solutions, Design and Implement Big Data Analytics Solutions, Design and Implement Cloud Data Platform Solutions  Blog at  Former SQL Server MVP  Author of book “Reporting with Microsoft SQL Server 2012”
  3. 3. I tried to understand AI products on my own… And felt like I was body slammed by Randy Savage: Let’s prevent that from happening…
  4. 4. Intro to Azure Machine Learning Model Management Hardware Acceleration Edge Integration AICloud+Edge
  5. 5. Machine learning is a data science technique that allows computers to use existing data to forecast future behaviors, outcomes, and trends.
  6. 6. Prepare Data Build & Train Deploy Custom AI Building your own AI models for Transforming Data into Intelligence
  7. 7. © Microsoft Corporation Advanced analytics pattern in Azure Azure Data Lake store Azure Storage HDInsightAzure Databricks Azure ML Services ML server Model training Long-term storage Data processing Azure Data Lake Analytics Azure ML Studio SQL Server (in-database ML) Azure Databricks (Spark ML) Data Science VM Cosmos DB Serving storage SQL DB SQL DW Azure Analysis Services Cosmos DB Batch AI SQL DB Azure Data Factory Orchestration Azure Container Service Trained model hosting SQL Server (in-database ML) Data collection and understanding, modeling, and deployment Sensors and IoT (unstructured) Logs, files, and media (unstructured) Business/custom apps (structured) Applications Dashboards Power BI
  8. 8. © Microsoft Corporation Power BI – build your own ML models
  9. 9. Azure AI Services Azure Infrastructure Tools
  10. 10. © Microsoft Corporation Machine learning and AI portfolio What engines do you want to use? Deployment target Which experience do you want? Build your own or consume pre-trained models? Microsoft ML & AI products Build your own Azure Machine Learning Code first (On-prem) ML Server On-prem Hadoop SQL Server (cloud) BYOT SQL Server Hadoop Azure Batch DSVM Spark Visual tooling (cloud) AML Studio Consume Cognitive services, bots Spark ML, SparkR, SparklyR Notebooks Jobs Azure Databricks Spark When to use what AI Decision tree:
  11. 11. Cognitive Services capabilities Infuse your apps, websites, and bots with human-like intelligence
  12. 12. Model Management Hardware Acceleration Edge Integration
  14. 14. WHAT IS AZURE MACHINE LEARNING SERVICE? Set of Azure Cloud Services Python SDK  Prepare Data  Build Models  Train Models  Manage Models  Track Experiments  Deploy Models That enables you to:
  15. 15. What is Azure Machine Learning service? Start training on your local machine and then scale out to the cloud
  16. 16. Azure Machine Learning Services • Deprecate Azure Machine Learning Workbench • Unified SDK, CLI and UX for training and deploying models • Full Integration with Visual Studio Code and Azure DevOps • Improved support for multiple compute targets • Four new models for FPGA • Vision AI Dev Kit available to order on Oct 1
  17. 17. Azure Machine Learning Workspace layout
  18. 18. The Azure ML Deployment Pipeline
  19. 19. Understanding the Edge: Heavy Edge vs Light Edge Cloud: Azure Heavy Edge Light Edge Descriptio n An Azure host that spans from CPU to GPU and FPGA VMs A server with slots to insert CPUs, GPUs, and FPGAs or a X64 or ARM system that needs to be plugged in to work A Sensor with a SOC (ARM CPU, NNA, MCU) and memory that can operate on batteries Example DSVM / ACI / AKS / Batch AI - DataBox Edge - HPE - Azure Stack - DataBox Edge - Industrial PC -Video Gateway -DVR -Mobile Phones -VAIDK -Mobile Phones -IP Cameras -Azure Sphere - Appliances What runs model CPU,GPU or FPGA CPU,GPU or FPGA CPU, GPU x64 CPU Multi-ARM CPU Hw accelerated NNA CPU/GPU MCU Why Edge? latency, less data sent, filter, aggregate, work offline
  20. 20. Data Science Virtual Machines
  21. 21. AZURE MACHINE LEARNING STUDIO Platform for emerging data scientists to graphically build and deploy experiments • Rapid experiment composition • > 100 easily configured modules for data prep, training, evaluation • Extensibility through R & Python • Serverless training and deployment Some numbers: • 100’s of thousands of deployed models serving billions of requests
  22. 22. Comparable Table Azure Machine Learning Studio Machine Learning Services Pros • Rapid development (Drag and Drop) • Works well with relatively simple datasets • Pre-built ML algorithms • Cheap • Fast (VMs with GPUs) • Different optimization methods, CI/CD pipeline • Full control during training • Manage computing resources (choose VM size) • Use open source ML libraries Cons • Can be slow • Limited optimization methods, operationalized architecture • Less control during training • Fixed computing resources • More elaborate to build, require deeper knowledge of machine learning • Deeper models need much more data with much more memory • Higher costs for VM with GPU
  23. 23.
  24. 24. Model Management and Deployment Demo UX, Azure Dev Ops and CLI
  25. 25. Model Management Hardware Acceleration Edge Integration
  26. 26. AI in Action
  27. 27. Breakthroughs in deep learning demand real-time AI Convolutional Neural Networks (CNN) ht-1 ht ht+1 xt-1 xt xt+1 ht-1 ht ht+1 yt-1 yt yt+1 Recurrent Neural Networks (RNN) Deep neural networks (DNN) have enabled major advances in machine learning and AI Computer vision Language translation Speech recognition Question answering And more… Problem DNNs are challenging to serve and deploy in large-scale online services Heavily constrained by latency, cost, and power Size and complexity outpacing growth of commodity CPUs
  29. 29. hardware architecture designed to accelerate real-time AI calculations Project Brainwave unique advantage (DNN models and FPGA) No batching required Brainwave delivers the ideal combination: High hardware utilization Low latency Low batch sizes In short, it is a hardware architecture and learning platform designed to accelerate real-time AI calculations Batch Size Performance Brainwave NPU 1256
  30. 30. Credit: Henk Monster. Licensed under the Creative Commons Attribution 3.0 Unported license.
  31. 31. Licensed under the Creative Commons Attribution 2.0 Generic license
  32. 32. Project Brainwave use cases For accelerated real-time image processing: - Identify manufacturing defects - Detect spills or open freezer doors in a retail store - Conduct real-time medical image analysis - Tract endangered species - Detect cars parked in a fire lane
  33. 33. The power of deep learning on FPGA Performance Flexibility Scale Rapidly adapt to evolving ML Inference-optimized numerical precision Exploit sparsity, deep compression Excellent inference at low batch sizes Ultra-low latency | 10x < CPU/GPU World’s largest cloud investment in FPGAs Multiple Exa-Ops of aggregate AI capacity Runs on Microsoft’s scale infrastructure Low cost $0.21/million images on Azure FPGA (inferencing)
  34. 34. Project BrainWave A Scalable FPGA-Powered DNN Serving Platform Fast: Flexible Friendly: F F F L0 L1 F F F L0 Pretrained DNN Model in TensorFlow, CNTK, etc. Scalable DNN Hardware Microservice BrainWave Soft DPU Instr Decoder & Control Neural FU Network switches FPGAs
  35. 35. Azure ML and Project Brainwave • New DNN models • ResNet 152, DenseNet-121, VGG-16, SSD-VGG • Customizable weights Easily deploy models to FPGAs for ultra-low latency with Azure Machine Learning powered by Project Brainwave
  36. 36. DAWNBench
  37. 37. Model Management Hardware Acceleration Edge Integration
  38. 38. Why Intelligent Edge? High-speed data processing, analytics and shorter response times are more essential than ever. Intelligent Cloud • Business agility and scalability: unlimited computing power available on demand. Intelligent Edge • Can handle priority-one tasks locally even without cloud connection. • Can handle generated data that is too large to pull rapidly from the cloud. • Enables real-time processing through intelligence in or near to local devices. • Flexibility to accommodate data privacy related requirements.
  39. 39. The components of a ML application Vision AI dev kit Vision AI dev kit
  40. 40. Building AI Solutions for the Intelligent Edge
  41. 41. Vision AI Developer Kit Hardware Specification Tutorial: Develop a C# IoT Edge module and deploy to your simulated device
  42. 42. Vision AI Developer Kit
  43. 43. Vision AI Developer Kit A connected camera reference solution Altek version available to order soon at
  44. 44. Q & A ? James Serra, Big Data Evangelist Email me at: Follow me at: @JamesSerra Link to me at: Visit my blog at: (where this slide deck is posted via the “Presentations” link on the top menu)