Uploaded byDavid Chou
Designing Artificial Intelligence
Researchers used deep learning techniques like ResNet and data augmentation to improve the accuracy of detecting snow leopards from 63.4% to 90%. They used transfer learning on a ResNet model to extract features from images, then trained a logistic regression classifier on those features to detect snow leopards. They also averaged predictions from multiple images and doubled their training data by flipping images horizontally. This helped improve the model's ability to identify snow leopards in photos.
More Related Content
Similar to Designing Artificial Intelligence
![[第35回 Machine Learning 15minutes!] Microsoft AI Updates](https://cdn.slidesharecdn.com/ss_thumbnails/20190427ml15minmicrosoftai-190427060706-thumbnail.jpg?width=640&height=640&fit=bounds)
![[Connect(); // Japan 2016] Microsoft の AI 開発最新アップデート ～ Cognitive Services からA...](https://cdn.slidesharecdn.com/ss_thumbnails/20161121connectjapanmicrosoftai-161121075407-thumbnail.jpg?width=640&height=640&fit=bounds)
![[第34回 WBA若手の会勉強会] Microsoft AI platform](https://cdn.slidesharecdn.com/ss_thumbnails/20180411wbamicrosoftai-180411111025-thumbnail.jpg?width=640&height=640&fit=bounds)
Designing Artificial Intelligence
- 2.
- 4. Saving Snow Leopardswith Deep Learning Transfer learning using ResNet Output features used to train a “traditional” SparkML logistic regression classifier to specifically detect snow leopards “Ensembling” – averaging the predictions of the model across multiple images Dataset augmentation - flipping images horizontally to double training data Accuracy improved from 63.4% using a basic model, to 90% blogs.technet.microsoft.com/machinelearning/2017/06/27/saving-snow-leopards-with-deep-learning-and-computer-vision-on-spark/
- 5. Artificial Intelligence A setof technologies that enable machine intelligence to extend and amplify elements of human thinking deep learning machine learning neural networks search probabilistic reasoning etc. vision speech language knowledge problem solving etc.
- 6. Artificial Intelligence, MachineLearning, Deep Learning “What’s the Difference Between Artificial Intelligence, Machine Learning, and Deep Learning?”, Michael Copeland, 2016
- 7. How is itdifferent? Teach machines, not instruct Increasing compute power Massive growth in data Advanced statistical methods +
- 8. Face identification foridentity verification • Real-time ID Check feature • Photo-matching technology to an array of screening methods it uses • Fast and easy visual verification • Scalable to more than 1 million partners • Work well on a range of smartphones • Accommodate the variable photo sizes, resolutions, and aspect ratios • Compare photos in varying pose, focus, and lighting conditions blogs.microsoft.com/transform/feature/how-uber-is-using-driver-selfies-to-enhance-security-powered-by-microsoft-cognitive-services/
- 9. Grocery item objectdetection and recognition • Automated grocery inventory management in connected refrigerators • Implemented Fast R-CNN object detection in CNTK. REST API published using Python Flask in Azure • Annotated 311 images, split into 71 test and 240 training images. In total 2578 annotated objects, i.e. on average 123 examples per class • Prototype classifier has a precision of 98% at a recall of 80%, and 93% precision at recall of 90% blogs.technet.microsoft.com/machinelearning/2016/09/02/microsoft-and-liebherr-collaborating-on-new-generation-of-smart-refrigerators/
- 10.
- 11. Amplifying human ingenuity Trustedand flexible approach that puts you in control Powerful platform that makes AI accessible that extend your capabilities Innovate and accelerate with powerful tools and services that bring AI to every developer. Drive your digital transformation with accelerators, solutions, and practices to empower your organization with AI. Experience the intelligence built into Microsoft products and services you use every day. Cortana is helping you stay on top of it all so you can focus on what matters most. microsoft.com/ai
- 12. AI platform VS Tools forAI Azure ML Studio CODING & MANAGEMENT TOOLS Azure ML Workbench DEEP LEARNING FRAMEWORKS Cognitive Toolkit TensorFlow Caffe Others (Scikit-learn, MXNet, Keras, Chainer, Gluon…) 3rd Party Others (PyCharm, Jupyter Notebooks…) AI ON DATA Cosm os DB AI COMPUTE SQL DB SQL DW Data Lake Spark DS VM Batch AI ACS CPU, FPGA, GPU Edge CUSTOM SERVICESCONVERSATIONAL AI TRAINED SERVICES Bot Framework Azure Machine LearningCognitive Services Services Infrastructure Tools azure.microsoft.com/ai
- 13. How can Istart? AI-as-a-Service Leverage AI APIs Data + AI Add AI where the data is AI Create & train models
- 14. SQL Server 2017Machine Learning Services In-database Python & R integration Run Python & R in stored procedures Remote compute contexts for Python & R Gain access to libraries from open source ecosystem Built-in Machine Learning Algorithms MicrosoftML package includes customizable deep neural networks, fast decision trees and decision forests, linear regression, and logistic regression Access to pre-trained models such as image recognition Real-time and native scoring Model stored in optimized binary format, enabling faster scoring operations without calling R runtime Native T-SQL function for fast scoring
- 15. Microsoft Cognitive Services Vision ComputerVision Content Moderator Custom Vision Service Emotion API Face API Video Indexer Speech Bing Speech Service Custom Speech Service Speaker Recognition Translator Speech Language Bing Spell Check Language Understanding Intelligent Service (LUIS) Linguistics Analysis Text Analytics Translator Text Web Language Model Knowledge Custom Decision Service Entity Linking Knowledge Exploration Service QnA Maker Recommendations Academic Knowledge Search Bing News Bing Video Search Bing Web Search Bing Autosuggest Bing Custom Search Bing Entity Search Bing Image Search List of available released and preview services as of November 2017
- 16. Microsoft Cognitive Toolkit Differentiatesautomatically and trains the net when users implement the forward direction of the network Unified framework supporting a wide range of uses • FNN, RNN, LSTM, CNN, DSSM, GAN, etc. • All types of deep learning applications: e.g., speech, vision and text C++, C#, Java, Python; Linux and Windows Distributed training • Can scale to hundreds of GPUs and VM’s Open source • Hosted on GitHub – Jan 25, 2016 • Contributors from Microsoft and external (MIT, Stanford, etc.) Input layer Hidden layer 1 Hidden layer 2 Output layer A A x x x + tanh tanhơ ơ ơ
- 17. Deep Learning inAzure Create model using Azure Data Science Virtual Machine with GPU Ubuntu 16.04 LTS, OpenLogic 7.2 CentOS, Windows Server 2012 CNTK, Tensorflow, MXNet, Caffe & Caffe2, Torch, Theano, Keras, Nvidia Digits, etc. Train and score models using Azure Batch AI Training with Dockerized tools Provision multi-node CPU/GPU and VM set jobs Execute massively parallel computational workflows Hardware microservices using FPGA Deploy trained models as web API’s Multiple compute technologies – Virtual Machines, Container Service, Service Fabric, App Service, Edge, etc.
- 18. Azure Machine Learning Spark SQLServer Virtual machines GPUs Container services Notebooks IDEs Azure Machine Learning Workbench SQL Server Machine Learning Server ON- PREMISES EDGE COMPUTING Azure IoT Edge Experimentation and Model Management AZURE MACHINE LEARNING SERVICES TRAIN & DEPLOY OPTIONS A ZURE Built with open source tools Jupyter Notebook, Apache Spark, Docker, Kubernetes, Python, Conda Studio Workbench Experimentation Service Model Management Service Libraries for Apache Spark (MMLSpark Library) Visual Studio Code Tools for AI
- 19.
- 20. Deep Learning Labrador Larger anddeeper networks Many layers; some up to 150 layers Billions of learnable parameters Feed Forward, Recurrent, Convolutional, Sparse, etc. Trained on big data sets 10,000+ hours of speech Millions of images Years of click data Highly parallelized computation Long-running training jobs (days, weeks, months) Acceleration with GPU Recent advances in more computer power and big data
- 21. Designing a solutionfor deep learning TestingPreparation Development Training Operationalize • Evaluate the model on separate data sets (ground truth) • Data access • Data preparation • Labeled data set • Data management • Storage performance • Network performance • Re-training automation • Data reading • Data pre-processing • Model creation (e.g. layer architecture) • Learning & evaluation • Model optimization (e.g., parameter tuning, SGD, batch sizes, backpropagation, convergence & regularization strategies, etc.) • High-scale job scheduling • On-demand compute infrastructure • Managed task execution • Data / model parallelism • Data transfer • Compute infrastructure • Deploy and serve the model • Model dependencies • Feedback loop • Application architecture • DevOps toolchain A subset of tasks in Microsoft Team Data Science Process Lifecycle (TDSP)
- 22. Model Development Used inMicrosoft first-party AI implementations Unified framework supporting a wide range of uses FNN, RNN, LSTM, CNN, DSSM, etc. All types of deep learning applications: e.g., speech, vision and text C++, C#, Java, Python; Linux and Windows Distributed training Can scale to hundreds of GPUs and VM’s Open source Hosted on GitHub – Jan 25, 2016 Contributors from Microsoft and external (MIT, Stanford, etc.) 15K 15K 15K 15K 15K 500 500 500 max max ... ... ... max 500 ... ... Word hashing layer: ft Convolutional layer: ht Max pooling layer: v Semantic layer: y <s> w1 w2 wT <s>Word sequence: xt Word hashing matrix: Wf Convolution matrix: Wc Max pooling operation Semantic projection matrix: Ws ... ... 500
- 23. Model Development Data setof hand written digits with 60,000 training images 10,000 test images Each image is: 28 x 28 pixels Vector (array) of 784 elements Labels encoded using 1-hot encoding (e.g., 5 = “labels 0 0 0 0 0 1 0 0 0 0”) Apply data transformations Shuffle training data Add noise (e.g., numpy.random) Distort images with affline transformation (translations or rotations) 1 5 4 3 5 3 5 3 5 9 0 6 Corresponding labelsHandwritten images
- 24. Model Development S S 0.10.1 0.3 0.9 0.4 0.2 0.1 0.1 0.6 0.3 Model SBias (10) (𝑏) 0 1 9 … 784 pixels ( 𝑥) 28 pix 28pix S = Sum (weights x pixels) = 𝑤0 ∙ 𝑥 𝑇 784 784 General solution approach • A corresponding weight array for each element in the input array • Find the suitable weights to classify the image vector into corresponding digit • Repeat the process 10 times; each for the digits from 0-9 • Compute the output of the classifiers (10 of them) by multiplying all the weights with the corresponding pixels • Add a scalar value called bias to each of the summation units • Normalize output of summation units to a 0-1 range using a sigmoid activation function
- 25. Model Development softmax import cntkas C input_dim = 784 num_output_classes = 10 input = C.input_variable(input_dim) label = C.input_variable(num_output_classes) def create_model(features): with C.layers.default_options(init = C.glorot_uniform()): r = C.layers.Dense(num_output_classes, activation = None)(features) return r
- 26. Model Development num_hidden_layers =2 hidden_layers_dim = 400 def create_model(features): with C.layers.default_options(init = C.layers.glorot_uniform(), activation = C.ops.relu): h = features for _ in range(num_hidden_layers): h = C.layers.Dense(hidden_layers_dim)(h) r = C.layers.Dense(num_output_classes, activation = None)(h) return r softmax
- 27. Model Development def create_model(features): withC.layers.default_options(init=C.glorot_uniform(), activation=C.relu): h = features h = C.layers.Convolution2D(filter_shape=(5,5), num_filters=8, strides=(2,2), pad=True, name='first_conv')(h) h = C.layers.Convolution2D(filter_shape=(5,5), num_filters=16, strides=(2,2), pad=True, name='second_conv')(h) r = C.layers.Dense(num_output_classes, activation=None, name='classify')(h) return r
- 28. Model Development 1. Trainon ImageNet 3. Medium dataset: finetuning 2. Small dataset: feature extractor Freeze these Train this more data = retrain more of the network (or all of it) Freeze these Train this
- 29. Model Development Initialization Data loadingand reading Network setup Loss function Error function Learning algorithms (SGD, AdaGrad, etc.) Minibatch sizing Learning rate Training Evaluation / testing
- 30. Designing a solutionfor deep learning TestingPreparation Development Training Operationalize • Evaluate the model on separate data sets (ground truth) • Data access • Data preparation • Labeled data set • Data management • Storage performance • Network performance • Re-training automation • Data reading • Data pre-processing • Model creation (e.g. layer architecture) • Learning & evaluation • Model optimization (e.g., parameter tuning, SGD, batch sizes, backpropagation, convergence & regularization strategies, etc.) • High-scale job scheduling • On-demand compute infrastructure • Managed task execution • Data / model parallelism • Data transfer • Compute infrastructure • Deploy and serve the model • Model dependencies • Feedback loop • Application architecture • DevOps toolchain A subset of tasks in Microsoft Team Data Science Process Lifecycle (TDSP)
- 31.
- 32.
- 33.
- 34.
- 35.
- 36.
- 37. Training 1. Create aDNN training script with any DL framework 2. Package the DNN as a Docker image and upload it to the Azure Container Registry 3. Create a pool with GPU VMs 4. Add a job with tasks to run a hyper-parameter sweep experiment tasks 5. Tasks are scheduled to the pool and the Docker image is downloaded if required 6. Data is copied to the container 7. Tasks as containers perform the DNN training 8. Tasks write results and trained models to storage DSVM
- 38. Operationalization Sample application workflow: 1.Develop and test locally a flask web-service that load the model in memory and handles requests 2. Create a deployment script to set-up the dependencies on the Azure App Service environment 3. Git commit and push to repo 4. Deployment triggered on Azure Web App instance configured with Github continuous deployment 5. Send requests to the Web App DSVM DSVM Sample container workflow: 1. Develop and test locally a flask- based web-service container that loads the model in memory and handles requests 2. Build and upload the Docker image to a registry 3. Trigger deployment of Azure Web App 4. Send requests to the Web App
- 39.
- 40.
- 41.
- 42.
- 43. © Copyright MicrosoftCorporation. All rights reserved. Thank you!
Editor's Notes
- #4 Can you spot the snow leopard?
- #7 Here we offer a bit more clarity around the more detailed terms, as there is a significant amount of confusion how artificial intelligence compares or relates to machine learning, and to deep learning. Here is one way of looking at this. Artificial intelligence, or AI, is the highest level term describing the intelligence exhibited by machines, implemented or delivered via many different fields of study in computer science, and technical methods. Machine learning, is one of those fields within the scope of artificial intelligence; it generally describes the ability for machines to learn without being explicitly programmed. And then we have deep learning, which is a form of machine learning, that is credited for driving a lot of the recent success and progress in artificial intelligence. And the distinction of ‘deep’ versus conventional machine learning, is that deep learning generally is associated with the use of artificial neural networks that have many layers in the network architecture; or ‘hidden layers’ to be more technically correct. And this is the view we will use for the rest of our conversation as well.
- #8 So, artificial intelligence actually has been around for a long time, since it was first conceptualized back in the 1950’s, and developed over the years. How is it any different than mainstream computing as we have known, and why is it getting all the hype now? First, artificial intelligence generally is created via a different way of ‘programming’. Actually, it’s not really ‘programming’ as we have known. Traditionally, we’d write code to instruct the computer, and the computer would execute those same lines of code exactly the way they were written and generate the expected output, given the data that it works with. And the static code would crash if encountering a data scenario that it doesn’t know how to handle. Sounds familiar right? With artificial intelligence software, the model is flipped around, where we teach a computer with data and intended output, which will then generate the code it’d use. We don’t write static lines of instructions, but we teach a software with data and intended output; the machine learning software would then ‘learn’ through the data to understand how find the answer we’re looking for. This is similar to how we learn as humans as well. We have to be taught, in order to distinguish between a dog and a cat, or even different dog breeds, by assessing the features and conditions and then to arrive at our best guess at the answer. That, is the primary difference between AI and structured programming. And that is how AI can be used to solve unstructured problems; by simulating or mimicking how we as humans learn and think. So this is all research and theory that have been around for a long time. Why is artificial intelligence suddenly more relevant and useful now? Well, for computers to learn, effectively, how to think like humans, these machine learning software need a ton of computational cycles, and a ton of data. And as you know, through the big data and cloud computing waves, we now have access to massive compute power and massive data sets to work with. Plus recent advances in statistical/math methods, we now start to see artificial intelligence software being more effective at tasks that simulate elements of human thinking.
- #12 AI intelligently senses, processes, and acts on information—learning and adapting over time. We believe that, when designed with people at the center, AI can extend your capabilities, free you up for more creative and strategic endeavors, and help you or your organization achieve more. Innovations that extend your capabilities Intelligence infused into products like Office 365, Cortana, Bing and Skype are helping millions of people save time and be more productive. Whether you’re looking to break down language barriers or bring professional design to your presentations, Microsoft AI can extend your capabilities today. Powerful platform that makes AI accessible Built on breakthrough advances in AI research and the power of the cloud, we’re delivering a flexible platform for organizations and developers to infuse intelligence into their products and services using tools and services like Microsoft Cognitive Services, Azure Machine Learning, and the Bot Framework. Trusted approach that puts you in control Our transparent approach to AI puts your privacy first. Built on our enterprise-grade security practices, it helps protect your information and puts you in control. Our principles lead with ethics, accountability, and inclusive design to empower people and organizations, and positively impact society. Please go visit microsoft.com/ai to learn more about the overall approach. Today we will focus on various technologies in the AI platform.
- #13 AI intelligently senses, processes, and acts on information—learning and adapting over time. We believe that, when designed with people at the center, AI can extend your capabilities, free you up for more creative and strategic endeavors, and help you or your organization achieve more. Innovations that extend your capabilities Intelligence infused into products like Office 365, Cortana, Bing and Skype are helping millions of people save time and be more productive. Whether you’re looking to break down language barriers or bring professional design to your presentations, Microsoft AI can extend your capabilities today. Powerful platform that makes AI accessible Built on breakthrough advances in AI research and the power of the cloud, we’re delivering a flexible platform for organizations and developers to infuse intelligence into their products and services using tools and services like Microsoft Cognitive Services, Azure Machine Learning, and the Bot Framework. Trusted approach that puts you in control Our transparent approach to AI puts your privacy first. Built on our enterprise-grade security practices, it helps protect your information and puts you in control. Our principles lead with ethics, accountability, and inclusive design to empower people and organizations, and positively impact society. Please go visit microsoft.com/ai to learn more about the overall approach. Today we will focus on various technologies in the AI platform.
- #41 Social sites, forums, and other text-heavy Q&A services rely heavily on tagging, which enables indexing and user search. Without appropriate tagging, these sites are far less effective. Often, however, tagging is left to the users’ discretion. And since users don’t have lists of commonly searched terms or a deep understanding of the categorization or information architecture of a site, posts are frequently mislabeled. This makes it difficult or impossible to find that content when it’s needed later. By combining deep learning and natural language processing (NLP) with data on site-specific search terms, this solution helps greatly improve tagging accuracy on your site. As your user types their post, it offers highly used terms as suggested tags, making it easier for others to find the information they’re providing. Components: Microsoft SQL Server - Data is stored, structured, and indexed using Microsoft SQL Server. GPU based Azure Data Science Virtual Machine - The core development environment is the Microsoft Windows Server 2016 GPU DSVM NC24. Azure Machine Learning Workbench - The Workbench is used for data cleaning and transformation, and it serves as the primary interface to the Experimentation and Model Management services. Azure Machine Learning Experimentation Service - The Experimentation Service is used for model training, including hyperparameter tuning. Azure Machine Learning Model Management Service - The Model Management service is used for deployment of the final model, including scaling out to a Kubernetes-managed Azure cluster. Jupyter Notebooks on Azure Data Science VM - Jupyter Notebooks is used as the base IDE for the model, which was developed in Python. Azure Container Registry - The Model Management Service creates and packages real-time web services as Docker containers. These containers are uploaded and registered via Azure Container Registry. Azure Container Service Cluster - Deployment for this solution uses Azure Container Service running a Kubernetes-managed cluster. The containers are deployed from images stored in Azure Container Registry.
- #42 Lean manufacturing, cost control, and waste reduction are imperative for manufacturing to remain competitive. In circuit-board manufacturing, faulty boards can cost manufacturers money and productivity. Assembly lines rely on human operators to quickly review and validate boards flagged as potentially faulty by assembly-line test machines. This solution analyzes electronic component images generated by assembly-line cameras in a circuit-board manufacturing plant and detects their error status. The goal is to minimize or remove the need for human intervention. The solution builds an image classification system using a convolutional neural network with 50 hidden layers, pretrained on 350,000 images in an ImageNet dataset to generate visual features of the images by removing the last network layer. These features are then used to train a boosted decision tree to classify the image as “pass” or “fail” and final scoring conducted on edge machines at the plant. The classification performance results are good (time-based cross-validation AUC>.90) which indicates the solution is suitable to drastically minimize human intervention for electronic-components failure detection in assembled circuit boards. Using this solution to automate failure detection instead of relying solely on human operators helps improve the identification of faulty electronic components and boost productivity. Components: Azure Blob Storage - Data is ingested and stored in Azure Blob Storage. GPU based Azure Data Science Virtual Machine - The core development environment is the Azure Ubuntu-based GPU DSVM. The data is pulled from blob onto an Azure virtual hard disk (VHD) attached to the DSVM. On that VHD, the data is processed, the images are featurized using a Deep Neural Network, and a Boosted Tree model is trained. DSVM IPython Notebook server is used for solution development. Azure Batch AI training (BAIT) - As an alternative to DSVM-based training, for computing-intensive jobs that use deep-learning image processing, we use BAIT as a managed Azure Batch framework for parallel and distributed computing using clusters of GPU compute nodes. Microsoft Machine Learning for Apache Spark HDInsight Spark Cluster - As an alternative to DSVM-based training, for big datasets, we use MMLSpark to build a highly scalable training solution. Azure Container Registry - The model and web application are packaged into a Docker image and written to Azure Container Registry. Azure Machine Learning Model Management service - used to deploy and manage the final model on a VM and to scale out using Azure Container Service to a Kubernetes managed Azure cluster. A predictive web service and a Java ETL service are also written onto the VM, each in its own container. Azure Container Service Cluster - Deployment for this solution uses Azure Container Service running a Kubernetes-managed cluster. The containers are deployed from images stored in Azure Container Registry.
- #43 The services used by modern IT departments generate large volumes of telemetry data to track various aspects of operational health, system performance, usage insights, business metrics, alerting, and many others. Often, however, monitoring and gathering insights from all of this data isn’t fully automated and can be error prone, making it hard to effectively and accurately determine the health of the system at any given point in time. This customizable anomaly-detection solution uses machine learning to ensure high IT-systems availability, and it provides an end-to-end pipeline that ingests data from on-premises and cloud data sources and reports anomalous events to downstream monitoring and ticketing systems. With this solution, you’ll quickly detect and fix issues based on underlying health metrics from IT infrastructure (CPU, memory, etc.), services (timeouts, SLA variations, brownouts, etc.), and other key performance indicators (order backlog, login and payment failures, etc.). Components: Event Hubs - This is the entry point of the pipeline, where the raw timeseries data is ingested. Stream Analytics - performs aggregation at 5-minute intervals, and aggregates raw data points by metric name. Storage - Azure Storage stores data aggregated by the Stream Analytics job. Data Factory - calls the Anomaly Detection API at regular intervals (every 15 minutes by default) on the data in Azure Storage. It stores the results in a SQL database. SQL Database - stores the results from the Anomaly Detection API, including binary detections and detection scores. It also stores optional metadata sent with the raw data points to allow for more complicated reporting. Machine Learning Studio – This hosts the Anomaly Detection API. Note that the API itself is stateless and requires historical data points to be sent in each API call. Service Bus - Detected anomalies are published to a service bus topic to enable consumption by external monitoring services. Application Insights - allows for monitoring of the pipeline. Power BI - provides dashboards showing the raw data, as well as detected anomalies.