Successfully reported this slideshow.
Your SlideShare is downloading. ×

Monitoring Models

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Upcoming SlideShare
Responsible Machine Learning
Responsible Machine Learning
Loading in …3
×

Check these out next

1 of 11 Ad

Monitoring Models

Download to read offline

After a model has been deployed, it's important to understand how the model is being used in production, and to detect any degradation in its effectiveness due to data drift. This module describes tech- niques for monitoring models and their data.

After a model has been deployed, it's important to understand how the model is being used in production, and to detect any degradation in its effectiveness due to data drift. This module describes tech- niques for monitoring models and their data.

Advertisement
Advertisement

More Related Content

More from Eng Teong Cheah (20)

Recently uploaded (20)

Advertisement

Monitoring Models

  1. 1. Monitor Azure Machine Learning
  2. 2. Eng Teong Cheah Microsoft MVP
  3. 3. Monitor Azure Machine Learning When you have critical applications and business processes relying on Azure resources, you want to monitor those resources for their availability, performance, and operation. This article describes the monitoring data generated by Azure Machine Learning and how to analyze and alert on this data with Azure Monitor.
  4. 4. What is Azure Monitor? Azure Machine Learning creates monitoring data using Azure Monitor, which is a full stack monitoring service in Azure. Azure Monitor provides a complete set of features to monitor your Azure resources. It can also monitor resources in other clouds and on- premises.
  5. 5. Monitor data from Azure Machine Learning Azure Machine Learning collects the same kinds of monitoring data as other Azure resources that are described in Monitoring data from Azure resources.
  6. 6. Collection and routing Platform metrics and the Activity log are collected and stored automatically, but can be routed to other locations by using a diagnostic setting. Resource Logs are not collected and stored until you create a diagnostic setting and route them to one or more locations. When you need to manage multiple Azure Machine Learning workspaces, you could route logs for all workspaces into the same logging destination and query all logs from a single place.
  7. 7. Analyzing metrics You can analyze metrics for Azure Machine Learning, along with metrics from other Azure services, by opening Metrics from the Azure Monitor menu. See Getting started with Azure Metrics Explorer for details on using this tool.
  8. 8. Filtering and splitting For metrics that support dimensions, you can apply filters using a dimension value. For example, filtering Active Cores for a Cluster Name of cpu-cluster. You can also split a metric by dimension to visualize how different segments of the metric compare with each other. For example, splitting out the Pipeline Step Type to see a count of the types of steps used in the pipeline.
  9. 9. Transparency When AI systems help inform decisions that have tremendous impacts on people's lives, it's critical that people understand how those decisions were made. For example, a bank might use an AI system to decide whether a person is creditworthy. A company might use an AI system to determine the most qualified candidates to hire. A crucial part of transparency is interpretability: the useful explanation of the behavior of AI systems and their components. Improving interpretability requires stakeholders to comprehend how and why AI systems function the way they do. The stakeholders can then identify potential performance issues, fairness issues, exclusionary practices, or unintended outcomes.
  10. 10. Analyzing logs Using Azure Monitor Log Analytics requires you to create a diagnostic configuration and enable Send information to Log Analytics.
  11. 11. References Microsoft Docs

×