Downloaded 153 times
Uploaded bySquared Up
Full stack monitoring across apps & infrastructure with Azure Monitor
The document outlines Azure Monitor's comprehensive capabilities for monitoring and securing applications, infrastructure, and networks. It highlights features such as custom metrics, integration with DevOps tools, advanced telemetry, and pricing changes to a pay-as-you-go model for data ingestion. Additionally, it emphasizes the unification of monitoring services, enabling stakeholders to gain end-to-end visibility and efficient management of resources across Azure and on-premises environments.
More Related Content
![[2017 Windows on AWS] AWS 를 활용한 Active Directory 연동 및 이관 방안](https://cdn.slidesharecdn.com/ss_thumbnails/4-171027022544-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Full stack monitoring across apps & infrastructure with Azure Monitor
Full stack monitoring across apps & infrastructure with Azure Monitor
- 2. PROTECT SECURE MONITOR CONFIGURE GOVERN Security Management Threat Protection Backup DisasterRecovery Policy Management Cost Management Configuration Update Management Automation Scripting Azure Log Analytics App, Infra & Network Monitoring MIGRATE
- 3. Azure Monitor Documentation– https://docs.microsoft.com/en-us/azure/azure- monitor/ - new consolidated landing page for LA/AppInsight https://aka.ms/MonitoringDocs Resources, Tutorials & What’s New – https://aka.ms/AzMonOverview
- 4. https://aka.ms/Ignite2018/AzureMonitor Azure Monitor E2Ew/ Announcements Azure Monitor for VMs/Containers & Log Analytics Continuous Monitoring for DevOps Network Monitoring Telemetry & ITSM/SIEM Partners Smart Alert Management
- 5.
- 6. One Metrics. OneLogs. One Alerts. Log Analytics query experience integrated into Azure Portal Integration into native Azure resource blades Configure Azure AD to send audit & sign-up logs to Azure Monitor Ability to send Custom Metrics Azure Monitor for resource groups Azure Monitor for VMs (health, performance, and maps) Multi-cluster health rollup view for AKS Distributed Tracing for Python/Go in addition to .NET, Java & Node.js apps Java Local Forwarder, Micrometer & Spring Boot support Onboard VMs at scale via Azure Policy Secured monitoring inside Virtual Networks Store logs in firewall restricted secured storage accounts Monitor ER and store NSG flow logs across subscriptions Scale & SecurityData Driven InsightsUnified Monitoring
- 7. Advanced diagnostics and analyticspowered by machine learning capabilities Data Driven Insights Rich ecosystem of popular DevOps, issue management, SIEM, and ITSM tools Workflow Integrations A common platform for all metrics, logs and other monitoring telemetry Unified Monitoring Metrics Log Common Store Full observability for your infra, app and network
- 8. Azure Subscriptions Security Center Resource Manager ServiceHealth Azure Resources Network Security Groups Virtual Machines Storage Accounts Guest OS (‘user space’) Linux syslog Windows Perf Counters Application User telemetry Application logs Azure Tenants Azure Active Directory Signals Sources
- 9. Metrics Logs Application Container VMMonitoring Solutions Insights Dashboards Views Power BI Workbooks Visualize Metrics Explorer Log Analytics Analyze Alerts Autoscale Respond Event Hubs Ingest & Export APIs Logic Apps Integrate Azure Monitor Custom Sources Application Operating System Azure Resources Azure Subscription Azure Tenant
- 10. Metrics Logs Azure Monitor Custom Sources Application OperatingSystem Azure Resources Azure Subscription Azure Tenant Logs & Metrics emitted by Azure Diag. Extensions + Agents Windows + Linux Support Workload Agnostic Application Insights SDK Driven Multi-Language Support For everything else
- 11. Unified Monitoring Integration intonative Azure resource blades One Metrics, One Logs, One Alerts across Azure/on-prem resources Ability to send custom metrics & custom logs Unified offering with App Insights & Log Analytics as integrated features
- 12. Jump to ApplicationMap or VM Map Monitor health state of all resources Drill down into failures or perf issues See alerts firing across app & infra
- 13. Track E2E distributedtransactions (including for Python & Go) NEW! Monitor apps in .NET, JS, Java, Node.js or any language with OSS SDKs NEW! Drill down to code-level with Snapshot Debugging & Profiling Visualize server/client connections & dependencies with App Map Understand end-user cohorts, behavior & engagement for planning
- 14. Visualize service dependencies& connection failures in Maps Monitor single VMs or at scale Onboard at scale using PowerShell or Azure Policy Troubleshoot perf issues like CPU, memory, disk, and network Identify & isolate host-level or guest-level health problems
- 15. Understand cluster capacityneeds under average or heaviest loads Monitor multi-cluster health & node/pod status NEW! Monitor containers on demand in AKS with virtual nodes NEW! Analyze Kubernetes event & container logs for troubleshooting View overall perf across nodes, controllers and containers
- 16. Monitor ExpressRoute connectivity tovirtual networks and O365 Secure and audit your network with Network Watcher Traffic Analytics Monitor connectivity to LoB apps with Service Connectivity Monitor Discover and monitor ExpressRoute circuits, across subscriptions
- 17. Advanced Queries withLog Analytics Run queries for investigations, statistics & root cause/trend analyses Log Analytics advanced query experience now in Azure Portal NEW! Utilize ML algorithms for clustering and anomaly detection Central Analytics Platform across Monitoring, Management, Security
- 18. Integration with Monitoring& SIEM Tools Integrate your existing APM/Monitoring solutions with Azure Monitor Azure Monitor is best for Azure, and provides both APM & SIEM capabilities Route telemetry to your SIEM solutions for analytics & security management Open and extensible to continue using your favorite tools & solutions
- 19. Partner Solutions forApps & Workloads
- 20. Rich insightsfor Azure Storage, SQL & Service Fabric Mesh RBAC per data type in Log Analytics Support for containers on demand for AKS with virtual nodes Support live logs for containers and Kubernetes events Smart detections and alerts with Dynamic Thresholds Expanded Azure support and interoperability for Distributed Tracing Exception tracking and custom metrics for Python and Go apps Auto enablement of app monitoring for App Services / Windows VMs (with WAD) Expanded region availability across public & sovereign clouds Latency improvements for Alerts, Logs & Metrics
- 21. Visibility – Getthe Big Picture Near Real-time Alerts & Notifications Multi-Dimensional Metrics Health & Availability Monitoring Azure Dashboards Insights – Find & Fix Problems Composite Application & Service Maps Distributed Transaction Tracing Advanced Analytics with ML Automated Actions & Remediations Optimization – Build, Measure, Learn Performance Optimization & Profiling User Behaviour & Customer Insights Impact Correlation Integration with Dev/DevOps Tools
- 22. Gain visibility acrossworkloads Enable consistent control and compliance Respond faster to security threats Ensure availability of apps and data Insight & Analytics Protection & Recovery Security & Compliance Automation & Control https://docs.microsoft.com/en-gb/azure/azure-monitor/azure- monitor-rebrand#retirement-of-operations-management-suite- brand
- 23. System Center System Center 2007 System Center 2012 SystemCenter 2012 R2 System Center 2016/2019… Operations Management Suite / Log Analytics / Security Center Alerts ~30sec to 5mins
- 26. PROTECT SECURE MONITOR CONFIGURE GOVERN Security Management Threat Protection Backup DisasterRecovery Policy Management Cost Management Configuration Update Management Automation Scripting Azure Log Analytics App, Infra & Network Monitoring MIGRATE
- 27. Full Stack Monitoring& Analytics across Apps and Infra Application Insights Scenario Specific Monitoring – Customized Data Ingestion & Diagnostics Log Analytics Service Map Container Health …Network Performance Monitor Monitoring Fundamentals – Available out of the box with Azure Platform Activity LogsDiagnostic Logs Service HealthMetrics Dashboards Alerts Action Groups Autoscale Unified pricing model Only pay what you use Data ingestion per GB
- 28. Private or hostedthird-party cloud, Rackspace, etc. WINDOWS WINDOWS WINDOWS WINDOWS Public cloud Azure or AWS Simplified guest and workload management, both on-premises and in the cloud On-premises with System Center or Direct Agent WINDOWS HYPER-V WINDOWS VMWare WINDOWS Log Analytics / ASC
- 29. https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-manage-access Today, a workspaceprovides: • A geographic location for data storage • Granularity for billing • Data isolation • Scope for configuration Based on the preceding characteristics, you may want to create multiple workspaces if: • You are a global company and you need data stored in specific regions for data sovereignty or compliance reasons. • You are using Azure and you want to avoid outbound data transfer charges by having a workspace in the same region as the Azure resources it manages. • You want to allocate charges to different departments or business groups based on their usage. When you create a workspace for each department or business group, your Azure bill and usage statement shows the charges for each workspace separately. • You are a managed service provider and need to keep the Log Analytics data for each customer you manage isolated from other customer’s data. • You manage multiple customers and you want each customer / department / business group to see their own data but not the data for others. My addition: To set different retentions for different data types i.e. Security vs. Events
- 30. Guidance: Please try andcreate as few large workspaces as you can! Have a playground workspace as well for testing and to monitor your actual usage and costs! An average Azure VM ingests ~1 GB to 3 GB of data per month. Log Analytics + Azure Security Center = all data in one place, less joins or cross workspace issues E.g. Today One Workspace in a region, with all data and all users You may need secondary (or more) workspaces because: 1. Geographical concerns 2. Compliance 3. Latency – esp if you are a global org Note: data that is sent from other regions includes outbound data transfer charges. 4. RBAC (being addresses – in preview stage) https://docs.microsoft.com/en- us/azure/log-analytics/log-analytics- manage-access#determine-the- number-of-workspaces-you-need
- 32.
- 34. •Windows or Linuxcomputers and virtual machines. You install the Microsoft Monitoring Agent on Windows and Linux •Azure services. Log Analytics collects telemetry from Azure Diagnostics and Azure Monitoring into the repository •Data Collector API. Log Analytics has a REST API for populating data from any client.
- 35. Note: near-real timealerting is listed here: https://docs.microsoft.com/en-us/azure/monitoring- and-diagnostics/monitoring-near-real-time-metric-alerts
- 37. Please use theAzure Portal to access the product not the Legacy OMS portal. Simply from Portal.Azure.Com, select your Log Analytics workspace – then press WORKSPACE SUMMARY or Log Search rather than OMS Portal. The old portal is being phased out and many features are no longer available there.
- 39.
- 40.
- 42. Creating dashboards with contentspanning Log Analytics, Application Insights and Azure http://blogs.catapultsyste ms.com/…/creating- dashboards-wit…/ and this one
- 45. Application Insights isan extensible Application Performance Management (APM) service for web developers on multiple platforms. Use it to monitor your live web application. It will automatically detect performance anomalies. It includes powerful analytics tools to help you diagnose issues and to understand what users actually do with your app. It's designed to help you continuously improve performance and usability. It works for apps on a wide variety of platforms including .NET, Node.js and J2EE, hosted on-premises or in the cloud. It integrates with your DevOps process, and has connection points to a variety of development tools. It can monitor and analyze telemetry from mobile apps by integrating with Visual Studio App Center and HockeyApp. https://docs.microsoft.com/en-us/azure/application- insights/app-insights-overview Uses a workspace like Log Analytics – You can query App Insights from Log Analytics e.g.
- 47. Query //per computer union withsource= tt * | where _IsBillable == true | summarize Bytes=sum(_BilledSize) by Computer | sort by Bytes nulls last Links: https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-usage?toc=/azure/azure- monitor/toc.json#troubleshooting-why-usage-is-higher-than-expected October 2018 updated SLA https://azure.microsoft.com/en-us/support/legal/sla/log-analytics/v1_3/ New Ingestion time defined Nov 2018 https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-data-ingestion- time?fbclid=IwAR08sXlFCyOcnTvlwU4QE12P38CxuIFwWyOWhHwlGfHlyxE2LlSisaaQtk4
- 49. https://azure.microsoft.com/en-us/blog/introducing-a-new-way-to-purchase-azure-monitoring-services/ Should I moveto the new April 1st 2018 licence model – use the cost estimator ?Should https://docs.microsoft.com/en-us/azure/monitoring-and-diagnostics/monitoring-usage-and-estimated-cos 1. Consistent pay-as-you-go pricing We are adopting a simple “pay-as-you-go” model across the complete portfolio of monitoring services. You have full control and transparency, so you pay for only what you use. 2. Consistent per gigabyte (GB) metering for data ingestion We are changing the pricing model for data ingestion from “per node” to “per GB”. Customers told us that the value in monitoring came from the amount of data received and the insight built on top of that, rather than the number of nodes. In addition, this new model works best for the future of containers and microservices where the definition of a node is less clear. “Per GB” data ingestion is the new basis for pricing across application, infrastructure, and networking monitoring. Learn more about the new pricing model by visiting the updated pricing calculator or the individual product pricing pages for Log Analytics, Network Watcher, Azure Monitor, and Application Insights.
- 51. We support theCEF format of logs and can accept those, OMS Security supports collection of logs using CEF over Syslogs and Cisco ASA logs, Arcsight etc... What is CEF? Common Event Format (CEF) is an industry standard format on top of Syslog messages, used by many security vendors to allow event interoperability among different platforms. OMS Security and Audit Solution support data ingestion using CEF, which enables you to connect your security products with OMS Security.
- 52. This is aperformance expensive operation – its an exception. E.g. for GDPR “forget me” requests. https://docs.microsoft.com/en-us/rest/api/loganalytics/workspace/purge Top Tip: Be VERY careful with this, it’s a one time action – if you have support maybe involve them
- 53.
- 54.
- 55.
- 56.
- 57. Getting started withqueries This is a series of posts to get you started with the query language Azure Log Analytics: Queries, the basics explained – Part 1 Azure Log Analytics: Queries, the basics explained – Part 2 Azure Log Analytics: Queries, the basics explained – Part 3 Azure Log Analytics: Queries, the basics explained – Part 4
- 58. Kusto Query Language(KQL) course Free course on the Query language (should be free mid July, in the meantime you will need Pluralsight access) https://www.pluralsight.com/courses/kusto-query- language-kql-from-scratch
- 59.
- 60. Azure Advisors –if you have an NDA with us? http://aka.ms/azureadvisors http://aka.ms/azureadvisorsaddcontact http://aka.ms/joinadvisors
- 61.
- 62. FAQ Most Common Questions: Todeploy agents, do we need to have target computer objects in Azure? OMS agents can be installed 3 ways. Directly from OMS portal, SCOM agent can be used as well to upload data to OMS via Sys Center Operations Manager or by simply enabling OMS agent as extension in Azure VMs. Connect Windows computers to the Log Analytics service in Azure https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-windows-agents Connect Azure virtual machines to Log Analytics with a Log Analytics agent https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-azure-vm-extension Connect Operations Manager to Log Analytics https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-om-agents How do we deploy the agent, via OMS portal, or can we do it with SCCM? You cannot push agent via OMS portal but in Azure portal via one click (Connect button) you can install agent on VM and yes via SCCM you can push OMS agent as well. Do we configure the agent to pull the data we want to or it just collects all by default and we can then filter it, or make required report? For this I encourage to understand the concept of Data sources in Log Analytics. Lot of data is captured out of the box but certain data types can be captured on demand or tuned on or off like events, perf counters etc. https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-data-sources What is the limitation in trial, can you deploy agent to 50 or 5000 machines? An OMS workspace on Free Tier won’t allow more than 500 MB data upload from all machines. There is no number of agents limit but total data allowed to upload and data retention (7 days on Free tier) limit. And in the end, what is the estimated cost of OMS under subscription, is it calculated per device? On Licensing https://www.microsoft.com/en-cy/cloud-platform/operations-management- suite-pricing ADDITIONAL INFORMATION: Computer groups in Log Analytics log searches https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-computer-groups Computer groups in OMS https://blogs.technet.microsoft.com/msoms/2016/04/04/computer-groups-in-oms/ How to verify that your Microsoft OMS agents are working properly https://blogs.technet.microsoft.com/omsblog/2016/01/05/how-to-verify-that-your- microsoft-oms-agents-are-working-properly/ How to troubleshoot Operations Management Suite onboarding issues https://support.microsoft.com/en-us/kb/3126513
Editor's Notes
- #3 2
- #6 We typically see: Hybrid Environments. Multiple VMs. Containers & Microservices. Custom Workloads. Apps written with diverse languages/platforms. You might have monolithic apps deployed on on-premises servers/VMs or microservices based apps deployed on AKS. We understand that you need to be able to monitor your entire stack end-to-end, from an overview to drilling down into specific components And we understand that you would like a single tool/product/service to enable you to do that.
- #7 What’s New in Network Monitoring: Multi-subscription capability for ER monitor Navigate to resources from Topology view NSG Flow logs support for Classic NSG resources Multi-subscription single storage support to store flow logs In resource monitoring and troubleshooting experience for VM, VNET
- #14 Java Support: Micrometer (Auto collection of Java Metrics like Tomcat, JVM, GC, etc.), Local Forwarder (Adaptive Sampling & Live Metrics), Spring Boot Starter (Single package to enable Monitoring for Java Apps, managed through Maven/Gradle)
- #17 Azure provides network troubleshooting and monitoring capabilities Our solutions can monitor networks In Azure, hybrid and on-premises networks We have released several new monitoring capabilities, since previous Ignite: Traffic Analytics - Network Watcher enables the generation of traffic flow logs corresponding to Network Security Groups set up by customers. These flow logs are used as the basis by the Traffic Analytics solution to visualize traffic flow characteristics into/out of customers’ virtual networks, correlated with other information such as malicious IP. TA gives insights for capacity planning, traffic flow audit and identifying anomalies. NPM can now monitor ExpressRoute circuits and connectivity to SaaS and LoB applications ExpressRoute - ExpressRoute is a key connectivity solution offered to connect customer premises with Azure using high-bandwidth private connectivity. Customers can monitor the characteristics of their ER connections including loss/latency/bandwidth utilization as well as topology details and usage of primary and secondary ER connections. Service Connectivity Monitor - Customers can also monitor connectivity to cloud services, such as O365 and Salesforce. All of the above solutions are generally available.
- #21 “Expanded Azure Support” tactically means Functions, API Management and IoT Hub, but strategically points to our Azure Fundamentals work and goals to onboard all of Azure. “Interoperability” refers to our work as innovators by standardizing within the W3C, as well as the tactical work of switching out SDKs to support the standard rather than our current proprietary format.
- #22 21
- #24 Since the release of System Center 2007, Microsoft has evolved its management tools with two key aims always in mind: continuous innovation and customer focus. With each release, Microsoft has provided leading-edge management capabilities to meet the needs of enterprise customers: System Center 2007: System Center 2007 focused on the management of highly virtualized heterogeneous datacenters. System Center 2012: Built on the lessons learned in previous generations, System Center 2012 was designed to provide a high-performance enterprise datacenter management tool that scaled to meet the new demands of virtualized n-tier application architectures. System Center 2012 R2: Moving from the virtualized datacenter to a software-defined datacenter, System Center 2012 R2 brought private cloud and hybrid cloud management based on Microsoft Azure expertise to enterprise IT. System Center 2016: System Center 2016 uses cloud-first design principles to simplify the deployment, configuration, management, and monitoring of the virtualized, software-defined datacenter and hybrid cloud infrastructure, including new capabilities in Windows Server such as Nano Server and Docker containers, and for managing across diverse environments with full LAMP stack compatibility. Additionally, OMS was released in 2015 as a cloud-based (Azure-based) management tool that takes data collection and analysis to a new level, and integrates seamlessly with Azure services.
- #27 26
- #28 27
- #29 Customer scenarios Easy to onboard to OMS – same Microsoft management agent Visualize SCOM alerts in OMS Run Orchestration from OMS through hybrid worker DPM connection to Azure Backup Cross-sell / up-sell Why use OMS when SCOM is still running? https://blogs.technet.microsoft.com/msoms/2016/01/11/why-use-oms-while-scom-is-running/ - it’s fast, easy to setup, and you can start for free - Microsoft is investing heavily into it and you’ll see new features and capabilities every month