What is going on - Application diagnostics on Azure - TechDays Finland
Report
Share
•1 like•746 views
![And here’s a typical log...
App.exe [12:13:03:985] Information: 0 : Customer address updated.
App.exe [12:13:04:011] Error: 8 : System.NullReferenceException occurred. Value
can not be null.
App.exe [12:13:04:567] Information: 0 : Machine policy value 'Debug' is 0
App.exe [12:13:04:569] Verbose: 9 : ******* RunEngine
******* Action:
******* CommandLine: **********
App.exe [12:13:04:578] Information: 9 : Entered CheckResult()
App.exe [12:13:04:689] Debug: 0 : Created. Req=0, Ret=8, Font: Req=null
Does this help troubleshooting? Improve the application? Analyze trends?](https://image.slidesharecdn.com/techdays-whatisgoingon-applicationdiagnosticsonazure-180322121446/75/what-is-going-on-application-diagnostics-on-azure-techdays-finland-5-2048.jpg?cb=1668959061)
What is going on - Application diagnostics on Azure - TechDays Finland
•1 like•746 views
Report
Share
Download to read offline
We all like building and deploying cloud applications. But what happens once that’s done? How do we know if our application behaves like we expect it to behave? Of course, logging! But how do we get that data off of our machines? How do we sift through a bunch of seemingly meaningless diagnostics? In this session, we’ll look at how we can keep track of our Azure application using structured logging, AppInsights and AppInsights analytics to make all that data more meaningful.
More Related Content
What's hot
What's hot(20)
Similar to What is going on - Application diagnostics on Azure - TechDays Finland
Similar to What is going on - Application diagnostics on Azure - TechDays Finland(20)
![[Webinar] Camunda Optimize Release 3.0](https://cdn.slidesharecdn.com/ss_thumbnails/optreleasewebinar3-200408171409-thumbnail.jpg?width=768&fit=bounds)
More from Maarten Balliauw
More from Maarten Balliauw(20)
Recently uploaded
Recently uploaded(20)
What is going on - Application diagnostics on Azure - TechDays Finland
- 1. What is going on? Application Diagnostics on Azure Maarten Balliauw @maartenballiauw
- 2. Agenda What is going on? Application Insights the service the developer side (SDK) Application Insights Analytics
- 3. What is going on?
- 4. Log all the things! System.Diagnostics.Trace.TraceInformation( "Something happened"); System.Diagnostics.Trace.TraceWarning( "Error! " + ex.Message);
- 5. And here’s a typical log... App.exe [12:13:03:985] Information: 0 : Customer address updated. App.exe [12:13:04:011] Error: 8 : System.NullReferenceException occurred. Value can not be null. App.exe [12:13:04:567] Information: 0 : Machine policy value 'Debug' is 0 App.exe [12:13:04:569] Verbose: 9 : ******* RunEngine ******* Action: ******* CommandLine: ********** App.exe [12:13:04:578] Information: 9 : Entered CheckResult() App.exe [12:13:04:689] Debug: 0 : Created. Req=0, Ret=8, Font: Req=null Does this help troubleshooting? Improve the application? Analyze trends?
- 6. No.
- 7. Log files suck. Just stupid string data Typical log file has no “context” Typical log file has no correlation Log files do not measure CPU, memory, I/O, ... How to get data off our machines? How to report/analyze?
- 8. Log files suck. Just stupid string data Typical log file has no “context” Typical log file has no correlation Log files do not measure CPU, memory, I/O, ... How to get data off our machines? How to report/analyze? Semantic Logging (e.g. ETW, Serilog, ...)
- 9. Log files suck. Just stupid string data Typical log file has no “context” Typical log file has no correlation Log files do not measure CPU, memory, I/O, ... How to get data off our machines? How to report/analyze? Windows Server tooling, NewRelic, PRTG, SNMP, ...
- 10. Log files suck. Just stupid string data Typical log file has no “context” Typical log file has no correlation Log files do not measure CPU, memory, I/O, ... How to get data off our machines? How to report/analyze? Vendor tooling, NewRelic, PRTG, SNMP, ...
- 11. Log files suck. Just stupid string data Typical log file has no “context” Typical log file has no correlation Log files do not measure CPU, memory, I/O, ... How to get data off our machines? How to report/analyze? Splunk, LogStash, Kibana, ...
- 12. Analytics suck! Either use vendor tooling What with x-plat deployments? Either build it ourselves Lots of components, management overhead, ... All we wanted was logging and correlation...
- 14. Application Insights ”Application Insights is an extensible Application Performance Management (APM) service for web developers on multiple platforms.”
- 15. Application Insights Azure Service + Library/SDK Solves “where to store”, “how to ship”, “how to analyze” Enriches data with telemetry Correlates data (e.g. client + server + dependency/DB/...) Allows structured logging Allows rich querying, alerting Works for many, many platforms and languages Web, Windows, Xamarin, any application type really .NET, Java, JavaScript, Objective-C, PHP, Python, Ruby, ... Docker (host and container)
- 18. Do I have to run on Azure? REST API to send data to Various tools and SDK’s to collect data specific to language/platform https://docs.microsoft.com/en-us/azure/application-insights/app-insights-platforms docker run -v /var/run/docker.sock:/docker.sock -d microsoft/applicationinsights ikey=000000-1111-2222-3333-444444444 IIS Docker host and containers
- 19. Application Insights – the service
- 20. Application Insights Let’s focus on this part for a bit.
- 22. Input, processing, output DATA INGESTION Performance Counters Requests (both server/client side) Traces Exceptions Dependencies Custom Metrics & Events ... CAPABILITIES Monitoring Alerting Querying Exporting
- 23. Application Insights – the developer side
- 24. Application Insights Let’s focus on this part for a bit.
- 25. Data being collected AUTOMATICALLY Performance Counters Requests (both server/client side) Traces Exceptions Dependencies Custom Metrics & Events ... ENRICH MANUALLY Better tracing (semantic logging!) Events Metrics Exceptions Telemetry pipeline ...
- 26. Better tracing Don’t do stupid strings. System.Diagnostics.Trace is nice, but also not nice. Do semantic logging! Serilog, .NET Core logging have an Application Insights sink All else: write a sink or log directly to TelemetryClient.Current
- 28. Events and Metrics Send custom events and metrics Name of event/metric + a value Events – help find how the application is used and can be optimized User resized column in a grid User logged in User clicked “Share” ... Metrics – help measuring quantifyable data # Items in shopping cart ...
- 30. Exceptions Exceptions are unpleasant. How to make them more pleasant? Stack traces? MiniDump?
- 31. Exceptions DEMO
- 32. Snapshots! (MiniDump) NuGet package MinidumpUploader.exe separate process which monitors snapshot requests SnapshotHolder_x64.exe creates a shadow process in 10-20ms, captures the minidump, returns it to MinidumpUploader.exe https://docs.microsoft.com/en-us/azure/application-insights/app-insights- snapshot-debugger
- 33. Enriching telemetry Why? Enriching telemetry – adding properties to all data Obfuscating telemetry – GDPR, sensitive data, ... How? Directly adding data on TelemetryClient.Current Using the telemetry pipeline (middlewares on sending data to AI service) https://blog.maartenballiauw.be/post/2017/01/31/application-insights-telemetry-processors.html
- 36. Application Insights Analytics Former MS-internal tool “Kusto” Near-realtime log ingestion and analysis Lets you run custom queries requests | where timestamp > ago(24h) | summarize count() by client_CountryOrRegion | top 10 by count_ | render piechart
- 37. Queries over logs! Input dataset traces customEvents pageViews requests dependencies exceptions availabilityResults customMetrics performanceCounters browserTimings | Operators where count project join limit order | Renderer table chart (bar, pie, time, area, scatter) https://docs.microsoft.com/en-us/azure/application-insights/app-insights-analytics-reference
- 38. Smart Detection Uses Application Insights Analytics data and searches for trends and anomalies Potential memleaks – memory increase Increase in specific exceptions Abnormal rise in failed requests ... Each detection comes with a query!
- 40. What else is there? Other client types (mobile, Java, ...) Sampling our data (in portal) Do we need all data? Or just data that is representative enough? Release annotations Show a marker in the timeline when a release happens https://docs.microsoft.com/en-us/azure/application-insights/app-insights-annotations Profiler Preview that I can’t get to work, but should do profiling when interesting things happen https://docs.microsoft.com/en-us/azure/application-insights/app-insights-profiler
- 41. Conclusion
- 42. Conclusion Logging sucks Application Insights the service – collect and ingest, analyze, different views the developer side (SDK) – enrich, customize, … Application Insights Analytics rich querying over all data base for automated insights/smart detection
- 43. Thank you! Please do not forget to evaluate the session before you leave by using our Lollipolls! http://blog.maartenballiauw.be @maartenballiauw
Editor's Notes
- https://pixabay.com/p-960806/
- Highlight which customer? Awesome, stack trace for Exception, “Debug is 0” great! RunEngine entry very useful. What if we want to aggregate “Req” / “Ret”?
- https://pixabay.com/p-960806/
- Open portal.azure.com Show how to create Application Insights service, open an existing one Explain the main view there: List of all data sources we can look at on the left Main chart showing response time, page view load time, server requests, failed requests -> health! Sidenote: every chart can be pinned to a portal dashboard in case you want a nice live TV thingy Charts can be drilled into Drill into server response time and show some things Do the same with exceptions Show that if we look at an event or use search there is a view of all data points as well, including traces and logging etc. Display details and show properties of the event – who, where, what, … Open MusicStore and display a few things that have been added: 2 NuGet package references Show AppInsights package reference added, show Startup.cs where to configure (3 simple “Use...”) Show _Layout.cshtml, run app, show snippet Explain data comes from many locations! The NuGet package does a lot here, but not all! Open service, click “Getting Started” Explain there are a few entry points to collect all data
- https://pixabay.com/en/cat-animal-eyes-grey-view-views-351926/
- There are a few things AppInsights detects automagically for us! Open MyGet AppInsights Live Metrics Stream Connects to our running application and inspects the data that comes in real-time Metrics, health, server CPU and memory, ... We can filter per server as well Application Map Show Availability tests test the server-side Show client-side uses server-side components Click around, explain what we can see here Smart Detection Smart Detection tries to be proactive in detecting things that are “not normal” Alerts (open Alerts) Add alert, see on which metrics we can alert Show examples, show web tests feeding into alerts, ... Continuous export Mention what it does (blob storage) Show MyGet dashboard in PowerBI Quickly go over these because we will show them later on! Availability – explain web tests, explain these can be simple or actual tests created in VS that have multiple steps and checks Failures – we can see diagrams of exceptions vs. other metrics, as well as Exceptions Performance – mention profiler available only for Azure App Services Mention slowest pages client/server side, … Servers – Some server details, quickly go through this Browser – Browser stats, verify client-side code, page render times, find slowest rendering pages, ... Usage – Correlate technical side with user-side sessions, users, custom events (more later), ... Pretty cool for installing a NuGet package and adding five lines of code, no?
- https://pixabay.com/en/cat-animal-eyes-grey-view-views-351926/
- Open MusicStore and show Startup.cs, method Configure Mention we are configuring logging here, adding AppInsights with the Information level Open AccountController and show we are structured tracing the login event. Do a login, then open AppInsights and find the event “logged in …” Details reveal username property We get extra info And we can show the request in which this was logged and get more info
- Open MusicStore and explain tracking cart AlbumAdded metric Click in the site a bit. Perform a checkout. Complete.cshtml contains client-side tracking as well! Angular! Vue.js! Show metrics in portal, show events as well Show we can correlate Show funnels as well (explain we could add multiple metrics to look at e.g. a sign-up funnel)
- Open MusicStore and explain tracking cart AlbumAdded metric Click in the site a bit. Perform a checkout. Complete.cshtml contains client-side tracking as well! Angular! Vue.js! Show metrics in portal, show events as well Show we can correlate Show funnels as well (explain we could add multiple metrics to look at e.g. a sign-up funnel)
- Open MusicStore In Startup.cs, find Configure. We register two .NET middlewares here, show how they work and what they do. Explain this is something we can always do to enrich data. Telemetry pipeline that runs before shipping off telemetry is also interesting Find Startup.cs ConfigureServices where we add two telemetry processors One to filter out synthetic traffic Another one to filter out sensitive URLs Explain we can capture any type of metric, change the result, …
- https://pixabay.com/p-960806/
- Open portal.azure.com on MyGet AppInsights Demo AppInsights Analytics Show some of the sample queries, show the editor, show the various rendering types (auto complete in there) Run a few sample queries and find custom dimensions Show smart detection and the queries it generates
- Click the links and show what it is as “things to look at after this talk”