SlideShare a Scribd company logo
1 of 51
What is going on?
Application Diagnostics on Azure
Maarten Balliauw
@maartenballiauw
Who am I?
Maarten Balliauw
Antwerp, Belgium
Developer Advocate, JetBrains
Founder, MyGet
AZUG
Focus on web
ASP.NET MVC, Azure, SignalR, ...
Former MVP Azure & ASPInsider
Big passion: Azure
http://blog.maartenballiauw.be
@maartenballiauw
Agenda
Logging
Why?
Logging sucks!
The need for semantic/structured logging
Application Insights
SDK, Azure portal
Application Insights Analytics
Find the needle in the haystack!
Build, measure, improve
Logging
Why logging?
Troubleshooting – Did a problem occur? Where? Our code? The machine?
Performance / Cost – Did we make the application slower?
Improvement – Can a problem be detected or avoided?
Trends – Need more storage? Need more servers?
Customer Experience – Are people happy? Or seeing error after error?
Business Decisions – Can we increase sales based on log data / telemetry?
So here’s what we typically do...
System.Diagnostics.Trace.TraceInformation(
"Something happened");
System.Diagnostics.Trace.TraceWarning(
"Error! " + ex.Message);
Does this help troubleshooting? Improve the application? Analyze trends?
And here’s a typical log...
App.exe [12:13:03:985] Information: 0 : Customer address updated.
App.exe [12:13:04:011] Error: 8 : System.NullReferenceException occurred. Value
can not be null.
App.exe [12:13:04:567] Information: 0 : Machine policy value 'Debug' is 0
App.exe [12:13:04:569] Verbose: 9 : ******* RunEngine
******* Action:
******* CommandLine: **********
App.exe [12:13:04:578] Information: 9 : Entered CheckResult()
App.exe [12:13:04:689] Debug: 0 : Created. Req=0, Ret=8, Font: Req=null
Does this help troubleshooting? Improve the application? Analyze trends?
No.
Log files suck.
Stupid string data
Unless using ETW / proper System.Diagnostics listener
Typical log file has no “context”
Typical log file has no correlation
How to get data off our machines?
How to report/analyze?
Log files suck.
Stupid string data
Typical log file has no “context”
Process/thread
Machine name
User performing the action
Data specific to the event being logged
Typical log file has no correlation
How to get data off our machines?
How to report/analyze?
Log files suck.
Stupid string data
Typical log file has no “context”
Typical log file has no correlation, e.g.:
What was the server load when this was logged?
How much memory was consumed at the time?
Which web request invoked this action?
How to get data off our machines?
How to report/analyze?
Log files suck.
Stupid string data
Typical log file has no “context”
Typical log file has no correlation
How to get data off our machines?
How to report/analyze?
Log files suck.
Stupid string data
Typical log file has no “context”
Typical log file has no correlation
How to get data off our machines?
How to report/analyze?
Log processing and analysis – log, store, process, query, proactive analysis
Logstash, ElasticSearch, ... – cumbersome to setup
What if we had (better)
contextual information?
Structured/Semantic logging
“Of or relating to meaning, especially meaning in language”
– the dictionary
Windows Event Log
Event Tracing for Windows (ETW)
Semantic Logging Application Blog (PnP SLAB)
Serilog
…
Event Source
XML representation containing
contextual information
application-specific data elements
Can be read by humans (just a message, like any log)
Can be read by machines (XML, baby!)
add “meaning” – we know what the data points can be
allow automated aggregaton / trend analysis
Fast!
(not very developer friendly)
Using EventSource
DEMO
Event Source
Super awesome
It has structured logging and context
Can collect all kinds of data combined (e.g. HTTP + own event source)
Hard to maintain
Versioning
Ceremony
One method for every single thing we want to log (or at least an event id)
Requires thinking about the events to log
Lots of data
Serilog - https://serilog.net/
Logging library like any other, mostly
Sinks https://github.com/serilog/serilog/wiki/Provided-Sinks
Console, file, event log
Database
ElasticSearch
AppInsights
Structured data as a first-class concept
Enrichers, filters, …
Structured data in Serilog
Simple scalar values (bool, string, int, …)
Collections and dictionaries
Full objects
Rendered by a sink to, for example, a string
Can be stored by a sink to provide additional dimensions on data
Serilog - Basics
DEMO
Enrichment of data
Enrich log entries with additional context
Machine name
User name
Process / thread information
ASP.NET client IP / hostname
ASP.NET user agent
…
Using Serilog enrichers (when applicable)
In custom ASP.NET middleware
But still...
How to get the data off our machine(s)?
How to analyze/aggregate/predict?
Which customer had this Exception and why?
Did our sales go up after we made this page faster?
What button on the front-end crashes this microservice?
Are # Exceptions increasing?
Do I need special infrastructure to store and process logs?
The cloud to the rescue? (Azure)
Azure Insights
Azure platform’s core monitoring and alerting services (standard Metrics, Alerts,
Autoscale, Email Notifications and Audit Logs)
Application Insights
Application monitoring & logging for your applications (web, mobile, desktop, …) in
cloud or other server
Log Analytics (OMS)
Log Analytics, part of the Operations Management Suite (OMS) ingests data from
Azure servers and provides rich ‘global’ view & advanced log search based alerting
capabilities
Application Insights
Application Insights
Azure Service + library/SDK
Solves “where to store”, “how to ship”, “how to analyze”
Enriches data
Correlates data (e.g. client + server + dependency/DB/...)
Telemetry!
Allows structured logging
Allows rich querying, alerting
Works for many, many platforms and languages
Web, Windows, Xamarin, any application type really
.NET, Java, JavaScript, Objective-C, PHP, Python, Ruby, ...
Getting Started
DEMO
Do I have to run on Azure?
No. REST API to send data to
Various tools and SDK’s to collect data specific to language/platform
Status Monitor on non-Azure machines
Data collected
Performance Counters
Requests (both server/client side)
Traces (more later)
Exceptions
Dependencies
Custom Metrics & Events
That is a lot of data that can be correlated!
Different Views
Application Insights portal
Dashboard
Basic metrics (# Alerts, Response times, Exceptions, Failed Requests, ...)
Gateway to all of the below
Application map
Smart Detection / Alerts
Live Metrics Stream
Search, Availability, Failures, Performance, ...
Metrics (and events)
Analytics
Application Map
DEMO
Smart Detection / Alerts
DEMO
Live Metrics Stream
DEMO
Search, Availability,
Failures, Performance, ...
DEMO
Metrics and Events
Send custom events and metrics
Name of event/metric + a value
Events – help find how the application is used and can be optimized
User resized column in a grid
User logged in
User clicked “Share”
No more stock
...
Metrics – help alerting / finding missing functionality
Purchase occurred
Search returned zero results
# Support calls
...
Metrics and Events
DEMO
So what about logging...
Configure Serilog to use Application Insights as a sink
Custom data becomes available (*)
So we can correlate our own data with performance, requests, traces,
exceptions, depenencies, ...
But how to do this? And how to consume?
(*) Make sure the entire object is in the log message, no tracking of non-printed data
Logging
DEMO
Application Insights
Analytics
Application Insights Analytics
Former MS-internal tool “Kusto”
Near-realtime log ingestion and analysis
Lets you run custom queries
requests
| where timestamp > ago(24h)
| summarize count() by client_CountryOrRegion
| top 10 by count_
| render piechart
Queries over logs!
Input dataset
traces
customEvents
pageViews
requests
dependencies
exceptions
availabilityResults
customMetrics
performanceCounters
browserTimings
| Operators
where
count
project
join
limit
order
| Renderer
table
chart (bar, pie, time, area, scatter)
https://docs.microsoft.com/en-us/azure/application-insights/app-insights-analytics-reference
Application Insights
Analytics
DEMO
Not covered in this talk
VS plugin for Application Insights
Run some analysis inside VS
Continuous export
Export data from Application Insights continuously if you want to analyze in
other tools
Retain data longer than 7 days
Join with other data (preview)
https://docs.microsoft.com/en-us/azure/application-insights/app-insights-
analytics-using#import-data
Join logging and metrics with other data sources, query with AI Analytics
Conclusion
But... Why?
Seriously, why are you in this session...
But... Why?
Troubleshooting – Did a problem occur? Where? Our code? The machine?
Performance / Cost – Did we make the application slower?
Improvement – Can a problem be detected or avoided?
Trends – Need more storage? Need more servers?
Customer Experience – Are people happy? Or seeing error after error?
Business Decisions – Can we increase sales based on log data/telemetry?
Structured logging
Enrich technical logs with business data
Allows correlating when things go wrong
Allows making business decisions when using a good data analysis tool on top
Application Insights
Is that good analysis tool! (for web, mobile, desktop!)
Solves
Collecting various sources
Ingesting various sources
Enrichment with structured logging (e.g. Serilog)
Helps analyzing all that data
Default views, metrics, alerts, monitoring
Custom querying – Application Insights Analytics
Build, measure, improve!
Both technical and business – there isn’t always a divide between both...
Thank you!
Need training, coaching, mentoring, performance analysis?
Hire me! https://blog.maartenballiauw.be/hire-me.html
http://blog.maartenballiauw.be
@maartenballiauw

More Related Content

What's hot

Deep Dive on Log Analytics with Elasticsearch Service
Deep Dive on Log Analytics with Elasticsearch ServiceDeep Dive on Log Analytics with Elasticsearch Service
Deep Dive on Log Analytics with Elasticsearch ServiceAmazon Web Services
 
AWS Webcast - Amazon Kinesis and Apache Storm
AWS Webcast - Amazon Kinesis and Apache StormAWS Webcast - Amazon Kinesis and Apache Storm
AWS Webcast - Amazon Kinesis and Apache StormAmazon Web Services
 
AWS re:Invent 2016: How Fulfillment by Amazon (FBA) and Scopely Improved Resu...
AWS re:Invent 2016: How Fulfillment by Amazon (FBA) and Scopely Improved Resu...AWS re:Invent 2016: How Fulfillment by Amazon (FBA) and Scopely Improved Resu...
AWS re:Invent 2016: How Fulfillment by Amazon (FBA) and Scopely Improved Resu...Amazon Web Services
 
213 event processingtalk-deviewkorea.key
213 event processingtalk-deviewkorea.key213 event processingtalk-deviewkorea.key
213 event processingtalk-deviewkorea.keyNAVER D2
 
Log Analytics with Amazon Elasticsearch Service & Kibana
Log Analytics with Amazon Elasticsearch Service & KibanaLog Analytics with Amazon Elasticsearch Service & Kibana
Log Analytics with Amazon Elasticsearch Service & KibanaAmazon Web Services
 
Modern Data Warehousing with Amazon Redshift
Modern Data Warehousing with Amazon RedshiftModern Data Warehousing with Amazon Redshift
Modern Data Warehousing with Amazon RedshiftAmazon Web Services
 
Real-time Analytics using Data from IoT Devices - AWS Online Tech Talks
Real-time Analytics using Data from IoT Devices - AWS Online Tech TalksReal-time Analytics using Data from IoT Devices - AWS Online Tech Talks
Real-time Analytics using Data from IoT Devices - AWS Online Tech TalksAmazon Web Services
 
Serverless Datalake Day with AWS
Serverless Datalake Day with AWSServerless Datalake Day with AWS
Serverless Datalake Day with AWSAmazon Web Services
 
Big Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWSBig Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWSAmazon Web Services
 
Introduction to AWS Step Functions:
Introduction to AWS Step Functions: Introduction to AWS Step Functions:
Introduction to AWS Step Functions: Amazon Web Services
 
Interactive Analytics on AWS - AWS Summit Tel Aviv 2017
Interactive Analytics on AWS - AWS Summit Tel Aviv 2017Interactive Analytics on AWS - AWS Summit Tel Aviv 2017
Interactive Analytics on AWS - AWS Summit Tel Aviv 2017Amazon Web Services
 
FSI202 Machine Learning in Capital Markets
FSI202 Machine Learning in Capital MarketsFSI202 Machine Learning in Capital Markets
FSI202 Machine Learning in Capital MarketsAmazon Web Services
 
BDA309 Building Your Data Lake on AWS
BDA309 Building Your Data Lake on AWSBDA309 Building Your Data Lake on AWS
BDA309 Building Your Data Lake on AWSAmazon Web Services
 
Big Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWSBig Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWSAmazon Web Services
 
Time Series Analytics Azure ADX
Time Series Analytics Azure ADXTime Series Analytics Azure ADX
Time Series Analytics Azure ADXRiccardo Zamana
 
AWS re:Invent 2016: Deep Learning in Alexa (MAC202)
AWS re:Invent 2016: Deep Learning in Alexa (MAC202)AWS re:Invent 2016: Deep Learning in Alexa (MAC202)
AWS re:Invent 2016: Deep Learning in Alexa (MAC202)Amazon Web Services
 
Black Friday and Cyber Monday- Best Practices for Your E-Commerce Database
Black Friday and Cyber Monday- Best Practices for Your E-Commerce DatabaseBlack Friday and Cyber Monday- Best Practices for Your E-Commerce Database
Black Friday and Cyber Monday- Best Practices for Your E-Commerce DatabaseTim Vaillancourt
 
Serverless Streaming Data Processing using Amazon Kinesis Analytics
Serverless Streaming Data Processing using Amazon Kinesis AnalyticsServerless Streaming Data Processing using Amazon Kinesis Analytics
Serverless Streaming Data Processing using Amazon Kinesis AnalyticsAmazon Web Services
 

What's hot (20)

Deep Dive on Log Analytics with Elasticsearch Service
Deep Dive on Log Analytics with Elasticsearch ServiceDeep Dive on Log Analytics with Elasticsearch Service
Deep Dive on Log Analytics with Elasticsearch Service
 
AWS Webcast - Amazon Kinesis and Apache Storm
AWS Webcast - Amazon Kinesis and Apache StormAWS Webcast - Amazon Kinesis and Apache Storm
AWS Webcast - Amazon Kinesis and Apache Storm
 
AWS re:Invent 2016: How Fulfillment by Amazon (FBA) and Scopely Improved Resu...
AWS re:Invent 2016: How Fulfillment by Amazon (FBA) and Scopely Improved Resu...AWS re:Invent 2016: How Fulfillment by Amazon (FBA) and Scopely Improved Resu...
AWS re:Invent 2016: How Fulfillment by Amazon (FBA) and Scopely Improved Resu...
 
213 event processingtalk-deviewkorea.key
213 event processingtalk-deviewkorea.key213 event processingtalk-deviewkorea.key
213 event processingtalk-deviewkorea.key
 
Log Analytics with Amazon Elasticsearch Service & Kibana
Log Analytics with Amazon Elasticsearch Service & KibanaLog Analytics with Amazon Elasticsearch Service & Kibana
Log Analytics with Amazon Elasticsearch Service & Kibana
 
Modern Data Warehousing with Amazon Redshift
Modern Data Warehousing with Amazon RedshiftModern Data Warehousing with Amazon Redshift
Modern Data Warehousing with Amazon Redshift
 
Real-time Analytics using Data from IoT Devices - AWS Online Tech Talks
Real-time Analytics using Data from IoT Devices - AWS Online Tech TalksReal-time Analytics using Data from IoT Devices - AWS Online Tech Talks
Real-time Analytics using Data from IoT Devices - AWS Online Tech Talks
 
Serverless Datalake Day with AWS
Serverless Datalake Day with AWSServerless Datalake Day with AWS
Serverless Datalake Day with AWS
 
Big Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWSBig Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWS
 
Introduction to AWS Step Functions:
Introduction to AWS Step Functions: Introduction to AWS Step Functions:
Introduction to AWS Step Functions:
 
Amazon Kinesis Data Streams
Amazon Kinesis Data StreamsAmazon Kinesis Data Streams
Amazon Kinesis Data Streams
 
Interactive Analytics on AWS - AWS Summit Tel Aviv 2017
Interactive Analytics on AWS - AWS Summit Tel Aviv 2017Interactive Analytics on AWS - AWS Summit Tel Aviv 2017
Interactive Analytics on AWS - AWS Summit Tel Aviv 2017
 
FSI202 Machine Learning in Capital Markets
FSI202 Machine Learning in Capital MarketsFSI202 Machine Learning in Capital Markets
FSI202 Machine Learning in Capital Markets
 
BDA309 Building Your Data Lake on AWS
BDA309 Building Your Data Lake on AWSBDA309 Building Your Data Lake on AWS
BDA309 Building Your Data Lake on AWS
 
Big Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWSBig Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWS
 
MCT Virtual Summit 2021
MCT Virtual Summit 2021MCT Virtual Summit 2021
MCT Virtual Summit 2021
 
Time Series Analytics Azure ADX
Time Series Analytics Azure ADXTime Series Analytics Azure ADX
Time Series Analytics Azure ADX
 
AWS re:Invent 2016: Deep Learning in Alexa (MAC202)
AWS re:Invent 2016: Deep Learning in Alexa (MAC202)AWS re:Invent 2016: Deep Learning in Alexa (MAC202)
AWS re:Invent 2016: Deep Learning in Alexa (MAC202)
 
Black Friday and Cyber Monday- Best Practices for Your E-Commerce Database
Black Friday and Cyber Monday- Best Practices for Your E-Commerce DatabaseBlack Friday and Cyber Monday- Best Practices for Your E-Commerce Database
Black Friday and Cyber Monday- Best Practices for Your E-Commerce Database
 
Serverless Streaming Data Processing using Amazon Kinesis Analytics
Serverless Streaming Data Processing using Amazon Kinesis AnalyticsServerless Streaming Data Processing using Amazon Kinesis Analytics
Serverless Streaming Data Processing using Amazon Kinesis Analytics
 

Similar to What is going on? Application Diagnostics on Azure - Copenhagen .NET User Group

What is going on - Application diagnostics on Azure - TechDays Finland
What is going on - Application diagnostics on Azure - TechDays FinlandWhat is going on - Application diagnostics on Azure - TechDays Finland
What is going on - Application diagnostics on Azure - TechDays FinlandMaarten Balliauw
 
Building an Observability Platform in 389 Difficult Steps
Building an Observability Platform in 389 Difficult StepsBuilding an Observability Platform in 389 Difficult Steps
Building an Observability Platform in 389 Difficult StepsDigitalOcean
 
Monitoring as Software Validation
Monitoring as Software ValidationMonitoring as Software Validation
Monitoring as Software ValidationBioDec
 
Practical operability techniques for teams - Matthew Skelton - Agile in the C...
Practical operability techniques for teams - Matthew Skelton - Agile in the C...Practical operability techniques for teams - Matthew Skelton - Agile in the C...
Practical operability techniques for teams - Matthew Skelton - Agile in the C...Skelton Thatcher Consulting Ltd
 
Semantic logging with etw and slab from DCC 10/16
Semantic logging with etw and slab from DCC 10/16Semantic logging with etw and slab from DCC 10/16
Semantic logging with etw and slab from DCC 10/16Chris Holwerda
 
SplunkLive! Zurich 2018: Integrating Metrics and Logs
SplunkLive! Zurich 2018: Integrating Metrics and LogsSplunkLive! Zurich 2018: Integrating Metrics and Logs
SplunkLive! Zurich 2018: Integrating Metrics and LogsSplunk
 
Internet of Things Chicago - Meetup
Internet of Things Chicago - MeetupInternet of Things Chicago - Meetup
Internet of Things Chicago - MeetupJason Lobel
 
Azure Data Explorer deep dive - review 04.2020
Azure Data Explorer deep dive - review 04.2020Azure Data Explorer deep dive - review 04.2020
Azure Data Explorer deep dive - review 04.2020Riccardo Zamana
 
Log Data Analysis Platform by Valentin Kropov
Log Data Analysis Platform by Valentin KropovLog Data Analysis Platform by Valentin Kropov
Log Data Analysis Platform by Valentin KropovSoftServe
 
Log Data Analysis Platform
Log Data Analysis PlatformLog Data Analysis Platform
Log Data Analysis PlatformValentin Kropov
 
Apache Spark Streaming -Real time web server log analytics
Apache Spark Streaming -Real time web server log analyticsApache Spark Streaming -Real time web server log analytics
Apache Spark Streaming -Real time web server log analyticsANKIT GUPTA
 
SplunkLive! Munich 2018: Data Onboarding Overview
SplunkLive! Munich 2018: Data Onboarding OverviewSplunkLive! Munich 2018: Data Onboarding Overview
SplunkLive! Munich 2018: Data Onboarding OverviewSplunk
 
Observability foundations in dynamically evolving architectures
Observability foundations in dynamically evolving architecturesObservability foundations in dynamically evolving architectures
Observability foundations in dynamically evolving architecturesBoyan Dimitrov
 
SplunkLive! Frankfurt 2018 - Data Onboarding Overview
SplunkLive! Frankfurt 2018 - Data Onboarding OverviewSplunkLive! Frankfurt 2018 - Data Onboarding Overview
SplunkLive! Frankfurt 2018 - Data Onboarding OverviewSplunk
 
Event Stream Processing SAP
Event Stream Processing SAPEvent Stream Processing SAP
Event Stream Processing SAPGaurav Ahluwalia
 
IMC Summit 2016 Breakout - Matt Coventon - Test Driving Streaming and CEP on ...
IMC Summit 2016 Breakout - Matt Coventon - Test Driving Streaming and CEP on ...IMC Summit 2016 Breakout - Matt Coventon - Test Driving Streaming and CEP on ...
IMC Summit 2016 Breakout - Matt Coventon - Test Driving Streaming and CEP on ...In-Memory Computing Summit
 
Azure Monitoring Overview
Azure Monitoring OverviewAzure Monitoring Overview
Azure Monitoring Overviewgjuljo
 
Microsoft SQL Server - StreamInsight Overview Presentation
Microsoft SQL Server - StreamInsight Overview PresentationMicrosoft SQL Server - StreamInsight Overview Presentation
Microsoft SQL Server - StreamInsight Overview PresentationMicrosoft Private Cloud
 
Big data on_aws in korea by abhishek sinha (lunch and learn)
Big data on_aws in korea by abhishek sinha (lunch and learn)Big data on_aws in korea by abhishek sinha (lunch and learn)
Big data on_aws in korea by abhishek sinha (lunch and learn)Amazon Web Services Korea
 
Responding to extended events in near real time
Responding to extended events in near real timeResponding to extended events in near real time
Responding to extended events in near real timeGianluca Sartori
 

Similar to What is going on? Application Diagnostics on Azure - Copenhagen .NET User Group (20)

What is going on - Application diagnostics on Azure - TechDays Finland
What is going on - Application diagnostics on Azure - TechDays FinlandWhat is going on - Application diagnostics on Azure - TechDays Finland
What is going on - Application diagnostics on Azure - TechDays Finland
 
Building an Observability Platform in 389 Difficult Steps
Building an Observability Platform in 389 Difficult StepsBuilding an Observability Platform in 389 Difficult Steps
Building an Observability Platform in 389 Difficult Steps
 
Monitoring as Software Validation
Monitoring as Software ValidationMonitoring as Software Validation
Monitoring as Software Validation
 
Practical operability techniques for teams - Matthew Skelton - Agile in the C...
Practical operability techniques for teams - Matthew Skelton - Agile in the C...Practical operability techniques for teams - Matthew Skelton - Agile in the C...
Practical operability techniques for teams - Matthew Skelton - Agile in the C...
 
Semantic logging with etw and slab from DCC 10/16
Semantic logging with etw and slab from DCC 10/16Semantic logging with etw and slab from DCC 10/16
Semantic logging with etw and slab from DCC 10/16
 
SplunkLive! Zurich 2018: Integrating Metrics and Logs
SplunkLive! Zurich 2018: Integrating Metrics and LogsSplunkLive! Zurich 2018: Integrating Metrics and Logs
SplunkLive! Zurich 2018: Integrating Metrics and Logs
 
Internet of Things Chicago - Meetup
Internet of Things Chicago - MeetupInternet of Things Chicago - Meetup
Internet of Things Chicago - Meetup
 
Azure Data Explorer deep dive - review 04.2020
Azure Data Explorer deep dive - review 04.2020Azure Data Explorer deep dive - review 04.2020
Azure Data Explorer deep dive - review 04.2020
 
Log Data Analysis Platform by Valentin Kropov
Log Data Analysis Platform by Valentin KropovLog Data Analysis Platform by Valentin Kropov
Log Data Analysis Platform by Valentin Kropov
 
Log Data Analysis Platform
Log Data Analysis PlatformLog Data Analysis Platform
Log Data Analysis Platform
 
Apache Spark Streaming -Real time web server log analytics
Apache Spark Streaming -Real time web server log analyticsApache Spark Streaming -Real time web server log analytics
Apache Spark Streaming -Real time web server log analytics
 
SplunkLive! Munich 2018: Data Onboarding Overview
SplunkLive! Munich 2018: Data Onboarding OverviewSplunkLive! Munich 2018: Data Onboarding Overview
SplunkLive! Munich 2018: Data Onboarding Overview
 
Observability foundations in dynamically evolving architectures
Observability foundations in dynamically evolving architecturesObservability foundations in dynamically evolving architectures
Observability foundations in dynamically evolving architectures
 
SplunkLive! Frankfurt 2018 - Data Onboarding Overview
SplunkLive! Frankfurt 2018 - Data Onboarding OverviewSplunkLive! Frankfurt 2018 - Data Onboarding Overview
SplunkLive! Frankfurt 2018 - Data Onboarding Overview
 
Event Stream Processing SAP
Event Stream Processing SAPEvent Stream Processing SAP
Event Stream Processing SAP
 
IMC Summit 2016 Breakout - Matt Coventon - Test Driving Streaming and CEP on ...
IMC Summit 2016 Breakout - Matt Coventon - Test Driving Streaming and CEP on ...IMC Summit 2016 Breakout - Matt Coventon - Test Driving Streaming and CEP on ...
IMC Summit 2016 Breakout - Matt Coventon - Test Driving Streaming and CEP on ...
 
Azure Monitoring Overview
Azure Monitoring OverviewAzure Monitoring Overview
Azure Monitoring Overview
 
Microsoft SQL Server - StreamInsight Overview Presentation
Microsoft SQL Server - StreamInsight Overview PresentationMicrosoft SQL Server - StreamInsight Overview Presentation
Microsoft SQL Server - StreamInsight Overview Presentation
 
Big data on_aws in korea by abhishek sinha (lunch and learn)
Big data on_aws in korea by abhishek sinha (lunch and learn)Big data on_aws in korea by abhishek sinha (lunch and learn)
Big data on_aws in korea by abhishek sinha (lunch and learn)
 
Responding to extended events in near real time
Responding to extended events in near real timeResponding to extended events in near real time
Responding to extended events in near real time
 

More from Maarten Balliauw

Bringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptxBringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptxMaarten Balliauw
 
Nerd sniping myself into a rabbit hole... Streaming online audio to a Sonos s...
Nerd sniping myself into a rabbit hole... Streaming online audio to a Sonos s...Nerd sniping myself into a rabbit hole... Streaming online audio to a Sonos s...
Nerd sniping myself into a rabbit hole... Streaming online audio to a Sonos s...Maarten Balliauw
 
Building a friendly .NET SDK to connect to Space
Building a friendly .NET SDK to connect to SpaceBuilding a friendly .NET SDK to connect to Space
Building a friendly .NET SDK to connect to SpaceMaarten Balliauw
 
Microservices for building an IDE - The innards of JetBrains Rider - NDC Oslo...
Microservices for building an IDE - The innards of JetBrains Rider - NDC Oslo...Microservices for building an IDE - The innards of JetBrains Rider - NDC Oslo...
Microservices for building an IDE - The innards of JetBrains Rider - NDC Oslo...Maarten Balliauw
 
Indexing and searching NuGet.org with Azure Functions and Search - .NET fwday...
Indexing and searching NuGet.org with Azure Functions and Search - .NET fwday...Indexing and searching NuGet.org with Azure Functions and Search - .NET fwday...
Indexing and searching NuGet.org with Azure Functions and Search - .NET fwday...Maarten Balliauw
 
NDC Sydney 2019 - Microservices for building an IDE – The innards of JetBrain...
NDC Sydney 2019 - Microservices for building an IDE – The innards of JetBrain...NDC Sydney 2019 - Microservices for building an IDE – The innards of JetBrain...
NDC Sydney 2019 - Microservices for building an IDE – The innards of JetBrain...Maarten Balliauw
 
JetBrains Australia 2019 - Exploring .NET’s memory management – a trip down m...
JetBrains Australia 2019 - Exploring .NET’s memory management – a trip down m...JetBrains Australia 2019 - Exploring .NET’s memory management – a trip down m...
JetBrains Australia 2019 - Exploring .NET’s memory management – a trip down m...Maarten Balliauw
 
.NET Conf 2019 - Indexing and searching NuGet.org with Azure Functions and Se...
.NET Conf 2019 - Indexing and searching NuGet.org with Azure Functions and Se....NET Conf 2019 - Indexing and searching NuGet.org with Azure Functions and Se...
.NET Conf 2019 - Indexing and searching NuGet.org with Azure Functions and Se...Maarten Balliauw
 
CloudBurst 2019 - Indexing and searching NuGet.org with Azure Functions and S...
CloudBurst 2019 - Indexing and searching NuGet.org with Azure Functions and S...CloudBurst 2019 - Indexing and searching NuGet.org with Azure Functions and S...
CloudBurst 2019 - Indexing and searching NuGet.org with Azure Functions and S...Maarten Balliauw
 
NDC Oslo 2019 - Indexing and searching NuGet.org with Azure Functions and Search
NDC Oslo 2019 - Indexing and searching NuGet.org with Azure Functions and SearchNDC Oslo 2019 - Indexing and searching NuGet.org with Azure Functions and Search
NDC Oslo 2019 - Indexing and searching NuGet.org with Azure Functions and SearchMaarten Balliauw
 
Approaches for application request throttling - Cloud Developer Days Poland
Approaches for application request throttling - Cloud Developer Days PolandApproaches for application request throttling - Cloud Developer Days Poland
Approaches for application request throttling - Cloud Developer Days PolandMaarten Balliauw
 
Indexing and searching NuGet.org with Azure Functions and Search - Cloud Deve...
Indexing and searching NuGet.org with Azure Functions and Search - Cloud Deve...Indexing and searching NuGet.org with Azure Functions and Search - Cloud Deve...
Indexing and searching NuGet.org with Azure Functions and Search - Cloud Deve...Maarten Balliauw
 
Approaches for application request throttling - dotNetCologne
Approaches for application request throttling - dotNetCologneApproaches for application request throttling - dotNetCologne
Approaches for application request throttling - dotNetCologneMaarten Balliauw
 
CodeStock - Exploring .NET memory management - a trip down memory lane
CodeStock - Exploring .NET memory management - a trip down memory laneCodeStock - Exploring .NET memory management - a trip down memory lane
CodeStock - Exploring .NET memory management - a trip down memory laneMaarten Balliauw
 
ConFoo Montreal - Microservices for building an IDE - The innards of JetBrain...
ConFoo Montreal - Microservices for building an IDE - The innards of JetBrain...ConFoo Montreal - Microservices for building an IDE - The innards of JetBrain...
ConFoo Montreal - Microservices for building an IDE - The innards of JetBrain...Maarten Balliauw
 
ConFoo Montreal - Approaches for application request throttling
ConFoo Montreal - Approaches for application request throttlingConFoo Montreal - Approaches for application request throttling
ConFoo Montreal - Approaches for application request throttlingMaarten Balliauw
 
Microservices for building an IDE – The innards of JetBrains Rider - TechDays...
Microservices for building an IDE – The innards of JetBrains Rider - TechDays...Microservices for building an IDE – The innards of JetBrains Rider - TechDays...
Microservices for building an IDE – The innards of JetBrains Rider - TechDays...Maarten Balliauw
 
JetBrains Day Seoul - Exploring .NET’s memory management – a trip down memory...
JetBrains Day Seoul - Exploring .NET’s memory management – a trip down memory...JetBrains Day Seoul - Exploring .NET’s memory management – a trip down memory...
JetBrains Day Seoul - Exploring .NET’s memory management – a trip down memory...Maarten Balliauw
 
DotNetFest - Let’s refresh our memory! Memory management in .NET
DotNetFest - Let’s refresh our memory! Memory management in .NETDotNetFest - Let’s refresh our memory! Memory management in .NET
DotNetFest - Let’s refresh our memory! Memory management in .NETMaarten Balliauw
 
VISUG - Approaches for application request throttling
VISUG - Approaches for application request throttlingVISUG - Approaches for application request throttling
VISUG - Approaches for application request throttlingMaarten Balliauw
 

More from Maarten Balliauw (20)

Bringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptxBringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptx
 
Nerd sniping myself into a rabbit hole... Streaming online audio to a Sonos s...
Nerd sniping myself into a rabbit hole... Streaming online audio to a Sonos s...Nerd sniping myself into a rabbit hole... Streaming online audio to a Sonos s...
Nerd sniping myself into a rabbit hole... Streaming online audio to a Sonos s...
 
Building a friendly .NET SDK to connect to Space
Building a friendly .NET SDK to connect to SpaceBuilding a friendly .NET SDK to connect to Space
Building a friendly .NET SDK to connect to Space
 
Microservices for building an IDE - The innards of JetBrains Rider - NDC Oslo...
Microservices for building an IDE - The innards of JetBrains Rider - NDC Oslo...Microservices for building an IDE - The innards of JetBrains Rider - NDC Oslo...
Microservices for building an IDE - The innards of JetBrains Rider - NDC Oslo...
 
Indexing and searching NuGet.org with Azure Functions and Search - .NET fwday...
Indexing and searching NuGet.org with Azure Functions and Search - .NET fwday...Indexing and searching NuGet.org with Azure Functions and Search - .NET fwday...
Indexing and searching NuGet.org with Azure Functions and Search - .NET fwday...
 
NDC Sydney 2019 - Microservices for building an IDE – The innards of JetBrain...
NDC Sydney 2019 - Microservices for building an IDE – The innards of JetBrain...NDC Sydney 2019 - Microservices for building an IDE – The innards of JetBrain...
NDC Sydney 2019 - Microservices for building an IDE – The innards of JetBrain...
 
JetBrains Australia 2019 - Exploring .NET’s memory management – a trip down m...
JetBrains Australia 2019 - Exploring .NET’s memory management – a trip down m...JetBrains Australia 2019 - Exploring .NET’s memory management – a trip down m...
JetBrains Australia 2019 - Exploring .NET’s memory management – a trip down m...
 
.NET Conf 2019 - Indexing and searching NuGet.org with Azure Functions and Se...
.NET Conf 2019 - Indexing and searching NuGet.org with Azure Functions and Se....NET Conf 2019 - Indexing and searching NuGet.org with Azure Functions and Se...
.NET Conf 2019 - Indexing and searching NuGet.org with Azure Functions and Se...
 
CloudBurst 2019 - Indexing and searching NuGet.org with Azure Functions and S...
CloudBurst 2019 - Indexing and searching NuGet.org with Azure Functions and S...CloudBurst 2019 - Indexing and searching NuGet.org with Azure Functions and S...
CloudBurst 2019 - Indexing and searching NuGet.org with Azure Functions and S...
 
NDC Oslo 2019 - Indexing and searching NuGet.org with Azure Functions and Search
NDC Oslo 2019 - Indexing and searching NuGet.org with Azure Functions and SearchNDC Oslo 2019 - Indexing and searching NuGet.org with Azure Functions and Search
NDC Oslo 2019 - Indexing and searching NuGet.org with Azure Functions and Search
 
Approaches for application request throttling - Cloud Developer Days Poland
Approaches for application request throttling - Cloud Developer Days PolandApproaches for application request throttling - Cloud Developer Days Poland
Approaches for application request throttling - Cloud Developer Days Poland
 
Indexing and searching NuGet.org with Azure Functions and Search - Cloud Deve...
Indexing and searching NuGet.org with Azure Functions and Search - Cloud Deve...Indexing and searching NuGet.org with Azure Functions and Search - Cloud Deve...
Indexing and searching NuGet.org with Azure Functions and Search - Cloud Deve...
 
Approaches for application request throttling - dotNetCologne
Approaches for application request throttling - dotNetCologneApproaches for application request throttling - dotNetCologne
Approaches for application request throttling - dotNetCologne
 
CodeStock - Exploring .NET memory management - a trip down memory lane
CodeStock - Exploring .NET memory management - a trip down memory laneCodeStock - Exploring .NET memory management - a trip down memory lane
CodeStock - Exploring .NET memory management - a trip down memory lane
 
ConFoo Montreal - Microservices for building an IDE - The innards of JetBrain...
ConFoo Montreal - Microservices for building an IDE - The innards of JetBrain...ConFoo Montreal - Microservices for building an IDE - The innards of JetBrain...
ConFoo Montreal - Microservices for building an IDE - The innards of JetBrain...
 
ConFoo Montreal - Approaches for application request throttling
ConFoo Montreal - Approaches for application request throttlingConFoo Montreal - Approaches for application request throttling
ConFoo Montreal - Approaches for application request throttling
 
Microservices for building an IDE – The innards of JetBrains Rider - TechDays...
Microservices for building an IDE – The innards of JetBrains Rider - TechDays...Microservices for building an IDE – The innards of JetBrains Rider - TechDays...
Microservices for building an IDE – The innards of JetBrains Rider - TechDays...
 
JetBrains Day Seoul - Exploring .NET’s memory management – a trip down memory...
JetBrains Day Seoul - Exploring .NET’s memory management – a trip down memory...JetBrains Day Seoul - Exploring .NET’s memory management – a trip down memory...
JetBrains Day Seoul - Exploring .NET’s memory management – a trip down memory...
 
DotNetFest - Let’s refresh our memory! Memory management in .NET
DotNetFest - Let’s refresh our memory! Memory management in .NETDotNetFest - Let’s refresh our memory! Memory management in .NET
DotNetFest - Let’s refresh our memory! Memory management in .NET
 
VISUG - Approaches for application request throttling
VISUG - Approaches for application request throttlingVISUG - Approaches for application request throttling
VISUG - Approaches for application request throttling
 

Recently uploaded

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Victor Rentea
 
2024 May Patch Tuesday
2024 May Patch Tuesday2024 May Patch Tuesday
2024 May Patch TuesdayIvanti
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Jeffrey Haguewood
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxFIDO Alliance
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfdanishmna97
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingScyllaDB
 
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...caitlingebhard1
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistandanishmna97
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxMarkSteadman7
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMKumar Satyam
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxFIDO Alliance
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctBrainSell Technologies
 
Navigating Identity and Access Management in the Modern Enterprise
Navigating Identity and Access Management in the Modern EnterpriseNavigating Identity and Access Management in the Modern Enterprise
Navigating Identity and Access Management in the Modern EnterpriseWSO2
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe中 央社
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Zilliz
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewDianaGray10
 
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....rightmanforbloodline
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptxFIDO Alliance
 

Recently uploaded (20)

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
2024 May Patch Tuesday
2024 May Patch Tuesday2024 May Patch Tuesday
2024 May Patch Tuesday
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptx
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cf
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptx
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDM
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage Intacct
 
Navigating Identity and Access Management in the Modern Enterprise
Navigating Identity and Access Management in the Modern EnterpriseNavigating Identity and Access Management in the Modern Enterprise
Navigating Identity and Access Management in the Modern Enterprise
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
 

What is going on? Application Diagnostics on Azure - Copenhagen .NET User Group

  • 1. What is going on? Application Diagnostics on Azure Maarten Balliauw @maartenballiauw
  • 2. Who am I? Maarten Balliauw Antwerp, Belgium Developer Advocate, JetBrains Founder, MyGet AZUG Focus on web ASP.NET MVC, Azure, SignalR, ... Former MVP Azure & ASPInsider Big passion: Azure http://blog.maartenballiauw.be @maartenballiauw
  • 3. Agenda Logging Why? Logging sucks! The need for semantic/structured logging Application Insights SDK, Azure portal Application Insights Analytics Find the needle in the haystack! Build, measure, improve
  • 5. Why logging? Troubleshooting – Did a problem occur? Where? Our code? The machine? Performance / Cost – Did we make the application slower? Improvement – Can a problem be detected or avoided? Trends – Need more storage? Need more servers? Customer Experience – Are people happy? Or seeing error after error? Business Decisions – Can we increase sales based on log data / telemetry?
  • 6. So here’s what we typically do... System.Diagnostics.Trace.TraceInformation( "Something happened"); System.Diagnostics.Trace.TraceWarning( "Error! " + ex.Message); Does this help troubleshooting? Improve the application? Analyze trends?
  • 7. And here’s a typical log... App.exe [12:13:03:985] Information: 0 : Customer address updated. App.exe [12:13:04:011] Error: 8 : System.NullReferenceException occurred. Value can not be null. App.exe [12:13:04:567] Information: 0 : Machine policy value 'Debug' is 0 App.exe [12:13:04:569] Verbose: 9 : ******* RunEngine ******* Action: ******* CommandLine: ********** App.exe [12:13:04:578] Information: 9 : Entered CheckResult() App.exe [12:13:04:689] Debug: 0 : Created. Req=0, Ret=8, Font: Req=null Does this help troubleshooting? Improve the application? Analyze trends?
  • 8. No.
  • 9. Log files suck. Stupid string data Unless using ETW / proper System.Diagnostics listener Typical log file has no “context” Typical log file has no correlation How to get data off our machines? How to report/analyze?
  • 10. Log files suck. Stupid string data Typical log file has no “context” Process/thread Machine name User performing the action Data specific to the event being logged Typical log file has no correlation How to get data off our machines? How to report/analyze?
  • 11. Log files suck. Stupid string data Typical log file has no “context” Typical log file has no correlation, e.g.: What was the server load when this was logged? How much memory was consumed at the time? Which web request invoked this action? How to get data off our machines? How to report/analyze?
  • 12. Log files suck. Stupid string data Typical log file has no “context” Typical log file has no correlation How to get data off our machines? How to report/analyze?
  • 13. Log files suck. Stupid string data Typical log file has no “context” Typical log file has no correlation How to get data off our machines? How to report/analyze? Log processing and analysis – log, store, process, query, proactive analysis Logstash, ElasticSearch, ... – cumbersome to setup
  • 14. What if we had (better) contextual information?
  • 15. Structured/Semantic logging “Of or relating to meaning, especially meaning in language” – the dictionary Windows Event Log Event Tracing for Windows (ETW) Semantic Logging Application Blog (PnP SLAB) Serilog …
  • 16. Event Source XML representation containing contextual information application-specific data elements Can be read by humans (just a message, like any log) Can be read by machines (XML, baby!) add “meaning” – we know what the data points can be allow automated aggregaton / trend analysis Fast! (not very developer friendly)
  • 18. Event Source Super awesome It has structured logging and context Can collect all kinds of data combined (e.g. HTTP + own event source) Hard to maintain Versioning Ceremony One method for every single thing we want to log (or at least an event id) Requires thinking about the events to log Lots of data
  • 19. Serilog - https://serilog.net/ Logging library like any other, mostly Sinks https://github.com/serilog/serilog/wiki/Provided-Sinks Console, file, event log Database ElasticSearch AppInsights Structured data as a first-class concept Enrichers, filters, …
  • 20. Structured data in Serilog Simple scalar values (bool, string, int, …) Collections and dictionaries Full objects Rendered by a sink to, for example, a string Can be stored by a sink to provide additional dimensions on data
  • 22. Enrichment of data Enrich log entries with additional context Machine name User name Process / thread information ASP.NET client IP / hostname ASP.NET user agent … Using Serilog enrichers (when applicable) In custom ASP.NET middleware
  • 23. But still... How to get the data off our machine(s)? How to analyze/aggregate/predict? Which customer had this Exception and why? Did our sales go up after we made this page faster? What button on the front-end crashes this microservice? Are # Exceptions increasing? Do I need special infrastructure to store and process logs?
  • 24. The cloud to the rescue? (Azure) Azure Insights Azure platform’s core monitoring and alerting services (standard Metrics, Alerts, Autoscale, Email Notifications and Audit Logs) Application Insights Application monitoring & logging for your applications (web, mobile, desktop, …) in cloud or other server Log Analytics (OMS) Log Analytics, part of the Operations Management Suite (OMS) ingests data from Azure servers and provides rich ‘global’ view & advanced log search based alerting capabilities
  • 26. Application Insights Azure Service + library/SDK Solves “where to store”, “how to ship”, “how to analyze” Enriches data Correlates data (e.g. client + server + dependency/DB/...) Telemetry! Allows structured logging Allows rich querying, alerting Works for many, many platforms and languages Web, Windows, Xamarin, any application type really .NET, Java, JavaScript, Objective-C, PHP, Python, Ruby, ...
  • 28. Do I have to run on Azure? No. REST API to send data to Various tools and SDK’s to collect data specific to language/platform Status Monitor on non-Azure machines
  • 29.
  • 30. Data collected Performance Counters Requests (both server/client side) Traces (more later) Exceptions Dependencies Custom Metrics & Events That is a lot of data that can be correlated!
  • 32. Application Insights portal Dashboard Basic metrics (# Alerts, Response times, Exceptions, Failed Requests, ...) Gateway to all of the below Application map Smart Detection / Alerts Live Metrics Stream Search, Availability, Failures, Performance, ... Metrics (and events) Analytics
  • 34. Smart Detection / Alerts DEMO
  • 37. Metrics and Events Send custom events and metrics Name of event/metric + a value Events – help find how the application is used and can be optimized User resized column in a grid User logged in User clicked “Share” No more stock ... Metrics – help alerting / finding missing functionality Purchase occurred Search returned zero results # Support calls ...
  • 39. So what about logging... Configure Serilog to use Application Insights as a sink Custom data becomes available (*) So we can correlate our own data with performance, requests, traces, exceptions, depenencies, ... But how to do this? And how to consume? (*) Make sure the entire object is in the log message, no tracking of non-printed data
  • 42. Application Insights Analytics Former MS-internal tool “Kusto” Near-realtime log ingestion and analysis Lets you run custom queries requests | where timestamp > ago(24h) | summarize count() by client_CountryOrRegion | top 10 by count_ | render piechart
  • 43. Queries over logs! Input dataset traces customEvents pageViews requests dependencies exceptions availabilityResults customMetrics performanceCounters browserTimings | Operators where count project join limit order | Renderer table chart (bar, pie, time, area, scatter) https://docs.microsoft.com/en-us/azure/application-insights/app-insights-analytics-reference
  • 45. Not covered in this talk VS plugin for Application Insights Run some analysis inside VS Continuous export Export data from Application Insights continuously if you want to analyze in other tools Retain data longer than 7 days Join with other data (preview) https://docs.microsoft.com/en-us/azure/application-insights/app-insights- analytics-using#import-data Join logging and metrics with other data sources, query with AI Analytics
  • 47. But... Why? Seriously, why are you in this session...
  • 48. But... Why? Troubleshooting – Did a problem occur? Where? Our code? The machine? Performance / Cost – Did we make the application slower? Improvement – Can a problem be detected or avoided? Trends – Need more storage? Need more servers? Customer Experience – Are people happy? Or seeing error after error? Business Decisions – Can we increase sales based on log data/telemetry?
  • 49. Structured logging Enrich technical logs with business data Allows correlating when things go wrong Allows making business decisions when using a good data analysis tool on top
  • 50. Application Insights Is that good analysis tool! (for web, mobile, desktop!) Solves Collecting various sources Ingesting various sources Enrichment with structured logging (e.g. Serilog) Helps analyzing all that data Default views, metrics, alerts, monitoring Custom querying – Application Insights Analytics Build, measure, improve! Both technical and business – there isn’t always a divide between both...
  • 51. Thank you! Need training, coaching, mentoring, performance analysis? Hire me! https://blog.maartenballiauw.be/hire-me.html http://blog.maartenballiauw.be @maartenballiauw

Editor's Notes

  1. https://pixabay.com/p-960806/
  2. Highlight which customer? Awesome, stack trace for Exception, “Debug is 0” great! RunEngine entry very useful. What if we want to aggregate “Req” / “Ret”?
  3. More structured/contextual data! Yay!
  4. Open Demo-EventSource Look at PerformBusinessLogic – some actions happening there, logging some items Look at CRMSystemEventSource In itself, pretty straightforward – attributes to inform the even tracing infrastructure about what something is and does One method per event we want to trace – easy to use BUT – how to version this? Do we really need to add a new method, decorate it with an attribute, ... For every single thing we want to log? And then invoke that method when needed? Cumbersome...
  5. Open Demo-Serilog Look at NuGet packages installed – Serilog and some sinks Browse NuGet for more Serilog packages and see millions Setting up Serilog – check Main method and explain what happens, show multiple sinks, show enrichers Show business logic again Still has log statements, no way around that Logging is now structured – we provide a message which can contain some info about the data we are logging But for example when catching the Exception, we can write a simple message and log the actual Exception with it Run the application, see colored console output, see a file has been created Explore the collapsed region: we can write to other sinks e.g. Windows Event Log Or attach an observable – explore the code and mention that even data that is not rendered by a sink is still present with the message, so if we could ship this to a system we could search/track/measure Exceptions, Customer objects, ... Based on property name
  6. https://pixabay.com/p-960806/
  7. Open portal.azure.com Show how to create Application Insights service Open service, click “Getting Started” Explain there are a few entry points to collect all data Client-side (JS) Server-side (or mobile) – different data points from inside app, from inside server, Exceptions, ... Custom metrics and events Open Demo_WebApp Show AppInsights package reference added, show Startup.cs where to configure (3 simple “Use...”) Show _Layout.cshtml, run app, show snippet Back to portal – data! (overview) Response times (collected from server and perf counters) Page view load time (correlated from client-side JS) # requests # failures Click server response time, see more detail, grouping, aggregation We can search data, correlate other data, ...
  8. https://pixabay.com/en/cat-animal-eyes-grey-view-views-351926/
  9. Open portal.azure.com MyGet AppInsights Application Map Show Availability tests test the server-side Show client-side uses server-side Click around, explain what we can see here
  10. Open portal.azure.com MyGet AppInsights Smart Detection We see nothing! Which is good. Smart Detection tries to be proactive in detecting things that are “not normal” Look at settings for list of examples Alerts (open Alerts) Add alert, see on which metrics we can alert Show examples, show web tests feeding into alerts, ...
  11. Open portal.azure.com MyGet AppInsights Live Metrics Stream Connects to our running application and inspects the data that comes in real-time Metrics, health, server CPU and memory, ... We can filter per server as well
  12. Open portal.azure.com MyGet AppInsights Browse through the blades Availability – explain web tests, explain these can be simple or actual tests created in VS that have multiple steps and checks Failures – we can see diagrams of exceptions vs. other metrics, as well as Exceptions Open Exception, see the occurences of this specific one See Exception message, stack trace, ... Analyze trends / root cause: what happened 5 minutes before this event? Performance – what is our slowest page? Drill into details, find out why Also lets us improve then see if the page drops in the ranking, continuous improvement Servers – Some server details, quickly go through this Browser – Browser stats, verify client-side code, page render times, find slowest rendering pages, ... Usage – Correlate technical side with user-side Sessions, users, custom events (more later), ...
  13. Open portal.azure.com MyGet AppInsights In Usage tab, note these “feature names” Custom events! We track feature usage there to see if features are actually being used or not Switch to Demo_WebApp In HomeController -> About, show the way to track metrics In Contact.cshtml, show how to track metrics/events client-side as well Switch to Demo AppInsights, show the portal’s Metrics tab, show how we can filter/correlate
  14. Open Demo_WebApp Show the HomeController – still same code as before, this time using _logger from ASP.NET Core Mention using full object data to make sure the object is streamed to AppInsights later on Startup.cs Show Serilog configuration – log to AppInsights Show how it ties into ASP.NET Core logging infrastructure Run the application, make a few requests to /Home/About, /Home/Contact, /Home/About?throw, ... Switch to Demo appinsights, use Search Lots of data – verbosity should probably be tuned... Search for “invoice” – yay custom log data! Expand one of them, click “All available telemetry” -> correlates all traces we had for this specific one, correlating our trace with request, exception, ... Expand the Exception, see that we even have customer id and customer email with it – super useful to have it all tied together
  15. https://pixabay.com/p-960806/
  16. Open portal.azure.com Demo AppInsights Show some of the sample queries, show the editor, show the various rendering types (auto complete in there) Run a few sample queries and find custom dimensions All traces for our About page traces | where operation_Name == "GET /Home/About" | order by timestamp desc | limit 500 Number of invoices per customer (customMetrics!) customMetrics | extend customerId = tostring(customDimensions.['Customer.Id']) | project customerId, value | summarize avg(value) by customerId | render piechart MyGet AppInsights All requests from yesterday that have an Exception requests | where timestamp > ago(1d) | where success == "False" | join kind = leftouter ( exceptions | where timestamp > ago(1d) ) on operation_Id | summarize exceptionCount = count() by operation_Name, outerMessage | order by exceptionCount asc Are we being DDoS-ed by one specific client? requests | summarize count() by client_IP | top 10 by count_ | render piechart Explain concept of package sources – at one point we DDoS-ed ourselves (infinite loop) Found using AI Analytics: