SlideShare a Scribd company logo
Kusto
Azure Data Explorer
For Taboola LA R&D
Monitoring in Production
Maher Odeh (Taboola Production IT), Adi Eldar (Microsoft), Tal Bar Zvi (Taboola R&D) 1
https://youtu.be/iWay1PeoGhg
Click here to watch
the recorded session
Maher Odeh, Taboola
Production IT
2
Adi Eldar, Microsoft
Principal Data Scientist
Tal Bar Zvi, Taboola
R&D, User Data
3
4
Applause
Gift Economy
5
Goals of This Training
6
1. Kusto Queries
1. Dashboards
1. Alerts
1. Bonus: Data Science
Let’s begin
7
Kusto is...
8
A new way to look at data / logs
What is it actually?What is it actually?
It’s a new, innovative thing
Developed by Microsoft
We are one of the first to use it
It helps us to get the picture of our service in a few
seconds
What is it actually?
It’s a new, innovative thing
Developed by Microsoft
We are one of the first to use it
It helps us to get the picture of our service in a few
seconds
Wow, sounds cool
True Story
9
Now Really Let’s begin
10
✓ Big Data
✓ Database
✓ Tables
✓ Functions
✓ Scripting
✓ Join
✓ Union
✓ Fast Search
✓ Graphs
✓ Dashboards
✓ Alerts
✓ HTTP Logs (for now)
✓ Notebooks
✓ Python
Why Kusto?
● Kibana-Fastly replacement
● It has a WOW effect
● It is easy to use and learn
11
It is new, for all,
we learn it together.
(This is Rare!)
12
Different payment model.
Kusto is already paid - flat.
Queries do not* cost extra money.
*prod-it are gonna hate me after this slide
✓ Credits: Shaked Zychlinski
What’s in?
13
Which Data / Logs Are In Kusto?
14
15
Request
URL
Referrer
HTTP Status
Response Time
+
DC
Server IP
more...
16
Javascript files (*.js)
(loader.js, impl, newsroom, userx...)
Image files (*.jpg, *.png...)
Events (available, visible, click, social,
debug, performance…)
Etc.
Architecture
17
Log files - from Fastly (CDN)
Kusto
Web interface
● Query
● Graphs
● Dashboards
API
● Alerts (Sensu)
● Scripting
● Jupyter
● Programming
What is a CDN (Fastly & Akamai for example)?
18
50 server farms
7 Data Centers
Caching our HTTP responses
HTTP Logs
CDN = Content Delivery Network
Kusto Database Sizes (as of Jan 2019)
19
Database Size RETENTION
COLD / HOT (CACHED)
fastly-
backstage
15 GB 60 days (31 days 🔥)
fastly-c3 10 TB 30 days (3 days 🔥)
fastly-trc 250 TB 30 days (3 days 🔥)
SLIDE | 20
Take Away
Messages No. 1
20
1. Kusto is BigData database
1. It holds our HTTP requests
1. Hot vs. Cold
Query
21
22
Tabs
Select:
Cluster & Database
Docs Settings
Output
Query
Tabs, Statistics, Info
Column
Selection
Pivoting
Deep link sharingExport ImportRun Recall output
Documentation
Click on Query Language
23
Query - KQL
24
● Query = statement ; statement ; ….. ; statement
● At least one statement is a tabular expression
● Returns result back
source |
operator1 |
[ | operator2 ]
[ | render ]
(Taboolar?!)
KQL vs. SQL
25
Example No. 1 of 7
26
● trc_access | count
Hot vs. Cold...
Example No. 1 of 7 - corrected
27
● trc_access | where timestamp > ago(1d) | count
Example No. 2 of 7 - by publisher
28
trc_access |
where timestamp > ago(1d) |
where publisher_name == ‘msn-msn’ |
count
Example No. 3 of 7 - take (like “limit”)
29
trc_access |
where timestamp > ago(5m) |
where publisher_name == ‘msn-msn’ |
take 5Geo Referrer Time Action URL
Example No. 4 of 7 - summarize & top
30
trc_access | where timestamp > ago(1h) |
summarize count() by geo_country_code |
top 5 by count_ desc;
trc_access | where timestamp > ago(1h) |
summarize count() by action |
top 5 by count_ desc
; Semicolon
Example No. 5 of 7 - render
31
trc_access | where timestamp > ago(1h) |
summarize count() by geo_country_code |
top 5 by count_ desc; | render piechart
WOW
Alert:
32
Example No. 6 of 7 - timechart
33
trc_access | where timestamp > ago(10d) |
summarize count() by bin(timestamp, 30m) |
render timechart
Example No. 7 of 7 - extract & extend
34
Complex Example
True Story from Last Week
35
Exmple - HTTP errors, where? what?
36
● Step 1 - See HTTP error increased
● Step 2 - Summarize by data center
● Step 3 - Summarize by action
● Step 4 - Union with normal traffic
HTTP Error Spike
Step 1
NJ & CH are
suffering
Step 2Step 2`Step 3
Found the actionsUnion
Project
Alias
Low
errors
Normal
Traffic
Both Normal
and Errors rise
Errors
Gone
SLIDE | 37
Take Away
Messages No. 2
37
1. Kusto has fast query capacities
1. It can create graphs
1. Can aggregate and create fields on-the-fly
1. Helps in:
a. Find root cause
b. Traffic sampling
c. Insights & trends
d. Integration validations
Brain Notebooks
38
Want more use cases? Use Brain. Sharing is Caring.
39
Team’s
wisdom
Your personal
wisdom
Here
some
40
Click to run on Kusto
(deep link)
Calculates response
time percentiles
Credits:
Taboola News
Some
more
41
1. action == ‘json’
2. unkown pub
3. extend data (add column)
4. url_decode(%20 - out)
5. parse_json
6. extend (again)
7. project
8. summarize by pub, json field
9. top 30 by count
Credits:
Taboola Mobile
Kusto Community
42
#kusto @kusto-mentors
43
SLIDE | 44
Take Away
Messages No. 3
44
1. Use Slack and Brain to share
1. Document your usage for others to learn
Dashboards
Lens Explorer
46
47
48
Lens Explorer - Rich Data Visualisations
Alerts
50
Kusto Sensu Integration
Elastic based check
Same check w/ Kusto
51
Alerts (using Sensu)
Period &
Threshold
Kusto
Query
Data Science
&
Notebooks
Jupyter Notebooks - Kqlmagic (Azure & Locally)
53
Kqlmagic Connect
Run queries
Output saved
Standardized
Use make-series (it’s fast)
To see the HTTP error
spike
Remember the example from 15 min. ago?
54
Use autocluster to find
similar error characteristics
DC is CH
Newsroom
affected
This is
the host
Using diffpatterns to find
clues
DC is CH Newsroom
affected
This is
the host
Kusto + Grafana
56
Summary
57
1. You know where to find me (tal.b@taboola.com)
1. You know you have accessible Resources
(Brain, WWW, Pluralsight free course, Videos, #kusto, Microsoft)
1. You saw how easy it is to run Kusto queries
1. You saw that there are Dashboards & Alerts
1. You are aware of the existence of built-in Data Science power
@kusto-mentors
58
Thank You
59
FAQ
60
1. Does it cost money? It is prepaid
2. What about Kibana, Grafana, BQ? Here to stay for now
3. What about applicative logs / my data? Currently Fastly logs
4. Will my elastic-fastly alerts be converted to Kusto for me? No
5. When will the other fastly logs be available? Updates in slack #kusto
6. Can we have more Kusto trainings? Dashboard? Workshops? Yes
7. Does Kusto support distinct count? Yes
8. Does Kusto have materialized views? Yes
9. Can we add to the schema our common recommendation fields? Yes
10. What about API 2.0 HTTP POST payload? It is in discussions
11. Can I look in all fields like in Kibana? Yes
12. Do all have access? Many have, or else ticket to prod-it
13. Can I use the alerts? Work in progress
14. Can I automatically derive smaller tables? Yes
Demo Time!
61
Pics - Atmosphere
62

More Related Content

What's hot

Architect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh ArchitectureArchitect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh Architecture
Databricks
 
Elk
Elk Elk
Understanding Query Plans and Spark UIs
Understanding Query Plans and Spark UIsUnderstanding Query Plans and Spark UIs
Understanding Query Plans and Spark UIs
Databricks
 
Azure Synapse Analytics Overview (r2)
Azure Synapse Analytics Overview (r2)Azure Synapse Analytics Overview (r2)
Azure Synapse Analytics Overview (r2)
James Serra
 
Apache Spark on K8S Best Practice and Performance in the Cloud
Apache Spark on K8S Best Practice and Performance in the CloudApache Spark on K8S Best Practice and Performance in the Cloud
Apache Spark on K8S Best Practice and Performance in the Cloud
Databricks
 
Data Observability Best Pracices
Data Observability Best PracicesData Observability Best Pracices
Data Observability Best Pracices
Andy Petrella
 
Data In Motion Paris 2023
Data In Motion Paris 2023Data In Motion Paris 2023
Data In Motion Paris 2023
confluent
 
Apache Iceberg: An Architectural Look Under the Covers
Apache Iceberg: An Architectural Look Under the CoversApache Iceberg: An Architectural Look Under the Covers
Apache Iceberg: An Architectural Look Under the Covers
ScyllaDB
 
Real-time Analytics with Trino and Apache Pinot
Real-time Analytics with Trino and Apache PinotReal-time Analytics with Trino and Apache Pinot
Real-time Analytics with Trino and Apache Pinot
Xiang Fu
 
OLTP+OLAP=HTAP
 OLTP+OLAP=HTAP OLTP+OLAP=HTAP
OLTP+OLAP=HTAP
EDB
 
Log analysis with elastic stack
Log analysis with elastic stackLog analysis with elastic stack
Log analysis with elastic stack
Bangladesh Network Operators Group
 
Elastic SIEM (Endpoint Security)
Elastic SIEM (Endpoint Security)Elastic SIEM (Endpoint Security)
Elastic SIEM (Endpoint Security)
Kangaroot
 
Big data architectures and the data lake
Big data architectures and the data lakeBig data architectures and the data lake
Big data architectures and the data lake
James Serra
 
New Directions in pySpark for Time Series Analysis: Spark Summit East talk by...
New Directions in pySpark for Time Series Analysis: Spark Summit East talk by...New Directions in pySpark for Time Series Analysis: Spark Summit East talk by...
New Directions in pySpark for Time Series Analysis: Spark Summit East talk by...
Spark Summit
 
Splunk Architecture
Splunk ArchitectureSplunk Architecture
Splunk Architecture
Kishore Chaganti
 
Introduction to influx db
Introduction to influx dbIntroduction to influx db
Introduction to influx db
Roberto Gaudenzi
 
Standing on the Shoulders of Open-Source Giants: The Serverless Realtime Lake...
Standing on the Shoulders of Open-Source Giants: The Serverless Realtime Lake...Standing on the Shoulders of Open-Source Giants: The Serverless Realtime Lake...
Standing on the Shoulders of Open-Source Giants: The Serverless Realtime Lake...
HostedbyConfluent
 
Building Robust ETL Pipelines with Apache Spark
Building Robust ETL Pipelines with Apache SparkBuilding Robust ETL Pipelines with Apache Spark
Building Robust ETL Pipelines with Apache Spark
Databricks
 
Log analysis using elk
Log analysis using elkLog analysis using elk
Log analysis using elk
Rushika Shah
 
Some Iceberg Basics for Beginners (CDP).pdf
Some Iceberg Basics for Beginners (CDP).pdfSome Iceberg Basics for Beginners (CDP).pdf
Some Iceberg Basics for Beginners (CDP).pdf
Michael Kogan
 

What's hot (20)

Architect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh ArchitectureArchitect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh Architecture
 
Elk
Elk Elk
Elk
 
Understanding Query Plans and Spark UIs
Understanding Query Plans and Spark UIsUnderstanding Query Plans and Spark UIs
Understanding Query Plans and Spark UIs
 
Azure Synapse Analytics Overview (r2)
Azure Synapse Analytics Overview (r2)Azure Synapse Analytics Overview (r2)
Azure Synapse Analytics Overview (r2)
 
Apache Spark on K8S Best Practice and Performance in the Cloud
Apache Spark on K8S Best Practice and Performance in the CloudApache Spark on K8S Best Practice and Performance in the Cloud
Apache Spark on K8S Best Practice and Performance in the Cloud
 
Data Observability Best Pracices
Data Observability Best PracicesData Observability Best Pracices
Data Observability Best Pracices
 
Data In Motion Paris 2023
Data In Motion Paris 2023Data In Motion Paris 2023
Data In Motion Paris 2023
 
Apache Iceberg: An Architectural Look Under the Covers
Apache Iceberg: An Architectural Look Under the CoversApache Iceberg: An Architectural Look Under the Covers
Apache Iceberg: An Architectural Look Under the Covers
 
Real-time Analytics with Trino and Apache Pinot
Real-time Analytics with Trino and Apache PinotReal-time Analytics with Trino and Apache Pinot
Real-time Analytics with Trino and Apache Pinot
 
OLTP+OLAP=HTAP
 OLTP+OLAP=HTAP OLTP+OLAP=HTAP
OLTP+OLAP=HTAP
 
Log analysis with elastic stack
Log analysis with elastic stackLog analysis with elastic stack
Log analysis with elastic stack
 
Elastic SIEM (Endpoint Security)
Elastic SIEM (Endpoint Security)Elastic SIEM (Endpoint Security)
Elastic SIEM (Endpoint Security)
 
Big data architectures and the data lake
Big data architectures and the data lakeBig data architectures and the data lake
Big data architectures and the data lake
 
New Directions in pySpark for Time Series Analysis: Spark Summit East talk by...
New Directions in pySpark for Time Series Analysis: Spark Summit East talk by...New Directions in pySpark for Time Series Analysis: Spark Summit East talk by...
New Directions in pySpark for Time Series Analysis: Spark Summit East talk by...
 
Splunk Architecture
Splunk ArchitectureSplunk Architecture
Splunk Architecture
 
Introduction to influx db
Introduction to influx dbIntroduction to influx db
Introduction to influx db
 
Standing on the Shoulders of Open-Source Giants: The Serverless Realtime Lake...
Standing on the Shoulders of Open-Source Giants: The Serverless Realtime Lake...Standing on the Shoulders of Open-Source Giants: The Serverless Realtime Lake...
Standing on the Shoulders of Open-Source Giants: The Serverless Realtime Lake...
 
Building Robust ETL Pipelines with Apache Spark
Building Robust ETL Pipelines with Apache SparkBuilding Robust ETL Pipelines with Apache Spark
Building Robust ETL Pipelines with Apache Spark
 
Log analysis using elk
Log analysis using elkLog analysis using elk
Log analysis using elk
 
Some Iceberg Basics for Beginners (CDP).pdf
Some Iceberg Basics for Beginners (CDP).pdfSome Iceberg Basics for Beginners (CDP).pdf
Some Iceberg Basics for Beginners (CDP).pdf
 

Similar to Kusto (Azure Data Explorer) Training for R&D - January 2019

Interconnection Automation For All - Extended - MPS 2023
Interconnection Automation For All - Extended - MPS 2023Interconnection Automation For All - Extended - MPS 2023
Interconnection Automation For All - Extended - MPS 2023
Chris Grundemann
 
Monitoring Big Data Systems Done "The Simple Way" - Demi Ben-Ari - Codemotion...
Monitoring Big Data Systems Done "The Simple Way" - Demi Ben-Ari - Codemotion...Monitoring Big Data Systems Done "The Simple Way" - Demi Ben-Ari - Codemotion...
Monitoring Big Data Systems Done "The Simple Way" - Demi Ben-Ari - Codemotion...
Codemotion
 
Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...
Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...
Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...
Demi Ben-Ari
 
Real Time Processing Using Twitter Heron by Karthik Ramasamy
Real Time Processing Using Twitter Heron by Karthik RamasamyReal Time Processing Using Twitter Heron by Karthik Ramasamy
Real Time Processing Using Twitter Heron by Karthik Ramasamy
Data Con LA
 
Monitoring your Python with Prometheus (Python Ireland April 2015)
Monitoring your Python with Prometheus (Python Ireland April 2015)Monitoring your Python with Prometheus (Python Ireland April 2015)
Monitoring your Python with Prometheus (Python Ireland April 2015)
Brian Brazil
 
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017
Demi Ben-Ari
 
NTTドコモ様 導入事例 OpenStack Summit 2015 Tokyo 講演「After One year of OpenStack Cloud...
NTTドコモ様 導入事例 OpenStack Summit 2015 Tokyo 講演「After One year of OpenStack Cloud...NTTドコモ様 導入事例 OpenStack Summit 2015 Tokyo 講演「After One year of OpenStack Cloud...
NTTドコモ様 導入事例 OpenStack Summit 2015 Tokyo 講演「After One year of OpenStack Cloud...
VirtualTech Japan Inc.
 
Monitoring Big Data Systems - "The Simple Way"
Monitoring Big Data Systems - "The Simple Way"Monitoring Big Data Systems - "The Simple Way"
Monitoring Big Data Systems - "The Simple Way"
Demi Ben-Ari
 
Cloud Dataflow - A Unified Model for Batch and Streaming Data Processing
Cloud Dataflow - A Unified Model for Batch and Streaming Data ProcessingCloud Dataflow - A Unified Model for Batch and Streaming Data Processing
Cloud Dataflow - A Unified Model for Batch and Streaming Data Processing
DoiT International
 
Apache Beam (incubating)
Apache Beam (incubating)Apache Beam (incubating)
Apache Beam (incubating)
Apache Apex
 
Application Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouseApplication Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouse
VictoriaMetrics
 
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...
Altinity Ltd
 
Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Austin Ogilvie
 
Lessons learned from designing QA automation event streaming platform(IoT big...
Lessons learned from designing QA automation event streaming platform(IoT big...Lessons learned from designing QA automation event streaming platform(IoT big...
Lessons learned from designing QA automation event streaming platform(IoT big...
Omid Vahdaty
 
MCSoC'13 Keynote Talk "Taming Big Data Streams"
MCSoC'13 Keynote Talk "Taming Big Data Streams"MCSoC'13 Keynote Talk "Taming Big Data Streams"
MCSoC'13 Keynote Talk "Taming Big Data Streams"Hideyuki Kawashima
 
Distributed real time stream processing- why and how
Distributed real time stream processing- why and howDistributed real time stream processing- why and how
Distributed real time stream processing- why and how
Petr Zapletal
 
Microsoft Dryad
Microsoft DryadMicrosoft Dryad
Microsoft Dryad
Colin Clark
 
Why And When Should We Consider Stream Processing In Our Solutions Teqnation ...
Why And When Should We Consider Stream Processing In Our Solutions Teqnation ...Why And When Should We Consider Stream Processing In Our Solutions Teqnation ...
Why And When Should We Consider Stream Processing In Our Solutions Teqnation ...
Soroosh Khodami
 
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...
Demi Ben-Ari
 
Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion...
Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion...Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion...
Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion...
Codemotion
 

Similar to Kusto (Azure Data Explorer) Training for R&D - January 2019 (20)

Interconnection Automation For All - Extended - MPS 2023
Interconnection Automation For All - Extended - MPS 2023Interconnection Automation For All - Extended - MPS 2023
Interconnection Automation For All - Extended - MPS 2023
 
Monitoring Big Data Systems Done "The Simple Way" - Demi Ben-Ari - Codemotion...
Monitoring Big Data Systems Done "The Simple Way" - Demi Ben-Ari - Codemotion...Monitoring Big Data Systems Done "The Simple Way" - Demi Ben-Ari - Codemotion...
Monitoring Big Data Systems Done "The Simple Way" - Demi Ben-Ari - Codemotion...
 
Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...
Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...
Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...
 
Real Time Processing Using Twitter Heron by Karthik Ramasamy
Real Time Processing Using Twitter Heron by Karthik RamasamyReal Time Processing Using Twitter Heron by Karthik Ramasamy
Real Time Processing Using Twitter Heron by Karthik Ramasamy
 
Monitoring your Python with Prometheus (Python Ireland April 2015)
Monitoring your Python with Prometheus (Python Ireland April 2015)Monitoring your Python with Prometheus (Python Ireland April 2015)
Monitoring your Python with Prometheus (Python Ireland April 2015)
 
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017
 
NTTドコモ様 導入事例 OpenStack Summit 2015 Tokyo 講演「After One year of OpenStack Cloud...
NTTドコモ様 導入事例 OpenStack Summit 2015 Tokyo 講演「After One year of OpenStack Cloud...NTTドコモ様 導入事例 OpenStack Summit 2015 Tokyo 講演「After One year of OpenStack Cloud...
NTTドコモ様 導入事例 OpenStack Summit 2015 Tokyo 講演「After One year of OpenStack Cloud...
 
Monitoring Big Data Systems - "The Simple Way"
Monitoring Big Data Systems - "The Simple Way"Monitoring Big Data Systems - "The Simple Way"
Monitoring Big Data Systems - "The Simple Way"
 
Cloud Dataflow - A Unified Model for Batch and Streaming Data Processing
Cloud Dataflow - A Unified Model for Batch and Streaming Data ProcessingCloud Dataflow - A Unified Model for Batch and Streaming Data Processing
Cloud Dataflow - A Unified Model for Batch and Streaming Data Processing
 
Apache Beam (incubating)
Apache Beam (incubating)Apache Beam (incubating)
Apache Beam (incubating)
 
Application Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouseApplication Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouse
 
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...
 
Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
 
Lessons learned from designing QA automation event streaming platform(IoT big...
Lessons learned from designing QA automation event streaming platform(IoT big...Lessons learned from designing QA automation event streaming platform(IoT big...
Lessons learned from designing QA automation event streaming platform(IoT big...
 
MCSoC'13 Keynote Talk "Taming Big Data Streams"
MCSoC'13 Keynote Talk "Taming Big Data Streams"MCSoC'13 Keynote Talk "Taming Big Data Streams"
MCSoC'13 Keynote Talk "Taming Big Data Streams"
 
Distributed real time stream processing- why and how
Distributed real time stream processing- why and howDistributed real time stream processing- why and how
Distributed real time stream processing- why and how
 
Microsoft Dryad
Microsoft DryadMicrosoft Dryad
Microsoft Dryad
 
Why And When Should We Consider Stream Processing In Our Solutions Teqnation ...
Why And When Should We Consider Stream Processing In Our Solutions Teqnation ...Why And When Should We Consider Stream Processing In Our Solutions Teqnation ...
Why And When Should We Consider Stream Processing In Our Solutions Teqnation ...
 
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...
 
Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion...
Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion...Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion...
Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion...
 

Recently uploaded

Enterprise Resource Planning System in Telangana
Enterprise Resource Planning System in TelanganaEnterprise Resource Planning System in Telangana
Enterprise Resource Planning System in Telangana
NYGGS Automation Suite
 
top nidhi software solution freedownload
top nidhi software solution freedownloadtop nidhi software solution freedownload
top nidhi software solution freedownload
vrstrong314
 
Cyaniclab : Software Development Agency Portfolio.pdf
Cyaniclab : Software Development Agency Portfolio.pdfCyaniclab : Software Development Agency Portfolio.pdf
Cyaniclab : Software Development Agency Portfolio.pdf
Cyanic lab
 
A Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of PassageA Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of Passage
Philip Schwarz
 
2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx
Georgi Kodinov
 
Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024
Globus
 
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptx
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptxTop Features to Include in Your Winzo Clone App for Business Growth (4).pptx
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptx
rickgrimesss22
 
Globus Connect Server Deep Dive - GlobusWorld 2024
Globus Connect Server Deep Dive - GlobusWorld 2024Globus Connect Server Deep Dive - GlobusWorld 2024
Globus Connect Server Deep Dive - GlobusWorld 2024
Globus
 
Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"
Donna Lenk
 
Enhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdfEnhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdf
Globus
 
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamOpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
takuyayamamoto1800
 
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Mind IT Systems
 
Using IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New ZealandUsing IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New Zealand
IES VE
 
Into the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdfInto the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdf
Ortus Solutions, Corp
 
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
informapgpstrackings
 
How to Position Your Globus Data Portal for Success Ten Good Practices
How to Position Your Globus Data Portal for Success Ten Good PracticesHow to Position Your Globus Data Portal for Success Ten Good Practices
How to Position Your Globus Data Portal for Success Ten Good Practices
Globus
 
Prosigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology Solutions
Prosigns
 
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
Juraj Vysvader
 
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdfDominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
AMB-Review
 
Graphic Design Crash Course for beginners
Graphic Design Crash Course for beginnersGraphic Design Crash Course for beginners
Graphic Design Crash Course for beginners
e20449
 

Recently uploaded (20)

Enterprise Resource Planning System in Telangana
Enterprise Resource Planning System in TelanganaEnterprise Resource Planning System in Telangana
Enterprise Resource Planning System in Telangana
 
top nidhi software solution freedownload
top nidhi software solution freedownloadtop nidhi software solution freedownload
top nidhi software solution freedownload
 
Cyaniclab : Software Development Agency Portfolio.pdf
Cyaniclab : Software Development Agency Portfolio.pdfCyaniclab : Software Development Agency Portfolio.pdf
Cyaniclab : Software Development Agency Portfolio.pdf
 
A Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of PassageA Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of Passage
 
2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx
 
Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024
 
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptx
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptxTop Features to Include in Your Winzo Clone App for Business Growth (4).pptx
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptx
 
Globus Connect Server Deep Dive - GlobusWorld 2024
Globus Connect Server Deep Dive - GlobusWorld 2024Globus Connect Server Deep Dive - GlobusWorld 2024
Globus Connect Server Deep Dive - GlobusWorld 2024
 
Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"
 
Enhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdfEnhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdf
 
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamOpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
 
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
 
Using IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New ZealandUsing IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New Zealand
 
Into the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdfInto the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdf
 
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
 
How to Position Your Globus Data Portal for Success Ten Good Practices
How to Position Your Globus Data Portal for Success Ten Good PracticesHow to Position Your Globus Data Portal for Success Ten Good Practices
How to Position Your Globus Data Portal for Success Ten Good Practices
 
Prosigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology Solutions
 
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
 
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdfDominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
 
Graphic Design Crash Course for beginners
Graphic Design Crash Course for beginnersGraphic Design Crash Course for beginners
Graphic Design Crash Course for beginners
 

Kusto (Azure Data Explorer) Training for R&D - January 2019

  • 1. Kusto Azure Data Explorer For Taboola LA R&D Monitoring in Production Maher Odeh (Taboola Production IT), Adi Eldar (Microsoft), Tal Bar Zvi (Taboola R&D) 1 https://youtu.be/iWay1PeoGhg Click here to watch the recorded session
  • 2. Maher Odeh, Taboola Production IT 2 Adi Eldar, Microsoft Principal Data Scientist Tal Bar Zvi, Taboola R&D, User Data
  • 3. 3
  • 6. Goals of This Training 6 1. Kusto Queries 1. Dashboards 1. Alerts 1. Bonus: Data Science
  • 8. Kusto is... 8 A new way to look at data / logs What is it actually?What is it actually? It’s a new, innovative thing Developed by Microsoft We are one of the first to use it It helps us to get the picture of our service in a few seconds What is it actually? It’s a new, innovative thing Developed by Microsoft We are one of the first to use it It helps us to get the picture of our service in a few seconds Wow, sounds cool
  • 10. Now Really Let’s begin 10 ✓ Big Data ✓ Database ✓ Tables ✓ Functions ✓ Scripting ✓ Join ✓ Union ✓ Fast Search ✓ Graphs ✓ Dashboards ✓ Alerts ✓ HTTP Logs (for now) ✓ Notebooks ✓ Python
  • 11. Why Kusto? ● Kibana-Fastly replacement ● It has a WOW effect ● It is easy to use and learn 11 It is new, for all, we learn it together. (This is Rare!)
  • 12. 12 Different payment model. Kusto is already paid - flat. Queries do not* cost extra money. *prod-it are gonna hate me after this slide ✓ Credits: Shaked Zychlinski
  • 14. Which Data / Logs Are In Kusto? 14
  • 16. 16 Javascript files (*.js) (loader.js, impl, newsroom, userx...) Image files (*.jpg, *.png...) Events (available, visible, click, social, debug, performance…) Etc.
  • 17. Architecture 17 Log files - from Fastly (CDN) Kusto Web interface ● Query ● Graphs ● Dashboards API ● Alerts (Sensu) ● Scripting ● Jupyter ● Programming
  • 18. What is a CDN (Fastly & Akamai for example)? 18 50 server farms 7 Data Centers Caching our HTTP responses HTTP Logs CDN = Content Delivery Network
  • 19. Kusto Database Sizes (as of Jan 2019) 19 Database Size RETENTION COLD / HOT (CACHED) fastly- backstage 15 GB 60 days (31 days 🔥) fastly-c3 10 TB 30 days (3 days 🔥) fastly-trc 250 TB 30 days (3 days 🔥)
  • 20. SLIDE | 20 Take Away Messages No. 1 20 1. Kusto is BigData database 1. It holds our HTTP requests 1. Hot vs. Cold
  • 22. 22 Tabs Select: Cluster & Database Docs Settings Output Query Tabs, Statistics, Info Column Selection Pivoting Deep link sharingExport ImportRun Recall output Documentation
  • 23. Click on Query Language 23
  • 24. Query - KQL 24 ● Query = statement ; statement ; ….. ; statement ● At least one statement is a tabular expression ● Returns result back source | operator1 | [ | operator2 ] [ | render ] (Taboolar?!)
  • 26. Example No. 1 of 7 26 ● trc_access | count Hot vs. Cold...
  • 27. Example No. 1 of 7 - corrected 27 ● trc_access | where timestamp > ago(1d) | count
  • 28. Example No. 2 of 7 - by publisher 28 trc_access | where timestamp > ago(1d) | where publisher_name == ‘msn-msn’ | count
  • 29. Example No. 3 of 7 - take (like “limit”) 29 trc_access | where timestamp > ago(5m) | where publisher_name == ‘msn-msn’ | take 5Geo Referrer Time Action URL
  • 30. Example No. 4 of 7 - summarize & top 30 trc_access | where timestamp > ago(1h) | summarize count() by geo_country_code | top 5 by count_ desc; trc_access | where timestamp > ago(1h) | summarize count() by action | top 5 by count_ desc ; Semicolon
  • 31. Example No. 5 of 7 - render 31 trc_access | where timestamp > ago(1h) | summarize count() by geo_country_code | top 5 by count_ desc; | render piechart WOW
  • 33. Example No. 6 of 7 - timechart 33 trc_access | where timestamp > ago(10d) | summarize count() by bin(timestamp, 30m) | render timechart
  • 34. Example No. 7 of 7 - extract & extend 34
  • 35. Complex Example True Story from Last Week 35
  • 36. Exmple - HTTP errors, where? what? 36 ● Step 1 - See HTTP error increased ● Step 2 - Summarize by data center ● Step 3 - Summarize by action ● Step 4 - Union with normal traffic HTTP Error Spike Step 1 NJ & CH are suffering Step 2Step 2`Step 3 Found the actionsUnion Project Alias Low errors Normal Traffic Both Normal and Errors rise Errors Gone
  • 37. SLIDE | 37 Take Away Messages No. 2 37 1. Kusto has fast query capacities 1. It can create graphs 1. Can aggregate and create fields on-the-fly 1. Helps in: a. Find root cause b. Traffic sampling c. Insights & trends d. Integration validations
  • 39. Want more use cases? Use Brain. Sharing is Caring. 39 Team’s wisdom Your personal wisdom
  • 40. Here some 40 Click to run on Kusto (deep link) Calculates response time percentiles Credits: Taboola News
  • 41. Some more 41 1. action == ‘json’ 2. unkown pub 3. extend data (add column) 4. url_decode(%20 - out) 5. parse_json 6. extend (again) 7. project 8. summarize by pub, json field 9. top 30 by count Credits: Taboola Mobile
  • 44. SLIDE | 44 Take Away Messages No. 3 44 1. Use Slack and Brain to share 1. Document your usage for others to learn
  • 46. 46
  • 47. 47
  • 48. 48 Lens Explorer - Rich Data Visualisations
  • 50. 50 Kusto Sensu Integration Elastic based check Same check w/ Kusto
  • 51. 51 Alerts (using Sensu) Period & Threshold Kusto Query
  • 53. Jupyter Notebooks - Kqlmagic (Azure & Locally) 53 Kqlmagic Connect Run queries Output saved Standardized
  • 54. Use make-series (it’s fast) To see the HTTP error spike Remember the example from 15 min. ago? 54 Use autocluster to find similar error characteristics DC is CH Newsroom affected This is the host Using diffpatterns to find clues DC is CH Newsroom affected This is the host
  • 56. 56
  • 57. Summary 57 1. You know where to find me (tal.b@taboola.com) 1. You know you have accessible Resources (Brain, WWW, Pluralsight free course, Videos, #kusto, Microsoft) 1. You saw how easy it is to run Kusto queries 1. You saw that there are Dashboards & Alerts 1. You are aware of the existence of built-in Data Science power
  • 60. FAQ 60 1. Does it cost money? It is prepaid 2. What about Kibana, Grafana, BQ? Here to stay for now 3. What about applicative logs / my data? Currently Fastly logs 4. Will my elastic-fastly alerts be converted to Kusto for me? No 5. When will the other fastly logs be available? Updates in slack #kusto 6. Can we have more Kusto trainings? Dashboard? Workshops? Yes 7. Does Kusto support distinct count? Yes 8. Does Kusto have materialized views? Yes 9. Can we add to the schema our common recommendation fields? Yes 10. What about API 2.0 HTTP POST payload? It is in discussions 11. Can I look in all fields like in Kibana? Yes 12. Do all have access? Many have, or else ticket to prod-it 13. Can I use the alerts? Work in progress 14. Can I automatically derive smaller tables? Yes

Editor's Notes

  1. Myself: 2 year in Taboola, Crazy about Taboola. User Data R&D Been developing software since I was 13 Maher from production id is the go-to person who set all this magic
  2. Recently added this in my Linkedin
  3. Evident you really care We are in it together
  4. Will help anyone that have something big to show
  5. EXPECTATIONS Who is the audience? DIVERSE AUDIENCE BIG This is the PART OF IT WILL BE TOO FAST AND PART TOO SLOW This is the VALUE
  6. You know other tools
  7. Simple Hebrew
  8. Insights were never so accessible - you will find stuff, you never had the ability to find. it is very simple. This tool enables you to investigate Taboola’s crazy traffic with ease.
  9. Killer combination
  10. Who is using Kusto? Who plans to use it? Questions in the end, this is a big crowd, use #Slack
  11. This is המחשה - סבלנות To make sure we are on the same page
  12. Request URL very useful
  13. This is production IT’s great work
  14. This technical gap that we need to close Thanks production IT!
  15. Use error for fast insight
  16. 1,000,000$ Question Are you happy about it?
  17. Take your time
  18. Fluent (like Java streams)
  19. For the SQL lovers Another language, a good one
  20. Learning by Example Ask people what is wrong here? Use hot
  21. New stuff in yellow
  22. Now ago(5m)
  23. SQL Group by == summarize Aggregation functions - avg(num-field)
  24. Powerful graphs Best is HORIZONTAL - BELLA GUTIE
  25. Most useful - timechart Is this clear? Black theme for cool guys
  26. Adi Dan / Michael Dragetski THEY KNEW NOW I KNOW TOO
  27. SQL Having Walkthrough Stay tuned to see this root cause drill down was done by Kusto Data Science Capabilities Prod-IT can see who to wake up even easier ALL is in BRAIN ASK - was it too fast? Too complex? OK? Ask me...
  28. Don’t sleep just yet!
  29. BRAIN
  30. Copy it to your own use
  31. Meetings (like this one), Meetups, Guild, and… #Slack
  32. Make its UX cool, use nice formatting
  33. Kibana Based
  34. Super rich, however Alerts come first Only when there is an alert - you can open dashboard - to see the history
  35. Tips & Best Practices
  36. We use sensu for alerting Over 50 errors in Yahoo Japan - Call Media PD On Call
  37. Advanced topics
  38. Jupyter
  39. Of course it works. This is our beloved visualisation tool.
  40. Grafana 5
  41. The first point is the most important one Even if you didn’t catch much, you can always ask
  42. Let’s works hard, play hard, and drink up
  43. Tips & Best Practices
  44. This explains what is Kusto Who believes that this is a true chat?