This document provides an overview of an Auto Root Cause Analysis (RCA) project at Microsoft. The goal of the project is to detect developing performance issues in Azure SQL Database and surface actionable insights to customers and support engineers. The document outlines the project objectives, business impact, example insights, workflow, current progress, challenges, learnings, and next steps. It also lists the collaborators on the project including the engineer, data scientist, and PM.

1. URL PREVIEW
Microsoft Whiteboard
Appx Team
Ritik Garg

2. Feature Crew
Ritik
Dev Intern
Saahiljeet
Dev
Deepika
Nishant
EM Designer
Sushma
PM

3. Problem Statement
Current WBD behaviour
Legacy UWP behaviour

4. Customer’s Ask

5. Important Use Cases
• Adding sources/references for information.
• Accessing 1P and 3P content during meetings and knowledge sharing
sessions.
• Connecting additional educational/workshop activities and increase
collaboration.
• Building relations across whiteboards.
• Cross referencing content within same board.

6. Our Competitors
Scenarios
Clickable Link (Type)
Clickable Link (Copy
– Paste)
Smart Links
(meaningful title)
URL Preview Limited
Support
Add link to text (ctrl
+ k)

7. Outline
• Blockers and Hiccups
• Different services, CSP issues, scraping problems,
• Solution
• Metascraping in UWP and CSP Issues
• 1P Services ( Hubble & OneServices)  Interactive content
• URLP Service  Getting meta data
• Iframe, layer, schema changes, internal services, copy paste detection,
regex matching, stamp changes

8. • Currently teaching mathematics and computer science
fundamentals to high school students
• Pursuing research in convex analysis and optimization
of transformer neural networks for driver's risk inference problem
• I also love reading all sorts of books – especially biographies and
memoirs
BS in Computer Science (specialization in machine learning)
BS in Computational and Applied Math
The University of Chicago
Graduating June 2022
Blaise Munyampirwa
Manager:
Mentor:
Extracurriculars and Hobbies

9. The Scope of
the Auto RCA
Project
The Auto RCA project is a long-term effort whose goal is to
detect developing problems around database performance
and other customer-reported issues and to surface actionable
insights to customers (via Azure portal) and support
engineers through Azure Support Center (ASC).
Objectives
• To help customers self-serve to fix problems through Azure
portal
• To assist support engineers and product group engineers to
reduce the time spent mitigating and resolving cases
Business Impact
• The anticipated impact of this effort is to reduce support
costs for Azure SQL DB while not negatively impacting
customer satisfaction.

10. Example of A Trend Insight surfaced in ASC: CPU Usage Increase
Auto RCA v1 Design Specification.docx (sharepoint.com)

11. © Copyright Microsoft Corporation. All rights reserved.
A regular feature of Relational
Database Management Systems that
takes effect when one transaction
attempts to obtain control on a
resource that is currently being held
by another transaction
What is Locking?

12. © Copyright Microsoft Corporation. All rights reserved.
Example of a Locking Scenario in SQL Server
Two processes want to
access resources that
are mutually being
blocked by each other
Deadlock

13. © Copyright Microsoft Corporation. All rights reserved.
Why Would Lock Waits Increase Beyond Regular Patterns?
• A few specific queries ran by
a customer may account for
a significant proportion of
the abnormal lock wait
time increase.
• Customer workload increase
(running more heavy-weight
queries or increasing the
number of query executions
per day).
Fig. Query hashes with increased total lock wait times
Y-axis: Total Query Wait Time in Milliseconds

14. © Copyright Microsoft Corporation. All rights reserved.
Project Workflow
Cosmos (cold
path data)
Kusto (hot
path data)
Kusto and
SCOPE
queries
SCOPE script integration
into the existing Auto
RCA pipeline
Surfacing Insights in ASC
or Azure Portal
ADF Kusto

15. Current
Progress
 Completed the project's design document and
shared it with stakeholders
doc: Auto RCA Wait Stats.docx (sharepoint.com)
 Wrote an initial SCOPE script to analyze cold path
data in order to detect queries with the highest
average lock wait times
1. Examining specific queries (top 5) with the
highest lock wait times for different databases
2. Running the SCOPE script to observe the
variation for these queries for an extended
period of time
3. Determining potential impactful increases for
these queries by utilizing statistical distributions
of wait time changes
Analysis Points:

16. Challenges
and Learnings
Challenges
• Initially struggled with understanding the Kusto
telemetry and the whole pipeline, but my
understanding continues to grow overtime
• Navigating large documentation and code base
Learnings
• Improving my ability to effectively approach
complex open-ended projects
• Understanding the nitty-gritties of locking in
RDBMS and the Kusto telemetry
• Learning a lot about the architecture of SQL
Server
• Completed a two-day bootcamp on Azure SQL
DB and cloud computing

17. Next Steps
Next Steps
• Continue iterating on the best approach to
determine impactful increases in lock waits
(potentially moving beyond statistical
significance)
• Integrating the changes into the existing Auto
RCA pipeline

18. Project Collaborators
Engineer:
Data Scientist:
PM:
Kendall Thomas
Tim Goodman
Srini Acharya