1
© 1992–2023 Cisco Systems, Inc. All rights reserved.
2
© 1992–2023 Cisco Systems, Inc. All rights reserved.
Featured Speakers
Chris Villemez
Technical Marketing Engineer
Kemal Sanjta
Principal Internet Analyst
3
© 1992–2023 Cisco Systems, Inc. All rights reserved.
Before We Begin...
• If you have any questions, please type them in the Questions window.
• If you have any audio problems, please chat us for help.
• A recording of this presentation will be sent to you in a few days.
3
@ThousandEyes
© 1992–2023 Cisco Systems, Inc. All rights reserved.
4
© 1992–2023 Cisco Systems, Inc. All rights reserved.
Agenda
• About ThousandEyes
• Digital Service Building Blocks
• Microsoft Outage Summary
• Live Data/Sharelink Review
• Lessons & Takeaways
• Q&A
4
@ThousandEyes
5
© 1992–2023 Cisco Systems, Inc. All rights reserved.
Actionable Insight for Internet, Cloud, and SaaS
Correlated Insights
Quickly isolate issues to app, network,
or service
Network Visibility
Overlay, hop-by-hop underlay, ISP
performance, and BGP routing
App Experience
SaaS, API, and internal app
performance and user experience
6
© 1992–2023 Cisco Systems, Inc. All rights reserved.
CDN
Cloud
BGP
DNS
The Building Blocks of Today’s Digital Services
SaaS
© 1992–2023 Cisco Systems, Inc. All rights reserved. 7
Microsoft Outage Summary
• Significant disruption that impacted connectivity to many
services, including Azure, Teams, Outlook and Sharepoint
– High levels of packet loss
– Both Microsoft and other services became unavailable due to
connectivity disruptions
• Outage started around 7:05 AM UTC and lasted until 8:50
AM UTC, at which time packet loss returned to 0%
• Triggered by an external BGP change by Microsoft that
impacted connected service providers
– Led to destabilization of global routes to its prefixes
– Significant packet loss and diminished reachability of its services
– During the incident, DNS changes were observed that may have
indicated steps to steer traffic away from impacted areas within
Microsoft’s network.
8
© 1992–2023 Cisco Systems, Inc. All rights reserved.
ThousandEyes Global View During The Outage
Live Data/Sharelink
Review
10
© 1992–2023 Cisco Systems, Inc. All rights reserved.
Gain Network and Application Outage Visibility Using Internet Insights
• Internet Insights provides a macro-
level view of ongoing network and app
outages
• Global data collected from different
networks, ISP’s, and transit providers
• Data is fed into the platform to help
enterprise users correlate performance
issues with broader outages and
quickly pinpoint where an issue is
occuring
Learn More About Internet Insights
11
© 1992–2023 Cisco Systems, Inc. All rights reserved.
Key Lessons and Takeaways
• Don’t underestimate the importance of change management
– With diligent process, change reviews, prepared rollback plans, and quality
assurance testing prior and after any change, the risks of disruption can be
greatly reduced.
• Robust visibility is essential
– This includes end-to-end visibility across all technology domains involved
in delivering the service to users.
– Knowing as quickly as possible that performance has drifted from desired
levels is critical to reducing the pain to your customers, users, and
partners.
– IT teams must also be able to quickly pinpoint where the fault is and who
owns and can fix it.
© 1992–2021 Cisco Systems, Inc. All rights reserved. 12
@ThousandEyes
Learn
more
Free
Trial /
Demo
Next Steps
© 1992–2023 Cisco Systems, Inc. All rights reserved.
• Subscribe! https://blog.thousandeyes.com
• Get a real-time view of the health of the Internet
https://thousandeyes.com/outages
• Sign up for a Free Trial:
https://www.thousandeyes.com/signup
• Request a demo:
https://www.thousandeyes.com/request-demo
Q&A
Microsoft Outage Analysis

Microsoft Outage Analysis

  • 1.
    1 © 1992–2023 CiscoSystems, Inc. All rights reserved.
  • 2.
    2 © 1992–2023 CiscoSystems, Inc. All rights reserved. Featured Speakers Chris Villemez Technical Marketing Engineer Kemal Sanjta Principal Internet Analyst
  • 3.
    3 © 1992–2023 CiscoSystems, Inc. All rights reserved. Before We Begin... • If you have any questions, please type them in the Questions window. • If you have any audio problems, please chat us for help. • A recording of this presentation will be sent to you in a few days. 3 @ThousandEyes © 1992–2023 Cisco Systems, Inc. All rights reserved.
  • 4.
    4 © 1992–2023 CiscoSystems, Inc. All rights reserved. Agenda • About ThousandEyes • Digital Service Building Blocks • Microsoft Outage Summary • Live Data/Sharelink Review • Lessons & Takeaways • Q&A 4 @ThousandEyes
  • 5.
    5 © 1992–2023 CiscoSystems, Inc. All rights reserved. Actionable Insight for Internet, Cloud, and SaaS Correlated Insights Quickly isolate issues to app, network, or service Network Visibility Overlay, hop-by-hop underlay, ISP performance, and BGP routing App Experience SaaS, API, and internal app performance and user experience
  • 6.
    6 © 1992–2023 CiscoSystems, Inc. All rights reserved. CDN Cloud BGP DNS The Building Blocks of Today’s Digital Services SaaS
  • 7.
    © 1992–2023 CiscoSystems, Inc. All rights reserved. 7 Microsoft Outage Summary • Significant disruption that impacted connectivity to many services, including Azure, Teams, Outlook and Sharepoint – High levels of packet loss – Both Microsoft and other services became unavailable due to connectivity disruptions • Outage started around 7:05 AM UTC and lasted until 8:50 AM UTC, at which time packet loss returned to 0% • Triggered by an external BGP change by Microsoft that impacted connected service providers – Led to destabilization of global routes to its prefixes – Significant packet loss and diminished reachability of its services – During the incident, DNS changes were observed that may have indicated steps to steer traffic away from impacted areas within Microsoft’s network.
  • 8.
    8 © 1992–2023 CiscoSystems, Inc. All rights reserved. ThousandEyes Global View During The Outage
  • 9.
  • 10.
    10 © 1992–2023 CiscoSystems, Inc. All rights reserved. Gain Network and Application Outage Visibility Using Internet Insights • Internet Insights provides a macro- level view of ongoing network and app outages • Global data collected from different networks, ISP’s, and transit providers • Data is fed into the platform to help enterprise users correlate performance issues with broader outages and quickly pinpoint where an issue is occuring Learn More About Internet Insights
  • 11.
    11 © 1992–2023 CiscoSystems, Inc. All rights reserved. Key Lessons and Takeaways • Don’t underestimate the importance of change management – With diligent process, change reviews, prepared rollback plans, and quality assurance testing prior and after any change, the risks of disruption can be greatly reduced. • Robust visibility is essential – This includes end-to-end visibility across all technology domains involved in delivering the service to users. – Knowing as quickly as possible that performance has drifted from desired levels is critical to reducing the pain to your customers, users, and partners. – IT teams must also be able to quickly pinpoint where the fault is and who owns and can fix it.
  • 12.
    © 1992–2021 CiscoSystems, Inc. All rights reserved. 12 @ThousandEyes Learn more Free Trial / Demo Next Steps © 1992–2023 Cisco Systems, Inc. All rights reserved. • Subscribe! https://blog.thousandeyes.com • Get a real-time view of the health of the Internet https://thousandeyes.com/outages • Sign up for a Free Trial: https://www.thousandeyes.com/signup • Request a demo: https://www.thousandeyes.com/request-demo
  • 13.