SlideShare a Scribd company logo
Outage Analysis:
March 5th/6th 2024
© 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential 1
Meta, Comcast, and LinkedIn
Featured speakers
Brian Tobia
Technical Marketing Engineer
Kemal Sanjta
Principal Internet Analyst
© 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
• If you have any questions, please type them in the Questions window.
• If you have any audio problems, please chat us for help.
• A recording of this presentation will be sent to you in a few days.
• Interested in more outage analysis and Internet insights? Check out
the ThousandEyes blog and The Internet Report podcast.
Before We Begin…
Meta
Started ~ 15:00 UTC and resolved
at 19:27 UTC (4+ hour outage)
Affected multiple Meta services
(Facebook, Instagram, Messenger,
Threads)
Network connectivity was un-
interrupted
Root cause was backend
authentication system failures
causing the application to be
unavailable
March 5, 2024
Started at 19:45 UTC and was
resolved at 21:40 UTC (2 hour
outage)
ThousandEyes saw increased
network latency and
timeouts/loss of connectivity
within the Comcast backbone
network (Texas)
Affected routing/connectivity to
multiple applications such as
Webex, Salesforce, and AWS
March 5, 2024
Started at 20:45 UTC and
resolved at 21:50 UTC (1 hour
outage)
Network connectivity was
unaffected
Application/HTTP service errors
were seen, making the service
inaccessible
No ramp up/ramp down,
indicating a possible back-end
failure
March 6, 2024
Outages Summary
Meta Outage
March 5th, 2024
5
6
High Level Summary
© 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential
● Beginning at approximately 15:00 UTC on 3/5/2024,
Meta experienced a significant global disruption that
impacted connectivity to many of its applications,
including Facebook, Instagram, Messenger, and
Threads.
● Platform was reachable, but users could not
proceed past authentication
● Users saw rejected passwords, feeds not being
refreshed, and were being logged out.
● No significant network conditions were observed
that could have been contributing to or causing the
outage.
Meta Outage
7
Network Paths Clear
© 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential
Meta Outage
8
HTTP Server Reachable
© 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential
Meta Outage
9
Issue Confirmation and Recovery
© 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential
● At around 15:17 UTC, Meta confirmed that they
were experiencing issues with their login services.
The issue was likely caused by a failure in one of
the dependencies that the login system relies on.
● ThousandEyes observed a gradual recovery of
impacted Meta services. At approximately 16:50
UTC, it appeared that the services had been
restored for some users
● ThousandEyes was able to confirm that by 18:40
UTC the majority of regions were able to connect.
● Finally, at 19:27 UTC, Meta made the official
announcement that the issue was fully resolved.
Meta Outage
Sharelink Demo
10
Comcast Outage
March 5th, 2024
11
12
High Level Summary
© 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential
● At approximately 19:45 UTC on March 5th,
ThousandEyes began observing outage
conditions in parts of Comcast’s network
● Impacted the reachability of many
applications and services, including Webex,
Salesforce, and AWS
● The outage appears to have impacted
traffic as it traversed Comcast’s network
backbone in Houston, Texas, including
traffic that originated in regions such as
California and Colorado
Comcast Outage
13
Network Path Failures and Packet Loss
© 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential
Comcast Outage
14
Comcast POP Network Architecture
© 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential
Comcast Outage
15
Comcast’s Houston POP During the Outage
© 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential
Comcast Outage
16
Issue Identification and Confirmation
© 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential
● This outage was initially misattributed as application
issues vs. a network outage
● Many consumer applications did not see issues due
to content for these services being cached and
served locally from CDN’s
● This outage mainly affected enterprise applications
which require traffic to be routed to centralized
locations or east/west within provider networks to
access data, rather than it being more localized
● Application providers confirmed ongoing issues due
to an external network provider
Comcast Outage
17
Issue Recovery
© 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential
● The outage continued for around two hours
and was eventually resolved by 21:40 UTC
● Part of the impact of this issue is due to
how AWS routes traffic into its networks
● AWS relies more on public internet to
transit users into their backbone networks
whereas Azure and GCP prefer users to
enter their network closer to the source
● Applications hosted in AWS might have felt
more impact due to users traversing over
Comcast networks to get to them
● How can you recover once you’ve detected
a provider issue? – leverage VPN/SSE to
route around the issue with multiple
providers
Comcast Outage
Sharelink Demo
18
LinkedIn Outage
March 6th, 2024
19
20
High Level Summary
© 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential
● On March 6, ThousandEyes detected an almost two-
hour service disruption for global users of LinkedIn
that manifested as service unavailable error
messages, suggesting a backend application issue.
● This global outage was a lights on/ lights off type of
scenario
○ No gradual performance decline which would
have indicated load issues
○ Timed out at origin means the resource
suddenly went away
● HTTP and receive errors were seen throughout the
issue, indicating that an application component had
failed
○ Page load time increased and many tests saw
operation cancelled errors
● Network paths were unaffected and showed no errors
or packet loss
LinkedIn Outage
21
Issue Confirmation and Recovery
© 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential
● LinkedIn acknowledged the issue at 20:50 UTC
● As the outage progressed, we saw tests returning
HTTP 503 service unavailable messages
○ Indicating that the back-end service is not
accessible
● As the service began to recover, the tests switched
to showing HTTP 502 bad gateway
○ This can indicate an invalid response was
received - backend was available, but not
sending proper data
○ This can also sometimes be seen as a sign
of system load as a service is being
restarted/restored
● ThousandEyes saw access to the LinkedIn web
application beginning to recover around 21:38 UTC
and LinkedIn posted that the issue was resolved as
of 22:05 UTC
LinkedIn Outage
Sharelink Demo
22
23
Lessons and Takeaways
© 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential
● Keep business moving by having a backup solution in place when an outage
happens.
○ For example, while LinkedIn was down, some companies and individuals
may have turned to posting updates on X, Facebook, or other social
media platforms.
● Authentication is a crucial step in the process of accessing a service.
A failure at this step can impact the entire application delivery chain, causing
major disruptions for users.
● It is crucial to have a complete view of the entire system in order to identify any
decrease in performance or functionality
○ Make sure you are testing all components, including 3rd party API’s
● Knowing as quickly as possible that performance has drifted from desired levels
is critical to reducing the pain to your customers, users, and partners.
● When services you rely on experience issues, a good first question is whether
the problem lies with you, that application, or a third-party provider.
24
Sharelinks
© 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential
Comcast: https://abxcsltbdauvkratsbjcuveindsjxenn.share.thousandeyes.com/view/internet-
insights/?roundId=1709668800&metric=interfaces&scenarioId=outageTraffic&filters=N4IgZglg
NgLgpgJwM4gFygIYActQgYwxggHsA7AERIFsMIyAVDAczQG0BdAGhG1wKKkyAeQCuMFnA
DK%2BEljjsQAQQBiKgKIBhBuooB9HVIZ7hAJT0BJAHJ7NShkoAywgOIhuvJGXYB2AJwATAE
eMAgYYJD4YhLM0rLyiqoa2roG6kYm5ta29k6u7gC%2BBUA
LinkedIn: https://aqhlwhhcznjuwoweapziutmrjmkrrexg.share.thousandeyes.com
Next Steps
Subscribe to our blog to keep up-to-date!​
thousandeyes.com/blog/​
Tune in to The Internet Report Podcast.​
thousandeyes.com/the-internet-report/
Check out the in-depth Comcast Outage
Analysis:
thousandeyes.com/blog/comcast-outage-
analysis-march-5-2024
New tutorial videos on our features​
thousandeyes.com/resources/?cat=tutorial​
New Getting Started Guides​
docs.thousandeyes.com/product-
documentation/getting-started
Blog &
Podcast
Learning
Resources
26
© 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential

More Related Content

What's hot

Creating MuleSoft API Template Project Using Maven Archetype
Creating MuleSoft API Template Project Using Maven ArchetypeCreating MuleSoft API Template Project Using Maven Archetype
Creating MuleSoft API Template Project Using Maven Archetype
Manish Kumar Yadav
 
Lightning-fast Analytics for Workday transactional data
Lightning-fast Analytics for Workday transactional dataLightning-fast Analytics for Workday transactional data
Lightning-fast Analytics for Workday transactional data
Pavel Hardak
 
MuleSoft Surat Meetup#54 - MuleSoft Automation
MuleSoft Surat Meetup#54 - MuleSoft AutomationMuleSoft Surat Meetup#54 - MuleSoft Automation
MuleSoft Surat Meetup#54 - MuleSoft Automation
Jitendra Bafna
 
Three layer API Design Architecture
Three layer API Design ArchitectureThree layer API Design Architecture
Three layer API Design Architecture
Harish Kumar
 
MuleSoft Event Driven Architecture (EDA Patterns in MuleSoft) - VirtualMuleys63
MuleSoft Event Driven Architecture (EDA Patterns in MuleSoft) - VirtualMuleys63MuleSoft Event Driven Architecture (EDA Patterns in MuleSoft) - VirtualMuleys63
MuleSoft Event Driven Architecture (EDA Patterns in MuleSoft) - VirtualMuleys63
Angel Alberici
 
Combining Logs, Metrics, and Traces for Unified Observability
Combining Logs, Metrics, and Traces for Unified ObservabilityCombining Logs, Metrics, and Traces for Unified Observability
Combining Logs, Metrics, and Traces for Unified Observability
Elasticsearch
 
High Availability (HA) Explained
High Availability (HA) ExplainedHigh Availability (HA) Explained
High Availability (HA) Explained
Maciej Lasyk
 
apidays LIVE Singapore - Next-generation microservice architecture based on A...
apidays LIVE Singapore - Next-generation microservice architecture based on A...apidays LIVE Singapore - Next-generation microservice architecture based on A...
apidays LIVE Singapore - Next-generation microservice architecture based on A...
apidays
 
Red hat ansible automation technical deck
Red hat ansible automation technical deckRed hat ansible automation technical deck
Red hat ansible automation technical deck
Juraj Hantak
 
CDC patterns in Apache Kafka®
CDC patterns in Apache Kafka®CDC patterns in Apache Kafka®
CDC patterns in Apache Kafka®
confluent
 
Meetup - Automate your project lifecycle using MuleSoft and Azure DevOps
Meetup - Automate your project lifecycle using MuleSoft and Azure DevOpsMeetup - Automate your project lifecycle using MuleSoft and Azure DevOps
Meetup - Automate your project lifecycle using MuleSoft and Azure DevOps
Renato de Oliveira
 
Building a Large Scale SolarWinds Installation
Building a Large Scale SolarWinds InstallationBuilding a Large Scale SolarWinds Installation
Building a Large Scale SolarWinds Installation
SolarWinds
 
Reference design for v mware nsx
Reference design for v mware nsxReference design for v mware nsx
Reference design for v mware nsx
solarisyougood
 
Observability
ObservabilityObservability
IBM MQ - High Availability and Disaster Recovery
IBM MQ - High Availability and Disaster RecoveryIBM MQ - High Availability and Disaster Recovery
IBM MQ - High Availability and Disaster Recovery
MarkTaylorIBM
 
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...
Kai Wähner
 
Next Generation Network Automation
Next Generation Network AutomationNext Generation Network Automation
Next Generation Network Automation
Laurent Ciavaglia
 
MuleSoft Surat Meetup#45 - Anypoint Flex Gateway as a Kubernetes Ingress Cont...
MuleSoft Surat Meetup#45 - Anypoint Flex Gateway as a Kubernetes Ingress Cont...MuleSoft Surat Meetup#45 - Anypoint Flex Gateway as a Kubernetes Ingress Cont...
MuleSoft Surat Meetup#45 - Anypoint Flex Gateway as a Kubernetes Ingress Cont...
Jitendra Bafna
 
Introduction to Microservices
Introduction to MicroservicesIntroduction to Microservices
Introduction to Microservices
Amazon Web Services
 
Developing real-time data pipelines with Spring and Kafka
Developing real-time data pipelines with Spring and KafkaDeveloping real-time data pipelines with Spring and Kafka
Developing real-time data pipelines with Spring and Kafka
marius_bogoevici
 

What's hot (20)

Creating MuleSoft API Template Project Using Maven Archetype
Creating MuleSoft API Template Project Using Maven ArchetypeCreating MuleSoft API Template Project Using Maven Archetype
Creating MuleSoft API Template Project Using Maven Archetype
 
Lightning-fast Analytics for Workday transactional data
Lightning-fast Analytics for Workday transactional dataLightning-fast Analytics for Workday transactional data
Lightning-fast Analytics for Workday transactional data
 
MuleSoft Surat Meetup#54 - MuleSoft Automation
MuleSoft Surat Meetup#54 - MuleSoft AutomationMuleSoft Surat Meetup#54 - MuleSoft Automation
MuleSoft Surat Meetup#54 - MuleSoft Automation
 
Three layer API Design Architecture
Three layer API Design ArchitectureThree layer API Design Architecture
Three layer API Design Architecture
 
MuleSoft Event Driven Architecture (EDA Patterns in MuleSoft) - VirtualMuleys63
MuleSoft Event Driven Architecture (EDA Patterns in MuleSoft) - VirtualMuleys63MuleSoft Event Driven Architecture (EDA Patterns in MuleSoft) - VirtualMuleys63
MuleSoft Event Driven Architecture (EDA Patterns in MuleSoft) - VirtualMuleys63
 
Combining Logs, Metrics, and Traces for Unified Observability
Combining Logs, Metrics, and Traces for Unified ObservabilityCombining Logs, Metrics, and Traces for Unified Observability
Combining Logs, Metrics, and Traces for Unified Observability
 
High Availability (HA) Explained
High Availability (HA) ExplainedHigh Availability (HA) Explained
High Availability (HA) Explained
 
apidays LIVE Singapore - Next-generation microservice architecture based on A...
apidays LIVE Singapore - Next-generation microservice architecture based on A...apidays LIVE Singapore - Next-generation microservice architecture based on A...
apidays LIVE Singapore - Next-generation microservice architecture based on A...
 
Red hat ansible automation technical deck
Red hat ansible automation technical deckRed hat ansible automation technical deck
Red hat ansible automation technical deck
 
CDC patterns in Apache Kafka®
CDC patterns in Apache Kafka®CDC patterns in Apache Kafka®
CDC patterns in Apache Kafka®
 
Meetup - Automate your project lifecycle using MuleSoft and Azure DevOps
Meetup - Automate your project lifecycle using MuleSoft and Azure DevOpsMeetup - Automate your project lifecycle using MuleSoft and Azure DevOps
Meetup - Automate your project lifecycle using MuleSoft and Azure DevOps
 
Building a Large Scale SolarWinds Installation
Building a Large Scale SolarWinds InstallationBuilding a Large Scale SolarWinds Installation
Building a Large Scale SolarWinds Installation
 
Reference design for v mware nsx
Reference design for v mware nsxReference design for v mware nsx
Reference design for v mware nsx
 
Observability
ObservabilityObservability
Observability
 
IBM MQ - High Availability and Disaster Recovery
IBM MQ - High Availability and Disaster RecoveryIBM MQ - High Availability and Disaster Recovery
IBM MQ - High Availability and Disaster Recovery
 
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...
 
Next Generation Network Automation
Next Generation Network AutomationNext Generation Network Automation
Next Generation Network Automation
 
MuleSoft Surat Meetup#45 - Anypoint Flex Gateway as a Kubernetes Ingress Cont...
MuleSoft Surat Meetup#45 - Anypoint Flex Gateway as a Kubernetes Ingress Cont...MuleSoft Surat Meetup#45 - Anypoint Flex Gateway as a Kubernetes Ingress Cont...
MuleSoft Surat Meetup#45 - Anypoint Flex Gateway as a Kubernetes Ingress Cont...
 
Introduction to Microservices
Introduction to MicroservicesIntroduction to Microservices
Introduction to Microservices
 
Developing real-time data pipelines with Spring and Kafka
Developing real-time data pipelines with Spring and KafkaDeveloping real-time data pipelines with Spring and Kafka
Developing real-time data pipelines with Spring and Kafka
 

Similar to Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn

The Top Outages of 2023: Analyses and Takeaways
The Top Outages of 2023: Analyses and TakeawaysThe Top Outages of 2023: Analyses and Takeaways
The Top Outages of 2023: Analyses and Takeaways
ThousandEyes
 
The Top Outages of 2023: Analysis and Takeaways
The Top Outages of 2023: Analysis and TakeawaysThe Top Outages of 2023: Analysis and Takeaways
The Top Outages of 2023: Analysis and Takeaways
ThousandEyes
 
The Top Outages of 2023: Analysis and Takeaways
The Top Outages of 2023: Analysis and TakeawaysThe Top Outages of 2023: Analysis and Takeaways
The Top Outages of 2023: Analysis and Takeaways
ThousandEyes
 
The Top Outages of 2022: Analysis and Takeaways
The Top Outages of 2022: Analysis and TakeawaysThe Top Outages of 2022: Analysis and Takeaways
The Top Outages of 2022: Analysis and Takeaways
ThousandEyes
 
The Top Outages of 2022: Analysis and Takeaways
The Top Outages of 2022: Analysis and TakeawaysThe Top Outages of 2022: Analysis and Takeaways
The Top Outages of 2022: Analysis and Takeaways
ThousandEyes
 
EMEA.23.02.23_Top_Outages_of_2022_Webinar_Slides.pptx
EMEA.23.02.23_Top_Outages_of_2022_Webinar_Slides.pptxEMEA.23.02.23_Top_Outages_of_2022_Webinar_Slides.pptx
EMEA.23.02.23_Top_Outages_of_2022_Webinar_Slides.pptx
ThousandEyes
 
Getting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of ConceptsGetting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of Concepts
ThousandEyes
 
The Top Outages of 2021: Analysis and Takeaways
The Top Outages of 2021: Analysis and TakeawaysThe Top Outages of 2021: Analysis and Takeaways
The Top Outages of 2021: Analysis and Takeaways
ThousandEyes
 
Getting Started With ThousandEyes Proof of Concepts: End User Digital Experience
Getting Started With ThousandEyes Proof of Concepts: End User Digital ExperienceGetting Started With ThousandEyes Proof of Concepts: End User Digital Experience
Getting Started With ThousandEyes Proof of Concepts: End User Digital Experience
ThousandEyes
 
Introduction to ThousandEyes
Introduction to ThousandEyesIntroduction to ThousandEyes
Introduction to ThousandEyes
ThousandEyes
 
Troubleshooting Webex and Microsoft Teams with ThousandEyes
Troubleshooting Webex and Microsoft Teams with ThousandEyesTroubleshooting Webex and Microsoft Teams with ThousandEyes
Troubleshooting Webex and Microsoft Teams with ThousandEyes
ThousandEyes
 
Getting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of ConceptsGetting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of Concepts
ThousandEyes
 
Optimizing and Troubleshooting Digital Experience for a Hybrid Workforce
Optimizing and Troubleshooting Digital Experience for a Hybrid WorkforceOptimizing and Troubleshooting Digital Experience for a Hybrid Workforce
Optimizing and Troubleshooting Digital Experience for a Hybrid Workforce
ThousandEyes
 
EMEA Optimizing and Troubleshooting Digital Experience for a Hybrid Workforce
EMEA Optimizing and Troubleshooting Digital Experience for a Hybrid WorkforceEMEA Optimizing and Troubleshooting Digital Experience for a Hybrid Workforce
EMEA Optimizing and Troubleshooting Digital Experience for a Hybrid Workforce
ThousandEyes
 
Getting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of ConceptsGetting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of Concepts
ThousandEyes
 
Getting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of ConceptsGetting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of Concepts
ThousandEyes
 
Introduction to ThousandEyes
Introduction to ThousandEyesIntroduction to ThousandEyes
Introduction to ThousandEyes
ThousandEyes
 
Getting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of ConceptsGetting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of Concepts
ThousandEyes
 
Cisco Connect 2018 Singapore - En06 jason pernell
Cisco Connect 2018 Singapore - En06 jason pernellCisco Connect 2018 Singapore - En06 jason pernell
Cisco Connect 2018 Singapore - En06 jason pernell
NetworkCollaborators
 
Cisco Connect 2018 Indonesia - Introducing cisco dna assurance
Cisco Connect 2018 Indonesia - Introducing cisco dna assurance Cisco Connect 2018 Indonesia - Introducing cisco dna assurance
Cisco Connect 2018 Indonesia - Introducing cisco dna assurance
NetworkCollaborators
 

Similar to Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn (20)

The Top Outages of 2023: Analyses and Takeaways
The Top Outages of 2023: Analyses and TakeawaysThe Top Outages of 2023: Analyses and Takeaways
The Top Outages of 2023: Analyses and Takeaways
 
The Top Outages of 2023: Analysis and Takeaways
The Top Outages of 2023: Analysis and TakeawaysThe Top Outages of 2023: Analysis and Takeaways
The Top Outages of 2023: Analysis and Takeaways
 
The Top Outages of 2023: Analysis and Takeaways
The Top Outages of 2023: Analysis and TakeawaysThe Top Outages of 2023: Analysis and Takeaways
The Top Outages of 2023: Analysis and Takeaways
 
The Top Outages of 2022: Analysis and Takeaways
The Top Outages of 2022: Analysis and TakeawaysThe Top Outages of 2022: Analysis and Takeaways
The Top Outages of 2022: Analysis and Takeaways
 
The Top Outages of 2022: Analysis and Takeaways
The Top Outages of 2022: Analysis and TakeawaysThe Top Outages of 2022: Analysis and Takeaways
The Top Outages of 2022: Analysis and Takeaways
 
EMEA.23.02.23_Top_Outages_of_2022_Webinar_Slides.pptx
EMEA.23.02.23_Top_Outages_of_2022_Webinar_Slides.pptxEMEA.23.02.23_Top_Outages_of_2022_Webinar_Slides.pptx
EMEA.23.02.23_Top_Outages_of_2022_Webinar_Slides.pptx
 
Getting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of ConceptsGetting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of Concepts
 
The Top Outages of 2021: Analysis and Takeaways
The Top Outages of 2021: Analysis and TakeawaysThe Top Outages of 2021: Analysis and Takeaways
The Top Outages of 2021: Analysis and Takeaways
 
Getting Started With ThousandEyes Proof of Concepts: End User Digital Experience
Getting Started With ThousandEyes Proof of Concepts: End User Digital ExperienceGetting Started With ThousandEyes Proof of Concepts: End User Digital Experience
Getting Started With ThousandEyes Proof of Concepts: End User Digital Experience
 
Introduction to ThousandEyes
Introduction to ThousandEyesIntroduction to ThousandEyes
Introduction to ThousandEyes
 
Troubleshooting Webex and Microsoft Teams with ThousandEyes
Troubleshooting Webex and Microsoft Teams with ThousandEyesTroubleshooting Webex and Microsoft Teams with ThousandEyes
Troubleshooting Webex and Microsoft Teams with ThousandEyes
 
Getting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of ConceptsGetting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of Concepts
 
Optimizing and Troubleshooting Digital Experience for a Hybrid Workforce
Optimizing and Troubleshooting Digital Experience for a Hybrid WorkforceOptimizing and Troubleshooting Digital Experience for a Hybrid Workforce
Optimizing and Troubleshooting Digital Experience for a Hybrid Workforce
 
EMEA Optimizing and Troubleshooting Digital Experience for a Hybrid Workforce
EMEA Optimizing and Troubleshooting Digital Experience for a Hybrid WorkforceEMEA Optimizing and Troubleshooting Digital Experience for a Hybrid Workforce
EMEA Optimizing and Troubleshooting Digital Experience for a Hybrid Workforce
 
Getting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of ConceptsGetting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of Concepts
 
Getting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of ConceptsGetting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of Concepts
 
Introduction to ThousandEyes
Introduction to ThousandEyesIntroduction to ThousandEyes
Introduction to ThousandEyes
 
Getting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of ConceptsGetting Started with ThousandEyes Proof of Concepts
Getting Started with ThousandEyes Proof of Concepts
 
Cisco Connect 2018 Singapore - En06 jason pernell
Cisco Connect 2018 Singapore - En06 jason pernellCisco Connect 2018 Singapore - En06 jason pernell
Cisco Connect 2018 Singapore - En06 jason pernell
 
Cisco Connect 2018 Indonesia - Introducing cisco dna assurance
Cisco Connect 2018 Indonesia - Introducing cisco dna assurance Cisco Connect 2018 Indonesia - Introducing cisco dna assurance
Cisco Connect 2018 Indonesia - Introducing cisco dna assurance
 

More from ThousandEyes

Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
ThousandEyes
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
ThousandEyes
 
How To Troubleshoot Collaboration Apps for the Modern Connected Worker
How To Troubleshoot Collaboration Apps for the Modern Connected WorkerHow To Troubleshoot Collaboration Apps for the Modern Connected Worker
How To Troubleshoot Collaboration Apps for the Modern Connected Worker
ThousandEyes
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
ThousandEyes
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
ThousandEyes
 
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesAssure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
ThousandEyes
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
ThousandEyes
 
New ThousandEyes Product Features and Release Highlights: March 2024
New ThousandEyes Product Features and Release Highlights: March 2024New ThousandEyes Product Features and Release Highlights: March 2024
New ThousandEyes Product Features and Release Highlights: March 2024
ThousandEyes
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? Webinar
ThousandEyes
 
Assure Patient and Clinician Digital Experiences with ThousandEyes for Health...
Assure Patient and Clinician Digital Experiences with ThousandEyes for Health...Assure Patient and Clinician Digital Experiences with ThousandEyes for Health...
Assure Patient and Clinician Digital Experiences with ThousandEyes for Health...
ThousandEyes
 
AMER Introduction to ThousandEyes Webinar
AMER Introduction to ThousandEyes WebinarAMER Introduction to ThousandEyes Webinar
AMER Introduction to ThousandEyes Webinar
ThousandEyes
 
New ThousandEyes Product Features and Release Highlights: February 2024
New ThousandEyes Product Features and Release Highlights: February 2024New ThousandEyes Product Features and Release Highlights: February 2024
New ThousandEyes Product Features and Release Highlights: February 2024
ThousandEyes
 
Enhancing SaaS Performance: A Hands-on Workshop for Partners
Enhancing SaaS Performance: A Hands-on Workshop for PartnersEnhancing SaaS Performance: A Hands-on Workshop for Partners
Enhancing SaaS Performance: A Hands-on Workshop for Partners
ThousandEyes
 
ThousandEyes Enterprise Digital Workshop - Spanish
ThousandEyes Enterprise Digital Workshop - SpanishThousandEyes Enterprise Digital Workshop - Spanish
ThousandEyes Enterprise Digital Workshop - Spanish
ThousandEyes
 
ThousandEyes Enterprise Digital Workshop - German
ThousandEyes Enterprise Digital Workshop - GermanThousandEyes Enterprise Digital Workshop - German
ThousandEyes Enterprise Digital Workshop - German
ThousandEyes
 
ThousandEyes Enterprise Digital Workshop
ThousandEyes Enterprise Digital WorkshopThousandEyes Enterprise Digital Workshop
ThousandEyes Enterprise Digital Workshop
ThousandEyes
 
Introduction to ThousandEyes and Meraki MX for Partners
Introduction to ThousandEyes and Meraki MX for PartnersIntroduction to ThousandEyes and Meraki MX for Partners
Introduction to ThousandEyes and Meraki MX for Partners
ThousandEyes
 

More from ThousandEyes (20)

Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
How To Troubleshoot Collaboration Apps for the Modern Connected Worker
How To Troubleshoot Collaboration Apps for the Modern Connected WorkerHow To Troubleshoot Collaboration Apps for the Modern Connected Worker
How To Troubleshoot Collaboration Apps for the Modern Connected Worker
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesAssure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
New ThousandEyes Product Features and Release Highlights: March 2024
New ThousandEyes Product Features and Release Highlights: March 2024New ThousandEyes Product Features and Release Highlights: March 2024
New ThousandEyes Product Features and Release Highlights: March 2024
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? Webinar
 
Assure Patient and Clinician Digital Experiences with ThousandEyes for Health...
Assure Patient and Clinician Digital Experiences with ThousandEyes for Health...Assure Patient and Clinician Digital Experiences with ThousandEyes for Health...
Assure Patient and Clinician Digital Experiences with ThousandEyes for Health...
 
AMER Introduction to ThousandEyes Webinar
AMER Introduction to ThousandEyes WebinarAMER Introduction to ThousandEyes Webinar
AMER Introduction to ThousandEyes Webinar
 
New ThousandEyes Product Features and Release Highlights: February 2024
New ThousandEyes Product Features and Release Highlights: February 2024New ThousandEyes Product Features and Release Highlights: February 2024
New ThousandEyes Product Features and Release Highlights: February 2024
 
Enhancing SaaS Performance: A Hands-on Workshop for Partners
Enhancing SaaS Performance: A Hands-on Workshop for PartnersEnhancing SaaS Performance: A Hands-on Workshop for Partners
Enhancing SaaS Performance: A Hands-on Workshop for Partners
 
ThousandEyes Enterprise Digital Workshop - Spanish
ThousandEyes Enterprise Digital Workshop - SpanishThousandEyes Enterprise Digital Workshop - Spanish
ThousandEyes Enterprise Digital Workshop - Spanish
 
ThousandEyes Enterprise Digital Workshop - German
ThousandEyes Enterprise Digital Workshop - GermanThousandEyes Enterprise Digital Workshop - German
ThousandEyes Enterprise Digital Workshop - German
 
ThousandEyes Enterprise Digital Workshop
ThousandEyes Enterprise Digital WorkshopThousandEyes Enterprise Digital Workshop
ThousandEyes Enterprise Digital Workshop
 
Introduction to ThousandEyes and Meraki MX for Partners
Introduction to ThousandEyes and Meraki MX for PartnersIntroduction to ThousandEyes and Meraki MX for Partners
Introduction to ThousandEyes and Meraki MX for Partners
 

Recently uploaded

Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Precisely
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
MichaelKnudsen27
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
Javier Junquera
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
BibashShahi
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
Fwdays
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Antonios Katsarakis
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
DianaGray10
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
Ajin Abraham
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
Safe Software
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
Pablo Gómez Abajo
 

Recently uploaded (20)

Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
 

Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn

  • 1. Outage Analysis: March 5th/6th 2024 © 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential 1 Meta, Comcast, and LinkedIn
  • 2. Featured speakers Brian Tobia Technical Marketing Engineer Kemal Sanjta Principal Internet Analyst © 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
  • 3. • If you have any questions, please type them in the Questions window. • If you have any audio problems, please chat us for help. • A recording of this presentation will be sent to you in a few days. • Interested in more outage analysis and Internet insights? Check out the ThousandEyes blog and The Internet Report podcast. Before We Begin…
  • 4. Meta Started ~ 15:00 UTC and resolved at 19:27 UTC (4+ hour outage) Affected multiple Meta services (Facebook, Instagram, Messenger, Threads) Network connectivity was un- interrupted Root cause was backend authentication system failures causing the application to be unavailable March 5, 2024 Started at 19:45 UTC and was resolved at 21:40 UTC (2 hour outage) ThousandEyes saw increased network latency and timeouts/loss of connectivity within the Comcast backbone network (Texas) Affected routing/connectivity to multiple applications such as Webex, Salesforce, and AWS March 5, 2024 Started at 20:45 UTC and resolved at 21:50 UTC (1 hour outage) Network connectivity was unaffected Application/HTTP service errors were seen, making the service inaccessible No ramp up/ramp down, indicating a possible back-end failure March 6, 2024 Outages Summary
  • 6. 6 High Level Summary © 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential ● Beginning at approximately 15:00 UTC on 3/5/2024, Meta experienced a significant global disruption that impacted connectivity to many of its applications, including Facebook, Instagram, Messenger, and Threads. ● Platform was reachable, but users could not proceed past authentication ● Users saw rejected passwords, feeds not being refreshed, and were being logged out. ● No significant network conditions were observed that could have been contributing to or causing the outage. Meta Outage
  • 7. 7 Network Paths Clear © 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential Meta Outage
  • 8. 8 HTTP Server Reachable © 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential Meta Outage
  • 9. 9 Issue Confirmation and Recovery © 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential ● At around 15:17 UTC, Meta confirmed that they were experiencing issues with their login services. The issue was likely caused by a failure in one of the dependencies that the login system relies on. ● ThousandEyes observed a gradual recovery of impacted Meta services. At approximately 16:50 UTC, it appeared that the services had been restored for some users ● ThousandEyes was able to confirm that by 18:40 UTC the majority of regions were able to connect. ● Finally, at 19:27 UTC, Meta made the official announcement that the issue was fully resolved. Meta Outage
  • 12. 12 High Level Summary © 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential ● At approximately 19:45 UTC on March 5th, ThousandEyes began observing outage conditions in parts of Comcast’s network ● Impacted the reachability of many applications and services, including Webex, Salesforce, and AWS ● The outage appears to have impacted traffic as it traversed Comcast’s network backbone in Houston, Texas, including traffic that originated in regions such as California and Colorado Comcast Outage
  • 13. 13 Network Path Failures and Packet Loss © 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential Comcast Outage
  • 14. 14 Comcast POP Network Architecture © 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential Comcast Outage
  • 15. 15 Comcast’s Houston POP During the Outage © 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential Comcast Outage
  • 16. 16 Issue Identification and Confirmation © 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential ● This outage was initially misattributed as application issues vs. a network outage ● Many consumer applications did not see issues due to content for these services being cached and served locally from CDN’s ● This outage mainly affected enterprise applications which require traffic to be routed to centralized locations or east/west within provider networks to access data, rather than it being more localized ● Application providers confirmed ongoing issues due to an external network provider Comcast Outage
  • 17. 17 Issue Recovery © 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential ● The outage continued for around two hours and was eventually resolved by 21:40 UTC ● Part of the impact of this issue is due to how AWS routes traffic into its networks ● AWS relies more on public internet to transit users into their backbone networks whereas Azure and GCP prefer users to enter their network closer to the source ● Applications hosted in AWS might have felt more impact due to users traversing over Comcast networks to get to them ● How can you recover once you’ve detected a provider issue? – leverage VPN/SSE to route around the issue with multiple providers Comcast Outage
  • 20. 20 High Level Summary © 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential ● On March 6, ThousandEyes detected an almost two- hour service disruption for global users of LinkedIn that manifested as service unavailable error messages, suggesting a backend application issue. ● This global outage was a lights on/ lights off type of scenario ○ No gradual performance decline which would have indicated load issues ○ Timed out at origin means the resource suddenly went away ● HTTP and receive errors were seen throughout the issue, indicating that an application component had failed ○ Page load time increased and many tests saw operation cancelled errors ● Network paths were unaffected and showed no errors or packet loss LinkedIn Outage
  • 21. 21 Issue Confirmation and Recovery © 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential ● LinkedIn acknowledged the issue at 20:50 UTC ● As the outage progressed, we saw tests returning HTTP 503 service unavailable messages ○ Indicating that the back-end service is not accessible ● As the service began to recover, the tests switched to showing HTTP 502 bad gateway ○ This can indicate an invalid response was received - backend was available, but not sending proper data ○ This can also sometimes be seen as a sign of system load as a service is being restarted/restored ● ThousandEyes saw access to the LinkedIn web application beginning to recover around 21:38 UTC and LinkedIn posted that the issue was resolved as of 22:05 UTC LinkedIn Outage
  • 23. 23 Lessons and Takeaways © 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential ● Keep business moving by having a backup solution in place when an outage happens. ○ For example, while LinkedIn was down, some companies and individuals may have turned to posting updates on X, Facebook, or other social media platforms. ● Authentication is a crucial step in the process of accessing a service. A failure at this step can impact the entire application delivery chain, causing major disruptions for users. ● It is crucial to have a complete view of the entire system in order to identify any decrease in performance or functionality ○ Make sure you are testing all components, including 3rd party API’s ● Knowing as quickly as possible that performance has drifted from desired levels is critical to reducing the pain to your customers, users, and partners. ● When services you rely on experience issues, a good first question is whether the problem lies with you, that application, or a third-party provider.
  • 24. 24 Sharelinks © 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential Comcast: https://abxcsltbdauvkratsbjcuveindsjxenn.share.thousandeyes.com/view/internet- insights/?roundId=1709668800&metric=interfaces&scenarioId=outageTraffic&filters=N4IgZglg NgLgpgJwM4gFygIYActQgYwxggHsA7AERIFsMIyAVDAczQG0BdAGhG1wKKkyAeQCuMFnA DK%2BEljjsQAQQBiKgKIBhBuooB9HVIZ7hAJT0BJAHJ7NShkoAywgOIhuvJGXYB2AJwATAE eMAgYYJD4YhLM0rLyiqoa2roG6kYm5ta29k6u7gC%2BBUA LinkedIn: https://aqhlwhhcznjuwoweapziutmrjmkrrexg.share.thousandeyes.com
  • 25. Next Steps Subscribe to our blog to keep up-to-date!​ thousandeyes.com/blog/​ Tune in to The Internet Report Podcast.​ thousandeyes.com/the-internet-report/ Check out the in-depth Comcast Outage Analysis: thousandeyes.com/blog/comcast-outage- analysis-march-5-2024 New tutorial videos on our features​ thousandeyes.com/resources/?cat=tutorial​ New Getting Started Guides​ docs.thousandeyes.com/product- documentation/getting-started Blog & Podcast Learning Resources
  • 26. 26 © 2024 Cisco Systems, Inc. and/or its affiliates. All rights reserved. Cisco Confidential