SlideShare a Scribd company logo
1 of 68
Download to read offline
TAMMY BUTOW & ANA MEDINA, GREMLIN
QCON SAN FRANCISCO, 2018
TAMMY BUTOW
PRINCIPAL SRE , GREMLIN
@TAMMYBUTOW
ANA MEDINA
CHAOS ENGINEER , GREMLIN
@ANA_M_MEDINA
v
• =
gremlin.com/slack
JOIN THE #QCONNY18 SLACK CHANNEL
CHAOS ENGINEERING SLACK
@TAMMYBUTOW @ANA_M_MEDINA
1. DISCOVER AND EXPLORE THE PRACTICE OF CHAOS ENGINEERING

2. IMMERSE YOURSELF IN A DISCUSSION ON CHAOS ENGINEERING

3. DELVE INTO CHAOS ENGINEERING ON DISTRIBUTED SYSTEMS

4.EXPLORE THE APPLICATION OF CHAOS ENGINEERING IN YOUR COMPANY

5. LEARN HOW TO CRAFT YOUR OWN CHAOS ENGINEERING EXPERIMENTS

6. LEARN TECHNIQUES TO EVALUATE YOUR CHAOS ENGINEERING PRACTICE
@TAMMYBUTOW @ANA_M_MEDINA
+ LAYING THE FOUNDATIONS (9:00 - 10:00)

+ CHAOS ENGINEERING DISCUSSION (10:00 - 10:30)

+ MORNING BREAK (10:30 - 10:45) 

+ DISTRIBUTED SYSTEMS CHAOS (10:45 - 11:00)

+ CHAOS ENGINEERING IN YOUR COMPANY (11:00 - 11:15)

+ CRAFT YOUR OWN EXPERIMENTS (11:15- 11:30)

+ ADVANCED TOPICS & Q + A (11:30 - 12:00)
@TAMMYBUTOW @ANA_M_MEDINA
@TAMMYBUTOW @ANA_M_MEDINA
THOUGHTFUL PLANNED EXPERIMENTS 

DESIGNED TO REVEAL THE
WEAKNESSES 

IN OUR SYSTEMS.
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
@ANA_M_MEDINA
https://goo.gl/MEbiLj
@ANA_M_MEDINAhttps://www.datadoghq.com/workshop/gremlin/
v
IN GROUPS OF 3, 

SSH INTO YOUR 

KUBERNETES CLUSTER USING THE

CHAOS USER


VISIT YOUR PRIMARY IN 

YOUR BROWSER (PORT 30001)
TIME TO USE YOUR SERVERS
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
1. GO TO YOUR CHAOS REPO
$ su - experiments

$ cd chaosengineeringbootcamp
@TAMMYBUTOW @ANA_M_MEDINA
v
1. GO TO YOUR CHAOS REPO$ ls chaosengineeringbootcamp

$ chmod +x chaos_cpu.sh

$ ./chaos_cpu.sh

$ top
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
v
• v
$ pkill -u experiments
@TAMMYBUTOW @ANA_M_MEDINA
v
v
@TAMMYBUTOW @ANA_M_MEDINA
v
WHAT KIND OF CHAOS
CAN YOU INJECT?
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
+ LAYING THE FOUNDATIONS (9:00 - 10:00)

+ MORNING BREAK (10:30 - 10:45) 

+ CHAOS ENGINEERING DISCUSSION (10:45 - 11:00)

+ DISTRIBUTED SYSTEMS CHAOS (11:00 - 11:05)

+ CHAOS ENGINEERING IN YOUR COMPANY (11:05 - 11:15)

+ CRAFT YOUR OWN EXPERIMENTS (11:15- 11:30)

+ ADVANCED TOPICS & Q + A (11:30 - 12:00)
@TAMMYBUTOW @ANA_M_MEDINA
v
CHAOS DISCUSSION
@TAMMYBUTOW @ANA_M_MEDINA
CHAOS ENGINEERING DEBATE TIME
✦ FOUR VOLUNTEERS

✦ TWO TEAMS - 1 TEAM IS FOR, 1 TEAM IS AGAINST

✦ TOPIC: “EVERY COMPANY SHOULD BE DOING CHAOS ENGINEERING”

✦ EACH PERSON GETS A GO AND SPEAKS FOR 2 MINS MAX

✦ WE ALL VOTE ON A WINNER (APPLAUSE-O-METER)
@TAMMYBUTOW @ANA_M_MEDINA
+ LAYING THE FOUNDATIONS (9:00 - 10:00)

+ MORNING BREAK (10:30 - 10:45) 

+ CHAOS ENGINEERING DISCUSSION (10:45 - 11:00)

+ DISTRIBUTED SYSTEMS CHAOS (11:00 - 11:05)

+ CHAOS ENGINEERING IN YOUR COMPANY (11:05 - 11:15)

+ CRAFT YOUR OWN EXPERIMENTS (11:15- 11:30)

+ ADVANCED TOPICS & Q + A (11:30 - 12:00)
@TAMMYBUTOW @ANA_M_MEDINA
v
PART III: DISTRIBUTED SYSTEMS CHAOS
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
• ]
GREMLIN IS BUILDING A

CHAOS ENGINEERING PLATFORM. 

FIRST COMPANY FOUNDED TO DO THIS.

RUN GREMLIN AGENTS ON YOUR 

HOSTS OR IN CONTAINERS. 

11 PRE-BUILT ATTACKS.

SCHEDULE ATTACKS WITH THE UI, API OR CLI.

GREMLIN.COM

@GREMLININC

@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
+ LAYING THE FOUNDATIONS (9:00 - 10:00)

+ MORNING BREAK (10:30 - 10:45) 

+ CHAOS ENGINEERING DISCUSSION (10:45 - 11:00)

+ DISTRIBUTED SYSTEMS CHAOS (11:00 - 11:05)

+ CHAOS ENGINEERING IN YOUR COMPANY (11:05 - 11:15)

+ CRAFT YOUR OWN EXPERIMENTS (11:15- 11:30)

+ ADVANCED TOPICS & Q + A (11:30 - 12:00)
@TAMMYBUTOW @ANA_M_MEDINA
v
PART IV: CHAOS ENGINEERING IN
YOUR OWN COMPANY
@TAMMYBUTOW @ANA_M_MEDINA
+ LAYING THE FOUNDATIONS (9:00 - 10:00)

+ MORNING BREAK (10:30 - 10:45) 

+ CHAOS ENGINEERING DISCUSSION (10:45 - 11:00)

+ DISTRIBUTED SYSTEMS CHAOS (11:00 - 11:05)

+ CHAOS ENGINEERING IN YOUR COMPANY (11:05 - 11:15)

+ CRAFT YOUR OWN EXPERIMENTS (11:15- 11:30)

+ ADVANCED TOPICS & Q + A (11:30 - 12:00)
@TAMMYBUTOW @ANA_M_MEDINA
v
• v
PART V: CRAFT YOUR OWN
CHAOS ENGINEERING EXPERIMENTS
@TAMMYBUTOW @ANA_M_MEDINA
+ LAYING THE FOUNDATIONS (9:00 - 10:00)

+ MORNING BREAK (10:30 - 10:45) 

+ CHAOS ENGINEERING DISCUSSION (10:45 - 11:00)

+ DISTRIBUTED SYSTEMS CHAOS (11:00 - 11:05)

+ CHAOS ENGINEERING IN YOUR COMPANY (11:05 - 11:15)

+ CRAFT YOUR OWN EXPERIMENTS (11:15- 11:30)

+ ADVANCED TOPICS & Q + A (11:30 - 12:00)
@TAMMYBUTOW @ANA_M_MEDINA
v
• B
PART VII: ADVANCED CHAOS + Q & A
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW @ANA_M_MEDINA
v
v
v
HOW CAN YOU CONTINUE YOUR

CHAOS ENGINEERING JOURNEY?
@TAMMYBUTOW @ANA_M_MEDINA
v
gremlin.com/slack
@TAMMYBUTOW @ANA_M_MEDINA
v
• v
CONTINUE BREAKING THINGS
ON PURPOSE TOGETHER
@TAMMYBUTOW @ANA_M_MEDINA
v
@TAMMYBUTOW
@TAMMYBUTOW & @ANA_M_MEDINA

More Related Content

More from Ana Medina

Velocity London - Chaos Engineering Bootcamp
Velocity London - Chaos Engineering Bootcamp Velocity London - Chaos Engineering Bootcamp
Velocity London - Chaos Engineering Bootcamp Ana Medina
 
The Practice of Chaos Engineering - Reactive Summit 2018 - Montreal, QC
The Practice of Chaos Engineering - Reactive Summit 2018 - Montreal, QCThe Practice of Chaos Engineering - Reactive Summit 2018 - Montreal, QC
The Practice of Chaos Engineering - Reactive Summit 2018 - Montreal, QCAna Medina
 
DevOpsDays Kansas City - Getting Started with Chaos Engineering
DevOpsDays Kansas City - Getting Started with Chaos Engineering DevOpsDays Kansas City - Getting Started with Chaos Engineering
DevOpsDays Kansas City - Getting Started with Chaos Engineering Ana Medina
 
#AllDayDevOps Getting Started with Chaos Engineering
#AllDayDevOps Getting Started with Chaos Engineering#AllDayDevOps Getting Started with Chaos Engineering
#AllDayDevOps Getting Started with Chaos EngineeringAna Medina
 
Chaos Engineering with Kubernetes - Berlin / Hamburg Chaos Engineering Meetup...
Chaos Engineering with Kubernetes - Berlin / Hamburg Chaos Engineering Meetup...Chaos Engineering with Kubernetes - Berlin / Hamburg Chaos Engineering Meetup...
Chaos Engineering with Kubernetes - Berlin / Hamburg Chaos Engineering Meetup...Ana Medina
 
SRECon Europe - Chaos Engineering Bootcamp | August 2018
SRECon Europe - Chaos Engineering Bootcamp | August 2018 SRECon Europe - Chaos Engineering Bootcamp | August 2018
SRECon Europe - Chaos Engineering Bootcamp | August 2018 Ana Medina
 
Introduction to Chaos Engineering | SRECon Asia - Ana Medina
Introduction to Chaos Engineering | SRECon Asia - Ana MedinaIntroduction to Chaos Engineering | SRECon Asia - Ana Medina
Introduction to Chaos Engineering | SRECon Asia - Ana MedinaAna Medina
 

More from Ana Medina (7)

Velocity London - Chaos Engineering Bootcamp
Velocity London - Chaos Engineering Bootcamp Velocity London - Chaos Engineering Bootcamp
Velocity London - Chaos Engineering Bootcamp
 
The Practice of Chaos Engineering - Reactive Summit 2018 - Montreal, QC
The Practice of Chaos Engineering - Reactive Summit 2018 - Montreal, QCThe Practice of Chaos Engineering - Reactive Summit 2018 - Montreal, QC
The Practice of Chaos Engineering - Reactive Summit 2018 - Montreal, QC
 
DevOpsDays Kansas City - Getting Started with Chaos Engineering
DevOpsDays Kansas City - Getting Started with Chaos Engineering DevOpsDays Kansas City - Getting Started with Chaos Engineering
DevOpsDays Kansas City - Getting Started with Chaos Engineering
 
#AllDayDevOps Getting Started with Chaos Engineering
#AllDayDevOps Getting Started with Chaos Engineering#AllDayDevOps Getting Started with Chaos Engineering
#AllDayDevOps Getting Started with Chaos Engineering
 
Chaos Engineering with Kubernetes - Berlin / Hamburg Chaos Engineering Meetup...
Chaos Engineering with Kubernetes - Berlin / Hamburg Chaos Engineering Meetup...Chaos Engineering with Kubernetes - Berlin / Hamburg Chaos Engineering Meetup...
Chaos Engineering with Kubernetes - Berlin / Hamburg Chaos Engineering Meetup...
 
SRECon Europe - Chaos Engineering Bootcamp | August 2018
SRECon Europe - Chaos Engineering Bootcamp | August 2018 SRECon Europe - Chaos Engineering Bootcamp | August 2018
SRECon Europe - Chaos Engineering Bootcamp | August 2018
 
Introduction to Chaos Engineering | SRECon Asia - Ana Medina
Introduction to Chaos Engineering | SRECon Asia - Ana MedinaIntroduction to Chaos Engineering | SRECon Asia - Ana Medina
Introduction to Chaos Engineering | SRECon Asia - Ana Medina
 

Recently uploaded

Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
Azure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAzure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAndikSusilo4
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...HostedbyConfluent
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxnull - The Open Security Community
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsSnow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsHyundai Motor Group
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?XfilesPro
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Patryk Bandurski
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
Key Features Of Token Development (1).pptx
Key  Features Of Token  Development (1).pptxKey  Features Of Token  Development (1).pptx
Key Features Of Token Development (1).pptxLBM Solutions
 

Recently uploaded (20)

Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
Azure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAzure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & Application
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsSnow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
The transition to renewables in India.pdf
The transition to renewables in India.pdfThe transition to renewables in India.pdf
The transition to renewables in India.pdf
 
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?
 
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptxVulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
Key Features Of Token Development (1).pptx
Key  Features Of Token  Development (1).pptxKey  Features Of Token  Development (1).pptx
Key Features Of Token Development (1).pptx
 

Chaos Engineering Bootcamp - QCon SF 2018