SlideShare a Scribd company logo
@petecheslock
Everything @obfuscurity Taught
Me About Monitoring
Monitorama 2016
@petecheslock
“Just use Graphite”
- Jason Dixon
@petecheslock
Thank you
@petecheslock
Pete Cheslock
@petecheslock
Tech Ops & Support at Threat Stack
@petecheslock
get.threatstack.com/monitorama16
@petecheslock
constantly
@petecheslock
It is an AMAZING time to be
working in technology
@petecheslock
The “Good ‘ole Days”
kinda sucked
@petecheslock
@petecheslock
@petecheslock
@petecheslock
@petecheslock
@petecheslock
@petecheslock
I knew how to cd and ls
@petecheslock
# chown -Rf bob:bob files /
Whoops
@petecheslock
Should've used bcfg2?
<Path name="/home/file" mode="0644" group="bob" owner="bob">
<ACL type="access" scope="user" user="bob" perms="rwx"/>
</Path>
Won’t exist for 3+ more years…
@petecheslock
@petecheslock
ColdFusion
@petecheslock
* * * * * /scripts/mrtg-netmon
@petecheslock
@petecheslock
@petecheslock
@petecheslock
**PROBLEM Host Alert:
Everything Is FUCKED**
@petecheslock
@petecheslock
@petecheslock
Jen: Yo, are you ready to support ~10
beta customers on a completely
unautomated, unmonitored env?
Pete: New Phone, who dis?
Jen: 5 9s of availability.
@petecheslock
The Right Way™
@petecheslock
@petecheslock
@petecheslock
@petecheslock
@petecheslock
http://blog.packagecloud.io/eng/2016/04/05/the-definitive-guide-to-linux-system-calls/
@petecheslock
@petecheslock
Premature
optimization is the
root of all evil
@petecheslock
@petecheslock
@petecheslock
@petecheslock
collectd -> write_http ->
(60 Second)
@petecheslock
@petecheslock
@petecheslock
@petecheslock
@petecheslock
collectd -> write_http ->
(60 Second)
statsd plugin
app1 app2 appN
@petecheslock
@petecheslock
• Hosted TSDB is super useful and “just works”
• The opportunity cost of fully replacing it was
too high.
@petecheslock
• Use the same tools already deployed
• Devs wrote libraries to send metrics to statsd
• Don’t want to “redeploy the world”
• Limited time to invest
@petecheslock
• Use the same tools already deployed
• Devs wrote libraries to send metrics to statsd
• Don’t want to “redeploy the world”
• Limited time to invest
@petecheslock
collectd -> write_http ->
(60 Second)
collectd -> write_graphite ->
(10 Second)
statsd plugin
statsd plugin
@petecheslock
@petecheslock
https://github.com/obfuscurity/
synthesize
https://github.com/hw-cookbooks/
graphite
@petecheslock
https://github.com/obfuscurity/
synthesize
https://github.com/hw-cookbooks/
graphite
@petecheslock
@petecheslock
Want your Devs to Ops?
Build Consumable Services
@petecheslock
@petecheslock
@petecheslock
@petecheslock
collectd -> write_graphite ->
(10 Second)
statsd_plugin
app1 app2 appN
@petecheslock
A lot of people must have suddenly
just stopped caring
@petecheslock
Community Matters
Relationships Matter
@petecheslock
You are (probably)
not Google
@petecheslock
and that is OK
(Solve YOUR Problems)
get.threatstack.com/monitorama16
Thank You!

More Related Content

Viewers also liked

Userguide eeeims en_v1.0
Userguide eeeims en_v1.0Userguide eeeims en_v1.0
Userguide eeeims en_v1.0Abetu Bope
 
Assalamu’alaikum
Assalamu’alaikumAssalamu’alaikum
Assalamu’alaikumerlin_tyas
 
Why We Can't Have Nice Things, A Tale of Woe and a Hope For the Future
Why We Can't Have Nice Things, A Tale of Woe and a Hope For the FutureWhy We Can't Have Nice Things, A Tale of Woe and a Hope For the Future
Why We Can't Have Nice Things, A Tale of Woe and a Hope For the FuturePete Cheslock
 
Brico montaje soporte para tableta
Brico montaje soporte para tabletaBrico montaje soporte para tableta
Brico montaje soporte para tabletaALGOIG
 
Art and development of the town of angono
Art and development of the town of angonoArt and development of the town of angono
Art and development of the town of angonoChe Berto
 
Taite wk 1 term 3 13pdf2
Taite wk 1 term 3 13pdf2Taite wk 1 term 3 13pdf2
Taite wk 1 term 3 13pdf2takp
 
Turei wk 10 term 2 2013
Turei wk 10 term 2 2013Turei wk 10 term 2 2013
Turei wk 10 term 2 2013takp
 
Section B ( Know your vocabulary)
Section B ( Know your vocabulary)Section B ( Know your vocabulary)
Section B ( Know your vocabulary)Rafidah Roslan
 
One-min Intro to Spectator.do
One-min Intro to Spectator.do One-min Intro to Spectator.do
One-min Intro to Spectator.do Bao-Wen Chen
 
Turei week 1 term 4 pdf
Turei week 1 term 4 pdfTurei week 1 term 4 pdf
Turei week 1 term 4 pdftakp
 
Actividades casa por ambiente vertical
Actividades casa por ambiente verticalActividades casa por ambiente vertical
Actividades casa por ambiente verticaltommypalmieri
 
The Future of Digital Textbooks, Tools of Change, 2010
The Future of Digital Textbooks, Tools of Change, 2010The Future of Digital Textbooks, Tools of Change, 2010
The Future of Digital Textbooks, Tools of Change, 2010John Warren
 

Viewers also liked (20)

Userguide eeeims en_v1.0
Userguide eeeims en_v1.0Userguide eeeims en_v1.0
Userguide eeeims en_v1.0
 
Assalamu’alaikum
Assalamu’alaikumAssalamu’alaikum
Assalamu’alaikum
 
Gbi trabajo
Gbi trabajoGbi trabajo
Gbi trabajo
 
Visita a la usina del arte
Visita a la usina del arteVisita a la usina del arte
Visita a la usina del arte
 
Why We Can't Have Nice Things, A Tale of Woe and a Hope For the Future
Why We Can't Have Nice Things, A Tale of Woe and a Hope For the FutureWhy We Can't Have Nice Things, A Tale of Woe and a Hope For the Future
Why We Can't Have Nice Things, A Tale of Woe and a Hope For the Future
 
Brico montaje soporte para tableta
Brico montaje soporte para tabletaBrico montaje soporte para tableta
Brico montaje soporte para tableta
 
Muka depan sp hsp
Muka depan sp hspMuka depan sp hsp
Muka depan sp hsp
 
Art and development of the town of angono
Art and development of the town of angonoArt and development of the town of angono
Art and development of the town of angono
 
Rujak buah
Rujak buahRujak buah
Rujak buah
 
Taite wk 1 term 3 13pdf2
Taite wk 1 term 3 13pdf2Taite wk 1 term 3 13pdf2
Taite wk 1 term 3 13pdf2
 
Kelompok 4 wirus
Kelompok 4 wirusKelompok 4 wirus
Kelompok 4 wirus
 
Turei wk 10 term 2 2013
Turei wk 10 term 2 2013Turei wk 10 term 2 2013
Turei wk 10 term 2 2013
 
Section B ( Know your vocabulary)
Section B ( Know your vocabulary)Section B ( Know your vocabulary)
Section B ( Know your vocabulary)
 
One-min Intro to Spectator.do
One-min Intro to Spectator.do One-min Intro to Spectator.do
One-min Intro to Spectator.do
 
PERSONALFILE
PERSONALFILEPERSONALFILE
PERSONALFILE
 
Turei week 1 term 4 pdf
Turei week 1 term 4 pdfTurei week 1 term 4 pdf
Turei week 1 term 4 pdf
 
Pre.con.ex.
Pre.con.ex.Pre.con.ex.
Pre.con.ex.
 
T301
T301T301
T301
 
Actividades casa por ambiente vertical
Actividades casa por ambiente verticalActividades casa por ambiente vertical
Actividades casa por ambiente vertical
 
The Future of Digital Textbooks, Tools of Change, 2010
The Future of Digital Textbooks, Tools of Change, 2010The Future of Digital Textbooks, Tools of Change, 2010
The Future of Digital Textbooks, Tools of Change, 2010
 

More from Pete Cheslock

How to keep the people you need
How to keep the people you needHow to keep the people you need
How to keep the people you needPete Cheslock
 
Pick Any Three: Good, Fast, or Safe - Devops from Scratch
Pick Any Three: Good, Fast, or Safe - Devops from ScratchPick Any Three: Good, Fast, or Safe - Devops from Scratch
Pick Any Three: Good, Fast, or Safe - Devops from ScratchPete Cheslock
 
DevOpsDays - Pick any Three - Devops from scratch
DevOpsDays - Pick any Three - Devops from scratchDevOpsDays - Pick any Three - Devops from scratch
DevOpsDays - Pick any Three - Devops from scratchPete Cheslock
 
Everyone has a plan until... Automacon16
Everyone has a plan until...  Automacon16Everyone has a plan until...  Automacon16
Everyone has a plan until... Automacon16Pete Cheslock
 
DevOpsDays PGH: How to Fail With One Weird Trick
DevOpsDays PGH:  How to Fail With One Weird TrickDevOpsDays PGH:  How to Fail With One Weird Trick
DevOpsDays PGH: How to Fail With One Weird TrickPete Cheslock
 
A Tale of Two Workflows - ChefConf 2014
A Tale of Two Workflows - ChefConf 2014A Tale of Two Workflows - ChefConf 2014
A Tale of Two Workflows - ChefConf 2014Pete Cheslock
 
Recruiting is Broken - How Do We Fix It
Recruiting is Broken - How Do We Fix ItRecruiting is Broken - How Do We Fix It
Recruiting is Broken - How Do We Fix ItPete Cheslock
 
Sonian, Open Source and Sensu
Sonian, Open Source and SensuSonian, Open Source and Sensu
Sonian, Open Source and SensuPete Cheslock
 
Chef boston-workflows
Chef boston-workflowsChef boston-workflows
Chef boston-workflowsPete Cheslock
 

More from Pete Cheslock (11)

How to keep the people you need
How to keep the people you needHow to keep the people you need
How to keep the people you need
 
Pick Any Three: Good, Fast, or Safe - Devops from Scratch
Pick Any Three: Good, Fast, or Safe - Devops from ScratchPick Any Three: Good, Fast, or Safe - Devops from Scratch
Pick Any Three: Good, Fast, or Safe - Devops from Scratch
 
DevOpsDays - Pick any Three - Devops from scratch
DevOpsDays - Pick any Three - Devops from scratchDevOpsDays - Pick any Three - Devops from scratch
DevOpsDays - Pick any Three - Devops from scratch
 
The Vasa Redux
The Vasa ReduxThe Vasa Redux
The Vasa Redux
 
Everyone has a plan until... Automacon16
Everyone has a plan until...  Automacon16Everyone has a plan until...  Automacon16
Everyone has a plan until... Automacon16
 
DevOpsDays PGH: How to Fail With One Weird Trick
DevOpsDays PGH:  How to Fail With One Weird TrickDevOpsDays PGH:  How to Fail With One Weird Trick
DevOpsDays PGH: How to Fail With One Weird Trick
 
A Tale of Two Workflows - ChefConf 2014
A Tale of Two Workflows - ChefConf 2014A Tale of Two Workflows - ChefConf 2014
A Tale of Two Workflows - ChefConf 2014
 
Recruiting is Broken - How Do We Fix It
Recruiting is Broken - How Do We Fix ItRecruiting is Broken - How Do We Fix It
Recruiting is Broken - How Do We Fix It
 
Rubix cube
Rubix cubeRubix cube
Rubix cube
 
Sonian, Open Source and Sensu
Sonian, Open Source and SensuSonian, Open Source and Sensu
Sonian, Open Source and Sensu
 
Chef boston-workflows
Chef boston-workflowsChef boston-workflows
Chef boston-workflows
 

Recently uploaded

Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backElena Simperl
 
UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1DianaGray10
 
In-Depth Performance Testing Guide for IT Professionals
In-Depth Performance Testing Guide for IT ProfessionalsIn-Depth Performance Testing Guide for IT Professionals
In-Depth Performance Testing Guide for IT ProfessionalsExpeed Software
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxAbida Shariff
 
IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoTAnalytics
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...Elena Simperl
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...Product School
 
ODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User GroupODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User GroupCatarinaPereira64715
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
 
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya HalderCustom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya HalderCzechDreamin
 
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...CzechDreamin
 
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Julian Hyde
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...Product School
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Thierry Lestable
 
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...CzechDreamin
 
AI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří KarpíšekAI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří KarpíšekCzechDreamin
 

Recently uploaded (20)

Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical Futures
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
 
UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1
 
In-Depth Performance Testing Guide for IT Professionals
In-Depth Performance Testing Guide for IT ProfessionalsIn-Depth Performance Testing Guide for IT Professionals
In-Depth Performance Testing Guide for IT Professionals
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
 
IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
 
ODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User GroupODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User Group
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi
 
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya HalderCustom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
 
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
 
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
 
AI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří KarpíšekAI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří Karpíšek
 

Everything obfuscurity taught me about monitoring