SlideShare a Scribd company logo
The Snake Oil Cycle
Dave Cole, dave@openraven.com
OPEN RAVEN
Modern Attack Surface Management
Meet Clark Stanley
It’s not about Clark or even a type of person
Genuine problems lay beyond the
The obvious stuff
If you did a great job, many times nothing
bad will happen. Which is impossible to
prove. The really compelling “before &
after” stories are elusive.
I’m going to be unfair &
pick on another product
category & a single
company as an example.
Imagine this in security, esp
in the middle of an incident…
measure progress in months!
Standard expectation: real-
time or very close to it.
Our expected uninvited guests
I’m sure it’s hard to build an HRIS, but their
Alice & Bob don’t have an Eve. Or a Mallory.
Or a Trudy.
Or re-architect because a new attack type
changed their original assumptions.
Or crank a new remediation engine
because they can’t remove the latest
threat.
Or make sure that OS change didn’t just
bust your behavioral defenses.
Or properly identify every IoT device.
A short
story.
Building security products is hard.
Being a security professional is hard.
Natural empathy should drive us together.
It rarely does.
The U.S. employs
nearly 716,000 people
in cybersecurity
positions, with
approximately 314,000
current cybersecurity
openings
https://www.cyberseek.org/heatmap.html
Breach Level Index 2018
https://blog.rapid7.com/2018/04/30/cve-100k-by-the-numbers/
https://blog.rapid7.com/2018/04/30/cve-100k-by-the-numbers/
“7% of all S3 buckets have unrestricted
public access, and 35% are unencrypted”
SkyHigh Networks (now McAfee) September 2017… before Amazon changed policy settings to proactively warn users of public buckets
Who When What
Booz Allen Hamilton May 2017 Battlefield imagery & admin credentials to sensitive systems
Deep Root Analytics June 2017 Personal data about 198 million American voters
Dow Jones & Co July 2017 Personally identifiable information for 2.2 million people
WWE July 2017 Personally identifiable information of >3 million wrestling fans
Verizon Wireless July/Sept 2017 Personally identifiable information of ~6 million people &
sensitive corp. information about IT systems, incl. login creds
Time Warner Cable Sept 2017 Personally identifiable information about 4 million customers,
proprietary code & administrator creds
Pentagon Exposures Sept/Nov 2017 TBs of info from spying archive, resume for intelligence
positions—incl. security clearance & ops history, creds &
metadata from an intra-agency intelligence sharing platform
Accenture Sept 2017 Master access keys for Accenture's account w/ AWS Key Mgmt
System, plaintext customer password DBs & proprietary API data
Natl. Credit Federation Nov 2017 111GB financial info—incl. full credit reports-- ~47K people
Alteryx Dec 2017 Personal info of ~123 million American households
https://businessinsights.bitdefender.com/worst-amazon-breaches
On average, sec teams are understaffed,
undertrained, under intense pressure from
increasing threats & vulnerabilities with
regulatory mandates to make fast
decisions… & circumstances can change
w/o warning.
Let’s try to find them some help...
Security design can sometimes look like this
Which leads to this
Historically we have assumed the person
behind the console is an expert
Increasingly, we have products designed for
experts in the hands of novices.
The bar must be lower.
Before we move on, a word on how products
are evaluated prior to purchasing*
*This is hard to generalize, your mileage may vary
Proof of Concept
Normal Operations
OMG Operations
What if someone had a solid test lab that we
could all lean on for quality, unbiased
product reviews?
There’s no genuine equivalent for the
security community… why?
They are spread too thin, like every other
journalist on the 24x7 coverage beat.
They also have the same pressures as
political journalists– if you’re overly critical
you lose access.
They typically do not have the lab
environments they require. Most vendors
have someone dedicated to lab ops at scale.
Even if they had the labs they need, the skill
set required to test one product is not what
you need to test the next product.
And if they truly were this skilled, they would
be unlikely to remain a journalist for long.
So our security reviewers are unlikely to
have the focus, objectivity, labs & skills they
need to publish great reviews. And if they
did, we would hire them in a heartbeat.
What about the industry analysts?
Same
• Increasingly spread too thin; lack of focus
• Typically little or no experience as
practitioner
Worse
• Purely anecdotal-- no real tests
• $$$ of a subscription; not readily available
Better
• More direct customer feedback
• In-depth reports w/ longer horizon
AntiVirus Industry – An Exception
What Opinion
File scanning tests Worked fine until this was only a subset of defenses… early 2000s
On access tests About the same as file scanning
Retrospective tests Purgatory b/w file scanning & real world tests; misleading
Real world tests When done properly, effective; rarely done properly
Remediation tests Few and far in between…
Performance tests Largely a success, materially improved the industry
At their best, public tests push vendors to
make better products, but they are too often
designed poorly or easily misunderstood.
Usually, they simply don’t exist.
So what’s a security marketer supposed to
do?
Hypothetical example: new scan engine
Product Person
“Our year long effort to create an awesome network
scanner that we can improve more rapidly and
support better in the future just completed! In some
instances, you’re going to see some nice speed
improvements.”
Marketing
“Our year long effort to create a strong network
scanner that we can improve more rapidly and
support better in the future just completed! In some
instances, you’re going to see some nice speed
improvements.”
Customer value not
readily apparent to many
Vague
Post Marketing
“New scan engine frequently boosts scans by 30%,
reducing time needed to scan and easing network
impact in many instances.”
Customer
“New scan engine boosts scans by 30%,frequently
reducing time needed to scan and easing network
impact in many instances.”
Recap
Opaque product benefit
↓
Well-intentioned attempt at clear message
↓
Misunderstanding due to nuance, pressure
↓
Disappointment & loss of trust
Marketing security products often requires a
strong grasp of nuance, understanding not
only the tech, but also how the customer will
receive the message. These people are rare.
And even if they are really that good, things
change quickly. And they change security
markets… which changes everything.
Marketing is also affected by how dynamic
security companies are– more so than other
functions. And more so than other areas in
software.
Often distracted (2018 #s)
• 184 M&A transaction
• 406 Financing transactions
Even if you can avoid the distractions, any
company will change significantly as it
briskly moves from start-
up→growth→behemoth
This often alienates employees & users
“I have never seen such a fast-growing market
with so many companies on the losing side”
David Cowan, Partner, Bessemer Venture Partners
Under threat: Cyber security startups fall on harder times
Reuters, January 17, 2018
If you knew you had to show strong sales
performance to establish your company, but
were unsure of its longevity, how would you
build & sell your product?
Given the rough & tumble nature of the
industry, far too often companies optimize for
creating the “reach for your wallet” moment
over establishing a strong foundation
Closing Thoughts
Breaking the Cycle
It’s too easy to blame any one person or
type of person, the problem is pervasive
& self-reinforcing
Security Professionals
1. Do a process walk thru w/ vendor or 3rd party
2. Do whatever you can to best simulate genuine
production conditions before purchasing
3. Ask for a “reverse roadmap”
4. Insist on references, preferably long-term
ones
5. Take updates & new versions ASAP
Security Vendors
1. Definition of done: customer is successful
2. Hire & empower a real design team
3. Build out a complete customer feedback loop
4. If no one else will do it, produce your own
metrics & be ready to share & explain them
5. Invite Sales & Marketing into “the factory”
Thanks
Dave Cole, dave@openraven.com

More Related Content

What's hot

Invessed Webinar: Investor Portals are not Rocket Science
Invessed Webinar: Investor Portals are not Rocket ScienceInvessed Webinar: Investor Portals are not Rocket Science
Invessed Webinar: Investor Portals are not Rocket Science
Theo Paraskevopoulos
 
Cloud, DevOps and the New Security Practitioner
Cloud, DevOps and the New Security PractitionerCloud, DevOps and the New Security Practitioner
Cloud, DevOps and the New Security Practitioner
Adrian Sanabria
 
PredictiMx H4D Stanford 2019
PredictiMx H4D Stanford 2019PredictiMx H4D Stanford 2019
PredictiMx H4D Stanford 2019
Stanford University
 
Ops Happen: Improve Security Without Getting in the Way
Ops Happen: Improve Security Without Getting in the WayOps Happen: Improve Security Without Getting in the Way
Ops Happen: Improve Security Without Getting in the Way
SeniorStoryteller
 
Big Bang Theory: The Evolution of Pentesting High Security Environments
Big Bang Theory: The Evolution of Pentesting High Security EnvironmentsBig Bang Theory: The Evolution of Pentesting High Security Environments
Big Bang Theory: The Evolution of Pentesting High Security Environments
Chris Gates
 
PCI DSS: Myths, Mistakes, Misconceptions 2009 - TEASER Version
PCI DSS: Myths, Mistakes, Misconceptions 2009 - TEASER VersionPCI DSS: Myths, Mistakes, Misconceptions 2009 - TEASER Version
PCI DSS: Myths, Mistakes, Misconceptions 2009 - TEASER Version
guest3af00b8
 
Business Continuity for Humans: Keeping Your Business Running When Your Peopl...
Business Continuity for Humans: Keeping Your Business Running When Your Peopl...Business Continuity for Humans: Keeping Your Business Running When Your Peopl...
Business Continuity for Humans: Keeping Your Business Running When Your Peopl...
Rundeck
 
Salus H4D 2021 Lessons Learned
Salus H4D 2021 Lessons LearnedSalus H4D 2021 Lessons Learned
Salus H4D 2021 Lessons Learned
Stanford University
 
How to Replace Your Legacy Antivirus Solution with CrowdStrike
How to Replace Your Legacy Antivirus Solution with CrowdStrikeHow to Replace Your Legacy Antivirus Solution with CrowdStrike
How to Replace Your Legacy Antivirus Solution with CrowdStrike
Adam Barrera
 
PCI DSS Myths 2009: Myths and Reality
PCI DSS Myths 2009: Myths and RealityPCI DSS Myths 2009: Myths and Reality
PCI DSS Myths 2009: Myths and Reality
Anton Chuvakin
 
What we learned from three years sciencing the crap out of devops
What we learned from three years sciencing the crap out of devopsWhat we learned from three years sciencing the crap out of devops
What we learned from three years sciencing the crap out of devops
Nicole Forsgren
 
Stranded on Infosec Island: Defending the Enterprise with Nothing but Windows...
Stranded on Infosec Island: Defending the Enterprise with Nothing but Windows...Stranded on Infosec Island: Defending the Enterprise with Nothing but Windows...
Stranded on Infosec Island: Defending the Enterprise with Nothing but Windows...
Adrian Sanabria
 
Управление рисками: как перестать верить в иллюзии
Управление рисками: как перестать верить в иллюзииУправление рисками: как перестать верить в иллюзии
Управление рисками: как перестать верить в иллюзии
Positive Hack Days
 
PCI DSS Myths 2010: Why Are They STILL Alive by Dr. Anton Chuvakin
PCI DSS Myths 2010: Why Are They STILL Alive by Dr. Anton ChuvakinPCI DSS Myths 2010: Why Are They STILL Alive by Dr. Anton Chuvakin
PCI DSS Myths 2010: Why Are They STILL Alive by Dr. Anton Chuvakin
Anton Chuvakin
 
The R.O.A.D to DevOps
The R.O.A.D to DevOpsThe R.O.A.D to DevOps
The R.O.A.D to DevOps
SeniorStoryteller
 
MITRE ATT&CKcon 2.0: Keynote Address - The Friends We Made Along the Way; Ton...
MITRE ATT&CKcon 2.0: Keynote Address - The Friends We Made Along the Way; Ton...MITRE ATT&CKcon 2.0: Keynote Address - The Friends We Made Along the Way; Ton...
MITRE ATT&CKcon 2.0: Keynote Address - The Friends We Made Along the Way; Ton...
MITRE - ATT&CKcon
 

What's hot (17)

Invessed Webinar: Investor Portals are not Rocket Science
Invessed Webinar: Investor Portals are not Rocket ScienceInvessed Webinar: Investor Portals are not Rocket Science
Invessed Webinar: Investor Portals are not Rocket Science
 
Cloud, DevOps and the New Security Practitioner
Cloud, DevOps and the New Security PractitionerCloud, DevOps and the New Security Practitioner
Cloud, DevOps and the New Security Practitioner
 
PredictiMx H4D Stanford 2019
PredictiMx H4D Stanford 2019PredictiMx H4D Stanford 2019
PredictiMx H4D Stanford 2019
 
Ops Happen: Improve Security Without Getting in the Way
Ops Happen: Improve Security Without Getting in the WayOps Happen: Improve Security Without Getting in the Way
Ops Happen: Improve Security Without Getting in the Way
 
Big Bang Theory: The Evolution of Pentesting High Security Environments
Big Bang Theory: The Evolution of Pentesting High Security EnvironmentsBig Bang Theory: The Evolution of Pentesting High Security Environments
Big Bang Theory: The Evolution of Pentesting High Security Environments
 
PCI DSS: Myths, Mistakes, Misconceptions 2009 - TEASER Version
PCI DSS: Myths, Mistakes, Misconceptions 2009 - TEASER VersionPCI DSS: Myths, Mistakes, Misconceptions 2009 - TEASER Version
PCI DSS: Myths, Mistakes, Misconceptions 2009 - TEASER Version
 
Business Continuity for Humans: Keeping Your Business Running When Your Peopl...
Business Continuity for Humans: Keeping Your Business Running When Your Peopl...Business Continuity for Humans: Keeping Your Business Running When Your Peopl...
Business Continuity for Humans: Keeping Your Business Running When Your Peopl...
 
Salus H4D 2021 Lessons Learned
Salus H4D 2021 Lessons LearnedSalus H4D 2021 Lessons Learned
Salus H4D 2021 Lessons Learned
 
How to Replace Your Legacy Antivirus Solution with CrowdStrike
How to Replace Your Legacy Antivirus Solution with CrowdStrikeHow to Replace Your Legacy Antivirus Solution with CrowdStrike
How to Replace Your Legacy Antivirus Solution with CrowdStrike
 
PCI DSS Myths 2009: Myths and Reality
PCI DSS Myths 2009: Myths and RealityPCI DSS Myths 2009: Myths and Reality
PCI DSS Myths 2009: Myths and Reality
 
What we learned from three years sciencing the crap out of devops
What we learned from three years sciencing the crap out of devopsWhat we learned from three years sciencing the crap out of devops
What we learned from three years sciencing the crap out of devops
 
Stranded on Infosec Island: Defending the Enterprise with Nothing but Windows...
Stranded on Infosec Island: Defending the Enterprise with Nothing but Windows...Stranded on Infosec Island: Defending the Enterprise with Nothing but Windows...
Stranded on Infosec Island: Defending the Enterprise with Nothing but Windows...
 
Управление рисками: как перестать верить в иллюзии
Управление рисками: как перестать верить в иллюзииУправление рисками: как перестать верить в иллюзии
Управление рисками: как перестать верить в иллюзии
 
PCI DSS Myths 2010: Why Are They STILL Alive by Dr. Anton Chuvakin
PCI DSS Myths 2010: Why Are They STILL Alive by Dr. Anton ChuvakinPCI DSS Myths 2010: Why Are They STILL Alive by Dr. Anton Chuvakin
PCI DSS Myths 2010: Why Are They STILL Alive by Dr. Anton Chuvakin
 
Challenge_Opp_Usha
Challenge_Opp_UshaChallenge_Opp_Usha
Challenge_Opp_Usha
 
The R.O.A.D to DevOps
The R.O.A.D to DevOpsThe R.O.A.D to DevOps
The R.O.A.D to DevOps
 
MITRE ATT&CKcon 2.0: Keynote Address - The Friends We Made Along the Way; Ton...
MITRE ATT&CKcon 2.0: Keynote Address - The Friends We Made Along the Way; Ton...MITRE ATT&CKcon 2.0: Keynote Address - The Friends We Made Along the Way; Ton...
MITRE ATT&CKcon 2.0: Keynote Address - The Friends We Made Along the Way; Ton...
 

Similar to Security Snake Oil Cycle 2019

Vendors, and Risk, and Tigers, and Bears, Oh My: How to Create a Vendor Revie...
Vendors, and Risk, and Tigers, and Bears, Oh My: How to Create a Vendor Revie...Vendors, and Risk, and Tigers, and Bears, Oh My: How to Create a Vendor Revie...
Vendors, and Risk, and Tigers, and Bears, Oh My: How to Create a Vendor Revie...
Wendy Knox Everette
 
2010 04 28 The Lean Startup webinar for the Lean Enterprise Institute
2010 04 28 The Lean Startup webinar for the Lean Enterprise Institute2010 04 28 The Lean Startup webinar for the Lean Enterprise Institute
2010 04 28 The Lean Startup webinar for the Lean Enterprise Institute
Eric Ries
 
Protecting the Core of Your Network
Protecting the Core of Your Network Protecting the Core of Your Network
Protecting the Core of Your Network
Mighty Guides, Inc.
 
(ISC)2 Security Congress 2015 - The Cloud Trust Conundrum- You’re Asking all ...
(ISC)2 Security Congress 2015 - The Cloud Trust Conundrum- You’re Asking all ...(ISC)2 Security Congress 2015 - The Cloud Trust Conundrum- You’re Asking all ...
(ISC)2 Security Congress 2015 - The Cloud Trust Conundrum- You’re Asking all ...
Andrew O. Leeth
 
January 2017 Printed Newsletter
January 2017 Printed NewsletterJanuary 2017 Printed Newsletter
January 2017 Printed Newsletter
Yigal Behar
 
React Faster and Better: New Approaches for Advanced Incident Response
React Faster and Better: New Approaches for Advanced Incident ResponseReact Faster and Better: New Approaches for Advanced Incident Response
React Faster and Better: New Approaches for Advanced Incident Response
SilvioPappalardo
 
2010 10 19 the lean startup workshop for i_gap ireland
2010 10 19 the lean startup workshop for i_gap ireland2010 10 19 the lean startup workshop for i_gap ireland
2010 10 19 the lean startup workshop for i_gap irelandEric Ries
 
Selling 0days to governments and offensive security companies
Selling 0days to governments and offensive security companiesSelling 0days to governments and offensive security companies
Selling 0days to governments and offensive security companies
Maor Shwartz
 
InfinityQS_7 Habits of Quality Obsessed Manufacturers_Print final
InfinityQS_7 Habits of Quality Obsessed Manufacturers_Print finalInfinityQS_7 Habits of Quality Obsessed Manufacturers_Print final
InfinityQS_7 Habits of Quality Obsessed Manufacturers_Print finalVanessa Stirling
 
The Black Report - Hackers
The Black Report - HackersThe Black Report - Hackers
The Black Report - Hackers
Dendreon
 
Giving Organisations new capabilities to ask the right business questions 1.7
Giving Organisations new capabilities to ask the right business questions 1.7Giving Organisations new capabilities to ask the right business questions 1.7
Giving Organisations new capabilities to ask the right business questions 1.7
OReillyStrata
 
Selling Infosec to the CSuite
Selling Infosec to the CSuiteSelling Infosec to the CSuite
Selling Infosec to the CSuiteDave R. Taylor
 
Cybercrime and the Developer Java2Days 2016 Sofia
Cybercrime and the Developer Java2Days 2016 SofiaCybercrime and the Developer Java2Days 2016 Sofia
Cybercrime and the Developer Java2Days 2016 Sofia
Steve Poole
 
SOC Analyst Guide For Beginners SOC analysts work as members of a managed sec...
SOC Analyst Guide For Beginners SOC analysts work as members of a managed sec...SOC Analyst Guide For Beginners SOC analysts work as members of a managed sec...
SOC Analyst Guide For Beginners SOC analysts work as members of a managed sec...
Varun Mithran
 
WhiteHat’s Website Security Statistics Report 2015
WhiteHat’s Website Security Statistics Report 2015WhiteHat’s Website Security Statistics Report 2015
WhiteHat’s Website Security Statistics Report 2015Jeremiah Grossman
 
2009 10 28 The Lean Startup In Paris
2009 10 28 The Lean Startup In Paris2009 10 28 The Lean Startup In Paris
2009 10 28 The Lean Startup In ParisEric Ries
 
Ingram micro physical security ebook
Ingram micro physical security ebookIngram micro physical security ebook
Ingram micro physical security ebook
Jason Destein, ABCHS III
 
IT Security - TestArmy
IT Security - TestArmy IT Security - TestArmy
IT Security - TestArmy
TestArmy
 
Tech Talent Meetup Hacking Security Event Recap
Tech Talent Meetup Hacking Security Event RecapTech Talent Meetup Hacking Security Event Recap
Tech Talent Meetup Hacking Security Event RecapDominic Vogel
 
Einstein Analytics Prediction Builder
Einstein Analytics Prediction BuilderEinstein Analytics Prediction Builder
Einstein Analytics Prediction Builder
rikkehovgaard
 

Similar to Security Snake Oil Cycle 2019 (20)

Vendors, and Risk, and Tigers, and Bears, Oh My: How to Create a Vendor Revie...
Vendors, and Risk, and Tigers, and Bears, Oh My: How to Create a Vendor Revie...Vendors, and Risk, and Tigers, and Bears, Oh My: How to Create a Vendor Revie...
Vendors, and Risk, and Tigers, and Bears, Oh My: How to Create a Vendor Revie...
 
2010 04 28 The Lean Startup webinar for the Lean Enterprise Institute
2010 04 28 The Lean Startup webinar for the Lean Enterprise Institute2010 04 28 The Lean Startup webinar for the Lean Enterprise Institute
2010 04 28 The Lean Startup webinar for the Lean Enterprise Institute
 
Protecting the Core of Your Network
Protecting the Core of Your Network Protecting the Core of Your Network
Protecting the Core of Your Network
 
(ISC)2 Security Congress 2015 - The Cloud Trust Conundrum- You’re Asking all ...
(ISC)2 Security Congress 2015 - The Cloud Trust Conundrum- You’re Asking all ...(ISC)2 Security Congress 2015 - The Cloud Trust Conundrum- You’re Asking all ...
(ISC)2 Security Congress 2015 - The Cloud Trust Conundrum- You’re Asking all ...
 
January 2017 Printed Newsletter
January 2017 Printed NewsletterJanuary 2017 Printed Newsletter
January 2017 Printed Newsletter
 
React Faster and Better: New Approaches for Advanced Incident Response
React Faster and Better: New Approaches for Advanced Incident ResponseReact Faster and Better: New Approaches for Advanced Incident Response
React Faster and Better: New Approaches for Advanced Incident Response
 
2010 10 19 the lean startup workshop for i_gap ireland
2010 10 19 the lean startup workshop for i_gap ireland2010 10 19 the lean startup workshop for i_gap ireland
2010 10 19 the lean startup workshop for i_gap ireland
 
Selling 0days to governments and offensive security companies
Selling 0days to governments and offensive security companiesSelling 0days to governments and offensive security companies
Selling 0days to governments and offensive security companies
 
InfinityQS_7 Habits of Quality Obsessed Manufacturers_Print final
InfinityQS_7 Habits of Quality Obsessed Manufacturers_Print finalInfinityQS_7 Habits of Quality Obsessed Manufacturers_Print final
InfinityQS_7 Habits of Quality Obsessed Manufacturers_Print final
 
The Black Report - Hackers
The Black Report - HackersThe Black Report - Hackers
The Black Report - Hackers
 
Giving Organisations new capabilities to ask the right business questions 1.7
Giving Organisations new capabilities to ask the right business questions 1.7Giving Organisations new capabilities to ask the right business questions 1.7
Giving Organisations new capabilities to ask the right business questions 1.7
 
Selling Infosec to the CSuite
Selling Infosec to the CSuiteSelling Infosec to the CSuite
Selling Infosec to the CSuite
 
Cybercrime and the Developer Java2Days 2016 Sofia
Cybercrime and the Developer Java2Days 2016 SofiaCybercrime and the Developer Java2Days 2016 Sofia
Cybercrime and the Developer Java2Days 2016 Sofia
 
SOC Analyst Guide For Beginners SOC analysts work as members of a managed sec...
SOC Analyst Guide For Beginners SOC analysts work as members of a managed sec...SOC Analyst Guide For Beginners SOC analysts work as members of a managed sec...
SOC Analyst Guide For Beginners SOC analysts work as members of a managed sec...
 
WhiteHat’s Website Security Statistics Report 2015
WhiteHat’s Website Security Statistics Report 2015WhiteHat’s Website Security Statistics Report 2015
WhiteHat’s Website Security Statistics Report 2015
 
2009 10 28 The Lean Startup In Paris
2009 10 28 The Lean Startup In Paris2009 10 28 The Lean Startup In Paris
2009 10 28 The Lean Startup In Paris
 
Ingram micro physical security ebook
Ingram micro physical security ebookIngram micro physical security ebook
Ingram micro physical security ebook
 
IT Security - TestArmy
IT Security - TestArmy IT Security - TestArmy
IT Security - TestArmy
 
Tech Talent Meetup Hacking Security Event Recap
Tech Talent Meetup Hacking Security Event RecapTech Talent Meetup Hacking Security Event Recap
Tech Talent Meetup Hacking Security Event Recap
 
Einstein Analytics Prediction Builder
Einstein Analytics Prediction BuilderEinstein Analytics Prediction Builder
Einstein Analytics Prediction Builder
 

Recently uploaded

anas about venice for grade 6f about venice
anas about venice for grade 6f about veniceanas about venice for grade 6f about venice
anas about venice for grade 6f about venice
anasabutalha2013
 
April 2024 Nostalgia Products Newsletter
April 2024 Nostalgia Products NewsletterApril 2024 Nostalgia Products Newsletter
April 2024 Nostalgia Products Newsletter
NathanBaughman3
 
Enterprise Excellence is Inclusive Excellence.pdf
Enterprise Excellence is Inclusive Excellence.pdfEnterprise Excellence is Inclusive Excellence.pdf
Enterprise Excellence is Inclusive Excellence.pdf
KaiNexus
 
What is the TDS Return Filing Due Date for FY 2024-25.pdf
What is the TDS Return Filing Due Date for FY 2024-25.pdfWhat is the TDS Return Filing Due Date for FY 2024-25.pdf
What is the TDS Return Filing Due Date for FY 2024-25.pdf
seoforlegalpillers
 
Maksym Vyshnivetskyi: PMO Quality Management (UA)
Maksym Vyshnivetskyi: PMO Quality Management (UA)Maksym Vyshnivetskyi: PMO Quality Management (UA)
Maksym Vyshnivetskyi: PMO Quality Management (UA)
Lviv Startup Club
 
FINAL PRESENTATION.pptx12143241324134134
FINAL PRESENTATION.pptx12143241324134134FINAL PRESENTATION.pptx12143241324134134
FINAL PRESENTATION.pptx12143241324134134
LR1709MUSIC
 
Kseniya Leshchenko: Shared development support service model as the way to ma...
Kseniya Leshchenko: Shared development support service model as the way to ma...Kseniya Leshchenko: Shared development support service model as the way to ma...
Kseniya Leshchenko: Shared development support service model as the way to ma...
Lviv Startup Club
 
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
taqyed
 
LA HUG - Video Testimonials with Chynna Morgan - June 2024
LA HUG - Video Testimonials with Chynna Morgan - June 2024LA HUG - Video Testimonials with Chynna Morgan - June 2024
LA HUG - Video Testimonials with Chynna Morgan - June 2024
Lital Barkan
 
Memorandum Of Association Constitution of Company.ppt
Memorandum Of Association Constitution of Company.pptMemorandum Of Association Constitution of Company.ppt
Memorandum Of Association Constitution of Company.ppt
seri bangash
 
3.0 Project 2_ Developing My Brand Identity Kit.pptx
3.0 Project 2_ Developing My Brand Identity Kit.pptx3.0 Project 2_ Developing My Brand Identity Kit.pptx
3.0 Project 2_ Developing My Brand Identity Kit.pptx
tanyjahb
 
VAT Registration Outlined In UAE: Benefits and Requirements
VAT Registration Outlined In UAE: Benefits and RequirementsVAT Registration Outlined In UAE: Benefits and Requirements
VAT Registration Outlined In UAE: Benefits and Requirements
uae taxgpt
 
The Parable of the Pipeline a book every new businessman or business student ...
The Parable of the Pipeline a book every new businessman or business student ...The Parable of the Pipeline a book every new businessman or business student ...
The Parable of the Pipeline a book every new businessman or business student ...
awaisafdar
 
Discover the innovative and creative projects that highlight my journey throu...
Discover the innovative and creative projects that highlight my journey throu...Discover the innovative and creative projects that highlight my journey throu...
Discover the innovative and creative projects that highlight my journey throu...
dylandmeas
 
Cree_Rey_BrandIdentityKit.PDF_PersonalBd
Cree_Rey_BrandIdentityKit.PDF_PersonalBdCree_Rey_BrandIdentityKit.PDF_PersonalBd
Cree_Rey_BrandIdentityKit.PDF_PersonalBd
creerey
 
Business Valuation Principles for Entrepreneurs
Business Valuation Principles for EntrepreneursBusiness Valuation Principles for Entrepreneurs
Business Valuation Principles for Entrepreneurs
Ben Wann
 
ikea_woodgreen_petscharity_dog-alogue_digital.pdf
ikea_woodgreen_petscharity_dog-alogue_digital.pdfikea_woodgreen_petscharity_dog-alogue_digital.pdf
ikea_woodgreen_petscharity_dog-alogue_digital.pdf
agatadrynko
 
Meas_Dylan_DMBS_PB1_2024-05XX_Revised.pdf
Meas_Dylan_DMBS_PB1_2024-05XX_Revised.pdfMeas_Dylan_DMBS_PB1_2024-05XX_Revised.pdf
Meas_Dylan_DMBS_PB1_2024-05XX_Revised.pdf
dylandmeas
 
Unveiling the Secrets How Does Generative AI Work.pdf
Unveiling the Secrets How Does Generative AI Work.pdfUnveiling the Secrets How Does Generative AI Work.pdf
Unveiling the Secrets How Does Generative AI Work.pdf
Sam H
 
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdf
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdfSearch Disrupted Google’s Leaked Documents Rock the SEO World.pdf
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdf
Arihant Webtech Pvt. Ltd
 

Recently uploaded (20)

anas about venice for grade 6f about venice
anas about venice for grade 6f about veniceanas about venice for grade 6f about venice
anas about venice for grade 6f about venice
 
April 2024 Nostalgia Products Newsletter
April 2024 Nostalgia Products NewsletterApril 2024 Nostalgia Products Newsletter
April 2024 Nostalgia Products Newsletter
 
Enterprise Excellence is Inclusive Excellence.pdf
Enterprise Excellence is Inclusive Excellence.pdfEnterprise Excellence is Inclusive Excellence.pdf
Enterprise Excellence is Inclusive Excellence.pdf
 
What is the TDS Return Filing Due Date for FY 2024-25.pdf
What is the TDS Return Filing Due Date for FY 2024-25.pdfWhat is the TDS Return Filing Due Date for FY 2024-25.pdf
What is the TDS Return Filing Due Date for FY 2024-25.pdf
 
Maksym Vyshnivetskyi: PMO Quality Management (UA)
Maksym Vyshnivetskyi: PMO Quality Management (UA)Maksym Vyshnivetskyi: PMO Quality Management (UA)
Maksym Vyshnivetskyi: PMO Quality Management (UA)
 
FINAL PRESENTATION.pptx12143241324134134
FINAL PRESENTATION.pptx12143241324134134FINAL PRESENTATION.pptx12143241324134134
FINAL PRESENTATION.pptx12143241324134134
 
Kseniya Leshchenko: Shared development support service model as the way to ma...
Kseniya Leshchenko: Shared development support service model as the way to ma...Kseniya Leshchenko: Shared development support service model as the way to ma...
Kseniya Leshchenko: Shared development support service model as the way to ma...
 
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
 
LA HUG - Video Testimonials with Chynna Morgan - June 2024
LA HUG - Video Testimonials with Chynna Morgan - June 2024LA HUG - Video Testimonials with Chynna Morgan - June 2024
LA HUG - Video Testimonials with Chynna Morgan - June 2024
 
Memorandum Of Association Constitution of Company.ppt
Memorandum Of Association Constitution of Company.pptMemorandum Of Association Constitution of Company.ppt
Memorandum Of Association Constitution of Company.ppt
 
3.0 Project 2_ Developing My Brand Identity Kit.pptx
3.0 Project 2_ Developing My Brand Identity Kit.pptx3.0 Project 2_ Developing My Brand Identity Kit.pptx
3.0 Project 2_ Developing My Brand Identity Kit.pptx
 
VAT Registration Outlined In UAE: Benefits and Requirements
VAT Registration Outlined In UAE: Benefits and RequirementsVAT Registration Outlined In UAE: Benefits and Requirements
VAT Registration Outlined In UAE: Benefits and Requirements
 
The Parable of the Pipeline a book every new businessman or business student ...
The Parable of the Pipeline a book every new businessman or business student ...The Parable of the Pipeline a book every new businessman or business student ...
The Parable of the Pipeline a book every new businessman or business student ...
 
Discover the innovative and creative projects that highlight my journey throu...
Discover the innovative and creative projects that highlight my journey throu...Discover the innovative and creative projects that highlight my journey throu...
Discover the innovative and creative projects that highlight my journey throu...
 
Cree_Rey_BrandIdentityKit.PDF_PersonalBd
Cree_Rey_BrandIdentityKit.PDF_PersonalBdCree_Rey_BrandIdentityKit.PDF_PersonalBd
Cree_Rey_BrandIdentityKit.PDF_PersonalBd
 
Business Valuation Principles for Entrepreneurs
Business Valuation Principles for EntrepreneursBusiness Valuation Principles for Entrepreneurs
Business Valuation Principles for Entrepreneurs
 
ikea_woodgreen_petscharity_dog-alogue_digital.pdf
ikea_woodgreen_petscharity_dog-alogue_digital.pdfikea_woodgreen_petscharity_dog-alogue_digital.pdf
ikea_woodgreen_petscharity_dog-alogue_digital.pdf
 
Meas_Dylan_DMBS_PB1_2024-05XX_Revised.pdf
Meas_Dylan_DMBS_PB1_2024-05XX_Revised.pdfMeas_Dylan_DMBS_PB1_2024-05XX_Revised.pdf
Meas_Dylan_DMBS_PB1_2024-05XX_Revised.pdf
 
Unveiling the Secrets How Does Generative AI Work.pdf
Unveiling the Secrets How Does Generative AI Work.pdfUnveiling the Secrets How Does Generative AI Work.pdf
Unveiling the Secrets How Does Generative AI Work.pdf
 
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdf
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdfSearch Disrupted Google’s Leaked Documents Rock the SEO World.pdf
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdf
 

Security Snake Oil Cycle 2019

  • 1. The Snake Oil Cycle Dave Cole, dave@openraven.com
  • 2.
  • 3. OPEN RAVEN Modern Attack Surface Management
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10. Meet Clark Stanley It’s not about Clark or even a type of person
  • 11. Genuine problems lay beyond the
  • 12.
  • 13. The obvious stuff If you did a great job, many times nothing bad will happen. Which is impossible to prove. The really compelling “before & after” stories are elusive.
  • 14. I’m going to be unfair & pick on another product category & a single company as an example.
  • 15. Imagine this in security, esp in the middle of an incident… measure progress in months!
  • 16. Standard expectation: real- time or very close to it.
  • 18. I’m sure it’s hard to build an HRIS, but their Alice & Bob don’t have an Eve. Or a Mallory. Or a Trudy. Or re-architect because a new attack type changed their original assumptions. Or crank a new remediation engine because they can’t remove the latest threat. Or make sure that OS change didn’t just bust your behavioral defenses. Or properly identify every IoT device.
  • 20.
  • 21.
  • 22. Building security products is hard. Being a security professional is hard. Natural empathy should drive us together. It rarely does.
  • 23. The U.S. employs nearly 716,000 people in cybersecurity positions, with approximately 314,000 current cybersecurity openings https://www.cyberseek.org/heatmap.html
  • 24.
  • 28.
  • 29.
  • 30.
  • 31. “7% of all S3 buckets have unrestricted public access, and 35% are unencrypted” SkyHigh Networks (now McAfee) September 2017… before Amazon changed policy settings to proactively warn users of public buckets
  • 32. Who When What Booz Allen Hamilton May 2017 Battlefield imagery & admin credentials to sensitive systems Deep Root Analytics June 2017 Personal data about 198 million American voters Dow Jones & Co July 2017 Personally identifiable information for 2.2 million people WWE July 2017 Personally identifiable information of >3 million wrestling fans Verizon Wireless July/Sept 2017 Personally identifiable information of ~6 million people & sensitive corp. information about IT systems, incl. login creds Time Warner Cable Sept 2017 Personally identifiable information about 4 million customers, proprietary code & administrator creds Pentagon Exposures Sept/Nov 2017 TBs of info from spying archive, resume for intelligence positions—incl. security clearance & ops history, creds & metadata from an intra-agency intelligence sharing platform Accenture Sept 2017 Master access keys for Accenture's account w/ AWS Key Mgmt System, plaintext customer password DBs & proprietary API data Natl. Credit Federation Nov 2017 111GB financial info—incl. full credit reports-- ~47K people Alteryx Dec 2017 Personal info of ~123 million American households https://businessinsights.bitdefender.com/worst-amazon-breaches
  • 33.
  • 34. On average, sec teams are understaffed, undertrained, under intense pressure from increasing threats & vulnerabilities with regulatory mandates to make fast decisions… & circumstances can change w/o warning. Let’s try to find them some help...
  • 35.
  • 36. Security design can sometimes look like this
  • 38. Historically we have assumed the person behind the console is an expert
  • 39. Increasingly, we have products designed for experts in the hands of novices. The bar must be lower.
  • 40. Before we move on, a word on how products are evaluated prior to purchasing* *This is hard to generalize, your mileage may vary
  • 44. What if someone had a solid test lab that we could all lean on for quality, unbiased product reviews?
  • 45.
  • 46.
  • 47.
  • 48. There’s no genuine equivalent for the security community… why?
  • 49. They are spread too thin, like every other journalist on the 24x7 coverage beat.
  • 50.
  • 51. They also have the same pressures as political journalists– if you’re overly critical you lose access.
  • 52. They typically do not have the lab environments they require. Most vendors have someone dedicated to lab ops at scale.
  • 53. Even if they had the labs they need, the skill set required to test one product is not what you need to test the next product.
  • 54. And if they truly were this skilled, they would be unlikely to remain a journalist for long.
  • 55. So our security reviewers are unlikely to have the focus, objectivity, labs & skills they need to publish great reviews. And if they did, we would hire them in a heartbeat. What about the industry analysts?
  • 56. Same • Increasingly spread too thin; lack of focus • Typically little or no experience as practitioner Worse • Purely anecdotal-- no real tests • $$$ of a subscription; not readily available Better • More direct customer feedback • In-depth reports w/ longer horizon
  • 57.
  • 58.
  • 59.
  • 60.
  • 61. AntiVirus Industry – An Exception
  • 62. What Opinion File scanning tests Worked fine until this was only a subset of defenses… early 2000s On access tests About the same as file scanning Retrospective tests Purgatory b/w file scanning & real world tests; misleading Real world tests When done properly, effective; rarely done properly Remediation tests Few and far in between… Performance tests Largely a success, materially improved the industry
  • 63. At their best, public tests push vendors to make better products, but they are too often designed poorly or easily misunderstood. Usually, they simply don’t exist.
  • 64. So what’s a security marketer supposed to do?
  • 65.
  • 67. Product Person “Our year long effort to create an awesome network scanner that we can improve more rapidly and support better in the future just completed! In some instances, you’re going to see some nice speed improvements.”
  • 68. Marketing “Our year long effort to create a strong network scanner that we can improve more rapidly and support better in the future just completed! In some instances, you’re going to see some nice speed improvements.” Customer value not readily apparent to many Vague
  • 69. Post Marketing “New scan engine frequently boosts scans by 30%, reducing time needed to scan and easing network impact in many instances.”
  • 70. Customer “New scan engine boosts scans by 30%,frequently reducing time needed to scan and easing network impact in many instances.”
  • 71. Recap Opaque product benefit ↓ Well-intentioned attempt at clear message ↓ Misunderstanding due to nuance, pressure ↓ Disappointment & loss of trust
  • 72. Marketing security products often requires a strong grasp of nuance, understanding not only the tech, but also how the customer will receive the message. These people are rare.
  • 73. And even if they are really that good, things change quickly. And they change security markets… which changes everything.
  • 74. Marketing is also affected by how dynamic security companies are– more so than other functions. And more so than other areas in software.
  • 75.
  • 76.
  • 77. Often distracted (2018 #s) • 184 M&A transaction • 406 Financing transactions
  • 78. Even if you can avoid the distractions, any company will change significantly as it briskly moves from start- up→growth→behemoth This often alienates employees & users
  • 79. “I have never seen such a fast-growing market with so many companies on the losing side” David Cowan, Partner, Bessemer Venture Partners Under threat: Cyber security startups fall on harder times Reuters, January 17, 2018
  • 80. If you knew you had to show strong sales performance to establish your company, but were unsure of its longevity, how would you build & sell your product?
  • 81. Given the rough & tumble nature of the industry, far too often companies optimize for creating the “reach for your wallet” moment over establishing a strong foundation
  • 83. It’s too easy to blame any one person or type of person, the problem is pervasive & self-reinforcing
  • 84. Security Professionals 1. Do a process walk thru w/ vendor or 3rd party 2. Do whatever you can to best simulate genuine production conditions before purchasing 3. Ask for a “reverse roadmap” 4. Insist on references, preferably long-term ones 5. Take updates & new versions ASAP
  • 85. Security Vendors 1. Definition of done: customer is successful 2. Hire & empower a real design team 3. Build out a complete customer feedback loop 4. If no one else will do it, produce your own metrics & be ready to share & explain them 5. Invite Sales & Marketing into “the factory”

Editor's Notes

  1. The oil came from Chinese Water Snakes, which were somewhat lacking on the North American continent, so later on when Americans tried to sell off fraudulent cures, snake oil was one of the go-to products to try and reproduce. Starting with rattlesnake oil. Clark Stanley was one such entrepreneur who, at a Chicago exposition: ...reached into a sack, plucked out a snake, slit it open and plunged it into boiling water. When the fat rose to the top, he skimmed it off and used it on the spot to create 'Stanley's Snake Oil,' a liniment that was immediately snapped up by the throng that had gathered to watch the spectacle. Kinda wish I was there to see... The problem was that rattlesnake oil wasn't even nearly as effective as Chinese water snake, and Stanley's product wasn't even using oil to begin with. What investigators eventually found was that it contained: ...mineral oil, a fatty oil believed to be beef fat, red pepper and turpentine. From that day forward, Stanley's scam made snake oil symbolic of fraud. Stanley was charged $20 for his crime (a little under $500 equivalent now).
  2. Hits sec vendors too… need them to do a good job building products
  3. There are currently 90,000 published CVE’s 2017 alone saw a 100% increase in added CVE’s from the previous year A record-breaking number of 20,832 vulnerabilities were discovered in total in 2017, but only 13,160 of these received an official CVE identifier last year*
  4. There are currently 90,000 published CVE’s 2017 alone saw a 100% increase in added CVE’s from the previous year A record-breaking number of 20,832 vulnerabilities were discovered in total in 2017, but only 13,160 of these received an official CVE identifier last year*
  5. Anyone with a credit card can create a data center. New reality of the pub cloud world Really basic problems like simply knowing what servers you have (even what data centers) are back What could possibly go wrong? Previous research carried out by experts from Skyhigh Networks found that 7% of all Amazon S3 buckets are publicly accessible. Over the past few months, security researchers have found a large number of companies that leaked sensitive data this way, via S3 buckets left exposed online. A (most likely incomplete) list of the most notable incidents is included below. ⬨ Top defense contractor Booz Allen Hamilton leaks 60,000 files, including employee security credentials and passwords to a US government system. ⬨ Verizon partner leaks personal records of over 14 million Verizon customers, including names, addresses, account details, and for some victims — account PINs. ⬨ An AWS S3 server leaked the personal details of WWE fans who registered on the company's sites. 3,065,805 users were exposed. ⬨ Another AWS S3 bucket leaked the personal details of over 198 million American voters. The database contained information from three data mining companies known to be associated with the Republican Party. ⬨ Another S3 database left exposed only leaked the personal details of job applications that had Top Secret government clearance. ⬨ Dow Jones, the parent company of the Wall Street Journal, leaked the personal details of 2.2 million customers. ⬨ Omaha-based voting machine firm Election Systems & Software (ES&S) left a database exposed online that contained the personal records of 1.8 million Chicago voters. ⬨ Security researchers discovered a Verizon AWS S3 bucket containing over 100 MB of data about the company's internal system named Distributed Vision Services (DVS), used for billing operations. ⬨ An auto-tracking company leaked over a half of a million records with logins/passwords, emails, VIN (vehicle identification number), IMEI numbers of GPS devices and other data that is collected on their devices, customers and auto dealerships.
  6. Re-order
  7. Bought off via ads…
  8. They typically do not have the lab environments they require. Most vendors have someone dedicated to lab ops at scale.
  9. frequently boosts scans by 30%, reducing time needed to scan and easing network impact in many instances.”
  10. Beyond normal s/w pressures
  11. How different from normal s/w… more losers, more dynamic
  12. The oil came from Chinese Water Snakes, which were somewhat lacking on the North American continent, so later on when Americans tried to sell off fraudulent cures, snake oil was one of the go-to products to try and reproduce. Starting with rattlesnake oil. Clark Stanley was one such entrepreneur who, at a Chicago exposition: ...reached into a sack, plucked out a snake, slit it open and plunged it into boiling water. When the fat rose to the top, he skimmed it off and used it on the spot to create 'Stanley's Snake Oil,' a liniment that was immediately snapped up by the throng that had gathered to watch the spectacle. Kinda wish I was there to see... The problem was that rattlesnake oil wasn't even nearly as effective as Chinese water snake, and Stanley's product wasn't even using oil to begin with. What investigators eventually found was that it contained: ...mineral oil, a fatty oil believed to be beef fat, red pepper and turpentine. From that day forward, Stanley's scam made snake oil symbolic of fraud. Stanley was charged $20 for his crime (a little under $500 equivalent now).
  13. Care and feeding…. After care Cats and dogs w/ IDS and FW
  14. Understand the solution versus the feature… know the problem that product is trying to solve
  15. The oil came from Chinese Water Snakes, which were somewhat lacking on the North American continent, so later on when Americans tried to sell off fraudulent cures, snake oil was one of the go-to products to try and reproduce. Starting with rattlesnake oil. Clark Stanley was one such entrepreneur who, at a Chicago exposition: ...reached into a sack, plucked out a snake, slit it open and plunged it into boiling water. When the fat rose to the top, he skimmed it off and used it on the spot to create 'Stanley's Snake Oil,' a liniment that was immediately snapped up by the throng that had gathered to watch the spectacle. Kinda wish I was there to see... The problem was that rattlesnake oil wasn't even nearly as effective as Chinese water snake, and Stanley's product wasn't even using oil to begin with. What investigators eventually found was that it contained: ...mineral oil, a fatty oil believed to be beef fat, red pepper and turpentine. From that day forward, Stanley's scam made snake oil symbolic of fraud. Stanley was charged $20 for his crime (a little under $500 equivalent now).