Successfully reported this slideshow.
Your SlideShare is downloading. ×

Security Snake Oil Cycle 2019

Security Snake Oil Cycle 2019

Download to read offline

An exploration of the cyber security market factors that lend to pervasive issues with hyperbole and feelings of broken trust across the various participants. Much is left off the slide & was covered in narrative at a recent OWASP LA meetup, original done for B-Sides LV.

An exploration of the cyber security market factors that lend to pervasive issues with hyperbole and feelings of broken trust across the various participants. Much is left off the slide & was covered in narrative at a recent OWASP LA meetup, original done for B-Sides LV.

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Security Snake Oil Cycle 2019

  1. 1. The Snake Oil Cycle Dave Cole, dave@openraven.com
  2. 2. OPEN RAVEN Modern Attack Surface Management
  3. 3. Meet Clark Stanley It’s not about Clark or even a type of person
  4. 4. Genuine problems lay beyond the
  5. 5. The obvious stuff If you did a great job, many times nothing bad will happen. Which is impossible to prove. The really compelling “before & after” stories are elusive.
  6. 6. I’m going to be unfair & pick on another product category & a single company as an example.
  7. 7. Imagine this in security, esp in the middle of an incident… measure progress in months!
  8. 8. Standard expectation: real- time or very close to it.
  9. 9. Our expected uninvited guests
  10. 10. I’m sure it’s hard to build an HRIS, but their Alice & Bob don’t have an Eve. Or a Mallory. Or a Trudy. Or re-architect because a new attack type changed their original assumptions. Or crank a new remediation engine because they can’t remove the latest threat. Or make sure that OS change didn’t just bust your behavioral defenses. Or properly identify every IoT device.
  11. 11. A short story.
  12. 12. Building security products is hard. Being a security professional is hard. Natural empathy should drive us together. It rarely does.
  13. 13. The U.S. employs nearly 716,000 people in cybersecurity positions, with approximately 314,000 current cybersecurity openings https://www.cyberseek.org/heatmap.html
  14. 14. Breach Level Index 2018
  15. 15. https://blog.rapid7.com/2018/04/30/cve-100k-by-the-numbers/
  16. 16. https://blog.rapid7.com/2018/04/30/cve-100k-by-the-numbers/
  17. 17. “7% of all S3 buckets have unrestricted public access, and 35% are unencrypted” SkyHigh Networks (now McAfee) September 2017… before Amazon changed policy settings to proactively warn users of public buckets
  18. 18. Who When What Booz Allen Hamilton May 2017 Battlefield imagery & admin credentials to sensitive systems Deep Root Analytics June 2017 Personal data about 198 million American voters Dow Jones & Co July 2017 Personally identifiable information for 2.2 million people WWE July 2017 Personally identifiable information of >3 million wrestling fans Verizon Wireless July/Sept 2017 Personally identifiable information of ~6 million people & sensitive corp. information about IT systems, incl. login creds Time Warner Cable Sept 2017 Personally identifiable information about 4 million customers, proprietary code & administrator creds Pentagon Exposures Sept/Nov 2017 TBs of info from spying archive, resume for intelligence positions—incl. security clearance & ops history, creds & metadata from an intra-agency intelligence sharing platform Accenture Sept 2017 Master access keys for Accenture's account w/ AWS Key Mgmt System, plaintext customer password DBs & proprietary API data Natl. Credit Federation Nov 2017 111GB financial info—incl. full credit reports-- ~47K people Alteryx Dec 2017 Personal info of ~123 million American households https://businessinsights.bitdefender.com/worst-amazon-breaches
  19. 19. On average, sec teams are understaffed, undertrained, under intense pressure from increasing threats & vulnerabilities with regulatory mandates to make fast decisions… & circumstances can change w/o warning. Let’s try to find them some help...
  20. 20. Security design can sometimes look like this
  21. 21. Which leads to this
  22. 22. Historically we have assumed the person behind the console is an expert
  23. 23. Increasingly, we have products designed for experts in the hands of novices. The bar must be lower.
  24. 24. Before we move on, a word on how products are evaluated prior to purchasing* *This is hard to generalize, your mileage may vary
  25. 25. Proof of Concept
  26. 26. Normal Operations
  27. 27. OMG Operations
  28. 28. What if someone had a solid test lab that we could all lean on for quality, unbiased product reviews?
  29. 29. There’s no genuine equivalent for the security community… why?
  30. 30. They are spread too thin, like every other journalist on the 24x7 coverage beat.
  31. 31. They also have the same pressures as political journalists– if you’re overly critical you lose access.
  32. 32. They typically do not have the lab environments they require. Most vendors have someone dedicated to lab ops at scale.
  33. 33. Even if they had the labs they need, the skill set required to test one product is not what you need to test the next product.
  34. 34. And if they truly were this skilled, they would be unlikely to remain a journalist for long.
  35. 35. So our security reviewers are unlikely to have the focus, objectivity, labs & skills they need to publish great reviews. And if they did, we would hire them in a heartbeat. What about the industry analysts?
  36. 36. Same • Increasingly spread too thin; lack of focus • Typically little or no experience as practitioner Worse • Purely anecdotal-- no real tests • $$$ of a subscription; not readily available Better • More direct customer feedback • In-depth reports w/ longer horizon
  37. 37. AntiVirus Industry – An Exception
  38. 38. What Opinion File scanning tests Worked fine until this was only a subset of defenses… early 2000s On access tests About the same as file scanning Retrospective tests Purgatory b/w file scanning & real world tests; misleading Real world tests When done properly, effective; rarely done properly Remediation tests Few and far in between… Performance tests Largely a success, materially improved the industry
  39. 39. At their best, public tests push vendors to make better products, but they are too often designed poorly or easily misunderstood. Usually, they simply don’t exist.
  40. 40. So what’s a security marketer supposed to do?
  41. 41. Hypothetical example: new scan engine
  42. 42. Product Person “Our year long effort to create an awesome network scanner that we can improve more rapidly and support better in the future just completed! In some instances, you’re going to see some nice speed improvements.”
  43. 43. Marketing “Our year long effort to create a strong network scanner that we can improve more rapidly and support better in the future just completed! In some instances, you’re going to see some nice speed improvements.” Customer value not readily apparent to many Vague
  44. 44. Post Marketing “New scan engine frequently boosts scans by 30%, reducing time needed to scan and easing network impact in many instances.”
  45. 45. Customer “New scan engine boosts scans by 30%,frequently reducing time needed to scan and easing network impact in many instances.”
  46. 46. Recap Opaque product benefit ↓ Well-intentioned attempt at clear message ↓ Misunderstanding due to nuance, pressure ↓ Disappointment & loss of trust
  47. 47. Marketing security products often requires a strong grasp of nuance, understanding not only the tech, but also how the customer will receive the message. These people are rare.
  48. 48. And even if they are really that good, things change quickly. And they change security markets… which changes everything.
  49. 49. Marketing is also affected by how dynamic security companies are– more so than other functions. And more so than other areas in software.
  50. 50. Often distracted (2018 #s) • 184 M&A transaction • 406 Financing transactions
  51. 51. Even if you can avoid the distractions, any company will change significantly as it briskly moves from start- up→growth→behemoth This often alienates employees & users
  52. 52. “I have never seen such a fast-growing market with so many companies on the losing side” David Cowan, Partner, Bessemer Venture Partners Under threat: Cyber security startups fall on harder times Reuters, January 17, 2018
  53. 53. If you knew you had to show strong sales performance to establish your company, but were unsure of its longevity, how would you build & sell your product?
  54. 54. Given the rough & tumble nature of the industry, far too often companies optimize for creating the “reach for your wallet” moment over establishing a strong foundation
  55. 55. Closing Thoughts Breaking the Cycle
  56. 56. It’s too easy to blame any one person or type of person, the problem is pervasive & self-reinforcing
  57. 57. Security Professionals 1. Do a process walk thru w/ vendor or 3rd party 2. Do whatever you can to best simulate genuine production conditions before purchasing 3. Ask for a “reverse roadmap” 4. Insist on references, preferably long-term ones 5. Take updates & new versions ASAP
  58. 58. Security Vendors 1. Definition of done: customer is successful 2. Hire & empower a real design team 3. Build out a complete customer feedback loop 4. If no one else will do it, produce your own metrics & be ready to share & explain them 5. Invite Sales & Marketing into “the factory”
  59. 59. Thanks Dave Cole, dave@openraven.com

Editor's Notes

  • The oil came from Chinese Water Snakes, which were somewhat lacking on the North American continent, so later on when Americans tried to sell off fraudulent cures, snake oil was one of the go-to products to try and reproduce. Starting with rattlesnake oil.
    Clark Stanley was one such entrepreneur who, at a Chicago exposition:
    ...reached into a sack, plucked out a snake, slit it open and plunged it into boiling water. When the fat rose to the top, he skimmed it off and used it on the spot to create 'Stanley's Snake Oil,' a liniment that was immediately snapped up by the throng that had gathered to watch the spectacle.
    Kinda wish I was there to see...
    The problem was that rattlesnake oil wasn't even nearly as effective as Chinese water snake, and Stanley's product wasn't even using oil to begin with. What investigators eventually found was that it contained:
    ...mineral oil, a fatty oil believed to be beef fat, red pepper and turpentine.
    From that day forward, Stanley's scam made snake oil symbolic of fraud. Stanley was charged $20 for his crime (a little under $500 equivalent now).
  • Hits sec vendors too… need them to do a good job building products
  • There are currently 90,000 published CVE’s
    2017 alone saw a 100% increase in added CVE’s from the previous year
    A record-breaking number of 20,832 vulnerabilities were discovered in total in 2017, but only 13,160 of these received an official CVE identifier last year*

  • There are currently 90,000 published CVE’s
    2017 alone saw a 100% increase in added CVE’s from the previous year
    A record-breaking number of 20,832 vulnerabilities were discovered in total in 2017, but only 13,160 of these received an official CVE identifier last year*

  • Anyone with a credit card can create a data center.
    New reality of the pub cloud world
    Really basic problems like simply knowing what servers you have (even what data centers) are back
    What could possibly go wrong?

    Previous research carried out by experts from Skyhigh Networks found that 7% of all Amazon S3 buckets are publicly accessible.
    Over the past few months, security researchers have found a large number of companies that leaked sensitive data this way, via S3 buckets left exposed online. A (most likely incomplete) list of the most notable incidents is included below.
    ⬨ Top defense contractor Booz Allen Hamilton leaks 60,000 files, including employee security credentials and passwords to a US government system. ⬨ Verizon partner leaks personal records of over 14 million Verizon customers, including names, addresses, account details, and for some victims — account PINs. ⬨ An AWS S3 server leaked the personal details of WWE fans who registered on the company's sites. 3,065,805 users were exposed. ⬨ Another AWS S3 bucket leaked the personal details of over 198 million American voters. The database contained information from three data mining companies known to be associated with the Republican Party. ⬨ Another S3 database left exposed only leaked the personal details of job applications that had Top Secret government clearance. ⬨ Dow Jones, the parent company of the Wall Street Journal, leaked the personal details of 2.2 million customers. ⬨ Omaha-based voting machine firm Election Systems & Software (ES&S) left a database exposed online that contained the personal records of 1.8 million Chicago voters. ⬨ Security researchers discovered a Verizon AWS S3 bucket containing over 100 MB of data about the company's internal system named Distributed Vision Services (DVS), used for billing operations. ⬨ An auto-tracking company leaked over a half of a million records with logins/passwords, emails, VIN (vehicle identification number), IMEI numbers of GPS devices and other data that is collected on their devices, customers and auto dealerships.
  • Re-order
  • Bought off via ads…
  • They typically do not have the lab environments they require. Most vendors have someone dedicated to lab ops at scale.
  • frequently boosts scans by 30%, reducing time needed to scan and easing network impact in many instances.”
  • Beyond normal s/w pressures
  • How different from normal s/w… more losers, more dynamic
  • The oil came from Chinese Water Snakes, which were somewhat lacking on the North American continent, so later on when Americans tried to sell off fraudulent cures, snake oil was one of the go-to products to try and reproduce. Starting with rattlesnake oil.
    Clark Stanley was one such entrepreneur who, at a Chicago exposition:
    ...reached into a sack, plucked out a snake, slit it open and plunged it into boiling water. When the fat rose to the top, he skimmed it off and used it on the spot to create 'Stanley's Snake Oil,' a liniment that was immediately snapped up by the throng that had gathered to watch the spectacle.
    Kinda wish I was there to see...
    The problem was that rattlesnake oil wasn't even nearly as effective as Chinese water snake, and Stanley's product wasn't even using oil to begin with. What investigators eventually found was that it contained:
    ...mineral oil, a fatty oil believed to be beef fat, red pepper and turpentine.
    From that day forward, Stanley's scam made snake oil symbolic of fraud. Stanley was charged $20 for his crime (a little under $500 equivalent now).
  • Care and feeding…. After care
    Cats and dogs w/ IDS and FW
  • Understand the solution versus the feature… know the problem that product is trying to solve
  • The oil came from Chinese Water Snakes, which were somewhat lacking on the North American continent, so later on when Americans tried to sell off fraudulent cures, snake oil was one of the go-to products to try and reproduce. Starting with rattlesnake oil.
    Clark Stanley was one such entrepreneur who, at a Chicago exposition:
    ...reached into a sack, plucked out a snake, slit it open and plunged it into boiling water. When the fat rose to the top, he skimmed it off and used it on the spot to create 'Stanley's Snake Oil,' a liniment that was immediately snapped up by the throng that had gathered to watch the spectacle.
    Kinda wish I was there to see...
    The problem was that rattlesnake oil wasn't even nearly as effective as Chinese water snake, and Stanley's product wasn't even using oil to begin with. What investigators eventually found was that it contained:
    ...mineral oil, a fatty oil believed to be beef fat, red pepper and turpentine.
    From that day forward, Stanley's scam made snake oil symbolic of fraud. Stanley was charged $20 for his crime (a little under $500 equivalent now).

×