Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Security Snake Oil Cycle 2019

An exploration of the cyber security market factors that lend to pervasive issues with hyperbole and feelings of broken trust across the various participants. Much is left off the slide & was covered in narrative at a recent OWASP LA meetup, original done for B-Sides LV.

  • Be the first to comment

  • Be the first to like this

Security Snake Oil Cycle 2019

  1. 1. The Snake Oil Cycle Dave Cole,
  2. 2. OPEN RAVEN Modern Attack Surface Management
  3. 3. Meet Clark Stanley It’s not about Clark or even a type of person
  4. 4. Genuine problems lay beyond the
  5. 5. The obvious stuff If you did a great job, many times nothing bad will happen. Which is impossible to prove. The really compelling “before & after” stories are elusive.
  6. 6. I’m going to be unfair & pick on another product category & a single company as an example.
  7. 7. Imagine this in security, esp in the middle of an incident… measure progress in months!
  8. 8. Standard expectation: real- time or very close to it.
  9. 9. Our expected uninvited guests
  10. 10. I’m sure it’s hard to build an HRIS, but their Alice & Bob don’t have an Eve. Or a Mallory. Or a Trudy. Or re-architect because a new attack type changed their original assumptions. Or crank a new remediation engine because they can’t remove the latest threat. Or make sure that OS change didn’t just bust your behavioral defenses. Or properly identify every IoT device.
  11. 11. A short story.
  12. 12. Building security products is hard. Being a security professional is hard. Natural empathy should drive us together. It rarely does.
  13. 13. The U.S. employs nearly 716,000 people in cybersecurity positions, with approximately 314,000 current cybersecurity openings
  14. 14. Breach Level Index 2018
  15. 15.
  16. 16.
  17. 17. “7% of all S3 buckets have unrestricted public access, and 35% are unencrypted” SkyHigh Networks (now McAfee) September 2017… before Amazon changed policy settings to proactively warn users of public buckets
  18. 18. Who When What Booz Allen Hamilton May 2017 Battlefield imagery & admin credentials to sensitive systems Deep Root Analytics June 2017 Personal data about 198 million American voters Dow Jones & Co July 2017 Personally identifiable information for 2.2 million people WWE July 2017 Personally identifiable information of >3 million wrestling fans Verizon Wireless July/Sept 2017 Personally identifiable information of ~6 million people & sensitive corp. information about IT systems, incl. login creds Time Warner Cable Sept 2017 Personally identifiable information about 4 million customers, proprietary code & administrator creds Pentagon Exposures Sept/Nov 2017 TBs of info from spying archive, resume for intelligence positions—incl. security clearance & ops history, creds & metadata from an intra-agency intelligence sharing platform Accenture Sept 2017 Master access keys for Accenture's account w/ AWS Key Mgmt System, plaintext customer password DBs & proprietary API data Natl. Credit Federation Nov 2017 111GB financial info—incl. full credit reports-- ~47K people Alteryx Dec 2017 Personal info of ~123 million American households
  19. 19. On average, sec teams are understaffed, undertrained, under intense pressure from increasing threats & vulnerabilities with regulatory mandates to make fast decisions… & circumstances can change w/o warning. Let’s try to find them some help...
  20. 20. Security design can sometimes look like this
  21. 21. Which leads to this
  22. 22. Historically we have assumed the person behind the console is an expert
  23. 23. Increasingly, we have products designed for experts in the hands of novices. The bar must be lower.
  24. 24. Before we move on, a word on how products are evaluated prior to purchasing* *This is hard to generalize, your mileage may vary
  25. 25. Proof of Concept
  26. 26. Normal Operations
  27. 27. OMG Operations
  28. 28. What if someone had a solid test lab that we could all lean on for quality, unbiased product reviews?
  29. 29. There’s no genuine equivalent for the security community… why?
  30. 30. They are spread too thin, like every other journalist on the 24x7 coverage beat.
  31. 31. They also have the same pressures as political journalists– if you’re overly critical you lose access.
  32. 32. They typically do not have the lab environments they require. Most vendors have someone dedicated to lab ops at scale.
  33. 33. Even if they had the labs they need, the skill set required to test one product is not what you need to test the next product.
  34. 34. And if they truly were this skilled, they would be unlikely to remain a journalist for long.
  35. 35. So our security reviewers are unlikely to have the focus, objectivity, labs & skills they need to publish great reviews. And if they did, we would hire them in a heartbeat. What about the industry analysts?
  36. 36. Same • Increasingly spread too thin; lack of focus • Typically little or no experience as practitioner Worse • Purely anecdotal-- no real tests • $$$ of a subscription; not readily available Better • More direct customer feedback • In-depth reports w/ longer horizon
  37. 37. AntiVirus Industry – An Exception
  38. 38. What Opinion File scanning tests Worked fine until this was only a subset of defenses… early 2000s On access tests About the same as file scanning Retrospective tests Purgatory b/w file scanning & real world tests; misleading Real world tests When done properly, effective; rarely done properly Remediation tests Few and far in between… Performance tests Largely a success, materially improved the industry
  39. 39. At their best, public tests push vendors to make better products, but they are too often designed poorly or easily misunderstood. Usually, they simply don’t exist.
  40. 40. So what’s a security marketer supposed to do?
  41. 41. Hypothetical example: new scan engine
  42. 42. Product Person “Our year long effort to create an awesome network scanner that we can improve more rapidly and support better in the future just completed! In some instances, you’re going to see some nice speed improvements.”
  43. 43. Marketing “Our year long effort to create a strong network scanner that we can improve more rapidly and support better in the future just completed! In some instances, you’re going to see some nice speed improvements.” Customer value not readily apparent to many Vague
  44. 44. Post Marketing “New scan engine frequently boosts scans by 30%, reducing time needed to scan and easing network impact in many instances.”
  45. 45. Customer “New scan engine boosts scans by 30%,frequently reducing time needed to scan and easing network impact in many instances.”
  46. 46. Recap Opaque product benefit ↓ Well-intentioned attempt at clear message ↓ Misunderstanding due to nuance, pressure ↓ Disappointment & loss of trust
  47. 47. Marketing security products often requires a strong grasp of nuance, understanding not only the tech, but also how the customer will receive the message. These people are rare.
  48. 48. And even if they are really that good, things change quickly. And they change security markets… which changes everything.
  49. 49. Marketing is also affected by how dynamic security companies are– more so than other functions. And more so than other areas in software.
  50. 50. Often distracted (2018 #s) • 184 M&A transaction • 406 Financing transactions
  51. 51. Even if you can avoid the distractions, any company will change significantly as it briskly moves from start- up→growth→behemoth This often alienates employees & users
  52. 52. “I have never seen such a fast-growing market with so many companies on the losing side” David Cowan, Partner, Bessemer Venture Partners Under threat: Cyber security startups fall on harder times Reuters, January 17, 2018
  53. 53. If you knew you had to show strong sales performance to establish your company, but were unsure of its longevity, how would you build & sell your product?
  54. 54. Given the rough & tumble nature of the industry, far too often companies optimize for creating the “reach for your wallet” moment over establishing a strong foundation
  55. 55. Closing Thoughts Breaking the Cycle
  56. 56. It’s too easy to blame any one person or type of person, the problem is pervasive & self-reinforcing
  57. 57. Security Professionals 1. Do a process walk thru w/ vendor or 3rd party 2. Do whatever you can to best simulate genuine production conditions before purchasing 3. Ask for a “reverse roadmap” 4. Insist on references, preferably long-term ones 5. Take updates & new versions ASAP
  58. 58. Security Vendors 1. Definition of done: customer is successful 2. Hire & empower a real design team 3. Build out a complete customer feedback loop 4. If no one else will do it, produce your own metrics & be ready to share & explain them 5. Invite Sales & Marketing into “the factory”
  59. 59. Thanks Dave Cole,