Indistinguishable from Magic: How the Cybersecurity Market Reached a Trillion Dollars

Editor's Notes

1. First off, you might be thinking to yourself, “a trillion dollars? this guy is full of crap!” Well, congrats! This talk is especially for you, bullshit spotters! So, sure - the market hasn’t reached a trillion dollars yet, but it could get there. But it won’t get there by being transparent, easy to use, or by solving security problems. People will be the ones to solve these problems, and it won’t be easy for them. I’ll explain why.
2. About me
3. - Data is stored on floppies and hard drives with *********magnitism? What???********* Have you ever opened up a floppy? It’s just this cheap looking flimsy plastic disc. It looks like nothing - Modems! How do they work? - Hacking - Tell the Blind SQL mercs for hire story ← probably not enough time? - Ask some audience members why they got into the industry - Talk about the magic of seeing your code run for the first time, or seeing an exploit work.
4. Some things never change So I'll just reuse this slide from 2016 This is a different kind of magic - Marketing Magic - ML and AI will fix everything - disclaimer: not actually a vendor bashing talk - but: there's some unearned trust in products and vendors
5. Or maybe the products work, but don't do what the marketing promises? Sometimes, it's easy to tell if it works or not The claim isn't that hard to put to the test -> to mentos fountain example
6. Has anyone here NOT personally done a mentos fountain? Do you remember the first time they saw a video of someone dropping mentos into a diet coke bottle? Anyone? What was your reaction, that *very first time* you saw it? I’m the type of person that watches a video like that and thinks: *get the fuck outta here, that ain’t real - no way, it CAN’T be!* How many of you went out and bought diet coke and mentos after seeing that video, to do your own experiment? What was lovely about the mentos and diet coke was that it looked like magic, but anyone with $5 could go out and do the experiment for themselves. In fact, if you search the Internet for videos and pictures of people doing experiments with coke and mentos, it’s a list that doesn’t stop
7. Have you ever watched an ad for motor oil and thought, “you know, I think they’re right. If I change to that brand, all this sludge and buildup will leave my engine and it’s gonna start working better.” [pause] Then you bought $1000 worth of Mobile 1, or Royal Purple, maybe Giraffe urine You changed your oil, or had a garage do it Got in your car, (pantomime begins) Sat down, Started the car, (quieter) And listened. *Really* listened. (much quieter) And said to yourself, “I can *hear* it.” “It *worked*.” “Royal Purple is really extending the life of my engine” You’re out there with a stethoscope on the valve cover of your engine, like “Yeahhhhh” “Oh yeah, it’s doing its thing”
8. Of course not! There's no simple, objective way for you to figure out what's better than anything else on the motor oil shelf at your local auto parts store. So you buy what your uncle told you to. Or your mechanic. Or your friend. Or maybe an advertisement caught your attention. Of COURSE not! You don’t *actually know* Because there’s no simple way for you to know or figure this out We don’t what an oil is or isn’t doing, we have to trust the damn marketing, or find some scientific, independent reviews of the products You can’t - there’s no observable result to the average consumer. Instead, you might pay more, because you’re hoping it really will extend the life of your engine. Or maybe just because paying more gives you more comfort or peace of mind. In fact, this is a great time to point out that the word “Security” literally means “without care” or “carefree”. In other words, security CAN be an emotion, a *feeling*. So how does this explain the state of the security market?
9. These two stories illustrate two very different scenarios: 1. you’ve got scenarios where it’s possible - easy, even to build a feedback loop that tells you if something really works, or not. $5 worth of mentos and diet coke and any *child* can 100% demonstrate the fountain reaction. 2. and a scenario where you're more likely to look for some formal, scientific testing to help guide you. Or maybe you just take someone's recommendation, or buy based on price: not the most expensive, but not the cheapest. do most security products fit the mentos fountain metaphor or the motor oil metaphor?
10. Oh yeah, there's gonna be a lot of dogs Any dog lovers in here? Any dog haters? This is going to be very abbreviated due to time constraints and from my specific viewpoint, so keep those limitations in mind
11. 2002 what isn't here?: No pen testing, no incident response plan, no password complexity, no hardening, no baseline security standards or benchmarks, one giant flat network with no host-based firewalls, no full disk encryption, telnet everywhere, open Windows file shares everywhere
12. Tell the story of my first SIEM - emphasizing the wrong approach I took (product first, design second) - We bought it because PCI forced us to - I shoved all our log sources into it (1700+ devices, 100m events per day, didn’t even include endpoint logs) - Now, suddenly, it’s the job of 2 full-time people to keep the SIEM running and it’s my job to figure out what to do with it. None of the built-in reports were useful, so I started creating custom ones. - Now, because we’ve got this single system of record, IT is making us do their root cause analysis every time they have a major outage - Every day, some log source stops sending us logs and we’ve got to go investigate why - We put together our first SOC to just put eyeballs on everything the SIEM was pumping out - Half the security team has been retasked by SIEM ownership - is it worth it? Are we more secure? Are we even doing security work, or is the SIEM just generating busywork for us?
13. And this is what it evolved into What the fuck is this monstrosity??? Look at the level of complexity here. What do you think your chances are of getting all this right? "Classic SIEM" is a pain and it's just a tiny portion of it! 24+ Trillion TI signals per day? What the fuck do you do with that??? At least they do suggest you should have some sort of feedback loop...
14. This is a cultural thing that doesn't just happen in security, but in IT also. "Looks Good To Me" is similar. Folks don't know how things work, or get done and they're just not interested in taking that extra step to find out. It's why IDS appliances get plugged into the wrong SPAN port It's why I've seen Carbon Black deployed to 8000 endpoints, but inactive, because no one pushed a policy.
15. This is a cultural thing that doesn't just happen in security, but in IT also. "Looks Good To Me" is similar. Folks don't know how things work, or get done and they're just not interested in taking that extra step to find out. It's why IDS appliances get plugged into the wrong SPAN port It's why I've seen Carbon Black deployed to 8000 endpoints, but inactive, because no one pushed a policy.
16. But a lot of what I've seen isn't just "being bad at security", it's carelessness. This is a cultural thing that doesn't just happen in security, but in IT also. "Looks Good To Me" is similar. Folks don't know how things work, or get done and they're just not interested in taking that extra step to find out. It's why IDS appliances get plugged into the wrong SPAN port It's why I've seen Carbon Black deployed to 8000 endpoints, but inactive, because no one pushed a policy.
17. It's why third parties request too many permissions!
18. It's a brand of carelessness that stems from hoping that getting CLOSE to doing the right thing can get you full credit. It can't. This is how I often see security products deployed. I see code written this way and tossed over to QA - it doesn't even run This is a cultural thing. Again, leadership is important here. Processes that are respected, followed, and continuously improved are important. Let people get away with garbage, and well - it all ends up in the garbage.
19. MSSPs are a very special kind of magic. I'm sure there are good ones, but I haven't had any good experiences, and I haven't run into many people with good experiences either. "When you're tired of doing a crappy job of something yourself, let an MSSP do a crappy job for you" was my motto
20. MSSPs are a very special kind of magic. I'm sure there are good ones, but I haven't had any good experiences, and I haven't run into many people with good experiences either. "When you're tired of doing a crappy job of something yourself, let an MSSP do a crappy job for you" was my motto
21. MSSPs are a very special kind of magic. I'm sure there are good ones, but I haven't had any good experiences, and I haven't run into many people with good experiences either. "When you're tired of doing a crappy job of something yourself, let an MSSP do a crappy job for you" was my motto
22. Anyone? A high-level, big picture sort of pattern?
23. Everything that would have saved them was just basic security policies (don't store plaintext creds) Basic monitoring (MFA bombing) Asset inventories Knowing how to use SAST and DAST tools properly In fact, if I'm not mistaken, simply following PCI would have saved all these orgs
24. Has anyone looked at how these extortion campaigns are run? The actual hacking part of it?
25. Back in 2011, Haroon Meer did a talk at 44Con called Penetration Testing Considered Harmful The TL;DR was that penetration testers were not accurately emulating adversaries They were more or less doing their own thing, having a good time The irony is that now, life seems to be imitating art and the script has flipped these extortion attacks look like every ordinary, mediocre pen test you've ever seen They use pen test tools They're using pen test methodologies Some of the people carrying them out even appear to be ex-pen testers, or at least have some formal pen test training. W T F We literally TRAINED for this. For two plus decades! But pen tests were a checkbox, so they could be safely ignored, right?
26. It's now highly likely (and awkward) that there may now be overlaps between CEH holders and cyber criminals.
27. ⭐Did you know that products can have *negative* value? Someone in management *really* wants to buy a security product, mostly because it’s the new hotness and they’ve really got a fever for it after the sales pitch sunk its teeth in them. Besides, all their friends and peers in other companies are buying it They don’t want to pay $250k for it, so they bargain with the sales rep They eventually get it for $50k and look like a damn genius to the other execs. What a deal! Let’s say this is an appliance that stops malware. Since we can only afford one, it will have to go in HQ, not any of our sales centers, remote offices, or manufacturing centers, where 95% of our malware infections occur. We get hit with a malware infection only a week later. The new product doesn’t catch it. We call up support to figure out why - this should have been an opportunity for it to do its thing! Support doesn’t seem too surprised, which concerns you. They check a few settings, but quickly start asking you to run some commands to dump logs from the appliance. They then have you zip up the logs and FTP them to the company for analysis. They get back to you about a week later, suggesting that you should add the hash for the malware to the block list. That’s it. That’s the solution. This process happens 11 more times over the next 4 months. Each time: call support, dump the logs, zip them up, upload to the FTP server, and a week later, add hash to block list. Each time you go through this, it takes 4 hours of your time. If value for this product is defined as stopping malware infections, it has demonstrated zero value in its first 4 months. In fact, let’s say the analyst dealing with these false negatives is getting paid $150k per year. The total cost of employment (benefits, etc) is $187500. An hour of their time is worth roughly $90 to the business. Every time this product misses a piece of malware, this product creates a negative value of $360.58. In the first 4 months, it has generated $4326.92 in negative value. It also requires 4 hours of maintenance per month to keep running, for another $1442.32. We’re not even counting the value of the opportunity loss here. Every hour spent reporting a false negative is an hour this employee could be spending doing something of value for the company. The exec is still bragging about what a great deal he got. Meanwhile, the product represents a *negative* -$55769.24 value to the business.
28. ⭐Did you know that products can have *negative* value? Someone in management *really* wants to buy a security product, mostly because it’s the new hotness and they’ve really got a fever for it after the sales pitch sunk its teeth in them. Besides, all their friends and peers in other companies are buying it They don’t want to pay $250k for it, so they bargain with the sales rep They eventually get it for $50k and look like a damn genius to the other execs. What a deal! Let’s say this is an appliance that stops malware. Since we can only afford one, it will have to go in HQ, not any of our sales centers, remote offices, or manufacturing centers, where 95% of our malware infections occur. We get hit with a malware infection only a week later. The new product doesn’t catch it. We call up support to figure out why - this should have been an opportunity for it to do its thing! Support doesn’t seem too surprised, which concerns you. They check a few settings, but quickly start asking you to run some commands to dump logs from the appliance. They then have you zip up the logs and FTP them to the company for analysis. They get back to you about a week later, suggesting that you should add the hash for the malware to the block list. That’s it. That’s the solution. This process happens 11 more times over the next 4 months. Each time: call support, dump the logs, zip them up, upload to the FTP server, and a week later, add hash to block list. Each time you go through this, it takes 4 hours of your time. If value for this product is defined as stopping malware infections, it has demonstrated zero value in its first 4 months. In fact, let’s say the analyst dealing with these false negatives is getting paid $150k per year. The total cost of employment (benefits, etc) is $187500. An hour of their time is worth roughly $90 to the business. Every time this product misses a piece of malware, this product creates a negative value of $360.58. In the first 4 months, it has generated $4326.92 in negative value. It also requires 4 hours of maintenance per month to keep running, for another $1442.32. We’re not even counting the value of the opportunity loss here. Every hour spent reporting a false negative is an hour this employee could be spending doing something of value for the company. The exec is still bragging about what a great deal he got. Meanwhile, the product represents a *negative* -$55769.24 value to the business.
29. Go talk to smart people, all the time. Build a network of folks that are like-minded, questioning things, and regularly throw out old, broken things.
30. Don't assume a commercial product or service does what it says on the tin. Test them and hold them to their promises and claims.
31. These are all things that will help ensure a security program continuously evolves. Not all choices you make will bring the program forward. There will likely be room for feedback loops within each of these areas. Just make sure the loop is always closed.
32. the problem is more clear than the solution. I think my recommendations can help, but I’m not sure what other challenges will emerge when my suggestions meet reality at your particular organization. The important thing is that we make security testable and measurable. Otherwise we can’t see the needle and we don’t know where it’s moving.