What Ice Cube taught me about security metrics
•Download as PPTX, PDF•
0 likes•548 views
Presentation from Converge 2014 about IT Security Operations Metrics
Recommended
Recommended
More Related Content
Viewers also liked
Similar to What Ice Cube taught me about security metrics
Similar to What Ice Cube taught me about security metrics (20)
Recently uploaded
Recently uploaded (20)
What Ice Cube taught me about security metrics
- 1. Check Yo’self What Ice Cube taught me about security metrics
- 2. Disclaimer All opinions and thoughts in this presentation are my own and do not represent my employer All use of Ice Cube’s image, lyrics, movies, and music are for storytelling, not for profit The data used in this presentation comes from my employer, but is anonymized to protect the guilty and innocent
- 4. Speed If you're foul, you better run a make on that license plate You coulda had a V8 Instead of a tre-eight slug to the cranium I got six and I'm aimin em
- 5. Speed How fast did you find the breach? How fast did you stop the breach after it happened? How fast did you clean it up? How fast did you go from What? to So What? to Now What?
- 6. Speed Comp Find Alert Give a s&*t? Taking action Stop the s&*t Clean up the s&*t You better check yourself before you wreck yourself Cause I'm bad for your health, I come real stealth Dropping bombs on your moms, f*** car alarms Doing foul crime, I'm that fool with your Alpine - Check Yourself – Ice Cube
- 7. Intellectual Honesty Time’s are all in the same time zone – goes without saying The time of compromise is when something changed in the system – not when you or your system found it Missing that key fact means you miss Quality of intelligence Coverage of intelligence Time dropper hit the file table Time A/V reported finding the backdoor Difference = 7 months, 8 days, 13 hours, 34 minutes, 7 seconds
- 8. Trusted sources of truth Host Event logs MFTs Network Firewall logs Netflow logs SMTP logs (for phish) Proxy logs (for watering- holes)
- 9. Comp-to-Find Speed of intelligence deployment to your tools How fast did you get it? How fast did you know it? How fast did you use it? Frequency of scans Alertness of users Collection Processing Exploitation Dissemination Tasking Comp Find Alert Give a s&*t? Taking action Stop the s&*t Clean up the s&*t
- 10. How to find? Host AV logs Event logs Nagios Tripwire Network IDS/IPS alerts Firewall logs Proxy logs Email gateway logs
- 11. Find-to-Alert Speed of the sensor Are your alerts backing up on a DB somewhere? How often are sensors reporting back to their console? Knowledge of user (protein-based sensor) Do they know how to report shadiness? Comp Find Alert Give a s&*t? Taking action Stop the s&*t Clean up the s&*t
- 12. Alert-to-Give a s&*t How long do alerts linger? How long do emails about incidents bounced around inboxes? SIEM logs When analyst acknowledges the alert Comp Find Alert Give a s&*t? Taking action Stop the s&*t Clean up the s&*t
- 13. Give a s&*t-to-taking action Speed of triage & initial analysis Knowledge of internal organization Do your responders know who to call? Comprehensiveness of response plans and SOPs Comp Find Alert Give a s&*t? Taking action Stop the s&*t Clean up the s&*t I found the APT !!!
- 14. Taking Action-to-Stopping the s&*t Host Event log (shutdown) DHCP log AV log (deleted malz) Phish deleted Network ACL in switch IPS rule change log IP block added to router Firewall block added Proxy log Not when the rule was added, but when it was confirmed to be working Comp Find Alert Give a s&*t? Taking action Stop the s&*t Clean up the s&*t
- 15. Stopping-to-cleaning up the s&*t How long the business was impacted by the breach? Did the containment strategy conflict with or support recovery? How fast did you find other breaches? How effective was your recovery? The fed’s preferred recovery method Comp Find Alert Give a s&*t? Taking action Stop the s&*t Clean up the s&*t
- 16. Quality I hate motherf**kers claimin that they foldin bank But steady talkin s&*t in the holding tank First you wanna step to me Now you’re a** screamin for the deputy
- 17. Quality It’s great that you’re fast, but are you any good at it? Easy to confuse quality with forensic soundness Easy to confuse quality with expensive blinking boxes Quality really measures Are you focusing on what’s really important (customer)? Are you focusing on what really works (performance)? Do you track failures as much as you do successes (defects)? Do you learn from mistakes and do you repeat them (improvement)?
- 18. Comp Find Alert Give a s&*t? Taking action Stop the s&*t Clean up the s&*t First time right In this process, how often were mistakes made Do you track and categorize mistakes and misfires? How many times did you miss the breach? Did the alerts go to the right place the first time? Did the person viewing the alert make the right call? Did the person who gives a s&*t do the right thing? Did the actions actually stop the breach? Was your cleanup effective?
- 19. Measuring Quality Get granular Avoid “other” or “unknown” If given an option, analysts will choose “other” two out of every three times. Set goals What’s acceptable performance?
- 20. Forensics & Kill Chain Increasing ferocity of Ice Cube movie characters Increasing cost of response and recovery
- 22. Reconnaissance Weaponization Delivery Exploitation Installation C2 AoO Forensics & Kill Chain Know every system/person involved in the incident and how they performed – relative to the Kill Chain
- 23. Coverage Tricks wanna step to Cube and then they get played Cause they b&*ch made pullin out a switchblade That's kinda trifle, cause that's a knife-o [here’s an] AK-47, assault rifle
- 24. Coverage Are you looking for the right things in the right places? Filenames in IDS? IP addresses in AV logs? What percentage of your install base are you monitoring? First, check yo’self Use the Kill Chain Find your gaps
- 25. Check Yo’self How do you get got? Phishing? Watering holes? Thumbdrives? Websites getting popped? For one thing, you don’t know how the f**k my company be muthaf**king owned.
- 26. Check Yo’self Recon Weapo n Deliver Exploit Install C2 AoO #1 #2 #3 #4 #5 Attacks stopped by Kill Chain #6
- 27. Check Yo’self Recon Weapo n Deliver Exploit Install C2 AoO $$ $ $$ $$ $ $$ $$ $ $$ $ Cost of the intrusion
- 28. Check Yo’self Recon Weapo n Deliver Exploit Install C2 AoO $$ $$ $$ $$ $ $$ Cost of countermeasure s $
- 29. Finding Gaps Lack of process Misapplying Intel Bad deployment of web applications Lack of Training Developers building insecure apps Lack of technology Buy only when you have a clear blind spot Not every gap in yo’ security needs to be filled with cash money
- 30. Check yo ‘net Do you have every network ingress/egress point monitored? 3rd Parties/Suppliers VPN Mobile/BYOD Do you have monitoring on every network service? FTP, SFTP, Web, SMTP, Telnet (yes, telnet) Cloud services (*aaS) Gary’s manager found an un- instrumented PoP on the network
- 31. Check yo ‘boxes What is your host logging policy? Do your logs go to a central location? Do you have a method to search the endpoints and servers for IOCs? How agro are your patching policies? Will a Java patch f’ your network? http://bit.ly/1pTiodM - for other derp-ables referring to “the APT”
- 32. Takeaways Here to let you know boy, oh boy I make dough but don't call me DoughBoy This ain't no f**kin motion picture A guy or b^*ch-a, my fool get wit'cha And hit ya, takin that yack to the neck So you better run a check
- 33. Telling your story to management Know the real cost of your breach Your time Your team’s time Cost of recovery Client’s lost productivity Data loss Cost of R&D Profit Margin Know the real cost of countermeasures Training costs should include time away and travel Process improvements requires good data, discipline, and expertise If you’re buying a new tool, double the cost of deployment and add 50% to annual O&M
- 34. Telling your story to management $7 K $113 K $64 K $119 K $122 K $142 K $114 K $42 K $56 K $45 K $6 K$6 K $7 K $110 K $17 K $47 K $152 K $144 K $97 K $100 K $119 K $39 K $99 K $135 K $ K $20 K $40 K $60 K $80 K $100 K $120 K $140 K $160 K Jan-12Jan-12Jan-12Jan-12 Feb-12Feb-12Feb-12 Mar-12Mar-12Mar-12Mar-12 Apr-12Apr-12Apr-12 May-12May-12May-12 Jun-12Jun-12Jun-12Jun-12 Jul-12Jul-12Jul-12 Aug-12Aug-12Aug-12Aug-12 Sep-12Sep-12Sep-12 Oct-12Oct-12Oct-12 Nov-12Nov-12Nov-12Nov-12 Dec-12Dec-12Dec-12 Jan-13Jan-13Jan-13Jan-13 Feb-13Feb-13Feb-13 Mar-13Mar-13Mar-13 Apr-13Apr-13Apr-13 May-13May-13May-13May-13 Jun-13Jun-13Jun-13 Jul-13Jul-13Jul-13Jul-13 Aug-13Aug-13Aug-13 Sep-13Sep-13Sep-13 Oct-13Oct-13Oct-13Oct-13 Nov-13Nov-13Nov-13 Dec-13Dec-13Dec-13Dec-13 Jan-14Jan-14Jan-14 Feb-14Feb-14Feb-14 Mar-14Mar-14Mar-14Mar-14 Apr-14Apr-14Apr-14 May-14May-14May-14 Jun-14 Per-event cost of our large-scale intrusions (Jan ‘12 – Jul ’14) (# of days of full-scale response) x (daily rate of employee) x (# of employees involved in the response)
- 35. What point in the Kill Chain are attacks being stopped? Does it cost more to respond to events higher in the KC? Telling your story to management 0.00 2.00 4.00 6.00 8.00 10.00 12.00 14.00 0 100 200 300 400 500 600 700 800 900 1000 Recon Deliver Exploit Install C2 AoO Days NumberofIncidents
- 36. What systems are catching attacks from “the APT” Telling your story to management IDS 29% Host-Based Scanner 12% AV 12% Proxy Logs 7% User Report 6% Email Scanner 6% Frequency Analysis 5% Monthly Host Checker 4% IP/Domain Hotlist 4% SIEM Correlations 4% Event Logs 3% Other 2% Netflow 2% 3rd Party Notification 2% Cloud-based Proxy 1% IPS 1% Commercial Malware Analysis appliance 1% Registry Scanner 1% Email Logs 1%
- 37. Don’t buy me another chirping box Telling your story to management 0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 0 1 2 3 4 5 6 7 8 9 IDS Crmcl Malware Analysis Device McAfee User Report Email Scanner 3rd Party (Other) Event Logs Proxy Logs DaysofInvestigation #ofFalsePositives Detection Tool
- 38. y = -0.0958x + 12.279 R² = 0.01819 0 5 10 15 20 25 0 5 10 15 20 25 30 35 40 #ofDaysofFull-scaleResponse # of Analysts on IR Team More people, more problems Practically no correlation between having more people and being able to responding faster
- 39. Training vs. Tools Cost of Training an Analyst for a small network – 10K hosts SANS Course & Certification = ~$5,500 Travel & Meals = ~$1,500 Time Away from office = ~$1,750 Cost of OS IDS appliance(s) & management servers = $20,000 Cost of a commercial IDS solution = ~$50,000 - $150,000 Cost of a commercial SIEM product = ~$150,000-$200,000 Annual cost of MSSP services = ~$60,000-$120,000
- 40. Questions? @DaveTrollman (since Jul 10, 2014 – 245PM)