Cyber Security Breakfast Briefing

2. YOUR SPEAKER –
• 2014 HEAD OF INFORMATION SECURITY – WORLDLINE (ATOS GROUP)
• 2014 CISO LEVEL SECURITY, RISK & COMPLIANCE CONSULTANCY ACROSS EUROPE
• 2013 PCIDSS COMPLIANCE AT WALMART FOR ASDA & GEORGE (LEVEL ONE MERCHANT)
• 2011 - 2013 PCIDSS COMPLIANCE MANCHESTER AIRPORTS GROUP (LEVEL THREE MERCHANT)
• 2006-2011 PCIDSS COMPLIANCE HOMELOAN MANAGEMENT LIMITED (LEVEL ONE SERVICE PROVIDER)
• 2006 ECOMMERCE SECURITY– THOMAS COOK SCHEDULED BUSINESS

3. EXEC SUMMARY
• QUICK LOOK BACK OVER LAST TEN YEARS
• QUICK LOOK AT MY FAVOURITE BACKGROUND READING
• AT A FORK IN THE ROAD - KPI V SECVIZ

4. Looking Back…
35

5. INTRODUCTION (FROM 2003)
•IT SECURITY METRICS PROVIDE A PRACTICAL APPROACH TO MEASURING
INFORMATION SECURITY.
•EVALUATING SECURITY AT THE SYSTEM LEVEL, IT SECURITY METRICS ARE TOOLS THAT
FACILITATE DECISION MAKING AND ACCOUNTABILITY THROUGH COLLECTION,
ANALYSIS, AND REPORTING OF RELEVANT PERFORMANCE DATA.
34

6. OLD SCHOOL METRICS TUTORIAL
• DAN GEER
• 426 PAGES
• 2007
• GEER - HTTP://GEER.TINHO.NET/MEASURINGSECURITY.TUTORIAL.PDF
33

7. NIST SPECIAL PAPER
• NIST SP800 SERIES
• TECH STANDARDS
• US GOV
• 2008
32

8. CIS CONSENSUS ON IS METRICS
• HTTPS://BENCHMARKS.CISECURITY.ORG/DOWNLOADS/METRICS/
31

9. GOOD BOOKS
• THERE ARE PLENTY OF BOOKS OUT THERE – NEW ONES BEING PUBLISHED ALL THE TIME
• CAROLYN WONG -2011
• LANCE HAYDEN -2010
30

10. WWW.SANS.ORG READING ROOM
• FAVOURITE DASHBOARD PAPER
• 2010
29

11. SANS PAPER
• EXAMPLES
28

12. Know your enemy
27

13. HOW TO LIE WITH STATISTICS 1954
• DARRELL HUFF
26

14. HUFF HTLWS
• 2009
• AND NEW
• HTTP://CSEWEB.UCSD.EDU/~RICKO/CSE3/LIE_WITH_STATISTICS.PDF
25

15. 24

16. BLACK HAT 2013 - TALK
• STEVE CHRISTIE
• BRIAN MARTIN
23

17. BLACK HAT TALK – UPDATE 2015
• HTTP://BLOG.OSVDB.ORG/CATEGORY/VULNERABILITY-STATISTICS/
22

18. Sample
21

19. Why Vulnerability Stats Suck
• Stats are presented without understanding the limits of the data
• Even if explanations are provided, correlation is confused with causation:
20

20. Talking Points
• Defining Bias
• Researcher Bias
• Vendor Bias
• VDB Bias
• Bad Stats
• Good(ish) Stats
• Conclusion
19

21. Disease Research:
Epidemiology vs. Vulnerability Research
Epidemiology Vulnerability Research
Goal Improve the public health
SAVE ALL THE THINGZ ON THA INTERWEBZ! *
(attention whoring)
Objects of Study People/Diseases Software/Vulnerabilities
Populations Groups of people
Groups of vulnerabilities (as seen in multi-vuln
disclosures)
Measurement
Devices (Tools of the
Trade)
Blood pressure monitors,
thermometers, lab tests, observation
Automated code scanners w/high FP/FN rates,
fuzzers, coffee-fueled malcontents staring at code at
3 AM
Publication
Requirements
Refereed journals with peer review Ability to send email
Sampling Methods
Using industry established
methodologies and formal
documentation.
Using wildly erratic methodologies, no standards for
documentation or disclosure
* Goal not shared by all researchers. Please to be rolling with this, kthxbye
18

22. The Shocking Claim
• Bias and statistics in vulnerability research are far worse than
it is in other disciplines
• At least people don’t die (yet?), but still use vulnerable
equipment:
– SCADA
– Airplanes
– Automobiles
– Medical Devices
– Oh my…
17

23. KPI / KRI
16

24. SECURITY EFFORT / PERFORMANCE
• WITH KRI
15

25. SECURITY EFFORT / PERFORMANCE
• WITH KPI
14

26. lets get visual
13

27. THE BOOKS
• APPLIEDSECURITY VISUALIZATION DATA-DRIVEN SECURITY
12

28. SECVIZ AND AFTERGLOW
• SITE WWW.SECVIZ.ORG AND TOOL AFTERGLOW (PERL)
11

29. DATA DRIVEN SECURITY BLOG/PODCAST
10

30. INDEPENDENT REVIEW OF DDS
• HTTP://HOLISTICINFOSEC.ORG/TOOLSMITH/PDF/SEPTEMBER2014.PDF
9

31. THE OTHER DATA DRIVEN SECURITY
• HTTPS://WWW.TRUSTWORTHYINTERNET.ORG/DATA-DRIVEN-SECURITY/
8

32. PATERVA MALTEGO TRANSFORM TOOL
• MALWAREINVESTIGATIONS EXAMPLES
2010 2013 2014
7

33. CROWDSTRIKE – DEEP PANDA
6

34. OPENDNS VIDEO AND GRAPHICS
• HTTPS://WWW.OPENDNS.COM/2013
• HTTPS://WWW.OPENDNS.COM/2014
5

35. OPENDNS FREE TOOLS
4

36. REALTIME MAPS – PURE MARKETING
• HTTP://CYBERMAP.KASPERSKY.COM/ HTTP://HTTP://MAP.IPVIKING.COM/
3

37. OTHER RT MAPS
• HTTP://WWW.THREATMETRIX.COM/THREATMETRIX-LABS/WEB-FRAUD-MAP/
• HTTP://WWW.FIREEYE.COM/CYBER-MAP/THREAT-MAP.HTML
• HTTP://WWW.DIGITALATTACKMAP.COM/#ANIM=1&COLOR=0&COUNTRY=ALL&TIME=16352&VIEW=MAP
• HTTP://WWW.SICHERHEITSTACHO.EU/
• HTTP://MASTDB3.MCAFEE.COM/VIRUSMAP3.ASP?NAME=VIRUSMAP&B=IE&LEFT=-162.96&BOTTOM=13.2&RIGHT=-
42.96&TOP=73.2&LANG=EN&OVB=2&FT=JPEG&OCM=1&VIEWBY=2&TRACK=4&PERIOD=1&CHOOSEMAP=1&CMD=ZOO
MIN
• HTTP://CERT.EUROPA.EU/BIGSCREENMAP/
2

38. THREATBUTT
• CYBER POMPEII
1

39. Time is precious, thank you for yours
https://uk.linkedin.com/in/jmck4cybersecurity