The document discusses how big data and data analytics can help with security intelligence and detecting advanced threats. It notes that every day 2.5 quintillion bytes of data are created, with 90% created in the last two years. It then outlines current security threats like malware events occurring every 3 minutes and how most breaches are caused by human error or took months to discover. Big data analytics can help with tasks like anomaly detection, correlation of multiple data sources, and predictive analysis to help detect threats. However, big data also poses new security challenges around privacy, ownership of data, and introducing new vulnerabilities.

1. Data Analytics for Security
Intelligence
Camil Demetrescu
Dept. Computer, Control, and Management
Engineering
Credits: Peter Wood, First Base Technologies LLP
Data Driven Innovation Rome 2016 – Open Summit
Roma Tre University, May 20 2016

2. Outline
• Big data
• Advanced threats – current situation
• Why big data for security?
• How can big data help?
• Big data security challenges
• Conclusions
20/5/2016Data Driven Innovation Rome 2016 Page 2

3. Big data
Every day, we create 2.5 quintillion bytes of data.
90% of the data in the world today has been created in the
last two years alone.
http://www-01.ibm.c/software/data/bigdata/
2.5 quintillion = 2.5 exabytes = 2.5 x 1018 =
2.500.000.000.000.000.000 bytes
• Sensors used to gather climate information
• Posts to social media sites
• Digital pictures and videos
• Purchase transaction records
• Cell phone GPS signals
20/5/2016Data Driven Innovation Rome 2016 Page 3

4. 20/5/2016Data Driven Innovation Rome 2016 Page 4

5. Outline
• Big data
• Advanced threats – current situation
• Why big data for security?
• How can big data help?
• Big data security challenges
• Conclusions
20/5/2016Data Driven Innovation Rome 2016 Page 5

6. Malware events per hour
20/5/2016Data Driven Innovation Rome 2016 Page 6
Organisations on average are
experiencing malware-related
activities once every three
minutes.
Receipt of a malicious email, a
user clicking a link on an
infected website, or an infected
machine making a call back to a
command and control server.
FireEyeAdvancedThreatReport2012

7. How breach occurred
20/5/2016Data Driven Innovation Rome 2016 Page 7
The Post Breach Boom, Ponemon Institute 2015
Survey of 3,529 IT and IT security practitioners

8. When the breach was discovered
20/5/2016Data Driven Innovation Rome 2016 Page 8
The Post Breach Boom, Ponemon Institute 2015
Survey of 3,529 IT and IT security practitioners

9. Reasons for failing to prevent the breach
20/5/2016Data Driven Innovation Rome 2016 Page 9
ThePostBreachBoom,PonemonInstitute2015
Surveyof3,529ITandITsecuritypractitioners

10. Extrapolated cost of breach
20/5/2016Data Driven Innovation Rome 2016 Page 10
ThePostBreachBoom,PonemonInstitute2015
Surveyof3,529ITandITsecuritypractitioners

11. Outline
• Big data
• Advanced threats – current situation
• Why big data for security?
• How can big data help?
• Big data security challenges
• Conclusions
20/5/2016Data Driven Innovation Rome 2016 Page 11

12. Data driven information security: examples
20/5/2016Data Driven Innovation Rome 2016 Page 12
• Analyze system/applications log files
• Analyze network traffic
• Identify anomalies and suspicious activities
• Correlate multiple sources of information into a
coherent view

13. Why do we need big data systems?
20/5/2016Data Driven Innovation Rome 2016 Page 13
• System Log files that can grow by gigabytes per
second
• Network data captures, which can grow by 10s of
gigabytes per second
• Intrusion Detection/Protection log files that can
grow by 10s of gigabytes per second
• Application Log files that can grow by gigabytes per
second
http://www.virtualizationpractice.com/big-data-security-tools-22075/

14. Traditional scenarios
Traditional defences:
• Signature-based anti-virus
• Signature-based IDS/IDP
• Firewalls and perimeter devices
Traditional approach:
• Data collection for compliance
• Check-list mindset
• Tactical thinking
20/5/2016Data Driven Innovation Rome 2016 Page 14

15. New challenges
Complex threat landscape:
• Stealth malware
• Targeted attacks
• Social engineering
New technologies and challenges:
• Social networking
• Cloud
• BYOD / consumerisation
• Virtualisation
20/5/2016Data Driven Innovation Rome 2016 Page 15

16. Conventional vs. advanced approaches
20/5/2016Data Driven Innovation Rome 2016 Page 16
http://www.emc.com/collateral/industry-overview/sbic-rpt.pdf

17. Outline
• Big data
• Advanced threats – current situation
• Why big data for security?
• How can big data help?
• Big data security challenges
• Conclusions
20/5/2016Data Driven Innovation Rome 2016 Page 17

18. Data-driven information security:
early times
20/5/2016Data Driven Innovation Rome 2016 Page 18
• Bank fraud detection and anomaly-based intrusion
detection systems.
• Credit card companies have conducted fraud
detection for decades.
• Custom-built infrastructure to mine big data for fraud
detection was not economical to adapt for other
fraud detection uses (healthcare, insurance, etc.)
Cloud Security Alliance

19. Data analytics for intrusion detection
20/5/2016Data Driven Innovation Rome 2016 Page 19
Intrusion detection systems – Security architects
realized the need for layered security (e.g., reactive
security and breach response) because a system
with 100% protective security is impossible.
1st
generation
2nd
generation
Security information and event management (SIEM)
– aggregate and filter alarms from many sources
and present actionable information to security
analysts.
3rd
generation
Big data analytics in security (2nd generation SIEM)
– correlating, consolidating, and contextualizing
diverse security event information, correlating long-
term historical data for forensic purposes

20. How can big data analytics help?
• Advanced persistent threat (APT) detection?
• Integration of IT and physical security?
• Predictive analysis
• Real-time updates
• Behaviour models
• Correlation
• … advising the analysts?
• … active defence?
20/5/2016Data Driven Innovation Rome 2016 Page 20

21. How can big data analytics help?
20/5/2016Data Driven Innovation Rome 2016 Page 21

22. Outline
• Big data
• Advanced threats – current situation
• Why big data for security?
• How can big data help?
• Big data security challenges
• Conclusions
20/5/2016Data Driven Innovation Rome 2016 Page 22

23. Big data security challenges
• Bigger data = bigger breaches?
• New technology = security later?
• Information classification
• Information ownership (outputs and raw data)
• Big data in cloud + BYOD = more problems?
20/5/2016Data Driven Innovation Rome 2016 Page 23

24. Big data security risks
• New technology will introduce new vulnerabilities
• Attack surface of the nodes in a cluster may not have been
reviewed and servers adequately hardened
• User authentication and access to data from multiple
locations may not be sufficiently controlled
• Regulatory requirements may not be fulfilled, with access to
logs and audit trails problematic
• Significant opportunity for malicious data input and
inadequate data validation
20/5/2016Data Driven Innovation Rome 2016 Page 24

25. Big data privacy concerns
• De-identifed information may be re-identified
• Possible deduction of personally identifiable information
• Risk of data breach is increased
• "Creepy" Factor: consumers may feel that companies
know more about them than they are willing to volunteer
• Big brother: predictive policing and tracking potential
terrorist activities. Harm individual rights or deny
consumers important benefits (such as housing or
employment) in lieu of credit reports.
http://www.ftc.gov/public-statements/2012/03/big-data-big-issues

26. Outline
• Big data
• Advanced threats – current situation
• Why big data for security?
• How can big data help?
• Big data security challenges
• Conclusions
20/5/2016Data Driven Innovation Rome 2016 Page 26

27. Conclusions
20/5/2016Data Driven Innovation Rome 2016 Page 27
• As with all new technologies, security in big data use
cases seems to be an afterthought at best
• Big data breaches will be big too, with even more
serious reputational damage and legal repercussions
• All organisations need to invest in research and study of
the emerging big data security analytics landscape
• Big data has the potential to defend against advanced
threats, but requires a big re-think of approach
• Relevant skills are key to successful deployment, only the
largest organisations can invest in this now

28. Big data to collect
• Logs
• Network traffic
• IT assets
• Sensitive / valuable information
• Vulnerabilities
• Threat intelligence
• Application behaviour
• User behaviour
20/5/2016Data Driven Innovation Rome 2016 Page 28