Downloaded 334 times
Uploaded byPeter Wood
Big Data and Security - Where are we now? (2015)
Peter Wood, CEO of First Base Technologies, discusses the current state of big data and its implications for security in a comprehensive presentation. He highlights the rapid growth of data, the increasing sophistication of cyber threats, and the potential of big data analytics to enhance security measures, while also addressing the challenges and privacy concerns associated with its use. The presentation emphasizes the necessity for organizations to adapt their security strategies in light of these developments and invest in big data security analytics.
More Related Content
Similar to Big Data and Security - Where are we now? (2015)
![[DSC Europe 25] Paula Garcia Esteban -Building the Future: The Role of Data S...](https://cdn.slidesharecdn.com/ss_thumbnails/9ld1r1bsqpwve8qfvphy-paula-garcia-esteban-building-the-future-260122103838-4171f5cb-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Marcos Heidemann - Beyond the Hype: Making AI Coding Assistan...](https://cdn.slidesharecdn.com/ss_thumbnails/eexkhvldrjsopspdjbur-marcos-heidemann-beyond-the-hype-getting-real-value-out-of-ai-assisted-coding-260121115910-7e9d41ec-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Tali Fulman - Guild Meetings, Then What? Building Data Commun...](https://cdn.slidesharecdn.com/ss_thumbnails/fgohhi33rwmhqdowdj5k-tali-fulman-guild-meetings-then-what-building-data-communities-that-actually-ch-260120105855-528492c3-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Egor Krasheninnikov - The Control Stack: Building Guardrails ...](https://cdn.slidesharecdn.com/ss_thumbnails/3lzcz7hxqmo51mtalv4u-the-control-stack-260119101520-ea90841a-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Srdj Stanisic - Local and Private AI in UX.pdf](https://cdn.slidesharecdn.com/ss_thumbnails/vwmetykqmztgmokmmkfa-3-srdjan-stanisic-local-and-small-ai-in-ux-260120105855-55a31869-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Bojan Banjac - AI is always right when it comes to the matter...](https://cdn.slidesharecdn.com/ss_thumbnails/syoxtqierpydwxm5srcb-4-bojan-banjac-ai-is-always-right-when-it-comes-to-the-matters-of-taste-260119101519-694ee7d7-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Milovan Jovicic - Beyond AI's Reach: The Enduring Value of Ev...](https://cdn.slidesharecdn.com/ss_thumbnails/pyeij0hurgwq5jugmtnv-2-milovan-jovicic-beyond-ais-reach-the-enduring-value-of-evergreen-design-v2-260120105856-d6ee57e5-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Mikhail Rozhkov - AI Product Canvas: From Business Goals to T...](https://cdn.slidesharecdn.com/ss_thumbnails/d53doddtpgfqivmzqel6-mikhail-rozhkov-ai-product-canvas-v1-260121115910-9dd517a7-thumbnail.jpg?width=640&height=640&fit=bounds)
Big Data and Security - Where are we now? (2015)
- 1. Peter Wood Chief ExecutiveOfficer First Base Technologies LLP Big Data and Security Where are we now?
- 2. Slide 2 ©First Base Technologies 2015 Who is Peter Wood? Worked in computers & electronics for 45 years Founded First Base in 1989 (the first ethical hackers in UK) Ethical hacker, security evangelist and public speaker • Fellow of the BCS, the Chartered Institute for IT • Chartered IT Professional • CISSP • Senior Member of the Information Systems Security Association (ISSA) • 15 Year+ Member of ISACA, Member of the ISACA Security Advisory Group • Member of the Institute of Information Security Professionals • Member of the BCS Information Risk Management and Assurance Group • Chair of white-hats.co.uk • UK Programme Chair for the Corporate Executive Programme • Member of ACM, IEEE, First Forensic Forum (F3), Institute of Directors • Member of Mensa
- 3. Slide 3 ©First Base Technologies 2015 Agenda • Big Data 101 • Advanced threats – state of play • Why Big Data for security? • How can Big Data help? • Big Data security challenges • Summing up
- 4. Slide 4 ©First Base Technologies 2015 Agenda • Big Data 101 • Advanced threats – state of play • Why Big Data for security? • How can Big Data help? • Big Data security challenges • Summing up
- 5. Slide 5 ©First Base Technologies 2015 Big Data is quite large Every day, we create 2.5 quintillion bytes of data — so much that 90% of the data in the world today has been created in the last two years alone. This data comes from everywhere: sensors used to gather climate information, posts to social media sites, digital pictures and videos, purchase transaction records, and cell phone GPS signals to name a few. http://www-01.ibm.com/software/data/bigdata/ IDC projects that the digital universe will reach 40 zettabytes by 2020, resulting in a 50-fold growth from the beginning of 2010 http://uk.emc.com/about/news/press/2012/20121211-01.htm 2.5 quintillion = 2.5 exabytes = 2.5x1018 bytes 40 zettabytes = 40x1021 bytes 57 times all the grains of sand on all the beaches on earth
- 6. Slide 6 ©First Base Technologies 2015 Big Data can be useful • Creating transparency by making relevant data more accessible • Enabling experimentation to discover needs, expose variability and improve performance - use data to analyse variability in performance and understand the root causes • Segmenting populations to customise actions and tailor products and services to meet specific needs • Replacing/supporting human decision-making with automated algorithms in order to minimise risk • Innovating new business models, products and services McKinsey Global Institute: “Big data: The next frontier for innovation, competition, and productivity”, May 2011
- 7. Slide 7 ©First Base Technologies 2015 Apache Hadoop • Created by Doug Cutting and Mike Cafarella in 2005 • Cutting named it after his son's toy elephant • The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models • It is designed to scale up from single servers to thousands of machines, each offering local computation and storage • Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer, so delivering a highly-available service on top of a cluster of computers, each of which may be prone to failures http://hadoop.apache.org/
- 8. Slide 8 ©First Base Technologies 2015 http://bradhedlund.com/2011/09/10/understanding-hadoop-clusters-and-the-network/
- 9. Slide 9 ©First Base Technologies 2015 http://bradhedlund.com/2011/09/10/understanding-hadoop-clusters-and-the-network/
- 10. Slide 10 ©First Base Technologies 2015
- 11. Slide 11 ©First Base Technologies 2015 Agenda • Big Data 101 • Advanced threats – state of play • Why Big Data for security? • How can Big Data help? • Big Data security challenges • Summing up
- 12. Slide 12 ©First Base Technologies 2015 Advanced Threats • Massive increase in advanced malware bypassing traditional security defences • Volumes vary substantially among different industries • Email-based attacks are growing, with link- and attachment-based malware presenting significant risks • Cybercriminals are increasingly employing limited-use domains in their spear phishing emails • Malicious email attachments growing more diverse, evading traditional security defences FireEye Advanced Threat Report
- 13. Slide 13 ©First Base Technologies 2015 Organisations on average are experiencing malware-related activities once every three minutes. This includes receipt of a malicious email, a user clicking a link on an infected website, or an infected machine making a call back to a command and control server. FireEye Advanced Threat Report
- 14. Slide 14 ©First Base Technologies 2015 The Post Breach Boom, Ponemon Institute Survey of 3,529 IT and IT security practitioners in US, Canada, UK, Australia, Brazil, Japan, Singapore and UAE
- 15. Slide 15 ©First Base Technologies 2015 The Post Breach Boom, Ponemon Institute Survey of 3,529 IT and IT security practitioners in US, Canada, UK, Australia, Brazil, Japan, Singapore and UAE
- 18. Slide 18 ©First Base Technologies 2015 Agenda • Big Data 101 • Advanced threats – state of play • Why Big Data for security? • How can Big Data help? • Big Data security challenges • Summing up
- 19. Slide 19 ©First Base Technologies 2015 The tipping point inputs Complex threat landscape: • Stealth malware • Targeted attacks • Social engineering New technologies and challenges: • Social networking • Cloud • BYOD / consumerisation • Virtualisation
- 20. Slide 20 ©First Base Technologies 2015 What do we do today? Traditional defences: • Signature-based anti-virus • Signature-based IDS/IDP • Firewalls and perimeter devices Traditional approach: • Data collection for compliance • Check-list mindset • Tactical thinking
- 21. Slide 21 ©First Base Technologies 2015 Why we need big data tools • System Log files that can grow by gigabytes per second • Network data captures, which can grow by 10s of gigabytes per second • Intrusion Detection/Protection log files that can grow by 10s of gigabytes per second • Application Log files that can grow by gigabytes per second http://www.virtualizationpractice.com/big-data-security-tools-22075/
- 22. Slide 22 ©First Base Technologies 2015 http://www.emc.com/collateral/industry-overview/sbic-rpt.pdf
- 23. Slide 23 ©First Base Technologies 2015 Agenda • Big Data 101 • Advanced threats – state of play • Why Big Data for security? • How can Big Data help? • Big Data security challenges • Summing up
- 24. Slide 24 ©First Base Technologies 2015 How can Big Data help? • SIEM on steroids? • Fraud detection • APT detection? • Integration of IT and physical security? • SIEM + IDS/IPS? • Predictive analysis
- 25. Slide 25 ©First Base Technologies 2015 Big Data to Collect • Logs • Network traffic • IT assets • Senstitive / valuable information • Vulnerabilities • Threat intelligence • Application behaviour • User behaviour
- 26. Slide 26 ©First Base Technologies 2015 Big Data Analytics • Real-time updates • Behaviour models • Correlation • Heuristic capability • Interoperability • … advising the analysts? • … active defence?
- 27. Slide 27 ©First Base Technologies 2015 Agenda • Big Data 101 • Advanced threats – state of play • Why Big Data for security? • How can Big Data help? • Big Data security challenges • Summing up
- 28. Slide 28 ©First Base Technologies 2015 Big Data Security Challenges • Bigger data = bigger breaches? • New technology = security later? • Information classification • Information ownership (outputs and raw data) • Big data in cloud + BYOD = more problems? • New security technologies (e.g. ABE)
- 29. Slide 29 ©First Base Technologies 2015 Big Data Security Risks • New technology will introduce new vulnerabilities • Potential for back doors and default credentials • Attack surface of the nodes in a cluster may not have been reviewed and servers adequately hardened • User authentication and access to data from multiple locations may not be sufficiently controlled • Regulatory requirements may not be fulfilled, with access to logs and audit trails problematic • Significant opportunity for malicious data input and inadequate data validation
- 30. Slide 30 ©First Base Technologies 2015 Big Data Privacy Concerns (1) • “De-Identifed” Information Can Be “Re-Identified”: data collectors claim that the aggregated information has been “de- identified”, however, it is possible to re-associate “anonymous” data with specific individuals, especially since so much information is linked with smartphones • Possible Deduction of Personally Identifiable Information: non- personal data could be used to make predictions of a sensitive nature, like sexual orientation, financial status, and the like • Risk of Data Breach Is Increased: The higher concentration of data, the more appealing a target it makes for hackers, and the greater impact as a result of the breach http://www.ftc.gov/public-statements/2012/03/big-data-big-issues
- 31. Slide 31 ©First Base Technologies 2015 Big Data Privacy Concerns (2) • "Creepy" Factor: Consumers are often unnerved when they feel that companies know more about them than they are willing to volunteer (the anecdote of Target sending baby related coupons to a teenage girl before she had even told her immediate family members about her new bundle of joy still stands as the benchmark horror story of invasive marketing) • Big Brother or Big Data: Municipalities are using Big Data for predictive policing and tracking potential terrorist activities. Concerns have been raised that such uses could become a slippery slope to using Big Data in a manner that infringes on individual rights, or could be used to deny consumers important benefits (such as housing or employment) in lieu of credit reports. http://www.ftc.gov/public-statements/2012/03/big-data-big-issues
- 32. Slide 32 ©First Base Technologies 2015 Agenda • Big Data 101 • Advanced threats – state of play • Why Big Data for security? • How can Big Data help? • Big Data security challenges • Summing up
- 33. Slide 33 ©First Base Technologies 2015 Big Data Adoption RSA said in 2013: Within the next two years, we predict big data analytics will disrupt the status quo in most information security product segments, including SIEM; network monitoring; user authentication and authorization; identity management; fraud detection; and governance, risk & compliance. Big Data Holds Big Promise For Security – RSA Security Brief, January 2013
- 34. Slide 34 ©First Base Technologies 2015 Summary • As with all new technologies, security in Big Data use cases seems to be an afterthought at best • Big Data breaches will be big too, with even more serious reputational damage and legal repercussions • All organisations need to invest in research and study of the emerging Big Data Security Analytics landscape • Big Data has the potential to defend against advanced threats, but requires a Big Re-think of approach • Relevant skills are key to successful deployment, only the largest organisations can invest in this now • Offerings exist for the other 97% that can enhance existing technologies using cloud-based solutions
- 35. Slide 35 ©First Base Technologies 2015 Peter Wood Chief Executive Officer First Base Technologies LLP peter@firstbase.co.uk http://firstbase.co.uk http://white-hats.co.uk http://peterwood.com Twitter: @peterwoodx Need more information?
Editor's Notes
- #9 The three major categories of machine roles in a Hadoop deployment are Client machines, Masters nodes, and Slave nodes. The Master nodes oversee the two key functional pieces that make up Hadoop: storing lots of data (HDFS), and running parallel computations on all that data (Map Reduce). The Name Node oversees and coordinates the data storage function (HDFS), while the Job Tracker oversees and coordinates the parallel processing of data using Map Reduce. Slave Nodes make up the vast majority of machines and do all the dirty work of storing the data and running the computations. Each slave runs both a Data Node and Task Tracker daemon that communicate with and receive instructions from their master nodes. The Task Tracker daemon is a slave to the Job Tracker, the Data Node daemon a slave to the Name Node. Client machines have Hadoop installed with all the cluster settings, but are neither a Master or a Slave. Instead, the role of the Client machine is to load data into the cluster, submit Map Reduce jobs describing how that data should be processed, and then retrieve or view the results of the job when its finished. In smaller clusters (~40 nodes) you may have a single physical server playing multiple roles, such as both Job Tracker and Name Node. With medium to large clusters you will often have each role operating on a single server machine. In real production clusters there is no server virtualization, no hypervisor layer. That would only amount to unnecessary overhead impeding performance. Hadoop runs best on Linux machines, working directly with the underlying hardware.
- #25 The deployment of Big Data for fraud detection, and in place of security incident and event management (SIEM) systems, is attractive to many organisations. The overheads of managing the output of traditional SIEM and logging systems are proving too much for most IT departments and Big Data is seen as a potential saviour. There are commercial replacements available for existing log management systems, or the technology can be deployed to provide a single data store for security event management and enrichment. Taking the idea a step further, the challenge of detecting and preventing advanced persistent threats may be answered by using Big Data style analysis. These techniques could play a key role in helping detect threats at an early stage, using more sophisticated pattern analysis, and combining and analysing multiple data sources. There is also the potential for anomaly identification using feature extraction. Today logs are often ignored unless an incident occurs. Big Data provides the opportunity to automatically consolidate and analyse logs from multiple sources rather than in isolation. This could provide insight that individual logs cannot, and potentially enhance intrusion detection systems (IDS) and intrusion prevention systems (IPS) through continual adjustment and effectively learning “good” and “bad” behaviours. Integrating information from physical security systems, such as building access controls and even CCTV, could also significantly enhance IDS and IPS to a point where insider attacks and social engineering are factored in to the detection process. This presents the possibility of significantly more advanced detection of fraud and criminal activities. We know that organisational silos often reduce the effectiveness of security systems, so businesses must be aware that the potential effectiveness of Big Data style analysis can also be diluted unless these issues are addressed. At the very least, Big Data could result in far more practical and successful SIEM, IDS and IPS implementations.
- #27 Data collection and storage The ability to collect information from multiple dimensions of the organisation is essential to provide visibility across the infrastructure and to ensure that there are no gaps in protection. This should include perimeter security controls such as antivirus and firewalls, all endpoints and every system connected to the network, including custom applications, embedded systems, removable media and physical access control records. For incident response and forensic purposes, all information should be encrypted, compressed, time stamped and stored in a secure archive. This will also enable the organisation to comply with the data retention requirements of the regulations and industry standards that apply to them. Big data analytics The sheer volume of the data requires that the system is integrated, scalable and extensible, with all processes highly automated. Early SIEM and log management systems were criticised for their inability to effectively analyse all the data collected, as many sources were stored in isolation and involved too many manual processes. What is required is big data analytics capabilities that provide advanced data aggregation, event correlation and pattern recognition across all dimensions of the big data sets collected using techniques that include statistical and heuristic analysis. It is necessary that the system performs continuous monitoring on a real time basis in order to be able to detect threats as they occur and that all the information is stored in a secure repository for use in forensic investigations to find the root cause of events that have occurred. Behavioural analysis The system should include integrated behavioural analytical capabilities that can automatically establish what constitutes expected and accepted behaviour for all systems, devices and users connected to the network—a process that all too often requires manual intervention in many first-generation SIEM and log management systems. Accepted behaviour for all those systems can then be whitelisted so that unexpected or suspicious behaviour can be flagged and alerted so that remediation steps can be taken. This also means that known good behaviour can be eliminated from any forensic review that is required. Integrity monitoring To ensure that internal threats are countered, such as changes made to files or configurations that could introduce vulnerabilities, organisations should look for a security intelligence platform with integrated file integrity and change management capabilities. Using behavioural analytics, multiple disparate data sets can be combined to look for behavioural patterns and risk factors that can provide indications of when and where advanced attacks have occurred so that remediation can be taken faster, focused on the highest priority events that have been uncovered. Threat intelligence feeds To turn log and event feeds into actionable security intelligence that can drive automated remediation, intelligence feeds should be included from other sources that include vulnerability data, identity and access management events, asset classification information, metadata, geolocation information and real-time threat intelligence feeds garnered from a variety of sources. Making sense of this information and its dependencies requires advanced correlation and pattern recognition capabilities that can uncover all data patterns and associate them with particular users and devices. Real time, continuous monitoring In early systems, much of the information that was uncovered through analysis and correlation would show events that had occurred for forensic investigation. However, whilst this is still a key requirement, this is insufficient for countering the dynamic, advanced threats seen today. Rather, the threat of a breach occurring that exposes sensitive information requires that all information is analysed and correlated in real time. This is only possible if the system provides continuous, real time protective monitoring of all activity, including network and host connections, user access events and behaviour, removable media activity, and processes and services that are running on all systems connected to the network. The types of activity that should be continuously monitored in real time are shown in Appendix 1 6 Unified management platform One further criticism of early SIEM and log management systems was that they were difficult to manage and use. To ease management tasks, organisations should look for a system that combines the capabilities described above into one integrated security intelligence platform, accessed through one central console that provides an intuitive user interface to wizard-driven processes. This will provide organisations with a single, consolidated view across events occurring in all parts of the network and will allow them to investigate those events in context. That console should provide access to easy-to-understand reports related to security, compliance and operational issues throughout the entire technology stack of the network.
- #29 Many businesses already use Big Data for marketing and research, yet may not have the fundamentals right, particularly from a security perspective. As with all new technologies, security seems to be an afterthought at best. Big Data breaches will be big too, with the potential for even more serious reputational damage and legal repercussions than at present. A growing number of companies are using the technology to store and analyse petabytes of data including web logs, click stream data and social media content to gain better insights about their customers and their business. As a result, information classification becomes even more critical; and information ownership must be addressed to facilitate any reasonable classification. Most organisations already struggle with implementing these concepts, making this a significant challenge. We will need to identify owners for the outputs of Big Data processes as well as the raw data. Thus Data Ownership will be distinct from Information Ownership, perhaps with IT owning the raw data and business units taking responsibility for the outputs. Very few organisations are likely to build a Big Data environment in-house, so cloud and Big Data will be inextricably linked. As many businesses are aware, storing data in the cloud does not remove their responsibility for protecting it - from both a regulatory and a commercial perspective. Techniques such as Attribute Based Encryption may be necessary to protect sensitive data and apply access controls (being attributes of the data itself, rather than the environment in which it is stored). Many of these concepts are foreign to businesses today.