Big Data Analytics to Enhance Security
Predictive Analtycis and Data Science Conference May 27-28
Anapat Pipatkitibodee
Technical Manager
anapat.p@Stelligence.com
Traditional data storage and analytic tools no longer provide the agility and flexibility required to deliver relevant business insights. That’s why organizations are shifting to a data lake architecture. This approach allows you to store massive amounts of data in a central location so it's readily available to be categorized, processed, analyzed, and consumed by diverse organizational groups. In this session, we’ll assemble a data lake using services such as Amazon S3, Amazon Kinesis, Amazon Athena, Amazon EMR, and AWS Glue.
The complexity of implementing and maintaining IBM Guardium or a native audit solution within an enterprise environment can quickly run into trouble. Escalating costs, manularity, and gaps in coverage put your company at risk of a failed audit or data breach. This presentation will share the experiences of Imperva customers who have moved from native audit or Guardium to Imperva SecureSphere for database audit and protection (DAP).
Viewers will leave with an understanding of:
- Security and compliance factors that organizations should consider
- The methods of deployment within an enterprise environment
- The monetary and human costs associated with each DAP architecture
DAS Slides: Data Quality Best PracticesDATAVERSITY
Tackling Data Quality problems requires more than a series of tactical, one-off improvement projects. By their nature, many Data Quality problems extend across and often beyond an organization. Addressing these issues requires a holistic architectural approach combining people, process, and technology. Join Nigel Turner and Donna Burbank as they provide practical ways to control Data Quality issues in your organization.
The right architecture is key for any IT project. This is especially the case for big data projects, where there are no standard architectures which have proven their suitability over years. This session discusses the different Big Data Architectures which have evolved over time, including traditional Big Data Architecture, Streaming Analytics architecture as well as Lambda and Kappa architecture and presents the mapping of components from both Open Source as well as the Oracle stack onto these architectures.
Traditional data storage and analytic tools no longer provide the agility and flexibility required to deliver relevant business insights. That’s why organizations are shifting to a data lake architecture. This approach allows you to store massive amounts of data in a central location so it's readily available to be categorized, processed, analyzed, and consumed by diverse organizational groups. In this session, we’ll assemble a data lake using services such as Amazon S3, Amazon Kinesis, Amazon Athena, Amazon EMR, and AWS Glue.
The complexity of implementing and maintaining IBM Guardium or a native audit solution within an enterprise environment can quickly run into trouble. Escalating costs, manularity, and gaps in coverage put your company at risk of a failed audit or data breach. This presentation will share the experiences of Imperva customers who have moved from native audit or Guardium to Imperva SecureSphere for database audit and protection (DAP).
Viewers will leave with an understanding of:
- Security and compliance factors that organizations should consider
- The methods of deployment within an enterprise environment
- The monetary and human costs associated with each DAP architecture
DAS Slides: Data Quality Best PracticesDATAVERSITY
Tackling Data Quality problems requires more than a series of tactical, one-off improvement projects. By their nature, many Data Quality problems extend across and often beyond an organization. Addressing these issues requires a holistic architectural approach combining people, process, and technology. Join Nigel Turner and Donna Burbank as they provide practical ways to control Data Quality issues in your organization.
The right architecture is key for any IT project. This is especially the case for big data projects, where there are no standard architectures which have proven their suitability over years. This session discusses the different Big Data Architectures which have evolved over time, including traditional Big Data Architecture, Streaming Analytics architecture as well as Lambda and Kappa architecture and presents the mapping of components from both Open Source as well as the Oracle stack onto these architectures.
Big Data - The 5 Vs Everyone Must KnowBernard Marr
This slide deck, by Big Data guru Bernard Marr, outlines the 5 Vs of big data. It describes in simple language what big data is, in terms of Volume, Velocity, Variety, Veracity and Value.
A changing market landscape and open source innovations are having a dramatic impact on the consumability and ease of use of data science tools. Join this session to learn about the impact these trends and changes will have on the future of data science. If you are a data scientist, or if your organization relies on cutting edge analytics, you won't want to miss this!
Deep Learning Explained: The future of Artificial Intelligence and Smart Netw...Melanie Swan
This talk provides an overview of an important emerging artificial intelligence technology, deep learning neural networks. Deep learning is a branch of computer science focused on machine learning algorithms that model and make predictions about data. A key distinction is that deep learning is not merely a software program, but a new class of information technology that is changing the concept of the modern technology project by replacing hard-coded software with a capacity to learn and execute tasks. In the future, deep learning smart networks might comprise a global computational infrastructure tackling real-time data science problems such as global health monitoring, energy storage and transmission, and financial risk assessment.
DataOps - The Foundation for Your Agile Data ArchitectureDATAVERSITY
Achieving agility in data and analytics is hard. It’s no secret that most data organizations struggle to deliver the on-demand data products that their business customers demand. Recently, there has been much hype around new design patterns that promise to deliver this much sought-after agility.
In this webinar, Chris Bergh, CEO and Head Chef of DataKitchen will cut through the noise and describe several elegant and effective data architecture design patterns that deliver low errors, rapid development, and high levels of collaboration. He’ll cover:
• DataOps, Data Mesh, Functional Design, and Hub & Spoke design patterns;
• Where Data Fabric fits into your architecture;
• How different patterns can work together to maximize agility; and
• How a DataOps platform serves as the foundational superstructure for your agile architecture.
Data Lake Architecture – Modern Strategies & ApproachesDATAVERSITY
Data Lake or Data Swamp? By now, we’ve likely all heard the comparison. Data Lake architectures have the opportunity to provide the ability to integrate vast amounts of disparate data across the organization for strategic business analytic value. But without a proper architecture and metadata management strategy in place, a Data Lake can quickly devolve into a swamp of information that is difficult to understand. This webinar will offer practical strategies to architect and manage your Data Lake in a way that optimizes its success.
This deck is from Interpol Conference 2017, these slides shows the holistic view of machine learning in cyber security for better organization readiness
The data lake has become extremely popular, but there is still confusion on how it should be used. In this presentation I will cover common big data architectures that use the data lake, the characteristics and benefits of a data lake, and how it works in conjunction with a relational data warehouse. Then I’ll go into details on using Azure Data Lake Store Gen2 as your data lake, and various typical use cases of the data lake. As a bonus I’ll talk about how to organize a data lake and discuss the various products that can be used in a modern data warehouse.
This presentation is prepared by one of our renowned tutor "Suraj"
If you are interested to learn more about Big Data, Hadoop, data Science then join our free Introduction class on 14 Jan at 11 AM GMT. To register your interest email us at info@uplatz.com
Big Data - The 5 Vs Everyone Must KnowBernard Marr
This slide deck, by Big Data guru Bernard Marr, outlines the 5 Vs of big data. It describes in simple language what big data is, in terms of Volume, Velocity, Variety, Veracity and Value.
A changing market landscape and open source innovations are having a dramatic impact on the consumability and ease of use of data science tools. Join this session to learn about the impact these trends and changes will have on the future of data science. If you are a data scientist, or if your organization relies on cutting edge analytics, you won't want to miss this!
Deep Learning Explained: The future of Artificial Intelligence and Smart Netw...Melanie Swan
This talk provides an overview of an important emerging artificial intelligence technology, deep learning neural networks. Deep learning is a branch of computer science focused on machine learning algorithms that model and make predictions about data. A key distinction is that deep learning is not merely a software program, but a new class of information technology that is changing the concept of the modern technology project by replacing hard-coded software with a capacity to learn and execute tasks. In the future, deep learning smart networks might comprise a global computational infrastructure tackling real-time data science problems such as global health monitoring, energy storage and transmission, and financial risk assessment.
DataOps - The Foundation for Your Agile Data ArchitectureDATAVERSITY
Achieving agility in data and analytics is hard. It’s no secret that most data organizations struggle to deliver the on-demand data products that their business customers demand. Recently, there has been much hype around new design patterns that promise to deliver this much sought-after agility.
In this webinar, Chris Bergh, CEO and Head Chef of DataKitchen will cut through the noise and describe several elegant and effective data architecture design patterns that deliver low errors, rapid development, and high levels of collaboration. He’ll cover:
• DataOps, Data Mesh, Functional Design, and Hub & Spoke design patterns;
• Where Data Fabric fits into your architecture;
• How different patterns can work together to maximize agility; and
• How a DataOps platform serves as the foundational superstructure for your agile architecture.
Data Lake Architecture – Modern Strategies & ApproachesDATAVERSITY
Data Lake or Data Swamp? By now, we’ve likely all heard the comparison. Data Lake architectures have the opportunity to provide the ability to integrate vast amounts of disparate data across the organization for strategic business analytic value. But without a proper architecture and metadata management strategy in place, a Data Lake can quickly devolve into a swamp of information that is difficult to understand. This webinar will offer practical strategies to architect and manage your Data Lake in a way that optimizes its success.
This deck is from Interpol Conference 2017, these slides shows the holistic view of machine learning in cyber security for better organization readiness
The data lake has become extremely popular, but there is still confusion on how it should be used. In this presentation I will cover common big data architectures that use the data lake, the characteristics and benefits of a data lake, and how it works in conjunction with a relational data warehouse. Then I’ll go into details on using Azure Data Lake Store Gen2 as your data lake, and various typical use cases of the data lake. As a bonus I’ll talk about how to organize a data lake and discuss the various products that can be used in a modern data warehouse.
This presentation is prepared by one of our renowned tutor "Suraj"
If you are interested to learn more about Big Data, Hadoop, data Science then join our free Introduction class on 14 Jan at 11 AM GMT. To register your interest email us at info@uplatz.com
Dat de snelle ontwikkelingen op het gebied van digitale technologie de samenleving veranderen is overduidelijk. We beseffen alleen nog niet altijd hoe ingrijpend die veranderingen zijn. Het gaat niet enkel over automatisering, of over het efficiënter maken van wat we al deden door het inzetten van nieuwe technologie. Steeds meer merk je dat de digitale revolutie de regels van het spel zelf verandert. Neem Uber als voorbeeld: het grootste taxibedrijf ter wereld bezit zelf geen enkele auto en zet zo het klassieke model van zaken doen helemaal op zijn kop.
Ook in de sfeer van educatie, gemeenschapsvorming, maatschappelijke en culturele actie zie je gelijkaardige ontwikkelingen.
In een dergelijke context verandert de manier waarop we aan sociaal-cultureel werk doen fundamenteel. De digitale transformatie dwingt je om de rol die je als organisatie speelt en de manier waarop je je doelen nastreeft grondig te bevragen en voor een stuk te herdefiniëren.
Hoe kunnen we digitale transformatie echt begrijpen? Wat betekenen deze ontwikkelingen voor het sociaal-cultureel werk? En hoe kunnen we ons ertoe verhouden? Jo Caudron, co-auteur van het boek 'Digital transformation', legt aan de hand van 7 metaforen uit wat digitale transformatie precies inhoudt.
Presentatie op studiedag "Sociaal-cultureel werk in digitale transformatie" (9 juni 2016).
Big Data Analytics to Enhance Security คุณอนพัทย์ พิพัฒน์กิติบดี Technical Ma...BAINIDA
Big Data Analytics to Enhance Security คุณอนพัทย์ พิพัฒน์กิติบดี Technical Manager, Stelligence ในงาน THE FIRST NIDA BUSINESS ANALYTICS AND DATA SCIENCES CONTEST/CONFERENCE จัดโดย คณะสถิติประยุกต์และ DATA SCIENCES THAILAND
Delivering User Behavior Analytics at Apache Hadoop Scale : A new perspective...Cloudera, Inc.
Learn how to:
* Detect threats automatically and accurately
* Reduce threat response times from 7 days to 4 hour
* Ingest and process 100+TB per day for automated machine learning and behavior-based detection
Preparing for the Cybersecurity RenaissanceCloudera, Inc.
We are in the midst of a fundamental shift in the way in which organizations protect themselves from the modern adversary.
Traditional rules based cybersecurity applications of the past are not able to protect organizations in the new mobile, social, and hyper-connected world they now operate within. However, the convergence of big data technology, analytic advancements, and a variety of other factors have sparked a cybersecurity renaissance that will forever change the way in which organizations protect themselves.
Join Rocky DeStefano, Cloudera's Cybersecurity subject matter expert, as he explores how modern organizations are protecting themselves from more frequent, sophisticated attacks.
During this webinar you will learn about:
The current challenges cybersecurity professionals are facing today
How big data technologies are extending the capabilities of cybersecurity applications
Cloudera customers that are future proofing their cybersecurity posture with Cloudera’s next generation data and analytics management system
Big Data For Threat Detection & ResponseHarry McLaren
Slides used at the University of Edinburgh SIGINT group (cybersecurity society). Covering what is big data, the value for security use cases, hunting for threats/actions, using Splunk to detect and respond, SIEM use and some useful searches (which were demoed).
2016 Cybersecurity Analytics State of the UnionCloudera, Inc.
3 Things to Learn About:
-Ponemon Institute's 2016 big data cybersecurity analytics research report
-Quantifiable returns organizations are seeing with big data cybersecurity analytics
-Trends in the industry that are affecting cybersecurity strategies
DataWorks 2018: How Big Data and AI Saved the DayInterset
In this presentation titled "How Big Data and AI Saved the Day: Critical IP Almost Walked Out the Door," Interset Field Data Scientist Roy Wilds discussed real-world examples of how businesses can expand their threat analysis using security analytics powered by artificial intelligence in a big data environment. This was presented at DataWorks Summit 2018.
[CONFidence 2016] Gaweł Mikołajczyk - Making sense out of the Security Operat...PROIDEA
As the SOC Manager with Cisco Active Threat Analytics (ATA), Gawel is responsible for building, growing and operating Cisco Managed Security Services SOC in Krakow, Poland and Tokyo, Japan.
Before that, Gawel spent half a decade in various Architect and Consulting Security roles at Cisco. He holds numerous industry certificates, including CCIE #24987, CISSP-ISSAP, CISA, C|EH and SFCE. Gawel is a frequent speaker at IT events, such as Cisco Live! Europe/Australia, PLNOG, EuroNOG, Security B-Sides, CONFidence, Cisco Connect, Cisco Expo and Cisco Forum.
Before Gawel has joined Cisco, he was a UNIX System Administrator and a Systems Engineer with one of the leading system integrators in Poland. He was also a Cisco Networking Academy Instructor. Gawel graduated from Warsaw University of Technology with degree in Telecommunications.
SplunkLive! Stockholm 2015 breakout - Analytics based securitySplunk
Splunk products provide a flexible and fast security intelligence platform that makes security personnel and processes more efficient by providing quick and flexible access to all of the data and information needed to detect, investigate and remediate threats. This presentation will discuss best practices for building out or enhancing an analytics based security strategy and how Splunk products can make people, process, and technology work better together. Presented at SplunkLive! Stockholm October 2015 for more information please visit http://live.splunk.com/stockholm
Protecting your data against cyber attacks in big data environmentsat MicroFocus Italy ❖✔
This article discusses the inherent risk of big data environments such as Hadoop and how
companies can take steps to protect the data in such an environment from current attacks.
It describes the best practices in applying current technology to secure sensitive data
without removing analytical capabilities.
Protecting your data against cyber attacks in big data environmentsat MicroFocus Italy ❖✔
This article discusses the inherent risk of big data environments such as Hadoop and how
companies can take steps to protect the data in such an environment from current attacks.
It describes the best practices in applying current technology to secure sensitive data
without removing analytical capabilities.
A Community Approach to Fighting Cyber ThreatsCloudera, Inc.
3 Things to Learn About:
*Infinitely scale data storage, access, and machine learning
*Provide community defined open data models for complete enterprise visibility
*Open up application flexibility while building on a future proofed architecture
Security Challenges and Use Cases in the Modern Application Build-and-Deploy ...Amazon Web Services
Modern application build-and-deploy workflows are creating new challenges for traditional security models. Traditional workflows need to be recast in new datasets, and new workflows need to be added to cover the expanding threat surface area. In this session, we explore the security challenges created by modern application build-and-deploy pipelines. We also discuss basic considerations for security defense, example use cases, and a customer case study to illustrate the concepts. This session is brought to you by AWS partner, Sumo Logic.
An overview on the application of data science methods and data analytics tools to complement cyber risk quantification, cyber insurance valuation, and cyber risk assessment.
Similar to Big Data Analytics to Enhance Security (20)
Data Science fuels Creativity
DAAT Day - Digital Advertisitng Association Thailand
Komes Chandavimol, Data Science Thailand
Data Scientists Data Science Lab, Thailand
Marketing analytics
PREDICTIVE ANALYTICS AND DATA SCIENCECONFERENCE (MAY 27-28)
Surat Teerakapibal, Ph.D.
Lecturer, Department of Marketing
Program Director, Doctor of Philosophy Program in Business Administration
Single Nucleotide Polymorphism Analysis
Predictive Analytics and Data Science Conference May 27-28
Asst. Prof. Vitara Pungpapong, Ph.D.
Department of Statistics
Faculty of Commerce and Accountancy
Chulalongkorn University
Explore our comprehensive data analysis project presentation on predicting product ad campaign performance. Learn how data-driven insights can optimize your marketing strategies and enhance campaign effectiveness. Perfect for professionals and students looking to understand the power of data analysis in advertising. for more details visit: https://bostoninstituteofanalytics.org/data-science-and-artificial-intelligence/
Show drafts
volume_up
Empowering the Data Analytics Ecosystem: A Laser Focus on Value
The data analytics ecosystem thrives when every component functions at its peak, unlocking the true potential of data. Here's a laser focus on key areas for an empowered ecosystem:
1. Democratize Access, Not Data:
Granular Access Controls: Provide users with self-service tools tailored to their specific needs, preventing data overload and misuse.
Data Catalogs: Implement robust data catalogs for easy discovery and understanding of available data sources.
2. Foster Collaboration with Clear Roles:
Data Mesh Architecture: Break down data silos by creating a distributed data ownership model with clear ownership and responsibilities.
Collaborative Workspaces: Utilize interactive platforms where data scientists, analysts, and domain experts can work seamlessly together.
3. Leverage Advanced Analytics Strategically:
AI-powered Automation: Automate repetitive tasks like data cleaning and feature engineering, freeing up data talent for higher-level analysis.
Right-Tool Selection: Strategically choose the most effective advanced analytics techniques (e.g., AI, ML) based on specific business problems.
4. Prioritize Data Quality with Automation:
Automated Data Validation: Implement automated data quality checks to identify and rectify errors at the source, minimizing downstream issues.
Data Lineage Tracking: Track the flow of data throughout the ecosystem, ensuring transparency and facilitating root cause analysis for errors.
5. Cultivate a Data-Driven Mindset:
Metrics-Driven Performance Management: Align KPIs and performance metrics with data-driven insights to ensure actionable decision making.
Data Storytelling Workshops: Equip stakeholders with the skills to translate complex data findings into compelling narratives that drive action.
Benefits of a Precise Ecosystem:
Sharpened Focus: Precise access and clear roles ensure everyone works with the most relevant data, maximizing efficiency.
Actionable Insights: Strategic analytics and automated quality checks lead to more reliable and actionable data insights.
Continuous Improvement: Data-driven performance management fosters a culture of learning and continuous improvement.
Sustainable Growth: Empowered by data, organizations can make informed decisions to drive sustainable growth and innovation.
By focusing on these precise actions, organizations can create an empowered data analytics ecosystem that delivers real value by driving data-driven decisions and maximizing the return on their data investment.
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...pchutichetpong
M Capital Group (“MCG”) expects to see demand and the changing evolution of supply, facilitated through institutional investment rotation out of offices and into work from home (“WFH”), while the ever-expanding need for data storage as global internet usage expands, with experts predicting 5.3 billion users by 2023. These market factors will be underpinned by technological changes, such as progressing cloud services and edge sites, allowing the industry to see strong expected annual growth of 13% over the next 4 years.
Whilst competitive headwinds remain, represented through the recent second bankruptcy filing of Sungard, which blames “COVID-19 and other macroeconomic trends including delayed customer spending decisions, insourcing and reductions in IT spending, energy inflation and reduction in demand for certain services”, the industry has seen key adjustments, where MCG believes that engineering cost management and technological innovation will be paramount to success.
MCG reports that the more favorable market conditions expected over the next few years, helped by the winding down of pandemic restrictions and a hybrid working environment will be driving market momentum forward. The continuous injection of capital by alternative investment firms, as well as the growing infrastructural investment from cloud service providers and social media companies, whose revenues are expected to grow over 3.6x larger by value in 2026, will likely help propel center provision and innovation. These factors paint a promising picture for the industry players that offset rising input costs and adapt to new technologies.
According to M Capital Group: “Specifically, the long-term cost-saving opportunities available from the rise of remote managing will likely aid value growth for the industry. Through margin optimization and further availability of capital for reinvestment, strong players will maintain their competitive foothold, while weaker players exit the market to balance supply and demand.”
Techniques to optimize the pagerank algorithm usually fall in two categories. One is to try reducing the work per iteration, and the other is to try reducing the number of iterations. These goals are often at odds with one another. Skipping computation on vertices which have already converged has the potential to save iteration time. Skipping in-identical vertices, with the same in-links, helps reduce duplicate computations and thus could help reduce iteration time. Road networks often have chains which can be short-circuited before pagerank computation to improve performance. Final ranks of chain nodes can be easily calculated. This could reduce both the iteration time, and the number of iterations. If a graph has no dangling nodes, pagerank of each strongly connected component can be computed in topological order. This could help reduce the iteration time, no. of iterations, and also enable multi-iteration concurrency in pagerank computation. The combination of all of the above methods is the STICD algorithm. [sticd] For dynamic graphs, unchanged components whose ranks are unaffected can be skipped altogether.
1. Copyright Stelligence Co.,Ltd. 2016 All rights reserved
Big Data Analytics to
Enhance Security
Anapat Pipatkitibodee
Technical Manager
anapat.p@Stelligence.com
2. Copyright Stelligence Co.,Ltd. 2016 All rights reserved
Agenda
• Big Data Analytics
• Security Trends
• Example Security Attacks
• Integrated Security Analytics with Open Source
• How to Apply ?
10. Copyright Stelligence Co.,Ltd. 2016 All rights reserved
Big Data Analytics
• The process of examining large data sets
containing a variety of data types i.e., big
data.
• Big Data analytics enables organizations to
analyze a mix of structured, semi-structured,
and unstructured data in search of valuable
information and insights.
12. Copyright Stelligence Co.,Ltd. 2016 All rights reserved
Data Analytics for Intrusion Detection
• 1st generation: Intrusion detection
systems
• 2nd generation: Security information
and event management (SIEM)
13. Copyright Stelligence Co.,Ltd. 2016 All rights reserved
Limitation of Traditional SIEMs
Storing and retaining a large quantity of data was not economically feasible.
Normalization & datastore schema reduces data
Traditional tools did not leverage Big Data technologies.
Closed platform with limited customization & integration options
15. Copyright Stelligence Co.,Ltd. 2016 All rights reserved
Security Trend from Y2015 to Y2016
• Threats are hard to investigate
Fireeye M-Trends Report 2016
16. Copyright Stelligence Co.,Ltd. 2016 All rights reserved
All Data is Security Relevant = Big Data
Servers
Storage
DesktopsEmail Web
Transaction
Records
Network
Flows
DHCP/ DNS
Hypervisor
Custom
Apps
Physical
Access
Badges
Threat
Intelligence
Mobile
CMDB
Intrusion
Detection
Firewall
Data Loss
Prevention
Anti-
Malware
Vulnerability
Scans
Traditional
Authentication
17. Copyright Stelligence Co.,Ltd. 2016 All rights reserved
Data Analytics for Intrusion Detection
• 1st generation: Intrusion detection
systems
• 2nd generation: Security information
and event management (SIEM)
• 3rd generation: Big Data analytics in
security (Next generation SIEM)
19. Copyright Stelligence Co.,Ltd. 2016 All rights reserved
Advanced Persistent Threats
• Advanced
– The attack can cope with traditional security solutions
– In many cases is based on Zero-day vulnerabilities
• Persistent
– Attack has a specific goal
– Remain on the system as long as the attack goal is not met.
• Threat
– Collect and steal information-Confidentiality.
– Make the victim's system unavailable-Availability.
– Modify the victim's system data-Integrity.
20. Copyright Stelligence Co.,Ltd. 2016 All rights reserved
Example of Advanced Threat Activities
HTTP (web) session to
command & control
server
Remote control,
Steal data,
Persist in company,
Rent as botnet
WEB
Conduct
Business
Create additional
environment
Gain Access
to systemTransaction
.pdf
.pdf executes & unpacks malware
overwriting and running “allowed” programs
Svchost.exeCalc.exe
Attacker hacks website
Steals .pdf files
Web
Portal.pdf
Attacker creates
malware, embed in .pdf,
Emails
to the target
MAIL
Read email, open attachment
Threat intelligence
Auth - User Roles
Host
Activity/Security
Network
Activity/Security
21. Copyright Stelligence Co.,Ltd. 2016 All rights reserved
Link Events Together
Threat intelligence
Auth - User Roles,
Corp Context
Host
Activity/Security
Network
Activity/Security
WEB
Conduct
Business
Create additional
environment
Gain Access
to systemTransaction
MAIL
.pdf Svchost.exeCalc.exe
Events that
contain link to file
Proxy log
C2 communication
to blacklist
How was
process started?
What created the
program/process?
Process making
C2 traffic
Web
Portal.pdf
22. Copyright Stelligence Co.,Ltd. 2016 All rights reserved
Correlated Security Log
Aug 08 06:09:13 acmesep01.acmetech.com Aug 09 06:17:24 SymantecServer acmesep01: Virus found,Computer name:
ACME-002,Source: Real Time Scan,Risk name: Hackertool.rootkit,Occurrences: 1,C:/Documents and Settings/smithe/Local
Settings/Temp/evil.tmp,"""",Actual action: Quarantined,Requested action: Cleaned, time: 2009-01-23 03:19:12,Inserted:
2009-01-23 03:20:12,End: 2009-01-23 03:19:12,Domain: Default,Group: My CompanyACME Remote,Server:
acmesep01,User: smithe,Source computer: ,Source IP: 10.11.36.20
Aug 08 08:26:54 snort.acmetech.com {TCP} 10.11.36.20:5072 -> 10.11.36.26:443 itsec snort[18774]: [1:100000:3]
[Classification: Potential Corporate Privacy Violation] Credit Card Number Detected in Clear Text [Priority: 2]:
20130806041221.000000Caption=ACME-2975EBAdministrator Description=Built-in account for administering the
computer/domainDomain=ACME-2975EB InstallDate=NULLLocalAccount = IP: 10.11.36.20 TrueName=Administrator
SID =S-1-5-21-1715567821-926492609-725345543 500SIDType=1 Status=Degradedwmi_ type=UserAccounts
Sources
All three occurring within a 24-hour period
Source IP
Data Loss
Default Admin Account
Malware Found
Time Range
Intrusion
Detection
Endpoint
Security
Windows
Authentication
Source IP
Source IP
23. Copyright Stelligence Co.,Ltd. 2016 All rights reserved
Incident Analysis & Investigation
Search historically - back in time Watch for new evidence
Related
evidence
from other
security
devices
30. Copyright Stelligence Co.,Ltd. 2016 All rights reserved
Determining Data That Can Be Collected
Threat
intelligence
Auth - User
Roles
Service
Host
Network
Network Security Through Data Analysisby Michael S
CollinsPublished by O'Reilly Media, Inc., 2014
• Third-party Threat Intel
• Open source blacklist
• Internal threat intelligence
• Firewall
• IDS / IPS
• Web Proxy
• Vulnerability scanners
• VPNs
• Netflow
• TCP Collector
• OS logs
• Patching
• File Integrity
• Endpoint (AV/IPS/FW)
• Malware detection
• Logins, Logouts log
• Active Directory
• LDAP
• AAA, SSO
• Application logs
• Audit log
• Service / Process
31. Copyright Stelligence Co.,Ltd. 2016 All rights reserved
Option 1 : Replace All Solution
• Data sent to new Big Data
Analytic Platform
• Big Data Analytic Platform
– Static Visualizations /
Reports
– Threat detection, alerts,
workflow, compliance
– Incident
investigations/forensics
– Non-security use cases
Big Data Analytic Platform
Raw data
Alerts
Static
Visualizations
Forensics / Search
Interface
32. Copyright Stelligence Co.,Ltd. 2016 All rights reserved
Option 2 : Big Data to Traditional SIEM
• Data sent to both system
• Big Data Analytic Platform
– Incident
investigations/forensics
– Non-security use cases
• Traditional SIEM
– Static Visualizations /
Reports
– Threat detection, alerts,
workflow, compliance
Big Data Analytic
Platform
Raw data
Forensics / Search
Interface
SIEM
Alerts
Static
Visualizations
Connectors
33. Copyright Stelligence Co.,Ltd. 2016 All rights reserved
Factors for evaluating
Big Data Security Analytics Platforms
Factors for Evaluating Open Source
• Scalable data ingestion HDFS
• Unified data management platform Cassandra / Accumulo
• Support for multiple data types Ready to Customized
• Real time Spark / Strom
• Security analytic tools No
• Compliance reporting No
• Easy to deploy and manage Manage many 3rd Party
• Flexible search, report and create new
correlation rule
No