I gave the keynote at VizSec 2012. I used the opportunity to take a step back to see where security visualization is at and propose a challenge for how some of the problems we should be focusing on going forward.
Video recording is here: http://youtu.be/AEAs7IzTHMo
Creating Your Own Threat Intel Through Hunting & VisualizationRaffael Marty
The security industry is talking a lot about threat intelligence; external information that a company can leverage to understand where potential threats are knocking on the door and might have already perpetrated the network boundaries. Conversations with many CERTs have shown that we have to stop relying on knowledge about how attacks have been conducted in the past and start 'hunting' for signs of compromises and anomalies in our own environments.
In this presentation we explore how the decade old field of security visualization has emerged. We show how we have applied advanced analytics and visualization to create our own threat intelligence and investigated lateral movement in a Fortune 50 company.
Visualization. Data science. No machine learning. But pretty pictures.
Here is a blog post I wrote a bit ago about the general theme of internal threat intelligence:
http://www.darkreading.com/analytics/creating-your-own-threat-intel-through-hunting-and-visualization/a/d-id/1321225?
Visual Analytics and Security IntelligenceRaffael Marty
Big data and security intelligence are the two hot security topics in 2012. We are collecting more and more information from both the infrastructure, but increasingly also directly from our applications. Some companies are moving away from traditional log management and SIEM tools and are deploying big data products. But what is this big data craze all about? Why is it that we have more and more data to look at? And is big data the right approach or what is missing?
The presentation takes the audience on a journey through big data tools and show that analytical tools are needed to make use of these infrastructures. How can visualization be used to fill in the gap in analytics to move into gaining situational awareness and building up security intelligence.
Supercharging Visualization with Data MiningRaffael Marty
We are exploring how data mining can help visualization. I am giving examples of security visualizations and am discussing how data mining best augments visualization efforts.
AfterGlow is a script that assists with the visualization of log data. It reads CSV files and converts them into a Graph description. Check out http://afterglow.sf.net for more information also.
This short presentation gives an overview of AfterGlow and outlines the features and capabilities of the tool. It discusses some of the harder to understand features by showing some configuration examples that can be used as a starting point for some more sophisticated setups.
AftterGlow is one the most downloaded security visualization tools with over 17,000 downloads.
Creating Your Own Threat Intel Through Hunting & VisualizationRaffael Marty
The security industry is talking a lot about threat intelligence; external information that a company can leverage to understand where potential threats are knocking on the door and might have already perpetrated the network boundaries. Conversations with many CERTs have shown that we have to stop relying on knowledge about how attacks have been conducted in the past and start 'hunting' for signs of compromises and anomalies in our own environments.
In this presentation we explore how the decade old field of security visualization has emerged. We show how we have applied advanced analytics and visualization to create our own threat intelligence and investigated lateral movement in a Fortune 50 company.
Visualization. Data science. No machine learning. But pretty pictures.
Here is a blog post I wrote a bit ago about the general theme of internal threat intelligence:
http://www.darkreading.com/analytics/creating-your-own-threat-intel-through-hunting-and-visualization/a/d-id/1321225?
Visual Analytics and Security IntelligenceRaffael Marty
Big data and security intelligence are the two hot security topics in 2012. We are collecting more and more information from both the infrastructure, but increasingly also directly from our applications. Some companies are moving away from traditional log management and SIEM tools and are deploying big data products. But what is this big data craze all about? Why is it that we have more and more data to look at? And is big data the right approach or what is missing?
The presentation takes the audience on a journey through big data tools and show that analytical tools are needed to make use of these infrastructures. How can visualization be used to fill in the gap in analytics to move into gaining situational awareness and building up security intelligence.
Supercharging Visualization with Data MiningRaffael Marty
We are exploring how data mining can help visualization. I am giving examples of security visualizations and am discussing how data mining best augments visualization efforts.
AfterGlow is a script that assists with the visualization of log data. It reads CSV files and converts them into a Graph description. Check out http://afterglow.sf.net for more information also.
This short presentation gives an overview of AfterGlow and outlines the features and capabilities of the tool. It discusses some of the harder to understand features by showing some configuration examples that can be used as a starting point for some more sophisticated setups.
AftterGlow is one the most downloaded security visualization tools with over 17,000 downloads.
"Thousands of organizations around the world, including AT&T, Sears, Ford, Verizon, The Guardian, Elsevier, Cisco, Macy’s and more have found their solution: Lucene/Solr open source, the world’s most popular search technology. Our new white paper “A Manager’s Guide to Real World Open Source Search Applications” provides numerous case studies across various industries and business models to show how real-world businesses have turned Lucene/Solr open source search into competitive advantage.http://www.lucidimagination.com/files/file/whitepaper/LIWP_LuceneSolrRealWorldSearch.pdf
"
The Kenya Ushahidi Evaluation Project was 9-month Ushahidi evaluation project in partnership with the Harvard Humanitarian Initiative supported by the Knight Foundation. Jennifer Chan and Melissa Tully conducted research which lead to the creation of case studies and toolboxes. (2011) This is Toolbox #3: Real-Time Evaluation.
It takes a network to defend a network. Discover how to harness the power of the Booz Allen Cyber Solutions Network. http://www.boozallen.com/consulting/prepare-for-whats-next/cyber/cyber-solutions-network
Bitocast - A hybrid BitTorrent and IP Multicast content distribution solutionTiago Andrade e Silva
In recent years we have observed an increased use of the Internet as a means for transmitting
large content. There have been several technology attempts to attack this problem, including
costly distribution networks and, more recently, peer to peer (P2P) protocols. Amongst
these P2P protocols, BitTorrent has proven itself as an effective means for transmitting large
content items and today enjoys great popularity.
Numerous researchers have analyzed BitTorrent and proposed concepts and models to enhance
its reliability, efficiency and fairness. Further, there are proposals to extend BitTorrent
to support on-demand multimedia streaming. In this Dissertation we present Bitocast, a
content distribution system that combines IP Multicast and BitTorrent protocols in order to
achieve a more efficient usage of an Internet Service Provider’s network and reduce download
time when serving large sets of contents to large audiences.
Workforce Needs of the California Solar IndustryJoel West
An April 19, 2011 webinar hosted by SolarTech, featuring SJSU faculty Meg Virick and Joel West; reported results from the first employer survey of the SJSU Solar Workforce Project
How to protect, detect, and respond to your threats.
This is an MSP centric talk exploring how to detect, protect, and respond to cyber security threats. We first walk through the cyber defense matrix, explore what security intelligence needs to be and emphasize the concepts with two case studies of BlackCat.
Extended Detection and Response (XDR)An Overhyped Product Category With Ulti...Raffael Marty
Extended Detection and Response, or XDR for short, is one of the acronyms that are increasingly used by cybersecurity vendors to explain their approach to solving the cyber security problem. We have been spending trillions of dollars on approaches to secure our systems and data, with what success? Cybersecurity is still one of the biggest and most challenging areas that companies, small and large, are dealing with. XDR is another approach driven by security vendors to solve this problem. The challenge is that every vendor defines XDR slightly differently and makes it fit their own “challenge du jour” for marketing and selling their products.
In this presentation we will demystify the XDR acronym and put a working model behind it. Together, we will explore why XDR is a fabulous concept, but also discover that it’s nothing revolutionarily new. With an MSP lens, we will explore what the XDR benefits are for small and medium businesses and what it means to the security strategy of both MSPs and their clients. The audience will leave with a clear understanding of what XDR is, how the technology matters to them, and how XDR will ultimately help them secure their customers and enable trusted commerce.
More Related Content
Similar to Security Visualization - Let's Take A Step Back
"Thousands of organizations around the world, including AT&T, Sears, Ford, Verizon, The Guardian, Elsevier, Cisco, Macy’s and more have found their solution: Lucene/Solr open source, the world’s most popular search technology. Our new white paper “A Manager’s Guide to Real World Open Source Search Applications” provides numerous case studies across various industries and business models to show how real-world businesses have turned Lucene/Solr open source search into competitive advantage.http://www.lucidimagination.com/files/file/whitepaper/LIWP_LuceneSolrRealWorldSearch.pdf
"
The Kenya Ushahidi Evaluation Project was 9-month Ushahidi evaluation project in partnership with the Harvard Humanitarian Initiative supported by the Knight Foundation. Jennifer Chan and Melissa Tully conducted research which lead to the creation of case studies and toolboxes. (2011) This is Toolbox #3: Real-Time Evaluation.
It takes a network to defend a network. Discover how to harness the power of the Booz Allen Cyber Solutions Network. http://www.boozallen.com/consulting/prepare-for-whats-next/cyber/cyber-solutions-network
Bitocast - A hybrid BitTorrent and IP Multicast content distribution solutionTiago Andrade e Silva
In recent years we have observed an increased use of the Internet as a means for transmitting
large content. There have been several technology attempts to attack this problem, including
costly distribution networks and, more recently, peer to peer (P2P) protocols. Amongst
these P2P protocols, BitTorrent has proven itself as an effective means for transmitting large
content items and today enjoys great popularity.
Numerous researchers have analyzed BitTorrent and proposed concepts and models to enhance
its reliability, efficiency and fairness. Further, there are proposals to extend BitTorrent
to support on-demand multimedia streaming. In this Dissertation we present Bitocast, a
content distribution system that combines IP Multicast and BitTorrent protocols in order to
achieve a more efficient usage of an Internet Service Provider’s network and reduce download
time when serving large sets of contents to large audiences.
Workforce Needs of the California Solar IndustryJoel West
An April 19, 2011 webinar hosted by SolarTech, featuring SJSU faculty Meg Virick and Joel West; reported results from the first employer survey of the SJSU Solar Workforce Project
How to protect, detect, and respond to your threats.
This is an MSP centric talk exploring how to detect, protect, and respond to cyber security threats. We first walk through the cyber defense matrix, explore what security intelligence needs to be and emphasize the concepts with two case studies of BlackCat.
Extended Detection and Response (XDR)An Overhyped Product Category With Ulti...Raffael Marty
Extended Detection and Response, or XDR for short, is one of the acronyms that are increasingly used by cybersecurity vendors to explain their approach to solving the cyber security problem. We have been spending trillions of dollars on approaches to secure our systems and data, with what success? Cybersecurity is still one of the biggest and most challenging areas that companies, small and large, are dealing with. XDR is another approach driven by security vendors to solve this problem. The challenge is that every vendor defines XDR slightly differently and makes it fit their own “challenge du jour” for marketing and selling their products.
In this presentation we will demystify the XDR acronym and put a working model behind it. Together, we will explore why XDR is a fabulous concept, but also discover that it’s nothing revolutionarily new. With an MSP lens, we will explore what the XDR benefits are for small and medium businesses and what it means to the security strategy of both MSPs and their clients. The audience will leave with a clear understanding of what XDR is, how the technology matters to them, and how XDR will ultimately help them secure their customers and enable trusted commerce.
Blog Post: http://raffy.ch/blog. - Video: https://youtu.be/nk5uz0VZrxM
In this video we talk about the world of security data or log data. In the first section, we dive into a bit of a history lesson around log management, SIEM, and big data in security. We then shift to the present to discuss some of the challenges that we face today with managing all of that data and also discuss some of the trends in the security analytics space. In the third section, we focus on the future. What does tomorrow hold in the SIEM / security data space? What are some of the key features we will see and how does this matter to the user of these approaches.
Cyber Security Beyond 2020 – Will We Learn From Our Mistakes?Raffael Marty
The cyber security industry has spent trillions of dollars to keep external attackers at bay. To what effect? We still don't see an end to the cat and mouse game between attackers and the security industry; zero day attacks, new vulnerabilities, ever increasingly sophisticated attacks, etc. We need a paradigm shift in security. A shift away from traditional threat intelligence and indicators of compromise (IOCs). We need to look at understanding behaviors. Those of devices and those of humans.
What are the security approaches and trends that will make an actual difference in protecting our critical data and intellectual property; not just from external attackers, but also from malicious insiders? We will explore topics from the 'all solving' artificial intelligence to risk-based security. We will look at what is happening within the security industry itself, where startups are putting placing their bets, and how human factors will play an increasingly important role in security, along with all of the potential challenges that will create.
Artificial Intelligence – Time Bomb or The Promised Land?Raffael Marty
Companies have AI projects. Security products use AI to keep attackers out and insiders at bay. But what is this "AI" that everyone talks about? In this talk we will explore what artificial intelligence in cyber security is, where the limitations and dangers are, and in what areas we should invest more in AI. We will talk about some of the recent failures of AI in security and invite a conversation about how we verify artificially intelligent systems to understand how much trust we can place in them.
Alongside the AI conversation, we will discover that we need to make a shift in our traditional approach to cyber security. We need to augment our reactive approaches of studying adversary behaviors to understanding behaviors of users and machines to inform a risk-driven approach to security that prevents even zero day attacks.
In this presentation I explore the topic of artificial intelligence in cyber security. What is AI and how do we get to real intelligence in a cyber context. I outline some of the dangers of the way we are using algorithms (AI, ML) today and what that leads to. We then explore how we can add real intelligence through export knowledge to the problem of finding attackers and anomalies in our applications and networks.
Presented at AI 4 Cyber in NYC on April 30, 2019
AI & ML in Cyber Security - Why Algorithms are DangerousRaffael Marty
Link to the video of the presentation: https://www.youtube.com/watch?v=WG1k-Xh1TqM
Every single security company is talking in some way or another about how they are applying machine learning. Companies go out of their way to make sure they mention machine learning and not statistics when they explain how they work. Recently, that's not enough anymore either. As a security company you have to claim artificial intelligence to be even part of the conversation.
Guess what. It's all baloney. We have entered a state in cyber security that is, in fact, dangerous. We are blindly relying on algorithms to do the right thing. We are letting deep learning algorithms detect anomalies in our data without having a clue what that algorithm just did. In academia, they call this the lack of explainability and verifiability. But rather than building systems with actual security knowledge, companies are using algorithms that nobody understands and in turn discover wrong insights.
In this talk, I will show the limitations of machine learning, outline the issues of explainability, and show where deep learning should never be applied. I will show examples of how the blind application of algorithms (including deep learning) actually leads to wrong results. Algorithms are dangerous. We need to revert back to experts and invest in systems that learn from, and absorb the knowledge, of experts.
AI & ML in Cyber Security - Why Algorithms Are DangerousRaffael Marty
Every single security company is talking in some way or another about how they are applying machine learning. Companies go out of their way to make sure they mention machine learning and not statistics when they explain how they work. Recently, that's not enough anymore either. As a security company you have to claim artificial intelligence to be even part of the conversation.
Guess what. It's all baloney. We have entered a state in cyber security that is, in fact, dangerous. We are blindly relying on algorithms to do the right thing. We are letting deep learning algorithms detect anomalies in our data without having a clue what that algorithm just did. In academia, they call this the lack of explainability and verifiability. But rather than building systems with actual security knowledge, companies are using algorithms that nobody understands and in turn discover wrong insights.
In this talk I will show the limitations of machine learning, outline the issues of explainability, and show where deep learning should never be applied. I will show examples of how the blind application of algorithms (including deep learning) actually leads to wrong results. Algorithms are dangerous. We need to revert back to experts and invest in systems that learn from, and absorb the knowledge, of experts.
Delivering Security Insights with Data Analytics and VisualizationRaffael Marty
It's an interesting exercise to look back to the year 2000 to see how we approached cyber security. We just started to realize that data might be a useful currency, but for the most part, security pursued preventative avenues, such as firewalls, intrusion prevention systems, and anti-virus. With the advent of log management and security incident and event management (SIEM) solutions we started to gather gigabytes of sensor data and correlate data from different sensors to improve on their weaknesses and accelerate their strengths. But fundamentally, such solutions didn't scale that well and struggled to deliver real security insight.
Today, cybersecurity wouldn't work anymore without large scale data analytics and machine learning approaches, especially in the realm of malware classification and threat intelligence. Nonetheless, we are still just scratching the surface and learning where the real challenges are in data analytics for security.
This talk will go on a journey of big data in cybersecurity, exploring where big data has been and where it must go to make a true difference. We will look at the potential of data mining, machine learning, and artificial intelligence, as well as the boundaries of these approaches. We will also look at both the shortcomings and potential of data visualization and the human computer interface. It is critical that today's systems take into account the human expert and, most importantly, provide the right data.
AI & ML in Cyber Security - Welcome Back to 1999 - Security Hasn't ChangedRaffael Marty
We are writing the year 2017. Cyber security has been a discipline for many years and thousands of security companies are offering solutions to deter and block malicious actors in order to keep our businesses operating and our data confidential. But fundamentally, cyber security has not changed during the last two decades. We are still running Snort and Bro. Firewalls are fundamentally still the same. People get hacked for their poor passwords and we collect logs that we don't know what to do with. In this talk I will paint a slightly provocative and dark picture of security. Fundamentally, nothing has really changed. We'll have a look at machine learning and artificial intelligence and see how those techniques are used today. Do they have the potential to change anything? How will the future look with those technologies? I will show some practical examples of machine learning and motivate that simpler approaches generally win. Maybe we find some hope in visualization? Or maybe Augmented reality? We still have a ways to go.
Ensuring security of a company’s data and infrastructure has largely become a data analytics challenge. It is about finding and understanding patterns and behaviors that are indicative of malicious activities or deviations from the norm. Data, Analytics, and Visualization are used to gain insights and discover those malicious activities. These three components play off of each other, but also have their inherent challenges. A few examples will be given to explore and illustrate some of these challenges,
Creating Your Own Threat Intel Through Hunting & VisualizationRaffael Marty
The security industry is talking a lot about threat intelligence; external information that a company can leverage to understand where potential threats are knocking on the door and might have already perpetrated the network boundaries. Conversations with many CERTs have shown that we have to stop relying on knowledge about how attacks have been conducted in the past and start ‘hunting’ for signs of compromises and anomalies in our own environments.
In this presentation we explore how the decade old field of security visualization has emerged. We show how we have applied advanced analytics and visualization to create our own threat intelligence and investigated lateral movement in a Fortune 50 company.
Visualization. Data science. No machine learning. But pretty pictures.What is internal threat intelligence? Check out http://www.darkreading.com/analytics/creating-your-own-threat-intel-through-hunting-and-visualization/a/d-id/1321225
The extent and impact of recent security breaches is showing that current security approaches are just not working. But what can we do to protect our business? We have been advocating monitoring for a long time as a way to detect subtle, advanced attacks that are still making it through our defenses. However, products have failed to deliver on this promise.
Current solutions don't scale in both data volume and analytical insights. In this presentation we will explore what security monitoring is. Specifically, we are going to explore the question of how to visualize a billion log records. A number of security visualization examples will illustrate some of the challenges with big data visualization. They will also help illustrate how data mining and user experience design help us get a handle on the security visualization challenges - enabling us to gain deep insight for a number of security use-cases.
An overview of some methods and principles for big data visualization. The presentation quickly hits on the topic of dashboards and some cyber security uses. The topic of a big data lake is also briefly discussed in the context of a cyber security big data setup.
The Heatmap - Why is Security Visualization so Hard?Raffael Marty
The extent and impact of recent security breaches is showing that current approaches are just not working. But what can we do to protect our business? We have been advocating monitoring for a long time as a way to detect subtle, advanced attacks. However, products have failed to deliver on this promise. Current solutions don't scale in both data volume and analytical insights. In this presentation we will explore why it is so hard to come up with a security monitoring (or shall we call it security intelligence) approach that helps find sophisticated attackers in all the data collected. We are going to explore the question of how to visualize a billion events. We are going to look at a number of security visualization examples to illustrate the problem and some possible solutions. These examples will also help illustrate how data mining and user experience design help us get a handle of the security visualization challenges - enabling us to gain deep insight for a number of security use-cases.
Workshop: Big Data Visualization for SecurityRaffael Marty
Big Data is the latest hype in the security industry. We will have a closer look at what big data is comprised of: Hadoop, Spark, ElasticSearch, Hive, MongoDB, etc. We will learn how to best manage security data in a small Hadoop cluster for different types of use-cases. Doing so, we will encounter a number of big-data open source tools, such as LogStash and Moloch that help with managing log files and packet captures.
As a second topic we will look at visualization and how we can leverage visualization to learn more about our data. In the hands-on part, we will use some of the big data tools, as well as a number of visualization tools to actively investigate a sample data set.
Vision is a human’s dominant sense. It is the communication channel with the highest bandwidth into the human brain. Security tools and applications need to make better use of information visualization to enhance human computer interactions and information exchange.
In this talk we will explore a few basic principles of information visualization to see how they apply to cyber security. We will explore both visualization as a data presentation, as well as a data discovery tool. We will address questions like: What makes for effective visualizations? What are some core principles to follow when designing a dashboard? How do you go about visually exploring a terabyte of data? And what role do big data and data mining play in security visualization?
The presentation is filled with visualizations of security data to help translate the theoretical concepts into tangible applications.
The Heatmap - Why is Security Visualization so Hard?Raffael Marty
This presentation explores why it is so hard to come up with a security monitoring (or shall we call it security intelligence) approach that helps find sophisticated attackers in all the data collected. It explores the question of how to visualize a billion events. To do so, the presentation dives deeply into heatmaps - matrices - as an example of a simple type of visualization. While these heatmaps are very simple, they are incredibly versatile and help us think about the problem of security visualization. They help illustrate how data mining and user experience design help get a handle of the security visualization challenges - enabling us to gain deep insight for a number of security use-cases.
DAVIX - Data Analysis and Visualization LinuxRaffael Marty
DAVIX, a live CD for data analysis and visualization, brings the most important free tools for data processing and visualization to your desk. There is no hassle with installing an operating system or struggle to build the necessary tools to get started with visualization. You can completely dedicate your time to data analysis.
1. Security Visualization
Let’s Take A Step Back
VizSec - October 2012
Raffael Marty
http://youtu.be/maCfIMTSJgs
pixlcloud | turning data into actionable insights copyright (c) 2012
2. Doushuai's Three Barriers
‘You make your way through the darkness of abandoned grasses in a
search for meaning. As you do, where is the meaning?'
47th case of'The Gateless Barrier'
a collection of Zen koans
3. Raffael Marty
13 years of log analysis and information visualization
•Security visualization pixlcloud
•Big data and security intelligence
IBM Research
•Security data analyst
•Entrepreneur
•Trainer, speaker, and author
Advisor in
pixlcloud | turning data into actionable insights copyright (c) 2012
4. Security Visualization
• The Most Exciting Field
• SecViz is Hard ...
• ... But Not THAT Hard
• User Centric Design
• Scaling Visualization
• The Role of Data Mining
• What’s Happening Out There?
pixlcloud | turning data into actionable insights copyright (c) 2012
6. The Most Exciting Field
- help make data understandable for the masses - for everyone
- help protect everyone on the planet from the bad guys
pixlcloud | turning data into actionable insights copyright (c) 2012
7. The Most Exciting Field
- an opportunity to solve interesting, big data problems
- a possibility to apply data analytics to real-word problems
- a chance to play with the coolest visualization capabilities
YOU can make a difference
pixlcloud | turning data into actionable insights copyright (c) 2012