Recommended
Recommended
More Related Content
What's hot
What's hot (14)
Similar to Ask me anything: A Conversational Interface to Augment Information Security workers
Similar to Ask me anything: A Conversational Interface to Augment Information Security workers (20)
Recently uploaded
Recently uploaded (20)
Ask me anything: A Conversational Interface to Augment Information Security workers
- 1. ASK ME ANYTHING: A CONVERSATIONAL INTERFACE TO AUGMENT INFORMATION SECURITY WORKERS Symposium On Usable Privacy and Security 2017 Bobby Filar, Rich Seymour, Matthew Park
- 2. BOBBY FILAR Data Scientist Background in (NLP) @filar bfilar@endgame.com RICH SEYMOUR Data Scientist Background in HPC @rseymour rseymour@endgame.com 2 WHO ARE WE? MATTHEW PARK User Experience Designer Background in Big Data @muted_counts mpark@endgame.com
- 3. User-Centric Design 3 Three Stages • Discovery • Understanding our users, confirming/disproving biases, capturing organizational workflows • Concepting • Creating design requirements solutions • Prototyping and User Testing • Feature creation and taking it back into the ‘wild’ for testing
- 4. Our Initial Problems 4 Insufficient Resources • Onboarding & training new hires & Retention • Limited time to review alerts and incidents Lack of easy-to-use automated tools • Difficult for non- programmers to use • Easy for programmers to mess up! Security platforms are just difficult to use! • Forces conformity • Requires level of expertise to extract value
- 5. User-Centric Design Study GOAL: Capture team dynamics and worker roles within security organization to identify challenges common across security teams 5 User Group Team Type Environment Collection Method A Traditional SOC Day-to-day use User interviews B Novice Training Team Mock Scenario Side-by-side monitoring, Retrospective & User interviews C Internal Red vs. Blue Mock Scenario Mirrored Scenario as User Group B D Trad. SOC & Consulting group Day-to-day use User testing
- 6. Findings: Security Worker Roles 6 Tier 1 Analyst Tier 3 Analyst Forensic Hunter Have little to no prior experience (average of 1 year) in the cyber security space. First line of defense in a Security Operations Center. Main responsibility is to initially triage alerts and determine if escalation (to higher tiered) is required. Primarily rely on a platform’s GUI. Intimately understand network and platform architecture. Seen as domain experts on the SOC team and more comfortable working through the command line. Investigates escalated alerts, and determine root causes and extent to remediate problems. Expert in EDR platforms and sophisticated investigation tools Uses command line and scripting languages to bypass UI and collect large data feeds using 3rd party APIs. SOC Manager Skilled security practitioners, not necessarily subject matter experts. Extensive management experience, oversees day-to-day ops. Set schedules, assigns prioritization, generates reports.
- 7. Example: EDR Alert Alert Type: Suspicious Binary Alert Created: Feb 11, 2017 Severity: High Confidence: 73% File Path: C:Tempaaa.exe File Size: 45700 MD5: 5d41402abc4b2a76b9719d911017c592 File Created: Feb 11, 2017 What do you do when there are 100s of these each day? 7
- 8. Example: EDR Alert Lacks context • Is it actually bad? • Is it anywhere else? • Did it talk to the network? Lacks connectivity • Is this alert tied to any others? Pivot on single IOC • Hash • Filename • IP address Alert Type: Suspicious Binary Alert Created: Feb 11, 2017 Severity: High Confidence: 73% File Path: C:Tempaaa.exe File Size: 45700 MD5: 5d41402abc4b2a76b9719d911017c592 File Created: Feb 11, 2017 What do you do when there are 100s of these each day? 8
- 9. Findings: Day in Life of a Security Analyst Data Deluge Lack of Context Repetitive Processes Searching not Analyzing Lack of Expertise Lack of Time 9
- 10. Design Requirements 10 1. Eliminate query syntax via natural language 2. Educate users on platform features 3. Provide context-driven alert triage 4. Recommend next steps 5. Expedite focused collection
- 11. Solution A Bot is an application that assists in the automation of tasks • Mimics human conversation • Natural Language Understanding determines user intent Imagine an assistant that provides ability to: • Ask questions • Execute workflows • Educate users • Recommend next steps Hello, how can I help you? 11
- 12. Natural Language vs Query Language 12 Query Language SELECT * FROM TABLE process_event WHERE process_name == “odinaff.exe”; Natural Language Search process event data for odinaff.exe Reality Is odinaff.exe on any endpoints?
- 13. Confidential and Proprietary 13
- 14. Interaction Types Turn-based Conversation User: Search processes Artemis: Okay. Please provide a hash or filename User: odinaff.exe Artemis: Got it. Which endpoints would you like to target? User: Windows 10 machines. Artemis: Okay. Would you like to launch this search? User: Yes 14
- 15. Interaction Types Goal-oriented Conversation User: Show me process event data for odinaff.exe on all Windows 10 endpoints Artemis: Are you sure? User: Yep! 15
- 16. Interaction Types API-Driven Investigations curl 'api/v1/event_search' -H "Content-Type: application/json" -H 'authorization: <api_key> --data-binary '{"intent":"search_process", "parameters": { "process_name":"odinaff.exe", "filepath": "C:Temp*.exe" } }' 16
- 17. TRADITIONAL INVESTIGATION 1. Narrow scope to limited endpoints 2. Understand adversary TTPs 3. Gather events from limited endpoints 4. Analyze events from for signs of TTPs 5. Discover suspicious activity 6. Decode obfuscated commands 7. Pinpoint powershell activity 8. Expand scope to next set of endpoints 9. Repeat… ENDGAME ARTEMIS “Find powershell activity” Powershell Misuse Investigation Automatically discovers and analyzes malicious activity across your global enterprise in minutes 17
- 18. Looking Forward 18 1. Collaboration 2. Chat Integration 3. Improve via Active Learning
- 19. Thank You Contact: bfilar@endgame.com @filar rseymour@endgame.com @rseymour mpark@endgame.com @muted_counts
Editor's Notes
- Summary Security operations center (SOC) teams are burdened with a deluge of alerts, repetitive processes for data analysis, and lack of skills and tools to stop advanced threats. To address these challenges, it is crucial to empower junior analysts to stop advanced threats before damage and loss occurs. Just as digital assistants like Siri or Alexa have proved their ability to give time back to our day by tackling tasks, a security chatbot can streamline workflows, perform complex tasks, and make recommendations to the SOC analyst. Using a combination of subject matter expertise from SOC analysts and the power of machine learning, chatbots can help teams overcome resource shortcomings by using conversations to offload data collection and guide analysts through recommended courses of action. This process provides an intuitive interface to remediation/investigation workflows and complex storage structures so the analyst can spend less time on collection efforts and more on analysis and response.
- So lets address the security industry as a whole. Generally, there are streams of new exploits types and malicious attacks constantly threaten enterprises environments To combat this, security products have been created to detect and respond and remediate problem areas - in their environment But for these products to be deployed successfully, the right teams had to be employed to monitor and act on these suspicious and malicious events. These teams are know as SOC teams – Security Operations Center – Who physically sit in these centers and remediate a stream of alerts. When approaching our predisposed soc problem (which we will discussed in the next slide) – We wanted to better understand our soc users- In order to create the best tools for own personas, we wanted to understand who they were and their daily workflow habits. We split this approach into three distinct phases: 1) Discovery; 2) Concepting and 3) Prototyping and User Testing
- Bascially… what the bosses said... Forces conformity as opposed to integrating into teams & workflows
- I can get into this more in the Q/A - but during this discovery phase.. confirming our biases/ capturing team dynamics and creating user personas.
- When a SOC analyst starts their shift, they first participate in the shift handover from the analysts on the previous shift. Here they get a briefing on current ongoing investigations or open alerts, ticket numbers associated with those alerts, whois assigned to the investigations, and anything that needs attention. They then monitor a SIEM, an assigned endpoint UI dashboard, email and wait for a security event to occur. Typically that doesn't take very long - with the amount of tools generating alerts in a typical SOC environment (especially those monitoring large networks) - getting alerts is not the problem. Determining which alerts to focus on is the problem. The analyst is typically in reactive mode - where they respond to alerts as they come in, quickly identifying the high priority alerts. Typically, Tier 1 Analysts will have little to no authority to take immediate action on suspected malicious security events and will instead move the alert up the SOC chain. When escalating they will create a case/investigation/incident and assign that case/investigation/incident to the SOC investigator (or Tier 3 analysis). Both the SOC Investigator and the Tier 3 Analyst will take further steps in verifying the anomalous event, and will take the proper response in remediating the alert. While the Tier 3 Analyst will also sift through a SIEM alongside the Tier 1 Analyst, SOC Investigators will often times only work on escalated alerts. At the end of the shift, all levels of analysts needs to prepare a report of the alerts triaged, what was resolved, and what is still open in order to handover the activities to the oncoming shift. Larger reports depicting alert and investigation trends are generated for a SOC Manager on a daily or weekly basis. The SOC Manager will use these reports to focus in on key metadata in the coming weeks, determine the SOC shift schedule and build a custom summary report for the organization's executive level.
- Analysts must battle a tide of incoming alerts Data deluge Morning coffee situation Too much data, too many sources AV EDR Firewall Application logs Threat Intel Feeds No Context No shared context Hard to tie the events from feed A to the events from feed B Repetitive Searching through event logs can be a repetitive process. Alert, Search, Find or Not, Repeat Can we automate this work without creating another data stream? Searching not analyzing Lack of expertise Even tier 3 analysts have blind spots Tier 1 analysts want to build skills and be effective as the first line of defense Can a security product teach new users how to use it effectively while supporting users all skill levels? Lack of time
- Analysts must battle a tide of incoming alerts Data deluge Morning coffee situation Too much data, too many sources AV EDR Firewall Application logs Threat Intel Feeds No Context No shared context Hard to tie the events from feed A to the events from feed B Repetitive Searching through event logs can be a repetitive process. Alert, Search, Find or Not, Repeat Can we automate this work without creating another data stream? Searching not analyzing Lack of expertise Even tier 3 analysts have blind spots Tier 1 analysts want to build skills and be effective as the first line of defense Can a security product teach new users how to use it effectively while supporting users all skill levels? Lack of time
- User: Show me process event data for odinaff.exe on all Windows 10 endpoints Artemis: Okay.
- User: Show me process event data for odinaff.exe on all Windows 10 endpoints Artemis: Okay.
- User: Show me process event data for odinaff.exe on all Windows 10 endpoints Artemis: Okay.
- Analysts must battle a tide of incoming alerts Data deluge Morning coffee situation Too much data, too many sources AV EDR Firewall Application logs Threat Intel Feeds No Context No shared context Hard to tie the events from feed A to the events from feed B Repetitive Searching through event logs can be a repetitive process. Alert, Search, Find or Not, Repeat Can we automate this work without creating another data stream? Searching not analyzing Lack of expertise Even tier 3 analysts have blind spots Tier 1 analysts want to build skills and be effective as the first line of defense Can a security product teach new users how to use it effectively while supporting users all skill levels? Lack of time