More Related Content Similar to Good Guys vs Bad Guys: Using Big Data to Counteract Advanced Threats (20) Good Guys vs Bad Guys: Using Big Data to Counteract Advanced Threats1. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents herein contain confidential information not to be copied.© 2014 Global Technology Resources, Inc. All Rights Reserved. Contents herein contain confidential information not to be copied.
GOOD GUYS VS BAD GUYS:
USING BIG DATA TO COUNTERACT ADVANCED
THREATS
Presented by GTRI
Micah Montgomery
Solutions Architect, Big Data and InfoSec
August 27, 2014
2. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Security Presentation Template
Scare
them
Unscare
them
3. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Security Presentation Template
Big Data
Advanced
Threats
4. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Here Comes the Scary Part…..
5. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Cisco 2014 Annual Security Report
Key Discoveries
– Attacks against infrastructure are targeting significant
resources across the Internet
– Malicious actors are using trusted applications to exploit
gaps in perimeter security
– Investigations of multinational companies show evidence
of internal compromise. Suspicious traffic is emanating
from their networks and attempting to connect to
questionable sites (100 percent of companies are
calling malicious malware hosts)
6. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
The Pervasiveness of Malicious Traffic
7. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
• By 2018, the gigabyte equivalent of all movies ever
made will cross the global Internet every 3 minutes.
• Globally, IP traffic will reach 400 terabits per second
(Tbps) in 2018, the equivalent of 148 million people
streaming Internet HD video simultaneously, all day,
every day.
• Global IP traffic in 2018 will be equivalent to 395 billion
DVDs per year, 33 billion DVDs per month, or 45 million
DVDs per hour.
Traffic demands are only going up…
8. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
9. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Advanced Threats Outpace the Defenders
Adversary
You
Time
Technical
Capabilities
10. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Advanced Threats Are Hard to Detect
100%
Valid credentials
were used
40
Average # of systems
accessed
243
Median # of days
before detection
63%
Of victims were notified
by external entity
Source: Mandiant M-Trends Report 2012 and 2013
11. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Advanced Threat Pattern – Not Signature Based
Infiltration
Back
Door
Exfiltration
Data
GatheringRecon
Phishing
or web
drive-by.
Email has
attached
malware or
link to
malware
Malware
installs
remote
access
toolkit(s)
Malware
obtains
credentials
to key
systems
and
identifies
valuable
data
Data is
acquired
and staged
for
exfiltration
Data is
exfiltrated
as
encrypted
files via
HTTP/S,
FTP, DNS
12. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Traditional SIEMs Miss The Threats
• Limited view of security threats. Difficult to collect all
data sources. Costly, custom collectors. Datastore
w/schema
• Inflexible search/reporting hampers investigations and
threat detection
• Scale/speed issues impede ability to do fast analytics
• Difficult to deploy and manage; often multiple
products
13. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Better Defensive Cybersecurity Tools Needed
14. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Here Comes The Solution
Big Data
15. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
So What is Big Data Anyway?
16. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
“Big Data” Definition
• Wikipedia: Collection of data sets so large and complex
that it becomes difficult to process using database
management tools
• Gartner: The Three Vs
1. Data Volume
2. Data Variety
3. Data Velocity
• Security has always been a Big Data problem; now we
are finding ways to solve it
17. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Big Data is Used Across IT and the Business
App Mgmt IT Ops Security Compliance Fraud
Business
Intelligence
Big Data
18. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Machine Data / Logs are Big Data
2013-08-09 16:21:38 10.11.36.29 98483 148 TCP_HIT 200 200 0 622 - - OBSERVED GET
www.neverbeenseenbefore.com HTTP/1.1 0 "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;
.NET CLR 2.0.50727; InfoPath.1; MS-RTC LM 8; .NET CLR 1.1.4322; .NET CLR 3.0.4506.2152; ) User
John Doe,"
08/09/2013 16:23:51.0128event_status="(0)The operation completed successfully. "pid=1300
process_image="John DoeDeviceHarddiskVolume1WindowsSystem32neverseenbefore.exe“
registry_type ="CreateKey"key_path="REGISTRYMACHINESOFTWAREMicrosoftWindows
NTCurrentVersion Printers PrintProviders John Doe-PCPrinters{} NeverSeenbefore" data_type""
Endpoint
Logs
Web Proxy
Aug 08 06:09:13 acmesep01.acmetech.com Aug 09 06:17:24 SymantecServer acmesep01: Virus
found,Computer name: ACME-002,Source: Real Time Scan,Risk name: Hackertool.rootkit,Occurrences:
1,C:/Documents and Settings/smithe/Local Settings/Temp/evil.tmp,"""",Actual action:
Quarantined,Requested action: Cleaned, time: 2009-01-23 03:19:12,Inserted: 2009-01-23
03:20:12,End: 2009-01-23 03:19:12,Domain: Default,Group: My CompanyACME Remote,Server:
acmesep01,User: smithe,Source computer: ,Source IP: 10.11.36.20
20130806041221.000000Caption=ACME-2975EBAdministrator Description=Built-in account for
administering the computer/domainDomain=ACME-2975EB InstallDate=NULLLocalAccount = IP:
10.11.36.20 TrueName=Administrator SID =S-1-5-21-1715567821-926492609-725345543
500SIDType=1 Status=Degradedwmi_ type=UserAccounts
Anti-virus
Authentications
19. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Big Data Analytics
Security for Business Innovation
Council report, “When
Advanced Persistent Threats
Go Mainstream,”
Chuck Hollis
VP – CTO, EMC Corporation
“The core of the most effective [advanced threat]
response appears to be a new breed of security
analytics that help quickly detect anomalous
patterns -- basically power tools in the hands of a
new and important sub-category of data scientists:
the security analytics expert..”
“[Security teams need] an analytical engine to sift
through massive amounts of real-time and
historical data at high speeds to develop trending
on user and system activity and reveal anomalies
that indicate compromise.”
20. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Step 1: Collect ALL The Data in One Location
Servers
Service
Desk
Storage
DesktopsEmail Web
Call
Records
Network
Flows
DHCP/ DNS
Hypervisor
Custom
Apps
Industrial
Control /
HVAC
Badges
Databases
Mobile
Intrusion
Detection
Firewall
Data Loss
Prevention
Anti-
Malware
Vulnerability
Scans
Traditional SIEM
Authentication
21. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Need Both Network and Endpoint
And Inbound/Outbound!
Network
Based
Indicators
Host Based
Indicators
Best
chance of
Detecting
the APT
22. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Enrich Indexed Data with External Data / Lookups
Geo-IP
Mapping
3rd-party
threat
intel
Asset
Info
Prohibited
Services /
Apps
Critical
Network
Segments /
Honeypots
Employee
Info
23. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Step 2: Identify Threat Activity
• What’s the M.O. of the attacker? (think like a
criminal)
• What/who are the most critical assets and
employees?
• What minute patterns/correlations in ‘normal’ IT
activities would represent ‘abnormal’ activity?
• What in my environment is
different/new/changed?
• What is rarely seen or standard deviations off the
norm?
24. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Survey
• Obtain a full picture of an environment: network, endpoint, mobile, and virtual,
including the technologies deployed to secure the environment.
Write
• Create targeted, context-aware malware.
Test
• Ensure the malware works as intended, specifically so it can evade security tools in
place.
Execute
• Navigate through the extended network—being environmentally aware, evading
detection, and moving laterally until reaching the target.
Accomplish
the Mission
• Gather data, create disruption, or cause destruction.
The APT Attack Chain
Cisco Annual Security Report 2014
25. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
What a Big Data Solution looks like
Big Data Architecture
Data Inclusion Model
All the original data from any source
No database schema to limit investigations/detection
Lookups against external data sources
Search & reporting flexibility
Advanced correlations
Math/statistics to baseline and find
outliers/anomalies
Real-time indexing and alerting
“Known” and “Unknown” threat detection
Scales horizontally to
Ideally, One product, UI, and datastore
26. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Big Data Evolution
Reactive
Search
and
Investigate
Proactive
Monitoring
and Alerting
Operational
Visibility
ProactiveReal-time
Business
Insight
27. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
28. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Sample Correlation of Unknown Threats
2013-08-09 16:21:38 10.11.36.29 98483 148 TCP_HIT 200 200 0 622 - - OBSERVED GET
www.neverbeenseenbefore.com HTTP/1.1 0 "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;
.NET CLR 2.0.50727; InfoPath.1; MS-RTC LM 8;.NET CLR 3.0.4506.2152; ) User John Doe,"
08/09/2013 16:23:51.0128event_status="(0)The operation completed successfully. "pid=1300
process_image="John DoeDeviceHarddiskVolume1WindowsSystem32neverseenbefore.exe“
registry_type ="CreateKey"key_path="REGISTRYMACHINESOFTWAREMicrosoftWindows
NTCurrentVersion Printers PrintProviders John Doe-PCPrinters{} NeverSeenbefore" data_type""
2013-08-09T12:40:25.475Z,,exch-hub-den-01,,exch-mbx-cup-
00,,,STOREDRIVER,DELIVER,79426,<20130809050115.18154.11234@acme.com>,johndoe@acme.com,
,685191,1,,, hacker@neverseenbefore.com , Please open this attachment with payroll information,,
,2013-08-09T22:40:24.975Z
Endpoint
Logs
Web Proxy
Email Server
All three occurring within a 24-hour period
Example Correlation - Spearphishing
User Name
User Name
Rarely seen email domain
Rarely visited web site
User Name
Rarely seen service
29. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Fingerprints of an Advanced Threat
What to Look For Why
Data
Source
Attack
Phase
Rarely seen registry, service, DLL. Or they fail
hash checks.
Malware or remote access
toolkit
OS Back door
Account creation or privilege escalation without
corresponding IT service desk ticket
Creating new admin accounts AD/ Service
Desk logs
Lateral
movement
A non-IT machine logging directly into multiple
servers. Or chained logins.
Threat accessing multiple
machines
AD /asset
info
Lateral
movement
For single employee: Badges in at one location,
then logs in countries away
Stealing credentials Badge/
VPN/ Auth
Data
gathering
Employee makes standard deviations more data
requests from file server with confidential data
than normal
Gathering confidential data for
theft
OS Data
gathering
Standard deviations larger traffic flows (incl DNS)
from a host to a given IP
Exfiltration of info NetFlow Exfiltration
30. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Step 3: Remediate and Automate
• Where else in my environment do I see the
“Indicators of Compromise” (IOC)?
• Remediate infected machines
• Fix weaknesses, including employee education
• Turn IOC into a real-time search for future threats
31. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Security Realities…
• Big Data is only as good as the data in
it and people behind the UI
• No replacement for capable
practitioners
• Put math and statistics to work for you
• Encourage IT Security creativity and
thinking outside the box
• Fine tuning needed; always will be false
positives
32. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Recap
• Step 1: Collect ALL The Data in One Location
• Step 2: Identify Threat Activity
• Step 3: Remediate and Automate
33. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents
herein contain confidential information not to be copied.
Questions?
34. © 2014 Global Technology Resources, Inc. All Rights Reserved. Contents herein contain confidential information not to be copied.
Thank You
Micah Montgomery
Solutions Architect, Big Data and InfoSec
mmontgomery@gtri.com
Editor's Notes https://www.cisco.com/web/offers/lp/2014-annual-security-report/index.html
Here is why SIEMs are not catching the advanced threats and not generating value in general.
For point 1, traditional SIEMs suffer from:
Collectors or backend database require data reduction or normalization – hampers security use cases
Little support for custom data sources
Brittle collectors break when data format changes
Clock icon indicates slow searches and long time to deploy.
Dollar sign is the expensive costs behind the long deployment (lots of prof serv) and multiple products. Better visibility on advanced threats and also way to do incident investigations after the fact Important – initially IT Ops and App mgmt. Takes in non-security data Key part of IT security is protecting confidential data. Which means detecting advanced threats, like cybercriminals or malicious insiders, before they can steal your data. To detect or investigate them, you need non-security and security data because advanced threats avoid detection from signature-based security products; the fingerprints of an advanced threat often are in the “non-security” data. Most traditional SIEMs just focus on gathering signature-based threats which do *not* have the fingerprints of advanced threats.
Also the above scenario is worse if there is no SIEM. Instead point UIs and grep are used and aggregating data is very manual and time consuming. What is not big data - data warehouses/database. They cannot take in all the original data. Also often batch-oriented, not real-time.
Explain the key security use cases