This document discusses log centralization in cloud environments. It describes FINRA's role as an independent financial industry regulator and how it monitors the stock market and registered brokers. It then discusses challenges of collecting logs from various cloud services (SaaS, IaaS, PaaS) and providers (AWS, Cisco, etc.). It provides examples of using AWS services like CloudTrail, CloudWatch, and Elastic MapReduce with Hadoop to collect and analyze logs and metrics in the cloud.

1. Splunking the Cloud
Log Centralization in an Amorphous Environment

2. Copyright 2014 FINRA
Who We Are
￭ FINRA—The Financial Industry Regulatory Authority—is an
independent, non-governmental regulator for all securities
firms doing business with the public in the United States
￭ MARKET – We monitor over 6 billion shares traded on the
stock market each day
￭ MEMBER – We monitor activities of over 660,000 active
registered brokers and dealers
￭ EDUCATION – www.FINRA.org; Risk Meter, Broker Check,
Calculators, Games
￭ TECHNOLOGY - http://technology.finra.org/

3. Copyright 2014 FINRA
The Amorphous Cloud
￭ Public vs Private
￭ Software as a Service (SaaS)
￭ Infrastructure as a Service (IaaS)
￭ Platform as a Service (Paas)
￭ Cloud Providers
Cloud Choices

4. Copyright 2014 FINRA
Software as a Service (SaaS)
￭ How to Get Logs from SaaS into Splunk?
• Did you sign the Contract?
￭ Traditional SaaS Assessment Questions
• Is Access Restricted?
• Do They Centrally Store and Retain?
• Are Logs Reviewed? Via Automation?
• Do I Need to Map to
Regulations/Controls/Standards?
￭ What is my Access Method?
• UI
• Access to Raw Logs
￭ Cisco Cloud Web Security (CWS)
Cloud Consumer

5. Copyright 2014 FINRA
IOCs ->FS-ISAC -> Soltra -> Splice -> ScanSafe

6. Copyright 2014 FINRA
Infrastructure as a Service (IaaS)
￭ Same as On-Premises, Only Different
• Building a Server == No Manual Intervention
• ip-10-97-142-121.us-west-2.compute.internal
• Tags
– Naming Conventions
– Choosing vs Following
– Continuously Monitor
• AutoScaling
￭ Inventory Based Deployment Server
• Separation of Duties
• Manifest Interrogation
• ClientName
Cloud Hosting

7. Copyright 2014 FINRA
Detailed CPU Utilization

8. Copyright 2014 FINRA
Platform as a Service (PaaS)
￭ Floor Cleaner and Dessert Topping
￭ AWS Elastic MapReduce (EMR) Hadoop
Framework
• Log to AWS Simple Storage Service (S3)
• Log to Hadoop/Hunk
￭ EMR Exposes Bootstrap
• Store Installation Container in S3
• Bootstrap installs HF*
• Adjusts Host value/deploymentclient.conf
• Starts splunkd
￭ Real-Time Collection of M/R Jobs
Building On the
Cloud

9. Copyright 2014 FINRA
Hadoop/EMR Job Interrogator

10. Copyright 2014 FINRA
Cloud Providers
￭ Vendor Event Monitoring/Management
Varies
• Raw Log Access vs Service vs Next Release
￭ AWS
• CloudTrail – Logs AWS API Activities to S3
• CloudWatch – Collect/Track Metrics
• Config – Collect/Track AWS Configuration
Changes
• Billing – Collects all Billing Reports from Master
Account
￭ Splunk Add-on for AWS
￭ IAM, S3, EC2, SG, AMI Compliance
Service Providers

11. Copyright 2014 FINRA
AWS IAM Compliance

12. Copyright 2014 FINRA
Two Years Ago, Today
￭ Traditional SIEM
• Tech Refresh
• Confined to Security Team
• Ingress/Egress Problems
• Discouraging POCs
￭ Attended SplunkLive! DC
• How Others Leveraging Splunk
• Diverse Data Input Options
• Enthusiasm
• Vendor Integration/Adoption
￭ Deployed SplunkCloud
• Security – Application – Operational Tool
• Changing Corporate Culture
• Working the Problem
Decision Point