Build a dashboard using serverless security analytics - SDD201 - AWS re:Inforce 2019

1. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Build a dashboard using serverless security analytics Umesh Ramesh Cloud Infrastructure Architect AWS Professional Services Amazon Web Services S D D 2 0 1 Rohit Rangnekar Sr. IoT Architect AWS Professional Services Amazon Web Services

2. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda Introduction to AWS WAF and full logging Overview of serverless analytics Amazon Kinesis Data Firehose Amazon Athena AWS Glue Amazon QuickSight Solution architecture Demo Other ML-based architectures

4. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS WAF overview • Gatekeeper to your cloud applications • Protecting Amazon API Gateway, Amazon CloudFront, and Application Load Balancer • Inspect incoming HTTP requests and perform user-defined actions • Protect your application from common web exploits • Pay-as-you-use and DevOps-friendly (managed using API’s) AWS WAF Applications Attackers Customers CloudFront ALB API Gateway

5. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Tools availablewithin AWS WAF Malicious Traffic Blocking • SQL Injection Conditions • XSS Conditions • AWS Marketplace Managed Rules Traffic Filtering • Rate-Based Rules • IP-Match Filters • Geo-IP Filters • Regex & String Match Conditions • Size Constraint Conditions Visibility and Debugging • Amazon CloudWatch • Sampled Logs • Full Logs

6. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Key features of AWS WAF • Customization & Automation • AWS WAF Security Automations • Use Amazon GuardDuty and AWS web application firewall to automatically block suspicious hosts • Fast Rule Update/Propagation • Update existing rules and push to all regions in under 1 minute • Real-time visibility through CloudWatch • Incident response: react to attacks immediately • Scalability & High-Volume Performance • Designed for scale and speed • Available in all edge locations through CloudFront and growing • Request inspection done in ~1 milliseconds

10. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. The serverless operation model Zero infrastructure to manage; zero system administration Never pay for idle resources $ Availability, fault tolerance, and security features built in Automatically scales resources with usage or on demand

11. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS database and analytics stack Business intelligence & machine learning Data movement AWS DMS | AWS Snowball | Kinesis Data Firehose | Amazon Kinesis Data Streams | Amazon Managed Streaming for Kafka Amazon QuickSight (BI) Relational databases Amazon RDS Amazon Aurora Data lake Amazon S3 Glacier (Storage) AWS Glue (ETL & Data Catalog) Amazon SageMaker (ML) Amazon Macie (Data Protection) Non-relational databases Analytics Amazon DynamoDB Amazon ElastiCache Data Warehouse | Big data processing | Ad hoc Amazon Redshift Amazon EMR Athena Amazon Kinesis Data Analytics Amazon Elasticsearch Service Real-time AWS Lake Formation (Data Lake) Amazon DocumentDB

12. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Kinesis Data Firehose • Capture, load, and transform streaming data into Amazon S3, Amazon Redshift, Amazon ES, and Splunk • Fully managed and pay-as-you-go pricing • Scales elastically to handle varying data throughput • Integrated data transformation with AWS Lambda • Supports optional encryption for data Capture and submit streaming data to Kinesis Data Firehose Analyze streaming data using your favorite BI tools Kinesis Data Firehose loads streaming data continuously into Amazon S3, Amazon Redshift, Amazon ES, or Splunk

13. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Athena • Interactive query service that makes it easy to analyze data in Amazon S3 using ANSI SQL • Decouple storage from compute • Serverless—no infrastructure or resources to manage • Pay per query and only for data scanned • Security—AWS Identity and Access Management (IAM) for authentication; encryption at rest and in transit • Standards compliant and open storage file formats Athena Amazon S3 Data Catalog

15. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Glue Simple, flexible, cost-effective automation for the undifferentiated heavy lifting of ETL Data Catalog ETL Job Authoring Discover data and extract schema Auto-generates customizable ETL code in Python and Scala • AWS Glue crawler automatically discovers data and stores schema • Makes data searchable and available for ETL & queries • Generates customizable code • Schedules and runs your ETL jobs • Serverless, flexible, and built on open standards

16. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon QuickSight • Amazon QuickSight is a fully managed, serverless, cloud business intelligence system • Build visualizations, perform ad hoc analysis, and quickly get business insights from your data • Key Benefits • No server license or infrastructure costs • Connect to your data, wherever it is • Designed to scale • Simple and intuitive UX • Self-service BI for users and analysts • ML insights

20. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Solution overview AWS WAF Amazon Kinesis Data Firehose Amazon S3 Bucket AthenaAmazon QuickSight AWS Glue Data Catalog Enable logs and specify Kinesis Data Firehose Configure the desired destination (in this case Amazon S3) Store logs and use as a data source for analytics Transforms log data from JSON into format that Athena understands Use standard SQL queries to extract desired data Drag and drop to build your own visualizations on the data from Athena AWS Glue Crawlers Crawlers scan your datasets and populate the AWS Glue Data Catalog AWS WAF Logs Security Alerts Protect internet-facing web applications Notifications based on AWS WAF metrics Query data Metadata Users View operationalsecurity dashboards

23. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. GuardDuty and AWS WAF https://aws.amazon.com/blogs/security/how-to-use-amazon-guardduty-and-aws-web-application-firewall-to-automatically-block-suspicious-hosts/ GuardDuty Event Findings generated Lambda Parse findings AWS WAF Rule Update rules based on findings DynamoDB Table Stores state data for checking blocked host Automatically block suspicious hosts

24. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. Log anomaly detection with Amazon SageMaker AWS WAF AWS WAF Logs Amazon Kinesis Data Firehose Store logs in Amazon S3 Amazon S3 Amazon SageMaker Training Lambda Amazon SageMaker Inference Anomaly Decision Update AWS WAF rules Use machine learning to learn about and block additional suspicious events Additional Logs (VPC flow logs, etc.) Algorithms • Random Cut Forest • IP Insights • Deep AR https://aws.amazon.com/blogs/machine-learning/detect-suspicious-ip-addresses-with-the-amazon-sagemaker-ip-insights-algorithm/

25. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved. References • Enhanced Security Analytics Using AWS WAF Full Logging—AWS Online Tech Talks (Webinar) https://www.youtube.com/watch?v=Zrnro4ohXdA&t=558s • AWS Blog—Enabling serverless security analytics using AWS WAF full logs, Athena, and Amazon QuickSight https://aws.amazon.com/blogs/security/enabling-serverless-security-analytics-using-aws-waf-full-logs/