In this session, we walk you through a demo of how a security team can build dashboards in minutes without having to gain deep knowledge on analytics. The AWS serverless services we use include AWS WAF logs, AWS Glue, Amazon Athena, and Amazon QuickSight.

1. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
Build a dashboard using serverless
security analytics
Umesh Ramesh
Cloud Infrastructure Architect
AWS Professional Services
Amazon Web Services
S D D 2 0 1
Rohit Rangnekar
Sr. IoT Architect
AWS Professional Services
Amazon Web Services

2. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
Introduction to AWS WAF and full logging
Overview of serverless analytics
Amazon Kinesis Data Firehose
Amazon Athena
AWS Glue
Amazon QuickSight
Solution architecture
Demo
Other ML-based architectures

3. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.

4. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS WAF overview
• Gatekeeper to your cloud applications
• Protecting Amazon API Gateway, Amazon CloudFront, and Application Load Balancer
• Inspect incoming HTTP requests and perform user-defined actions
• Protect your application from common web exploits
• Pay-as-you-use and DevOps-friendly (managed using API’s)
AWS WAF
Applications
Attackers
Customers
CloudFront
ALB
API Gateway

5. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tools availablewithin AWS WAF
Malicious
Traffic Blocking
• SQL Injection
Conditions
• XSS Conditions
• AWS Marketplace
Managed Rules
Traffic
Filtering
• Rate-Based Rules
• IP-Match Filters
• Geo-IP Filters
• Regex & String Match
Conditions
• Size Constraint
Conditions
Visibility and
Debugging
• Amazon CloudWatch
• Sampled Logs
• Full Logs

6. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
Key features of AWS WAF
• Customization & Automation
• AWS WAF Security Automations
• Use Amazon GuardDuty and AWS web application firewall to automatically
block suspicious hosts
• Fast Rule Update/Propagation
• Update existing rules and push to all regions in under 1 minute
• Real-time visibility through CloudWatch
• Incident response: react to attacks immediately
• Scalability & High-Volume Performance
• Designed for scale and speed
• Available in all edge locations through CloudFront and growing
• Request inspection done in ~1 milliseconds

7. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS WAF full logging
Action
Rate-based rules
Terminating rules
& corresponding
actions
Excluded rules

8. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS WAF full logging (cont.)
Client IP
User-agent
Host
Country
HTTP method
HTTP URI
HTTP
headers
Query string

9. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.

10. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
The serverless operation model
Zero infrastructure to
manage; zero system
administration
Never pay for idle
resources
$
Availability, fault
tolerance, and
security features
built in
Automatically
scales resources
with usage or on
demand

11. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS database and analytics stack
Business intelligence & machine learning
Data movement
AWS DMS | AWS Snowball | Kinesis Data Firehose | Amazon Kinesis Data Streams | Amazon Managed Streaming for Kafka
Amazon QuickSight (BI)
Relational databases
Amazon
RDS
Amazon
Aurora
Data lake
Amazon S3 Glacier
(Storage)
AWS Glue
(ETL & Data Catalog)
Amazon SageMaker (ML)
Amazon Macie
(Data Protection)
Non-relational databases Analytics
Amazon
DynamoDB
Amazon ElastiCache
Data Warehouse | Big data processing | Ad hoc
Amazon
Redshift
Amazon
EMR
Athena
Amazon Kinesis Data
Analytics
Amazon
Elasticsearch
Service
Real-time
AWS Lake Formation
(Data Lake)
Amazon
DocumentDB

12. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kinesis Data Firehose
• Capture, load, and transform streaming data into Amazon S3, Amazon Redshift,
Amazon ES, and Splunk
• Fully managed and pay-as-you-go pricing
• Scales elastically to handle varying data throughput
• Integrated data transformation with AWS Lambda
• Supports optional encryption for data
Capture and submit streaming
data to Kinesis Data Firehose
Analyze streaming data using your
favorite BI tools
Kinesis Data Firehose loads streaming data continuously
into Amazon S3, Amazon Redshift, Amazon ES, or Splunk

13. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
Athena
• Interactive query service that makes it easy to analyze data in Amazon S3
using ANSI SQL
• Decouple storage from compute
• Serverless—no infrastructure or resources to manage
• Pay per query and only for data scanned
• Security—AWS Identity and Access Management (IAM) for authentication;
encryption at rest and in transit
• Standards compliant and open storage file formats
Athena
Amazon
S3
Data
Catalog

14. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
Athena

15. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Glue Simple, flexible, cost-effective automation
for the undifferentiated heavy lifting of ETL
Data Catalog
ETL Job
Authoring
Discover data and extract
schema
Auto-generates
customizable ETL code in
Python and Scala
• AWS Glue crawler automatically discovers
data and stores schema
• Makes data searchable and available for
ETL & queries
• Generates customizable code
• Schedules and runs your ETL jobs
• Serverless, flexible, and built on open
standards

16. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon QuickSight
• Amazon QuickSight is a fully managed, serverless, cloud business intelligence
system
• Build visualizations, perform ad hoc analysis, and quickly get business insights
from your data
• Key Benefits
• No server license or infrastructure costs
• Connect to your data, wherever it is
• Designed to scale
• Simple and intuitive UX
• Self-service BI for users and analysts
• ML insights

17. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon QuickSight

18. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
Serverless analyticson AWS
Amazon S3 AWS Glue
Crawler
AWS Glue
Data Catalog
Athena Amazon
QuickSight
QUERY DATA

19. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.

20. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
Solution overview
AWS WAF Amazon Kinesis
Data Firehose
Amazon S3
Bucket
AthenaAmazon
QuickSight
AWS Glue
Data Catalog
Enable logs and
specify Kinesis Data
Firehose
Configure the desired
destination (in this case
Amazon S3)
Store logs and use as a
data source for analytics
Transforms log data from
JSON into format that
Athena understands
Use standard SQL queries
to extract desired data
Drag and drop to build
your own visualizations on
the data from Athena
AWS Glue
Crawlers
Crawlers scan your datasets
and populate the AWS Glue
Data Catalog
AWS WAF Logs
Security Alerts
Protect internet-facing
web applications
Notifications based on
AWS WAF metrics
Query data
Metadata
Users
View operationalsecurity
dashboards

21. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.

22. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.

23. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
GuardDuty and AWS WAF
https://aws.amazon.com/blogs/security/how-to-use-amazon-guardduty-and-aws-web-application-firewall-to-automatically-block-suspicious-hosts/
GuardDuty
Event
Findings
generated
Lambda
Parse
findings
AWS WAF
Rule
Update rules based
on findings
DynamoDB Table
Stores state data for
checking blocked host
Automatically block suspicious hosts

24. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
Log anomaly detection with Amazon SageMaker
AWS
WAF
AWS WAF
Logs
Amazon Kinesis
Data Firehose
Store logs in Amazon
S3
Amazon S3
Amazon
SageMaker
Training
Lambda Amazon
SageMaker
Inference
Anomaly
Decision
Update AWS WAF rules
Use machine learning to learn about and block additional suspicious events
Additional Logs (VPC
flow logs, etc.)
Algorithms
• Random Cut Forest
• IP Insights
• Deep AR
https://aws.amazon.com/blogs/machine-learning/detect-suspicious-ip-addresses-with-the-amazon-sagemaker-ip-insights-algorithm/

25. © 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
References
• Enhanced Security Analytics Using AWS WAF Full Logging—AWS Online Tech
Talks (Webinar)
https://www.youtube.com/watch?v=Zrnro4ohXdA&t=558s
• AWS Blog—Enabling serverless security analytics using AWS WAF full logs,
Athena, and Amazon QuickSight
https://aws.amazon.com/blogs/security/enabling-serverless-security-analytics-using-aws-waf-full-logs/

26. Thank you!
© 2019,Amazon Web Services, Inc. or its affiliates. All rights reserved.
Umesh Ramesh
uramesh@amazon.com
Rohit Rangnekar
rangneka@amazon.com