Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

AWS re:Invent 2016: Security Automation: Spend Less Time Securing Your Applications (SAC316)

2,063 views

Published on

As attackers become more sophisticated, web application developers need to constantly update their security configurations. Static firewall rules are no longer good enough. Developers need a way to deploy automated security that can learn from the application behavior and identify bad traffic patterns to detect bad bots or bad actors on the Internet. This session showcases some of the real-world customer use cases that use machine learning and AWS WAF (a web application firewall) with automated incident response and machine learning to automatically identify bad actors. We also present tutorials and code samples that show how customers can analyze traffic patterns and deploy new AWS WAF rules on the fly.

Published in: Technology
  • Be the first to comment

AWS re:Invent 2016: Security Automation: Spend Less Time Securing Your Applications (SAC316)

  1. 1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. December 1, 2016 SAC316 Security Automation Using AWS WAF: Spend Less Time Securing Your Applications
  2. 2. What to expect from this session Introduction to AWS WAF AWS WAF 101
  3. 3. What to expect from this session Introduction to AWS WAF AWS WAF security automation strategies AWS WAF 101
  4. 4. What to expect from this session Introduction to AWS WAF AWS WAF security automation strategies AWS WAF 101 5 automation strategies 1. Provisioning WAF 2. Deploying WAF 3. Importing rules 4. Automated incident response 5. Learning-based protections
  5. 5. What to expect from this session AWS WAF security automation strategies AWS WAF 101 Demo and getting started Introduction to AWS WAF
  6. 6. What is AWS WAF AWS WAF 101
  7. 7. What is AWS WAF
  8. 8. Why AWS WAF? Application vulnerabilities Good users Bad guys Web server Database Exploit code AWS WAF
  9. 9. Why AWS WAF? Content abuse: Bots and scrapers Good users Bad guys Web server Database AWS WAF
  10. 10. Why AWS WAF? Application DDOS Good users Bad guys Web server Database AWS WAF
  11. 11. AWS WAF: Rules in action Monitor security events
  12. 12. AWS WAF: Integrated with AWS Amazon CloudFront Global content delivery network to accelerate websites, API, video content, and other web assets
  13. 13. AWS WAF: Integrated with AWS Amazon CloudFront Application Load Balancer Load balancer with advanced request routing, and support for microservices and container-based applications Global content delivery network to accelerate websites, API, video content, and other web assets Announcing today..
  14. 14. Why is ALB integration important?
  15. 15. Why is ALB integration important? Applications not using Amazon CloudFront Good users Bad guys AWS WAF region Amazon CloudFront Amazon S3
  16. 16. Why is ALB integration important? Block traffic that bypass any proxy, like CDN Good users Bad guys AWS WAF region Amazon CloudFront AWS WAF
  17. 17. Why is ALB integration important? Protect internal load balancer Good users Bad guys AWS WAF region Application server NGINX TLS termination TCP/SSL ELB
  18. 18. How to enable WAF on ALB Demo
  19. 19. What to expect from this session Introduction to AWS WAF AWS WAF security automation strategies AWS WAF 101 Demo and getting started
  20. 20. Why security automation Spend less time securing your applications Instead, focus on building applications
  21. 21. We built a WAF that has… Customizable and flexible rules APIs: Integration with DevOps …allowing several WAF automation strategies Quick rule update
  22. 22. AWS WAF security automation strategies.. Provisioning WAF Configuring rules Importing rules Automated incident response Learning-based protections … to spend less time securing applications
  23. 23. AWS WAF security automation strategies Provisioning WAF Configuring rules Importing rules Automated incident response Learning-based protections
  24. 24. Provisioning AWS WAF Step 1 – Create web ACL
  25. 25. Provisioning AWS WAF Rule 1: Whitelist [ALLOW] Rule 2: Blacklist [BLOCK] Rule 3: Common protection [BLOCK] Step 1 – Create web ACL Step 2 – Add rule
  26. 26. Provisioning AWS WAF IP whitelist SQL injection URL match Rule 1: Whitelist [ALLOW] IP blacklist Rule 2: Blacklist [BLOCK] Rule 3: Common protection [BLOCK] Step 1 – Create web ACL Step 2 – Add rule Step 3: Add condition
  27. 27. Provisioning AWS WAF IP Whitelist SQL injection URL match Rule 1: Whitelist [ALLOW] IP Blacklist Rule 2: Blacklist [BLOCK] Rule 3: Common protection [BLOCK] Step 1 – Create web ACL Step 2 – Add rule Step 3: Add condition Step 4: Associate CloudFront ALB
  28. 28. Provisioning AWS WAF: Reuse Spend less time by reusing WAF rules
  29. 29. Provisioning AWS WAF: Reuse IP whitelist internal IP SQL injection URL match Rule 1: Whitelist [ALLOW] IP blacklist known bad Rule 2: Blacklist [BLOCK] Rule 3: Common protection #1 [BLOCK] Web ACL #1 ALB 1 (dev env) Rule 4: Common protection #2 [BLOCK] XSS match Web ACL #2ALB 2 (prod env) Spend less time by reusing WAF rules
  30. 30. Provisioning AWS WAF: Reuse IP whitelist internal IP SQL injection URL match Rule 1: Whitelist [ALLOW] IP blacklist known bad Rule 2: Blacklist [BLOCK] Rule 3: Common protection #1 [BLOCK] Web ACL #1 ALB 1 (dev env) Rule 4: Common protection #2 [BLOCK] XSS match Web ACL #2ALB 2 (prod env) Spend less time by reusing WAF rules ALB 3 (new app)
  31. 31. Provisioning AWS WAF Quickly fix vulnerabilities Example: {CVE-2016-538} • Server-side web applications that utilize the HTTP_Proxy header as an environment variable • Attacker could intercept connections between a client and server. Quick solution: Use AWS WAF to configure a rule to detect and block web requests that contain a proxy header.
  32. 32. Provisioning AWS WAF IP whitelist internal IP SQL injection URL match Rule 1: Whitelist [ALLOW] IP blacklist known bad Rule 2: Blacklist [BLOCK] Rule 3: Common protection #1 [BLOCK] Web ACL #1 ALB 1 (dev env) Rule 4: Common protection #2 [BLOCK] XSS match Web ACL #2ALB 2 (prod env) Spend less time by reusing WAF rules ALB 3 (new app)
  33. 33. Provisioning AWS WAF IP whitelist internal IP SQL injection URL match Rule 1: Whitelist [ALLOW] IP blacklist known bad Rule 2: Blacklist [BLOCK] Rule 3: Common protection #1 [BLOCK] Web ACL #1 ALB 1 (dev env) Rule 4: Common protection #2 [BLOCK] XSS match Web ACL #2ALB 2 (prod env) Spend less time by reusing WAF rules ALB 3 (new app) Rule 5: CVE-2016-538 [BLOCK] Header match
  34. 34. Negative Typical of prod deployment ALLOW by default BLOCK known bad Provisioning AWS WAF: Rule strategy Positive Typical of restricted site BLOCK by default ALLOW known good Examples: • BLOCK MalwareIncIPRange • BLOCK “{;}” Examples: • ALLOW SeattleOfficeIPRange • ALLOW referrer header “example.com”
  35. 35. Provisioning AWS WAF Demo Show how to get started Reusing rules
  36. 36. AWS WAF security automation strategies Provisioning WAF Configuring rules Importing rules Automated incident response Learning-based protections
  37. 37. Configuring AWS WAF rules IP whitelist internal IP SQL injection URL match Rule 1: Whitelist [ALLOW] IP blacklist known bad Rule 2: Blacklist [BLOCK] Rule 3: Common protection #1 [BLOCK] Web ACL #1 ALB 1 (dev env) Rule 4: Common protection #2 [BLOCK] XSS match Web ACL #2ALB 2 (prod env) ALB 3 (new app) Rule 5: CVE-2016-538 [BLOCK] Header match
  38. 38. Configuring AWS WAF rules IP whitelist internal IP SQL injection URL match Rule 1: Whitelist [ALLOW] IP blacklist known bad Rule 2: Blacklist [BLOCK] Rule 3: Common protection #1 [BLOCK] Web ACL #1 ALB 1 (dev env) Rule 4: Common protection #2 [BLOCK] XSS match Web ACL #2ALB 2 (prod env) ALB 3 (new app) Rule 5: CVE-2016-538 [BLOCK] Header match
  39. 39. Configuring AWS WAF rules How to quickly get started with common protections?
  40. 40. Configuring AWS WAF rules Preconfigured AWS CloudFormation templates for common protection CloudFormation template AWS WAF Configuration
  41. 41. Configuring AWS WAF: Common protection Enable common protections  SQL injection  Cross-site scripting  Attack from known bad IP addresses
  42. 42. Preconfigured protections: Customer example Need quick setup and common protections like SQLi, XSS “Overall, the entire stack so far has been extremely helpful. I truly would say that this stack should almost be a standard built-in for anyone looking to use WAF as I cannot begin to tell you how useful and truly effective it is.” Describe eVitamins
  43. 43. Create a rule to block SQLi /login?x=test%20Id=10 /login?x=test%27%20UNION%20ALL%20select%20NULL%20-- /login?x=test’ UNION ALL select NULL -- Transform: URL decode True Match: SQL injection False
  44. 44. Configuring AWS WAF: Common protection Demo
  45. 45. AWS WAF security automation strategies Provisioning WAF Configuring rules Importing rules Automated incident response Learning-based protections
  46. 46. Configuring AWS WAF rules IP whitelist internal IP SQL injection URL match Rule 1: Whitelist [ALLOW] IP blacklist known bad Rule 2: Blacklist [BLOCK] Rule 3: Common protection #1 [BLOCK] Web ACL #1 ALB 1 (dev env) Rule 4: Common protection #2 [BLOCK] XSS match Web ACL #2ALB 2 (prod env) ALB 3 (new app) Rule 5: CVE-2016-538 [BLOCK] Header match
  47. 47. Configuring AWS WAF rules IP whitelist internal IP SQL injection URL match Rule 1: Whitelist [ALLOW] IP blacklist known bad Rule 2: Blacklist [BLOCK] Rule 3: Common protection #1 [BLOCK] Web ACL #1 ALB 1 (dev env) Rule 4: Common protection #2 [BLOCK] XSS match Web ACL #2ALB 2 (prod env) ALB 3 (new app) Rule 5: CVE-2016-538 [BLOCK] Header match Can we improve the common protections?
  48. 48. Configuring AWS WAF rules It is possible for almost any email server to block over 90% just based on IP reputation - http://www.spamrats.com/ip_reputation_spam_stats.pdf IP reputation lists can identify roughly 90% of all spam - http://www.acm.org/ - (http://dl.acm.org/citation.cfm?id=1831448)
  49. 49. Importing AWS WAF rules Import open source IP reputation lists
  50. 50. Importing AWS WAF rules Open source IP reputation lists
  51. 51. Importing AWS WAF rules
  52. 52. Configuring AWS WAF Rules IP whitelist internal IP SQL injection URL match Rule 1: Whitelist [ALLOW] IP blacklist known bad Rule 2: Blacklist [BLOCK] Rule 3: Common protection #1 [BLOCK] Web ACL #1 ALB 1 (dev env) Rule 4: Common protection #2 [BLOCK] XSS match Web ACL #2ALB 2 (prod env) ALB 3 (new app) Rule 5: CVE-2016-538 [BLOCK] Header match Rule 6: IP reputation [BLOCK] IP blacklist known bad
  53. 53. Configuring AWS WAF rules So far,  Whitelist known good  Blacklist known bad IP  Common protections like SQLi and XSS  Import IP reputation list
  54. 54. Configuring AWS WAF rules So far,  Whitelist known good  Blacklist known bad IP  Common protections like SQLi and XSS  Import IP reputation list How can you customize rules for your application?
  55. 55. AWS WAF security automation strategies Provisioning WAF Configuring rules Importing rules Automated incident response Learning-based protections
  56. 56. Why security automation • Set-and-forget rules are very effective • But are not customized for your applications • Malicious actors are adaptive and persistent • Incident response for threat mitigation
  57. 57. Why security automation Traditional incident response Good users Bad guys Server AWS WAF Logs Threat analysis Notification Security engineer
  58. 58. Why security automation We need.. • Sophisticated out of band analysis • Integrate application-specific data sources • Automated incident response
  59. 59. Why security automation Automated incident response Good users Bad guys Server AWS WAF Logs Threat analysis Rule updater Notification Security engineer
  60. 60. AWS WAF for automated incident response Automatically respond to incidents based on real-time analysis APIs for automation ~1 min rule updateReal-time processing
  61. 61. Security automation: Use cases HTTP floods Scans and probes Bots and scrapers Attackers Use cases that static rules cannot protect effectively
  62. 62. WAF example: A technical implementation Blocking bad bots dynamically with AWS WAF web ACLs
  63. 63. WAF example: Blocking bad bots What we need… • IPSet: Contains our list of blocked IP addresses • Rule: Blocks requests if requests match IP in our IPSet • Web ACL: Allow requests by default; contains our Rule and… • Mechanism to detect bad bots • Mechanism to add bad bot IP address to IPSet
  64. 64. WAF example: Detecting bad bots • Use robots.txt to specify which areas of your site or web app should not be scraped • Place file in your web root • Ensure there are links pointing to nonscrapable content • Hide a trigger script that normal users don’t see and good bots ignore $ cat webroot/robots.txt User-agent: * Disallow: /honeypot/ <a href="/honeypot/" class="hidden" aria- hidden="true">click me</a>
  65. 65. WAF example: Blacklist bad bots • Bad bots (ignoring your robots.txt) will request the hidden link • Trigger script will detect the source IP of the request • Trigger script requests change token • Trigger script adds source IP to IPSet blacklist • Web ACL will block subsequent request from that source $ aws --endpoint-url https://carrot.amazon.com/ carrot get- change-token { "ChangeToken": "acbc53f2-46db-4fbd- b8d5-dfb8c466927f” } $ aws --endpoint-url https://carrot.amazon.com/ carrot update- ip-set --cli-input-json '{ "IPSetId": ”<<IP SET ID>>", "ChangeToken": "acbc53f2-46db-4fbd-b8d5-dfb8c466927f", "Updates": [ { "Action": "INSERT", "IPSetDescriptor": { "Type": "IPV4", "Value": ”<<SOURCE IP>>/32" } } ] }’ { "ChangeToken": "acbc53f2-46db-4fbd- b8d5-dfb8c466927f” }
  66. 66. Automated incident response using AWS WAF Automated incident response is effective Customized for your application
  67. 67. Automated incident response: Customer example
  68. 68. MapBox uses WAF to protect from bots Good users Bad guys Serve r AWS WAF Logs Threat analysis Rule updater
  69. 69. Automated incident response using AWS WAF • But attackers are persistent • Adapt to firewall rules Can we adapt our firewall rules? Build continuously learning automated security
  70. 70. AWS WAF security automation strategies Provisioning WAF Configuring rules Importing rules Security Automation Learning-based protections
  71. 71. What is machine learning Machine learning is the technology that automatically finds patterns in your data and uses them to make predictions for new data points as they become available Your data + machine learning = smart applications
  72. 72. Amazon Machine Learning Easy-to-use, managed machine learning service built for developers Robust, powerful machine learning technology based on Amazon’s internal systems Create models using your data already stored in the AWS Cloud Deploy models to production in seconds
  73. 73. AWS WAF with Amazon Machine Learning Amazon Machine Learning GoodHTTPrequests BadHTTPrequests 2. Train model1. Build model 3. Evaluate model 4. Retrieve prediction ALLrealHTTPrequests UpdateAWSWAF AWS WAF
  74. 74. AWS WAF with Amazon Machine Learning A PoC on learning-based WAF
  75. 75. AWS WAF with Amazon Machine Learning The problem: Detect requests from domain generation algorithms Solution: Use referrer header to detect bad domains visiting my website based on machine learning
  76. 76. AWS WAF with Amazon Machine Learning 1. Data preparation – Feature engineering 2. Train model based on known good and bad domains 3. Evaluate using real data
  77. 77. AWS WAF with Amazon Machine Learning 1. Data preparation – Feature engineering
  78. 78. AWS WAF with Amazon Machine Learning 2. Train model based on known good and bad domains Good domains: Alexa 10,000 Bad domains: Known phishing domains
  79. 79. AWS WAF with Amazon Machine Learning 3. Evaluate using real data Use raw logs from CloudFront logs #Version: 1.0 #Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x- edge-result-type x-edge-request-id x-host-header cs-protocol cs-bytes time-taken x-forwarded-for ssl-protocol ssl-cipher x-edge-response-result- type cs-protocol-version 2014-05-23 01:13:11 FRA2 182 192.0.2.10 GET d111111abcdef8.cloudfront.net /view/my/file.html 200 www.displaymyfiles.com Mozilla/4.0%20(compatible;%20MSIE%205.0b1;%20Mac_PowerPC) - zip=98101 RefreshHit MRVMF7KydIvxMWfJIglgwHQwZsbG2IhRJ07sn9AkKUFSHS9EXAMPLE== d111111abcdef8.cloudfront.net http - 0.001 - - - RefreshHit HTTP/1.1 2014-05-23 01:13:12 LAX1 2390282 192.0.2.202 GET d111111abcdef8.cloudfront.net /soundtrack/happy.mp3 304 www.unknownsingers.com Mozilla/4.0%20(compatible;%20MSIE%207.0;%20Windows%20NT%205.1) a=b&c=d zip=50158 Hit xGN7KWpVEmB9Dp7ctcVFQC4E-nrcOcEKS3QyAez--06dV7TEXAMPLE== d111111abcdef8.cloudfront.net http - 0.002 - - - Hit HTTP/1.1
  80. 80. AWS WAF with Amazon Machine Learning
  81. 81. AWS WAF with Amazon Machine Learning Demo
  82. 82. AWS WAF with Amazon Machine Learning Category Result Accuracy 98% Recall true positive rate 78% False positive rate 1% True negative rate 99% How good is our machine learning model
  83. 83. Summary Spend less time securing your applications Instead, focus on building applications 1. Provisioning WAF – Reuse rules 2. Configuring WAF – Get started in minutes using CloudFormation template 3. Importing rules – 4. Automated incident response – DevOps WAF 5. Learning-based WAF –
  84. 84. Summary Spend less time securing your applications Instead, focus on building applications Provisioning WAF Reuse rules Configuring rules Configure common protections in minutes using CloudFormation templates Importing rules Automated reputation list from external sources Automated incident response Advanced application-specific firewall rules Learning-based protections Smart adaptive protections using Amazon ML
  85. 85. Remember to complete your evaluations!
  86. 86. Thank you! Get started with AWS WAF: https://console.aws.amazon.com/waf

×