Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Automating DDoS Response in the Cloud - SID324 - re:Invent 2017

1,254 views

Published on

If left unmitigated, Distributed Denial of Service (DDoS) attacks have the potential to harm application availability or impair application performance. DDoS attacks can also act as a smoke screen for intrusion attempts or as a harbinger for attacks against non-cloud infrastructure. Accordingly, it's crucial that developers architect for DDoS resiliency and maintain robust operational capabilities that allow for rapid detection and engagement during high-severity events. In this session, you learn how to build a DDoS-resilient application and how to use services like AWS Shield and Amazon CloudWatch to defend against DDoS attacks and automate response to attacks in progress.

  • Be the first to comment

Automating DDoS Response in the Cloud - SID324 - re:Invent 2017

  1. 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS re:INVENT Automating DDoS Response in the Cloud J e f f r e y L y o n , A W S S y s t e m D e v e l o p m e n t M a n a g e r Y a z i d B o u t e j d e r , A W S S o l u t i o n s A r c h i t e c t E r i c N e u s t a d t e r , V P o f T e c h n o l o g y , T h e P o k é m o n C o m p a n y I n t e r n a t i o n a l S I D 3 2 4 N o v e m b e r 3 0 , 2 0 1 7
  2. 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. TODAY’S OBJECTIVES • Types of DDoS threats • Evolution of DDoS mitigation strategy • PREPARE: build a DDoS-resilient application on AWS • MONITOR: awareness of the threat environment and application health • RESPOND: engaging the AWS DDoS Response Team (DRT)
  3. 3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. TYPES OF THREATS Bad BotsDDoS Application Attacks UDP floods SYN floods Slowloris SSL abuse HTTP floods UDP reflection Content scrapers Scanners & probes CrawlersApplication Layer Network/ Transport Layer SQL injection Application exploits
  4. 4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EVOLUTION OF DDOS MITIGATION On-Premises Cloud-Routed Cloud-Native
  5. 5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ON-PREMISES • Scale network and fixed infrastructure to mitigate DDoS attacks on-site • Visibility and control • Large capital expenditures, maintenance costs, and in-house expertise
  6. 6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CLOUD-ROUTED • Route traffic to other networks for better mitigation capacity, managed services • Mitigate larger DDoS attacks without upfront investment or in- house expertise • Black box solution—can introduce latency, additional points of failure, increased operating costs
  7. 7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CLOUD-NATIVE • Automatic, always-on DDoS protection for all applications on AWS • Leverage 16 AWS Regions and 107 Edge Locations to mitigate large attacks close to the source • Simple, flexible, and affordable • Robust capabilities without undifferentiated heavy lifting
  8. 8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS SHIELD Standard Protection Advanced Protection Available to ALL AWS customers at no additional cost Paid service that provides additional protections, features, and benefits
  9. 9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Automatic defense against the most common network and transport layer DDoS attacks for any AWS resource, in any AWS Region Comprehensive defense against all known network and transport layer attacks when using Amazon CloudFront and Amazon Route 53 Application layer defense available when using AWS WAF AWS SHIELD Standard Protection Automatically provided to all AWS customers at no additional cost
  10. 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Fast escalation to the AWS DDoS Response Team (DRT) to assist with complex edge cases Attack visibility and enhanced detection Cost Protection to mitigate economic attack vectors AWS WAF for application-layer defense, at no additional cost AWS SHIELD Advanced Protection Available globally on Amazon CloudFront, Amazon Route 53, and in select AWS Regions
  11. 11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Effective Against: • HTTP Floods • Bad Bots • Suspicious IPs Effective Against: • SSL Attacks • Slowloris • Malformed HTTP Effective Against: • SYN Floods • Reflection Attacks • Suspicious Sources DEFENSE IN DEPTH Border Network Network Layer Mitigations AWS Services Web Layer Mitigations Customer Infrastructure DDoS Detection Internet Internet- Layer Mitigations DDoS Effective Against: • Large-scale attacks Effective Against: • Sophisticated Layer 7 attacks DDoS Response Team
  12. 12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. PREPARE: DDOS-RESILIENT ARCHITECTURE Amazon Route 53 ALB Security Group Amazon EC2 Instances Application Load Balancer Amazon CloudFront Public Subnet Web Application Security Group Private Subnet AWS WAF Amazon API Gateway DDoS Attack Users Globally distributed attack mitigation capability SYN proxy feature that verifies three-way handshake before passing to the application Slowloris mitigation that reaps long-lived collections Mitigates complex attacks by allowing only the most reliable DNS queries Validates DNS Provides flexible rule language to block or rate-limit malicious requests
  13. 13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. MONITOR: DEMONSTRATION
  14. 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. R E SPO ND I NG T O HI GH- S E VE R I T Y E VE NT S YAZID BOUTEJDER, AWS SOLUTIONS ARCHITECT
  15. 15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ALARM RESPONSE • Opportunity to review CloudWatch or custom dashboards • Identify availability or performance concerns • Check for on-premises or smokescreen attacks • Escalate to AWS Support or the AWS DDoS Response Team (DRT)
  16. 16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. KEY CLOUDWATCH METRICS Metrics that can indicate a DDoS attack or anomalous volume of traffic • AWS WAF: AllowedRequests, CountedRequests, BlockedRequests • AWS Shield Advanced: DDoSDetected, DDoSAttackBitsPerSecond
  17. 17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. KEY CLOUDWATCH METRICS Indicators of application anomaly, not specific to DDoS • Amazon CloudFront: Requests, TotalErrorRate • Amazon Route 53: HealthCheckStatus • Classic Load Balancer: BackendConnectionErrors, HTTPCode.*, Latency, RequestCount, SpilloverCount, SurgeQueueLength, UnHealthyHostCount
  18. 18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. KEY CLOUDWATCH METRICS Indicators of application anomaly, not specific to DDoS • Application Load Balancer: ActiveConnectionCount, ConsumedLCUs, HTTPCode.*Count, NewConnectionCount, ProcessedBytes, RejectedConnectionCount, RequestCount, TargetConnectionErrorCount, TargetResponseTime, UnhealthyHostCount • Network Load Balancer: ActiveFlowCount, ConsumedLCUs, UnHealthyHostCount, NewFlowCount, ProcessedBytes, TCP_Client_Reset_Count, TCP_ELB_Reset_Count, TCP_Target_Reset_Count
  19. 19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. KEY CLOUDWATCH METRICS Indicators of application anomaly, not specific to DDoS • Amazon EC2: CPUUtilization, NetworkIn • Auto Scaling: GroupMaxSize
  20. 20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. IMMEDIATE ACTIONS • Verify the performance and availability of the application • Check Sampled Requests in AWS WAF • Use a regular rule to block malicious patterns • Use a rate-based rule to temporarily block heavy hitting IPs
  21. 21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DEPLOY CLOUDFRONT QUICKLY • Keep on standby or deploy in an emergency • Protects web applications on AWS or hosted elsewhere • Supports static and dynamic content • Follow the guide at http://amzn.to/2mYNX6A
  22. 22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ENGAGING WITH AWS • Open a case with service of “AWS Shield” via AWS Management Console or API • Select the highest available priority (e.g., “Urgent” or “Critical”) • Is there a better way?
  23. 23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. IMPROVING EMERGENCY ENGAGEMENT • Case generation time can be reduced by automating case creation and using standardized messaging • Predefined, unambiguous messaging can reduce the potential for human error • Time-to-escalate is reduced by parallelizing engagement workflows • Solution: Programmatically generate an AWS Support case and notify the AWS DDoS Response Team (DRT)
  24. 24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS SHIELD ENGAGEMENT LAMBDA Operations Engineer DRT Customer Account AWS Shield Engagement Lambda AWS Support AWS Lambda Event Trigger (e.g., AWS IoT button) DRT Notification Topic AWS Managed Capabilities
  25. 25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS SHIELD ENGAGEMENT LAMBDA • STEP 1: Download documentation from http://bit.ly/2ic3XAW • STEP 2: Follow the instructions to create the AWS Lambda function and configure an event trigger (like an AWS IoT button) • STEP 3: Configure variables in the provided function • STEP 4: Create an AWS IAM execution role and click “Create function”
  26. 26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. // User configurable options var config = { // Change this to ‘critical’ if you are subscribed to Enterprise Support severity: ‘urgent’, // Change this to ‘advanced’ if you are subscribed to AWS Shield Advanced shield: ‘standard’, // Change this to ‘off’ after testing test: ‘on’, CONFIGURE VARIABLES
  27. 27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. // Modify subject and message if not subscribed to AWS Shield Advanced // Change subject and message to the path of a .txt file that you created in S3 standardSubject: 'http://s3.amazonaws.com/aws-shield- lambda/EngagementSubject.txt', standardMessage: 'http://s3.amazonaws.com/aws-shield- lambda/EngagementBody.txt' CONFIGURE VARIABLES
  28. 28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. RESPOND: DEMONSTRATION
  29. 29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. A WS S HI E LD A T PO K É MO N ERIC NEUSTADTER VP OF TECHNOLOGY THE POKÉMON COMPANY INTERNATIONAL
  30. 30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. WELCOME TO THE POKÉMON TRAINER CLUB
  31. 31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. PLEASE ASK YOUR PARENTS FOR HELP
  32. 32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. THE POKÉMON TRAINER CLUB (PTC) • Used for minigames on Pokémon.com • Logging in to the Pokémon Global Link • Play the Pokémon Trading Card Game Online • Register for Play! Pokémon events
  33. 33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. THEN, POKÉMON GO • PTC was added to Pokémon GO late in the development cycle Without it, minors wouldn’t have been able to play • Pokémon GO was a success beyond anyone’s expectations Does anyone plan for 750 million downloads?
  34. 34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. FINDING POKÉMON “Your device will vibrate to alert you when a wild Pokémon is nearby. If you don’t see any Pokémon nearby, take a walk! Pokémon love places like parks, so try visiting a local recreational area.”
  35. 35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. POKÉMON GO BROUGHT NEW CHALLENGES • Massive increase in legitimate users and traffic • Massive, disproportional increase in illegitimate users and traffic • Bots • Scanners • DDoS attacks
  36. 36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. BOTS: FREE, PAID, OR SOURCE ON GITHUB Partial feature list from a bot on GitHub: • Search and spin Pokéstops and Gyms • Diverse options for humanlike behavior from movement to overall game play • Advanced catch, evolve, and transfer configuration using our PokémonOptimizer
  37. 37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. BOTS: FREE, PAID, OR SOURCE ON GITHUB • Determine which Pokéball to use • Rules to determine the use of Razz and Pinap Berries • Transfer Pokémon in bulk • Telegram integration—reporting of bot's events
  38. 38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. BOTS: FREE, PAID, OR SOURCE ON GITHUB • Issue command through Telegram: Activate Lucky Egg/Incense, Snipping • Docker support
  39. 39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SCANNERS • Simulate very large crowds to gather data • Let you skip the game play to get to the prize
  40. 40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. PTC AND THE CLOUD-ROUTED WAF For years, PTC had been protected by a cloud-routed WAF provider: • That had been sufficient without the focus on PTC brought by GO
  41. 41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. PTC AND THE CLOUD-ROUTED WAF The increase in traffic brought on by GO overwhelmed our provider: • Management interface would become unusable • Traffic would stop flowing altogether • Rapidly growing traffic volume meant we had to find a new solution and implement it quickly
  42. 42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. MOVING TO AWS SHIELD ADVANCED • Existing application on AWS • The next major Pokémon GO event was only two weeks away: • Pokémon DevOps and InfoSec worked closely with AWS • Started slowly moving traffic in a week • 100% of GO login traffic was protected by AWS Shield Advanced in less than two weeks from “go”
  43. 43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. LIFE WITH AWS SHIELD ADVANCED Cloud-routed WAF issues are behind us: • No more WAF capacity issues taking us offline Pokémon is now seeing: • Lower latency through the WAF • Superior analytics and logging
  44. 44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. LIFE WITH AWS SHIELD ADVANCED Close cooperation with AWS: • Regular roadmap and feature discussions • Engaging the AWS Shield team via AWS IoT button enables rapid creation of incident bridge and reduces time-to-engage
  45. 45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CLOSING THOUGHTS • Bots and scanners will not go away • AWS Shield makes it easier to protect applications on AWS (or elsewhere) • AWS WAF is not a black box, provides better latency and throughput • Greatly simplified incident response process • What other operational processes can we automate?
  46. 46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thank you!

×