Your SlideShare is downloading. ×
0
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

DDoS Resiliency with Amazon Web Services (SEC305) | AWS re:Invent 2013

14,767

Published on

It's a rough world out there, filled with mega bot nets that threaten the availability of your web service. How do you keep your service running in the event of a 10,000x increase in traffic? …

It's a rough world out there, filled with mega bot nets that threaten the availability of your web service. How do you keep your service running in the event of a 10,000x increase in traffic? Maximizing service availability under DDoS conditions requires thoughtful service architecture, and at times, fast acting operations teams. This presentation covers best practices for DDoS-resilient services.

Published in: Technology
0 Comments
18 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
14,767
On Slideshare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
212
Comments
0
Likes
18
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. DDoS Resilience with Amazon Web Services nated@amazon.com November 14, 2013 © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 2. Agenda • • • • Anatomy of DDoS Things We Do So You Don’t Have To Designing for Availability Attack Response
  • 3. DDoS Facts • Yes, DDoS attacks are on the rise and the big ones are getting bigger • …although those attacks average out to ~14Gbps* and target services owners ~1 per year *source: Arbor Networks
  • 4. DDoS Facts Percentile Max Gbps 10 20 30 40 50 60 70 80 90 95 99 2.39 4.28 6.55 8.27 10.49 11.85 13.97 17.38 25.45 35.74 84.90 Max 299.43 Average Duration (minutes) 5.87 7.68 9.00 10.53 13.23 16.80 23.12 35.87 66.13 141.74 906.80 13.81 *source: Arbor Networks
  • 5. DDoS Anatomy Application Exhaustion /search.php?expensive-params service attacker
  • 6. DDoS Anatomy Host Exhaustion attacker service attacker
  • 7. DDoS Anatomy attacker Traditional Datacenter Exhaustion attacker traditional datacenter transit attacker
  • 8. DDoS Anatomy attacke r attacke r Intermediary Exhaustion attacke r transit traditional datacenter attacke r transit transit attacke r transit attacke r attacke r
  • 9. DDoS Anatomy • Large enough attacks consume the capacity of application layer, host, datacenter connectivity, Internet connectivity, or intermediary networks
  • 10. How can we help you? • Scale and Diversity of AWS • Resilient Service Designs • Business or Enterprise Support
  • 11. Things We Do So You Don’t Have To
  • 12. model credit: Scale
  • 13. Scale traditional datacenter transit
  • 14. Scale More Bandwidth transit AWS region transit transit
  • 15. Scale More Compute transit AWS region transit transit
  • 16. Scale More Points of Presence transit AWS region AWS edge AWS edge transit transit AWS edge
  • 17. Scale Attack Absorbed attacker transit AWS region AWS edge transit transit attacker AWS edge AWS edge attacker
  • 18. Diversity transit peer Internet Transit and Peering peer peer peer transit AWS region peer transit
  • 19. Diversity
  • 20. Diversity
  • 21. Diversity Amazon Route 53 Example - Anycast Striping • Leverages Resolver Behavior • Edge Location Diversity • Network Path Diversity
  • 22. Delegation Set [nated@xyz ~]$ dig NS internetkitties.com ;; QUESTION SECTION: ;internetkitties.com. ;; ANSWER SECTION: internetkitties.com. internetkitties.com. internetkitties.com. internetkitties.com. IN NS 172800 172800 172800 172800 IN IN IN IN NS NS NS NS ns-1131.awsdns-13.org. ns-1751.awsdns-26.co.uk. ns-340.awsdns-42.com. ns-952.awsdns-55.net.
  • 23. Delegation Set [nated@xyz ~]$ dig NS internetkitties.com ;; QUESTION SECTION: ;internetkitties.com. ;; ANSWER SECTION: internetkitties.com. internetkitties.com. internetkitties.com. internetkitties.com. IN NS 172800 172800 172800 172800 IN IN IN IN NS NS NS NS ns-1131.awsdns-13.org. ns-1751.awsdns-26.co.uk. ns-340.awsdns-42.com. ns-952.awsdns-55.net.
  • 24. Edge Location Diversity awsdns-13.org. awsdns-26.co.uk. awsdns-42.com. awsdns-55.net.
  • 25. Edge Location Diversity awsdns-13.org. awsdns-26.co.uk. awsdns-42.com. awsdns-55.net.
  • 26. Edge Location Diversity awsdns-13.org. awsdns-26.co.uk. awsdns-42.com. awsdns-55.net.
  • 27. Edge Location Diversity awsdns-13.org. awsdns-26.co.uk. awsdns-42.com. awsdns-55.net.
  • 28. Edge Location Diversity awsdns-13.org. awsdns-26.co.uk. awsdns-42.com. awsdns-55.net.
  • 29. Network Path Diversity awsdns-13.org. awsdns-26.co.uk. awsdns-42.com. awsdns-55.net. [nated@xyz ~]$ traceroute ns-1131.awsdns-13.org. [nated@xyz ~]$ traceroute ns-1751.awsdns-26.co.uk. traceroute to ns-1131.awsdns-13.org (205.251.196.107), 64 hops max, 52 byte packets traceroute to ~]$ traceroute ns-340.awsdns-42.com. 1 (192.168.1.1) 1.748 ms 0.830 ms 0.750 ms [nated@xyz ns-1751.awsdns-26.co.uk (205.251.198.215), 64 hops max, 52 byte packets 1 *traceroute to ns-340.awsdns-42.com (205.251.193.84), 64 hops max, 52 byte packets (192.168.1.1) 1.298 ms 0.755 ms 0.694 ms 2 ** [nated@xyz ~]$ traceroute ns-952.awsdns-55.net. 2 *** 1 (192.168.1.1) 2.444 ms 1.676 (68.85.255.255) 14.634 ms 12.822 ms 10.774 ms 3 cat.seattle.wa.seattle.comcast.net ms 1.028 ms (205.251.195.184), 64 hops max, 52 byte packets traceroute to ns-952.awsdns-55.net 3 cat.seattle.wa.seattle.comcast.net (68.85.255.255) 9.254 ms 24.156 ms 19.167 ms 2 *** 4 ae-20-0-ar03.burien.wa.seattle.comcast.net (69.139.164.125) 31.766 ms 13.898 ms 1 (192.168.1.1) 1.352 ms 0.642 ms 0.630 ms 4 ae-20-0-ar03.seattle.wa.seattle.comcast.net (69.139.164.129) 17.281 ms 18.580 ms 17.906 3 cat.seattle.wa.seattle.comcast.net (68.85.255.255) 19.842 ms 23.018 ms 26.469 ms 5 ae-20-0-ar03.seattle.wa.seattle.comcast.net (69.139.164.129) 20.108 ms 2 *** 5 he-1-5-0-0-11-cr01.seattle.wa.ibone.comcast.net (68.86.94.65) 20.753 ms 29.955 ms 4 ae-20-0-ar03.seattle.wa.seattle.comcast.net (69.139.164.129) 24.366 ms 20.842 ms 3 cat.seattle.wa.seattle.comcast.net (68.86.93.5) 18.781 ms 16.253 6 he-1-7-0-0-11-cr01.seattle.wa.ibone.comcast.net (68.85.255.255)34.612 ms 30.382 ms 17.851 ms 6 ae12.edge2.seattle3.level3.net (4.68.63.65)(68.86.93.173) 30.211 ms ms 17.221 ms 5 he-1-12-0-0-10-cr01.seattle.wa.ibone.comcast.net 38.159 ms 4 be-1-ur08.seattle.wa.seattle.comcast.net (69.139.164.134) ms 13.561 ms 7 ae12.edge2.seattle3.level3.net (4.68.63.65) 34.371 ms 36.504 msms 49.457 ms 49.945 ms 7 ae-31-51.ebr1.seattle1.level3.net (4.69.147.150) 31.948 ms 29.775 ms 6 ae12.edge2.seattle3.level3.net (4.68.63.65) 33.596 ms 48.510 27.301 5 ae-1-0-ar03.seattle.wa.seattle.comcast.net (68.85.240.94) 21.009 ms ms 8 ae-31-51.ebr1.seattle1.level3.net (4.69.147.150) 48.557 ms 60.610 43.456 ms ms 8 ae-7-7.ebr2.sanjose1.level3.net (4.69.132.49) 45.286167.112 ms 161.82143.219 7 ae-32-52.ebr2.seattle1.level3.net (4.69.147.182) 162.580 ms ms ms 56.751 ms 6 he-1-12-0-0-11-cr01.seattle.wa.ibone.comcast.net 9 ae-7-7.ebr2.sanjose1.level3.net (4.69.132.49) 58.662 ms 46.830 ms 62.458 ms 9 ae-62-62.csw1.sanjose1.level3.net (4.69.153.18) 44.181(68.86.93.177) 17.366 ms 19.162 ms 8 ae-2-2.ebr2.denver1.level3.net (4.69.132.54) 163.723 ms 159.037 ms 174.670 ms ms 7 be-12-pe03.seattle.wa.ibone.comcast.net (68.86.84.106) ae-3-80.edge1.sanjose3.level3.net (4.69.152.144) 46.817 ms 19.949 ms 22.968 ms 24.976 ms 10 10 9 ae-3-3.ebr1.chicago2.level3.net (4.69.132.62) 60.700 ms 47.997 ms 54.477 ms ae-2-2.ebr2.sanjose5.level3.net (4.69.148.141) 169.379 ms 167.307 ms 168.454 ms 8 *** 4.53.208.22 (4.53.208.22) 54.634 ms 60.111 ms 44.187 ms 11 11 10 ae-6-6.ebr1.chicago1.level3.net (4.69.148.201)166.002 ms 168.125 ms 164.232 ms ae-6-6.ebr2.losangeles1.level3.net (4.69.140.189) 55.190 ms 58.829 ms 55.751 ms 9 *** 205.251.229.155 (205.251.229.155) 47.758 ms 12 12 11 ae-2-2.ebr2.newyork2.level3.net (4.69.132.66) 167.861 ms ms ae-92-92.csw4.losangeles1.level3.net (4.69.137.30) 49.261 167.893 ms 160.681 ms 10 * 65-122-235-178.dia.static.qwest.net (65.122.235.178) 40.707 ms 30.916 ms 205.251.230.91 (205.251.230.91) 52.714 ms 43.560 53.091 ms 13 13 12 ae-1-100.ebr1.newyork2.level3.net (4.69.144.139)163.919 ms ms ae-3-80.edge5.losangeles1.level3.net (4.69.135.253) 58.707 ms166.782 ms 161.686 ms 11 14 13 4.69.201.45 (4.69.201.45) 164.023 ms SFO5 205.251.225.22 (205.251.225.22) 85.275 ms 14 amazon.com.edge5.losangeles1.level3.net (205.129.4.26) 46.477 ms 36.525 ms 42.110 ms 12 205.251.225.122 (205.251.225.122) 35.017 160.461 ms 14 ae-42-42.ebr2.london1.level3.net (4.69.137.69) 165.560 msms 38.568 ms 15 LAX3 13 205.251.226.136 (205.251.226.136) 36.560 ms 15 16 17 18 19 ae-46-46.ebr2.amsterdam1.level3.net (4.69.143.73) 165.627 ms 14 SEA50 ae-59-224.csw2.amsterdam1.level3.ne (t4.69.153.214) 172.909 ms 166.052 ms 4.69.162.154 (4.69.162.154) 166.353 ms 212.72.41.162 (212.72.41.162) 171.714 ms 174.033 ms 179.219 ms AMS50
  • 30. Striping in Action
  • 31. Striping in Action
  • 32. Striping in Action awsdns-13.org. awsdns-26.co.uk. awsdns-42.com. awsdns-55.net.
  • 33. Striping in Action awsdns-13.org. awsdns-26.co.uk. awsdns-42.com. awsdns-55.net.
  • 34. Striping in Action awsdns-13.org. awsdns-26.co.uk. awsdns-42.com. awsdns-55.net.
  • 35. Diversity client AWS edge AWS edge AWS region AWS edge transit client AWS edge attacker
  • 36. Diversity • Amazon Route 53 - Anycast Striping • Amazon CloudFront Edge Locations • AWS Regions
  • 37. How can we help you? • Scale and Diversity of AWS • Resilient Service Designs • Business or Enterprise Support
  • 38. How can we help you? • Amazon Route 53 and Amazon CloudFront • Resilient Service Designs • Business or Enterprise Support
  • 39. Designing for Resilience
  • 40. Designing for Resilience • • • • • N+1 Failover Resilient Clients Capped Workloads Process Isolation Shuffle Sharding
  • 41. Designing for Resilience • • • • • N+1 Failover Resilient Clients Capped Workloads Process Isolation Shuffle Sharding
  • 42. N+1 Failover • Scale Out, Plus Redundancy
  • 43. N+1 Failover • Scale Out, Plus Redundancy • Failure of 1/100 < Failure of 1/10
  • 44. N+1 Failover • Scale Out, Plus Redundancy • Failure of 1/100 < Failure of 1/10 • Automatic Failover with Health Checked DNS
  • 45. N+1 Failover attacker client
  • 46. N+1 Failover attacker client
  • 47. N+1 Failover Check out Amazon Route 53 Health Checks
  • 48. Designing for Resilience • • • • • N+1 Failover Resilient Clients Capped Workloads Process Isolation Shuffle Sharding
  • 49. Resilient Clients • Use multi-record RRSets • Randomize the record on connect retry • Popular HTTP clients already do this!
  • 50. Resilient Clients [nated@xyz ~]$ dig www.internetkitties.com ;; QUESTION SECTION: ;www.internetkitties.com. IN ;; ANSWER SECTION: www.internetkitties.com. 32 IN d3g5kqnbrlf3fg.cloudfront.net. d3g5kqnbrlf3fg.cloudfront.net. d3g5kqnbrlf3fg.cloudfront.net. d3g5kqnbrlf3fg.cloudfront.net. d3g5kqnbrlf3fg.cloudfront.net. d3g5kqnbrlf3fg.cloudfront.net. d3g5kqnbrlf3fg.cloudfront.net. d3g5kqnbrlf3fg.cloudfront.net. A CNAME 30 IN A 30 IN A 30 IN A 30 IN A 30 IN A 30 IN A 30 IN A 30 IN A d3g5kqnbrlf3fg.cloudfront.net. 54.230.69.190 54.230.71.141 54.230.71.172 54.230.71.233 54.240.188.66 54.230.68.41 54.230.68.212 54.230.69.141
  • 51. Resilient Clients Browser Packet Capture Num 4 5 6 7 8 9 11 12 13 14 15 16 Time 2.535515 2.736659 2.93782 3.138996 3.339767 3.540963 3.541123 3.742296 3.824502 3.824515 4.024809 4.225094 Source 10.61.60.17 10.61.60.17 10.61.60.17 10.61.60.17 10.61.60.17 10.61.60.17 10.61.60.17 10.61.60.17 10.61.60.17 10.61.60.17 10.61.60.17 10.61.60.17 Destination 54.230.69.141 54.230.69.190 54.230.71.141 54.230.71.172 54.230.71.233 54.240.188.66 54.230.68.41 54.230.68.212 54.230.69.190 54.230.69.141 54.230.71.141 54.230.71.172 [SYN] [SYN] [SYN] [SYN] [SYN] [SYN] [SYN] [SYN] [SYN] [SYN] [SYN] [SYN]
  • 52. Client Retry Behavior, SYN Timeout Browser OS Rotates IPs Time to Rotation Chrome 30.0.1599 Windows 7 Yes 12 Internet Explorer 8 Windows 7 Yes 12 Firefox 25 Windows 7 Yes 20 Safari 5.0.5 Windows 7 Yes 20 Safari 6.0.5 OSX 10.7.5 Yes <1 Firefox 25 OSX 10.7.5 Yes (2) <1 Chrome 32.0.1678 OSX 10.7.5 Yes (2) DNS TTL, or Refresh
  • 53. Resilient Clients attacker service client
  • 54. Resilient Clients
  • 55. Designing for Resilience • • • • • N+1 Failover Resilient Clients Capped Workloads Process Isolation Shuffle Sharding
  • 56. Capped Workloads • Protect Application Layer Capacity • Strive for Sameness • Throttle or Sample Request Workloads
  • 57. Strive for Sameness Application Exhaustion /search.php?expensive-params service attacker
  • 58. Strive for Sameness /search.php?expensive-params attacker service Search_Result_Page_1
  • 59. Capped Workloads AppLayer ~1K to ~10K rps Host/OS ~500K to 5M pps
  • 60. Capped Workloads AppLayer ~1K to ~10K rps Host/OS ~500K to 5M pps Core DAL Auth Logging
  • 61. Capped Workloads AppLayer ~1K to ~10K rps Throttle ~10 to ~100K rps Core DAL Auth Logging Host/OS ~500K to 5M pps
  • 62. Capped Workloads AppLayer ~1K to ~10K rps Throttle ~10 to ~100K rps Host/OS ~500K to 5M pps Core DAL Auth Logging 1,000 samples / sec
  • 63. Designing for Resilience • • • • • N+1 Failover Resilient Clients Capped Workloads Process Isolation Shuffle Sharding
  • 64. Process Isolation • Isolate application components across processes • Let the OS protect critical resources
  • 65. Process Isolation Core DAL Auth Logging
  • 66. Process Isolation Core DAL Auth Logging
  • 67. Designing for Resilience • • • • • N+1 Failover Resilient Clients Capped Workloads Process Isolation Shuffle Sharding
  • 68. Evolution of Resilience client client
  • 69. Evolution of Resilience client client
  • 70. Evolution of Resilience client client
  • 71. Evolution of Resilience client client
  • 72. Evolution of Resilience client client
  • 73. Evolution of Resilience client client
  • 74. Evolution of Resilience client client
  • 75. Evolution of Resilience client client
  • 76. Evolution of Resilience client client
  • 77. N Choose M Isolation • 2 endpoints 2 AZs = 4 permutations
  • 78. N Choose M Isolation • 2 endpoints 2 AZs = 4 permutations • 8 endpoints 2 AZs = 64
  • 79. N Choose M Isolation • 2 endpoints 2 AZs = 4 permutations • 8 endpoints 2 AZs = 64 • 8 endpoints 3 AZs = 512
  • 80. Shuffle Sharding – Amazon Route 53 • Define Availability Lattice • Stripes – Edge Location • Braids – Host Isolation • Assign Endpoints to the Lattice • Virtual Name Servers • Allocate Endpoints to Resources • Hosted Zone Delegate Set
  • 81. Non-Overlapping Delegation Sets ;; QUESTION SECTION: ;gray.internetkitties.com. IN NS ;; QUESTION SECTION: ;orange.internetkitties.org. IN NS ;; ANSWER SECTION: ns-1131.awsdns-13.org. ns-1751.awsdns-26.co.uk. ns-340.awsdns-42.com. ns-952.awsdns-55.net. ;; ANSWER SECTION: ns-1140.awsdns-14.org. ns-1773.awsdns-29.co.uk. ns-290.awsdns-36.com. ns-989.awsdns-59.net.
  • 82. Shuffle Sharding .com .net .co.uk .org
  • 83. Shuffle Sharding .com .net .co.uk .org ns-1773.awsdns-29.co.uk. ns-1140.awsdns-14.org.
  • 84. Shuffle Sharding A B C D .com .net .co.uk .org ns-1773.awsdns-29.co.uk. ns-1140.awsdns-14.org.
  • 85. Shuffle Sharding A .com .net .co.uk .org B gray.internetkitties.com orange.internetkitties.org C D
  • 86. Shuffle Sharding A .com .net .co.uk .org B gray.internetkitties.com orange.internetkitties.org C D
  • 87. Non-Overlapping Delegation Sets ;; QUESTION SECTION: ;gray.internetkitties.com. IN NS ;; QUESTION SECTION: ;orange.internetkitties.org. IN NS ;; ANSWER SECTION: ns-1131.awsdns-13.org. ns-1751.awsdns-26.co.uk. ns-340.awsdns-42.com. ns-952.awsdns-55.net. ;; ANSWER SECTION: ns-1140.awsdns-14.org. ns-1773.awsdns-29.co.uk. ns-290.awsdns-36.com. ns-989.awsdns-59.net.
  • 88. Shuffle Sharding Resilience attacke r .co.uk A B C D client .org A B C D gray.internetkitties.com orange.internetkitties.org
  • 89. Shuffle Sharding Resilience attacke r .co.uk A B C D client .org A B C D gray.internetkitties.com orange.internetkitties.org
  • 90. Shuffle Sharding Toolkit • • • • Define a Lattice of Availability Allocate Service Resources to the Lattice Assign Customers Isolated Resources https://github.com/awslabs/route53-infima
  • 91. Lattice Configuration // Create a 1-D lattice with "AvailabilityZone” as the dimension OneDimensionalLattice<HealthCheckedRecordSet> myServiceLayout = new OneDimensionalLattice<HealthCheckedRecordSet>("AvailabilityZone”);
  • 92. Lattice Configuration // Add endpoints in the us-west-1a Availability zone myServiceLayout.addEndpoint("us-west-1a”, new HealthCheckedRecordSet("192.0.2.1")); myServiceLayout.addEndpoint("us-west-1a”, new HealthCheckedRecordSet("192.0.2.2")); myServiceLayout.addEndpoint("us-west-1a”, new HealthCheckedRecordSet("192.0.2.3")); … // Add endpoints in the us-west-1b Availability zone myServiceLayout.addEndpoint("us-west-1b” new HealthCheckedRecordSet("192.0.2.11")); …
  • 93. Lattice Configuration // Add endpoints in the us-west-1a Availability zone myServiceLayout.addEndpoint("us-west-1a”, new HealthCheckedRecordSet("192.0.2.1")); myServiceLayout.addEndpoint("us-west-1a”, new HealthCheckedRecordSet("192.0.2.2")); myServiceLayout.addEndpoint("us-west-1a”, new HealthCheckedRecordSet("192.0.2.3")); … // Add endpoints in the us-west-1b Availability zone myServiceLayout.addEndpoint("us-west-1b” new HealthCheckedRecordSet("192.0.2.11")); …
  • 94. Shuffle Shard // Create a shuffle sharder SimpleSignatureShuffleSharder shuffleSharder = new SimpleSignatureShuffleSharder(5353L); Lattice shard = shuffleSharder.shuffleShard(myServiceLayout, "v123543234", 1);
  • 95. Shuffle Shard // Create a shuffle sharder SimpleSignatureShuffleSharder shuffleSharder = new SimpleSignatureShuffleSharder(5353L); Lattice shard = shuffleSharder.shuffleShard(myServiceLayout, "v123543234", 1);
  • 96. Shuffle Shard // Create a shuffle sharder SimpleSignatureShuffleSharder shuffleSharder = new SimpleSignatureShuffleSharder(5353L); Lattice shard = shuffleSharder.shuffleShard(myServiceLayout, "v123543234", 1);
  • 97. Vulcanized Lattice // Create a shuffle sharder SimpleSignatureShuffleSharder shuffleSharder = new SimpleSignatureShuffleSharder(5353L); Lattice shard = shuffleSharder.shuffleShard(myServiceLayout, "v123543234", 1); // Create a RubberTree of DNS records Route53RubberTree rubberTree = new Route53RubberTree(”v123543234.video.internetkitties.com", shard); List rrsets = rubberTree.vulcanize();
  • 98. Lattice Shard RRSet [nated@xyz ~]$ dig v123543234.video.internetkitties.com ;; QUESTION SECTION: ; v123543234.video.internetkitties.com. IN A ;; ANSWER SECTION: v123543234.video.internetkitties.com. 60 IN A v123543234.video.internetkitties.com. 60 IN A v123543234.video.internetkitties.com. 60 IN A 192.0.2.12 192.0.1.45 192.0.3.24 us-west-1b us-west-1a us-west-1c
  • 99. Designing for Resilience • • • • • N+1 Failover Resilient Clients Capped Workloads Process Isolation Shuffle Sharding
  • 100. Attack Response
  • 101. Attack Response • Detection • Src-IP Blocking • Engaging Customer Support
  • 102. Attack Response • Detection • Src-IP Blocking • Engaging Customer Support
  • 103. Detect • Traffic Spikes, Drops • CPU Utilization • Network Stats
  • 104. Detect • Use Resilience Patterns to Access Logs • X-Forwarded-For • Sort and Sum
  • 105. X-Forwarded-For • Use a trusted load balancer or proxy
  • 106. X-Forwarded-For • Use a trusted load balancer or proxy • Enable logging
  • 107. X-Forwarded-For • Use a trusted load balancer or proxy • Enable logging – IIS7 • Install ‘IIS Advanced Logging’ • Configure X-Forwarded-For field
  • 108. X-Forwarded-For Enable Logging if($http_x_forwarded_for !='-’) { nginx: log_format main '$http_x_forwarded_for - $remote_user [$time_local] $status ' '"$request" $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$remote_addr"'; } else { log_format main '$remote_addr - $remote_user [$time_local] $status ' '"$request" $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; }
  • 109. X-Forwarded-For • Use a trusted load balancer or proxy • Enable X-Forwarded-For logging
  • 110. Sort & Sum • Used to identify “top talkers” [nated@xyz.com ~]$ grep 'expensive-param' ./access.log | awk '{print $1}' | sort | uniq -c | tail 2 10.54.4.1 3 10.63.34.1 5 10.23.97.212 1182 10.54.0.183
  • 111. Sort & Sum • Used to identify “top talkers” [nated@xyz.com ~]$ grep 'expensive-param' ./access.log | awk '{print $1}' | sort | uniq -c | tail 2 10.54.4.1 3 10.63.34.1 5 10.23.97.212 1182 10.54.0.183
  • 112. Src-IP Blacklisting • • • • Host-Level Firewalling Web-Server Configuration VPC Network ACLs Web Application Firewall
  • 113. Src-IP Blacklisting • • • • Host-Level Firewalling (IPTables) Web-Server Configuration (Nginx / Apache, IIS) VPC Network ACLs Web Application Firewall
  • 114. Src-IP Blacklisting • • • • Host-Level Firewalling Web-Server Configuration VPC Network ACLs Web Application Firewall
  • 115. VPC Network ACLs • Apply to a VPC subnet • Supports DENY rules
  • 116. VPC Network ACLs • Enter each source IP • Set DENY
  • 117. Src-IP Blacklisting • Host-Level Firewalling • VPC Network ACLs • Web Application Firewall
  • 118. Web Application Firewall • • • • Src-IP Blacklist HTTP Headers (X-Forwarded-For) URI-Based Filtering Advanced Throttling
  • 119. Attack Response • Detection • Src-IP Blocking • Engaging Customer Support
  • 120. Engaging Customer Support http://aws.amazon.com/premiumsupport/
  • 121. Summary How can we help? Resilient Design • Scale and Diversity • Route 53 and CloudFront • Business and Enterprise Support • • • • • • Attack Response • • • • Enable X-Forwarded-For Logging Detect, Sum and Sort Src-IP Blacklist Engage Customer Support Availability Lattice Shuffle Sharding N+1 Failover Resilient Clients Capped Workloads Process Isolation
  • 122. Summary How can we help? Resilient Design • Scale and Diversity • Route 53 and CloudFront • Business and Enterprise Support • • • • • • Attack Response • • • • Enable X-Forwarded-For Logging Detect, Sum and Sort Src-IP Blacklist Engage Customer Support Availability Lattice Shuffle Sharding N+1 Failover Resilient Clients Capped Workloads Process Isolation
  • 123. Please give us your feedback on this presentation SEC305 As a thank you, we will select prize winners daily for completed surveys!

×