Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Cyber-security

499 views

Published on

A Presentation in Tokopedia Tech A Break about some big DDos Attacks we saw

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Cyber-security

  1. 1. fighting the cyber threats Qasim Zaidi
  2. 2. Text We were DDos’ed we must be doing something right !
  3. 3. Text Denial of Service Legitimate users are denied service
  4. 4. Types Volumetric (UDP Floods) State Exhaustion (TCP Syn Attacks) Application Layer Attacks (HTTP, DNS query flood)
  5. 5. Application 15% State Exhaustion 20% Volumetric 65%
  6. 6. Reflection Attacks Do not directly attack the Target. Forge Reply to Address Send request to normal servers Trick them to reply to the Target Makes it distributed and harder to deal with.
  7. 7. Amplification A new class of reflection
  8. 8. Amplification attacks Because a small question can have a big answer. Why? How?
  9. 9. ; <<>> DiG 9.8.3-P1 <<>> ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 64739 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0 ;; QUESTION SECTION: ;dig. IN ANY ;; AUTHORITY SECTION: . 73193 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2016041601 1800 900 604800 86400 ;; Query time: 80 msec ;; SERVER: 8.8.8.8#53(8.8.8.8) ;; WHEN: Sun Apr 17 10:19:42 2016 ;; MSG SIZE rcvd: 96 ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39944 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 6, ADDITIONAL: 10 ;; QUESTION SECTION: ;yahoo.com. IN A ;; ANSWER SECTION: yahoo.com. 1762 IN A 98.139.183.24 yahoo.com. 1762 IN A 206.190.36.45 yahoo.com. 1762 IN A 98.138.253.109 ;; AUTHORITY SECTION: yahoo.com. 17439 IN NS ns3.yahoo.com. yahoo.com. 17439 IN NS ns5.yahoo.com. yahoo.com. 17439 IN NS ns2.yahoo.com. yahoo.com. 17439 IN NS ns1.yahoo.com. yahoo.com. 17439 IN NS ns6.yahoo.com. yahoo.com. 17439 IN NS ns4.yahoo.com. ;; ADDITIONAL SECTION: ns1.yahoo.com. 1197500 IN A 68.180.131.16 ns1.yahoo.com. 66008 IN AAAA 2001:4998:130::1001 ns2.yahoo.com. 1197500 IN A 68.142.255.16 ns2.yahoo.com. 85955 IN AAAA 2001:4998:140::1002 ns3.yahoo.com. 1197585 IN A 203.84.221.53 ns3.yahoo.com. 73296 IN AAAA 2406:8600:b8:fe03::1003 ns4.yahoo.com. 1198687 IN A 98.138.11.157 ns5.yahoo.com. 1197585 IN A 119.160.247.124 ns6.yahoo.com. 160785 IN A 121.101.144.139 ns6.yahoo.com. 1762 IN AAAA 2406:2000:108:4::1006 ;; Query time: 27 msec ;; SERVER: 8.8.8.8#53(8.8.8.8) ;; WHEN: Sun Apr 17 10:19:42 2016 ;; MSG SIZE rcvd: 391 dig ANY yahoo.com @8.8.8.8 (64 bytes) A (391 bytes) 6x amplification
  10. 10. The D in DDos
  11. 11. SSDP Simple Service Discovery Protocol (UPnP) Example: Used to discover printers on your network SSDP Discovery - HTTP over UDP sent to a multicast address.
  12. 12. 1. Recruiting Zombies
  13. 13. 2. Flooding the victim
  14. 14. First Attack Happened at 6 PM on a Monday Website seemed slow SSH to servers even slower
  15. 15. public ips private ips
  16. 16. dmesg output UDP: bad checksum. From 182.215.214.137:1900 to 182.253.224.184:80 ulen 318 UDP: bad checksum. From 182.215.214.137:1900 to 182.253.224.184:80 ulen 318 UDP: bad checksum. From 190.129.30.190:1900 to 182.253.224.184:80 ulen 347 UDP: bad checksum. From 190.129.169.3:1900 to 182.253.224.184:80 ulen 291 UDP: bad checksum. From 190.129.169.3:1900 to 182.253.224.184:80 ulen 291 UDP: bad checksum. From 200.87.245.44:1900 to 182.253.224.184:80 ulen 311 UDP: bad checksum. From 190.129.81.203:1900 to 182.253.224.184:80 ulen 281 UDP: bad checksum. From 190.129.6.33:1900 to 182.253.224.184:80 ulen 301 UDP: bad checksum. From 73.201.211.248:1900 to 182.253.224.184:80 ulen 253 UDP: bad checksum. From 190.129.199.12:1900 to 182.253.224.184:80 ulen 347 UDP: bad checksum. From 200.87.155.100:1900 to 182.253.224.184:80 ulen 285 UDP: bad checksum. From 182.215.214.137:1900 to 182.253.224.184:80 ulen 300 UDP: bad checksum. From 190.129.182.57:1900 to 182.253.224.184:80 ulen 280 UDP: bad checksum. From 190.129.165.180:1900 to 182.253.224.184:80 ulen 237 UDP: bad checksum. From 182.215.214.137:1900 to 182.253.224.184:80 ulen 306 UDP: bad checksum. From 190.129.81.26:1900 to 182.253.224.184:80 ulen 283 UDP: bad checksum. From 182.215.214.137:1900 to 182.253.224.184:80 ulen 300 UDP: bad checksum. From 172.97.240.102:1900 to 182.253.224.184:80 ulen 343 UDP: bad checksum. From 190.129.195.29:1900 to 182.253.224.184:80 ulen 246 UDP: bad checksum. From 172.97.240.102:1900 to 182.253.224.184:80 ulen 237 UDP: bad checksum. From 190.129.165.171:1900 to 182.253.224.184:80 ulen 301 UDP: bad checksum. From 172.97.240.102:1900 to 182.253.224.184:80 ulen 237 UDP: bad checksum. From 182.215.214.137:1900 to 182.253.224.184:80 ulen 302 UDP: bad checksum. From 190.129.30.176:1900 to 182.253.224.184:80 ulen 289
  17. 17. First Response sudo iptables -A INPUT -p udp -sport 1900 -j DROP Drops all incoming packets with source Port 1900 saves some resources, but remember that packets still have to be processed by NIC card, and the pipe is still clogged. dmesg output goes away, but recovery isn’t complete.
  18. 18. GEO IP Lookup
  19. 19. But we knew we haven’t yet
  20. 20. During Q4 (2015), repeat DDoS attacks were the norm, with an average of 24 attacks per targeted customer in Q4. Three targets were subject to more than 100 attacks each and one customer suffered 188 attacks – an average of more than two per day. Source: Akamai Attacker’s persist, especially if they don’t get what they wanted.
  21. 21. Attack 2 The very next day, at 2 PM Same attack vector, but more distributed Lot’s of Indonesian IP addresses Attacked all of our public IP’s, not DNS based.
  22. 22. identify netstat dmesg iptraf netstat -i Kernel Interface table Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg em1 1500 0 266063410705 0 327198 0 269121217381 0 2 em2 1500 0 19266620548 0 197 0 20700650229 0 0 0 lo 16436 0 79744956 0 0 0 79744956 0 0 0 LRU
  23. 23. iptables/netfilter/tuning kernel parameters tuning NIC TX/RX Buffer tuning sudo iptables -A INPUT -p udp --sport 1900 -j DROP netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n iptables -I INPUT -s <ipaddress> —j DROP tcpkill / cutter synproxy (against syn flood attacks) sudo ethtool -g em1 Ring parameters for em1: Pre-set maximums: RX: 2047 RX Mini: 0 RX Jumbo: 0 TX: 511 Current hardware settings: RX: 200 RX Mini: 0 RX Jumbo: 0 TX: 511
  24. 24. Know who to call @ ISP
  25. 25. tc / firehol Ensure you can ssh to the server when your network is congested Limit bandwidth class ssh commit 2Mbit server ssh client ssh class rsync commit 2Mbit max 10Mbit server rsync client rsync
  26. 26. private net Minimize Attack Surface
  27. 27. private net under attack normal whois tokopedia.com 1 2 2 Use a WAF / hide origin
  28. 28. –Johnny Appleseed “Type a quote here.”

×