BGP Route Stability

616
-1

Published on

The presentation was delivered at the RIPE 64 Meeting on 16 – 20 April 2012 in Ljubljana by Alexander Asimov, Qrator network engineer at HLL.

Alexander Asimov expressed his concerns about BGP Route Stability and it was suggested that RIPE Atlas may be able to address these issues.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
616
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
29
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

BGP Route Stability

  1. 1. BGP Route Stability Alexander Asimov <aa@highloadlab.com> Highload Lab
  2. 2. Research history• 2009-2010 – study of BGP convergence process: • Imitation testbed (BGPC + PRIME); • BGP convergence time equations; • Experiments with BGP timers;• Since 2010 – BGP policy recovery model;• Since 2011 – BGP route monitor.
  3. 3. Route instability1. Router misconfiguration;2. BGP route loops;3. Link failure.
  4. 4. Default route Default RouteIncomplete route Route loop at last table upstream No prefix atloopback interface
  5. 5. Default route : Stat• Prefixes affected 12514• Vulnerability: could be used for DDoS amplification on AS (~ TTL) • Link exhaustion • Billable bandwidth
  6. 6. DDoS amplifier• Prefixes affected 688• Vulnerability: exploit makes AS work like DDoS amplifier. • DDoS at multiple ASes at the same time with N-time increase of attack magnitude
  7. 7. BGP Route LoopsBuilt-in defense for static loopsDynamic loops!
  8. 8. BGP dynamic loops• Route withdraw;• Changing prepend policy;• Changing LOCAL_PREF.
  9. 9. BGP convergence process• 𝐸𝑇 𝑢𝑝 = 𝜃 𝑑 × 𝐸𝑡 𝑤𝑎𝑖𝑡 , where d – diameter of AS graph;• 𝐸𝑇 𝑑𝑜𝑤𝑛 = 𝜃 𝐷 × 𝐸𝑡 𝑤𝑎𝑖𝑡 , where D – Hamiltonian path of AS graph;Alternatives paths!
  10. 10. Route Withdraw AS2 AS3 default AS-X route200 AS1 AS4 100 AS1 AS2 AS3 AS-X AS2 AS3 AS-X AS3 AS-X AS4 AS-X
  11. 11. Route Withdraw AS2 AS3 W default AS-X route 200 AS1 AS4 100 AS1 AS2 AS3 AS-X AS2 AS3 AS-XLoop AS2 – AS3 AS3 --Packet loss AS1, AS2,AS3 AS4 AS-X
  12. 12. Route Withdraw W AS2 AS3 default AS-X route 200 AS1 AS4 100 AS1 AS2 AS3 AS-X AS2 --Loop AS2 – AS3 AS3 --Packet loss AS1, AS2,AS3 AS4 AS-X
  13. 13. Route Withdraw AS2 AS3 AS-X W A default route200 AS1 AS4 100 AS1 AS2 AS3 AS-X AS2 AS1 AS4 AS-X1 minute problem! AS3 -- ~𝜌 𝑐2𝑝 ∗ 30𝑠 AS4 AS-X
  14. 14. Route FlapsOne route withdraw could causeunstable dynamic loop.Multiple flaps could make your netunreachable.
  15. 15. Prepend Policy AS1200 AS4 AS2 AS-X AS3 AS1 AS-X AS2 AS3 AS-X AS3 AS-X AS4 AS1 AS-X
  16. 16. Prepend Policy AS1 200 A AS4 AS2 AS-X A AS3 AS1 AS2 AS3 AS-X AS2 AS3 AS-Xas-path prepend 5 AS3 AS4 AS1 AS-XLoop AS1-AS2-AS3-AS4 AS4 AS1 AS-X
  17. 17. Prepend Policy AS1 A 200 AS4 AS2 AS-X W A AS3 AS1 AS2 AS3 AS-X AS2 AS3 AS4 AS1 AS-XLoop AS1-AS2-AS3-AS4 AS3 AS4 AS1 AS-X AS4 AS1 AS2 AS3 AS-X
  18. 18. Prepend Policy AS1 200 A AS4 AS2 AS-X A AS3 AS1 AS-X(5) AS2 AS3 AS4 AS1 AS-XMore then 1 minute problem AS3 AS-X(5) AS4 AS1 AS2 AS3 AS-X
  19. 19. Core of the problemTraffic flow engineering looks like…
  20. 20. Automated traffic flow engineering• No opportunity for traffic flow prediction;• No use of BGP convergence process;• Could make networks partially unavailable.
  21. 21. Route flaps : Stat• Prefixes affected 1720• Vulnerability: • Packet loss; • BGP message noise.
  22. 22. BGP Route Policy Loops 200 AS2 AS3 A 200 AS-X200 AS1 AS4 A 200 AS1 -- AS2 -- AS3 AS-X AS4 AS-X
  23. 23. BGP Route Policy Loops 200 AS2 A AS3 200 AS-X A200 AS1 AS4 200 AS1 -- AS2 AS3 AS-X AS3 AS4 AS-X AS4 AS-X
  24. 24. BGP Route Policy Loops 200 AS2 A AS3 200 AS-X A200 AS1 AS4 200 AS1 AS2 AS3 AS5 AS-X AS2 AS3 AS4 AS5 AS-X AS3 AS4 AS5 AS-X AS4 AS5 AS-X
  25. 25. BGP Route Policy Loops 200 AS2 AS3 200 AS-X A 200 AS1 A AS4 200 AS1 AS2 AS3 AS4 AS5 AS-X AS2 AS3 AS4 AS5 AS-XLoop AS1-AS2-AS3-AS4 AS3 AS4 AS5 AS-X AS4 AS1 AS2 AS3 AS5 AS-X
  26. 26. BGP Route Policy Loops 200 AS2 AS3 200 AS-X A200 AS1 A AS4 200 AS1 AS2 AS3 AS4 AS5 AS-X AS2 AS3 AS4 AS5 AS-X AS3 AS5 AS-X Again! AS4 AS5 AS-X
  27. 27. BGP Route Policy Loops : Stat• Prefixes affected 149• Vulnerability: • Packet loss • BGP message noise
  28. 28. Breaking down1. Route flap2. Change prepend policy AS-X AS-X3. AS_PATH poisoning AS-X AS2 AS-X Makes AS2 ignore your route
  29. 29. Breaking down 1. Route flap 2. Change prepend policy AS-X AS-X 3. AS_PATH poisoning AS-X AS2 AS-X Makes AS2 ignore your routeHow not to fall into another loop?
  30. 30. One “killer” featureAutonomous Systems ReverseMap:1. Traffic flow engineering;2. AS architecture design;3. BGP loop prediction.
  31. 31. AS Mapping projects• Physical links discoveryNo use of route policy;• Macro mapShows links that are used by somebody,not by your AS;• Route policy dataOutdated, incomplete.• Model abstraction too far from reality
  32. 32. BGP Policy recovery project: Current state• Mathematical model for BGP route policy recovery;• Mathematical model for AS graph;• New active verification methods;• Working prototype;• Negotiations with RIPE ATLAS.
  33. 33. They had problems… AS174 AS3356 AS7018 AS6939 AS701AS3549 AS209 AS4323 AS1239 AS12389AS2848 AS3257 AS6461 AS2914 AS8468AS23148 AS8447 AS20485 AS6830 AS8220AS8928 AS3303 AS4589 AS42708 AS6453AS6730 AS31130 AS3491 AS3320 AS8218 AS286 AS702 AS3561 AS20764 AS31323AS20632 AS4766 AS680 AS29686 AS5089AS10026 AS12350 AS2516 AS3786 AS12741AS7575 AS1916 AS2273 AS9498 AS1785 And more then 4k other ASes!
  34. 34. ExamplesAS174• increases DDoS 17 times• Default route: 25 prefixes affectedAS3356-AS3549• increases DDoS 8 times• Route flap: 12 prefixes affected• Default route: 86 prefixes affected
  35. 35. Errors grouped by RR ARINAPNIC Errors in RR % Errors in ASes % RIPE 0 20 40 60
  36. 36. Duration90,00%80,00%70,00%60,00% Flap50,00% Route loop40,00% Default route30,00% DDoS amplifier20,00%10,00% 0,00% <1h <2h <4h <8h
  37. 37. Trends16000140001200010000 8000 6000 Found Errors 4000 Fixed Errors 2000 0
  38. 38. Robin HoodHello.During our research project we have detected IP address “*.*.*.*" inAS***, which multiplies ICMP packets. 11 times is a mean value.It is dangerous vulnerability because this IP could be used by attackers toN-times increase DoS attack.This could be harmful for your own network infrastructure and also breakdown attacked network.Could you tell me for what purpose your router was configured by suchway?HelloThis is assigned to one of our customers. We cannot see how thereequipment is configured. If you believe this is in violation of ourAcceptable use policy you can email abuse@***.com to report that.
  39. 39. In the core of the Internet BGP 2.2.2.0/24 4 second loop = DDoS*4 ASX2 Default ASX1 route
  40. 40. In the core of the Internet Client ASes BGP 2.2.2.0/24Max = 350*DDoS > 50 amplifiers 4 second loopMean = 10*DDoS = DDoS*4 ASX2 Default ASX1 route
  41. 41. In the core of the Internet Client ASes BGP 2.2.2.0/24Max = 350*DDoS > 50 amplifiers 4 second loopMean = 10*DDoS = DDoS*4 ASX2 Default ASX1 route dst: aplifiers src: 2.2.2.1
  42. 42. In the core of the Internet Client ASes BGP 2.2.2.0/24Max = 350*DDoS > 50 amplifiers 4 second loopMean = 10*DDoS = DDoS*4 ASX2 Default ASX1 route dst: aplifiers src: 2.2.2.1 Result = 40*DDoS
  43. 43. Results• The problem in client network is your problem;• The problem in your upstream network is your problem too;• Unstable prefix is invisible from origin AS;• BGP balancing without prediction could make network unavailable;• We have monitor. Community needs some trusted mechanism for notification.
  44. 44. Thank you for listening! Questions?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×