Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

IX Best Practices by Tay Chee Yong

4,430 views

Published on

IX Best Practices by Tay Chee Yong

Published in: Technology
  • Be the first to comment

IX Best Practices by Tay Chee Yong

  1. 1. IXP Best Practices Tay Chee Yong MyNOG 3 28 November 2013 1
  2. 2. IXP Essentials •  Layer 2 Ethernet network consisting of one or more switches •  Members connects to the network with an assigned IP address •  Only BGP is allowed –  Bi-lateral (BGP between members) –  Multi-lateral (BGP with route servers) 2
  3. 3. IXP Essentials •  Announce own origin and customer routes •  Exchange traffic with all other members to improve traffic gravity and performance –  Members save cost on Internet transit –  Better user experience (reduced latency) •  One port with many peers –  Allows exchange of routes/traffic among all IXP members 3
  4. 4. IXP Benefits •  Keep the local traffic local! –  ISP within the country/region peer with each other –  Doesn’t need to take a long route out and return –  Improved latency and efficiency •  Save money! –  Traffic stays local means save transit bandwidth = save money •  Improve network performance –  Better RTT between end points –  Direct traffic forwarding instead of sub-optimal routing 4
  5. 5. Be responsible! •  IXP operator responsible to ensure infrastructure is stable and secure –  Choice of hardware/software –  Stability of route server daemon –  Security measures –  Competent operational staffs •  Usual BGP best practices still apply to all members •  IXP best practices and etiquettes to be adhered 5
  6. 6. Leaking of IX prefix to Internet •  Announce IXP prefix outside of AS boundary is not a good idea •  Providing free transit for IXP prefix •  Vulnerable to DDOS attacks •  Common reason : redistribute connected to bgp •  Prefix list/route maps to deny IXP prefix announcement 6
  7. 7. Routing control discipline •  Same set of routes should be announced over both transit links and IX port •  Consistent routing policy over different IXP •  Members announcing more specific routes, may result in transit over the IXP •  No Static/Default route! 7
  8. 8. Unwanted protocols towards IXP •  Interior routing protocols : OSPF, IS-IS, EIGRP, RIP -  Generates unwanted broadcast/multicast traffic •  Layer 2 protocols : -  STP, VTP, Proxy Arp •  Network discovery : -  CDP, LLDP, EDP 8
  9. 9. Proxy ARP •  Members acting as a arp relay, potentially very dangerous •  Leading to hijacking of packets destined to other members •  Usual culprits are of Cisco equipment •  IOS : enabled by default •  IOS-XR : disabled by default •  JUNOS : disabled by default #sh arp 219 202.yyy.yyy.yyy 225 202.yyy.yyy.yyy 242 202.yyy.yyy.yyy 316 202.yyy.yyy.yyy 0012.7fxx.xxxx 0012.7fxx.xxxx 0012.7fxx.xxxx 0012.7fxx.xxxx Dynamic Dynamic Dynamic Dynamic 0 0 0 0 15/20 15/20 15/20 15/20 9
  10. 10. Proxy ARP •  Tools to detect members with proxy arp enabled •  Violation logs to be sent to NMS monitoring •  Enhance internal monitoring & operational process •  Follow up , Follow up 10
  11. 11. Looping back an Ethernet Port… •  Loopback towards on an IXP port is never a good idea •  Result : broadcast storm towards all other members •  Cripple the IXP, and disrupting traffic 11
  12. 12. Peering with route servers •  Facilitate implementation of peering arrangement •  Allow new members to join the community easily •  Generally have 2 route servers for redundancy •  Single routing daemon •  Dual routing daemon •  Reduced the number of peering sessions •  Just peer with 2 to get all routes from all members •  Ability to manipulate routing policy via bgp communities 12
  13. 13. Port Security •  MAC address filtering •  Only permit specific IP ethertypes •  IPv4, ARP, IPv6 •  Drop everything else •  Enforce one-mac-address-per-port rule •  No additional devices are permitted •  Prevent noise from any intermediate L2 devices (eg. STP) •  Inform your IXP if you are doing any migration or change of device •  Mac address change 13
  14. 14. Prefix Filtering •  Applied on route servers •  Per neighbor prefix filtering •  Pros •  Prevent unintentional route hijack or route leak by members •  Treat IXP as a normal upstream provider to update prefix list •  Cons •  Accidental of route denial – reduction in traffic •  Solutions : Route update using IRR where possible •  Challenge : Route objects should be updated regularly 14
  15. 15. Configuration Automation •  Fat fingers and human nature at times cause issues in IXP -  Applying incorrect switch configuration -  Forgot to apply port security -  Typo error -  etc •  Reduce errors during provisioning of switch or route servers •  Increase IXP productivity and efficiency •  Standardize configuration across IXP platform 15
  16. 16. Transparent AS •  AS-PATH Transparency : Route servers do not insert its own AS number in the AS-PATH updates to members •  In route servers, well-known BGP attributes (AS-Path, MED, next-hop, communities) are not modified before redistributing to other members. •  Peering sessions appears to be directly between members, but the RS is mediating the session. •  Common problem seen with Cisco routers due to default behavior •  IOS : no bgp enforce-first-as •  IOS XR : bgp enforce-first-as disable 16
  17. 17. Transparent AS •  Non route server setup 10.10.0.0/16 20.20.0.0/16 AS10 Prefix 20.20.0.0/16 AS100 AS-PATH 100 20 AS20 Prefix 10.10.0.0/16 AS-PATH 100 10 17
  18. 18. Transparent AS •  With route server setup 10.10.0.0/16 20.20.0.0/16 IXP A AS 100 AS10 Prefix 20.20.0.0/16 AS20 AS-PATH 20 Prefix 10.10.0.0/16 AS-PATH 10 18
  19. 19. Storm Control •  Broadcast storm into an IXP a major challenge for the operator – beyond their control •  IXP hardware to have better storm control capability or features to counter •  Various hardware vendors has employed certain level of storm control detection and mitigation feature Vendor Mechanism/Capability Cisco Nexus •  Interface level (Threshold : Interface bandwidth) Brocade MLX •  Interface level ACL/rate-limit •  Global Level / VPLS Level (Threshold : # of packets) Extreme •  Interface level ACL/rate-limit •  Global/CPU level (Threshold : # of packets) 19
  20. 20. Summary of Best Practices Members Operator •  Disable unwanted traffic towards IXP •  Do not loop towards IXP •  Do not leak IXP prefix to Internet •  Peering with route servers •  Consistent route announcement •  •  •  •  •  Port Security Prefix Filtering Configuration Automation Transparent AS Storm Control 20
  21. 21. Reference •  AMS-IX •  https://www.ams-ix.net/technical/specifications-descriptions/ config-guide •  Euro-IX •  https://www.euro-ix.net/ixp-bcp 21
  22. 22. chee-yong.tay@ap.equinix.com 22

×