IX Best Practices by Tay Chee Yong

3,191 views
2,957 views

Published on

IX Best Practices by Tay Chee Yong

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,191
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
85
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

IX Best Practices by Tay Chee Yong

  1. 1. IXP Best Practices Tay Chee Yong MyNOG 3 28 November 2013 1
  2. 2. IXP Essentials •  Layer 2 Ethernet network consisting of one or more switches •  Members connects to the network with an assigned IP address •  Only BGP is allowed –  Bi-lateral (BGP between members) –  Multi-lateral (BGP with route servers) 2
  3. 3. IXP Essentials •  Announce own origin and customer routes •  Exchange traffic with all other members to improve traffic gravity and performance –  Members save cost on Internet transit –  Better user experience (reduced latency) •  One port with many peers –  Allows exchange of routes/traffic among all IXP members 3
  4. 4. IXP Benefits •  Keep the local traffic local! –  ISP within the country/region peer with each other –  Doesn’t need to take a long route out and return –  Improved latency and efficiency •  Save money! –  Traffic stays local means save transit bandwidth = save money •  Improve network performance –  Better RTT between end points –  Direct traffic forwarding instead of sub-optimal routing 4
  5. 5. Be responsible! •  IXP operator responsible to ensure infrastructure is stable and secure –  Choice of hardware/software –  Stability of route server daemon –  Security measures –  Competent operational staffs •  Usual BGP best practices still apply to all members •  IXP best practices and etiquettes to be adhered 5
  6. 6. Leaking of IX prefix to Internet •  Announce IXP prefix outside of AS boundary is not a good idea •  Providing free transit for IXP prefix •  Vulnerable to DDOS attacks •  Common reason : redistribute connected to bgp •  Prefix list/route maps to deny IXP prefix announcement 6
  7. 7. Routing control discipline •  Same set of routes should be announced over both transit links and IX port •  Consistent routing policy over different IXP •  Members announcing more specific routes, may result in transit over the IXP •  No Static/Default route! 7
  8. 8. Unwanted protocols towards IXP •  Interior routing protocols : OSPF, IS-IS, EIGRP, RIP -  Generates unwanted broadcast/multicast traffic •  Layer 2 protocols : -  STP, VTP, Proxy Arp •  Network discovery : -  CDP, LLDP, EDP 8
  9. 9. Proxy ARP •  Members acting as a arp relay, potentially very dangerous •  Leading to hijacking of packets destined to other members •  Usual culprits are of Cisco equipment •  IOS : enabled by default •  IOS-XR : disabled by default •  JUNOS : disabled by default #sh arp 219 202.yyy.yyy.yyy 225 202.yyy.yyy.yyy 242 202.yyy.yyy.yyy 316 202.yyy.yyy.yyy 0012.7fxx.xxxx 0012.7fxx.xxxx 0012.7fxx.xxxx 0012.7fxx.xxxx Dynamic Dynamic Dynamic Dynamic 0 0 0 0 15/20 15/20 15/20 15/20 9
  10. 10. Proxy ARP •  Tools to detect members with proxy arp enabled •  Violation logs to be sent to NMS monitoring •  Enhance internal monitoring & operational process •  Follow up , Follow up 10
  11. 11. Looping back an Ethernet Port… •  Loopback towards on an IXP port is never a good idea •  Result : broadcast storm towards all other members •  Cripple the IXP, and disrupting traffic 11
  12. 12. Peering with route servers •  Facilitate implementation of peering arrangement •  Allow new members to join the community easily •  Generally have 2 route servers for redundancy •  Single routing daemon •  Dual routing daemon •  Reduced the number of peering sessions •  Just peer with 2 to get all routes from all members •  Ability to manipulate routing policy via bgp communities 12
  13. 13. Port Security •  MAC address filtering •  Only permit specific IP ethertypes •  IPv4, ARP, IPv6 •  Drop everything else •  Enforce one-mac-address-per-port rule •  No additional devices are permitted •  Prevent noise from any intermediate L2 devices (eg. STP) •  Inform your IXP if you are doing any migration or change of device •  Mac address change 13
  14. 14. Prefix Filtering •  Applied on route servers •  Per neighbor prefix filtering •  Pros •  Prevent unintentional route hijack or route leak by members •  Treat IXP as a normal upstream provider to update prefix list •  Cons •  Accidental of route denial – reduction in traffic •  Solutions : Route update using IRR where possible •  Challenge : Route objects should be updated regularly 14
  15. 15. Configuration Automation •  Fat fingers and human nature at times cause issues in IXP -  Applying incorrect switch configuration -  Forgot to apply port security -  Typo error -  etc •  Reduce errors during provisioning of switch or route servers •  Increase IXP productivity and efficiency •  Standardize configuration across IXP platform 15
  16. 16. Transparent AS •  AS-PATH Transparency : Route servers do not insert its own AS number in the AS-PATH updates to members •  In route servers, well-known BGP attributes (AS-Path, MED, next-hop, communities) are not modified before redistributing to other members. •  Peering sessions appears to be directly between members, but the RS is mediating the session. •  Common problem seen with Cisco routers due to default behavior •  IOS : no bgp enforce-first-as •  IOS XR : bgp enforce-first-as disable 16
  17. 17. Transparent AS •  Non route server setup 10.10.0.0/16 20.20.0.0/16 AS10 Prefix 20.20.0.0/16 AS100 AS-PATH 100 20 AS20 Prefix 10.10.0.0/16 AS-PATH 100 10 17
  18. 18. Transparent AS •  With route server setup 10.10.0.0/16 20.20.0.0/16 IXP A AS 100 AS10 Prefix 20.20.0.0/16 AS20 AS-PATH 20 Prefix 10.10.0.0/16 AS-PATH 10 18
  19. 19. Storm Control •  Broadcast storm into an IXP a major challenge for the operator – beyond their control •  IXP hardware to have better storm control capability or features to counter •  Various hardware vendors has employed certain level of storm control detection and mitigation feature Vendor Mechanism/Capability Cisco Nexus •  Interface level (Threshold : Interface bandwidth) Brocade MLX •  Interface level ACL/rate-limit •  Global Level / VPLS Level (Threshold : # of packets) Extreme •  Interface level ACL/rate-limit •  Global/CPU level (Threshold : # of packets) 19
  20. 20. Summary of Best Practices Members Operator •  Disable unwanted traffic towards IXP •  Do not loop towards IXP •  Do not leak IXP prefix to Internet •  Peering with route servers •  Consistent route announcement •  •  •  •  •  Port Security Prefix Filtering Configuration Automation Transparent AS Storm Control 20
  21. 21. Reference •  AMS-IX •  https://www.ams-ix.net/technical/specifications-descriptions/ config-guide •  Euro-IX •  https://www.euro-ix.net/ixp-bcp 21
  22. 22. chee-yong.tay@ap.equinix.com 22

×