Confidential – © 2017 Equinix Inc. Equinix.com 1
APRICOT 2017
Unknown Unicast Traffic
and Ping Pollers
Lim Kian Sim
Presented on March 2nd 2017
Confidential – © 2017 Equinix Inc. Equinix.com 2
2
Content
vUnknown Unicast Traffic
Ø What is unknown unicast traffic?
Ø Where does it happen?
Ø When does it happen?
Ø What is the impact caused by unknown unicast traffic?
Ø Unknown unicast traffic problem
vPing Pollers
Ø What is ping poller?
Ø Why ping poller?
Ø How does it helps to stop unknown unicast?
Ø Traditional ping poller
Ø Enhanced ping poller with SHA/SPA
Confidential – © 2017 Equinix Inc. Equinix.com 3
3
Unknown Unicast Traffic
v What is unknown Unicast traffic?
Ø Unknown unicast is a unicast traffic flooded on LAN switch
when the destination mac-address is not present in the cam-
table.
v Where does it happen?
Ø L2 broadcast domain (e.g. flat layer-2, VPLS, VxLAN)
v When/where does it happen?
Ø Busy port went down.
Ø Silent host in switched network.
v What is the impact caused by unknown unicast traffic?
Ø LAN switch performance (CPU, buffers,…).
Ø Bandwidth utilization.
Ø End host performance (CPU, buffers,…).
Source:
Source: Packetlife.net
Confidential – © 2017 Equinix Inc. Equinix.com 4
4
How does it happen? Example
1. All peers are exchanging traffic.
2. Peer-A to Peer-B traffic flow through IX switch path.
3. Peer-B (Silent Peer) to Peer-A traffic flow through IP Transit
path.
4. Since Peer-B is not sending traffic via IX switch, the mac-
address for Peer-B in IX Switch is going to be expired (default
300 seconds).
5. In Peer-A router, Peer-B’s ARP entry is still present (Cisco
default ARP timeout 4 hours) and it keeps sending traffic to
next-hop Peer-B router address without ARP query.
6. When the traffic is received by IX Switch, as the destination
mac-address does not exist in CAM table, the switch will flood
the frame to all ports (except receiving port) out.
7. As a result, Peer-C is receiving the traffic sending from Peer-A
to Peer-B.
8. Another Silent Peer scenario can be found as below:
A B
C
A to B
B to A
(4) Peer-B mac-
address is aging out,
removed from CAM
table.
(5) My ARP for Peer-B
not expired yet, I will
keep sending the
traffic.
(without ARP query)
Why am I receiving
the packets for Peer-
B?
(6) I will flood all
ports as I do not
know Peer-B mac-
address
A B
C
A to B
B to A
.100
.101
.102
.200
Why am I receiving
the packets for
.101?
(3) Silent
Peer
(8) Silent
Peer to SW1
Transit
IX
IX
SW1
SW2
Confidential – © 2017 Equinix Inc. Equinix.com 5
5
Ping Poller
v What is ping poller?
Ø Light weight Linux machine installed in IX network.
v Why ping poller?
Ø To mitigate unknown unicast traffic from flooding
through IX network.
v How does it help to stop unknown unicast traffic?
Ø “Keep-alive” traffic refreshes switch cam-table.
v Disadvantage?
Ø Requires one ping poller per switch.
Confidential – © 2017 Equinix Inc. Equinix.com 6
6
Ping Poller with SHA/SPA
v What is SHA/SPA?
Ø Fields in ARP packet header.
Ø SHA (Source hardware address).
Ø SPA (Source protocol address).
Ø Ping poller to send this modified ARP request to
each peers.
v Why ping poller with SHA/SPA?
Ø To mitigate unknown unicast traffic from flooding
with lesser machine implementation.
v How does it help to stop unknown unicast traffic?
Ø Ping poller send ARP request with modified
SHA/SPA to all peers.
Ø Switches learn source MAC from peer’s ARP reply.
Ø MAC address is propagated to all switches through
this unknown unicast flooding.
Ø ARP reply is discard by all other peers.
Confidential – © 2017 Equinix Inc. Equinix.com 7
7
How does it happen?
Host SA
Source MAC
DA
Destination MAC
SHA
Source Hardware Address
SPA
Source IP Address
THA
Target Hardware Address
TPA
Target IP Address
B
P P all ‘f’ U U all ‘0’ B
ARP Header
P
SWITCH_C
SWITCH_A SWITCH_B
A B
CPing Poller send ARP request with
modified SHA/SPA to target
Peer-B
Switch A Switch B Switch C
A
C
P
A
C
P
A
C
P
MAC Table
Confidential – © 2017 Equinix Inc. Equinix.com 8
8
How does it happen?
Host SA
Source MAC
DA
Destination MAC
SHA
Source Hardware Address
SPA
Source IP Address
THA
Target Hardware Address
TPA
Target IP Address
B B U B B U U
P
ARP Header
1. Peer-B send ARP reply back to
the modified SHA/SPA.
2. Switch B and other switches
does not have information of
the modified SHA/SPA, hence
flooded to the network.
3. As a result, MAC address of
Peer-B is propagated
through the network
P
SWITCH_C
SWITCH_A SWITCH_B
A B
C
Switch A Switch B Switch C
A
C
P
B
A
C
P
B
A
C
P
B
MAC Table
Switch A Switch B Switch C
A
C
P
B
A
C
P
B
A
C
P
B
Confidential – © 2017 Equinix Inc. Equinix.com 9
9
Sample
P
#!/bin/bash
PEERING_SUBNET="100.100.100.0/24"
echo "1" > /proc/sys/net/ipv4/ip_nonlocal_bind
SOURCE_EIE_SWITCH="SWITCH_A"
SOURCE_MAC_ADDRESS="11:11:11:11:11:11“
SOURCE_IP_ADDRESS="100.100.100.101"
arp-scan --arpsha=$SOURCE_MAC_ADDRESS --interface=eth1 --arpspa=$SOURCE_IP_ADDRESS $PEERING_SUBNET > /dev/null
SOURCE_EIE_SWITCH="SWITCH_B"
SOURCE_MAC_ADDRESS="22:22:22:22:22:22"
SOURCE_IP_ADDRESS="100.100.100.102”
arp-scan --arpsha=$SOURCE_MAC_ADDRESS --interface=eth1 --arpspa=$SOURCE_IP_ADDRESS $PEERING_SUBNET > /dev/null
SOURCE_EIE_SWITCH="SWITCH_C"
SOURCE_MAC_ADDRESS="33:33:33:33:33:33"
SOURCE_IP_ADDRESS="100.100.100.103"
arp-scan --arpsha=$SOURCE_MAC_ADDRESS --interface=eth1 --arpspa=$SOURCE_IP_ADDRESS $PEERING_SUBNET > /dev/null
SWITCH_C
SWITCH_A SWITCH_B
A B
C
MAC=22:22:22:22:22:22
IP=100.100.100.102
MAC=11:11:11:11:11:11
IP=100.100.100.101
MAC=33:33:33:33:33:33
IP=100.100.100.103
$ arp-scan --arpsha=11:11:11:11:11:11 --interface=eth1 --arpspa=100.100.100.101 100.100.100.0/24
Command to send Spoof ARP
Spoof Source
Mac Address
Spoof Source
IP Address
IX
Subnet
(Sourcec: https://github.com/royhills/arp-scan)
P
P
Existing
Ping
Pollers
Confidential – © 2017 Equinix Inc. Equinix.com 10
10
Ping Poller Versus
v Traditional ping poller
Ø Use ICMP ping sweep to all destination host in subnet.
Ø IX switch learn peer’s MAC address from ICMP reply.
Ø Requires one ping poller per switch.
Ø Simpler & less scalable in large network.
Ø Multiple machines required.
v Enhanced ping poller with SHA/SPA deployment
Ø Use ARP request packets with spoof SPA/SHA in ARP
packet.
Ø IX switch learn peer’s MAC address from ARP reply.
Ø Requires one ping poller per metro.
Ø More complex but scalable.
Ø Minimum one machine is required.
Ping
Poller
#3
SWITCH_C
SWITCH_A SWITCH_B
Ping
Poller
#1
Ping
Poller
#2
Ping
Poller
SWITCH_C
SWITCH_A SWITCH_B
Confidential – © 2017 Equinix Inc. Equinix.com 11
11
Thank You

Unknown Unicast Traffic and Ping Pollers

  • 1.
    Confidential – ©2017 Equinix Inc. Equinix.com 1 APRICOT 2017 Unknown Unicast Traffic and Ping Pollers Lim Kian Sim Presented on March 2nd 2017
  • 2.
    Confidential – ©2017 Equinix Inc. Equinix.com 2 2 Content vUnknown Unicast Traffic Ø What is unknown unicast traffic? Ø Where does it happen? Ø When does it happen? Ø What is the impact caused by unknown unicast traffic? Ø Unknown unicast traffic problem vPing Pollers Ø What is ping poller? Ø Why ping poller? Ø How does it helps to stop unknown unicast? Ø Traditional ping poller Ø Enhanced ping poller with SHA/SPA
  • 3.
    Confidential – ©2017 Equinix Inc. Equinix.com 3 3 Unknown Unicast Traffic v What is unknown Unicast traffic? Ø Unknown unicast is a unicast traffic flooded on LAN switch when the destination mac-address is not present in the cam- table. v Where does it happen? Ø L2 broadcast domain (e.g. flat layer-2, VPLS, VxLAN) v When/where does it happen? Ø Busy port went down. Ø Silent host in switched network. v What is the impact caused by unknown unicast traffic? Ø LAN switch performance (CPU, buffers,…). Ø Bandwidth utilization. Ø End host performance (CPU, buffers,…). Source: Source: Packetlife.net
  • 4.
    Confidential – ©2017 Equinix Inc. Equinix.com 4 4 How does it happen? Example 1. All peers are exchanging traffic. 2. Peer-A to Peer-B traffic flow through IX switch path. 3. Peer-B (Silent Peer) to Peer-A traffic flow through IP Transit path. 4. Since Peer-B is not sending traffic via IX switch, the mac- address for Peer-B in IX Switch is going to be expired (default 300 seconds). 5. In Peer-A router, Peer-B’s ARP entry is still present (Cisco default ARP timeout 4 hours) and it keeps sending traffic to next-hop Peer-B router address without ARP query. 6. When the traffic is received by IX Switch, as the destination mac-address does not exist in CAM table, the switch will flood the frame to all ports (except receiving port) out. 7. As a result, Peer-C is receiving the traffic sending from Peer-A to Peer-B. 8. Another Silent Peer scenario can be found as below: A B C A to B B to A (4) Peer-B mac- address is aging out, removed from CAM table. (5) My ARP for Peer-B not expired yet, I will keep sending the traffic. (without ARP query) Why am I receiving the packets for Peer- B? (6) I will flood all ports as I do not know Peer-B mac- address A B C A to B B to A .100 .101 .102 .200 Why am I receiving the packets for .101? (3) Silent Peer (8) Silent Peer to SW1 Transit IX IX SW1 SW2
  • 5.
    Confidential – ©2017 Equinix Inc. Equinix.com 5 5 Ping Poller v What is ping poller? Ø Light weight Linux machine installed in IX network. v Why ping poller? Ø To mitigate unknown unicast traffic from flooding through IX network. v How does it help to stop unknown unicast traffic? Ø “Keep-alive” traffic refreshes switch cam-table. v Disadvantage? Ø Requires one ping poller per switch.
  • 6.
    Confidential – ©2017 Equinix Inc. Equinix.com 6 6 Ping Poller with SHA/SPA v What is SHA/SPA? Ø Fields in ARP packet header. Ø SHA (Source hardware address). Ø SPA (Source protocol address). Ø Ping poller to send this modified ARP request to each peers. v Why ping poller with SHA/SPA? Ø To mitigate unknown unicast traffic from flooding with lesser machine implementation. v How does it help to stop unknown unicast traffic? Ø Ping poller send ARP request with modified SHA/SPA to all peers. Ø Switches learn source MAC from peer’s ARP reply. Ø MAC address is propagated to all switches through this unknown unicast flooding. Ø ARP reply is discard by all other peers.
  • 7.
    Confidential – ©2017 Equinix Inc. Equinix.com 7 7 How does it happen? Host SA Source MAC DA Destination MAC SHA Source Hardware Address SPA Source IP Address THA Target Hardware Address TPA Target IP Address B P P all ‘f’ U U all ‘0’ B ARP Header P SWITCH_C SWITCH_A SWITCH_B A B CPing Poller send ARP request with modified SHA/SPA to target Peer-B Switch A Switch B Switch C A C P A C P A C P MAC Table
  • 8.
    Confidential – ©2017 Equinix Inc. Equinix.com 8 8 How does it happen? Host SA Source MAC DA Destination MAC SHA Source Hardware Address SPA Source IP Address THA Target Hardware Address TPA Target IP Address B B U B B U U P ARP Header 1. Peer-B send ARP reply back to the modified SHA/SPA. 2. Switch B and other switches does not have information of the modified SHA/SPA, hence flooded to the network. 3. As a result, MAC address of Peer-B is propagated through the network P SWITCH_C SWITCH_A SWITCH_B A B C Switch A Switch B Switch C A C P B A C P B A C P B MAC Table Switch A Switch B Switch C A C P B A C P B A C P B
  • 9.
    Confidential – ©2017 Equinix Inc. Equinix.com 9 9 Sample P #!/bin/bash PEERING_SUBNET="100.100.100.0/24" echo "1" > /proc/sys/net/ipv4/ip_nonlocal_bind SOURCE_EIE_SWITCH="SWITCH_A" SOURCE_MAC_ADDRESS="11:11:11:11:11:11“ SOURCE_IP_ADDRESS="100.100.100.101" arp-scan --arpsha=$SOURCE_MAC_ADDRESS --interface=eth1 --arpspa=$SOURCE_IP_ADDRESS $PEERING_SUBNET > /dev/null SOURCE_EIE_SWITCH="SWITCH_B" SOURCE_MAC_ADDRESS="22:22:22:22:22:22" SOURCE_IP_ADDRESS="100.100.100.102” arp-scan --arpsha=$SOURCE_MAC_ADDRESS --interface=eth1 --arpspa=$SOURCE_IP_ADDRESS $PEERING_SUBNET > /dev/null SOURCE_EIE_SWITCH="SWITCH_C" SOURCE_MAC_ADDRESS="33:33:33:33:33:33" SOURCE_IP_ADDRESS="100.100.100.103" arp-scan --arpsha=$SOURCE_MAC_ADDRESS --interface=eth1 --arpspa=$SOURCE_IP_ADDRESS $PEERING_SUBNET > /dev/null SWITCH_C SWITCH_A SWITCH_B A B C MAC=22:22:22:22:22:22 IP=100.100.100.102 MAC=11:11:11:11:11:11 IP=100.100.100.101 MAC=33:33:33:33:33:33 IP=100.100.100.103 $ arp-scan --arpsha=11:11:11:11:11:11 --interface=eth1 --arpspa=100.100.100.101 100.100.100.0/24 Command to send Spoof ARP Spoof Source Mac Address Spoof Source IP Address IX Subnet (Sourcec: https://github.com/royhills/arp-scan) P P Existing Ping Pollers
  • 10.
    Confidential – ©2017 Equinix Inc. Equinix.com 10 10 Ping Poller Versus v Traditional ping poller Ø Use ICMP ping sweep to all destination host in subnet. Ø IX switch learn peer’s MAC address from ICMP reply. Ø Requires one ping poller per switch. Ø Simpler & less scalable in large network. Ø Multiple machines required. v Enhanced ping poller with SHA/SPA deployment Ø Use ARP request packets with spoof SPA/SHA in ARP packet. Ø IX switch learn peer’s MAC address from ARP reply. Ø Requires one ping poller per metro. Ø More complex but scalable. Ø Minimum one machine is required. Ping Poller #3 SWITCH_C SWITCH_A SWITCH_B Ping Poller #1 Ping Poller #2 Ping Poller SWITCH_C SWITCH_A SWITCH_B
  • 11.
    Confidential – ©2017 Equinix Inc. Equinix.com 11 11 Thank You