- Interconnection between internet networks in regional and secondary markets is often suboptimal, resulting in inefficient routing of traffic that can trombone through primary markets. - In cities like Tokyo, Dublin, and Singapore, large percentages of traffic from Tier 1 internet providers' peered routes are learned in other countries rather than being exchanged locally. - The lack of robust local peering harms user experience, network resilience, and regional economic development. While some improvements have been made, more progress is still needed to drive better interconnection in secondary markets.

1. Interconnection in Regional Markets
How it’s broken and why it’s such a problem.
-Tom Paseka

2. Disclaimer
• I’m going to talk about Tier 1 Networks
• I define a Tier 1 as a network whose routes are not transited by any other network,
except to their customers
• I don’t care about paid settlement.
• I’m looking at primary locations where multiple Tier-1’s interconnect
• I’m not looking at a number of locations, for example: Sydney, where three Tier 1’s are
present, but with next-to-no network.
• All of this data is based on the BGP view of AS13335. Yours may be different.

3. Primary Markets

4. Historical Interconnection

5. Historical Interconnection
• These locations evolved in part due to the IXPs that operate there.
• Eg:
Frankfurt - DE-CIX
Amsterdam - AMS-IX
London - LINX
Ashburn - Equinix
Palo Alto - PAIX (now Equinix)
etc.
• These locations continue to grow as major interconnection points with a huge density of
networks.

6. Understanding Current Interconnection
• Looking at our Transit Providers tables in San Jose
• Their Peer routes only , no customers.
• We receive 5 x Full Tables but looking at 3 transits:
• NTT Com AS2914
• TATA AS6453
• Cogent AS174
• Looking at combination of their peer BGP community tag, as well as location based tags,
we can see how many routes are locally exchanged.

7. San Jose/Palo Alto/SF Bay Area
• AS2914 NTT Communications:
• Looking at community 2914:420, for all their peer (not customer) routes:
• 372,244 Routes
• Now, combination [2914:420 2914:1008], for SF Bay Area location + peer routes:
• 360,033 Routes
• That’s roughly 97% local!

8. San Jose/Palo Alto/SF Bay Area
• AS6453 TATA Communications:
• Looking at community 6453:86, for all their peer (not customer) routes:
• 401,220 Routes
• Now, combination [6453:86 6453:1300], for SF Bay Area location + peer routes:
• 383,941 Routes
• That’s roughly 96% local!

9. San Jose/Palo Alto/SF Bay Area
• AS174 Cogent:
• Looking at community 174:21[01]00, for all their peer (not customer) routes:
• 346,788 Routes
• Now, combination [ 174:21[01]00 174:22013 ], for USA location + peer routes:
• 337,181 Routes
• That’s roughly 97% in country!
(Cogent doesn’t have, or doesn’t expose city based communities).

10. Understanding Current Interconnection
• Almost all interconnection between peers on the Tier 1 networks happens locally.
• This is EXCELLENT news for interconnection
• The story is completed over most of the traditional primary markets for interconnection.
• USA location considered because of interconnection
• Europe not the same, Many Tier 1’s have little or no presence
• Eg: AT&T AS7018, XO AS2828, Verizon AS701, Sprint AS1239, et al.
• Numbers hover around 75% of routes (Because of above).
• Some funky traffic engineering pollutes the numbers, but for the most part things are good.

12. Secondary Markets / Regional Interconnection
A bleak outlook

13. Regional Growth

14. Regional Growth
• Getting closer to eyeballs matters!
• It’s how CDN’s beat the speed of light with performance / delivery.
• But, so many users are far away from these “primary markets”.

15. Tokyo, Japan
• Looking at two Tier 1’s
• NTT AS2914 and TATA AS6453
• Same principal as previous tests
• NTT: Just 45,842 routes or 12% of peered routes are learnt in Tokyo
• TATA: Just 29,557 routes or 7.5% of peered routes as learnt in Tokyo
• This is despite a number of Tier 1 networks being present there.
• But, Tokyo might be a bad example because of “other” reasons.

16. Tokyo, Japan
Asia
31%
North
America
68%
Europe
1%
South
America
<1%
REGIONAL BREAKDOWN OF
ROUTE LOCATION - NTT TOKYO
Tokyo
40%
Osaka
<1%
Hong
Kong
38%
Singapore
22%
ASIA CITY BREAKDOWN OF
ROUTE LOCATION - NTT TOKYO

17. Dublin, Ireland
• Cogent AS174
• 452 routes learned in Ireland
• 0 Routes from Peers.
• GTT AS3257
• 2718 routes learned in Ireland
• 2162 routes learned from Peers
• Majority of these from a (major) eyeball
network, not another Tier 1
• (though this is still good for delivery!  )
                 

18. Dublin, Ireland
• Almost no inter-connection between Tier 1 networks
• Even though multiple have presence in the same buildings
• Saving grace is INEX (@comepeerwithme), but this is a talk about transit interconnection.
One star. Would not interconnect again.

19. Singapore, Singapore
• Looking at two Tier 1’s
• NTT AS2914 and TATA AS6453
• Same principal as previous tests
• NTT: 87,616 routes or an improved 23% of peered routes are learnt in Singapore
• TATA: 85,544 routes or 22% of peered routes as learnt in Singapore
• Definitely an improvement, but shows that connectivity here might be suboptimal.

20. Singapore, Singapore
• Local routes, looking at two largest local ISPs: Singnet and Starhub
• As seen from NTT and Tata, not more than 80% of local routes are seen within Singapore
• Across all of Asia, it improves slightly to 82%
• For at least 18% of traffic going over these two providers, traffic will trombone through the
USA to come back to Singapore

21. Singapore, Singapore
Asia
Not Asia
NTT SINGNET/STARHUB ROUTES
Asia
Not Asia
TATA SINGNET/STARHUB ROUTES

22. This sounds broken!

23. Issues arise
• Enterprise internet connectivity can be inefficient
• Applications suffer
• Economy suffers
• Network is less resilient

24. Issues arise
• What makes a market secondary?
• San Francisco Bay Area: population ~7.5million
• Phoenix Metropolitan Area: population ~4.5million
• Does that mean cities/regions with less than 5 million people suffer?
• Los Angeles <> Phoenix = ~370 miles
• Los Angeles <> San Jose = ~330 miles

25. Issues arise
• Dublin: population ~1.8Million
• Need more people to move in before better internet connectivity comes
around?

26. Issues arise
• What can drive improvement?
• Is this lack of facilities?
• Regulatory?
• Do we care?

27. Take Backs

28. Take Backs
In a regional, or secondary market, don’t be
surprised to have traffic trombone!

29. Take Backs
User Experience Suffers…

30. Take Backs
Peering can help!

31. Take Backs
But, things are getting better….

32. Take Backs
More still needs to happen!

33. Questions?