Martin Levy, Director, Hurricaine Electric "The Background of Internet Interconnection"

  • 969 views
Uploaded on

BEREC expert workshop on IP-Interconnection in cooperation with OECD …

BEREC expert workshop on IP-Interconnection in cooperation with OECD
November 2nd, Bloom Hotel Brussels, 9:00-17:30

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
969
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
19
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. The background of Internet interconnection ! g! pe erin et tI n tern ou m e abHurricane Electric AskIPv6 Native Backbone – Massive Peering! BEREC expert workshop on IP-Interconnection in cooperation with OECD Brussels – 2nd November 2011 Martin J. Levy, Director IPv6 Strategy Hurricane Electric
  • 2. Agenda – Internet interconnections (Peering?)!! Defining terms "! Long understood within the industry "! Subject to change and interpretation as needed "! Useful numbers!! Reality "! It’s not as clear-cut as it seems!! The cost model "! Highly simplified!! Dissecting a peering agreement "! Some serious hidden gotcha’s2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 2 OECD - The background of Internet interconnection
  • 3. PEERING & TRANSIT2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 3 OECD - The background of Internet interconnection
  • 4. Redefining the terms!! Peering "! The exchange of traffic between two networks "! Only for traffic associated with the network, or its customers "! Can be via an IX/NAP or a private interconnect "! Subject to a bilateral or unilateral contract "! Either zero-cost, charged-for service or as part of other contracts "! Majority of negotiations done on a bilateral basis!! Transit "! A network providing another network with a full connection to the Internet "! Sometimes provided as a “default route” or as full routes!! Partial Transit "! A network providing access to only some of the global Internet "! Sometimes sold as access to a specific provider or destination2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 4 OECD - The background of Internet interconnection
  • 5. Internet peering points (IXs/NAPs) exist globally Oslo Paris NIX London LINX EQUINIX Shanghai Taipei LONAP FRANCEIX Stockholm National IX TWIX SFINX NetNod Seattle NJ & NY FreeIX STHIX Hong Kong SIX EQUINIX SOL-IX HKIX Seoul Brussels EQUINIX TIE TELX EQUINIX KIDC Toronto BNIX EQUINIX NYIIX FreeBiX Frankfurt KINX Dublin TORIX BigApe INEX Amsterdam DE-CIX BeijingSan Jose AMSIX KleyReX National IX Major IXs/NAPs represented; plenty more exist Chicago EQUINIX EQUINIX NL-IX ANY-2 Chicago IX AMES Guangzhou National IX Boston MXP MSK-IX Moscow NIX CZ Prague JPNAP GigaPix VIX JPIX EQUINIX Lisbon Vienna ANY2 DIX-IE PAIX MIX EQUINIX Virginia area ESPANIX MINAP EQUINIX NOTA Miami Madrid Milan Tokyo ANY-2 LAIIX CERN Geneva NIXILos Angeles Mumbai TIE TELX SwissIX Sydney EQUINIX EQUINIX EQUINIX Atlanta Zurich NIXI PIPE New Delhi EQUINIX MyIX TIE TELX Kuala Lumpur Dallas EQUINIX SOX Singapore IIX KIXP Jakarta NAP do Brasil Nairobi PTTMetro São Paulo JINX Johannesburg WIX Wellington NAP Chile CABASE APE Santiago Buenos Aires V6IX Auckland2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 5 OECD - The background of Internet interconnection
  • 6. Internet IXs/NAPs – fundamental to the Internet!! Internet peering points exist globally "! High traffic densities in the US, EU & parts of Asia !! London, Amsterdam, Frankfurt, Bay Area, Virginia, Tokyo, etc.!! Corporate structure: MAE-West in San Jose California Back of the envelope design for "! Member-based or Commercial!! Nearly always public stats "! Member list & traffic stats vital!! Highly unregulated "! Slight exceptions: India, Chile, etc.2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 6 OECD - The background of Internet interconnection
  • 7. Is the Internet growing? Is there more peering? ASNs Network Prefixes 10% Growth in IPv4 table http://bgp.he.net/report/prefixes 78% Growth in IPv6 table!! Internet Growth still happening "! More growth within IPv6; which makes sense "! More networks joining the global Internet!! This is just a measure of destinations on the Internet "! It’s not measure of peering or transit relationships2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with OECD - Page 7 The background of Internet interconnection
  • 8. Who are the big Internet peering players? IPv4 IPv6 http://bgp.he.net/report/peers!! This is a measure of BGP adjacencies "! Customers are adjacencies "! Peers are adjacencies!! A high adjacencies means there’s a short path "! Short as in “AS PATH” vs. physical path2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with OECD - Page 8 The background of Internet interconnection
  • 9. Can one diagram peering and transit? Tele2 (Sweden)Sprint – providing transit AOL Level3 NTT/Verio Verizon AT&T Qwest/CenturyLink Savvis TATA Global Crossing Verizon– providing transit? (nah!) IIJ DTAG AboveNet Telia IIJ TInet Cogent Telecom Italia Hurricane Electric C&W Telecom Italia 2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with OECD - Page 9 The background of Internet interconnection
  • 10. Can one diagram peering and transit? BICS (Belgacom Intl’) Global Crossing – providing transit NTT – providing transit Level3 – providing transit TATA – providing transit2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with OECD - Page 10 The background of Internet interconnection
  • 11. REALITY2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 11 OECD - The background of Internet interconnection
  • 12. The reality within the peering and transit world!! Peering as a donut model "! Smaller players peer around the core "! This bypasses “Tier1 players” – and works somewhat well!! Peering can be more expensive than transit "! Over the last few years the cost of building a peering-based network has grown to be greater than buying transit "! Peering requirements force applicants to expand to match those requirements "! Peering requirements chance!! Peering sometimes means non-peering related costs "! You can get peering if your paying for something else2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 12 OECD - The background of Internet interconnection
  • 13. Peering – “I felt a great disturbance in the Force” 1997 was a good year for the peering world to “wake up” …2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 13 OECD - The background of Internet interconnection
  • 14. HOW WE GOT HERE2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 14 OECD - The background of Internet interconnection
  • 15. January 1st! 1983! First com puter w RFC791! distribut ed via th orms! e Intern 1988! et! 1981! UUnet fo unded! InterNIC!Mosaic! NS! s! 10,000 host D IETF ki 1987! cks off 1993! IPng (I1984! Pv6)!ARPAnet W NSFn ! et DS3’ upgrade s (45 1994! WW EXPLOSIVE GROWTH!!!!! s name goe Mbp to! s away! s)! 1990! CIX! 1992! 1995 1996 1997 1998 1999 2000 2001 2002 1991! 2003 2004 2005 2006 2007 2008 2009 2010! 2011! NT ! EYEBA CONTE LLS!2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 15 OECD - The background of Internet interconnection
  • 16. What’s all the fuss about?Commercial pressure andclassic business sense vs.2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 16 OECD - The background of Internet interconnection
  • 17. TRANSIT COSTS2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 17 OECD - The background of Internet interconnection
  • 18. Transit costs – but how much depends on where Moderate Cheap CheapCheap Expensive Moderate Expensive Very Expensive Expensive Expensive2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 18 OECD - The background of Internet interconnection
  • 19. THE INVERSE COST MODEL2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 19 OECD - The background of Internet interconnection
  • 20. Transit costs – counter intuitive – inverse distance !!!!!! !!!!! !!!!! Country-Based !!!! ISP/Operator ! Content Content Content Content Cheaper transit cost for content / more expensive for operator Higher latency (and potential packet loss) Content closer to customer (perceived to be better) Proven better performance2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 20 OECD - The background of Internet interconnection
  • 21. PEERING AGREEMENT REVIEW2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 21 OECD - The background of Internet interconnection
  • 22. Peering Agreement – The review!! What follows is a sentence-by-sentence review of the Comcast Settlement-Free Interconnect (SFI) Policy!! Can be found at: http://www.comcast.com/peering/ "! Text dated Jan/2008 June/2009 July/2011!! Anything with strikethrough is from an earlier version!! This peering language can change at any time!! Not actually called “peering” but “SFI”2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 22 OECD - The background of Internet interconnection
  • 23. Sample peering agreement (public information)Comcast Communications, Inc., Comcast Cable Communications Management, LLC, through itsoperating subsidiaries ("Comcast"), seeks to interconnect its IP network with other Internet backboneproviders on a settlement-free basis when such interconnection provides tangible benefits to Comcastand its customers.For purposes of this SFI policy, an Internet Network must be a single Autonomous System (“ASN”).The criteria involving traffic measurement below applies to IPv4 IPv4 and IPv6 and must be met by theApplicant for three consecutive months. To apply for IPv4 or IPv6 settlement-free interconnection withComcast an email must be sent to peering@comcast.com containing evidence of meeting all criterialisted in the policy. Applicants will be responded to within a reasonable timeframe to discuss theirrequest.Networks that do not meet the requirements of the settlement-free interconnection policy, but remaininterested in Comcast dedicated internet access services may also address their requests topeering@comcast.com. In this case the subject line of the e-mail should indicate a need for salesassistance. The body of the message should include as many details as possible, such as locationand telephone number, to expedite the redirect for assistance.2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 23 OECD - The background of Internet interconnection
  • 24. Sample peering agreement (public information)•! IPv4 IPv4 and IPv6 Settlement-Free Interconnection Requirements for Comcast AS7922•! Applicant must operate a US-wide IP backbone whose links are primarily 10 Gbps or greater.•! Applicant must meet Comcast at a minimum of four mutually agreeable geographically diverse points in the US. Interconnection points must include at least one city on the US east coast, one in the central region, and one on the US west coast, and must currently be chosen from Comcast peering points in the following list of metropolitan areas: New York City/Newark NJ, Ashburn, Atlanta, Miami, Chicago, Denver, Dallas, Los Angeles, Palo Alto/San Jose, and Seattle.•! Applicant’s traffic to/from the Comcast network must be on-net only and must amount to at least 3 Gbps peak 7 Gpbs peak 10 Gbps average in the dominant direction. Interconnection bandwidth must be at least 1 Gbps 10 Gbps at each interconnection point.•! A network (ASN) that is a customer of a Comcast network for any dedicated IP services may not simultaneously be a settlement-free network peer.•! Applicant must have a professionally managed 24x7 NOC and agree to repair or otherwise remedy any problems within a reasonable timeframe. Applicant must also agree to actively cooperate to resolve security incidents, denial of service attacks, and other operational problems.2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 24 OECD - The background of Internet interconnection
  • 25. Sample peering agreement (public information)•! Applicant must maintain responsive abuse contacts for reporting and dealing with UCE (Unsolicited Commercial Email), technical contact information for capacity planning and provisioning and administrative contacts for all legal notices.•! Applicant must agree to participate in joint capacity reviews at pre-set intervals and work towards timely augments as identified.•! Applicant must maintain a traffic scale between its network and Comcast that enables a general balance of inbound versus outbound traffic. The end-to-end costs of network cost burden for carrying traffic between networks shall be similar to justify SFI.•! Applicant must abide by the following routing policy: •! Applicant must use the same peering AS at each US interconnection point and must announce a consistent set of routes at each point, unless otherwise mutually agreed. •! No transit or third party routes are to be announced; all routes exchanged must be Applicants and Applicants customers routes. •! Applicant must filter route announcements from their customers by prefix. •! Neither party shall abuse the SFI network peering relationship by engaging in activities such as, but not limited to: pointing a default route at the other or otherwise forwarding traffic for destinations not explicitly advertised, resetting next-hop, selling or giving next-hop to others.2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 25 OECD - The background of Internet interconnection
  • 26. Sample peering agreement (public information)•! Applicant should be willing to enter into an NDA before formal discussions begin.•! Applicant should be advised that the SFI processes will start with a 90 day trial. On successful completion of that trial, a formal interconnect agreement will be processed. This agreement will renew annually, subject to the then current SFI Policy. During the year if there is a violation of the policy, the agreement and interconnections may be terminated upon written notice to the contacts specified in the agreement.•! Applicant shall not be permitted to offer or sell any IP transit services providing only AS7922.•! Applicant must be financially stable.•! Comcast requires that Applicants seeking SFI in the United States agree to provide reciprocal SFI arrangement with Comcast in the Applicant’s home market.2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 26 OECD - The background of Internet interconnection
  • 27. Sample peering agreement (public information)•! Ongoing SFI Reviews with Comcast AS7922•! Meeting the SFI network peering guidelines set forth herein is not a guarantee that an SFI network peering relationship with Comcast will be established. Comcast shall evaluate a number of business factors and reserves the right not to enter into a SFI network peering agreement with an otherwise qualified applicant. Applicants may apply for settlement-free interconnection once every six (6) months.•! Existing SFI network peers of Comcast will have their status reviewed periodically to ensure that joint capacity planning intervals are sufficient for growth, contacts are refreshed for operations purposes, and all criteria continue to be met.•! Comcast reserves the right to terminate SFI network peering, upon a notice period as determined by the parties agreement, with peers who do not meet the criteria described above.•! Periodic review of the policies contained here will be conducted to ensure that the criteria and eligibility requirements are consistent with Comcasts business needs. Comcast may modify this SFI policy at any time.2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 27 OECD - The background of Internet interconnection
  • 28. SUMMARY2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 28 OECD - The background of Internet interconnection
  • 29. Summary!! Starting in the peering world is easy !! Fraught with “landmines” (once you play big) !! Financially challenging Financial Times – November 2nd 2011 – http://ft.com/ !! It’s the road to “Tier-1” status!! Wholesale transit has evolved as a commodity-play with very low margins !! Enterprise pricing is still good !! Bulk hosting sales – not so good!! Complaining to the regulator? … !! Jimmy Wales said it best! “the greatest threat to the internet was misguided and over-reaching government policy”2 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with 29 OECD - The background of Internet interconnection
  • 30. Q&A Contact: ?Wa alwa ? ering urrica nt pe ” at H y s “ye s ! er is ! answ ectric - The ne El Martin J. Levy Director, IPv6 Strategy Hurricane Electric 760 Mission Court Fremont, CA 94539, USA http://he.net/ martin at he dot net +1 (510) 580 41672 Nov 2011 Martin J. Levy - Hurricane Electric - BEREC in cooperation with OECD - Page 30 The background of Internet interconnection