SlideShare a Scribd company logo
1 of 10
Download to read offline
Version 1.0                                                                               3/13/2011




                               International R/E Routing

               Chris Robb – CIREN/TransPAC2 Engineer and Abilene Engineer
                 James Williams – CIREN/TransPAC2 Principal Investigator



This brief White Paper was prepared at the request of Kevin Thompson of the National Science
Foundation. The objective of the paper is to provide a vehicle for discussion of the particular
issues connected with “efficient” international routing. The paper is organized in seven sections
as follows:

        1.  Description of the problem
        2.  Description of inter-networking in both R/E and commercial networks
        3.  Example of a R/E routing problem
        4.  Recommendations of immediate and mid-term activities that would address the
            problem
        5. Longer-term issues and the evolution of R/E networking
        6. Summary and Recommendations
        7. Appendix ( which contains some additional detailed routing examples)

1. Problem Description

Although labeled or “called” a routing problem, the issues surrounding inefficient traffic flow on
international R/E network links have a number of different aspects. Examples of these follow.

On occasion (and sometimes for long periods of time) traffic between two locations does not
travel by the “most desirable” path. The desirability of a particular unidirectional path can vary
based on the application requirements and preference of the network administrators. For
example, certain applications respond well to low latency, but don’t necessarily require large
amounts of bandwidth. Other applications are written to take advantage of bandwidth, but may
put less emphasis on latency. Often times, these applications coexist within a particular network,
creating the need for more granular path selection methodologies.

Regardless of the definition of desirable, there are very few pervasive tools available to the
operators of R/E networks to understand the extent to which undesirable traffic paths are being
utilized. In addition to the widely implemented ability to traceroute into a network, several projects
are providing useful tools that the R/E community should embrace. Primary amongst them is the
operationally focused RouteViews project1, based at the University of Oregon. Building upon that,
the Computer Networks Research Group’s2 BGPlay project3 provides the dimension of time,
giving operators a 10 day view into RouteViews peer networks routing tables. In addition to these
centralized and coordinated efforts, individual networks often have their own disjoint data
collection tools and operational-focused debug tools for outsiders to utilize. Unfortunately, these
individual network looking glasses and netflow data analysis tools are often difficult to reconcile in
any programmatic fashion between disparate networks.


1
  http://www.routeviews.org/
2
  http://www.dia.uniroma3.it/%7Ecompunet/
3
  http://bgplay.routeviews.org/bgplay/
Version 1.0                                                                             3/13/2011

In addition to the lack of debug tools, engineers often challenged with understanding how they
should handle the import and export of routes into or out of their network. Router vendors and
standards bodies have provided an extensive set of methodologies to implement traffic
engineering policies, but the community lacks the necessary metadata to align policy with a
particular researcher’s desired outcome.

Finally, there is limited communication among international R/E network operators except in times
of emergency/crisis or perhaps near some particularly significant event such as SuperComputing
or iGrid. Within the North American commercial network operators, NANOG4 provides a forum
for NOC-NOC engineering and operational discussions. But, R&E participation in NANOG has
historically been limited and the discussion is generally limited to North American issues, as one
might expect from the name.

In summary, the “routing problem” is not very well-defined and there are few, or at least not very
well-known, tools for providing assistance/information to a not very well-organized group of R/E
NOC operators.




4
    http://www.nanog.org/
Version 1.0                                                                               3/13/2011


2. Description of Inter-network routing in Research and Education networks and
commercial networks

There are significant differences between the commercial world (CW) and the R/E world with
respect to routing and the drivers for “efficient” routing.

Commercial networks
In the CW very large networks (Tier1 ISPs) peer with each other and exchange traffic by a
mechanism called Settlement Free Interconnects (SFI)5. With this type of peering large ISPs
interconnect with each other in multiple geographic locations with no financial exchange between
the ISPs. The theory behind SFI is that traffic between these large ISPs is approximately equal.
Consequently, Tier1 ISP treat each other as equals. However, Tier1 ISPs regularly monitor these
traffic flows to ensure that this concept of equal flow remains accurate. In general, Tier1 ISPs
have several motivating factors to establish SFI interconnects:
     • Lowering transit costs
     • Increased control over routers; direct peering always yields more ability to shift traffic
           around
     • Decreased latency for their customers; the shorter distances between endpoints cause
           TCP backoff algorithms to allow an application more bandwidth- hence more traffic (and
           better performance) from their customers, whom they charge
Further, there are several reasons why a network would choose not to SFI peer with another
network:
     • Loss of potential revenue with a network you might otherwise charge for transit
     • Traffic load asymmetry. (i.e. if the other network provides service to a large group of
           content providers, the asymmetric nature of the traffic volume combined with a hot-potato
           routing methodology, may mean that you would bear the greater burden in the creation of
           a bidirectional data exchange; the traffic would be on your network longer than it would
           be on your peers network
     • Peering may strengthen a potential competitor, thus drawing away your customer base

In the CW smaller networks (Tier2 ISPs) and some content providers purchase transit (access to
routes) from larger Tier1 ISPs. This is a tiered network model, in contrast to the R/E flatter
network model described below. There is a financial exchange between Tier2 and Tier1
networks. Consequently, it is in the interests both sides of this exchange to closely monitor traffic
passing between networks and be certain that it passes in the most efficient and effective
manner.

In more recent years, smaller ISPs (cable modem providers, regional ISPs, etc.) and large
content providers (Google, Apple, Microsoft) have begun to peer directly with each other in a
settlement free context. This more distributed approach has somewhat reduced the importance of
larger ISPs, though they are still critical as a “gateway of last resort” for networks that don’t have
presence at all the major exchange points6.

In the CW, there are real costs and real dollars paid for connectivity. Consequently, decisions are
made based on cost minimizing considerations. Network provisioning closely matches network
usage. Significant attention is paid to routing, routing tables and traffic balances. There is careful
analysis of imbalances and possible routing problems. Each of the facets of peering are analyzed
by engineers and lawyers before decisions are rendered and peering is established. If there is
any uncertainty, the NANOG list is a useful reference.


Research and Education Networks
5
    http://en.wikipedia.org/wiki/Settlement-free_interconnect
6
    http://en.wikipedia.org/wiki/Internet_Exchange_Point#North_America
Version 1.0                                                                               3/13/2011


Research and Education networks are created, designed and provision quite differently (for quite
different reasons) than commercial networks.

The most significant difference between the CW and R/E worlds is that in the R/E world,
bandwidth is usually (not always) purchased by government organizations via grants or
consortiums. There is less incentive to reduce costs via more efficiency, whether in usage or
routing. Essentially, by design, there is little connection between costs and usage.

Furthermore, there is less incentive to manage bandwidth efficiently in the R&E networks due to
the over provisioned nature of the R&E infrastructure. It is not uncommon for R/E networks to
carry an average of 100Mbps of traffic on a 10Gbps connection. In a network over provisioned by
a factor of 100, it is difficult to convince network operators to devote resources to the idea of
efficiency. As is discussed later, this over provisioning is purposeful.

Following on the point made above, in the R/E world links are provisioned for the “most extreme
individual experiment” rather than to insure generally optimal and cost-effective service. Given
the funding from governments or granting agencies and the fundamental purpose of R/E
networks, this is not an unreasonable approach.

The R&E world has the additional dynamic of catering to individual researcher’s needs. At any
given time, though more frequently around large conferences such as Supercomputing7, network
engineers may utilize several different methodologies to alter network paths to satisfy the needs
of a particular researcher. Often times, this has the unfortunate side effect of becoming a
permanent, and difficult-to-track feature of the network landscape. Depending on the willingness
of the network engineer to properly address the problem, this network “gaming” will often pull
significant portions of an organizations network in a direction that is well suited to a particular
researcher, but not to the larger population.

Additionally, this lack of coordination of routing is definitely due, in part, to network engineers
simply not having a complete (or even partial) understanding of global routing. As was pointed
out previously, there is both a lack of tools (or, more specifically, a lack of knowledge about
available tools) and the lack of a forum for the international discussion of this issue.

The issues above, when combined with the more general lack of strict legal definitions of
appropriate peering and financial reparations, the traditional over-commitment of engineering
resources and an (appropriate) focus of resources on providing means for the development of
science collaborations, create a more relaxed attitude to network management and routing policy
enforcement.

Finally, international routing paths, and in fact entire international networks, may develop for
reasons connected to political rather than technical objectives. It is quite common for countries to
have two smaller bandwidth connections to the United States rather than one, most larger and
possibly less expensive connection. Political reasons trump technical (and financial) reasons
anytime.

However, it is important to note that Research and Educations networks are designed to create
opportunities and collaborations rather than simply pass/exchange traffic and generate revenue.
Consequently, R/E networks are sized for anticipated users rather than existing users, greatly
reducing the operational connection and attention between traffic and network size and costs.
This fundamental difference between commercial and R/E networks argues strongly for a parallel
but separate forum for the discussion of R/E routing issues and in no way diminishes the
necessity for development of global routing visualization and management tools.


7
    http://www.supercomputing.org/
Version 1.0                                                                              3/13/2011



3. An Example of a R/E routing problem




                          Figure 1: Merit Route Propagation to Abilene

The diagram above illustrates a specific issue that was observed in March 2006 while
troubleshooting a connectivity problem between Abilene (AS11537) and Merit (AS237).
Specifically, traffic between AS11537 and AS237, which should travel directly between these
nodes, was instead traveling through Korea!


   1. Merit, an MREN connector, has direct peerings with both Abilene and MREN. It
      announces its routes to all it’s peers.
   2. MREN has a peering with KREONET2, so it announces it’s customer routes.
   3. KREONET2 has a transit agreement with KOREN to provide transit to US customers, so
      KREONET2 announces the Merit route to KOREN
   4. KOREN announces it’s entire routing table, including non-customer routes to APAN
   5. APAN trusts that KOREN will filter its non-customer routes out and passes all KOREN-
      learned prefixes to TransPAC2 in the US.
   6. TransPAC2 trusts that APAN is only sending APAN customer prefixes so it passes all
      APAN-learned routes to Abilene.
   7. Abilene now has received two advertisements. One via the direct MREN peering and one
      via the much longer path via Asia.
   8. Abilene doesn’t have the Merit route in it’s inbound prefix filter facing Merit, so the direct
      MERIT-learned route is dropped.
   9. Even if Abilene had the MERIT route in it’s filter, MERIT was tagging the route with
      Abilene community 11537:140, which causes it to be preferred with a localpref of 140.
      This is done because of a backup agreement with a separate US-based network not
      depicted.
Version 1.0                                                                              3/13/2011


    10. APAN tags all its routes with Abilene community 11537:160. This is passed to Abilene
        and causes APAN routes learned via TransPAC2 to be local-preferenced with a weight of
        160- higher than the direct Merit local-preference. APAN tags their routes with 11537:160
        to influence return traffic over TransPAC2 instead of over a separate circuit they maintain
        to the US.

What are some of the issues that caused the problem above?

            A. KREONET2 should be indicating to KOREN which routes are transit routes and
               which routes are their internal customer routes
            B. KOREN should not announce non-customer routes to APAN. Or they should
               announce those routes with tags that allow APAN to filter out non-KOREN
               customer routes
            C. APAN should be able to distinguish between it’s customers routes and its
               customers transit routes
            D. Abilene has local-preference values that can be manipulated remotely. An
               engineer needs to be familiar with Abilene’s internal BGP community
               infrastructure to successfully navigate the problem
            E. With the exception of Abilene’s direct peering with Merit, all networks along the
               path appear to have very loose restrictions on what networks they will accept
               from their BGP neighbors.

In this case, as in many (all?) international routing problems, problem resolution is often relegated
to multiple point to point discussions that resolve individual problems, but rarely address the
larger issues. The routing problem is fixed. But, the underlying issues that lead to the problem
remain. The lack of problem “broadcast” relegates the discussion to one-time, one-on-one
immediate triage with little follow-up on the greater problem. Additionally, this lack of “broadcast”
limits knowledge of the problem and the underlying issues to only the directly involved parties.


4. What immediate and mid-term steps can be taken to address these problems?

The R/E community should move toward using international common technical forums to focus
attention on the problem. Obvious suggestions are the APAN meetings, the Internet2 meetings,
the TEIN2 meetings, the TERENA meetings and other international networking forums. There
was a full-session discussion of international routing at APAN22 in Singapore
(http://www.apan.net/meetings/singapore2006/proposals/APAN-Routing.html). A follow-up
routing meeting is planned for the December Internet2 meeting in Chicago and for the APAN23
meeting in Manila.

The R/E community should encourage use of (participation in) existing tools such as RouteViews
to make current routing visible to the entire community. Again, see the notes from APAN22
referenced above. It was agreed at this meeting that all APAN countries who were not
participating in RouteViews would participate by APAN23.

The R/E community should encourage international NOC-NOC engineering and operational
cooperation. To that end, rather than simply grumbling about inefficient or incorrect routing
engineers and attempting one-to-one solutions NOC operators need a mechanism (forum) to
address the problems, in “broadcast” mode.

The Coordinating Committee for Internatinal Research Networking (CCIRN) has taken a step
toward addressing this lack of a forum problem. At the April 2006 meeting the CCIRN charged
Indiana University and Internet2 with the development of the Research and Education Network
Operators Group (RENOG). See the following for details: http://www.ccirn.org/CCIRN2006-
minutes-final.pdf. RENOG, is a light weight “mailing list organization” designed to provide the
ability to broadcast both international network routing problems and the solutions to these
Version 1.0                                                                               3/13/2011

problems. At the present time, the success of RENOG, as measured by traffic on the mailing list
is limited. Perhaps this approach is too limited or another mechanism for addressing this problem
needs to be tried. Our view is that the ability of the international routing community to interact
more publicly is particularly important because of the parallel, but fundamentally different, goals of
commercial and R/E networks.




5. Longer-term issues and the evolution of R/E networking

How will the direction of evolution of R/E networking will effect “routing” ? That is impossible to
predict (at least for us). We offer three questions that we view as important speculation relating
to international R/E networks and routing.

    1. Will switched on-demand bandwidth become a primary mechanism for handling large
       data flows? If so, then routing will take on an additional meaning. Routing may mean
       allocation of light paths rather than only the direction of packets.

    2. What is the future of backbone networks? Will networking become a collection of
       regional networks connected by on-demand circuits with much smaller backbone
       connections, thus increasing the number of entities involved in resolving performance
       problems and routing between these regional organizations? Or, will the continually
       declining cost of raw bandwidth mean that backbone networks will simply expand and
       offer services as good as on-demand networks in a much simpler environment?


    3. How will R/E networking in the United States be delivered and how will this effect
       international networking and routing? The NLR-Interent2 controversy in the US certainly
       has the potential to make international routing more confusing, as international networks
       connect to one or both major US R/E networks, not considering the already existing
       issues of FedNets in the US. As US entities “take sides” in the NLR-I2 dispute, it is
       possible that a connection between two US universities, one connected to NLR and one
       connected to Abilene/I2 would be routed to an international exchange point and then
       back into the US rather then routed directly over US terrestrial infrastructure.
Version 1.0                                                                           3/13/2011



6. Summary and Recommendations

Commercial networks and R/E networks are built for different purposes and are managed to
different objectives. Commercial networks are concerned about costs and efficiency. R/E
networks are designed to create collaborations and support research rather than efficiently
exchange traffic or generate revenue. However, while the managing principles may be different,
the underlying needs of routing engineers in both types of networks are surprisingly similar.

This paper makes three recommendations.

   1. Networks need to participate in (make use of) existing routing tools such as RouteViews.
      This directly relates to Recommendation #2.

   2. The issue of international routing needs to be given attention on the international stage
      (at meetings such as CCIRN, APAN and TERENA).

   3. The international R/E community needs a regular forum where routing issues can be
      discussed and information exchanged among international routing engineers.
Version 1.0                                                                           3/13/2011

                                          Appendix A

The following exhibits are further examples to illustrate some of the unique challenges that the
authors of this paper have been involved with. They have been chosen as representative
examples that highlight a facet of the discussion and are not meant to represent the complete set
of challenges that have arisen in the past several years.

Exhibit A: R/E “Fish” Traffic

Given the common US interconnection methodology of bringing connectors behind large regional
aggregation points (called gigapops), and given that the service offerings of most gigapops
include both commodity and R/E peerings, the R/E community has faced the following
asymmetric behavior for quite a number of years. Consider the following topology:




                                    Figure 2: R/E Fish Traffic

In the topology above, US Gigapop A provides commodity connectivity for one of it’s connectors.
It does not provide R/E access, so it’s routes aren’t sent to the R/E network (nor would the R/E
network accept them). Gigapop B provides both R/E and commodity service to its connector. As
such, it advertises its customer’s routes to both the commodity and the R&E network. Gigapop A
receives these routes from both the commodity and R/E peerings. Gigapop A will typically
localpreference routes from the R/E network higher than routes on the commodity network. As
such, Gigapop A prefers the R/E path toward Gigapop B’s customer, and Gigapop A prefers the
Commodity path for Gigapop A’s customer.

This is a fairly old problem with some well-known solutions. Most US Gigapops today have solved
the issue by using Virtual Routing and Forwarding (VRFs) tables so they can keep distinct sets of
destinations for use by their connectors. The question will be complicated somewhat in the
coming months as US R/E networks (notably Abilene and the National TransitRail project) are
looking toward providing all or part of commodity routes as part of their service offering.
Version 1.0                                                                            3/13/2011



Exhibit B: TEIN2 Routing

In late 2005, the TEIN2 network was brought online, creating the following topology:




                                    Figure 3: TEIN2 Routing

TEIN2 administrators had some very specific traffic flow directions in mind when creating the
network traffic engineering policy. One specific desire was for US-bound traffic to flow in the
direction of APAN, with GEANT as a backup. This decision was based on the network latency
and available bandwidth between the various locations. As such, the TEIN2 network receives US
network routes from both GEANT and from APAN. In addition, TEIN2 provides backup
connectivity between APAN and GEANT, so it passes GEANT-learned and APAN-learned routes
between the two networks. A difficulty arises when the set of routes that Abilene advertises to
GEANT isn’t congruent with the set of routes it advertises to APAN.

Such a problem arose in early 2006 when GEANT engineers noticed that their traffic to various
US Federal networks was taking the very long path via Asia to get to the United States. At issue
was the Internet2 Federal Network policy. Due to some contractual constraints with Abilene’s
carrier, Abilene cannot do blanket advertisements of federal networks to its international peers.
Some networks have agreements in place to receive all or some of the set of federal networks.
TransPAC2 was such a network with an agreement to receive all Federal network routes. GEANT
was only receiving a subset of those destination advertisements. Since TransPAC2 passes those
routes to APAN, and APAN passes all their US-learned routes to TEIN2 (for the GEANT backup),
this created an incongruent set of routes that GEANT received between it’s TEIN2 and Abilene
peerings. Given the latency involved, a commodity path was more preferable from their
perspective.

More Related Content

What's hot

Building a Regional 100G Collaboration Infrastructure
Building a Regional 100G Collaboration InfrastructureBuilding a Regional 100G Collaboration Infrastructure
Building a Regional 100G Collaboration InfrastructureLarry Smarr
 
History and overview of csnet
History and overview of csnetHistory and overview of csnet
History and overview of csnetsugeladi
 
Lambda data grid: communications architecture in support of grid computing
Lambda data grid: communications architecture in support of grid computingLambda data grid: communications architecture in support of grid computing
Lambda data grid: communications architecture in support of grid computingTal Lavian Ph.D.
 
The Exascale Computing Project and the future of HPC
The Exascale Computing Project and the future of HPCThe Exascale Computing Project and the future of HPC
The Exascale Computing Project and the future of HPCinside-BigData.com
 
ECE 528T1: Design of Communication Systems Module 2
ECE 528T1:  Design of Communication Systems Module 2ECE 528T1:  Design of Communication Systems Module 2
ECE 528T1: Design of Communication Systems Module 2Jeffrey Des Binwag
 
IRJET- Internet Video Streaming Service for Social Network
IRJET- Internet Video Streaming Service for Social NetworkIRJET- Internet Video Streaming Service for Social Network
IRJET- Internet Video Streaming Service for Social NetworkIRJET Journal
 
Studying performance barriers to cloud services in Africa's public sector
Studying performance barriers to cloud services in Africa's public sectorStudying performance barriers to cloud services in Africa's public sector
Studying performance barriers to cloud services in Africa's public sectorAFRINIC
 
CARLETON_SrSysEng08.11.2015
CARLETON_SrSysEng08.11.2015CARLETON_SrSysEng08.11.2015
CARLETON_SrSysEng08.11.2015Rich Carleton
 
Datacom module 1: Introduction to Data Communications
Datacom module 1:  Introduction to Data CommunicationsDatacom module 1:  Introduction to Data Communications
Datacom module 1: Introduction to Data CommunicationsJeffrey Des Binwag
 

What's hot (13)

Building a Regional 100G Collaboration Infrastructure
Building a Regional 100G Collaboration InfrastructureBuilding a Regional 100G Collaboration Infrastructure
Building a Regional 100G Collaboration Infrastructure
 
History and overview of csnet
History and overview of csnetHistory and overview of csnet
History and overview of csnet
 
[.ppt]
[.ppt][.ppt]
[.ppt]
 
Lambda data grid: communications architecture in support of grid computing
Lambda data grid: communications architecture in support of grid computingLambda data grid: communications architecture in support of grid computing
Lambda data grid: communications architecture in support of grid computing
 
Tackling Protocol Diversity: ISOC@IETF Panel at IETF 93
Tackling Protocol Diversity: ISOC@IETF Panel at IETF 93Tackling Protocol Diversity: ISOC@IETF Panel at IETF 93
Tackling Protocol Diversity: ISOC@IETF Panel at IETF 93
 
The Exascale Computing Project and the future of HPC
The Exascale Computing Project and the future of HPCThe Exascale Computing Project and the future of HPC
The Exascale Computing Project and the future of HPC
 
ECE 528T1: Design of Communication Systems Module 2
ECE 528T1:  Design of Communication Systems Module 2ECE 528T1:  Design of Communication Systems Module 2
ECE 528T1: Design of Communication Systems Module 2
 
CV-RPLiu-2014
CV-RPLiu-2014CV-RPLiu-2014
CV-RPLiu-2014
 
IRJET- Internet Video Streaming Service for Social Network
IRJET- Internet Video Streaming Service for Social NetworkIRJET- Internet Video Streaming Service for Social Network
IRJET- Internet Video Streaming Service for Social Network
 
Studying performance barriers to cloud services in Africa's public sector
Studying performance barriers to cloud services in Africa's public sectorStudying performance barriers to cloud services in Africa's public sector
Studying performance barriers to cloud services in Africa's public sector
 
CARLETON_SrSysEng08.11.2015
CARLETON_SrSysEng08.11.2015CARLETON_SrSysEng08.11.2015
CARLETON_SrSysEng08.11.2015
 
Datacom module 1: Introduction to Data Communications
Datacom module 1:  Introduction to Data CommunicationsDatacom module 1:  Introduction to Data Communications
Datacom module 1: Introduction to Data Communications
 
DeStephen Derek RESUME
DeStephen Derek RESUMEDeStephen Derek RESUME
DeStephen Derek RESUME
 

Similar to International R/E Routing (v1.0)

Standard Protocols for Heterogeneous P2P Vehicular Networks
Standard Protocols for Heterogeneous P2P Vehicular NetworksStandard Protocols for Heterogeneous P2P Vehicular Networks
Standard Protocols for Heterogeneous P2P Vehicular Networksijtsrd
 
How Web Analytics Can Help Businesses Or Organizations...
How Web Analytics Can Help Businesses Or Organizations...How Web Analytics Can Help Businesses Or Organizations...
How Web Analytics Can Help Businesses Or Organizations...Kendra Cote
 
The International Journal of Engineering and Science (The IJES)
The International Journal of Engineering and Science (The IJES)The International Journal of Engineering and Science (The IJES)
The International Journal of Engineering and Science (The IJES)theijes
 
CONCEPTUAL FRAMEWORK OF REDUNDANT LINK AGGREGATION
CONCEPTUAL FRAMEWORK OF REDUNDANT LINK AGGREGATIONCONCEPTUAL FRAMEWORK OF REDUNDANT LINK AGGREGATION
CONCEPTUAL FRAMEWORK OF REDUNDANT LINK AGGREGATIONcseij
 
OPERATORS CAN SAVE $14 MILLION YEARLY THROUGH DATA OFFLOADING
 OPERATORS CAN SAVE $14 MILLION YEARLY THROUGH DATA OFFLOADING OPERATORS CAN SAVE $14 MILLION YEARLY THROUGH DATA OFFLOADING
OPERATORS CAN SAVE $14 MILLION YEARLY THROUGH DATA OFFLOADINGGreen Packet
 
IRJET- An Incentive Framework for Cellular Traffic Offloading
IRJET-  	  An Incentive Framework for Cellular Traffic OffloadingIRJET-  	  An Incentive Framework for Cellular Traffic Offloading
IRJET- An Incentive Framework for Cellular Traffic OffloadingIRJET Journal
 
Extended summary of "Cloudy with a chance of short RTTs Analyzing Cloud Conne...
Extended summary of "Cloudy with a chance of short RTTs Analyzing Cloud Conne...Extended summary of "Cloudy with a chance of short RTTs Analyzing Cloud Conne...
Extended summary of "Cloudy with a chance of short RTTs Analyzing Cloud Conne...IsabellaFilippo
 
Quick Routing for Communication in MANET using Zone Routing Protocol
Quick Routing for Communication in MANET using Zone Routing ProtocolQuick Routing for Communication in MANET using Zone Routing Protocol
Quick Routing for Communication in MANET using Zone Routing Protocolijceronline
 
M-Lab Study of ISP Interconnection
M-Lab Study of ISP InterconnectionM-Lab Study of ISP Interconnection
M-Lab Study of ISP InterconnectionEd Dodds
 
CPCRT: Crosslayered and Power Conserved Routing Topology for congestion Cont...
CPCRT: Crosslayered and Power Conserved Routing Topology  for congestion Cont...CPCRT: Crosslayered and Power Conserved Routing Topology  for congestion Cont...
CPCRT: Crosslayered and Power Conserved Routing Topology for congestion Cont...IOSR Journals
 
Edge device multi-unicasting for video streaming
Edge device multi-unicasting for video streamingEdge device multi-unicasting for video streaming
Edge device multi-unicasting for video streamingTal Lavian Ph.D.
 
REUSABILITY-AWARE ROUTING WITH ENHANCED SECURE DATA TRANSMISSION USING HOP-BY...
REUSABILITY-AWARE ROUTING WITH ENHANCED SECURE DATA TRANSMISSION USING HOP-BY...REUSABILITY-AWARE ROUTING WITH ENHANCED SECURE DATA TRANSMISSION USING HOP-BY...
REUSABILITY-AWARE ROUTING WITH ENHANCED SECURE DATA TRANSMISSION USING HOP-BY...AM Publications,India
 
An Optimal Algorithm For Relay Node Assignment In Cooperative Ad Hoc Networks
An Optimal Algorithm For Relay Node Assignment In Cooperative Ad Hoc NetworksAn Optimal Algorithm For Relay Node Assignment In Cooperative Ad Hoc Networks
An Optimal Algorithm For Relay Node Assignment In Cooperative Ad Hoc NetworksNancy Ideker
 
E2IOPN_Elsevier.pdf
E2IOPN_Elsevier.pdfE2IOPN_Elsevier.pdf
E2IOPN_Elsevier.pdftaha karram
 
Types of Networks,Network Design Issues,Design Tools
Types of Networks,Network Design Issues,Design ToolsTypes of Networks,Network Design Issues,Design Tools
Types of Networks,Network Design Issues,Design ToolsSurabhi Gosavi
 
IRJET-A_AODV: A Modern Routing Algorithm for Mobile Ad-Hoc Network
IRJET-A_AODV: A Modern Routing Algorithm for Mobile Ad-Hoc NetworkIRJET-A_AODV: A Modern Routing Algorithm for Mobile Ad-Hoc Network
IRJET-A_AODV: A Modern Routing Algorithm for Mobile Ad-Hoc NetworkIRJET Journal
 
An Insight Into The Qos Techniques
An Insight Into The Qos TechniquesAn Insight Into The Qos Techniques
An Insight Into The Qos TechniquesKatie Gulley
 
Transfer reliability and congestion control strategies in opportunistic netwo...
Transfer reliability and congestion control strategies in opportunistic netwo...Transfer reliability and congestion control strategies in opportunistic netwo...
Transfer reliability and congestion control strategies in opportunistic netwo...IEEEFINALYEARPROJECTS
 
JAVA 2013 IEEE NETWORKING PROJECT Transfer reliability and congestion control...
JAVA 2013 IEEE NETWORKING PROJECT Transfer reliability and congestion control...JAVA 2013 IEEE NETWORKING PROJECT Transfer reliability and congestion control...
JAVA 2013 IEEE NETWORKING PROJECT Transfer reliability and congestion control...IEEEGLOBALSOFTTECHNOLOGIES
 

Similar to International R/E Routing (v1.0) (20)

Standard Protocols for Heterogeneous P2P Vehicular Networks
Standard Protocols for Heterogeneous P2P Vehicular NetworksStandard Protocols for Heterogeneous P2P Vehicular Networks
Standard Protocols for Heterogeneous P2P Vehicular Networks
 
How Web Analytics Can Help Businesses Or Organizations...
How Web Analytics Can Help Businesses Or Organizations...How Web Analytics Can Help Businesses Or Organizations...
How Web Analytics Can Help Businesses Or Organizations...
 
Wiki2010 Unit 4
Wiki2010 Unit 4Wiki2010 Unit 4
Wiki2010 Unit 4
 
The International Journal of Engineering and Science (The IJES)
The International Journal of Engineering and Science (The IJES)The International Journal of Engineering and Science (The IJES)
The International Journal of Engineering and Science (The IJES)
 
CONCEPTUAL FRAMEWORK OF REDUNDANT LINK AGGREGATION
CONCEPTUAL FRAMEWORK OF REDUNDANT LINK AGGREGATIONCONCEPTUAL FRAMEWORK OF REDUNDANT LINK AGGREGATION
CONCEPTUAL FRAMEWORK OF REDUNDANT LINK AGGREGATION
 
OPERATORS CAN SAVE $14 MILLION YEARLY THROUGH DATA OFFLOADING
 OPERATORS CAN SAVE $14 MILLION YEARLY THROUGH DATA OFFLOADING OPERATORS CAN SAVE $14 MILLION YEARLY THROUGH DATA OFFLOADING
OPERATORS CAN SAVE $14 MILLION YEARLY THROUGH DATA OFFLOADING
 
IRJET- An Incentive Framework for Cellular Traffic Offloading
IRJET-  	  An Incentive Framework for Cellular Traffic OffloadingIRJET-  	  An Incentive Framework for Cellular Traffic Offloading
IRJET- An Incentive Framework for Cellular Traffic Offloading
 
Extended summary of "Cloudy with a chance of short RTTs Analyzing Cloud Conne...
Extended summary of "Cloudy with a chance of short RTTs Analyzing Cloud Conne...Extended summary of "Cloudy with a chance of short RTTs Analyzing Cloud Conne...
Extended summary of "Cloudy with a chance of short RTTs Analyzing Cloud Conne...
 
Quick Routing for Communication in MANET using Zone Routing Protocol
Quick Routing for Communication in MANET using Zone Routing ProtocolQuick Routing for Communication in MANET using Zone Routing Protocol
Quick Routing for Communication in MANET using Zone Routing Protocol
 
M-Lab Study of ISP Interconnection
M-Lab Study of ISP InterconnectionM-Lab Study of ISP Interconnection
M-Lab Study of ISP Interconnection
 
CPCRT: Crosslayered and Power Conserved Routing Topology for congestion Cont...
CPCRT: Crosslayered and Power Conserved Routing Topology  for congestion Cont...CPCRT: Crosslayered and Power Conserved Routing Topology  for congestion Cont...
CPCRT: Crosslayered and Power Conserved Routing Topology for congestion Cont...
 
Edge device multi-unicasting for video streaming
Edge device multi-unicasting for video streamingEdge device multi-unicasting for video streaming
Edge device multi-unicasting for video streaming
 
REUSABILITY-AWARE ROUTING WITH ENHANCED SECURE DATA TRANSMISSION USING HOP-BY...
REUSABILITY-AWARE ROUTING WITH ENHANCED SECURE DATA TRANSMISSION USING HOP-BY...REUSABILITY-AWARE ROUTING WITH ENHANCED SECURE DATA TRANSMISSION USING HOP-BY...
REUSABILITY-AWARE ROUTING WITH ENHANCED SECURE DATA TRANSMISSION USING HOP-BY...
 
An Optimal Algorithm For Relay Node Assignment In Cooperative Ad Hoc Networks
An Optimal Algorithm For Relay Node Assignment In Cooperative Ad Hoc NetworksAn Optimal Algorithm For Relay Node Assignment In Cooperative Ad Hoc Networks
An Optimal Algorithm For Relay Node Assignment In Cooperative Ad Hoc Networks
 
E2IOPN_Elsevier.pdf
E2IOPN_Elsevier.pdfE2IOPN_Elsevier.pdf
E2IOPN_Elsevier.pdf
 
Types of Networks,Network Design Issues,Design Tools
Types of Networks,Network Design Issues,Design ToolsTypes of Networks,Network Design Issues,Design Tools
Types of Networks,Network Design Issues,Design Tools
 
IRJET-A_AODV: A Modern Routing Algorithm for Mobile Ad-Hoc Network
IRJET-A_AODV: A Modern Routing Algorithm for Mobile Ad-Hoc NetworkIRJET-A_AODV: A Modern Routing Algorithm for Mobile Ad-Hoc Network
IRJET-A_AODV: A Modern Routing Algorithm for Mobile Ad-Hoc Network
 
An Insight Into The Qos Techniques
An Insight Into The Qos TechniquesAn Insight Into The Qos Techniques
An Insight Into The Qos Techniques
 
Transfer reliability and congestion control strategies in opportunistic netwo...
Transfer reliability and congestion control strategies in opportunistic netwo...Transfer reliability and congestion control strategies in opportunistic netwo...
Transfer reliability and congestion control strategies in opportunistic netwo...
 
JAVA 2013 IEEE NETWORKING PROJECT Transfer reliability and congestion control...
JAVA 2013 IEEE NETWORKING PROJECT Transfer reliability and congestion control...JAVA 2013 IEEE NETWORKING PROJECT Transfer reliability and congestion control...
JAVA 2013 IEEE NETWORKING PROJECT Transfer reliability and congestion control...
 

More from International Networking at Indiana University

More from International Networking at Indiana University (16)

ACE: Opportunities For African Connectivity To The USA
ACE: Opportunities For African Connectivity To The USAACE: Opportunities For African Connectivity To The USA
ACE: Opportunities For African Connectivity To The USA
 
IN@IU - Who We Are & Where We're At
IN@IU - Who We Are & Where We're AtIN@IU - Who We Are & Where We're At
IN@IU - Who We Are & Where We're At
 
Pakistan Network Connectivity via ACE-TP3
Pakistan Network Connectivity via ACE-TP3Pakistan Network Connectivity via ACE-TP3
Pakistan Network Connectivity via ACE-TP3
 
TransPAC3 2010 NSF Annual Report
TransPAC3 2010 NSF Annual ReportTransPAC3 2010 NSF Annual Report
TransPAC3 2010 NSF Annual Report
 
ACE and TransPAC3 - Cooperative Partnerships to facilitate Global R/E Collabo...
ACE and TransPAC3 - Cooperative Partnerships to facilitate Global R/E Collabo...ACE and TransPAC3 - Cooperative Partnerships to facilitate Global R/E Collabo...
ACE and TransPAC3 - Cooperative Partnerships to facilitate Global R/E Collabo...
 
Catalysing inter-regional network-enabled collaboration through workshopping ...
Catalysing inter-regional network-enabled collaboration through workshopping ...Catalysing inter-regional network-enabled collaboration through workshopping ...
Catalysing inter-regional network-enabled collaboration through workshopping ...
 
Ace+tp3 smm sa sig
Ace+tp3 smm sa sigAce+tp3 smm sa sig
Ace+tp3 smm sa sig
 
TransPAC2 Workplan - IRNC Final Review
TransPAC2 Workplan - IRNC Final ReviewTransPAC2 Workplan - IRNC Final Review
TransPAC2 Workplan - IRNC Final Review
 
Us india workshop workshop program v 7.1
Us india workshop workshop program v 7.1Us india workshop workshop program v 7.1
Us india workshop workshop program v 7.1
 
US-India Workshop Report
US-India Workshop ReportUS-India Workshop Report
US-India Workshop Report
 
Tein3 safs talk
Tein3 safs talkTein3 safs talk
Tein3 safs talk
 
Research and Education Connectivity to Pakistan
Research and Education Connectivity to PakistanResearch and Education Connectivity to Pakistan
Research and Education Connectivity to Pakistan
 
Dedicated R&E Network Connects Pakistan And The US
Dedicated R&E Network Connects Pakistan And The USDedicated R&E Network Connects Pakistan And The US
Dedicated R&E Network Connects Pakistan And The US
 
International Networking @ Indiana University
International Networking @ Indiana UniversityInternational Networking @ Indiana University
International Networking @ Indiana University
 
BP Talk - IN@IU
BP Talk - IN@IUBP Talk - IN@IU
BP Talk - IN@IU
 
GLIF Geneva - ACE & TransPAC3
GLIF Geneva - ACE & TransPAC3GLIF Geneva - ACE & TransPAC3
GLIF Geneva - ACE & TransPAC3
 

International R/E Routing (v1.0)

  • 1. Version 1.0 3/13/2011 International R/E Routing Chris Robb – CIREN/TransPAC2 Engineer and Abilene Engineer James Williams – CIREN/TransPAC2 Principal Investigator This brief White Paper was prepared at the request of Kevin Thompson of the National Science Foundation. The objective of the paper is to provide a vehicle for discussion of the particular issues connected with “efficient” international routing. The paper is organized in seven sections as follows: 1. Description of the problem 2. Description of inter-networking in both R/E and commercial networks 3. Example of a R/E routing problem 4. Recommendations of immediate and mid-term activities that would address the problem 5. Longer-term issues and the evolution of R/E networking 6. Summary and Recommendations 7. Appendix ( which contains some additional detailed routing examples) 1. Problem Description Although labeled or “called” a routing problem, the issues surrounding inefficient traffic flow on international R/E network links have a number of different aspects. Examples of these follow. On occasion (and sometimes for long periods of time) traffic between two locations does not travel by the “most desirable” path. The desirability of a particular unidirectional path can vary based on the application requirements and preference of the network administrators. For example, certain applications respond well to low latency, but don’t necessarily require large amounts of bandwidth. Other applications are written to take advantage of bandwidth, but may put less emphasis on latency. Often times, these applications coexist within a particular network, creating the need for more granular path selection methodologies. Regardless of the definition of desirable, there are very few pervasive tools available to the operators of R/E networks to understand the extent to which undesirable traffic paths are being utilized. In addition to the widely implemented ability to traceroute into a network, several projects are providing useful tools that the R/E community should embrace. Primary amongst them is the operationally focused RouteViews project1, based at the University of Oregon. Building upon that, the Computer Networks Research Group’s2 BGPlay project3 provides the dimension of time, giving operators a 10 day view into RouteViews peer networks routing tables. In addition to these centralized and coordinated efforts, individual networks often have their own disjoint data collection tools and operational-focused debug tools for outsiders to utilize. Unfortunately, these individual network looking glasses and netflow data analysis tools are often difficult to reconcile in any programmatic fashion between disparate networks. 1 http://www.routeviews.org/ 2 http://www.dia.uniroma3.it/%7Ecompunet/ 3 http://bgplay.routeviews.org/bgplay/
  • 2. Version 1.0 3/13/2011 In addition to the lack of debug tools, engineers often challenged with understanding how they should handle the import and export of routes into or out of their network. Router vendors and standards bodies have provided an extensive set of methodologies to implement traffic engineering policies, but the community lacks the necessary metadata to align policy with a particular researcher’s desired outcome. Finally, there is limited communication among international R/E network operators except in times of emergency/crisis or perhaps near some particularly significant event such as SuperComputing or iGrid. Within the North American commercial network operators, NANOG4 provides a forum for NOC-NOC engineering and operational discussions. But, R&E participation in NANOG has historically been limited and the discussion is generally limited to North American issues, as one might expect from the name. In summary, the “routing problem” is not very well-defined and there are few, or at least not very well-known, tools for providing assistance/information to a not very well-organized group of R/E NOC operators. 4 http://www.nanog.org/
  • 3. Version 1.0 3/13/2011 2. Description of Inter-network routing in Research and Education networks and commercial networks There are significant differences between the commercial world (CW) and the R/E world with respect to routing and the drivers for “efficient” routing. Commercial networks In the CW very large networks (Tier1 ISPs) peer with each other and exchange traffic by a mechanism called Settlement Free Interconnects (SFI)5. With this type of peering large ISPs interconnect with each other in multiple geographic locations with no financial exchange between the ISPs. The theory behind SFI is that traffic between these large ISPs is approximately equal. Consequently, Tier1 ISP treat each other as equals. However, Tier1 ISPs regularly monitor these traffic flows to ensure that this concept of equal flow remains accurate. In general, Tier1 ISPs have several motivating factors to establish SFI interconnects: • Lowering transit costs • Increased control over routers; direct peering always yields more ability to shift traffic around • Decreased latency for their customers; the shorter distances between endpoints cause TCP backoff algorithms to allow an application more bandwidth- hence more traffic (and better performance) from their customers, whom they charge Further, there are several reasons why a network would choose not to SFI peer with another network: • Loss of potential revenue with a network you might otherwise charge for transit • Traffic load asymmetry. (i.e. if the other network provides service to a large group of content providers, the asymmetric nature of the traffic volume combined with a hot-potato routing methodology, may mean that you would bear the greater burden in the creation of a bidirectional data exchange; the traffic would be on your network longer than it would be on your peers network • Peering may strengthen a potential competitor, thus drawing away your customer base In the CW smaller networks (Tier2 ISPs) and some content providers purchase transit (access to routes) from larger Tier1 ISPs. This is a tiered network model, in contrast to the R/E flatter network model described below. There is a financial exchange between Tier2 and Tier1 networks. Consequently, it is in the interests both sides of this exchange to closely monitor traffic passing between networks and be certain that it passes in the most efficient and effective manner. In more recent years, smaller ISPs (cable modem providers, regional ISPs, etc.) and large content providers (Google, Apple, Microsoft) have begun to peer directly with each other in a settlement free context. This more distributed approach has somewhat reduced the importance of larger ISPs, though they are still critical as a “gateway of last resort” for networks that don’t have presence at all the major exchange points6. In the CW, there are real costs and real dollars paid for connectivity. Consequently, decisions are made based on cost minimizing considerations. Network provisioning closely matches network usage. Significant attention is paid to routing, routing tables and traffic balances. There is careful analysis of imbalances and possible routing problems. Each of the facets of peering are analyzed by engineers and lawyers before decisions are rendered and peering is established. If there is any uncertainty, the NANOG list is a useful reference. Research and Education Networks 5 http://en.wikipedia.org/wiki/Settlement-free_interconnect 6 http://en.wikipedia.org/wiki/Internet_Exchange_Point#North_America
  • 4. Version 1.0 3/13/2011 Research and Education networks are created, designed and provision quite differently (for quite different reasons) than commercial networks. The most significant difference between the CW and R/E worlds is that in the R/E world, bandwidth is usually (not always) purchased by government organizations via grants or consortiums. There is less incentive to reduce costs via more efficiency, whether in usage or routing. Essentially, by design, there is little connection between costs and usage. Furthermore, there is less incentive to manage bandwidth efficiently in the R&E networks due to the over provisioned nature of the R&E infrastructure. It is not uncommon for R/E networks to carry an average of 100Mbps of traffic on a 10Gbps connection. In a network over provisioned by a factor of 100, it is difficult to convince network operators to devote resources to the idea of efficiency. As is discussed later, this over provisioning is purposeful. Following on the point made above, in the R/E world links are provisioned for the “most extreme individual experiment” rather than to insure generally optimal and cost-effective service. Given the funding from governments or granting agencies and the fundamental purpose of R/E networks, this is not an unreasonable approach. The R&E world has the additional dynamic of catering to individual researcher’s needs. At any given time, though more frequently around large conferences such as Supercomputing7, network engineers may utilize several different methodologies to alter network paths to satisfy the needs of a particular researcher. Often times, this has the unfortunate side effect of becoming a permanent, and difficult-to-track feature of the network landscape. Depending on the willingness of the network engineer to properly address the problem, this network “gaming” will often pull significant portions of an organizations network in a direction that is well suited to a particular researcher, but not to the larger population. Additionally, this lack of coordination of routing is definitely due, in part, to network engineers simply not having a complete (or even partial) understanding of global routing. As was pointed out previously, there is both a lack of tools (or, more specifically, a lack of knowledge about available tools) and the lack of a forum for the international discussion of this issue. The issues above, when combined with the more general lack of strict legal definitions of appropriate peering and financial reparations, the traditional over-commitment of engineering resources and an (appropriate) focus of resources on providing means for the development of science collaborations, create a more relaxed attitude to network management and routing policy enforcement. Finally, international routing paths, and in fact entire international networks, may develop for reasons connected to political rather than technical objectives. It is quite common for countries to have two smaller bandwidth connections to the United States rather than one, most larger and possibly less expensive connection. Political reasons trump technical (and financial) reasons anytime. However, it is important to note that Research and Educations networks are designed to create opportunities and collaborations rather than simply pass/exchange traffic and generate revenue. Consequently, R/E networks are sized for anticipated users rather than existing users, greatly reducing the operational connection and attention between traffic and network size and costs. This fundamental difference between commercial and R/E networks argues strongly for a parallel but separate forum for the discussion of R/E routing issues and in no way diminishes the necessity for development of global routing visualization and management tools. 7 http://www.supercomputing.org/
  • 5. Version 1.0 3/13/2011 3. An Example of a R/E routing problem Figure 1: Merit Route Propagation to Abilene The diagram above illustrates a specific issue that was observed in March 2006 while troubleshooting a connectivity problem between Abilene (AS11537) and Merit (AS237). Specifically, traffic between AS11537 and AS237, which should travel directly between these nodes, was instead traveling through Korea! 1. Merit, an MREN connector, has direct peerings with both Abilene and MREN. It announces its routes to all it’s peers. 2. MREN has a peering with KREONET2, so it announces it’s customer routes. 3. KREONET2 has a transit agreement with KOREN to provide transit to US customers, so KREONET2 announces the Merit route to KOREN 4. KOREN announces it’s entire routing table, including non-customer routes to APAN 5. APAN trusts that KOREN will filter its non-customer routes out and passes all KOREN- learned prefixes to TransPAC2 in the US. 6. TransPAC2 trusts that APAN is only sending APAN customer prefixes so it passes all APAN-learned routes to Abilene. 7. Abilene now has received two advertisements. One via the direct MREN peering and one via the much longer path via Asia. 8. Abilene doesn’t have the Merit route in it’s inbound prefix filter facing Merit, so the direct MERIT-learned route is dropped. 9. Even if Abilene had the MERIT route in it’s filter, MERIT was tagging the route with Abilene community 11537:140, which causes it to be preferred with a localpref of 140. This is done because of a backup agreement with a separate US-based network not depicted.
  • 6. Version 1.0 3/13/2011 10. APAN tags all its routes with Abilene community 11537:160. This is passed to Abilene and causes APAN routes learned via TransPAC2 to be local-preferenced with a weight of 160- higher than the direct Merit local-preference. APAN tags their routes with 11537:160 to influence return traffic over TransPAC2 instead of over a separate circuit they maintain to the US. What are some of the issues that caused the problem above? A. KREONET2 should be indicating to KOREN which routes are transit routes and which routes are their internal customer routes B. KOREN should not announce non-customer routes to APAN. Or they should announce those routes with tags that allow APAN to filter out non-KOREN customer routes C. APAN should be able to distinguish between it’s customers routes and its customers transit routes D. Abilene has local-preference values that can be manipulated remotely. An engineer needs to be familiar with Abilene’s internal BGP community infrastructure to successfully navigate the problem E. With the exception of Abilene’s direct peering with Merit, all networks along the path appear to have very loose restrictions on what networks they will accept from their BGP neighbors. In this case, as in many (all?) international routing problems, problem resolution is often relegated to multiple point to point discussions that resolve individual problems, but rarely address the larger issues. The routing problem is fixed. But, the underlying issues that lead to the problem remain. The lack of problem “broadcast” relegates the discussion to one-time, one-on-one immediate triage with little follow-up on the greater problem. Additionally, this lack of “broadcast” limits knowledge of the problem and the underlying issues to only the directly involved parties. 4. What immediate and mid-term steps can be taken to address these problems? The R/E community should move toward using international common technical forums to focus attention on the problem. Obvious suggestions are the APAN meetings, the Internet2 meetings, the TEIN2 meetings, the TERENA meetings and other international networking forums. There was a full-session discussion of international routing at APAN22 in Singapore (http://www.apan.net/meetings/singapore2006/proposals/APAN-Routing.html). A follow-up routing meeting is planned for the December Internet2 meeting in Chicago and for the APAN23 meeting in Manila. The R/E community should encourage use of (participation in) existing tools such as RouteViews to make current routing visible to the entire community. Again, see the notes from APAN22 referenced above. It was agreed at this meeting that all APAN countries who were not participating in RouteViews would participate by APAN23. The R/E community should encourage international NOC-NOC engineering and operational cooperation. To that end, rather than simply grumbling about inefficient or incorrect routing engineers and attempting one-to-one solutions NOC operators need a mechanism (forum) to address the problems, in “broadcast” mode. The Coordinating Committee for Internatinal Research Networking (CCIRN) has taken a step toward addressing this lack of a forum problem. At the April 2006 meeting the CCIRN charged Indiana University and Internet2 with the development of the Research and Education Network Operators Group (RENOG). See the following for details: http://www.ccirn.org/CCIRN2006- minutes-final.pdf. RENOG, is a light weight “mailing list organization” designed to provide the ability to broadcast both international network routing problems and the solutions to these
  • 7. Version 1.0 3/13/2011 problems. At the present time, the success of RENOG, as measured by traffic on the mailing list is limited. Perhaps this approach is too limited or another mechanism for addressing this problem needs to be tried. Our view is that the ability of the international routing community to interact more publicly is particularly important because of the parallel, but fundamentally different, goals of commercial and R/E networks. 5. Longer-term issues and the evolution of R/E networking How will the direction of evolution of R/E networking will effect “routing” ? That is impossible to predict (at least for us). We offer three questions that we view as important speculation relating to international R/E networks and routing. 1. Will switched on-demand bandwidth become a primary mechanism for handling large data flows? If so, then routing will take on an additional meaning. Routing may mean allocation of light paths rather than only the direction of packets. 2. What is the future of backbone networks? Will networking become a collection of regional networks connected by on-demand circuits with much smaller backbone connections, thus increasing the number of entities involved in resolving performance problems and routing between these regional organizations? Or, will the continually declining cost of raw bandwidth mean that backbone networks will simply expand and offer services as good as on-demand networks in a much simpler environment? 3. How will R/E networking in the United States be delivered and how will this effect international networking and routing? The NLR-Interent2 controversy in the US certainly has the potential to make international routing more confusing, as international networks connect to one or both major US R/E networks, not considering the already existing issues of FedNets in the US. As US entities “take sides” in the NLR-I2 dispute, it is possible that a connection between two US universities, one connected to NLR and one connected to Abilene/I2 would be routed to an international exchange point and then back into the US rather then routed directly over US terrestrial infrastructure.
  • 8. Version 1.0 3/13/2011 6. Summary and Recommendations Commercial networks and R/E networks are built for different purposes and are managed to different objectives. Commercial networks are concerned about costs and efficiency. R/E networks are designed to create collaborations and support research rather than efficiently exchange traffic or generate revenue. However, while the managing principles may be different, the underlying needs of routing engineers in both types of networks are surprisingly similar. This paper makes three recommendations. 1. Networks need to participate in (make use of) existing routing tools such as RouteViews. This directly relates to Recommendation #2. 2. The issue of international routing needs to be given attention on the international stage (at meetings such as CCIRN, APAN and TERENA). 3. The international R/E community needs a regular forum where routing issues can be discussed and information exchanged among international routing engineers.
  • 9. Version 1.0 3/13/2011 Appendix A The following exhibits are further examples to illustrate some of the unique challenges that the authors of this paper have been involved with. They have been chosen as representative examples that highlight a facet of the discussion and are not meant to represent the complete set of challenges that have arisen in the past several years. Exhibit A: R/E “Fish” Traffic Given the common US interconnection methodology of bringing connectors behind large regional aggregation points (called gigapops), and given that the service offerings of most gigapops include both commodity and R/E peerings, the R/E community has faced the following asymmetric behavior for quite a number of years. Consider the following topology: Figure 2: R/E Fish Traffic In the topology above, US Gigapop A provides commodity connectivity for one of it’s connectors. It does not provide R/E access, so it’s routes aren’t sent to the R/E network (nor would the R/E network accept them). Gigapop B provides both R/E and commodity service to its connector. As such, it advertises its customer’s routes to both the commodity and the R&E network. Gigapop A receives these routes from both the commodity and R/E peerings. Gigapop A will typically localpreference routes from the R/E network higher than routes on the commodity network. As such, Gigapop A prefers the R/E path toward Gigapop B’s customer, and Gigapop A prefers the Commodity path for Gigapop A’s customer. This is a fairly old problem with some well-known solutions. Most US Gigapops today have solved the issue by using Virtual Routing and Forwarding (VRFs) tables so they can keep distinct sets of destinations for use by their connectors. The question will be complicated somewhat in the coming months as US R/E networks (notably Abilene and the National TransitRail project) are looking toward providing all or part of commodity routes as part of their service offering.
  • 10. Version 1.0 3/13/2011 Exhibit B: TEIN2 Routing In late 2005, the TEIN2 network was brought online, creating the following topology: Figure 3: TEIN2 Routing TEIN2 administrators had some very specific traffic flow directions in mind when creating the network traffic engineering policy. One specific desire was for US-bound traffic to flow in the direction of APAN, with GEANT as a backup. This decision was based on the network latency and available bandwidth between the various locations. As such, the TEIN2 network receives US network routes from both GEANT and from APAN. In addition, TEIN2 provides backup connectivity between APAN and GEANT, so it passes GEANT-learned and APAN-learned routes between the two networks. A difficulty arises when the set of routes that Abilene advertises to GEANT isn’t congruent with the set of routes it advertises to APAN. Such a problem arose in early 2006 when GEANT engineers noticed that their traffic to various US Federal networks was taking the very long path via Asia to get to the United States. At issue was the Internet2 Federal Network policy. Due to some contractual constraints with Abilene’s carrier, Abilene cannot do blanket advertisements of federal networks to its international peers. Some networks have agreements in place to receive all or some of the set of federal networks. TransPAC2 was such a network with an agreement to receive all Federal network routes. GEANT was only receiving a subset of those destination advertisements. Since TransPAC2 passes those routes to APAN, and APAN passes all their US-learned routes to TEIN2 (for the GEANT backup), this created an incongruent set of routes that GEANT received between it’s TEIN2 and Abilene peerings. Given the latency involved, a commodity path was more preferable from their perspective.