Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. 2012 13th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing Experiments on Multi-Layer Network Virtualization towards the Software Defined Transport Network Akeo Masuda, Akinori Isogai, Daisaku Shimazaki, Yoshihiko Uematsu and Atsushi Hiramatsu NTT Network Service Systems Laboratories, NTT Corporation Musashino-shi, Tokyo, Japan Email: Abstract—This paper proposes a novel architecture which change the route of the existing flows. In this case, they will enables software defined networking not only at the routing be able to achieve better performance by acquiring additional layer but also at the transport layer. Proposed architecture network resources, or optimizing the topology of tunnel paths. provides multiple SDTNs with wide range of controllability level, in spite that the SDTNs coexist upon a shared multi- For another example, they can achieve high availability if layered network infrastructure. We have conducted a nation- they can prepare a SRLG(Shared Risk Link Group)-aware wide experiment where we have provided SDTNs to practical protection path at the transport layer, designed in accordance users such as broadcasting studios. Through the experiments, we to the design of server redundancy. have successfully verified the resource management mechanism and network control functionalities. On the other hand, network carriers do not devote their net- work resources to a single service or single user. They logically I. I NTRODUCTION slice their network resources to launch a new service including inter-cloud connection, and provide the portion of the slice to Recently the main players and the drivers of the devel- each user. Sharing the infrastructure by multiple usage of the opment of networking technologies seem to be shifting to network is usually done in most of the network providers to operators and users of datacenter. Software developers of cloud keep their competitiveness by the cost efficient operation of operators are eager to totally program the operation of not their infrastructure. This can be seen as a virtualization of the only their computing equipments, but also the network. Inside network infrastructure. The difficulty of network virtualization and between the datacenters, there are numerous dataflows is to offer the programmability at the same time. between virtual machines (VMs) running upon numerous com- puters, and they keep being generated and changed dynami- Speaking generally, network providers do not desire to allow cally. The concept of software defined network is expected users to freely configure the network equipments. It may to release the network operation from time-consuming tasks cause serious problem if a certain user of the network directly of manual configuration of each network equipment along configures the functionalities of the routers and switches. which the flow traverses. This enables programmed control It will prevent the fair use of the network among multiple of the dataflow routing in order to achieve optimization, users and services, and causes conflict between the controls scalability and resiliency of the network, similar to the way of from multiple users. In order to offer programmability of the management where the cloud operators program the usage of transport layer, we need a new technology to overcome this computing resources. OpenFlow[1] is expected to be the main problem. enabler of SDN (Software Defined Network). This technology Several works had addressed the architecture of total control lets cloud operators to explicitly designate the route at flow of the network including the SDN layer and transport layer [2], level granularity, and slice the network capacity to multiple [3]. However, previous researches only focus on the integrated independent tenants. control of both layers by unification of the control plane. However, at least in the past, SDN had been seen to be only Software defined control of the transport layer where multiple the enabler of control function at the flow routing level. We users share the infrastructure is still an open issue at this believe that controllability should be enhanced deeply to the moment. transport level for full utilization of network resources. The main contribution of this paper is to address an architecture We propose the SDTN architecture that enables network of SDTN (Software Defined Transport Network) that enables virtualization in the transport layer, which provides secure virtualization and programmability of the transport layer. shared use and programmability at the same time to multiple users. Virtualization and programmability is the major require- ments for future network operation. From the user’s point This paper is organized as follows. Next section explains of view, they can be able to achieve much flexibility, high the architecture of SDTN. In section III, we illustrate the performance and resiliency if they can also program the design of the experimental network. Section IV discuses about transport layer of the inter-cloud network. For example, they what we confirmed through the experiments. Finally, section may lack of bandwidth in case they newly generate a flow or V concludes the paper.978-0-7695-4761-9/12 $26.00 © 2012 IEEE 661DOI 10.1109/SNPD.2012.134
  2. 2. II. T HE S OFTWARE D EFINED T RANSPORT N ETWORK Virtual Network (VN) #1 Virtual Network (VN) #2A. SDTN Architecture VN Topology SDTN architecture is designed to enable network virtualiza-tion in the transport layer, that provides secure shared use and (4) Setup Pathsprogrammability the same time to multiple users. Key concept SDTN (Allocatedis that we provide multiple SDTNs upon a shared multi- Resources)layered network infrastructure for users. Note that “users” SDTN Optical Pathscould be the cloud tenants, cloud service providers and other Controlleroperators of network services. (3) Allocate resources SDTN is made of set of network resources such as links,wavelengths, unit of bandwidth and switching capabilities.Each unit of resources is assigned permission to users. Users (2) Configure VN#1 VN#2 Shared Private permission to VNs Dedicated Dedicatedare allowed to setup optical and packet transport paths mak- Resource Linking use of the resources that are assigned permission to Routerthemselves. This ensures the portion of the network to be ᾉindependently controlled without any contention. OXC The key component of the architecture is the Physical Net- PN Managerwork Manager (PN Manager), which is the unified controller (1) Collect Resource Info L2 switch, (OSPF-TE/LLDP) Physical network(PN) Routerof the optical network. It provides functions for the usersto invoke network control such as resource allocation and Fig. 1. Construction of VNTs using the resources allocated from the PNpath setup in order to program their own software of network Manager.topology designing (Fig. 1). PN Manager provides API [4] forSDTN operators to develop a software to control their SDTN. Network providers are able to optimize the operation of theirnetwork infrastructure. For example, optimization of resource layer-2 link will be provided by connecting a pair of layer-2allocation to each slice according to the traffic demands will switches by an optical path through layer-1 nodes using theprovide statistical multiplexing effect. Furthermore, sharing layer-1 resources (e.g. GMPLS TE-links). Then, those layer-2redundant resources prepared for forecasted future demand and links can be seen as resources to setup a path in the layer-2,detour routes in case of failures will provide high efficiency by which the layer-3 IP routers can be connected in orderof capital expenditure. Since the SDTN is logically formed to form an IP link. In this manner, VNT in a certain layerby set of circuits that can be provisioned automatically, it also can be provided dynamically and recursively. SDTNs areenables fast launch of new services by making use of available provided to the users as the VNTs at the desired resources. It may provide survivability of services in Consequently, layer-2 and 3 SDTNs are provided by uti-case of disaster, by letting the slices to share a small portion lizing network resources of layer-1 and 2. Resources usedof the remained part of the network. to setup layer-1 optical paths are routers, OXCs, fibers and On the other hand, SDTN benefits the users in terms of the wavelengths. They are handled in a unit called TE-link whichprogrammability of the transport network. As mentioned in defined in the GMPLS technology. We can describe andthe previous section, users can optimize the transport layer as utilize the resource to setup optical paths because proper-well as the flow routing layer. Cloud operators can be able to ties of TE-link provides sufficient information of the linkprogram the total system including the computing resources, such as connected node address, link address, maximum andflow routing, underlying circuits, and the amount of allocated minimum reservable bandwidth, switching capability (fiber,network resource to configure the circuits. lambda, TDM and packet), SRLG, and so on. Information ofB. Recursive VNT construction upon multi-layer network in- the existing resources are automatically collected by listeningfrastructure to OSPF-TE[6] advertisement in GMPLS. For the physical network infrastructure, we assume a multi- Resources used to setup layer-2 paths can be handled bylayer network which is consisted of layer-1, 2, 3 nodes such L2SC TE-links that are also defined in GMPLS. However, asas optical cross-connects (OXCs), L2 switches and IP routers. L2SC is not actually popular in the market, we can make use ofThis can be prepared with ordinary products that are already ethernet related technologies such as LLDP (IEEE 802.1AB).available in the market. To be precise, there are no exact technologies to be named In each layer, resources are defined in order to as layer-2 path. What is needed here is actually a technologysetup a path. Here we incorporate the notion of to setup a packet transport path to slice the huge bandwidthV irtualN etworkT opology(V N T )[5]. When a pair of provided by the layer-1 path that is too much to offer to users.nodes in a certain layer is connected with a path in the lower Here we can employ MPLS-TP LSPs, or S-VLANs defined inlayer, it will form a link in the upper layer. For example, PBB (Provider Backbone Bridge) configured with rate limits. 662
  3. 3. IP Link Allocated exclusively Allocation to each SDTN #A #B #C Layer-3 Return Resource VN Operator Path setup Path Release Obtain/Release Equivalent Path PN Operator #A #A #B #B #C L2 path between Permission Dedi- Shared Dedi- Dedi- L3 Router cated cated cated Layer-2 VN Operator Assignment of Permission PN Operator Resource detection Initially permitted only to PN (OSPF-TE/LLDP) administrator L1 path between L2 switches Layer-1 Fig. 3. Resource access control model. VN Operator L1 path between L2 switches PN Operator to design the SDTN at that level. Therefore, abstraction of Fig. 2. Multi-layer network resource state machine. network resources may provide much usefulness to the users. We assume following three types of abstraction: type-T , a topology which contains links and nodes, type-P , a set ofC. Multi-layer resource state machine point-to-point paths, and type-S, a virtual switch. For each unit of resource, the administrator of the physical In type-T , users are provided with links and nodes in ordernetwork will apply permission for SDTNs to obtain them. to setup transport paths by their own. Users are provided aUsing the obtained resources, SDTN operators are allowed to large range of freedom to control the network, such as de-setup paths in order to form their own VNT. Fig.2 shows the signing multi-layer topology optimization or capacity planningstate machine we have designed for the multi-layer resource according to the traffic demands, and provisioning protectionmanagement model. Users are permitted to obtain layer-1 paths. This type can be seen as an abstraction at the mostresources. Using the layer-1 resources, users can setup layer-1 lower level.paths between layer-2 or 3 node pairs in order to form links In type-P , users are provided with a set of point-to-pointat layer-2 or 3. In addition, resources can also be assigned to paths. Users only request the paths that connect the desiredthe administrator of the total physical network infrastructure, endpoint in order to connect the nodes owned by the users.which we call the PN (Physical Network) operator. PN oper- Users do not have the level of controllability as much as type-ator can setup layer-1 paths to produce layer-2 resources, and T , but still it is their work to design the topology formed bythen assign permission to users. By this, users are also able to the provided paths and their nodes.start from obtaining layer-2 resources in order to form layer-3 In type-S, the provided SDTN is seen as a single switch.VNT by connecting IP routers by layer-2 paths. Users are provided with connection points, as if they are Resources are permitted as either dedicated or shared. provided with several ports of a big switch. Users only need toShared resources can be noticed by multiple VNs, but it will connect their equipments to those ports, and the packets willbe allocated to only one of that VNs. Sharing the unallocated be forwarded to any of the points they have connected. Thisresources enables capital cost reduction of the physical net- type can be seen as an abstraction at the most higher infrastructure, by sharing the redundant resource that III. NATION - WIDE EXPERIMENTAL NETWORKshould have been prepared for each of the network service ifno virtualization is adopted. Fig.3 shows the resource access As shown in Fig.4, we have implemented a network in-control model. frastructure for experiments, upon a national R&E network Balance of the amount of resources allocated to each virtual in Japan, called JGN-X[7]. Through June 2011 to Februarynetwork can be modified flexibly by changing the permission 2012, we have connected four OXCs, ten Layer-2 switches,of each resource. This enables efficient utilization of the and six IP routers upon JGN-X. Scale of the network in-resources in accordance to the change in traffic demands. frastructure changed at each experiment event. At most the number of nodes was 14. Network spanned over the nation,D. Resource abstraction and variety of controllability level from Hokkaido to Okinawa, which are the north and south Here we discuss on abstracting the network resources. end of Japan. Some of the links had 10 Gbps capacity, andPreviously we explained that users form SDTN for them by others had 1 Gbps.themselves, utilizing the resources obtained at the granularity We have implemented an SDTN controller software withof links. However, we should be aware that not all of the GUI that invoke the PN Manager API in order to let SDTNusers of the network require controllability at that level. Some users to obtain resources and setup paths.of them don’t need to, some of them don’t want to, and For some of the users, layer-1 resources were directlysome of them are not the network experts skilled enough allocated. Those users formed IP links by connecting IP router 663
  4. 4. 2) Dynamic resource allocation: In the experiment event in Sapporo February 2012, we have provided layer-2 SDTNs to four TV broadcasting studio groups. As bandwidth capacity of most of Koganei the links was 1 Gbps, we sliced the network to provide SDTNs Otemachi with limited capacity of 150 Mbps each. As the topologies of Fukuoka SDTNs were different according to the required access point among users, reserved and residual capacity at each physical links were different. Residual capacity was maintained as a bandwidth pool that can be allocated dynamically according to user’s requests. Two of the broadcasting studios turned out to require larger Okinawa Osaka Musashino amount of bandwidth capacity for their video transmission. In one case, they needed to simultaneously transmit video file IP Router Layer-2 Switch OXC for remote TV program editorial and live streaming for news program. Total bandwidth usage exceeded the default alloca- Fig. 4. Experimental network infrastructure implemented upon JGN-X. tion of 150 Mbps, so we additional capacity was allocated to them to enhance the limit to be 200 Mbps. In another case, a broadcasting studio desired to try a new video encoder thatpairs with GMPLS optical paths. There was another case that consumes bandwidth of 150 Mbps. Also in this case, we addedthe PN operator connected layer-2 Ethernet switches with allocation to let it enhance to 200 Mbps. These operations ofoptical paths in order to produce layer-2 resources. These resource allocation was also done during the time when otherresources were divided by setting up point-to-point S-VLANs SDTN users were transmitting commercial video stream.with upper rate limit. SDTNs consisted of set of S-VLANs 3) Abstraction variety: Through the experiment, we werewere provided to users. Users setup C-VLANs between the able to test the usage of SDTNs with all three variations ofdesired access points in order to transmit their data flows. abstraction level which mentioned above in section II. Although we haven’t completed the evaluation from the SDTN for a research project that tested their proposal ofperformance point of view, we report that time needed to high-efficiency layer-4 protocol was provided in the mannersetup a single optical path was about 15 seconds, and that of type-T , a topology which contains links and nodes, Weof a single point-to-point S-VLAN connectivity was around also provided measurement functions that the user were able10-15 seconds. Note that these results may differ according to to check the precise performance in terms of data rate,conditions. These are expected to be shortened by additional jitter at multiple measurement point implemented inside thetuning efforts. network. By analysis of the performance degradation point, they were able to optimize the transmission path. As a result, IV. R ESULTS this user was successful in achieving their highest record of Through the network operation in the experiment which was performance. This experiment can be seen as a successful caseclose to practical use, we successfully confirmed the feasibility that the high level of controllability of SDTN had providedand the benefits provided by our control architecture. benefit to the user.A. Multiple SDTN operation Another experiment that we provided a SDTN for users to demonstrate their OpenFlow enabled equipments. As the user Totally we had provided 11 SDTNs to users including side nodes were capable of controlling the flow route withexperiment project of new generation network technologies, OpenFlow technology, they only needed a path to connect theirdemonstrations for international conference, and live video nodes. This experiment can be seen as a use case of type-P ,transmission for commercial TV program broadcasting. At a set of point-to-point paths. In addition, we have successfullymost, five SDTNs were operated simultaneously. 1) Independent control of multiple SDTNs: All of the users tested path switchover in the transport layer. As the transportof 11 SDTNs were able to completely carry out their event path was provided by Ether-over-MPLS circuit, the switchoverof such as experiment, demonstration, and broadcasting. This did not cause any packet losses, and we confirmed the isolatedmeans that, we confirmed that user traffic was successfully control in independent layers.isolated in terms that no user experienced any trouble caused Finally, SDTNs provided to most of the broadcasting stu-by network control or data traffic of other users that share the dios, except the ones that operated the topology changenetwork infrastructure. Indeed, two SDTNs used by broad- described above, was a case of resource abstraction type-S,casting studios had changed the topology of their SDTN in which the network can be seen as a virtual switch. Most ofadvance of a planned construction work that was known to the broadcasting studios do not care for the inner topologyforce outage of the connection at a certain physical link. of the network. They only desire to connect the camera crewEven in this case, network control to change topology had sites and editorial facilities, and broadcasting stations to thecarried out while broadcasting studios using other SDTNs access points of the network. In this experiment, topology ofwere transmitting their commercial video stream. the SDTN was designed and operated by physical network 664
  5. 5. operator. However, ideally the network should automaticallydesign and setup an SDTN with the optimal topology withoptimal bandwidth capacity according to the connectivityrequirements submitted from the users. This case had impliedmany future issues for us of the value-adding functions thatthe transport network can provide. V. C ONCLUSION SDTN is a slice of a physical network that can be controlledindependently by the user of it. As mentioned in this paper,we believe there should be many variety of how the SDTNis provided to the users, in terms of abstraction level andcontrollability level. The way to provide the SDTN should bedifferent according to the user’s requirements. For example,advanced users will be able to totally program the network ateach layer of the network, by making use of SDTN functionsin addition to the SDN functions at the flow routing layer.On the other hand, for users that do not care about the innernetworking technologies, it may be beneficial for them if thenetwork can offer useful functions to users such as automaticcapacity designing and topology optimization. The experimentresults shown in this paper are valuable findings derived frompractical use cases that suggests us of the future researchtopics. Further discussions are expected to be focused ondefining the total architecture and the interfaces between usersystems and the SDTN controllers such as our PN Manager. ACKNOWLEDGEMENTS The authors would like to thank Dr. Kazumasa Kobayashi,Yoshihiko Kanaumi and all of the JGN-X related researchersand engineers in NICT for strongly supporting us on theexperiments. R EFERENCES[1] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson, J. Rexford, S. Shenker, and J. Turner, “OpenFlow: enabling innovation in campus networks,” SIGCOMM Comput. Commun. Rev., vol. 38, no. 2, pp. 69–74, Mar. 2008.[2] S. Das, G. Parulkar, N. McKeown, P. Singh, D. Getachew, and L. Ong, “Packet and circuit network convergence with OpenFlow,” in Optical Fiber Communication Conference. Optical Society of America, 2010, p. OTuG1.[3] S. Azodolmolky, R. Nejabati, E. Escalona, R. Jayakumar, N. Efstathiou, and D. Simeonidou, “Integrated OpenFlow–GMPLS control plane: an overlay model for software defined packet over optical networks,” Opt. Express, vol. 19, no. 26, pp. B421–B428, Dec 2011.[4] A. Masuda, A. Isogai, T. Miyamura, K. Shiomoto, and A. Hiramatsu, “Application-defined control of virtual networks over IP-optical net- works,” in CNSM. IEEE, 2011, pp. 1–6.[5] K. Shiomoto, D. Papadimitriou, J. L. Roux, M. Vigoureux, and D. Brun- gard, “Requirements for GMPLS-Based Multi-Region and Multi-Layer Networks (MRN/MLN),” RFC 5212 (Informational), Internet Engineering Task Force, Jul. 2008.[6] K. Kompella and Y. Rekhter, “OSPF Extensions in Support of Gener- alized Multi-Protocol Label Switching (GMPLS),” RFC 4203 (Proposed Standard), Internet Engineering Task Force, Oct. 2005.[7] “New generation network testbed JGN-X,” 665