Published on

  • Be the first to comment

  • Be the first to like this


  1. 1. Resource Virtualisation of Network Routers Ross McIlroy, Joe Sventek (ross@dcs.gla.ac.uk, joe@dcs.gla.ac.uk) Department of Computing Science University of Glasgow
  2. 2. <ul><li>QoS in the Internet </li></ul><ul><li>Proposed Approach </li></ul><ul><li>Implementation of QuaSAR </li></ul><ul><li>Experimental Results </li></ul><ul><li>Experience Gained </li></ul><ul><li>Conclusions </li></ul>
  3. 3. The Problem <ul><li>The Internet provides an inherently best effort service </li></ul><ul><li>However, many recent Internet applications require Quality of Service guarantees </li></ul><ul><ul><li>Isochronous network flows </li></ul></ul><ul><ul><ul><li>Voice over IP </li></ul></ul></ul><ul><ul><ul><li>Streaming Video </li></ul></ul></ul><ul><ul><li>High Throughput applications </li></ul></ul><ul><ul><ul><li>Grid Utilities </li></ul></ul></ul><ul><ul><ul><li>Streaming Video </li></ul></ul></ul><ul><ul><ul><li>Peer to Peer </li></ul></ul></ul>
  4. 4. Previous Research into Network QoS <ul><li>Integrated / Differentiated Services </li></ul><ul><li>Traffic Engineering (e.g. MPLS) </li></ul><ul><li>Resource Reservation Protocols (e.g. RSVP) </li></ul><ul><li>However, less research into how router resources (e.g. processing time) should be provisioned to guarantee QoS to network flows </li></ul><ul><ul><li>Over provisioning typically used </li></ul></ul><ul><ul><ul><li>Expensive </li></ul></ul></ul><ul><ul><ul><li>Inflexible </li></ul></ul></ul><ul><ul><li>Weighted Fair Queuing </li></ul></ul><ul><ul><ul><li>Complex calculation for each packet </li></ul></ul></ul><ul><ul><ul><li>Simple queuing of packets within flows </li></ul></ul></ul>
  5. 5. <ul><li>QoS in the Internet </li></ul><ul><li>Proposed Approach </li></ul><ul><li>Implementation of QuaSAR </li></ul><ul><li>Experimental Results </li></ul><ul><li>Experience Gained </li></ul><ul><li>Conclusions </li></ul>
  6. 6. Overall Approach <ul><li>Each QoS network flow is assigned its own virtual router - routelet </li></ul><ul><li>Each routelet runs on its own virtual machine </li></ul><ul><li>Routelets are assigned a proportion of the router’s resources required to support the flow’s QoS requirements </li></ul><ul><li>This provides resource usage partitioning between network flows </li></ul>
  7. 7. QuaSAR Components <ul><li>Main Router </li></ul><ul><ul><li>Controls creation of routelets and their assignment to QoS flows </li></ul></ul><ul><ul><li>Routes best effort traffic </li></ul></ul><ul><li>Routelets </li></ul><ul><ul><li>Process and route packets from a QoS MPLS flow </li></ul></ul><ul><li>Packet Demultiplexing </li></ul><ul><ul><li>Identifies packets from QoS flows </li></ul></ul><ul><ul><li>Directs QoS packets to the correct Routelet for processing </li></ul></ul>
  8. 8. QuaSAR Architecture Overview Packet Demultiplexer Main Best-Effort Router Idle Routelet Pool QoS Routelets Input NIC Channels Output NIC Channels
  9. 9. Possible Applications <ul><li>Per QoS flow partitioning on access networks </li></ul><ul><li>Virtual trunking of traffic on access / medium area networks </li></ul><ul><li>Application specific traffic processing for different traffic flows </li></ul><ul><li>Packet scheduling within traffic flows </li></ul>
  10. 10. <ul><li>QoS in the Internet </li></ul><ul><li>Proposed Approach </li></ul><ul><li>Implementation of QuaSAR </li></ul><ul><li>Experimental Results </li></ul><ul><li>Experience Gained </li></ul><ul><li>Conclusions </li></ul>
  11. 11. Virtualisation in QuaSAR <ul><li>Virtualisation divides the resources of a computing system into multiple execution environments </li></ul><ul><li>A virtualisation layer provides the illusion of a real machine to multiple virtual machines </li></ul><ul><li>QuaSAR uses the Xen Virtual Machine Monitor to isolate QoS Routelets within virtual machines </li></ul><ul><ul><li>Xen chosen due to increase in performance and scalability caused by paravirtualisation approach </li></ul></ul><ul><ul><li>Paravirtualisation presents an idealised virtual machine interface, rather than attempting to fully emulate the real hardware </li></ul></ul>
  12. 12. QuaSAR Hierarchically Main Best-Effort Router Input NIC Channels Output NIC Channels Packet Demultiplexer Packet Demultiplexer Idle Routelet Pool QoS Routelets Real Hardware Virtualisation Layer Virtual Machines Virtual Machine Virtual NIC
  13. 13. Routelets <ul><li>Routelets perform simple packet processing using Click Modular Router </li></ul><ul><ul><li>Each Routelet only processes one MPLS LSP, therefore only simple processing needed </li></ul></ul><ul><ul><li>Routelets could be specialised for certain types of traffic </li></ul></ul>
  14. 14. Main Router <ul><li>Routes any best effort traffic using standard Linux Routing Software </li></ul><ul><li>Creates a Pool of Routelet VMs (initially all idle) </li></ul><ul><li>Assigns QoS flows to routelets when new flow creation is signalled using RSVP messages </li></ul><ul><ul><li>Chooses a routelet for this new flow </li></ul></ul><ul><ul><li>Modifies Routelet’s packet forwarding engine for this flow </li></ul></ul><ul><ul><li>Assigns a proportion of the machine’s resources (CPU, NIC bandwidth, etc) to routelet based upon flow’s flowspec </li></ul></ul><ul><ul><li>Updates Demultiplexer to pass packets from this flow to the chosen routelet </li></ul></ul>
  15. 15. Demultiplexing Framework <ul><li>Click Modular Router used to create a switch which passes packets to the routelet processing their flow </li></ul><ul><ul><li>Peeks at MPLS label and compares this with a table of Label / Virtual Network Device pairs </li></ul></ul><ul><ul><li>Passes packet to the Virtual Network Device corresponding to the packet’s label </li></ul></ul><ul><ul><li>Any packets which don’t match are passed to the best effort router </li></ul></ul><ul><li>Routelets are linked to the demultiplexer through Virtual Network Devices </li></ul><ul><li>Bypasses the Linux Network Stack </li></ul>
  16. 16. Restricting Resource Usage <ul><li>CPU Usage </li></ul><ul><ul><li>Xen’s Virtual Machine Scheduler decides when routelets can make progress in processing packets </li></ul></ul><ul><ul><li>Soft Real-time Scheduler could provide guarantees about times to process packets associated with a certain routelet </li></ul></ul><ul><li>Network Bandwidth Usage </li></ul><ul><ul><li>Credit based system to limit each routelet’s maximum outgoing network usage </li></ul></ul><ul><li>Memory Usage </li></ul><ul><ul><li>Not significantly important with QuaSAR (Linux uses most of each routelet’s assigned memory) </li></ul></ul><ul><ul><li>Statically assigned at routelet startup </li></ul></ul>
  17. 17. <ul><li>QoS in the Internet </li></ul><ul><li>Proposed Approach </li></ul><ul><li>Implementation of QuaSAR </li></ul><ul><li>Experimental Results </li></ul><ul><li>Experience Gained </li></ul><ul><li>Conclusions </li></ul>
  18. 18. Results - Latency
  19. 19. Results - Partitioning
  20. 20. <ul><li>QoS in the Internet </li></ul><ul><li>Proposed Approach </li></ul><ul><li>Implementation of QuaSAR </li></ul><ul><li>Experimental Results </li></ul><ul><li>Experience Gained </li></ul><ul><li>Conclusions </li></ul>
  21. 21. Problems with QuaSAR <ul><li>Linux OS running for each routelet </li></ul><ul><li>Context switch overhead between routelets </li></ul><ul><li>Memory protection between domains </li></ul><ul><li>Much of the work per packet occurs during classification </li></ul><ul><ul><li>This is assigned to the best effort router, therefore much of the flow’s work does not occur in its routelet </li></ul></ul><ul><li>Routelets have no direct access to physical network devices </li></ul><ul><li>No soft real-time scheduler for Xen available during this project </li></ul>
  22. 22. <ul><li>QoS in the Internet </li></ul><ul><li>Proposed Approach </li></ul><ul><li>Implementation of QuaSAR </li></ul><ul><li>Experimental Results </li></ul><ul><li>Experience Gained </li></ul><ul><li>Conclusions </li></ul>
  23. 23. Conclusions <ul><li>Virtualisation can be used to partition a router into several distinct routelets </li></ul><ul><ul><li>QoS flow partitioning opportunities </li></ul></ul><ul><ul><li>Virtual segregation of network for management purposes </li></ul></ul><ul><li>Application specific routelets can be built for different classes of network traffic </li></ul><ul><li>It is possible to build a router which uses virtualisation without a huge loss in performance </li></ul>
  24. 24. QuaSAR Hierarchically (Actual) Packet Demultiplexer Domain 0 Domain 0 Real Hardware Virtualisation Layer Virtual Machines
  25. 25. Demultiplexer - Click Architecture best effort router best effort router best effort router best effort router
  26. 26. Results - Jitter
  27. 27. Results - Throughput