Virtual net performance
Upcoming SlideShare
Loading in...5

Virtual net performance



Slides from Linuxcon Japan 2011

Slides from Linuxcon Japan 2011



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • Performance is a complex topic, and hard to measure. The blind man who feels a leg says the elephant is like a pillar; the one who feels the tail says the elephant is like a rope; the one who feels the trunk says the elephant is like a tree branch; the one who feels the ear says the elephant is like a hand fan; the one who feels the belly says the elephant is like a wall; and the one who feels the tusk says the elephant is like a solid pipe. A wise man explains to them: "All of you are right. The reason every one of you is telling it differently is because each one of you touched the different part of the elephant. So, actually the elephant has all the features you mentioned."
  • I use Virt net every day but always felt uncomfortable that performance is being lost. Rules of thumb, today, on my hardware
  • In honor of Australia day wanted to graph it upside down!! Not a benchmark * different hardware * not precise * not allowed by License agreements
  • P7 Avoid unnecessary generality
  • Relax system requirements

Virtual net performance Virtual net performance Presentation Transcript

  • Virtual Networking Performance Stephen Hemminger Linuxcon Japan Virtualization Summit
  • Bi-directional Virtualization Loss Test 100% 75%Throughput 50% 25% 0% 0 500 1000 1500 Packet Size
  • Houston, we have a problem!
  • TopicsWhat is virtual router?Performance measurementsAnalysisCurrent workHow to help
  • Performance History2005 2006 2007 2008 2009 2010 2011 1G line rate Multiqueue NIC 10G line rate 1M ppsXen 3.0 KVM Hyper-V drivers
  • Performance measurementsForwarding RFC 2544 minimum packetThroughput TCP bulk transferLatency Request / Response
  • Router Benchmark RFC2544 Router Under TestSpirent
  • Virtualized Router Benchmark Router Under Guest Test Hypervisor Bridge BridgeSpirent
  • 1 Gigabit bi-directional forwarding 100% 75%Throughput 50% 25% 0% 0 200 400 600 800 1000 1200 1400 1600 Packet Size
  • Physical NetworksRate limited by slowest componentAdding more layers does not slow down throughputSeparate resources
  • Virtual Network overhead Hyper- Bridge out visor In Bridge In Hyper- visor Router Under Test
  • Network BaselineIntel i7 920 Intel 10G LOM 1G networkLoopback – 12G bits/secForwarding – 1M packets/sec per core Your mileage may vary...
  • Emulated Network Interface Fake PCI regionGuest Hypervisor To Network Packet Buffer
  • Legacy (emulated) NICRelative Performance vs Native % -25% -50% -75% Transmit -100% Receive Hyper-V VmWare KVM
  • Virtual NICGuest Hypervisor To Network Packet Buffer
  • Virtual NIC performance 0%-25%-50%-75%-100% Transmit Hyper-V VmWare Receive KVM
  • VNIC characteristics Hyper-V Vmware Xen KVM vmxnet3 netfront virtio-netMTU 1500 9000 65521 65535Checksum Y Y YoffloadSegmentation Y Y YoffloadNAPI Y Y YLRO YVLAN YMultiqueue Y ? ?
  • Guest to Guest Throughput TSO Tx csumNo offload 0 1000 2000 3000 4000 5000 Megabits/sec
  • VM to VM performance 2000Thorouhput (10^6 bits/sec) 0 100 1000 10000 100000 MTU (bytes)
  • TCP performance2 Tier NATRouteBridge Bare 0 2 4 6 8 10 Gigabytes / sec
  • ObservationsSlow Legacy emulated NIC NAT Firewall connection trackingFast Virtual NIC Hardware passthrough Bare Bridge
  • Multithread benchmark 20,000 UP Guest SMP GuestTotal Transactions/sec 15,000 10,000 5,000 0 Packet Steering (RPS) Baseline Multi Queue NIC
  • Multiple Queue HW or SW Guest VM Thread Device queue ThreadFlow Device VNICclas queuesfier Thread Device queue
  • TODOMore peformance measurement MACVTAP, VEPA, …Configuration Pinning, IRQ assignment, …Profiling
  • Future workVirtio-net optimization Michael S. TsirkinMultiqueue Krishna Kumar KVM ↔ QEMUZero copy Shirley Ma Skb lifetime SecurityBatching Rusty Russell Bufferbloat
  • How to helpAdministrator →Tuning and SizingTester →Measure and TestDeveloper →Multiqueue and Offload