XS Oracle 2009 PV USB


Published on

Noboru Iwamatsu: Paravirtualized USB Support

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

XS Oracle 2009 PV USB

  1. 1. Paravirtualized USB Support for Xen: Status Update Noboru Iwamatsu n_iwamatsu@jp.fujitsu.com FUJITSU LABORATORIES LTD. Xen Summit at Oracle Feb 24-25, 2009
  2. 2. Contents  Background and Motivation  Development Status  PVUSB Design  Demo (Video)  Test Results 1 Copyright 2009 FUJITSU LIMITED
  3. 3. Background: Client virtualization and USB  In client virtualization, special Service-VMs work in the background of User-VM, and provide various functions.  Both User-VM and Service-VMs require using the USB devices at the same time by the same USB host controller. User VM Service VM Service VM Usage example of client virtualization User OS Service OS Service OS Office/Music VoIP/ Authentication apps Messenger Hypervisor USB Host Controller Laptop USB devices 2 Copyright 2009 FUJITSU LIMITED
  4. 4. Motivation  USB support for Xen today  Qemu-dm UHCI emulation • USB 1.1 only  PCI pass-through with IOMMU • Works well. But the entire controller is assigned to a single domain and can’t be used from other domains.  Current options are not suitable for client virtualization!  We have started development to achieve a good performance and flexibility! Aug. 2008 Started development. Proposed in XCI (Xen Client Initiative). “PV USB support” was on the roadmap of Xen 3.4. Sep. 2008 Nov. 2008 Just started working! Xen Summit @Tokyo 2008 3 Copyright 2009 FUJITSU LIMITED
  5. 5. Development Status CHANGES from the last summit  What we did  Emulation of the CLEAR_HALT command added in the backend.  RING operations and queue usage changed.  Initial hotplug code added.  Results  Various devices worked (HID, Storage, Serial, Ethernet)  Performance improved (6MB/s to 33MB/s on USB storage)  Hotplug worked, but unplug not worked yet :-)  What we left  No code posted yet!  Disconnection and canceling functions not completed.  Not well tested. 4 Copyright 2009 FUJITSU LIMITED
  6. 6. PVUSB Design: Driver Architecture Driver domain kernel-space Guest domain kernel-space free or reuse usb_alloc_urb() usbback USB Function Driver urb urb Xen backend interface USB Core xenbus RING .urb_enqueue() usb_hcd_giveback_urb() usb_alloc_urb() Virtual USB free Cloned Cloned USB Function Driver Host Controller Driver urb urb usb_submit_urb () urb->complete RING xenbus USB Core Xen frontend interface Host Controller Driver usbfront Hardware (USB Host Controller) urb (USB Request Block structure): In linux kernel, all USB drivers communicate with urb. (described in include/linux/usb.h).  Frontend driver (usbfront) - Implemented as a USB host controller driver  Backend driver (usbback) - Implemented as a USB function driver 5 Copyright 2009 FUJITSU LIMITED
  7. 7. PVUSB Design: Hotplug  Hotplug-rule is set from sysfs interface in the backend driver.  The hotplug-rule format • <usbbusname>:<domid>:<vusb number>:<virtual port number>  Example settings % echo 1-2.3:1:0:3 > /sys/bus/usb/drivers/usbback/new_vport % echo 1-4:2:0:1 > /sys/bus/usb/drivers/usbback/new_vport % echo 1-2.1:2:0:2 > /sys/bus/usb/drivers/usbback/new_vport Claim the device, Guest VM 1 Guest VM 2 Dom 0 and notify by xenbus. (domain ID:1) (domain ID:2) usbback Hotplug notification usbfront usbfront Host Controller Host Controller Host Controller (vusb-0) (vusb-0) (usb1) Virtual Virtual roothub roothub roothub 1-4 device hub device device device device 1-2.1 1-2.3 device device 6 Copyright 2009 FUJITSU LIMITED
  8. 8. PVUSB Design: Source Code Tree Require no change in existing kernel code! Xen USB public I/O header usbif header file Xen USB backend driver Backend interface management Makefile USB backend RING operations and URB transmissions usbback header file USB stub driver – grabbing and managing USB devices Xenbus operations Xen USB frontend driver Makefile usbfront header file Debugging functions for Host Controller USB2.0 Host Controller interface Root Hub emulations for Host Controller USB frontend RING operations and URB transmissions Xenbus operations 7 Copyright 2009 FUJITSU LIMITED
  9. 9. Demo (Video)  Hotplug Keyboard, Mouse, Flash drive, HDD and Ethernet.  All devices successfully connected to guest domain and work fine. 8 Copyright 2009 FUJITSU LIMITED
  10. 10. Results: Tested Devices (USB2.0) Device Transfer Name Manufacturer Driver Status Type Type Flash drive RUF2-R2GS Buffalo Bulk usb-storage Flash drive RUF-C1G/U2 Buffalo Bulk usb-storage Flash drive DataTraveler DTI/1G Kingston Bulk usb-storage HDD HDCN-U500 IO DATA Bulk usb-storage Memory Card BSCRA38U2 Buffalo Bulk usb-storage Reader/Writer Ethernet ETX2-US2 IO DATA Bulk/Intr pegasus (100Base-TX) Ethernet ETG2-US2 IO DATA Bulk/Intr asix (1000Base-T) [1] USB2.0 SVGA VGA KAIREN Bulk/Intr sisusbvga Adapter [2] Works Works with issues Not work [1] Interrupt URB fails, and the device goes into a stall. [2] No URB fails, but fails to output the correct screen. 9 Copyright 2009 FUJITSU LIMITED
  11. 11. Results: Tested Devices (USB1.1) Device Transfer Name Manufacturer Driver Status Type Type Keyboard FMV-KB333 Fujitsu Intr usbhid Keyboard FKB-108-EU FILCO Intr usbhid Cordless Notebook Mouse Logitech Intr usbhid Mouse Mouse Optical Mouse USB Logitech Intr usbhid FDD USB-FDU Y-E DATA Bulk/Intr usb-storage Serial USB-RSAQ5 IO DATA Bulk/Intr pl2303 Webcam WebCam 3 USB Creative Labs Isoc ov511 [3] Webcam WebCam NX pro Creative Labs Isoc gspca [4] Works Works with issues Not work [3] No URB fails, but incorrect horizontal stripes are into the video image. [4] No URB fails, but the video image has incorrect blinking and flickers. 10 Copyright 2009 FUJITSU LIMITED
  12. 12. Results: Performance (USB Storage) USB Mass Storage Read/Write Throughput (MB/s) 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 92% vs native 29.0 Read USB Flash Drive 31.5 82% vs native 20.3 Write 24.9 99% vs native 33.3 Read 33.8 USB HDD 99% vs native 24.9 Paravirtualized USB Write 25.2 Dom0 nativedriver Test environment: Read/Write 1GB block using dd command. CPU: Intel Core 2 Duo 6700 2.66GHz Kernel: linux-2.6.18-xen (c/s 789), x86_64 USB Flash Drive: Buffalo RUF2-R2GS Xen: 3.3.0 release, x86_64 USB HDD: IODATA HDCN-U500 11 Copyright 2009 FUJITSU LIMITED
  13. 13. Results: Performance (USB Ethernet) USB Ethernet(100Base-TX) TCP Throughput (Mbps) 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0 86% vs native 77.7 Rx 89.9 USB Ethernet 67% vs native 62.2 Tx 92.5 Paravirtualized USB Dom0 native driver Test environment: TCP throughput measured by ttcp command. CPU: Intel Core 2 Duo 6700 2.66GHz Kernel: linux-2.6.18-xen (c/s 789), x86_64 USB Ethernet: IODATA ETX2-US2 Xen: 3.3.0 release, x86_64 12 Copyright 2009 FUJITSU LIMITED
  14. 14. Results: CPU Utilization (USB Storage) CPU utilization: USB HDD Read/Write 50 x1.9 vs native Paravirtualized USB Domain 0 native driver 40 Guest x2.6 vs native 30 CPU(%) Guest 20 Dom0 Dom0 Dom0 10 Dom0 0 Paravirtualized Dom0 native Paravirtualized Dom0 native USB driver USB driver Read Write Test environment: Xentop average while Read/Writ with dd command. CPU: Intel Core 2 Duo 6700 2.66GHz Kernel: linux-2.6.18-xen (c/s 789), x86_64 USB HDD: IODATA HDCN-U500 Xen: 3.3.0 release, x86_64 13 Copyright 2009 FUJITSU LIMITED
  15. 15. Results: CPU Utilization (USB Ethernet) CPU utilization: USB Ethernet Rx/Tx 90 x3 vs native Paravirtualized USB 80 Domain 0 native driver 70 Guest x3.4 vs native 60 Guest CPU(%) 50 40 Dom0 30 Dom0 20 Dom0 10 Dom0 0 Paravirtualized Dom0 native Paravirtualized Dom0 native USB driver USB driver Rx Tx Test environment: Xentop average while using ttcp command. CPU: Intel Core 2 Duo 6700 2.66GHz Kernel: linux-2.6.18-xen (c/s 789), x86_64 USB Ethernet: IODATA ETX2-US2 Xen: 3.3.0 release, x86_64 14 Copyright 2009 FUJITSU LIMITED
  16. 16. TODO & Future Work  TODO (for posting the code)  Complete the device disconnection and the URB canceling functions. • Current PVUSB can hotplug, but can’t unplug, and can’t cancel the URBs that already transferred to the backend. • When the device fails and actually stops, current PVUSB can do nothing.  Add Xend support. We take a month to post the code.  Future Work (after initial posted)  Isochronous transfer support (for WebCAM) • Driver has no errors, the problem seems timing related.  Some optimizations • Reducing the buffer allocation overhead • Better handling of the large buffer transfer • Netchannel2 support 15 Copyright 2009 FUJITSU LIMITED
  17. 17. Conclusion  Paravirtualized USB achieved:  Various devices worked. Worked devices Keyboard(2), Mouse(2), FDD(1), Serial(1), Flash drive(3), HDD(1), Ethernet(1)  Performance improved. • USB Storage: 99 % of the native driver • USB Ethernet: 67% - 86% of the native driver  Hotplug worked.  Toward the initial release  Complete the several essential functions.  More test • SMP(Quad-core), multiple-domain, heavy load, … 16 Copyright 2009 FUJITSU LIMITED
  18. 18. 17 Copyright 2009 FUJITSU LIMITED
  19. 19. Appendix: dmesg output dmesg output after usbfront loaded. 18 Copyright 2009 FUJITSU LIMITED
  20. 20. Appendix: lsusb output lsusb output after connected the devices. 19 Copyright 2009 FUJITSU LIMITED
  21. 21. Appendix: Webcam(ov511) output xawtv video streaming output 20 Copyright 2009 FUJITSU LIMITED
  22. 22. Appendix: USB-VGA(sisusbvga) output USB-VGA output: CentOS 5.2 default login screen (res. 1024x768) 21 Copyright 2009 FUJITSU LIMITED