Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
OS*: OSv on *
@ysmoo
BitVisor
BitVisor
Hardware
OS
App App App
http://www.justis.as-1.co.jp
https://bitbucket.org/bitvisor/bitvisor
OSv
VMM
Hardware
OSv
App
http://medical-care.feed.jp
https://github.com/cloudius-systems/osv
BitVisor
BitVisor OSv
BitVisor OSv
http://www.root.ne.jp/nishide/shs/
OSv on BitVisor
BitVisor
Hardware
OSv
App
OSv on BitVisor
BitVisor
Hardware
OSv
App
OSb
OSb Design
BitVisor
Hardware
OSv
App
Virtio NIC
With a small fix,
OSv can handle chipsets
OSv has an AHCI driver
BitVisor p...
OSb Design
BitVisor
Hardware
OSv
App
Virtio NIC
With a small fix,
OSv can handle chipsets
OSv has an AHCI driver
BitVisor p...
One thing to fix
Skipped the code!
(drivers/acpi.cc:early_init())
OSb Design
BitVisor
Hardware
OSv
App
Virtio NIC
With a small fix,
OSv can handle chipsets
OSv has an AHCI driver
BitVisor p...
Para Pass-through
Virtio
BitVisor
Hardware
Virtio Ring
PRO/1000 Ring
OSv
App
MSI-X etc.
Handle minimal operations.
PCI Fun...
for (12-billion times) val++;
0.00
6.50
13.00
19.50
26.00
OSb Linux (Host, Bare-metal)
25.51
20.75
ElapsedTime(sec)
Netperf TCP_STREAM
0.00
237.50
475.00
712.50
950.00
OSb Linux (Host, Bare-metal)
941.39921.83
Throughput(Mbps)
Ping
0.00
0.04
0.08
0.12
0.16
OSb Linux (Host, Bare-metal)
0.16
0.12
Latency(msec)
Thinner and thinner
Hardware
KVM/Linux
OSv
Hardware
BitVisor
OSv12.8M LOC
0.3M LOC
virtio
sched.
driver
http://cdn-ak.f.st...
For
ultimate thinness…
OSv on Physical Machine
Hardware
KVM/Linux
OSv
Hardware
BitVisor
OSv
Hardware
OSv
OSp
OSp Design
Hardware
OSv
App
Physical NIC
Driver
Implemented
Intel X540 10GbE Driver
in C++11
With a small fix,
OSv can hand...
About implementation
• Intel NIC TX/RX Basics
• Device initialization
• TX initialization/operations
• RX initialization/o...
Intel NIC TX/RX Basics
Descriptor 0
Descriptor 2
Descriptor 3
Descriptor 4
Descriptor 5
Descriptor 6
Descriptor 7
Ring Buf...
Intel NIC TX Basics
Descriptor 0
Descriptor 2
Descriptor 3
Descriptor 4
Descriptor 5
Descriptor 6
Descriptor 7
Ring Buffer...
Intel NIC TX Basics
Descriptor 0
Descriptor 2
Descriptor 3
Descriptor 4
Descriptor 5
Descriptor 6
Descriptor 7
Ring Buffer...
Intel NIC TX Basics
Descriptor 0
Descriptor 2
Descriptor 3
Descriptor 4
Descriptor 5
Descriptor 6
Descriptor 7
Ring Buffer...
Intel NIC RX Basics
Descriptor 0
Descriptor 2
Descriptor 3
Descriptor 4
Descriptor 5
Descriptor 6
Descriptor 7
Ring Buffer...
Device Init.
TX Init.
TX Interface
(OSv => Driver)
• OSv’s upper layer (TCP/IP) calls:
• int try_xmit_one_locked(void *mbuf_etc)
• Then, driver ...
try_xmit_one_locked()
to put mbuf on ring buffer
kick_hw()
to update producer reg.
RX Init.
RX Interface
(Driver => OSv)
• struct ifnet *_ifn = if_alloc (IFT_ETHER);
• On init., driver creates ifnet and…
• ether_if...
Under improvement
(A lot of TODOs)
• No interrupts… (bad polling impl.)
• No TSO, LRO, checksum assist…
• No advanced desc...
Netperf TCP_STREAM
0.00
2000.00
4000.00
6000.00
8000.00
OSp OSv
7229.00
4200.00
Throughput(Mbps)
Future Work
0.00
2000.00
4000.00
6000.00
8000.00
OSp OSv
7229.00
4200.00
Throughput(Mbps)
Revenge of the OSp
One more thing
KVM/Linux
Hardware
OSv
App
./scripts/bindctrl bind 8086 1528 0000:07:00.0
./scripts/run.py -a 07:00.0
Addit...
Output
$ sudo ./scripts/run.py -n -v -c 8 -a 07:00.0
ixgbe [00:04.0] 8086:1528
Address: a0:36:9f:14:5f:84
PHY Status:
Stat...
Summary
OSv → OSb → OSp
Thank you !
http://www.root.ne.jp/nishide/shs/
References
• The Ugly Patch of Intel X540 10GbE Driver
and Additional scripts for PCI Pass-through
Configuration
• http://g...
Upcoming SlideShare
Loading in …5
×

OS*: OSv on *

1,144 views

Published on

Through my practice of C++, I've added a simple piece of code to OSv. Intel 10GbE driver (dhcp/ping confirmed).

Published in: Software
  • Be the first to comment

OS*: OSv on *

  1. 1. OS*: OSv on * @ysmoo
  2. 2. BitVisor BitVisor Hardware OS App App App http://www.justis.as-1.co.jp https://bitbucket.org/bitvisor/bitvisor
  3. 3. OSv VMM Hardware OSv App http://medical-care.feed.jp https://github.com/cloudius-systems/osv
  4. 4. BitVisor
  5. 5. BitVisor OSv
  6. 6. BitVisor OSv http://www.root.ne.jp/nishide/shs/
  7. 7. OSv on BitVisor BitVisor Hardware OSv App
  8. 8. OSv on BitVisor BitVisor Hardware OSv App OSb
  9. 9. OSb Design BitVisor Hardware OSv App Virtio NIC With a small fix, OSv can handle chipsets OSv has an AHCI driver BitVisor providesVirtio NIC
  10. 10. OSb Design BitVisor Hardware OSv App Virtio NIC With a small fix, OSv can handle chipsets OSv has an AHCI driver BitVisor providesVirtio NIC
  11. 11. One thing to fix Skipped the code! (drivers/acpi.cc:early_init())
  12. 12. OSb Design BitVisor Hardware OSv App Virtio NIC With a small fix, OSv can handle chipsets OSv has an AHCI driver BitVisor providesVirtio NIC
  13. 13. Para Pass-through Virtio BitVisor Hardware Virtio Ring PRO/1000 Ring OSv App MSI-X etc. Handle minimal operations. PCI Function Pass-through! For more details, see: http://www.slideshare.net/yushiomote/osb-osv-on-bitvisor-2
  14. 14. for (12-billion times) val++; 0.00 6.50 13.00 19.50 26.00 OSb Linux (Host, Bare-metal) 25.51 20.75 ElapsedTime(sec)
  15. 15. Netperf TCP_STREAM 0.00 237.50 475.00 712.50 950.00 OSb Linux (Host, Bare-metal) 941.39921.83 Throughput(Mbps)
  16. 16. Ping 0.00 0.04 0.08 0.12 0.16 OSb Linux (Host, Bare-metal) 0.16 0.12 Latency(msec)
  17. 17. Thinner and thinner Hardware KVM/Linux OSv Hardware BitVisor OSv12.8M LOC 0.3M LOC virtio sched. driver http://cdn-ak.f.st-hatena.com/images/fotolife/j/jovi1kamiya/20140607/20140607114847.jpg
  18. 18. For ultimate thinness…
  19. 19. OSv on Physical Machine Hardware KVM/Linux OSv Hardware BitVisor OSv Hardware OSv OSp
  20. 20. OSp Design Hardware OSv App Physical NIC Driver Implemented Intel X540 10GbE Driver in C++11 With a small fix, OSv can handle chipsets OSv has an AHCI driver
  21. 21. About implementation • Intel NIC TX/RX Basics • Device initialization • TX initialization/operations • RX initialization/operations
  22. 22. Intel NIC TX/RX Basics Descriptor 0 Descriptor 2 Descriptor 3 Descriptor 4 Descriptor 5 Descriptor 6 Descriptor 7 Ring Buffer Descriptor 1 Producer I/O Register Consumer I/O Register OSv I/O Registers
  23. 23. Intel NIC TX Basics Descriptor 0 Descriptor 2 Descriptor 3 Descriptor 4 Descriptor 5 Descriptor 6 Descriptor 7 Ring Buffer Descriptor 1 Producer I/O Register Consumer I/O Register OSv I/O Registers
  24. 24. Intel NIC TX Basics Descriptor 0 Descriptor 2 Descriptor 3 Descriptor 4 Descriptor 5 Descriptor 6 Descriptor 7 Ring Buffer Descriptor 1 Producer I/O Register Consumer I/O Register OSv I/O Registers
  25. 25. Intel NIC TX Basics Descriptor 0 Descriptor 2 Descriptor 3 Descriptor 4 Descriptor 5 Descriptor 6 Descriptor 7 Ring Buffer Descriptor 1 Producer I/O Register Consumer I/O Register OSv I/O Registers Xmit!
  26. 26. Intel NIC RX Basics Descriptor 0 Descriptor 2 Descriptor 3 Descriptor 4 Descriptor 5 Descriptor 6 Descriptor 7 Ring Buffer Descriptor 1 Consumer I/O Register Producer I/O Register OSv I/O Registers Recv!
  27. 27. Device Init.
  28. 28. TX Init.
  29. 29. TX Interface (OSv => Driver) • OSv’s upper layer (TCP/IP) calls: • int try_xmit_one_locked(void *mbuf_etc) • Then, driver puts mbuf on ring buffer. • bool kick_hw() • Then, driver updates producer reg. • … EOP EOP mbuf
  30. 30. try_xmit_one_locked() to put mbuf on ring buffer
  31. 31. kick_hw() to update producer reg.
  32. 32. RX Init.
  33. 33. RX Interface (Driver => OSv) • struct ifnet *_ifn = if_alloc (IFT_ETHER); • On init., driver creates ifnet and… • ether_ifattach(_ifn, macaddr); • …passes it to the upper layer. (Then, the upper layer puts an callback function on ifnet.if_input) • (*_ifn->if_input)(_ifn, mbuf); • On packet reception, driver calls the callback function (if_input) by passing mbuf as an argument.
  34. 34. Under improvement (A lot of TODOs) • No interrupts… (bad polling impl.) • No TSO, LRO, checksum assist… • No advanced descriptor… (legacy descriptor seems slow) • …
  35. 35. Netperf TCP_STREAM 0.00 2000.00 4000.00 6000.00 8000.00 OSp OSv 7229.00 4200.00 Throughput(Mbps)
  36. 36. Future Work 0.00 2000.00 4000.00 6000.00 8000.00 OSp OSv 7229.00 4200.00 Throughput(Mbps) Revenge of the OSp
  37. 37. One more thing KVM/Linux Hardware OSv App ./scripts/bindctrl bind 8086 1528 0000:07:00.0 ./scripts/run.py -a 07:00.0 Additional scripts & options to start OSv with PCI pass-through config.
  38. 38. Output $ sudo ./scripts/run.py -n -v -c 8 -a 07:00.0 ixgbe [00:04.0] 8086:1528 Address: a0:36:9f:14:5f:84 PHY Status: State: Connected Type: MDI-X Speed: 10Gbps Duplex: Full MAC Status: Link: UP Speed: 10Gbps ixgbe [00:04.0] 8086:1528 eth0: 192.168.122.15 ixgbe0: 192.168.100.2 /# /# /# /# exit Goodbye
  39. 39. Summary OSv → OSb → OSp
  40. 40. Thank you ! http://www.root.ne.jp/nishide/shs/
  41. 41. References • The Ugly Patch of Intel X540 10GbE Driver and Additional scripts for PCI Pass-through Configuration • http://goo.gl/0yRcV4 • git diff -r 5e5c227b2aec23418e5e3fd104cc21e98f31729c • Usage & True Motivation • http://www.yushiomote.org/blog/?p=428

×