Lagopus Day #1 Tokyo 
䛾㝮䛳䛣䛷䜔䛳䛶䛯䛣䛸 
Aug 31, 2014 
m-oki@stratosphere.co.jp
䝬䝄䞊䝪䞊䝗䛸㟁※➼ᣢ䛱㎸䜏 
● SUPERMICRO X9SKV-1105 
○ Xeon E3-1105C v2 䜸䞁䝪䞊䝗(䝣䜯䞁䝺䝇) 
○ GbE x6 (i350 4ᮏ䚸i210 2ᮏ) 䜸䞁䝪䞊䝗 
○ ౛䛻䜘䛳䛶䝟䝋䝁䞁䝝䜴䝇ᮾᫎ䛥䜣䛷㈙䛔䜎䛧䛯 
● 16GB䝯䝰䝸䚸320GB HDD 
● PCIe x1䛾䝡䝕䜸䜹䞊䝗 
● 9䜲䞁䝏䝰䝙䝍䚸䜻䞊䝪䞊䝗 
● 䜿䞊䝇䛿䛺䛧
఍ሙ䛷䜔䛳䛯䛣䛸 
● QEMU䜢౑䛔௬᝿䝬䝅䞁䜢㉳ື 
● ௬᝿䝬䝅䞁ୖ䛷Ubuntu server䜲䞁䝇䝖䞊䝹 
● 䛭䛾Ubuntuୖ䛷dpdk䜢ྲྀᚓ䚸䝁䞁䝟䜲䝹 
● 䛥䜙䛻lagopus, Pktgen-DPDK䜢䜲䞁䝇䝖䞊䝹 
● Ryu䜒䜲䞁䝇䝖䞊䝹 
● VM䜲䝯䞊䝆䜢」〇䛧2䛴䛾VM䜢㉳ື 
● VMୖ䛷Pktgen-DPDK䛸lagopus䛾㏻ಙ䜢ᐇ⾜
320GB HDD䛾୰㌟ 
● Ubuntu 14.04LTS server 
● ᙜ᪥༗๓୰䛻⮬Ꮿ䛷䜲䞁䝇䝖䞊䝹䛧䛯 
● QEMU⏝䛾HDD䜲䝯䞊䝆䜢స䛳䛶䛚䛔䛯 
○ 4GB䛒䜜䜀Ubuntu server䛜ධ䜛 
qemu-img -f qcow2 create ubuntu-vm1.img 4G
䜔䛳䛯䛣䛸䜢⤮䛻䛩䜛䛸䛣䛖 
● VM䛿䛭䜜䛮䜜3ᮏ㊊ 
● 1ᮏ䛿䝩䝇䝖䝤䝸䝑䝆䚸2ᮏ䛿஫䛔䛻᥋⥆ 
Lagopus Pktgen-DPDK 
QEMU 
ubuntu-vm 
QEMU 
Ryu ubuntu-vm2 
alice: Ubuntu server on X9SKV-1105 
eth4 
10.75.30.10 
→会場ネットワーク 
eth0 
10.75.30.28 
eth0 
10.75.30.31
䝩䝇䝖䝤䝸䝑䝆䛾䛯䜑䛾‽ഛ 
/etc/network/interfaces䛾auto eth4䜢ኚ᭦ 
auto br0 
interface br0 inet dhcp 
bridge_ports eth4 
bridge_stp off 
bridge_maxwait 0 
bridge_fd 0 
QEMU䛾-net bridge䛿ᣦᐃ䛜䛺䛡䜜䜀br0䛻䛴䛺䛜䜛䚹
QEMU䛾㉳ື䝟䝷䝯䞊䝍 
● -machine pc,accel=kvm 
○ ᣦᐃ䛧䛺䛔䛸KVM䛺䛧䛷ື䛟(㐜䛔) 
● -display vnc=10.75.30.10:0 (䜒䛧䛟䛿:1) 
○ 䝜䞊䝖PC䛾VNC Viewer䛷䛴䛺䛔䛷VM䛾䝁䞁䝋䞊䝹䜢᧯స 
○ Viewerഃ䛷䛿port 5900 (䜒䛧䛟䛿5901)䜢ᣦᐃ 
● -m 1024 
○ 1GBᣦᐃ䚹↓ᣦᐃ䛰䛸128MB䛸䛺䜛(Ubuntu䜲䞁䝇䝖䞊䝷䛜ື䛛䛺䛔) 
● -cpu host -smp cores=4 
○ 䝩䝇䝖䛸ྠ䛨௬᝿䝥䝻䝉䝑䝃䜢4䝁䜰䛷 
○ ↓ᣦᐃ䛰䛸QEMU Virtual CPU䛸䛺䜚䚸DPDK䛜䝁䞁䝟䜲䝹䛷䛝䛺䛔 
○ Haswell䛺䛹䜒ᣦᐃ䛷䛝䜛䛜䚸䜶䝭䝳䝺䞊䝅䝵䞁䛜䛛䛛䜛䛸㐜䛟䛺䜛
QEMU䝛䝑䝖䝽䞊䜽㛵㐃 
● -net䜢」ᩘᣦᐃ䛩䜛䚹 
○ 䜂䛸䛴䛿௬᝿䝬䝅䞁ഃ䛾NICタᐃ䚸䜂䛸䛴䛿䝩䝇䝖ഃ䛷䛹䛖ᢅ䛖䛛䛾タᐃ 
○ ௬᝿䝛䝑䝖䝽䞊䜽␒ྕ䛷༊ู䛩䜛 (vlan=N䛷䚸䝕䝣䜷䝹䝖1) 
○ bridge᥋⥆䛩䜛䛻䛿⟶⌮⪅ᶒ㝈䛜ᚲせ䚹qemu䜢sudo௜䛷㉳ື䛩䜛䚹 
○ tapᣦᐃ䛩䜛䛸ྠ୍vlan␒ྕ䛾VMྠኈ䛜䛴䛺䛜䜛 
● -net nic -net bridge,macaddr=xx:xx:xx:xx:xx:xx 
● -net nic,vlan=2 -net tap,vlan=2 
● -net nic,vlan=3 -net tap,vlan=3 
○ eth0: 䝩䝇䝖䝤䝸䝑䝆䚹MACᣦᐃ䛜䛺䛔䛸ྠ୍䛻䛺䜚䚸」ᩘVM䛷ᅔ䜛 
○ eth1: VM㛫᥋⥆ 
○ eth2: VM㛫᥋⥆
QEMU䝇䝖䝺䞊䝆㛵㐃 
● -hda ubuntu-vm.img (䜒䛧䛟䛿ubuntu-vm2.img) 
● -cdrom ubuntu-server-14.04.1-amd64.iso 
● Ubuntu䜲䞁䝇䝖䞊䝹ᚋ䚸img䝣䜯䜲䝹䜢䝁䝢䞊䛧䛯䚹 
● ISO䛿Ubuntu䜲䞁䝇䝖䞊䝹᫬䛰䛡ᣦᐃ䚹
QEMU㉳ື 
ୗグ䜢1⾜䛷䚹ኴᏐ䛿2䛴┠䛾VM䛾䛸䛝ኚ᭦䛩䜛䚹 
sudo qemu-system-x86_64 -machine pc, 
accel=kvm -display vnc=10.75.30.10:0 -m 1024 
-cpu host,cores=4 -net nic -net bridge, 
macaddr=56:78:90:12:34:56 -net nic,vlan=2 - 
net tap,vlan=2 -net nic,vlan=3 -net tap, 
vlan=3 -hda ubuntu-vm1.img
䜲䞁䝇䝖䞊䝹䛸䛛 
DPDK, lagopus䛾䜲䞁䝇䝖䞊䝹䛿䝝䞁䝈䜸䞁䛾䜎䜎䛻䛴䛝␎䚹 
Pktgen-DPDK䛿ୗグ䛾䜘䛖䛺ឤ䛨䛷䚹 
cd src/Pktgen-DPDK 
RTE_SDK=`pwd` 
RTE_TARGET=x86_64-pktgen-linuxapp-gcc 
make config T=$RTE_TARGET 
make install 
cd example/pktgen 
make
䛭䜜䛮䜜䛾VM䛷䜰䝥䝸㉳ື 
● ryu-manager simple_switch_13.py 
● sudo lagopus -d -cf -n2 -- -p3 
○ -cf: 4䝁䜰 (DPDKඹ㏻䝟䝷䝯䞊䝍) 
○ -n2: 䝯䝰䝸䝏䝱䝛䝹ᣦᐃ (ྠୖ) 
○ -p3: Port0, Port1䜢฼⏝ 
● sudo app/pktgen -cf -n2 -- -m 1.0,2.1 
○ -m 1.0,2.1: 䝁䜰1䛷Port0䛾㏦ཷಙ䚸䝁䜰2䛷Port1
ື䛔䛯? 
● ື䛔䛯! 
● ື䛝䛯䛶䜋䜔䜋䜔䜢䝝䞁䝈䜸䞁䛷䛚ぢ䛫䛧䛯 
● ὶ䛧䛶䛔䜛䝟䜿䝑䝖䛿64byte䛾IPv4/TCP 
● 䝇䝹䞊䝥䝑䝖䛿䛛䛺䜚ప䛛䛳䛯Ẽ䛜䚹 
● タᐃ䜢ワ䜑䛯䜚䛷䛝䜛䛸㏿䛟䛺䜛䛛䜒?

Qemu上でlagopusを動かした話

  • 1.
    Lagopus Day #1Tokyo 䛾㝮䛳䛣䛷䜔䛳䛶䛯䛣䛸 Aug 31, 2014 m-oki@stratosphere.co.jp
  • 2.
    䝬䝄䞊䝪䞊䝗䛸㟁※➼ᣢ䛱㎸䜏 ● SUPERMICROX9SKV-1105 ○ Xeon E3-1105C v2 䜸䞁䝪䞊䝗(䝣䜯䞁䝺䝇) ○ GbE x6 (i350 4ᮏ䚸i210 2ᮏ) 䜸䞁䝪䞊䝗 ○ ౛䛻䜘䛳䛶䝟䝋䝁䞁䝝䜴䝇ᮾᫎ䛥䜣䛷㈙䛔䜎䛧䛯 ● 16GB䝯䝰䝸䚸320GB HDD ● PCIe x1䛾䝡䝕䜸䜹䞊䝗 ● 9䜲䞁䝏䝰䝙䝍䚸䜻䞊䝪䞊䝗 ● 䜿䞊䝇䛿䛺䛧
  • 3.
    ఍ሙ䛷䜔䛳䛯䛣䛸 ● QEMU䜢౑䛔௬᝿䝬䝅䞁䜢㉳ື ● ௬᝿䝬䝅䞁ୖ䛷Ubuntu server䜲䞁䝇䝖䞊䝹 ● 䛭䛾Ubuntuୖ䛷dpdk䜢ྲྀᚓ䚸䝁䞁䝟䜲䝹 ● 䛥䜙䛻lagopus, Pktgen-DPDK䜢䜲䞁䝇䝖䞊䝹 ● Ryu䜒䜲䞁䝇䝖䞊䝹 ● VM䜲䝯䞊䝆䜢」〇䛧2䛴䛾VM䜢㉳ື ● VMୖ䛷Pktgen-DPDK䛸lagopus䛾㏻ಙ䜢ᐇ⾜
  • 4.
    320GB HDD䛾୰㌟ ●Ubuntu 14.04LTS server ● ᙜ᪥༗๓୰䛻⮬Ꮿ䛷䜲䞁䝇䝖䞊䝹䛧䛯 ● QEMU⏝䛾HDD䜲䝯䞊䝆䜢స䛳䛶䛚䛔䛯 ○ 4GB䛒䜜䜀Ubuntu server䛜ධ䜛 qemu-img -f qcow2 create ubuntu-vm1.img 4G
  • 5.
    䜔䛳䛯䛣䛸䜢⤮䛻䛩䜛䛸䛣䛖 ● VM䛿䛭䜜䛮䜜3ᮏ㊊ ● 1ᮏ䛿䝩䝇䝖䝤䝸䝑䝆䚸2ᮏ䛿஫䛔䛻᥋⥆ Lagopus Pktgen-DPDK QEMU ubuntu-vm QEMU Ryu ubuntu-vm2 alice: Ubuntu server on X9SKV-1105 eth4 10.75.30.10 →会場ネットワーク eth0 10.75.30.28 eth0 10.75.30.31
  • 6.
    䝩䝇䝖䝤䝸䝑䝆䛾䛯䜑䛾‽ഛ /etc/network/interfaces䛾auto eth4䜢ኚ᭦ auto br0 interface br0 inet dhcp bridge_ports eth4 bridge_stp off bridge_maxwait 0 bridge_fd 0 QEMU䛾-net bridge䛿ᣦᐃ䛜䛺䛡䜜䜀br0䛻䛴䛺䛜䜛䚹
  • 7.
    QEMU䛾㉳ື䝟䝷䝯䞊䝍 ● -machinepc,accel=kvm ○ ᣦᐃ䛧䛺䛔䛸KVM䛺䛧䛷ື䛟(㐜䛔) ● -display vnc=10.75.30.10:0 (䜒䛧䛟䛿:1) ○ 䝜䞊䝖PC䛾VNC Viewer䛷䛴䛺䛔䛷VM䛾䝁䞁䝋䞊䝹䜢᧯స ○ Viewerഃ䛷䛿port 5900 (䜒䛧䛟䛿5901)䜢ᣦᐃ ● -m 1024 ○ 1GBᣦᐃ䚹↓ᣦᐃ䛰䛸128MB䛸䛺䜛(Ubuntu䜲䞁䝇䝖䞊䝷䛜ື䛛䛺䛔) ● -cpu host -smp cores=4 ○ 䝩䝇䝖䛸ྠ䛨௬᝿䝥䝻䝉䝑䝃䜢4䝁䜰䛷 ○ ↓ᣦᐃ䛰䛸QEMU Virtual CPU䛸䛺䜚䚸DPDK䛜䝁䞁䝟䜲䝹䛷䛝䛺䛔 ○ Haswell䛺䛹䜒ᣦᐃ䛷䛝䜛䛜䚸䜶䝭䝳䝺䞊䝅䝵䞁䛜䛛䛛䜛䛸㐜䛟䛺䜛
  • 8.
    QEMU䝛䝑䝖䝽䞊䜽㛵㐃 ● -net䜢」ᩘᣦᐃ䛩䜛䚹 ○ 䜂䛸䛴䛿௬᝿䝬䝅䞁ഃ䛾NICタᐃ䚸䜂䛸䛴䛿䝩䝇䝖ഃ䛷䛹䛖ᢅ䛖䛛䛾タᐃ ○ ௬᝿䝛䝑䝖䝽䞊䜽␒ྕ䛷༊ู䛩䜛 (vlan=N䛷䚸䝕䝣䜷䝹䝖1) ○ bridge᥋⥆䛩䜛䛻䛿⟶⌮⪅ᶒ㝈䛜ᚲせ䚹qemu䜢sudo௜䛷㉳ື䛩䜛䚹 ○ tapᣦᐃ䛩䜛䛸ྠ୍vlan␒ྕ䛾VMྠኈ䛜䛴䛺䛜䜛 ● -net nic -net bridge,macaddr=xx:xx:xx:xx:xx:xx ● -net nic,vlan=2 -net tap,vlan=2 ● -net nic,vlan=3 -net tap,vlan=3 ○ eth0: 䝩䝇䝖䝤䝸䝑䝆䚹MACᣦᐃ䛜䛺䛔䛸ྠ୍䛻䛺䜚䚸」ᩘVM䛷ᅔ䜛 ○ eth1: VM㛫᥋⥆ ○ eth2: VM㛫᥋⥆
  • 9.
    QEMU䝇䝖䝺䞊䝆㛵㐃 ● -hdaubuntu-vm.img (䜒䛧䛟䛿ubuntu-vm2.img) ● -cdrom ubuntu-server-14.04.1-amd64.iso ● Ubuntu䜲䞁䝇䝖䞊䝹ᚋ䚸img䝣䜯䜲䝹䜢䝁䝢䞊䛧䛯䚹 ● ISO䛿Ubuntu䜲䞁䝇䝖䞊䝹᫬䛰䛡ᣦᐃ䚹
  • 10.
    QEMU㉳ື ୗグ䜢1⾜䛷䚹ኴᏐ䛿2䛴┠䛾VM䛾䛸䛝ኚ᭦䛩䜛䚹 sudoqemu-system-x86_64 -machine pc, accel=kvm -display vnc=10.75.30.10:0 -m 1024 -cpu host,cores=4 -net nic -net bridge, macaddr=56:78:90:12:34:56 -net nic,vlan=2 - net tap,vlan=2 -net nic,vlan=3 -net tap, vlan=3 -hda ubuntu-vm1.img
  • 11.
    䜲䞁䝇䝖䞊䝹䛸䛛 DPDK, lagopus䛾䜲䞁䝇䝖䞊䝹䛿䝝䞁䝈䜸䞁䛾䜎䜎䛻䛴䛝␎䚹 Pktgen-DPDK䛿ୗグ䛾䜘䛖䛺ឤ䛨䛷䚹 cd src/Pktgen-DPDK RTE_SDK=`pwd` RTE_TARGET=x86_64-pktgen-linuxapp-gcc make config T=$RTE_TARGET make install cd example/pktgen make
  • 12.
    䛭䜜䛮䜜䛾VM䛷䜰䝥䝸㉳ື ● ryu-managersimple_switch_13.py ● sudo lagopus -d -cf -n2 -- -p3 ○ -cf: 4䝁䜰 (DPDKඹ㏻䝟䝷䝯䞊䝍) ○ -n2: 䝯䝰䝸䝏䝱䝛䝹ᣦᐃ (ྠୖ) ○ -p3: Port0, Port1䜢฼⏝ ● sudo app/pktgen -cf -n2 -- -m 1.0,2.1 ○ -m 1.0,2.1: 䝁䜰1䛷Port0䛾㏦ཷಙ䚸䝁䜰2䛷Port1
  • 13.
    ື䛔䛯? ● ື䛔䛯! ● ື䛝䛯䛶䜋䜔䜋䜔䜢䝝䞁䝈䜸䞁䛷䛚ぢ䛫䛧䛯 ● ὶ䛧䛶䛔䜛䝟䜿䝑䝖䛿64byte䛾IPv4/TCP ● 䝇䝹䞊䝥䝑䝖䛿䛛䛺䜚ప䛛䛳䛯Ẽ䛜䚹 ● タᐃ䜢ワ䜑䛯䜚䛷䛝䜛䛸㏿䛟䛺䜛䛛䜒?