SlideShare a Scribd company logo
1 of 75
Download to read offline
OSb: OSv on BitVisor 
@ysmoo 
Saturday, October 18, 14
BitVisor 
App App App 
BitVisor 
Hardware 
OS 
http://www.justis.as-1.co.jp 
Saturday, October 18, 14
OSv 
VMM 
Hardware 
App 
OSv 
http://medical-care.feed.jp 
Saturday, October 18, 14
BitVisor 
Saturday, October 18, 14
BitVisor OSv 
Saturday, October 18, 14
BitVisor OSv 
http://www.root.ne.jp/nishide/shs/ 
Saturday, October 18, 14
OSv on BitVisor 
BitVisor 
Hardware 
App 
OSv 
Saturday, October 18, 14
OSv on BitVisor 
BitVisor 
Hardware 
App 
OSv 
OSb 
Saturday, October 18, 14
OSv on BitVisor 
BitVisor 
Hardware 
App 
OSv 
What the fuck 
no Virtio ! 
OSb 
Physical interface 
sucks ! 
What the hell is 
PRO/1000 ! 
Saturday, October 18, 14
OSv on BitVisor 
BitVisor 
Hardware 
App 
OSv 
What the fuck 
no Virtio ! 
OSb 
Physical interface 
sucks ! 
What the hell is 
PRO/1000 ! 
You have no choice. 
Virtio 
Saturday, October 18, 14
~ The road to OSb ~ 
Saturday, October 18, 14
OSv code reading 
• Boot process 
• Some drivers 
Saturday, October 18, 14
OSv code reading 
• Boot process 
• Some drivers 
- MBR 
- Read local disk with INT13/42. 
- Load command line, boot loader, OSv kernel. 
- Get memory map with INT15/E820. 
- Setup segment descriptors/page tables. 
- Switch to 64-bit mode. 
- premain() 
- main() 
... 
Saturday, October 18, 14
OSv code reading 
• Boot process 
• Some drivers 
- MBR 
- Read local disk with INT13/42. 
- Load command line, boot loader, OSv kernel. 
- Get memory map with INT15/E820. 
- Setup segment descriptors/page tables. 
- Switch to 64-bit mode. 
- premain() 
- main() 
... 
- Serial / VGA output. 
- SATA. 
- Virtio NIC/BLK/RNG/SCSI. 
- ACPI. 
- APIC. 
... 
Saturday, October 18, 14
OSv code reading 
• Boot process 
• Some drivers 
- MBR 
- Read local disk with INT13/42. 
- Load command line, boot loader, OSv kernel. 
- Get memory map with INT15/E820. 
- Setup segment descriptors/page tables. 
- Switch to 64-bit mode. 
- premain() 
- main() 
... 
- Serial / VGA output. 
- SATA. 
- Virtio NIC/BLK/RNG/SCSI. 
- ACPI. 
- APIC. 
... 
Straightforward... 
Saturday, October 18, 14
Booting OSb 
./scripts/run.py 
Saturday, October 18, 14
Booting OSb 
In the case of normal OSv... 
./scripts/run.py 
OSv 
VMM 
Saturday, October 18, 14
Booting OSb 
Hardware 
./scripts/run.py 
Saturday, October 18, 14
Booting OSb 
Hardware 
./scripts/run.py 
Image 
Update 
Saturday, October 18, 14
Booting OSb 
Hardware 
./scripts/run.py 
Image 
Update 
IPMI 
Saturday, October 18, 14
Booting OSb 
Hardware 
./scripts/run.py 
Image 
Update 
PXE Boot 
IPMI 
Saturday, October 18, 14
Booting OSb 
Hardware 
./scripts/run.py 
Image 
Update 
PXE Boot 
IPMI 
BitVisor 
Saturday, October 18, 14
Booting OSb 
Hardware 
./scripts/run.py 
Image 
Update 
PXE Boot 
IPMI 
BitVisor 
OSv 
Network Boot 
Saturday, October 18, 14
Booting OSb 
Hardware 
./scripts/run.py 
Image 
Update 
PXE Boot 
IPMI 
BitVisor 
OSv 
Network Boot 
App 
Saturday, October 18, 14
Booting OSb 
BitVisor 
Hardware 
App 
OSv 
Network Boot 
IPMI 
./scripts/run.py 
TTY 
Image 
Update 
PXE Boot 
Saturday, October 18, 14
ACPI workaround 
drivers/acpi.cc: 
// Copy the root table list to dynamic memory 
if (!is_bitvisor()) { 
status = AcpiReallocateRootTable(); 
if (ACPI_FAILURE(status)) { 
acpi_e("AcpiReallocateRootTable failed: %sn", 
AcpiFormatException(status)); 
return; 
} 
} 
Saturday, October 18, 14
ACPI workaround 
drivers/acpi.cc: 
// Copy the root table list to dynamic memory 
if (!is_bitvisor()) { 
status = AcpiReallocateRootTable(); 
if (ACPI_FAILURE(status)) { 
acpi_e("AcpiReallocateRootTable failed: %sn", 
AcpiFormatException(status)); 
return; 
} 
} 
Triple fault 
Saturday, October 18, 14
ACPI workaround 
drivers/acpi.cc: 
// Copy the root table list to dynamic memory 
if (!is_bitvisor()) { 
status = AcpiReallocateRootTable(); 
if (ACPI_FAILURE(status)) { 
acpi_e("AcpiReallocateRootTable failed: %sn", 
AcpiFormatException(status)); 
return; 
} 
} 
Triple fault 
Skip in case of OSb 
Saturday, October 18, 14
ACPI workaround 
drivers/acpi.cc: 
// Copy the root table list to dynamic memory 
if (!is_bitvisor()) { 
status = AcpiReallocateRootTable(); 
if (ACPI_FAILURE(status)) { 
acpi_e("AcpiReallocateRootTable failed: %sn", 
AcpiFormatException(status)); 
return; 
} 
} 
Triple fault 
Skip in case of OSb 
Machine-specific...? 
Saturday, October 18, 14
Hello world ! 
Saturday, October 18, 14
Performance? 
20.75& 21.20& 
25.00& 
20.00& 
15.00& 
10.00& 
5.00& 
0.00& 
OSb& OSv& 
Elapsed(Time((sec) 
for((12(billion(6mes)(val++; 
Saturday, October 18, 14
Virtio NIC 
BitVisor 
Hardware 
App Actually PRO/1000... 
OSv 
Saturday, October 18, 14
Virtio NIC 
BitVisor 
Hardware 
App 
OSv 
Virtio NIC 
Actually PRO/1000... 
Fake Virtio NIC... 
Saturday, October 18, 14
Virtio NIC 
BitVisor 
Hardware 
App 
OSv 
Virtio NIC 
Actually PRO/1000... 
Fake Virtio NIC... 
Hook PCI config access, MMIO, PIO and fake! 
Saturday, October 18, 14
Faking 
PCI configuration space 
• Faking IDs 
• Faking BARs 
• Faking capabilities 
Saturday, October 18, 14
Faking IDs 0x1AF4 
(Virtio Device) 
Saturday, October 18, 14
Faking IDs 0x1AF4 
(Virtio Device) 
0x1000 
(Virtio NIC) 
Saturday, October 18, 14
Faking IDs 0x1AF4 
(Virtio Device) 
0x1000 
(Virtio NIC) 
0 (Legacy Virtio) 
Saturday, October 18, 14
Faking IDs 0x1AF4 
(Virtio Device) 
0x1000 
(Virtio NIC) 
0 (Legacy Virtio) 
1 (Virtio NIC) 
Saturday, October 18, 14
Faking BARs 
BAR0: IO space 
for Virtio 
BAR1: MMIO space 
for MSI-X 
Saturday, October 18, 14
Faking BARs 
BAR0: IO space 
for Virtio 
BAR1: MMIO space 
for MSI-X 
= PRO/1000 IO space 
Saturday, October 18, 14
Faking BARs 
BAR0: IO space 
for Virtio 
BAR1: MMIO space 
for MSI-X 
= PRO/1000 IO space 
= PRO/1000 MMIO space 
Saturday, October 18, 14
Faking capabilities 
Add MSI-X 
capability 
Saturday, October 18, 14
Faking capabilities 
Add MSI-X 
capability Support 
3 vectors 
Saturday, October 18, 14
Faking capabilities 
Add MSI-X 
capability Support 
3 vectors 
Vector 0: 
Reception 
Saturday, October 18, 14
Faking capabilities 
Add MSI-X 
capability Support 
3 vectors 
Vector 0: 
Reception 
Vector 1: 
Transmission 
Saturday, October 18, 14
Faking capabilities 
Add MSI-X 
capability Support 
3 vectors 
Vector 0: 
Reception 
Vector 1: 
Transmission 
Vector 2: 
Control 
Saturday, October 18, 14
Virtio NIC Operations 
• Virtio ring 
• Packet transmission 
• Packet reception 
Saturday, October 18, 14
Virtio ring 
Controlled by 
these registers 
BAR0 
(IO) 
Saturday, October 18, 14
Virtio ring 
Controlled by 
these registers 
3 rings 
available 
BAR0 
(IO) 
Saturday, October 18, 14
Virtio ring 
BAR0 Ring 0: Reception 
(IO) 
Controlled by 
these registers 
3 rings 
available 
Saturday, October 18, 14
Virtio ring 
Controlled by 
these registers 
3 rings 
available 
Ring 0: Reception 
Ring 1: Transmission 
BAR0 
(IO) 
Saturday, October 18, 14
Virtio ring 
Controlled by 
these registers 
3 rings 
available 
Ring 0: Reception 
Ring 1: Transmission 
Ring 2: Control 
BAR0 
(IO) 
Saturday, October 18, 14
Virtio ring 
Controlled by 
these registers 
3 rings 
available 
Ring 0: Reception 
Ring 1: Transmission 
Ring 2: Control 
BAR0 
(IO) 
(Memory) 
Saturday, October 18, 14
Virtio ring 
Controlled by 
these registers 
3 rings 
available 
Ring 0: Reception 
Ring 1: Transmission 
Ring 2: Control 
BAR0 
(IO) 
(Memory) 
Driver 2 Device 
Device 2 Driver 
Saturday, October 18, 14
Packet transmission 
OSv 
(Driver) 
BitVisor 
(Device) 
PRO/1000 
Saturday, October 18, 14
Packet transmission 
OSv 
(Driver) 
BitVisor 
(Device) 
Req. 
PRO/1000 
Req. Req. 
Saturday, October 18, 14
Packet transmission 
OSv 
(Driver) 
BitVisor 
(Device) 
Req. 
PRO/1000 
Req. Req. 
Req. Req. Req. 
Saturday, October 18, 14
Packet transmission 
OSv 
(Driver) 
BitVisor 
(Device) 
Req. 
PRO/1000 
Req. 
TX Ring 
Req. 
Req. Req. Req. 
Req. Req. Req. 
Saturday, October 18, 14
Packet transmission 
OSv 
(Driver) 
BitVisor 
(Device) 
Req. 
PRO/1000 
Req. 
TX Ring 
Req. 
Req. Req. Req. 
Req. Req. Req. 
Req. 
Saturday, October 18, 14
Packet transmission 
OSv 
(Driver) 
BitVisor 
(Device) 
Req. 
PRO/1000 
Req. 
TX Ring 
Req. 
Req. Req. Req. 
Req. Req. Req. 
Req. 
Notify! 
(PIO) 
Saturday, October 18, 14
Packet transmission 
OSv 
(Driver) 
BitVisor 
(Device) 
Req. 
PRO/1000 
Req. 
TX Ring 
Req. 
Req. Req. Req. 
Req. Req. Req. 
Req. 
Notify! 
(PIO) 
Req. 
Req. 
Saturday, October 18, 14
Packet transmission 
OSv 
(Driver) 
BitVisor 
(Device) 
Req. 
PRO/1000 
Req. 
TX Ring 
Req. 
Req. Req. Req. 
Req. Req. Req. 
Req. 
Notify! 
(PIO) 
Req. 
Req. 
Saturday, October 18, 14
Packet transmission 
OSv 
(Driver) 
BitVisor 
(Device) 
Req. 
PRO/1000 
Req. 
TX Ring 
Req. 
Req. Req. Req. 
Req. Req. Req. 
Req. 
Notify! 
(PIO) 
Req. 
Req. 
Reuse 
Saturday, October 18, 14
Packet reception 
OSv 
(Driver) 
BitVisor 
(Device) 
Buf. 
PRO/1000 
Buf. 
RX Ring 
Buf. 
Buf. Buf. Buf. 
Buf. Buf. Buf. 
Saturday, October 18, 14
Packet reception 
OSv 
(Driver) 
BitVisor 
(Device) 
Buf. 
PRO/1000 
Buf. 
RX Ring 
Buf. 
Buf. Buf. Buf. 
Buf. Buf. Buf. 
Buf. 
Saturday, October 18, 14
Packet reception 
OSv 
(Driver) 
BitVisor 
(Device) 
Buf. 
PRO/1000 
Buf. 
RX Ring 
Buf. 
Buf. Buf. Buf. 
Buf. Buf. Buf. 
Buf. 
Buf. 
Saturday, October 18, 14
Packet reception 
OSv 
(Driver) 
BitVisor 
(Device) 
Buf. 
PRO/1000 
Buf. 
RX Ring 
Buf. 
Buf. Buf. Buf. 
Buf. Buf. Buf. 
Buf. 
Buf. 
Buf. 
Saturday, October 18, 14
Packet reception 
OSv 
(Driver) 
BitVisor 
(Device) 
Buf. 
PRO/1000 
Buf. 
RX Ring 
Buf. 
Buf. Buf. Buf. 
Buf. Buf. Buf. 
Buf. 
Buf. 
Buf. 
Saturday, October 18, 14
Packet reception 
OSv 
(Driver) 
BitVisor 
(Device) 
Buf. 
PRO/1000 
Buf. 
RX Ring 
Buf. 
Buf. Buf. Buf. 
Buf. Buf. Buf. 
Buf. 
Buf. 
Buf. 
Int! 
Saturday, October 18, 14
Packet reception 
OSv 
(Driver) 
BitVisor 
(Device) 
Buf. 
PRO/1000 
Buf. 
RX Ring 
Buf. 
Buf. Buf. Buf. 
Buf. Buf. Buf. 
Buf. 
Int! 
Buf. 
Buf. 
Receive 
Saturday, October 18, 14
Interrupt BAR1 
(MMIO) 
MSI-X Message Data: 
Int. vector number (e.g. 0x26) 
VMX event injection 
for virtual interrupts 
Saturday, October 18, 14
DHCP succeeded ! 
Saturday, October 18, 14
Summary  
future work 
• Summary 
• Booting OSv on BitVisor 
• Virtio NIC 
• Future work 
• Further testing 
• Virtio BLK/RNG/... 
Saturday, October 18, 14
Thank you ! 
http://www.root.ne.jp/nishide/shs/ 
Saturday, October 18, 14

More Related Content

What's hot

What's hot (18)

Unveiling EMUX - ARM and MIPS IoT Emulation Framework
Unveiling EMUX - ARM and MIPS IoT Emulation FrameworkUnveiling EMUX - ARM and MIPS IoT Emulation Framework
Unveiling EMUX - ARM and MIPS IoT Emulation Framework
 
What's New in ES6 for Web Devs
What's New in ES6 for Web DevsWhat's New in ES6 for Web Devs
What's New in ES6 for Web Devs
 
Hack.LU 2018 ARM IoT Firmware Emulation Workshop
Hack.LU 2018 ARM IoT Firmware Emulation WorkshopHack.LU 2018 ARM IoT Firmware Emulation Workshop
Hack.LU 2018 ARM IoT Firmware Emulation Workshop
 
Stm32 develop tool introduction
Stm32 develop tool introductionStm32 develop tool introduction
Stm32 develop tool introduction
 
INSIDE ARM-X Cansecwest 2020
INSIDE ARM-X Cansecwest 2020INSIDE ARM-X Cansecwest 2020
INSIDE ARM-X Cansecwest 2020
 
Introducing ARM-X
Introducing ARM-XIntroducing ARM-X
Introducing ARM-X
 
IoT Chess 16th April Berlin
IoT Chess 16th April BerlinIoT Chess 16th April Berlin
IoT Chess 16th April Berlin
 
Retrospective: Seven VM Engineering Years
Retrospective: Seven VM Engineering YearsRetrospective: Seven VM Engineering Years
Retrospective: Seven VM Engineering Years
 
Kernel Recipes 2013 - kconfig-frontends, a packaging of the kconfig parser an...
Kernel Recipes 2013 - kconfig-frontends, a packaging of the kconfig parser an...Kernel Recipes 2013 - kconfig-frontends, a packaging of the kconfig parser an...
Kernel Recipes 2013 - kconfig-frontends, a packaging of the kconfig parser an...
 
Internet Technology for the Commodore 64
Internet Technology for the Commodore 64Internet Technology for the Commodore 64
Internet Technology for the Commodore 64
 
Control-M 800 - Infrastructure Example
Control-M 800 - Infrastructure ExampleControl-M 800 - Infrastructure Example
Control-M 800 - Infrastructure Example
 
Vm ware fuzzing - defcon russia 20
Vm ware fuzzing  - defcon russia 20Vm ware fuzzing  - defcon russia 20
Vm ware fuzzing - defcon russia 20
 
Mikhail Belopuhov: OpenBSD: Where is crypto headed?
Mikhail Belopuhov: OpenBSD: Where is crypto headed?Mikhail Belopuhov: OpenBSD: Where is crypto headed?
Mikhail Belopuhov: OpenBSD: Where is crypto headed?
 
ARM IoT Firmware Emulation Workshop
ARM IoT Firmware Emulation WorkshopARM IoT Firmware Emulation Workshop
ARM IoT Firmware Emulation Workshop
 
【Manifes2018】私が考えるPCの"最新"学習方法
【Manifes2018】私が考えるPCの"最新"学習方法【Manifes2018】私が考えるPCの"最新"学習方法
【Manifes2018】私が考えるPCの"最新"学習方法
 
DOD 2016 - Ignat Korchagin - Managing Server Secrets at Scale
DOD 2016 - Ignat Korchagin - Managing Server Secrets at ScaleDOD 2016 - Ignat Korchagin - Managing Server Secrets at Scale
DOD 2016 - Ignat Korchagin - Managing Server Secrets at Scale
 
VA-API rust-binding (GStreamer Conference 2017)
VA-API rust-binding (GStreamer Conference 2017)VA-API rust-binding (GStreamer Conference 2017)
VA-API rust-binding (GStreamer Conference 2017)
 
[5]投影片 futurewad樹莓派研習會 141218
[5]投影片 futurewad樹莓派研習會 141218[5]投影片 futurewad樹莓派研習會 141218
[5]投影片 futurewad樹莓派研習會 141218
 

Recently uploaded

ALCOHOL PRODUCTION- Beer Brewing Process.pdf
ALCOHOL PRODUCTION- Beer Brewing Process.pdfALCOHOL PRODUCTION- Beer Brewing Process.pdf
ALCOHOL PRODUCTION- Beer Brewing Process.pdf
Madan Karki
 

Recently uploaded (20)

"United Nations Park" Site Visit Report.
"United Nations Park" Site  Visit Report."United Nations Park" Site  Visit Report.
"United Nations Park" Site Visit Report.
 
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas SachpazisSeismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
 
ALCOHOL PRODUCTION- Beer Brewing Process.pdf
ALCOHOL PRODUCTION- Beer Brewing Process.pdfALCOHOL PRODUCTION- Beer Brewing Process.pdf
ALCOHOL PRODUCTION- Beer Brewing Process.pdf
 
Research Methodolgy & Intellectual Property Rights Series 1
Research Methodolgy & Intellectual Property Rights Series 1Research Methodolgy & Intellectual Property Rights Series 1
Research Methodolgy & Intellectual Property Rights Series 1
 
RESORT MANAGEMENT AND RESERVATION SYSTEM PROJECT REPORT.pdf
RESORT MANAGEMENT AND RESERVATION SYSTEM PROJECT REPORT.pdfRESORT MANAGEMENT AND RESERVATION SYSTEM PROJECT REPORT.pdf
RESORT MANAGEMENT AND RESERVATION SYSTEM PROJECT REPORT.pdf
 
Software Engineering - Modelling Concepts + Class Modelling + Building the An...
Software Engineering - Modelling Concepts + Class Modelling + Building the An...Software Engineering - Modelling Concepts + Class Modelling + Building the An...
Software Engineering - Modelling Concepts + Class Modelling + Building the An...
 
How to Design and spec harmonic filter.pdf
How to Design and spec harmonic filter.pdfHow to Design and spec harmonic filter.pdf
How to Design and spec harmonic filter.pdf
 
Instruct Nirmaana 24-Smart and Lean Construction Through Technology.pdf
Instruct Nirmaana 24-Smart and Lean Construction Through Technology.pdfInstruct Nirmaana 24-Smart and Lean Construction Through Technology.pdf
Instruct Nirmaana 24-Smart and Lean Construction Through Technology.pdf
 
15-Minute City: A Completely New Horizon
15-Minute City: A Completely New Horizon15-Minute City: A Completely New Horizon
15-Minute City: A Completely New Horizon
 
Intelligent Agents, A discovery on How A Rational Agent Acts
Intelligent Agents, A discovery on How A Rational Agent ActsIntelligent Agents, A discovery on How A Rational Agent Acts
Intelligent Agents, A discovery on How A Rational Agent Acts
 
The battle for RAG, explore the pros and cons of using KnowledgeGraphs and Ve...
The battle for RAG, explore the pros and cons of using KnowledgeGraphs and Ve...The battle for RAG, explore the pros and cons of using KnowledgeGraphs and Ve...
The battle for RAG, explore the pros and cons of using KnowledgeGraphs and Ve...
 
Online book store management system project.pdf
Online book store management system project.pdfOnline book store management system project.pdf
Online book store management system project.pdf
 
NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024
NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024
NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024
 
Attraction and Repulsion type Moving Iron Instruments.pptx
Attraction and Repulsion type Moving Iron Instruments.pptxAttraction and Repulsion type Moving Iron Instruments.pptx
Attraction and Repulsion type Moving Iron Instruments.pptx
 
Research Methodolgy & Intellectual Property Rights Series 2
Research Methodolgy & Intellectual Property Rights Series 2Research Methodolgy & Intellectual Property Rights Series 2
Research Methodolgy & Intellectual Property Rights Series 2
 
SLIDESHARE PPT-DECISION MAKING METHODS.pptx
SLIDESHARE PPT-DECISION MAKING METHODS.pptxSLIDESHARE PPT-DECISION MAKING METHODS.pptx
SLIDESHARE PPT-DECISION MAKING METHODS.pptx
 
Fabrication Of Automatic Star Delta Starter Using Relay And GSM Module By Utk...
Fabrication Of Automatic Star Delta Starter Using Relay And GSM Module By Utk...Fabrication Of Automatic Star Delta Starter Using Relay And GSM Module By Utk...
Fabrication Of Automatic Star Delta Starter Using Relay And GSM Module By Utk...
 
2024 DevOps Pro Europe - Growing at the edge
2024 DevOps Pro Europe - Growing at the edge2024 DevOps Pro Europe - Growing at the edge
2024 DevOps Pro Europe - Growing at the edge
 
Filters for Electromagnetic Compatibility Applications
Filters for Electromagnetic Compatibility ApplicationsFilters for Electromagnetic Compatibility Applications
Filters for Electromagnetic Compatibility Applications
 
Introduction to Machine Learning Unit-4 Notes for II-II Mechanical Engineering
Introduction to Machine Learning Unit-4 Notes for II-II Mechanical EngineeringIntroduction to Machine Learning Unit-4 Notes for II-II Mechanical Engineering
Introduction to Machine Learning Unit-4 Notes for II-II Mechanical Engineering
 

OSb: OSv on BitVisor

  • 1. OSb: OSv on BitVisor @ysmoo Saturday, October 18, 14
  • 2. BitVisor App App App BitVisor Hardware OS http://www.justis.as-1.co.jp Saturday, October 18, 14
  • 3. OSv VMM Hardware App OSv http://medical-care.feed.jp Saturday, October 18, 14
  • 5. BitVisor OSv Saturday, October 18, 14
  • 7. OSv on BitVisor BitVisor Hardware App OSv Saturday, October 18, 14
  • 8. OSv on BitVisor BitVisor Hardware App OSv OSb Saturday, October 18, 14
  • 9. OSv on BitVisor BitVisor Hardware App OSv What the fuck no Virtio ! OSb Physical interface sucks ! What the hell is PRO/1000 ! Saturday, October 18, 14
  • 10. OSv on BitVisor BitVisor Hardware App OSv What the fuck no Virtio ! OSb Physical interface sucks ! What the hell is PRO/1000 ! You have no choice. Virtio Saturday, October 18, 14
  • 11. ~ The road to OSb ~ Saturday, October 18, 14
  • 12. OSv code reading • Boot process • Some drivers Saturday, October 18, 14
  • 13. OSv code reading • Boot process • Some drivers - MBR - Read local disk with INT13/42. - Load command line, boot loader, OSv kernel. - Get memory map with INT15/E820. - Setup segment descriptors/page tables. - Switch to 64-bit mode. - premain() - main() ... Saturday, October 18, 14
  • 14. OSv code reading • Boot process • Some drivers - MBR - Read local disk with INT13/42. - Load command line, boot loader, OSv kernel. - Get memory map with INT15/E820. - Setup segment descriptors/page tables. - Switch to 64-bit mode. - premain() - main() ... - Serial / VGA output. - SATA. - Virtio NIC/BLK/RNG/SCSI. - ACPI. - APIC. ... Saturday, October 18, 14
  • 15. OSv code reading • Boot process • Some drivers - MBR - Read local disk with INT13/42. - Load command line, boot loader, OSv kernel. - Get memory map with INT15/E820. - Setup segment descriptors/page tables. - Switch to 64-bit mode. - premain() - main() ... - Serial / VGA output. - SATA. - Virtio NIC/BLK/RNG/SCSI. - ACPI. - APIC. ... Straightforward... Saturday, October 18, 14
  • 16. Booting OSb ./scripts/run.py Saturday, October 18, 14
  • 17. Booting OSb In the case of normal OSv... ./scripts/run.py OSv VMM Saturday, October 18, 14
  • 18. Booting OSb Hardware ./scripts/run.py Saturday, October 18, 14
  • 19. Booting OSb Hardware ./scripts/run.py Image Update Saturday, October 18, 14
  • 20. Booting OSb Hardware ./scripts/run.py Image Update IPMI Saturday, October 18, 14
  • 21. Booting OSb Hardware ./scripts/run.py Image Update PXE Boot IPMI Saturday, October 18, 14
  • 22. Booting OSb Hardware ./scripts/run.py Image Update PXE Boot IPMI BitVisor Saturday, October 18, 14
  • 23. Booting OSb Hardware ./scripts/run.py Image Update PXE Boot IPMI BitVisor OSv Network Boot Saturday, October 18, 14
  • 24. Booting OSb Hardware ./scripts/run.py Image Update PXE Boot IPMI BitVisor OSv Network Boot App Saturday, October 18, 14
  • 25. Booting OSb BitVisor Hardware App OSv Network Boot IPMI ./scripts/run.py TTY Image Update PXE Boot Saturday, October 18, 14
  • 26. ACPI workaround drivers/acpi.cc: // Copy the root table list to dynamic memory if (!is_bitvisor()) { status = AcpiReallocateRootTable(); if (ACPI_FAILURE(status)) { acpi_e("AcpiReallocateRootTable failed: %sn", AcpiFormatException(status)); return; } } Saturday, October 18, 14
  • 27. ACPI workaround drivers/acpi.cc: // Copy the root table list to dynamic memory if (!is_bitvisor()) { status = AcpiReallocateRootTable(); if (ACPI_FAILURE(status)) { acpi_e("AcpiReallocateRootTable failed: %sn", AcpiFormatException(status)); return; } } Triple fault Saturday, October 18, 14
  • 28. ACPI workaround drivers/acpi.cc: // Copy the root table list to dynamic memory if (!is_bitvisor()) { status = AcpiReallocateRootTable(); if (ACPI_FAILURE(status)) { acpi_e("AcpiReallocateRootTable failed: %sn", AcpiFormatException(status)); return; } } Triple fault Skip in case of OSb Saturday, October 18, 14
  • 29. ACPI workaround drivers/acpi.cc: // Copy the root table list to dynamic memory if (!is_bitvisor()) { status = AcpiReallocateRootTable(); if (ACPI_FAILURE(status)) { acpi_e("AcpiReallocateRootTable failed: %sn", AcpiFormatException(status)); return; } } Triple fault Skip in case of OSb Machine-specific...? Saturday, October 18, 14
  • 30. Hello world ! Saturday, October 18, 14
  • 31. Performance? 20.75& 21.20& 25.00& 20.00& 15.00& 10.00& 5.00& 0.00& OSb& OSv& Elapsed(Time((sec) for((12(billion(6mes)(val++; Saturday, October 18, 14
  • 32. Virtio NIC BitVisor Hardware App Actually PRO/1000... OSv Saturday, October 18, 14
  • 33. Virtio NIC BitVisor Hardware App OSv Virtio NIC Actually PRO/1000... Fake Virtio NIC... Saturday, October 18, 14
  • 34. Virtio NIC BitVisor Hardware App OSv Virtio NIC Actually PRO/1000... Fake Virtio NIC... Hook PCI config access, MMIO, PIO and fake! Saturday, October 18, 14
  • 35. Faking PCI configuration space • Faking IDs • Faking BARs • Faking capabilities Saturday, October 18, 14
  • 36. Faking IDs 0x1AF4 (Virtio Device) Saturday, October 18, 14
  • 37. Faking IDs 0x1AF4 (Virtio Device) 0x1000 (Virtio NIC) Saturday, October 18, 14
  • 38. Faking IDs 0x1AF4 (Virtio Device) 0x1000 (Virtio NIC) 0 (Legacy Virtio) Saturday, October 18, 14
  • 39. Faking IDs 0x1AF4 (Virtio Device) 0x1000 (Virtio NIC) 0 (Legacy Virtio) 1 (Virtio NIC) Saturday, October 18, 14
  • 40. Faking BARs BAR0: IO space for Virtio BAR1: MMIO space for MSI-X Saturday, October 18, 14
  • 41. Faking BARs BAR0: IO space for Virtio BAR1: MMIO space for MSI-X = PRO/1000 IO space Saturday, October 18, 14
  • 42. Faking BARs BAR0: IO space for Virtio BAR1: MMIO space for MSI-X = PRO/1000 IO space = PRO/1000 MMIO space Saturday, October 18, 14
  • 43. Faking capabilities Add MSI-X capability Saturday, October 18, 14
  • 44. Faking capabilities Add MSI-X capability Support 3 vectors Saturday, October 18, 14
  • 45. Faking capabilities Add MSI-X capability Support 3 vectors Vector 0: Reception Saturday, October 18, 14
  • 46. Faking capabilities Add MSI-X capability Support 3 vectors Vector 0: Reception Vector 1: Transmission Saturday, October 18, 14
  • 47. Faking capabilities Add MSI-X capability Support 3 vectors Vector 0: Reception Vector 1: Transmission Vector 2: Control Saturday, October 18, 14
  • 48. Virtio NIC Operations • Virtio ring • Packet transmission • Packet reception Saturday, October 18, 14
  • 49. Virtio ring Controlled by these registers BAR0 (IO) Saturday, October 18, 14
  • 50. Virtio ring Controlled by these registers 3 rings available BAR0 (IO) Saturday, October 18, 14
  • 51. Virtio ring BAR0 Ring 0: Reception (IO) Controlled by these registers 3 rings available Saturday, October 18, 14
  • 52. Virtio ring Controlled by these registers 3 rings available Ring 0: Reception Ring 1: Transmission BAR0 (IO) Saturday, October 18, 14
  • 53. Virtio ring Controlled by these registers 3 rings available Ring 0: Reception Ring 1: Transmission Ring 2: Control BAR0 (IO) Saturday, October 18, 14
  • 54. Virtio ring Controlled by these registers 3 rings available Ring 0: Reception Ring 1: Transmission Ring 2: Control BAR0 (IO) (Memory) Saturday, October 18, 14
  • 55. Virtio ring Controlled by these registers 3 rings available Ring 0: Reception Ring 1: Transmission Ring 2: Control BAR0 (IO) (Memory) Driver 2 Device Device 2 Driver Saturday, October 18, 14
  • 56. Packet transmission OSv (Driver) BitVisor (Device) PRO/1000 Saturday, October 18, 14
  • 57. Packet transmission OSv (Driver) BitVisor (Device) Req. PRO/1000 Req. Req. Saturday, October 18, 14
  • 58. Packet transmission OSv (Driver) BitVisor (Device) Req. PRO/1000 Req. Req. Req. Req. Req. Saturday, October 18, 14
  • 59. Packet transmission OSv (Driver) BitVisor (Device) Req. PRO/1000 Req. TX Ring Req. Req. Req. Req. Req. Req. Req. Saturday, October 18, 14
  • 60. Packet transmission OSv (Driver) BitVisor (Device) Req. PRO/1000 Req. TX Ring Req. Req. Req. Req. Req. Req. Req. Req. Saturday, October 18, 14
  • 61. Packet transmission OSv (Driver) BitVisor (Device) Req. PRO/1000 Req. TX Ring Req. Req. Req. Req. Req. Req. Req. Req. Notify! (PIO) Saturday, October 18, 14
  • 62. Packet transmission OSv (Driver) BitVisor (Device) Req. PRO/1000 Req. TX Ring Req. Req. Req. Req. Req. Req. Req. Req. Notify! (PIO) Req. Req. Saturday, October 18, 14
  • 63. Packet transmission OSv (Driver) BitVisor (Device) Req. PRO/1000 Req. TX Ring Req. Req. Req. Req. Req. Req. Req. Req. Notify! (PIO) Req. Req. Saturday, October 18, 14
  • 64. Packet transmission OSv (Driver) BitVisor (Device) Req. PRO/1000 Req. TX Ring Req. Req. Req. Req. Req. Req. Req. Req. Notify! (PIO) Req. Req. Reuse Saturday, October 18, 14
  • 65. Packet reception OSv (Driver) BitVisor (Device) Buf. PRO/1000 Buf. RX Ring Buf. Buf. Buf. Buf. Buf. Buf. Buf. Saturday, October 18, 14
  • 66. Packet reception OSv (Driver) BitVisor (Device) Buf. PRO/1000 Buf. RX Ring Buf. Buf. Buf. Buf. Buf. Buf. Buf. Buf. Saturday, October 18, 14
  • 67. Packet reception OSv (Driver) BitVisor (Device) Buf. PRO/1000 Buf. RX Ring Buf. Buf. Buf. Buf. Buf. Buf. Buf. Buf. Buf. Saturday, October 18, 14
  • 68. Packet reception OSv (Driver) BitVisor (Device) Buf. PRO/1000 Buf. RX Ring Buf. Buf. Buf. Buf. Buf. Buf. Buf. Buf. Buf. Buf. Saturday, October 18, 14
  • 69. Packet reception OSv (Driver) BitVisor (Device) Buf. PRO/1000 Buf. RX Ring Buf. Buf. Buf. Buf. Buf. Buf. Buf. Buf. Buf. Buf. Saturday, October 18, 14
  • 70. Packet reception OSv (Driver) BitVisor (Device) Buf. PRO/1000 Buf. RX Ring Buf. Buf. Buf. Buf. Buf. Buf. Buf. Buf. Buf. Buf. Int! Saturday, October 18, 14
  • 71. Packet reception OSv (Driver) BitVisor (Device) Buf. PRO/1000 Buf. RX Ring Buf. Buf. Buf. Buf. Buf. Buf. Buf. Buf. Int! Buf. Buf. Receive Saturday, October 18, 14
  • 72. Interrupt BAR1 (MMIO) MSI-X Message Data: Int. vector number (e.g. 0x26) VMX event injection for virtual interrupts Saturday, October 18, 14
  • 73. DHCP succeeded ! Saturday, October 18, 14
  • 74. Summary future work • Summary • Booting OSv on BitVisor • Virtio NIC • Future work • Further testing • Virtio BLK/RNG/... Saturday, October 18, 14
  • 75. Thank you ! http://www.root.ne.jp/nishide/shs/ Saturday, October 18, 14