Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
A bit of history
QEMU upstream
Linux-based Device Model Stubdomains in Qubes
OS
Marek Marczykowski-G´orecki, Invisible Thi...
A bit of history
QEMU upstream
A bit of history
1 Qubes 1.0 (2012) - PV domains only
Marek Marczykowski-G´orecki, Invisibl...
A bit of history
QEMU upstream
A bit of history
1 Qubes 1.0 (2012) - PV domains only
2 Qubes 2.0 (2014) - initial HVM supp...
A bit of history
QEMU upstream
A bit of history
1 Qubes 1.0 (2012) - PV domains only
2 Qubes 2.0 (2014) - initial HVM supp...
A bit of history
QEMU upstream
MiniOS based stubdomains in Qubes
No qemu in dom0 (hard requirement) - patched libxl
Displa...
A bit of history
QEMU upstream
Why change?
Hard to debug and develop
Marek Marczykowski-G´orecki, Invisible Things Lab Lin...
A bit of history
QEMU upstream
Why change?
Hard to debug and develop
Lack of newer device support (audio, vbkd, . . . )
Ma...
A bit of history
QEMU upstream
Why change?
Hard to debug and develop
Lack of newer device support (audio, vbkd, . . . )
Ho...
A bit of history
QEMU upstream
Why change?
Hard to debug and develop
Lack of newer device support (audio, vbkd, . . . )
Ho...
A bit of history
QEMU upstream
Why change?
Hard to debug and develop
Lack of newer device support (audio, vbkd, . . . )
Ho...
A bit of history
QEMU upstream
What we need?
No qemu in dom0
PCI passthrough
Custom GUI (instead of VNC/SDL)
DHCP server (...
A bit of history
QEMU upstream
Which stubdomain?
Rumprun, not progressing at that time, dead now
Marek Marczykowski-G´orec...
A bit of history
QEMU upstream
Which stubdomain?
Rumprun, not progressing at that time, dead now
Linux, deployed by OpenXT...
A bit of history
QEMU upstream
Which stubdomain?
Rumprun, not progressing at that time, dead now
Linux, deployed by OpenXT...
A bit of history
QEMU upstream
Stubdomain build blocks
dracut-based build - use build host binaries (busybox, glibc
etc)
m...
A bit of history
QEMU upstream
Make it work
Load kernel + initramfs (can be bundled into one binary later,
but simpler to ...
A bit of history
QEMU upstream
Make it work
Load kernel + initramfs (can be bundled into one binary later,
but simpler to ...
A bit of history
QEMU upstream
Make it work
Load kernel + initramfs (can be bundled into one binary later,
but simpler to ...
A bit of history
QEMU upstream
Make it work
Load kernel + initramfs (can be bundled into one binary later,
but simpler to ...
A bit of history
QEMU upstream
Make it work
Load kernel + initramfs (can be bundled into one binary later,
but simpler to ...
A bit of history
QEMU upstream
Problems
PCI passthrough related problems
Marek Marczykowski-G´orecki, Invisible Things Lab...
A bit of history
QEMU upstream
Problems
PCI passthrough related problems
RAM usage, CPU usage
Marek Marczykowski-G´orecki,...
A bit of history
QEMU upstream
Problems
PCI passthrough related problems
RAM usage, CPU usage
read-only disks (IDE vs AHCI...
A bit of history
QEMU upstream
Problems
PCI passthrough related problems
RAM usage, CPU usage
read-only disks (IDE vs AHCI...
A bit of history
QEMU upstream
Next steps
Cleanup libxl patches
Xenconsoled support for secondary consoles, fix
save/migrat...
A bit of history
QEMU upstream
Next steps
Cleanup libxl patches
Xenconsoled support for secondary consoles, fix
save/migrat...
A bit of history
QEMU upstream
Resources
github.com/QubesOS/qubes-vmm-xen
github.com/QubesOS/qubes-vmm-xen-stubdom-linux
M...
A bit of history
QEMU upstream
Questions
Questions?
Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device M...
Upcoming SlideShare
Loading in …5
×

XPDDS18: Linux-based Device Model Stubdomains in Qubes OS - Marek Marczykowski-Górecki, Invisible Things Lab

49 views

Published on

One of the killer features of Xen is the ability to contain qemu in a minimal stubdomain. But even though qemu-upstream has been supported by Xen for a long time, stubdomains are compatible only with the ancient qemu-traditional. There were multiple approaches to this problem discussed over time (rumprun, Linux, ...), including some PoC patches. In this presentation I'll explain why we've chosen the Linux solution in Qubes OS and what challenges we faced to make it really work.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

XPDDS18: Linux-based Device Model Stubdomains in Qubes OS - Marek Marczykowski-Górecki, Invisible Things Lab

  1. 1. A bit of history QEMU upstream Linux-based Device Model Stubdomains in Qubes OS Marek Marczykowski-G´orecki, Invisible Things Lab June 22, 2018 Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  2. 2. A bit of history QEMU upstream A bit of history 1 Qubes 1.0 (2012) - PV domains only Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  3. 3. A bit of history QEMU upstream A bit of history 1 Qubes 1.0 (2012) - PV domains only 2 Qubes 2.0 (2014) - initial HVM support, MiniOS based stubdomain for qemu-traditional (the only one at that time) Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  4. 4. A bit of history QEMU upstream A bit of history 1 Qubes 1.0 (2012) - PV domains only 2 Qubes 2.0 (2014) - initial HVM support, MiniOS based stubdomain for qemu-traditional (the only one at that time) 3 Qubes 4.0 (2018) - most PVHv2 domains, HVM with linux-based and qemu upstream stubdomains where needed Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  5. 5. A bit of history QEMU upstream MiniOS based stubdomains in Qubes No qemu in dom0 (hard requirement) - patched libxl Display using qubes-gui-agent (port for qemu) DHCP server based on LWIP Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  6. 6. A bit of history QEMU upstream Why change? Hard to debug and develop Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  7. 7. A bit of history QEMU upstream Why change? Hard to debug and develop Lack of newer device support (audio, vbkd, . . . ) Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  8. 8. A bit of history QEMU upstream Why change? Hard to debug and develop Lack of newer device support (audio, vbkd, . . . ) Hope to get better security support for currently maintained qemu (as we don’t consider PV bulletproof anymore) Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  9. 9. A bit of history QEMU upstream Why change? Hard to debug and develop Lack of newer device support (audio, vbkd, . . . ) Hope to get better security support for currently maintained qemu (as we don’t consider PV bulletproof anymore) MiniOS build system gives a lot of headache. . . Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  10. 10. A bit of history QEMU upstream Why change? Hard to debug and develop Lack of newer device support (audio, vbkd, . . . ) Hope to get better security support for currently maintained qemu (as we don’t consider PV bulletproof anymore) MiniOS build system gives a lot of headache. . . Direct kernel boot for HVM Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  11. 11. A bit of history QEMU upstream What we need? No qemu in dom0 PCI passthrough Custom GUI (instead of VNC/SDL) DHCP server (nice to have) Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  12. 12. A bit of history QEMU upstream Which stubdomain? Rumprun, not progressing at that time, dead now Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  13. 13. A bit of history QEMU upstream Which stubdomain? Rumprun, not progressing at that time, dead now Linux, deployed by OpenXT (use OpenEmbedded, and v4v for communication) Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  14. 14. A bit of history QEMU upstream Which stubdomain? Rumprun, not progressing at that time, dead now Linux, deployed by OpenXT (use OpenEmbedded, and v4v for communication) Linux, patches by Anthony Perard, later revived by Eric Shelton Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  15. 15. A bit of history QEMU upstream Stubdomain build blocks dracut-based build - use build host binaries (busybox, glibc etc) minimal Linux kernel (based on make tinyconfig) recent upstream QEMU build (2.10.1 as of today) udhcpd (busybox) Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  16. 16. A bit of history QEMU upstream Make it work Load kernel + initramfs (can be bundled into one binary later, but simpler to develop when separate) Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  17. 17. A bit of history QEMU upstream Make it work Load kernel + initramfs (can be bundled into one binary later, but simpler to develop when separate) Pass qemu command line via xenstore (as for MiniOS) - putting qemu-xen variant there Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  18. 18. A bit of history QEMU upstream Make it work Load kernel + initramfs (can be bundled into one binary later, but simpler to develop when separate) Pass qemu command line via xenstore (as for MiniOS) - putting qemu-xen variant there -append can contain spaces. . . , workaround: use x1b separator and set FS= x1b Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  19. 19. A bit of history QEMU upstream Make it work Load kernel + initramfs (can be bundled into one binary later, but simpler to develop when separate) Pass qemu command line via xenstore (as for MiniOS) - putting qemu-xen variant there -append can contain spaces. . . , workaround: use x1b separator and set FS= x1b Different disk configuration: format=host device Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  20. 20. A bit of history QEMU upstream Make it work Load kernel + initramfs (can be bundled into one binary later, but simpler to develop when separate) Pass qemu command line via xenstore (as for MiniOS) - putting qemu-xen variant there -append can contain spaces. . . , workaround: use x1b separator and set FS= x1b Different disk configuration: format=host device No direct access to QMP socket - pass selected commands via xenstore (as for MiniOS), then convert to QMP commands inside (a script) Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  21. 21. A bit of history QEMU upstream Problems PCI passthrough related problems Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  22. 22. A bit of history QEMU upstream Problems PCI passthrough related problems RAM usage, CPU usage Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  23. 23. A bit of history QEMU upstream Problems PCI passthrough related problems RAM usage, CPU usage read-only disks (IDE vs AHCI vs SCSI), supported by Windows installer by default (right now we use mptsas1068) Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  24. 24. A bit of history QEMU upstream Problems PCI passthrough related problems RAM usage, CPU usage read-only disks (IDE vs AHCI vs SCSI), supported by Windows installer by default (right now we use mptsas1068) no migration / save+restore without qemu in dom0 (only one console) Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  25. 25. A bit of history QEMU upstream Next steps Cleanup libxl patches Xenconsoled support for secondary consoles, fix save/migration Better design for QMP access PVH stubdomain? Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  26. 26. A bit of history QEMU upstream Next steps Cleanup libxl patches Xenconsoled support for secondary consoles, fix save/migration Better design for QMP access, how fragile is libxl parsing QMP response? PVH stubdomain? Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  27. 27. A bit of history QEMU upstream Resources github.com/QubesOS/qubes-vmm-xen github.com/QubesOS/qubes-vmm-xen-stubdom-linux Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS
  28. 28. A bit of history QEMU upstream Questions Questions? Marek Marczykowski-G´orecki, Invisible Things Lab Linux-based Device Model Stubdomains in Qubes OS

×