Xen is one of most popular virtualization project, many companies and individuals participate in the development, testing and using of Xen. In this presentation, Xudong will give an introduction of Intel QA team's test coverage for Xen, demonstrate the pre check in for new features testing, and post check in for regression testing, as well as test methodology and test framework.

1. 1
Xen Testing at Intel
Xudong Hao
xudong.hao@intel.com
June 2018

2. 2
Agenda
• Introduction
• Test Coverage
• Pre Silicon Test
• Post Silicon Test
• Test Cycles

3. 3
Introduction
• Involved in Xen testing since 2005
• Focus on Intel hardware new features
• Contribute to Xen quality

4. 4
Test Coverage
4
Xeon®
……
HVM X86_64
EPT
ACPI
RAS
VT-d
SR-IOV
SMAP
SMEP
Nested
CMT
Live migration
Altp2m
…
Broadwell Skylake Icelake
PML
VT-d PI
CAT
CDP
MBM
MPX
AVX512
XSAVES
RDTSC Scaling
Protection Key
Local MCE
MBA
UMIP
PT VMX
5 level paging
SPP
SGX

5. 5
Test Coverage (2)
5
L2 CAT
Atom®
TVX
288 vCPU
MCDRAM
PMU
NUMA Distance
System topology
KNM NIs
Xeon-Phi®Accelerator
FPGA
Denverton Knights Mill
Nervana

6. 6
Pre Silicon Test
• Shift left development and testing for Xen
• But hardware is not yet available – How to verify?
• Wind River® Simics®
6

7. 7
Test Xen on Simics
 Virtual/simulated target hardware
 Simulate any computer system,
such as: Single board, multiple
boards, custom chips, networks, …
 Run the same software as the
physical system
 Xen could run on the simulated
HW without modification
Wind River® Simics®
Host hardware
Host operating system
Xen hypervisor
Dom0
Apps
Target OS
Network
Virtual/simulated
target hardware
Simulated HW
Dom1 Dom2

8. 8
Post Silicon Test
 Automation Test
framework
 XVS (Xen Validation
System)
 Integrated XTF into
XVS

9. 9
Test Cycles
• Internal Testing
• Test Xen internal repo before patches sent out
• Upstream Testing
• Monitor Xen master
• Nightly and Weekly
• Cover the recent two generation HWs
• Bug report and verification
• Release Testing
• XVS + manual testing
• Big release. e.g. Xen 4.9.0, 4.10.0, 4.11.0… (each RC*)
• Test report to Xen community mail list

10. 10
Xen Release Pass Rate
10
68%
90%
96% 96%
0%
20%
40%
60%
80%
100%
120%
Xen 4.9.0-rc1 Xen 4.9.0 Xen 4.10.0 Xen 4.11.0-rc5
Xen Release Pass Rate
Xen 4.9 testing, closed
19 bugs, raised 2 out and
got fixed.

11. 11
References
• Wind River® Simics® product:
http://www.windriver.com/products/simics/
• Wind River academic program including Simics:
http://www.windriver.com/universities/
• Xen nested status wiki:
https://wiki.xenproject.org/wiki/Nested_Virtualization_in_Xen

12. 12
Q & A

13. Thank you!