The document outlines the Xen testing processes at Intel, focusing on pre-silicon and post-silicon testing methods, including the use of Wind River Simics for virtual hardware simulations. It details the test coverage for various Intel hardware features and describes the internal and upstream testing cycles, including bug reporting and release testing. Overall, the testing efforts aim to enhance Xen's quality and ensure its compatibility with new Intel hardware releases.

1. 1
Xen Testing at Intel
Xudong Hao
xudong.hao@intel.com
June 2018

2. 2
Agenda
• Introduction
• Test Coverage
• Pre Silicon Test
• Post Silicon Test
• Test Cycles

3. 3
Introduction
• Involved in Xen testing since 2005
• Focus on Intel hardware new features
• Contribute to Xen quality

4. 4
Test Coverage
4
Xeon®
……
HVM X86_64
EPT
ACPI
RAS
VT-d
SR-IOV
SMAP
SMEP
Nested
CMT
Live migration
Altp2m
…
Broadwell Skylake Icelake
PML
VT-d PI
CAT
CDP
MBM
MPX
AVX512
XSAVES
RDTSC Scaling
Protection Key
Local MCE
MBA
UMIP
PT VMX
5 level paging
SPP
SGX

5. 5
Test Coverage (2)
5
L2 CAT
Atom®
TVX
288 vCPU
MCDRAM
PMU
NUMA Distance
System topology
KNM NIs
Xeon-Phi®Accelerator
FPGA
Denverton Knights Mill
Nervana

6. 6
Pre Silicon Test
• Shift left development and testing for Xen
• But hardware is not yet available – How to verify?
• Wind River® Simics®
6

7. 7
Test Xen on Simics
 Virtual/simulated target hardware
 Simulate any computer system,
such as: Single board, multiple
boards, custom chips, networks, …
 Run the same software as the
physical system
 Xen could run on the simulated
HW without modification
Wind River® Simics®
Host hardware
Host operating system
Xen hypervisor
Dom0
Apps
Target OS
Network
Virtual/simulated
target hardware
Simulated HW
Dom1 Dom2

8. 8
Post Silicon Test
 Automation Test
framework
 XVS (Xen Validation
System)
 Integrated XTF into
XVS

9. 9
Test Cycles
• Internal Testing
• Test Xen internal repo before patches sent out
• Upstream Testing
• Monitor Xen master
• Nightly and Weekly
• Cover the recent two generation HWs
• Bug report and verification
• Release Testing
• XVS + manual testing
• Big release. e.g. Xen 4.9.0, 4.10.0, 4.11.0… (each RC*)
• Test report to Xen community mail list

10. 10
Xen Release Pass Rate
10
68%
90%
96% 96%
0%
20%
40%
60%
80%
100%
120%
Xen 4.9.0-rc1 Xen 4.9.0 Xen 4.10.0 Xen 4.11.0-rc5
Xen Release Pass Rate
Xen 4.9 testing, closed
19 bugs, raised 2 out and
got fixed.

11. 11
References
• Wind River® Simics® product:
http://www.windriver.com/products/simics/
• Wind River academic program including Simics:
http://www.windriver.com/universities/
• Xen nested status wiki:
https://wiki.xenproject.org/wiki/Nested_Virtualization_in_Xen

12. 12
Q & A

13. Thank you!