The document outlines the agenda and discussions from the Oracle Xen Developer Summit held on October 24-25, 2013, focusing on profiling methods, existing tools, and the performance (perf) framework within the Xen hypervisor. It details desired properties, high-level design features, current limitations, and future enhancements for profiling in Xen. The presentation emphasizes managing performance monitoring unit (PMU) contexts and the need for ongoing development and coordination with Linux patches.

1. Xen PMU
Boris Ostrovsky, Oracle
Xen Developer Summit
Oct 24-25, 2013
1Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

2. Agenda

Profiling methods

Existing tools

Why perf

Design requirements

High-level design

Status

Next steps
2Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

3. Profiling Methods

Timing

Counting



Software: counting probes
Hardware: performance counters
Sampling



Software: timer-based
Hardware: HW counters, IBS, PEBS
Tracing

Software events: recording probes

Hardware events (Intel BTS)
3Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

4. Existing Tools

Guests

SW-based methods: any guest, most tools

HW-based methods:



HVM: Counters, BTS: most tools
PV: xenoprofile
Hypervisor

SW-based methods: xentrace, xenalyze, xentop, xenperf

HW-based methods: xenoprofile
4Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

5. Why perf

Why not (xen)oprofile



not inthe tree
not in active development
perf

in the tree

active development

lots of features
5Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

6. Desired Properties

Use HVM VPMU for both PV and HVM

As little as possible non-Xen changes in Linux kernel


Staged development
Possibility of other tools using the same framework
6Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

7. (Very) High-Level Design Features

Manage PMU in current VPMU



VPMU states
VPMU context (i.e. registers)
On PMU interrupt stop all counters

Emulate PMU in Linux to avoid traps on MSR accesses

Domain re-enables PMU via hypercall when done processing
the interrupt

Global profile mode


Dom0 profiles everyone
APIC and NMI mode
7Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

8. Status

Limitations


Only profile hypervisor CPUs where dom0 is running

Only counters are supported (no IBS/PEBS)


PV VCPUs should be pinned
No backtrace support (for the hypervisor)
V3 for Xen in progress

4.4 is unlikely

coordinate with Linux patches

perf toolstack changes (not Xen-specific code)
8Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

9. Future Enhancements

Address limitations from stage 1

No pinning requirement

#PCPUs > #VCPUs in dom0

Tracing multiple guests in global mode

New HW features (PEBS, IBS)

Tracing (xentrace?)
9Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

10. Example (perf record/report)
10Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

11. Example (perf kvm record/report)
11Copyright © 2013, Oracle and/or its affiliates. All rights reserved.