With the rise of containerization, as well as the established adoption of virtualization technologies, run-time power and energy management is becoming one of the key challenges in modern cloud computing. This is also fundamental as power consumption contributes to the 20% of the Total Cost of Ownership of a datacenter and energy costs will exceed hardware costs in the near future. In this context, several goals towards power optimization can be achieved. On the one hand, power capping can be enforced and on top of that the system should be able to maximize performance. On the other hand, when performance are critical, the system should be able to provide a minimum SLA and optimize power consumption without violating it. Within this context, we propose a common autonomic methodology based on the ODA control loop for containers and virtual machines. The proposed methodology is able to achieve 25% power savings for containers and can improve performance under a power cap for virtual machines.

1. Run-%me power management in cloud
and containerized environments
NECST Group Conference 2018 @ MSR
05/31/2018
Rolando Brondolin
<rolando.brondolin@polimi.it>

2. 2
Computer Architecture
Heterogeneous systems
Autonomic computing
Operating systems
Runtime adaptation
Monitoring infrastructures
Computer Architecture and OS

3. ORCA Research topics
E2ASY: Energy Efficiency for Autonomic Scalable sYstems
• Making systems energy and power aware, in particular in a virtualized
and containerized environments
ML
• FPGA-based solutions and CAD frameworks for improving ML solutions
through high performance and adaptable architectures
3

4. Presenta%on outline
• Cloud technologies and perspec%ves
• Power awareness and power management
• A common power-aware run-%me methodology
- Monitoring and op%mising power consump%on for the Xen hypervisor
- Monitoring Docker container power consump%on
- Op%mizing power consump%on for Docker and Kubernetes
• Conclusion and future work
4

5. A cloudy landscape
• Cloud services became more structured and variegated in the last few years
5
Physical Hardware
VM1 VM2
C1 C2 C3 C4
The complexity of the
environment is left
to the Cloud provider

6. The need for power awareness
6
[3] Beloglazov, A., Buyya, R., Lee, Y. C., Zomaya, A., et Al, taxonomy and survey of energy-efficient datacenters
and cloud compu%ng systems. Advances in computers 82, 2 (2011) 47–111.
[2] Cui, Yan, et al. "Total cost of ownership model for data center technology evaluation." Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm),
2017 16th IEEE Intersociety Conference on. IEEE, 2017.
Power consumption is accounted for 20% of a data-center TCO [2]
Lifetime energy cost will exceed hardware cost in the near future [3]
Energy budgets and
power caps constraint the
performance of the system
Power consumption is
affected by a plethora of
different actors
Performance are key for
production systems
despite power consumption

7. Power@NECST: goals and requirements
7
Power consump%on is somehow unavoidable…
Maximize performance
under a power cap
Minimize power consumption while
meeting performance requirements
Goals
…but it should be opBmized (at run-%me)
Need to tackle different cloud technologies,
general methodology but specific tuning
Run-time power optimization
starts from accurate and fine-grain data
Similar problems, different “execu%on”
Requirements

8. A common autonomic methodology
8
O
A D
Lightweight observation
•Performance monitor
•Power monitor and attribution
•Low overhead
•Fine grain and precise
Decide and optimize
•Reactive power-aware control
•Control policy based on overall goals
Fast actuation
•Enforce policies to save power
•Tune power and resource allocation
Proposed general approach
Observe - Decide - Act loop
To cope with the complexity of power and performance management,
an autonomic approach is then needed

9. Research projects
9
XeMPower DEEP-mon
XeMPUPiL HyPPO
O O
A D A D

10. Research projects (virtualiza%on)
10
XeMPower DEEP-mon
XeMPUPiL HyPPO
O O
A D A D

11. XeMPower
• To observe behavior of VMs, two approaches possible:
- white box approach with code instrumentaBon
- black box approach with performance counter monitoring
• White box is too invasive, black box not available for run%me monitoring
• XeMPower is a lightweight hardware and resource monitoring
solu%on for the Xen hypervisor
• Basic Idea: monitor hardware events and precisely account them to each
virtual guest
• Use XeMPower to precisely account CPU power consumpBon to each VM
11
O
A D

12. Fine grain power asribu%on
• For each context switch measure:
- non-halted cycles of the VM
- RAPL socket power
• Collec%on of raw data into dom0
• Compute the non-halted cycles
percentage for each domain
• Use percentage to split evenly
socket power consumpBon
12
XeMPowerCLI
A1
1
B1
A2
2
B2
A1
1
B1
A3
3
B3
A2
2
1
A1
Core 0 Core N
Time
B2
…
…
…
context
switch
context
switch
context
switch
context
switch
XeMPowerDaemon
B2
B2
B1
B1
B3
B2
B2
B1
B1
B3
Xen Kernel Dom0
Hardware events per core,
energy per socket
…
O
A D

13. Fine grain power asribu%on
• For each context switch measure:
- non-halted cycles of the VM
- RAPL socket power
• Collec%on of raw data into dom0
• Compute the non-halted cycles
percentage for each domain
• Use percentage to split evenly
socket power consumpBon
13
XeMPowerCLI
A1
1
B1
A2
2
B2
A1
1
B1
A3
3
B3
A2
2
1
A1
Core 0 Core N
Time
B2
…
…
…
context
switch
context
switch
context
switch
context
switch
XeMPowerDaemon
B2
B2
B1
B1
B3
B2
B2
B1
B1
B3
Xen Kernel Dom0
Hardware events per core,
energy per socket
…
O
A D
Introduced system overhead: MAX 1.18W (1.58%), AVG < 1W (< 1%)

14. XeMPUPiL
• Star%ng from the data coming from XeMPower,
we want to act on the system, for each domain
• Enforce a strong power cap, maximize performance of VMs
14
O
A D
SOFTWARE APPROACH
✓ efficiency
✖ timeliness
MODEL BASED
MONITORING
THREAD
MIGRATION
RESOURCE
MANAGMENT DVFS
RAPL
CPU
QUOTA
HARDWARE APPROACH
✖ efficiency
✓ timeliness

15. XeMPUPiL
• Star%ng from the data coming from XeMPower,
we want to act on the system, for each domain
• Enforce a strong power cap, maximize performance of VMs
15
O
A D
RESOURCE
MANAGMENT
CPU
QUOTA
HYBRID APPROACH
✓ efficiency
✓ timeliness
SOFTWARE APPROACH
✓ efficiency
✖ timeliness
HARDWARE APPROACH
✖ efficiency
✓ timeliness
MODEL BASED
MONITORING
THREAD
MIGRATION
DVFS
RAPL

16. XeMPUPiL architecture
16
Xen
Hypervisor
Hardware
DomainU
Workload
DomainU
Workload
Dom0
XLCLI
Act
Decide
Observe
XeMPower
Hardware
Events
Counters
buffers
Hypercall manager
RAPL
interface
PUPiL
O
A D

17. Experimental results
17
• Server setup
- 2.8-GHz quad-core Intel Xeon E5-1410
processor, no HT enabled (4 physical core)
- 32GB of RAM
- Xen hypervisor version 4.4
- paravirtualized instance of Ubuntu 14.04
as Dom0, pinned on the 4 cores and with
4GB of RAM
EP
[1]
IOzone
[3]
cachebench
[2]
BT
[3]
CPU-bound YES NO NO YES
IO-bound NO YES NO YES
memory-
bound
NO NO YES YES
[1] Nas parallel benchmarks. http://www.nas.nasa.gov/publications/npb. html#url. Accessed: 2016-06-01.
[2] Openbenchmarking.org. https://openbenchmarking.org/test/pts/ cachebench. Accessed: 2016-06-01.
[3] Iozone filesystem benchmark. http://www.iozone.org. Accessed: 2016- 06-01.
Benchmarking
The baseline is noRAPL configuration
O
A D
0
0.5
1.0
PUPiL 40
RAPL 40
Normalizedperformance
0
0.5
1.0
EP cachebench IOzone BT
0
0.5
1.0
PUPiL 30
RAPL 30
Normalizedperformance
0
0.5
1.0
EP cachebench IOzone BT
0
0.5
1.0
PUPiL 20
RAPL 20
Normalizedperformance
0
0.5
1.0
EP cachebench IOzone BT

18. Research projects (containeriza%on)
18
XeMPower DEEP-mon
XeMPUPiL HyPPO
O O
A D A D

19. DEEP-mon
19
• DEEP-mon is a HT-aware fine-grain power monitor for containers
- precise power a[ribuBon to containers
- instrumentaBon free, watch workloads from outside
- lightweight, with lisle overhead on the target workloads and systems
- scalable and distributed, to observe Kubernetes clusters
• Monitoring ingredients:
Container
execution
Resource
usage
Power
consumption
Context
switch
Performance
Counter (cycle)*
Intel
RAPL
* cycles has 99% correlation w.r.t. CPU power usage
Zhai, Yan, et al. "HaPPy: Hyperthread-aware Power Profiling Dynamically." USENIX Annual Technical Conference. 2014.
O
A D

20. DEEP-mon architecture
20
user-space
kernel-space
Intel RAPL
DEEP-mon
Power attribution
Docker and Kubernetes metrics
kernel tracing
PMC
context switch
Linux CFS
Monitoring back-end
O
A D

21. DEEP-mon architecture
O
A D
21
user-space
kernel-space
Intel RAPL
DEEP-mon
Power attribution
Docker and Kubernetes metrics
PMC
Monitoring back-end
200K evts/s
kernel tracing context switch
Linux CFS

22. Kernel level data acquisi%on (1)
• We cannot send each context switch to user-space
- too many events per second to process
- too much overhead
• Introduce in-kernel data aggrega%on:
22
eBPF and BCC:
build, inject and execute code
in a Kernel VM
trace context switch,
count PMCs on the fly
store data in
eBPF data structures
send one big event instead
of many small ones
DEEP-mon
kernel

23. Correlate power and performance
• At fixed %me intervals we collect the thread map
- extrac%on %me depends on # of context switch
• Then we extract power measurement from RAPL and we
account it for each thread:
23
eBPF output
Thread1
Thread2
Thread3
thread map
G benchmarks from NPB with
HT experiments pin two threads
ysical core
run on a Dell PowerEdge
n E5-2680 v2 (10 cores
and with Ubuntu Linux
st experiment shows that
cal cores mapped on the
umption is ' 1.15 with
g on that same physical
execution periods in which the thread was co-running on the
same physical core via HT, weighted by the HTr ratio and
divided by 2 to equally divide the overlapping cycles among
the two threads. In this context an execution period is defined
as the time between context switches on the physical core
where the thread is scheduled.
Starting from Equation (1), we can now attribute the power
measured by RAPL for our thread T1 following Equation (2),
where |K| is the cardinality of the set K of threads running in
the server in a given period of time and |S| is the cardinality
of the set S of sockets in the system.
PT 1(t) =
|S|
X
s=0
RAPLcore(t, s) ·
CyclesT W1 (t, s)
P|K|
k=0 CyclesT Wk
(t, s)
!
(2)
Starting from this result, the next sections will provide
details on how we implemented power attribution for each
thread and container running in the system.
B. Kernel level data acquisition
The power attribution model described in Section III-A
needs a precise measurement of the performance counter
Power of Thread 1
Sum among all sockets
RAPL measurement of the socket
Thread weight inside
the socket power consumption
• Finally we group each thread by container
DEEP-mon
kernel

24. Monitoring containers at scale
• Once power data is collected, we can send it to a back-end
on a regular basis
- further aggrega%on of metrics data
- Kubernetes cluster level view
• Backend exposes data for visualizaBon and autonomic power management
24
O
A D

25. Benchmarks
Cloud Benchmarks: Phoronix test suite pts/apache, pts/Nginx, pts/fio
HPC Benchmarks: NAS Parallel benchmarks EP, MG, CG
Experimental results
25
Network and syscall intensive benchmarks CPU and memory intensive tasks
Cloud benchmarks
app overhead
< 3.3%
HPC benchmarks
app overhead
< 4%
Cloud benchmarks
power overhead
1.74% avg
HPC benchmarks
power overhead
0.90% avg
O
A D
EvaluaBon goals
Monitoring should introduce minimum overhead
We evaluated DEEP-mon w.r.t. its overhead on applicaBons and the target system

26. Research projects (containeriza%on)
26
XeMPower DEEP-mon
XeMPUPiL HyPPO
O O
A D A D

27. HyPPO
• HyPPO is a Hybrid Performance-aware Power-capping
Orchestrator for Kubernetes environments
- leverage run-%me monitoring data coming from DEEP-mon
- guarantee SLAs for each container
- autonomic management of SLAs and power consump%on
- hybrid: HW power capping, SW resource management
27
O
A D
Energy proporBonality
The resources I use == the energy bill I pay
Performance first
Guaranteed user experience, saving power

28. Distributed ODA loop
28
Master
Node Node
API
API
Pod Pod
API
Pod Pod
MONITORING
AGENT
MONITORING
AGENT
HyPPO
Backend
ACTUATOR
AGENT
ACTUATOR
AGENT
CONTROLLER

29. HyPPO controller
29
O
A D
mples of metrics and Kubernetes status from each monitoring agent in the GRPC collector.
base for later use. Monitoring samples are unpacked inside the Metrics workers and aggregated
ored in an InfluxDB database, which is queried by the monitoring frontend to show real-time
loop components.
hen access
the REST
ueries the
the latter
gramming
the others,
ge, Power
time and
metrics by
nt of our
composed
ch data to
utcome to
node of the cluster, powern,c is the power consumed by the
c-th container running on the n-th node.
powern = Pidle +
CX
c=0
(powern,c + i(c)) (1)
Equation (1) defines the power for the n-th node as the sum
of the idle power Pidle and the sums of all the powers
consumed by the c-th container running on the n-th node,
plus a contribute i(c). The contribution can be positive or
negative and is expressed in Equation (2), where cpu requestc
represents the CPU request expressed for the c-th container,
cpu usagec represent the actual CPU consumption for the c-
th container and P represents a proportional factor that can
be defined in controller configurations. Each container CPU
usage data point passes through the controller represented by
Equations (1) and (2). In this way, it contributes positively
queries the
e the latter
ogramming
g the others,
sage, Power
n time and
metrics by
ent of our
composed
uch data to
outcome to
ementing a
tuators.
ST endpoint
ashion (the
controller).
nformation:
ners power
first set of
dictionary
powern = Pidle +
CX
c=0
(powern,c + i(c)) (1)
Equation (1) defines the power for the n-th node as the sum
of the idle power Pidle and the sums of all the powers
consumed by the c-th container running on the n-th node,
plus a contribute i(c). The contribution can be positive or
negative and is expressed in Equation (2), where cpu requestc
represents the CPU request expressed for the c-th container,
cpu usagec represent the actual CPU consumption for the c-
th container and P represents a proportional factor that can
be defined in controller configurations. Each container CPU
usage data point passes through the controller represented by
Equations (1) and (2). In this way, it contributes positively
or negatively to the total power consumption of the node on
which the container is actually running.
i(c) =
(
(cpu usagec cpu requestc) ⇤ P if 9cpu usagec
0 if 6 9cpu usagec
(2)
The P parameter was chosen after several experiments and
represent the pace at which the controller tries to fill the
opportunity gap. It is defined in the configuration file of the
For each host we compute the power cap to be enforced depending on running containers
CONTROLLER
power of node n
power of idle system power of the container
power adjusting
under/over utilisation condition proportional factor (10W)
Power is then adjusted depending on how the CPU is used

30. Experimental setup
30
O
A D
Testbed
Kubernetes cluster composed by 2 homogenous nodes
Node specs: Dell PowerEdge r720xd, 2x Intel Xeon E5-2680 Ivy Bridge (20 HT), 2.80GHz, 380GB of RAM
Benchmarks: Phoronix Test Suite
apache-cpu CPU Request
CPU%
0
200
400
Execution Time [s]
0 20 40 60 80 100 120 140 160
Apache CPU Opportunity Gap
apache-cpu CPU Request
CPU%
0
200
400
Execution Time [s]
0 20 40 60 80 100 120 140 160
Apache CPU Opportunity Gap
Goal
HyPPO should be able to guarantee performance
and at the same %me try to reduce power usage

31. Experimental results
31
O
A D
apache-cpu apache-cpu-ctrl CPU Request
CPU%
0
200
400
600
Execution Time [s]
0 20 40 60 80 100 120 140 160
Apache CPU usage
apache-pw apache-pw-ctrl
Power[mW]
0
20000
40000
60000
Execution Time [s]
0 10 20 30 40 50 60 70 80
Apache Power consumed

32. Experimental results
32
O
A D
apache-cpu apache-cpu-ctrl CPU Request
CPU%
0
200
400
600
Execution Time [s]
0 20 40 60 80 100 120 140 160
Apache CPU usage
apache-pw apache-pw-ctrl
Power[mW]
0
20000
40000
60000
Execution Time [s]
0 10 20 30 40 50 60 70 80
Apache Power consumed
Preliminary tests conducted on PTS Apache, Nginx, Dbench, Fio, Postmark, IOzone
showed a power saving ranging from 5% to 45%

33. Conclusion
• We saw the main challenges behind monitoring and controlling power
- data gathering and behaviour observa%on
- power awareness and power op%miza%on
• Power@NECST
- general autonomic methodology
- accurate fine grain monitoring for Docker, Kubernetes and Xen
- power capping methodology designed according to system goals
33

34. Future work
34
Latency awareness:
monitoring and control
Seamless VM and container
integraBon

35. Thanks for your asen%on
35
NECST Group Conference 2018 @ MSR
05/31/2018
Rolando Brondolin
<rolando.brondolin@polimi.it>
Run-%me power management in cloud
and containerized environments