This document summarizes a study exploring the performance and energy consumption differences between recent Intel processors. The study found that codes ran on average 2.1x faster with 1.1x less energy on a Broadwell system compared to a Sandy Bridge system, and 3.7x faster with 1.3x less energy on a Skylake system compared to Sandy Bridge. These results were obtained by running benchmarks like STREAM, NPB, Rodinia, and measuring performance and power consumption using the RAPL-logger tool. Future work could extend this study to more benchmarks and processor architectures.

1. Exploring performance and energy
consumption differences between
recent Intel processors
Unai Lopez-Novoa
Data Innovation Research Institute
Cardiff University

2. Challenge in High Performance Computing:
Increasing power consumption of modern supercomputers
Top500 list (June 2019):
2
Motivation
Rank Name Performance Power cons.
# 1 Summit (USA) 148.6 PFlops 10.1 MW
# 2 Sierra (USA) 94.6 PFlops 7.4 MW
# 3 Sunway TaihuLight (China) 93.0 PFlops 15.3 MW
Source: www.top500.org

3. Some agencies are setting energy consumption objectives in HPC,
e.g. US Dpt. of Energy by year 20221:
◦ 1 ExaFlops at 20-40 MW
◦ 25-50 GigaFlops/W
Proposal: analyse the behaviour of common parallel codes in
recent compute devices
◦ This work is focused on recent Intel CPUs
Goal: obtain a yardstick to guide performance or energy-oriented
optimisation
3
Motivation
1: Z.-N. Chen, J. Dongarra, Z.-W. Xu, “Post-exascale supercomputing: research opportunities
abound” Frontiers of Information Technology & Electronic Engineering, 19, 10, Oct 2018.

4. Feature introduced by Intel w/ Sandy Bridge (~2011) to monitor
energy consumption and cap power consumption.
Domains:
◦ Package: whole socket
◦ Power plane 0: cores within a package
◦ Power plane 1: uncore elements within a package
◦ DRAM: memory attached to socket
In this work: Package & DRAM
4
Intel RAPL

5. Issue: many ways to use RAPL, but they require either:
a) Modifying the source code of the target app (PAPI library)
b) Administrative privileges (perf command)
RAPL-logger: a tool to monitor the power consumption of an app
in an Intel-based system from user space
Generates a report of the energy (J) and power (W) consumption
GitHub: https://github.com/ulopeznovoa/RAPL-logger
5
RAPL-logger
./rapl_logger <your-app> <params-of-your-app>

6. 3 hardware systems:
5 benchmarks:
◦ STREAM
◦ NPB: Conjugate Gradient (CG)
◦ NPB: Lower-Upper decomposition (LU)
Everything was compiled with GCC 7.3 and -O3 flag
6
Experimental setup
ID CPU Family # Cores Clock DRAM
SB 2 x Xeon E5-2620 Sandy Bridge 6 2.00 32 GB – DDR3
BW 2 x Xeon E5-2695 v4 Broadwell 18 2.10 126 GB – DDR4
SL 2 x Xeon Gold 6148 Skylake 20 2.40 377 GB – DDR4
◦ Rodinia: LavaMD
◦ Rodinia: Streamcluster

7. 7
Results: performance comparison

8. 8
Results: power consumption

9. SB BW SL
Package 0 9.7 19.2 51.6
Package 1 9.4 16.4 52.9
DRAM 0 2.9 1.2 10.1
DRAM 1 4.0 1.2 9.8
9
Results
SB BW SL
GFLOP/s 67.7 753.7 2000.3
DRAM GB/s 47.6 95.3 152.2
Idle power consumption (Watts)
Measured running sleep for 1 minute
Measured:
- GFLOP/s: FIRESTARTER benchmark
- DRAM GB/s: STREAM Triad
Raw computing power

10. On average:
◦ Codes run 2.1x faster with 1.1x less energy on BW than SB
◦ Codes run 3.7x faster with 1.3x less energy on SL than SB
RAPL-logger was used to collect measurements
◦ https://github.com/ulopeznovoa/RAPL-logger
Future work:
◦ Extend this study w/ more benchmarks and CPUs (ARM, IBM, …)
◦ Use this study to develop energy-aware performance models
10
Conclusions

11. Exploring performance and energy
consumption differences between
recent Intel processors
Unai Lopez-Novoa
LopezU@cardiff.ac.uk