Many software projects are configurable through compile-time options (e.g. using ./configure) and also through run-time options (e.g. command-line parameters, fed to the software at the execution). Several works have studied how to predict the effect of run-time options on performance. However it is unclear how these prediction models generalize when the software is built with different values of compile-time options. For instance, is the best run-time configuration always the best w.r.t. the chosen compilation options? In this paper, we investigate the effect of compile-time options on the performance distributions of 5 software systems. We prove there can exist an interplay between the compile-time and the execution levels, by exhibiting a case where different compilation options significantly alter the performance distributions of a software system.

1. The Interplay of Compile-time
& Run-time Options for Performance Prediction
Luc Lesoil, Mathieu Acher, Xhevahire Tërnava, Arnaud Blouin and Jean-Marc Jézéquel
The Systems and Software Product Line Conference 2021
06-09 september, Leicester, UK

2. Run-time configurations matter
Threads = ’auto-detect’
Tile size = 64 pixels
Progressive refine = True
Thread(s) = 1
Tile size = 12 pixels
Progressive refine = False
Rendering a 3D scene
1
2
~ 3 seconds
~ 3 minutes
Performances
Run-time
configurations
Input data
What about
compile-time
configurations?
2

3. git clone https://github.com/mirror/x264
./x264 --me tesa
Com
Dow d
Run
Use
./configure [--enable-asm] …
make
./configure --disable-asm …
make
./x264 --me umh ./x264 --me tesa ./x264 --me umh
10.6 seconds 3.4 seconds 81.5 seconds 25.9 seconds
What about compile-time options? An example with
A B
1 1
2 2
3

4. This paper investigates how compile-time options can affect software
performances and how compile-time options interact with run-time options
4

5. Motivations
Outperform the run-time performance of the default compile-time configuration
- Linux packages (fixed values)
- Manual compilation (default values)
5
Understand the interplay between the compile-time and the run-time levels
Are performance models only valid for default compile-time configuration ?

6. Research questions
RQ2 - How to tune software performances at the compile-time level?
RQ1 - Do compile-time options change the performance distributions of configurable systems?
6
RQ1.1. Do the run-time performances of configurable systems vary with compile-time options?
RQ1.2. How many performance can we gain/lose when changing the default compile-time configuration?
RQ2.1. Do compile-time options interact with the run-time options?
RQ2.2. How to use these interactions to find a set of good compile-time options and tune the configurable system?

7. Gathering data to answer research questions
7
1
2 3
4
5
6

8. RQ1.1. Do the run-time performances of configurable systems vary with compile-time options?
RQ1 - Do compile-time options change the performance distributions?
8
Fixed run-time config.
Different compile-time config.
Results:
- Size => stable
- xz => stable
- x264 => vary with run-time
- nodeJS => vary

9. RQ1.2. How many performance can we gain/lose when changing the default compile-time configuration?
RQ1 - Do compile-time options change the performance distributions?
9
Performance Ratio (r, c) =
performance of the run-time configuration r for the compile-time configuration c
performance of the run-time configuration r for the default compile-time configuration
Results:
- Size => no gain
- xz and poppler => negligible
- Good default performance
- Can vary with input data

10. RQ2.1. Do compile-time options interact with the run-time options?
10
RQ2 - How to tune software performances at the compile-time level?
Spearman correlations (3a)
Random Forest importances (3b)
Results (nodeJS):
- Compile-time options alter perf. rankings
-> Interplay
- Both compile- and run-time are useful
-> Interplay

11. RQ2.2. How to use these interactions to find a set of good compile-time options and tune the configurable system?
11
RQ2 - How to tune software performances at the compile-time level?
Predict the best compile-time configuration
Vary the training size : 1%, 5% and 10% of measurements
Depict the performance ratio per input and per training size in Table 3
Results (nodeJS):
Do not need too much data - 5% is enough to get close to the oracle
Up to 50 % improvement of performance

12. 12
Conclusion
Compile-time configurations matter
Future work : Automate the tuning of software systems at compile-time ?

13. THANKS!
Any questions?
13
Feel free to contact us if you want to discuss this topic.