This document summarizes a PhD student's presentation on their research into taming deep software variability. The key points are: 1. The PhD aims to identify new external factors that influence software performance, measure their effects, and reuse performance models across workloads. 2. Current work is analyzing input sensitivity in video compression software, and grouping inputs by similar performance profiles. 3. Ideas discussed include specializing software for specific workloads using feature importance analysis to remove unnecessary code.

1. Taming Deep Software Variability
Luc LESOIL
PhD Student - 1st
year
Mathieu ACHER
Supervisors:
Slimf
ast Kick-Off
15/10/2020
Arnaud BLOUIN
Jean-marc JÉZÉQUEL

2. Overview

3. Overview
1. PhD subject

4. Overview
1. PhD subject
2. Current work

5. Overview
3. Ideas & Discussion
1. PhD subject
2. Current work

6. 1. PhD subject
Deep software variability

7. 1. PhD subject
Deep software variability

8. 1. PhD subject
Deep software variability
?

9. Why “deep” ?

10. Why “deep” ?
users
contributors
Variability Factors
actions

11. Why “deep” ?
users Hardware
buy
contributors
Variability Factors
actions

12. Why “deep” ?
users Hardware Operating
System
buy install
contributors
Variability Factors
actions

13. Why “deep” ?
users Hardware Operating
System
buy install
package
“apt install package” download
contributors
Variability Factors
actions

14. Why “deep” ?
users Hardware Operating
System
buy install
package
“apt install package”
package
use
download
contributors
Variability Factors
actions

15. Why “deep” ?
users Hardware Operating
System
buy install
package
“apt install package”
package
use
download
contributors
Variability Factors
actions
apply on
Input
Data

16. Why “deep” ?
developpers
users Hardware Operating
System
buy install
package
“apt install package”
package
use
download
contributors
Variability Factors
actions
apply on
Input
Data

17. Why “deep” ?
developpers
users Hardware Operating
System
code
implement
buy install
maintain
package
“apt install package”
package
use
download
contributors
Variability Factors
actions
apply on
Input
Data

18. Why “deep” ?
developpers
users Hardware Operating
System
code
implement
buy install
maintain require
Dependencies
package
“apt install package”
package
use
download
contributors
Variability Factors
actions
apply on
Input
Data

19. Why “deep” ?
developpers
users Hardware Operating
System
code
implement
buy install
maintain
Compile
require
Dependencies
package
“apt install package”
package
use
download
contributors
Variability Factors
actions
apply on
Input
Data

20. Marginal eff
ects?
Software

21. Marginal eff
ects?
Compilation
Software
?

22. Marginal eff
ects?
Hardware
Compilation
Software
?
?

23. Marginal eff
ects?
Hardware Operating
System
Compilation
Software
?
?
?

24. Marginal eff
ects?
Hardware Operating
System
Compilation Input Data
Software
?
?
?
?

25. Marginal eff
ects?
Hardware Operating
System
Compilation Input Data
Software
?
?
?
?
Interaction
with configuration options ?

26. Research questions

27. Research questions
Identify new variability f
actors

28. Research questions
Identify new variability f
actors
Which variability f
actors
are the most influential?

29. Research questions
Identify new variability f
actors
Which variability f
actors
are the most influential?
Can we reuse perf
ormance models
trained on another workload?

30. On a concrete example ?

31. On a concrete example ?
Software
0.152.2854
--no-cabac –-ref 1
–-no-mbtree

32. On a concrete example ?
Software Input Data
0.152.2854
--no-cabac –-ref 1
–-no-mbtree
High complexity
Action scene

33. On a concrete example ?
Software Input Data
0.152.2854
7.5.0
--no-cabac –-ref 1
–-no-mbtree
–-disable-asm
-On
High complexity
Action scene
Compiler

34. On a concrete example ?
Software Input Data
0.152.2854
7.5.0
--no-cabac –-ref 1
–-no-mbtree
–-disable-asm
-On
High complexity
Action scene
Compiler Operating
System
18.04
LTS
GENERIC_CMOS_UPDATE
BH1750

35. On a concrete example ?
Software Input Data
0.152.2854
7.5.0
--no-cabac –-ref 1
–-no-mbtree
–-disable-asm
-On
High complexity
Action scene
Compiler
Hardware
Operating
System
18.04
LTS
GENERIC_CMOS_UPDATE
BH1750
Dell latitude 7400

36. On a concrete example ?
Software Input Data
0.152.2854
7.5.0
--no-cabac –-ref 1
–-no-mbtree
–-disable-asm
-On
High complexity
Action scene
Compiler
Hardware
Operating
System
18.04
LTS
What if ?
GCC 7.5.0 GCC 7.3.0
→ GCC 7.3.0
Ubuntu Fedora
→ GCC 7.3.0
We enable mbtree
GENERIC_CMOS_UPDATE
BH1750
Dell latitude 7400

37. On a concrete example ?
Software Input Data
0.152.2854
7.5.0
--no-cabac –-ref 1
–-no-mbtree
–-disable-asm
-On
High complexity
Action scene
Compiler
Hardware
Operating
System
18.04
LTS
What if ?
GCC 7.5.0 GCC 7.3.0
→ GCC 7.3.0
Ubuntu Fedora
→ GCC 7.3.0
We enable mbtree
GENERIC_CMOS_UPDATE
BH1750
Dell latitude 7400
Here we go !

38. 1. PhD subject - T
o sum up ?

39. 1. PhD subject - T
o sum up ?
Identify new external variability f
actors

40. 1. PhD subject - T
o sum up ?
Identify new external variability f
actors
Measure their eff
ects on software

41. 1. PhD subject - T
o sum up ?
Identify new external variability f
actors
Measure their eff
ects on software
Reuse perf
ormance models across workloads

42. 2. Current work

43. 2. Current work
Input sensitivity :
how input data aff
ect software perf
ormances

44. 2. Current work
Input sensitivity :
how input data aff
ect software perf
ormances
Data-oriented software

45. 2. Current work
Input sensitivity :
how input data aff
ect software perf
ormances
Data-oriented software

46. 2. Current work
Input sensitivity :
how input data aff
ect software perf
ormances
Data-oriented software

47. 2. Current work
Input sensitivity :
how input data aff
ect software perf
ormances
Data-oriented software

48. 2. Current work
Input sensitivity :
how input data aff
ect software perf
ormances
Data-oriented software

49. Motivation : why video compression?

50. Motivation : why video compression?
x264 commits : devs are aware of input sensitivity

51. Motivation : why video compression?
x264 commits : devs are aware of input sensitivity
x264 f
eatures : devs created a f
eature f
or special input videos

52. Idea
Optimize x264 compression f
or special video content

53. Idea
Optimize x264 compression f
or special video content
Wisely choose
configuration options

54. Idea
Optimize x264 compression f
or special video content
Bitrate
Wisely choose
configuration options

55. Idea
Optimize x264 compression f
or special video content
Bitrate
Wisely choose
configuration options
How to
classify video?

56. Dataset

57. Dataset
[1]
Wang, Y., Inguva, S., & Adsumilli, B. (2019 1, September). Youtube ugc dataset for video compression research.
In 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP) (pp. 1-5). IEEE.
Youtube UGC Dataset [1]

58. Dataset
[1]
Wang, Y., Inguva, S., & Adsumilli, B. (2019 1, September). Youtube ugc dataset for video compression research.
In 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP) (pp. 1-5). IEEE.
1300+ short videos
Youtube UGC Dataset [1]

59. Dataset
[1]
Wang, Y., Inguva, S., & Adsumilli, B. (2019 1, September). Youtube ugc dataset for video compression research.
In 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP) (pp. 1-5). IEEE.
1300+ short videos
Diff
erent resolutions (360p 2160p)
→ GCC 7.3.0
Youtube UGC Dataset [1]

60. Dataset
[1]
Wang, Y., Inguva, S., & Adsumilli, B. (2019 1, September). Youtube ugc dataset for video compression research.
In 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP) (pp. 1-5). IEEE.
1300+ short videos
Diff
erent resolutions (360p 2160p)
→ GCC 7.3.0
Diff
erent categories
Youtube UGC Dataset [1]

61. Dataset
[1]
Wang, Y., Inguva, S., & Adsumilli, B. (2019 1, September). Youtube ugc dataset for video compression research.
In 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP) (pp. 1-5). IEEE.
1300+ short videos
Diff
erent resolutions (360p 2160p)
→ GCC 7.3.0
Diff
erent categories
Video properties
Spatial/Temporal/Chunk complexity
Quality score
Youtube UGC Dataset [1]

62. Dataset
[1]
Wang, Y., Inguva, S., & Adsumilli, B. (2019 1, September). Youtube ugc dataset for video compression research.
In 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP) (pp. 1-5). IEEE.
https://media.withyoutube.com/
1300+ short videos
Diff
erent resolutions (360p 2160p)
→ GCC 7.3.0
Diff
erent categories
Video properties
Spatial/Temporal/Chunk complexity
Quality score
Youtube UGC Dataset [1]

63. Configurations

64. Configurations
Compromise bitrate-quality

65. Configurations
Compromise bitrate-quality
Preset configurations

66. Configurations
Compromise bitrate-quality
Preset configurations
200
configs

67. Configurations
Compromise bitrate-quality
Preset configurations
200
configs
Better
Better

68. Results (1)

69. Results (1)
Perf
ormances change with input videos.
Perf
ormance correlations can be low or negative between inputs

70. Results (1)
Perf
ormances change with input videos.
Feature positive
→ GCC 7.3.0 & negative eff
ects depending on inputs
Perf
ormance correlations can be low or negative between inputs

71. Results (1)
Perf
ormances change with input videos.
Feature positive
→ GCC 7.3.0 & negative eff
ects depending on inputs
Perf
ormance correlations can be low or negative between inputs
Group together videos having the same “encoding profile”

72. Results (2)

73. Results (2)
Include input properties in the model

74. Results (2)
Include input properties in the model
Specialize x264 f
or a new video
without measuring any new configuration

75. Results (2)
Include input properties in the model
Specialize x264 f
or a new video
without measuring any new configuration
Reduce the variability of the inputs

76. 2. Current work - T
o sum up ?

77. 2. Current work - T
o sum up ?
Input sensitivity matters

78. 2. Current work - T
o sum up ?
Input sensitivity matters
Group inputs by perf
ormance
& find discriminant (cheap) properties

79. 2. Current work - T
o sum up ?
Input sensitivity matters
Group inputs by perf
ormance
& find discriminant (cheap) properties
Include input properties in the model

80. 3. Ideas & Discussion

81. 3. Ideas & Discussion

82. 3. Ideas & Discussion
Specialize the code f
or a workload

83. 3. Ideas & Discussion
Specialize the code f
or a workload
Use f
eature importances to remove useless parts of code

84. 3. Ideas & Discussion
Specialize the code f
or a workload
Use f
eature importances to remove useless parts of code
branch selection choose the best branch
→ GCC 7.3.0

85. Questions?
Thanks f
or your attention !