Icecc is a distributed compiler that allows compilation jobs to be distributed across multiple computers to reduce build times. It works by having one computer act as a scheduler that distributes preprocessing, compilation, and assembly jobs to other connected computers. The key requirements are that all computers must have the same toolchain installed and be on a fast network. Setting up Icecc involves configuring one computer as the scheduler, installing Icecc on all computers, and ensuring all toolchains are packaged and accessible in a shared location. Icecc has been shown to reduce Linux kernel compilation time by 75% by utilizing 3 additional computers during the build.

1. Distributed Compiler
Icecc
Harder Better Faster Stronger
SZ LIN
szlin@cs.nctu.edu.tw
1
Date: 2013/5/8

2. 2
SZLIN (林上智)
 Debian Developer
 Industrial Grade Linux Distribution R&D
 https://szlin.me
About Me

3. 3

4. 4
Compiler takes lots of time
All we can do is wait and pray

5. 5

6. 6
RAM disk
(or flash based stroage)
Precompiled
headers
ccache
Parallelize make

7. 7

8. 8
High-end computer
Multi-core & Large RAM & $$$

9. 9
Low-end computers
Build a Distributed system

10. 10
Your PC
SZLIN’s PC
RJ’s PC
$4’s PC

11. 11
How it works
fundamental concept in C compilation

12. 12

13. 13
Steps of C Compilation

14. 14
1. Preprocessor must always run locally
• It needs to access various header files on the local
machine which may not be present, or may not be the
same, on the volunteer. The linker similarly needs to
examine libraries and object files, and so must run locally.
2. The linker similarly needs to examine libraries
and object files, and so must run locally.
3. Preprocessed source => compiler/assembler=>
object file
4. It runs the compiler/assembler remotely.

15. 15
Make –j option
Specifies the number of jobs to run
simultaneously

16. 16
man make.conf

17. 17
紙上得來終覺淺
絕知此事要躬行
陸游（1125-1210）

18. 18
IceCream based on Distcc’s source code
•Improve performance
•Dynamic scheduler of the compilation jobs

19. 19
Requirement
Same version (icecream)
• icecc-scheduler scheduler 1.0.90
• ICECC 1.0.90
Fast network connection
• > 100 Mbps
One scheduler
• neither less nor more
Toolchain Installed
• All toolchain should be the same

20. 20
git clone https://github.com/icecc/icecream.git
apt-get install automake build-essential libtool
./autogen.sh
./configure
make
make install
Install Icecc in Debian

21. 21
1. Set only one computer as a dispatcher role
vim /etc/default/icecc
2. Other device setting
vim /etc/default/icecc

22. 22
3. Modify configuration parameters
vi /etc/icecc/icecc.conf

23. 23
4. More verbose message for debugging
export ICECC_DEBUG=“debug”
5. Create soft link under icecc binary directory
cd /usr/local/bin/
ln -sf icecc arm-mv5sft-linux-gnueabi-gcc
ln -sf icecc arm-mv5sft-linux-gnueabi-g++
ln -sf icecc gcc
ln -sf icecc g++
ln -sf icecc arm-linux-gnueabihf-gcc
ln -sf icecc arm-linux-gnueabihf-g++

24. 24
6. Export path
declare -x
ICECC_VERSION="/dev/shm/toolchain/i386.tar.gz,/dev/shm/too
lchain/arm-mv5sft-linux-gnueabi.tar.gz=arm-mv5sft-
linux,/dev/shm/toolchain/arm-linux.tar.gz=arm-linux-
gcc,/dev/shm/toolchain/arm-linux-gnueabihf.tar.gz=arm-linux-
gnueabihf"
7. Add new user in /etc/passwd
vi /etc/passwd
 icecc:x:113:121::/var/cache/icecc:/bin/false

25. 25
7. Create a tarball file for each toolchain that you
want to use with icecream
/usr/local/libexec/icecc/icecc-create-env --gcc /usr/local/arm-mv5sft-linux-gnueabi/bin/arm-mv5sft-
linux-gnueabi-gcc /usr/local/arm-mv5sft-linux-gnueabi/bin/arm-mv5sft-linux-gnueabi-g++
/usr/local/libexec/icecc/icecc-create-env --gcc /usr/bin/gcc /usr/bin/g++
/usr/local/libexec/icecc/icecc-create-env --gcc /usr/local/arm-linux-4.4.2/bin/arm-linux-gcc
/usr/local/libexec/icecc/icecc-create-env --gcc /usr/local/arm-linux-gnueabihf-4.7-
20130415/bin/arm-linux-gnueabihf-gcc /usr/local/arm-linux-gnueabihf-4.7-
20130415/bin/arm-linux-gnueabihf-g++
8. Modify generated tar.gz file, and put it under
specific directory
mv xxxx.tar.gz i386.tar.gz
mv i386.tar.gz /dev/shm/toolchain/i386.tar.gz

26. 26
9. Set toolchain path
(set icecc path before toolchain path)
declare -x
PATH="/usr/local/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/
usr/bin:/sbin:/bin:/usr/local/arm-mv5sft-linux-
gnueabi/bin:/usr/local/arm-linux-4.4.2//bin:/usr/local/arm-
linux-gnueabihf-4.7-20130415/bin:."
Restart ICECC daemon
/etc/init.d/icecc restart

27. 27

28. 28
One Click Solution

29. 29

30. 30

31. 31

32. 32
Compatible
Add “m32 m64” in cflags is better

33. 33
Caution

34. 34
Only allow Trusted Network
All machines &users must be trusted

35. 35
0
1,000
2,000
3,000
4,000
5,000
Single PC Single PC + 3* P4 PC
Compile Linux kernel 3.8.4
Compile time
Reduced by
75%

36. 36
Thank you

37. 37
人才難找
如果讓每個人做事少花 75% 的時間
就像免費為公司多雇了 75% 的人