Make Debian compiler agnostic :        A rebuild with ClangSylvestre Ledru – sylvestre@debian.org
Disclamer !April, 04 2013   Make Debian compiler agnostic                       Sylvestre Ledru
Current status :     All C, C++, Objective-C sources are being built     with gcc for all supported Debian arches and     ...
gcc is THE FLOSS compiler for the last 25 years   Used for (pretty much) everywhere or anything            Still growing a...
Why a new compiler in Debian ?April, 04 2013            Make Debian compiler agnostic                                Sylve...
Other compilers can find programming errors that                gcc could not findApril, 04 2013    Make Debian compiler a...
Code built by many compilers is more likely to    be more strictly correct and more portable then                code only...
Some compilers can have advantages on some           archs (ex : clang on ARM)April, 04 2013    Make Debian compiler agnos...
Provide some interesting side projects (lldb, scan-           build, ASAN, libc++, etc)April, 04 2013     Make Debian comp...
As we were able to do with decoupling Linuxfrom Debian with kFreeBSD and the HURD, were       aiming to decouple gcc in De...
LLVM/ClangApril, 04 2013   Make Debian compiler agnostic                       Sylvestre Ledru
Started as an academic project       Versatile platform for compilation and virtual                         machine       ...
Developed initially as a base for a next major                      GCC release                 Source:                 ht...
Sponsored by Apple since 2005 to replace gcc                  (GPL vs BSD)          Has now a strong and diverse community...
April, 04 2013   Make Debian compiler agnostic                       Sylvestre Ledru
Clang                 C, C++ & Objective-C compiler.                          (no Fortran)                        Based on...
Some advantages :       More recent base code (ie less legacy code)   Strong interest of hardware manufactors (ARM,       ...
Example            Full build of Scilab (doc, essential tests)                         ~24 minutes gcc                    ...
Some advantages (bis)                  More intelligent detections– foo.c –                                     $ gcc -Wal...
Side effect          => Brings (friendly) competition in the free                        compiler world.April, 04 2013    ...
http://gcc.gnu.org/wiki/ClangDiagnosticsComparisonApril, 04 2013    Make Debian compiler agnostic                        S...
http://clang.llvm.org/comparison.html#gccApril, 04 2013           Make Debian compiler agnostic                           ...
Rebuild of Debian using ClangApril, 04 2013           Make Debian compiler agnostic                               Sylvestr...
Crappy method :VERSION=4.7cd /usr/binrm g++-$VERSION gcc-$VERSION cpp-$VERSIONln -s clang++ g++-$VERSIONln -s clang gcc-$V...
Testing the rebuild of the package under amd64.   NOT the performances (build time or execution)         nor the execution...
Full results published:                 http://clang.debian.net/      Done on the cloud-qa - EC2 (Amazon cloud)           ...
Rebuild with clang 3.0                    February 28, 2012         15658 packages built : 1381 (8.8 %) failed.April, 04 2...
Rebuild with clang 3.1                    June 23, 2012       17710 packages built : 2137 (12.1 %) failed.April, 04 2013  ...
Rebuild with clang 3.2                   January 28, 2013        18264 packages built : 2204 (12.1 %) failed.April, 04 201...
Why these differences between 3.0 vs 3.1/3.2?April, 04 2013      Make Debian compiler agnostic                          Sy...
Some information about -Wall & -Werror :                         -Wall enables many warnings                   -Werror tra...
-Werror & unused args                         40 occurrencesClang detects unused argument.clang --param ssp-buffer-size=4 ...
Security check introduced in clang 3.1                           9 occurences                                             ...
Some of the most common errorsApril, 04 2013           Make Debian compiler agnostic                               Sylvest...
Unsupported options                            50 occurrences      $ gcc -O9 foo.c && echo $?      0      $ clang -O9 foo....
Different default behavior                     133 occurrences– noreturn.c –                           $ gcc -c noreturn.c...
Different default behavior (bis)                        15 occurrences                                            $ gcc -c...
Different understanding of the C++ standard– mailboxField.cpp –                              $ g++ -c mailboxField.cpp && ...
Different set of warnings with -Wall                            Plenty of occurences echo $?                              ...
gcc extensions which wont be supported                   25 occurences– foo.cpp –                                $ g++ -c ...
gcc accepts stuff which should not                              34 occurences                                             ...
Last rebuild proved that clang is now ready                 Remaining problems are upstream        Some work started to re...
StatusApril, 04 2013   Make Debian compiler agnostic                       Sylvestre Ledru
Packages LLVM-toolchain containing           LLVM + Clang + compiler-rt + polly + lldb           version 3.2 & 3.3 waiting...
Build results integrated on the Debian                       Package Tracking System                             By Paul W...
Version of pbuilder/cowbuilder with a hook to                  change the compiler                   By Junich Uekawa     ...
Interest by upstreams (Google, Intel & Apple)    Publication of Debian (wheezy, unstable) and     Ubuntu (Quantal, Precise...
FutureApril, 04 2013   Make Debian compiler agnostic                       Sylvestre Ledru
Discussions to move the build as a new port             on http://www.debian-ports.org/ Various potential issues (not real...
Hack the Debian tools to :            Force dpkg to export CC=/usr/bin/cc,                 ●          CXX=/usr/bin/c++ and...
Update the debian policy to include something                       like : Hardcoded usage of CC or CXX (for example, CC=g...
Add a lintian warning like       W: yourpackage: Hardcoded call to gcc/g++.          Use /usr/bin/cc or /usr/bin/c++ inste...
Potential the rebuild of Debian with :●clang+plugin. Ex : polly : cache-localityoptimisation auto-parallelism and vectoriz...
Any questions ? Remarks ?                        Troll ? (+1)April, 04 2013          Make Debian compiler agnostic        ...
Upcoming SlideShare
Loading in...5
×

Distro Recipes 2013 : Make Debian and compiler agnostic

2,862

Published on

https://distro-recipes.org

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,862
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Distro Recipes 2013 : Make Debian and compiler agnostic

  1. 1. Make Debian compiler agnostic : A rebuild with ClangSylvestre Ledru – sylvestre@debian.org
  2. 2. Disclamer !April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  3. 3. Current status : All C, C++, Objective-C sources are being built with gcc for all supported Debian arches and Kernel.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  4. 4. gcc is THE FLOSS compiler for the last 25 years Used for (pretty much) everywhere or anything Still growing and maintained (cf gcc 4.8)April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  5. 5. Why a new compiler in Debian ?April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  6. 6. Other compilers can find programming errors that gcc could not findApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  7. 7. Code built by many compilers is more likely to be more strictly correct and more portable then code only built with gccApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  8. 8. Some compilers can have advantages on some archs (ex : clang on ARM)April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  9. 9. Provide some interesting side projects (lldb, scan- build, ASAN, libc++, etc)April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  10. 10. As we were able to do with decoupling Linuxfrom Debian with kFreeBSD and the HURD, were aiming to decouple gcc in Debian. Open the door to other compilers or analyzersApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  11. 11. LLVM/ClangApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  12. 12. Started as an academic project Versatile platform for compilation and virtual machine Designed originally for the investigation of dynamic compilation techniques for static and dynamic languagesApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  13. 13. Developed initially as a base for a next major GCC release Source: http://gcc.gnu.org/ml/gcc/2005-11/msg00888.htmlApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  14. 14. Sponsored by Apple since 2005 to replace gcc (GPL vs BSD) Has now a strong and diverse community (academics, individuals and corporates) Many universities/research centers are basing their research on LLVMApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  15. 15. April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  16. 16. Clang C, C++ & Objective-C compiler. (no Fortran) Based on LLVM Default compiler for Mac OS X (Xcode)/iOS [1] and FreeBSD [2] Sources: [1] https://developer.apple.com/technologies/tools/ [2] http://lists.freebsd.org/pipermail/freebsd-stable/2012-May/067486.htmlApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  17. 17. Some advantages : More recent base code (ie less legacy code) Strong interest of hardware manufactors (ARM, MIPS, Intel, Nvidia, etc) Supposed to be faster to build code than gcc Accept the same arguments as gccApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  18. 18. Example Full build of Scilab (doc, essential tests) ~24 minutes gcc ~22 minutes clangApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  19. 19. Some advantages (bis) More intelligent detections– foo.c – $ gcc -Wall -Werror foo.c ; echo $?int main() { 0 unsigned int i = 0; $ clang -Werror foo.c foo.c:3:17: error: comparison of return i < 0; unsigned expression < 0 is always} false [-Werror,-Wtautological-compare] return i < 0; ~^~ 1 error generated.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  20. 20. Side effect => Brings (friendly) competition in the free compiler world.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  21. 21. http://gcc.gnu.org/wiki/ClangDiagnosticsComparisonApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  22. 22. http://clang.llvm.org/comparison.html#gccApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  23. 23. Rebuild of Debian using ClangApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  24. 24. Crappy method :VERSION=4.7cd /usr/binrm g++-$VERSION gcc-$VERSION cpp-$VERSIONln -s clang++ g++-$VERSIONln -s clang gcc-$VERSIONln -s clang cpp-$VERSIONcd -April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  25. 25. Testing the rebuild of the package under amd64. NOT the performances (build time or execution) nor the execution of the binariesApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  26. 26. Full results published: http://clang.debian.net/ Done on the cloud-qa - EC2 (Amazon cloud) Thanks to Lucas NussbaumApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  27. 27. Rebuild with clang 3.0 February 28, 2012 15658 packages built : 1381 (8.8 %) failed.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  28. 28. Rebuild with clang 3.1 June 23, 2012 17710 packages built : 2137 (12.1 %) failed.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  29. 29. Rebuild with clang 3.2 January 28, 2013 18264 packages built : 2204 (12.1 %) failed.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  30. 30. Why these differences between 3.0 vs 3.1/3.2?April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  31. 31. Some information about -Wall & -Werror : -Wall enables many warnings -Werror transforms Warning to Errorint main() { $ gcc -Wall -Werror foo.c && echo $? unsigned int i = 0; 0 return i < 0;} $ clang -Wall -Werror foo.c && echo $? foo.c:3:14: error: comparison of unsigned expression < 0 is always false [-Werror,-Wtautological-compare] return i < 0; ~^~ 1 error generated. April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  32. 32. -Werror & unused args 40 occurrencesClang detects unused argument.clang --param ssp-buffer-size=4 clang: error: argument unused-Werror foo.c during compilation: --param ssp- buffer-size=4And generates a normal warning … 96 occurrencesWhich becomes an error with -Werror April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  33. 33. Security check introduced in clang 3.1 9 occurences $ gcc -Werror -c foo.c && echo $?#include <stdio.h> 0void foo(void) { $ clang -Werror -c foo.c && echo $? char buffer[1024]; foo.c:5:23: error: use of %n in sprintf(buffer, "%n", 2); format string discouraged (potentially insecure) [-Werror,-} Wformat-security] sprintf(buffer, "%n", 2); ~^ 1 error generated. April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  34. 34. Some of the most common errorsApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  35. 35. Unsupported options 50 occurrences $ gcc -O9 foo.c && echo $? 0 $ clang -O9 foo.c error: invalid value 9 in -O9 Record by libdbi-drivers with -O20 o/April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  36. 36. Different default behavior 133 occurrences– noreturn.c – $ gcc -c noreturn.c; echo $?int foo(void) { 0 # -Wall shows it as warning return; $ clang -c noreturn.c} noreturn.c:2:2: error: non-void function foo should return a value [-Wreturn-type] return; ^ 1 error generated.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  37. 37. Different default behavior (bis) 15 occurrences $ gcc -c returninvoid.c; echo $?– returninvoid.c – returninvoid.c: In function ‘foo’:void foo(void) { returninvoid.c:2:2: warning: ‘return’ with a return 42; value, in function returning void [enabled by default]} 0 $ clang -c returninvoid.c returninvoid.c:2:2: error: void function foo should not return a value [-Wreturn-type] return 42; ^ ~~ 1 error generated.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  38. 38. Different understanding of the C++ standard– mailboxField.cpp – $ g++ -c mailboxField.cpp && echo $?class address { 0protected: static int parseNext(int a); $ clang++ -c mailboxField.cpp}; mailboxField.cpp:17:22: error: parseNext is aclass mailbox : public address { protected member of address friend class mailboxField; address::parseNext(a);}; ^class mailboxField { mailboxField.cpp:4:16: note: declared protected void parse(int a) { here address::parseNext(a); static int parseNext(int a); // will work with: ^ // mailbox::parseNext(a); } References:}; http://llvm.org/bugs/show_bug.cgi?id=6840 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52136April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  39. 39. Different set of warnings with -Wall Plenty of occurences echo $? $ gcc -Wall -Werror -c foo.cpp &&– plop.c – 0void foo() { $ clang -Wall -Werror -c foo.cpp foo.cpp:3:13: error: equality comparison with extraneous int a=1; parentheses if ((a == 1)) { [-Werror,-Wparentheses-equality] if ((a == 1)) { return; ~~^~~~ foo.cpp:3:13: note: remove extraneous parentheses around the } comparison to} silence this warning if ((a == 1)) { ~ ^ ~ foo.cpp:3:13: note: use = to turn this equality comparison into an assignment if ((a == 1)) { ^~ = 1 error generated. April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  40. 40. gcc extensions which wont be supported 25 occurences– foo.cpp – $ g++ -c foo.cpp; echo $?#include <vector> 0 $ clang++ -c foo.cppvoid foo() { foo.cpp:4:29: error: variable length int N=2; array of non-POD element type std::vector<int> best[2][N]; std::vector<int>} std::vector<int> best[2][N]; ^ 1 error generated.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  41. 41. gcc accepts stuff which should not 34 occurences $ g++ -c foo.cpp; echo $?– foo.cpp – 0// Uncomment this line will fix the issue. $ clang++ -c foo.cpp// template<typename Value_t> foo.cpp:6:5: error: call to function b that is neither// void b(Value_t value) visible in the templatetemplate<typename Value_t> definition nor found by argument-dependent lookupvoid a(Value_t value) { b(value); b(value); ^} foo.cpp:15:5: note: in instantiation of function template specialization a<int> requested heretemplate<typename Value_t>void b(Value_t value) { a(y);} ^ foo.cpp:9:33: note: b should be declared prior to the call site template<typename Value_t> void b(Value_t value)void foo(int y) { ^ a(y); 1 error generated.} April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  42. 42. Last rebuild proved that clang is now ready Remaining problems are upstream Some work started to report bugs upstreamApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  43. 43. StatusApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  44. 44. Packages LLVM-toolchain containing LLVM + Clang + compiler-rt + polly + lldb version 3.2 & 3.3 waiting in Debian NEWApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  45. 45. Build results integrated on the Debian Package Tracking System By Paul WiseApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  46. 46. Version of pbuilder/cowbuilder with a hook to change the compiler By Junich Uekawa Exemple : sudo CHOOSE_COMPILER=clang cowbuilder --build yourpackage_1.00-1.dscApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  47. 47. Interest by upstreams (Google, Intel & Apple) Publication of Debian (wheezy, unstable) and Ubuntu (Quantal, Precise, Raring) of Nightly packages of LLVM, Clang, Compiler-rt, Polly & LLDB http://llvm.org/apt http://blog.llvm.org/2013/04/llvm-debianubuntu-nightly-packages.htmlApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  48. 48. FutureApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  49. 49. Discussions to move the build as a new port on http://www.debian-ports.org/ Various potential issues (not really new arch, mix of gcc/clang-built packages, etc)April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  50. 50. Hack the Debian tools to : Force dpkg to export CC=/usr/bin/cc, ● CXX=/usr/bin/c++ and OBJC=/usr/bin/objc ● Check for hardcoded CC=gcc in debian/rules ● Set the /usr/bin/{cc,c++,objc} alternatives to {clang, clang++}April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  51. 51. Update the debian policy to include something like : Hardcoded usage of CC or CXX (for example, CC=gcc) should be avoid and documented if necessary. Debian build tools must respect the CC and CXX variables if provided. If not, they shall default to /usr/bin/cc and /usr/bin/c++ See : http://lists.debian.org/debian-devel/2012/08/msg00783.htmlApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  52. 52. Add a lintian warning like W: yourpackage: Hardcoded call to gcc/g++. Use /usr/bin/cc or /usr/bin/c++ insteadApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  53. 53. Potential the rebuild of Debian with :●clang+plugin. Ex : polly : cache-localityoptimisation auto-parallelism and vectorization, etc● address sanitizer (ASAN)● scan-build : static C/C++ analyzer● Intel compilersApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  54. 54. Any questions ? Remarks ? Troll ? (+1)April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×