• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Distro Recipes 2013 : Make Debian and  compiler agnostic
 

Distro Recipes 2013 : Make Debian and compiler agnostic

on

  • 2,243 views

https://distro-recipes.org

https://distro-recipes.org

Statistics

Views

Total Views
2,243
Views on SlideShare
2,243
Embed Views
0

Actions

Likes
0
Downloads
6
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Distro Recipes 2013 : Make Debian and  compiler agnostic Distro Recipes 2013 : Make Debian and compiler agnostic Presentation Transcript

    • Make Debian compiler agnostic : A rebuild with ClangSylvestre Ledru – sylvestre@debian.org
    • Disclamer !April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Current status : All C, C++, Objective-C sources are being built with gcc for all supported Debian arches and Kernel.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • gcc is THE FLOSS compiler for the last 25 years Used for (pretty much) everywhere or anything Still growing and maintained (cf gcc 4.8)April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Why a new compiler in Debian ?April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Other compilers can find programming errors that gcc could not findApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Code built by many compilers is more likely to be more strictly correct and more portable then code only built with gccApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Some compilers can have advantages on some archs (ex : clang on ARM)April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Provide some interesting side projects (lldb, scan- build, ASAN, libc++, etc)April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • As we were able to do with decoupling Linuxfrom Debian with kFreeBSD and the HURD, were aiming to decouple gcc in Debian. Open the door to other compilers or analyzersApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • LLVM/ClangApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Started as an academic project Versatile platform for compilation and virtual machine Designed originally for the investigation of dynamic compilation techniques for static and dynamic languagesApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Developed initially as a base for a next major GCC release Source: http://gcc.gnu.org/ml/gcc/2005-11/msg00888.htmlApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Sponsored by Apple since 2005 to replace gcc (GPL vs BSD) Has now a strong and diverse community (academics, individuals and corporates) Many universities/research centers are basing their research on LLVMApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Clang C, C++ & Objective-C compiler. (no Fortran) Based on LLVM Default compiler for Mac OS X (Xcode)/iOS [1] and FreeBSD [2] Sources: [1] https://developer.apple.com/technologies/tools/ [2] http://lists.freebsd.org/pipermail/freebsd-stable/2012-May/067486.htmlApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Some advantages : More recent base code (ie less legacy code) Strong interest of hardware manufactors (ARM, MIPS, Intel, Nvidia, etc) Supposed to be faster to build code than gcc Accept the same arguments as gccApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Example Full build of Scilab (doc, essential tests) ~24 minutes gcc ~22 minutes clangApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Some advantages (bis) More intelligent detections– foo.c – $ gcc -Wall -Werror foo.c ; echo $?int main() { 0 unsigned int i = 0; $ clang -Werror foo.c foo.c:3:17: error: comparison of return i < 0; unsigned expression < 0 is always} false [-Werror,-Wtautological-compare] return i < 0; ~^~ 1 error generated.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Side effect => Brings (friendly) competition in the free compiler world.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • http://gcc.gnu.org/wiki/ClangDiagnosticsComparisonApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • http://clang.llvm.org/comparison.html#gccApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Rebuild of Debian using ClangApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Crappy method :VERSION=4.7cd /usr/binrm g++-$VERSION gcc-$VERSION cpp-$VERSIONln -s clang++ g++-$VERSIONln -s clang gcc-$VERSIONln -s clang cpp-$VERSIONcd -April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Testing the rebuild of the package under amd64. NOT the performances (build time or execution) nor the execution of the binariesApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Full results published: http://clang.debian.net/ Done on the cloud-qa - EC2 (Amazon cloud) Thanks to Lucas NussbaumApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Rebuild with clang 3.0 February 28, 2012 15658 packages built : 1381 (8.8 %) failed.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Rebuild with clang 3.1 June 23, 2012 17710 packages built : 2137 (12.1 %) failed.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Rebuild with clang 3.2 January 28, 2013 18264 packages built : 2204 (12.1 %) failed.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Why these differences between 3.0 vs 3.1/3.2?April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Some information about -Wall & -Werror : -Wall enables many warnings -Werror transforms Warning to Errorint main() { $ gcc -Wall -Werror foo.c && echo $? unsigned int i = 0; 0 return i < 0;} $ clang -Wall -Werror foo.c && echo $? foo.c:3:14: error: comparison of unsigned expression < 0 is always false [-Werror,-Wtautological-compare] return i < 0; ~^~ 1 error generated. April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • -Werror & unused args 40 occurrencesClang detects unused argument.clang --param ssp-buffer-size=4 clang: error: argument unused-Werror foo.c during compilation: --param ssp- buffer-size=4And generates a normal warning … 96 occurrencesWhich becomes an error with -Werror April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Security check introduced in clang 3.1 9 occurences $ gcc -Werror -c foo.c && echo $?#include <stdio.h> 0void foo(void) { $ clang -Werror -c foo.c && echo $? char buffer[1024]; foo.c:5:23: error: use of %n in sprintf(buffer, "%n", 2); format string discouraged (potentially insecure) [-Werror,-} Wformat-security] sprintf(buffer, "%n", 2); ~^ 1 error generated. April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Some of the most common errorsApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Unsupported options 50 occurrences $ gcc -O9 foo.c && echo $? 0 $ clang -O9 foo.c error: invalid value 9 in -O9 Record by libdbi-drivers with -O20 o/April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Different default behavior 133 occurrences– noreturn.c – $ gcc -c noreturn.c; echo $?int foo(void) { 0 # -Wall shows it as warning return; $ clang -c noreturn.c} noreturn.c:2:2: error: non-void function foo should return a value [-Wreturn-type] return; ^ 1 error generated.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Different default behavior (bis) 15 occurrences $ gcc -c returninvoid.c; echo $?– returninvoid.c – returninvoid.c: In function ‘foo’:void foo(void) { returninvoid.c:2:2: warning: ‘return’ with a return 42; value, in function returning void [enabled by default]} 0 $ clang -c returninvoid.c returninvoid.c:2:2: error: void function foo should not return a value [-Wreturn-type] return 42; ^ ~~ 1 error generated.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Different understanding of the C++ standard– mailboxField.cpp – $ g++ -c mailboxField.cpp && echo $?class address { 0protected: static int parseNext(int a); $ clang++ -c mailboxField.cpp}; mailboxField.cpp:17:22: error: parseNext is aclass mailbox : public address { protected member of address friend class mailboxField; address::parseNext(a);}; ^class mailboxField { mailboxField.cpp:4:16: note: declared protected void parse(int a) { here address::parseNext(a); static int parseNext(int a); // will work with: ^ // mailbox::parseNext(a); } References:}; http://llvm.org/bugs/show_bug.cgi?id=6840 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52136April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Different set of warnings with -Wall Plenty of occurences echo $? $ gcc -Wall -Werror -c foo.cpp &&– plop.c – 0void foo() { $ clang -Wall -Werror -c foo.cpp foo.cpp:3:13: error: equality comparison with extraneous int a=1; parentheses if ((a == 1)) { [-Werror,-Wparentheses-equality] if ((a == 1)) { return; ~~^~~~ foo.cpp:3:13: note: remove extraneous parentheses around the } comparison to} silence this warning if ((a == 1)) { ~ ^ ~ foo.cpp:3:13: note: use = to turn this equality comparison into an assignment if ((a == 1)) { ^~ = 1 error generated. April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • gcc extensions which wont be supported 25 occurences– foo.cpp – $ g++ -c foo.cpp; echo $?#include <vector> 0 $ clang++ -c foo.cppvoid foo() { foo.cpp:4:29: error: variable length int N=2; array of non-POD element type std::vector<int> best[2][N]; std::vector<int>} std::vector<int> best[2][N]; ^ 1 error generated.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • gcc accepts stuff which should not 34 occurences $ g++ -c foo.cpp; echo $?– foo.cpp – 0// Uncomment this line will fix the issue. $ clang++ -c foo.cpp// template<typename Value_t> foo.cpp:6:5: error: call to function b that is neither// void b(Value_t value) visible in the templatetemplate<typename Value_t> definition nor found by argument-dependent lookupvoid a(Value_t value) { b(value); b(value); ^} foo.cpp:15:5: note: in instantiation of function template specialization a<int> requested heretemplate<typename Value_t>void b(Value_t value) { a(y);} ^ foo.cpp:9:33: note: b should be declared prior to the call site template<typename Value_t> void b(Value_t value)void foo(int y) { ^ a(y); 1 error generated.} April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Last rebuild proved that clang is now ready Remaining problems are upstream Some work started to report bugs upstreamApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • StatusApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Packages LLVM-toolchain containing LLVM + Clang + compiler-rt + polly + lldb version 3.2 & 3.3 waiting in Debian NEWApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Build results integrated on the Debian Package Tracking System By Paul WiseApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Version of pbuilder/cowbuilder with a hook to change the compiler By Junich Uekawa Exemple : sudo CHOOSE_COMPILER=clang cowbuilder --build yourpackage_1.00-1.dscApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Interest by upstreams (Google, Intel & Apple) Publication of Debian (wheezy, unstable) and Ubuntu (Quantal, Precise, Raring) of Nightly packages of LLVM, Clang, Compiler-rt, Polly & LLDB http://llvm.org/apt http://blog.llvm.org/2013/04/llvm-debianubuntu-nightly-packages.htmlApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • FutureApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Discussions to move the build as a new port on http://www.debian-ports.org/ Various potential issues (not really new arch, mix of gcc/clang-built packages, etc)April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Hack the Debian tools to : Force dpkg to export CC=/usr/bin/cc, ● CXX=/usr/bin/c++ and OBJC=/usr/bin/objc ● Check for hardcoded CC=gcc in debian/rules ● Set the /usr/bin/{cc,c++,objc} alternatives to {clang, clang++}April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Update the debian policy to include something like : Hardcoded usage of CC or CXX (for example, CC=gcc) should be avoid and documented if necessary. Debian build tools must respect the CC and CXX variables if provided. If not, they shall default to /usr/bin/cc and /usr/bin/c++ See : http://lists.debian.org/debian-devel/2012/08/msg00783.htmlApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Add a lintian warning like W: yourpackage: Hardcoded call to gcc/g++. Use /usr/bin/cc or /usr/bin/c++ insteadApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Potential the rebuild of Debian with :●clang+plugin. Ex : polly : cache-localityoptimisation auto-parallelism and vectorization, etc● address sanitizer (ASAN)● scan-build : static C/C++ analyzer● Intel compilersApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru
    • Any questions ? Remarks ? Troll ? (+1)April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru