LCU14 209- LLVM Linux --------------------------------------------------- Speaker: Behan Webster Date: September 16, 2014 --------------------------------------------------- ★ Session Summary ★ This session will provide an update on the status of the LLVMLinux project; a project which is cooperating with both the Linux kernel and LLVM communities to build the Linux kernel with Clang/LLVM. This talk will also cover new things in LLVM which make clang even more attractive to the kernel community. LLVM is an extensive compiler technology suite which is already commonplace from Android/Renderscript and OpenCL through to high performance computing clusters. --------------------------------------------------- ★ Resources ★ Zerista: http://lcu14.zerista.com/event/member/137735 Google Event: https://plus.google.com/u/0/events/cp34l6uc5rmu189qn8eq3ps3bkc Video: https://www.youtube.com/watch?v=gcVQxjb6iQ4&list=UUIVqQKxCyQLJS6xvSmfndLA Etherpad: http://pad.linaro.org/p/lcu14-209 --------------------------------------------------- ★ Event Details ★ Linaro Connect USA - #LCU14 September 15-19th, 2014 Hyatt Regency San Francisco Airport --------------------------------------------------- http://www.linaro.org http://connect.linaro.org

1. LCU14-209: LLVMLinux
Behan Webster, LCU14
LCU14 BURLINGAME

2. Linaro Overview
● About the LLVMLinux project
● State of Linux kernel being compiled with clang
● Details of specific known problems/patches still being upstreamed
● How can you help?
● Contact info

3. The LLVMLinux Project Goals
● Fully build the Linux kernel for multiple
architectures, using the Clang/LLVM toolchain
● Discover LLVM/Kernel issues early and find fixes
quickly across both communities
● Upstream patches to the Linux Kernel and LLVM
projects
● Bring together like-minded developers
● Enable the kernel community to do more in depth
analysis of the kernel code

4. LLVMLinux Build/Test System
● Fetches, patches, builds, tests: clang, kernel, qemu, etc
○ git clone http://git.linuxfoundation.org/llvmlinux.git
○ cd llvmlinux/target/vexpress (or vexpress64)
○ make

5. Patched Mainline Kernel Tree
● A mainline kernel tree with all LLVMLinux patches applied on top is
now available:
○ git://git.linuxfoundation.org/llvmlinux/kernel.git
● Dated llvmlinux branches
○ remotes/origin/llvmlinux-2014.09.16
● The master branch is rebased regularly

6. LLVMLinux Project Status
● LLVM/clang:
○ All LLVMLinux patches for LLVM are Upstream
○ Newer LLVM patches to support the Linux kernel are mostly
being added by upstream maintainers
● Linux Kernel:
○ Roughly 49 kernel patches for various arches
○ LLVMLinux branch in linux-next

7. Remaining LLVMLinux Kernel Patches
● Patches still working their way upstream
Architecture Number of
Patches
Patches
Submitted
Patches
Accepted
all 23 17 2
arm 12 11 1 (+7)
aarch64 11 8 7
x86_64 3 1 1
TOTAL 49 37 11

8. LLVMLinux Kernel Patches
● Total patches currently required for a single architecture
Architecture Number of
Patches
Patches
Submitted
Patches
Accepted
arm (+all) 35 (12+23) 28 (11+17) 3 (1+2)
aarch64 (+all) 34 (11+23) 25 (8+17) 9 (7+2)
x86_64 (+all) 26 (3+23) 18 (1+17) 3 (1+2)

9. Kbuild
● Basic Kbuild support for clang and x86 has been upstreamed
● Specific support for ARM and aarch64 ready to be upstreamed

10. Integrated Assembly Status
● Renato Golin, Vicinius Tinti, Saleem Abdulrasool and Stepan
Dyatkovskiy are working on fixing IA issues in clang to support the
Linux ARM kernel code (and ultimately AARCH64)
● David Woodhouse has added .code16 support for X86 ASM
● For now we disable the IA and use GNU as instead

11. Different option passing
● gcc passes -march to GNU as
● clang doesn't... (Bug submitted PR)
● Probably should be fixed in clang
● Work around patch for now
-CFLAGS_aes-ce-cipher.o += -march=armv8-a+crypto
+CFLAGS_aes-ce-cipher.o += -march=armv8-a+crypto -Wa,-march=armv8-a+crypto

12. extern inline: Different for gnu89 and gnu99
● GNU89/GNU90 (used by gcc)
○ Function will be inlined where it is used
○ No function definition is emitted
○ A non-inlined function may also be provided
● GNU99/C99 (used by clang)
○ Function will be inlined where it is used
○ An external function is emitted
○ No other function of the same name may be provided.
● Solution? Use “static inline” instead.
● Only still an issue for ARM support for ftrace (submitted)

13. Attribute Order
● gcc is less picky about placement of __attribute__(())
● clang requires it at the end of the type or variable
-struct __read_mostly va_alignment va_align = {
+struct va_alignment __read_mostly va_align = {
● (This particular patch was just accepted)

14. Named Registers
● (Named registers for X86 kernel have been removed from the
mainline kernel by Andi Kleen)
● ARM and AARCH64 still like using named registers
● Clang now supports using a globally named register for the stack
pointer (Thanks Renato!)
● For ARM/AARCH64 move to using a global in asm/thread_info.h
register unsigned long current_stack_pointer asm ("sp");
● Patches for AARCH64 now accepted, acked for ARM

15. ARM percpu patch
● One of the uses of Named Registers in the ARM code is due to a
deficiency in gcc
● The new code which works with gcc fails in clang
● Solution, provide routines for both, and choose at compile time
● Gcc:
asm("mrc p15, 0, %0, c13, c0, 4" : "=r" (off) : "Q" (*sp));
● Clang:
asm("mrc p15, 0, %0, c13, c0, 4" : "=r" (off) : : "memory");

16. Missing “%a” for inline ASM
● The following error is generated with clang for AArch64:
error: invalid operand in inline asm: 'prfm pldl1keep, ${0:a}'
● Per comments by Tim Northover on the LLVM Bug database:
It's rather unclear how it's better than "prfm pstl1keep, [%0]" though. Not
all instructions can make use of any offset, so wouldn't we have to be
conservative and always map it to "[xN]"?
● When %a0 is changed to [%x0] it uncovered a GCC bug: https://bugs.
linaro.org/show_bug.cgi?id=635
● Changing the "p" to "r" resolves the issue for both clang and GCC.
- asm volatile("prfm pldl1keep, %a0n" : : "p" (ptr));
+ asm volatile("prfm pldl1keep, [%x0]n" : : "r" (ptr));

17. Section Mismatch Issues (MergedGlobals)
● By default clang merges globals with internal linkage into one:
MergedGlobals
● Allows globals to be addressed using offsets from a base pointer
● Can reduce the number of registers used
● Modpost script in the Linux kernel uses symbol names to look for
section mismatches (e.g. regular code calling init code)
● MergedGlobals breaks modpost (false positive section
mismatches)
● Current solution: use -mno-global-merge to stop global merging
● Updates to modpost may allow this optimization to be enabled
again in the future

18. ARM eabi support
● Clang emits code which uses the “aeabi” ARM calls which are
implemented in compiler-rt (equivalient to libgcc)
● Compiler-rt doesn't easily cross compile yet...
void __aeabi_memcpy(void *dest, const void *src, size_t n)
void __aeabi_memmove(void *dest, const void *src, size_t n)
void __aeabi_memset(void *s, size_t n, int c)
● Still needed for ARM
● No longer required for AARCH64

19. Variable Length Arrays In Structs
● VLAIS isn't supported by Clang (undocumented gcc extension)
char vla[n]; /* Supported, C99/C11 */
struct {
char flexible_member[]; /* Supported, C99/C11 */
} struct_with_flexible_member;
struct {
char vlais[n]; /* Explicitly not allowed by C99/C11 */
} variable_length_array_in_struct;
● VLAIS is used in the Linux kernel in a number of places, spreading
mostly through reusing patterns from data structures found in
crypto

20. VLAIS Removal Example (from crypto/hmac.c)
- struct {
- struct shash_desc shash;
- char ctx[crypto_shash_descsize(hash)];
- } desc;
+ char desc[sizeof(struct shash_desc)
+
+ crypto_shash_descsize(hash)] CRYPTO_MINALIGN_ATTR;
+ struct shash_desc *shash = (struct shash_desc *)desc;
unsigned int i;
- desc.shash.tfm = hash;
+ shash->tfm = hash;

21. VLAIS Removal Example (the missing pieces)
#define ARCH_KMALLOC_MINALIGN __alignof__(unsigned long long)
#define CRYPTO_MINALIGN ARCH_KMALLOC_MINALIGN
#define CRYPTO_MINALIGN_ATTR __attribute__ ((__aligned__(CRYPTO_MINALIGN)))
struct shash_desc {
struct crypto_shash *tfm;
u32 flags;
void *__ctx[] CRYPTO_MINALIGN_ATTR;
};

22. Status of VLAIS in the Linux Kernel
● USB Gadget patch is in mainline
● Mac80211 patch is in mainline
● Netfilter patch is in mainline
● apparmor patch accepted
● Bluetooth patch accepted
● Only the VLAIS patches for crypto are left:
(btrfs, dm-crypt, hmac, libcrc32c, testmgr, etc)
● However, we recently found a few previously unknown instances of
VLAIS in raid10 and exofs...

23. Huge use of VLAIS in fs/exofs/ore_raid.c
static int _sp2d_alloc(unsigned pages_in_unit, unsigned group_width,
unsigned parity, struct __stripe_pages_2d **psp2d)
[...]
unsigned data_devs = group_width - parity;
struct _alloc_all_bytes {
struct __alloc_stripe_pages_2d {
struct __stripe_pages_2d sp2d;
struct __1_page_stripe _1p_stripes[pages_in_unit];
} __asp2d;
struct __alloc_1p_arrays {
struct page *pages[group_width];
struct page *scribble[group_width];
char page_is_read[data_devs];
} __a1pa[pages_in_unit];
} *_aab;

24. How Can You Help?
● Don’t use the non-C99 practices we showed in previous slides
● Make it known you want to be able to use Clang to compile the
kernel (tell your Linaro representative!)
● Test LLVMLinux patches
● Report bugs to the LLVMLinux mailing list
● Help get LLVMLinux patches upstream
● Work on unsupported features and Bugs
○ http://llvm.linuxfoundation.org/index.php/Broken_kernel_options
● Submit new targets and arch support
● Patches welcome

25. Embrace the
Dragon.
He's cuddly.
Thank you
http://llvm.linuxfoundation.org

26. Contribute to the LLVMLinux Project
● Project wiki page
○ http://llvm.linuxfoundation.org
● Project Mailing List
○ http://lists.linuxfoundation.org/mailman/listinfo/llvmlinux
○ http://lists.linuxfoundation.org/pipermail/llvmlinux/
● IRC Channel
○ #llvmlinux on OFTC
○ http://buildbot.llvm.linuxfoundation.org/irclogs/OFTC/%23llvmlinux/
● LLVMLinux Community on Google Plus

27. More about Linaro Connect: connect.linaro.org
Linaro members: www.linaro.org/members
More about Linaro: www.linaro.org/about/