Abstract Building packages is a resource consuming task and can take very long on embedded devices or low power architectures. Being able to use the power of amd64 servers to build packages for arm or mips allows to make this task faster and less tedious. This talk will cover the following topic: State of art to build arm/mips packages on FreeBSD from a powerful amd64 box How to create a cross building environement How the ports tree does automatically handle cross building How dependencies are handled when cross building packages How to workaround non cross buildable or broken build system like perl and python Cross build monster ports: chromium, libreoffice, openjdk and firefox What are the current limitation Future directions for the cross building framework in the ports tree. Speaker bio Baptiste Daroussin is a unix system engineer, FreeBSD committer for both base and ports, a member of the port management team. He is responsible for a couple of the important changes that happened in the ports over the past few years: New options framework, pkgng, Stage support and more.

1. Cross build in the FreeBSD ports tree
Baptiste Daroussin
bapt@FreeBSD.org
EuroBSDCon 2014
Sofia - Bulgaria
September 28, 2014

2. Goals
Building packages for Tiers-2 arches
Building packages for low power machines
Building boostrap packages for non self hosting languages
Building ”emulation” ports (aka linuxulator)

3. Easier way: qemu user emulation
binary image activator
18k packages sucessfully build for armv6 (thanks sbruno!)
Requires no particular modification of the ports tree beside

4. Easier way: qemu user emulation
binary image activator
18k packages sucessfully build for armv6 (thanks sbruno!)
Requires no particular modification of the ports tree beside
qemu-bsd-user is still buggy and fragile
slow

5. Hybrid way: qemu user emulation + native cross tools
use qemu-bsd + binary image activator
native cross toolchain

6. Hybrid way: qemu user emulation + native cross tools
use qemu-bsd + binary image activator
native cross toolchain
qemu-bsd-user is still buggy and fragile
still slow

7. The one true way: cross compilation
Faster
Simpler
Easier to use for regular users

8. The one true way: cross compilation
Faster
Simpler
Easier to use for regular users
overhead some ports are built twice

9. Build systems
Good Players:

10. Build systems
Good Players:
autotools:

11. Build systems
Good Players:
autotools: really works out of box

12. Build systems
Good Players:
autotools: really works out of box ... when used correctly...

13. Build systems
Good Players:
autotools: really works out of box ... when used correctly...
cmake

14. Build systems
Good Players:
autotools: really works out of box ... when used correctly...
cmake
/usr/share/mk/* (somehow)

15. Build systems
Good Players:
autotools: really works out of box ... when used correctly...
cmake
/usr/share/mk/* (somehow)
The bad players

16. Build systems
Good Players:
autotools: really works out of box ... when used correctly...
cmake
/usr/share/mk/* (somehow)
The bad players
scons

17. Build systems
Good Players:
autotools: really works out of box ... when used correctly...
cmake
/usr/share/mk/* (somehow)
The bad players
scons
The cusom home made build systems

18. Build systems
Good Players:
autotools: really works out of box ... when used correctly...
cmake
/usr/share/mk/* (somehow)
The bad players
scons
The cusom home made build systems
./please build me.sh

19. Main complications

20. Main complications
Perl
Cross build friendly

21. Main complications
Perl
Cross build friendly

22. Main complications
Perl
Cross build friendly
by requiring a ssh connection to a target server ...

23. Main complications
Perl
Cross build friendly
by requiring a ssh connection to a target server ...
Python
Cross build friendly

24. Main complications
Perl
Cross build friendly
by requiring a ssh connection to a target server ...
Python
Cross build friendly ... almost

25. Main complications
Perl
Cross build friendly
by requiring a ssh connection to a target server ...
Python
Cross build friendly ... almost
try to run the built python instead of a native one :(

26. Main complications
Perl
Cross build friendly
by requiring a ssh connection to a target server ...
Python
Cross build friendly ... almost
try to run the built python instead of a native one :(
FreeBSD ports wtf

27. Main complications
Perl
Cross build friendly
by requiring a ssh connection to a target server ...
Python
Cross build friendly ... almost
try to run the built python instead of a native one :(
FreeBSD ports wtf (fixed now thanks python@)

28. Main complications
Perl
Cross build friendly
by requiring a ssh connection to a target server ...
Python
Cross build friendly ... almost
try to run the built python instead of a native one :(
FreeBSD ports wtf (fixed now thanks python@)
Working patches available for very very long still not fully in
python 3.3
Python 3.4?
OpenJDK

29. Main complications
Perl
Cross build friendly
by requiring a ssh connection to a target server ...
Python
Cross build friendly ... almost
try to run the built python instead of a native one :(
FreeBSD ports wtf (fixed now thanks python@)
Working patches available for very very long still not fully in
python 3.3
Python 3.4?
OpenJDK
Cross build friendly

30. Main complications
Perl
Cross build friendly
by requiring a ssh connection to a target server ...
Python
Cross build friendly ... almost
try to run the built python instead of a native one :(
FreeBSD ports wtf (fixed now thanks python@)
Working patches available for very very long still not fully in
python 3.3
Python 3.4?
OpenJDK
Cross build friendly ... It really is!

31. Toolchains

32. Toolchains
Clang
a cross build friendly compiler

33. Toolchains
Clang
a cross build friendly compiler
number of targets very limited (only sane arm on FreeBSD)

34. Toolchains
Clang
a cross build friendly compiler
number of targets very limited (only sane arm on FreeBSD)
GCC
gcc 4.2

35. Toolchains
Clang
a cross build friendly compiler
number of targets very limited (only sane arm on FreeBSD)
GCC
gcc 4.2 ... real world needs a modern compiler

36. Toolchains
Clang
a cross build friendly compiler
number of targets very limited (only sane arm on FreeBSD)
GCC
gcc 4.2 ... real world needs a modern compiler
FreeBSD people never upstream lots of patches

37. Toolchains
Clang
a cross build friendly compiler
number of targets very limited (only sane arm on FreeBSD)
GCC
gcc 4.2 ... real world needs a modern compiler
FreeBSD people never upstream lots of patches
Not really a cross build friendly compiler

38. Toolchains
Clang
a cross build friendly compiler
number of targets very limited (only sane arm on FreeBSD)
GCC
gcc 4.2 ... real world needs a modern compiler
FreeBSD people never upstream lots of patches
Not really a cross build friendly compiler
No consistent behaviour between gcc and clang

39. Toolchains
Clang
a cross build friendly compiler
number of targets very limited (only sane arm on FreeBSD)
GCC
gcc 4.2 ... real world needs a modern compiler
FreeBSD people never upstream lots of patches
Not really a cross build friendly compiler
No consistent behaviour between gcc and clang
binutils

40. Toolchains
Clang
a cross build friendly compiler
number of targets very limited (only sane arm on FreeBSD)
GCC
gcc 4.2 ... real world needs a modern compiler
FreeBSD people never upstream lots of patches
Not really a cross build friendly compiler
No consistent behaviour between gcc and clang
binutils
FreeBSD patches for arm were missing

41. Toolchains
Clang
a cross build friendly compiler
number of targets very limited (only sane arm on FreeBSD)
GCC
gcc 4.2 ... real world needs a modern compiler
FreeBSD people never upstream lots of patches
Not really a cross build friendly compiler
No consistent behaviour between gcc and clang
binutils
FreeBSD patches for arm were missing
Cross build friendly for all

42. Toolchains
Clang
a cross build friendly compiler
number of targets very limited (only sane arm on FreeBSD)
GCC
gcc 4.2 ... real world needs a modern compiler
FreeBSD people never upstream lots of patches
Not really a cross build friendly compiler
No consistent behaviour between gcc and clang
binutils
FreeBSD patches for arm were missing
Cross build friendly for all ... but gas

43. Making a cross building environment
make xdev

44. Making a cross building environment
make xdev
Create a sysroot
Create a cross build toolchain

45. Making a cross building environment
make xdev
Create a sysroot
Create a cross build toolchain
Inconsistent over versions
gcc/clang problems
Use clang
clang from base is available and recent enough
fall back on clang from ports otherwise
use binutils from ports all the time

46. Making a cross building environment
make xdev
Create a sysroot
Create a cross build toolchain
Inconsistent over versions
gcc/clang problems
Use clang
clang from base is available and recent enough
fall back on clang from ports otherwise
use binutils from ports all the time ...requires fixing our *.S
files

47. Making a cross building environment
make xdev
Create a sysroot
Create a cross build toolchain
Inconsistent over versions
gcc/clang problems
Use clang
clang from base is available and recent enough
fall back on clang from ports otherwise
use binutils from ports all the time ...requires fixing our *.S
files
create a ports cross building aware version of freebsd

48. Making a cross building environment
make xdev
Create a sysroot
Create a cross build toolchain
Inconsistent over versions
gcc/clang problems
Use clang
clang from base is available and recent enough
fall back on clang from ports otherwise
use binutils from ports all the time ...requires fixing our *.S
files
create a ports cross building aware version of freebsd ...
requires upstreaming our patches

49. Making a cross building environment
make xdev
Create a sysroot
Create a cross build toolchain
Inconsistent over versions
gcc/clang problems
Use clang
clang from base is available and recent enough
fall back on clang from ports otherwise
use binutils from ports all the time ...requires fixing our *.S
files
create a ports cross building aware version of freebsd ...
requires upstreaming our patches

50. Making a cross building environment (create the sysroot)
$make sysroot:
make: don’t know how to make sysroot. Stop

51. Making a cross building environment (create the sysroot)
$make sysroot:
make: don’t know how to make sysroot. Stop
any way manually that is easy

52. Making a cross building environment (create the sysroot)
$make sysroot:
make: don’t know how to make sysroot. Stop
any way manually that is easy
TARGET?= arm
TARGET_ARCH?= armv6
XCFLAGS= isystem ${WRKDIR}/tmp/usr/include -L${WRKDIR}/tmp/usr/lib
--sysroot=${WRKDIR}/tmp/ -B${LOCALBASE}/arm-gnueabi-freebsd/bin
-B/usr/bin
-target armv6-gnueabi-freebsd10.0
XMAKE_ENV= PATH=${LOCALBASE}/arm-gnueabi-freebsd/bin:/usr/bin:/usr/sbin:/bin
WITHOUT_PROFILE=yes __MAKE_CONF=/dev/null SRCCONF=/dev/null
NO_FSCHG=yes MAKEOBJDIRPREFIX=${WRKDIR}/obj
TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH}
MACHINE=${TARGET} MACHINE_ARCH=${TARGET_ARCH}
_SHLIBDIRPREFIX=${WRKDIR}/tmp
CC="${CC} ${XCFLAGS}"
CPP="${CPP} ${XCFLAGS}"
CXX="${CXX} ${XCFLAGS}"
NO_WERROR=yes NO_WARNS=yes
NOFUN= -DNO_FSCHG MK_HTML=no MK_INFO=no -DNO_LINT
MK_MAN=no MK_NLS=no -DNO_PROFILE
MK_KERBEROS=no MK_RESCUE=no MK_TESTS=no -DNO_WARNS
cd ${WRKSRC}/lib/ncurses/ncurses ;
MAKEOBJDIRPREFIX=${WRKDIR}/obj make build-tools
cd ${WRKSRC};
mtree -R uid,gid -deU -f etc/mtree/BSD.root.dist -p ${WRKDIR}/tmp >/dev/null ;
mtree -R uid,gid -deU -f etc/mtree/BSD.usr.dist -p ${WRKDIR}/tmp/usr >/dev/null ;
mtree -R uid,gid -deU -f etc/mtree/BSD.include.dist -p ${WRKDIR}/tmp/usr/include >/dev/null ;
setenv -i ${XMAKE_ENV} WITHOUT_MAN=yes -f Makefile.inc1 par-includes libraries
DESTDIR=${WRKDIR}/tmp

53. Changes to the ports infrastructure
Variable set when cross building
HCC/HCXX (host compiler)
CC/CXX (set to the cross compiler + special flags)
STRIP CMD to the cross binutils version
ABI FILE=${X SYSROOT}/usr/lib/crt1.o
PKG CONFIG SYSROOT DIR=”${X SYSROOT}”

54. Changes to the ports infrastructure
Behaviour changed
LIB DEPENDS BUILD DEPENDS are built twice: native and
target
native are installed on the host
target are installed to the sysroot
Automatically add dependencies to sysroot (if not provided)
and toolchain

55. Changes to the ports infrastructure
Behaviour changed
LIB DEPENDS BUILD DEPENDS are built twice: native and
target
native are installed on the host
target are installed to the sysroot
Automatically add dependencies to sysroot (if not provided)
and toolchain
tweaks have to be done ports by ports

56. Ports tweak
Perl
perl-cross (unofficial)
provide config.h per supported architecture/freebsd version

57. Ports tweak
Perl
perl-cross (unofficial)
provide config.h per supported architecture/freebsd version
Python
patch python 2.7 to 3.3 to use native python
check python 3.4

58. Ports tweak
Perl
perl-cross (unofficial)
provide config.h per supported architecture/freebsd version
Python
patch python 2.7 to 3.3 to use native python
check python 3.4
Scons
impossible to get a global solution
use a saner build system

59. Ports point of view
Without sysroot
# cd devel/pkgconf
# make X_BUILD_FOR=armv6-gnueabi-freebsd10.0 package
With sysroot
# cd devel/pkgconf
# make X_BUILD_FOR=armv6-gnueabi-freebsd10.0
X_SYSROOT=/path/to/sysroot package

60. Limitations
ports requiring a different compiler than the default are not
supported (meaning openmp and non libc++ ports ports
using C nested functions)
plateforms using gcc as a default compiler doesn’t work

61. Thank you!
Questions ?