Download to read offline
![chroot:
•
•
•
an operation that changes the apparent root directory
for the current running process [...].
A program that is run in such a modified environment
cannot name (and therefore normally not access) files
outside the designated directory tree.
(from wikipedia)](https://image.slidesharecdn.com/2-131107063916-phpapp02/75/Joe-Damato-31-2048.jpg)
Uploaded byOntico
Joe Damato
The document discusses best practices for building and packaging compiled applications across different platforms. It recommends setting up a build server like Jenkins to enable reproducible builds. Built artifacts should be backed up, such as copying to S3. The document also discusses using tools like chroot, mock, and pbuilder to build in clean environments. It recommends static linking of dependencies where possible and capturing debug symbols to aid in debugging. Automated testing of install/uninstall and correctness is also recommended to catch errors on supported platforms.
More Related Content
Joe Damato
- 1. scaling compiled applications JoeDamato @joedamato
- 2. about me • JoeDamato • San Francisco, CA • i like: • systems programming • debugging • performance analysis
- 3.
- 4.
- 5.
- 18.
- 21. Repeatable builds • themost important thing for compiled software • set up a build server • with jenkins (or another type of software) • backup your built objects • copying them to s3 is not a horrible idea • regardless of source control system or branching strategy, ensure you can always rebuild any version of your software
- 23. Jenkins problems (maybe • gitplugin didn’t work on windows fixed?) • branch building is painful, jenkins API can help • getting windows build agent working is painful
- 26.
- 29. tools for... • RPMs •DEBs • Everything else • windows installers (MSIs) • other linux/unix like OS’s • etc
- 30.
- 31. chroot: • • • an operation thatchanges the apparent root directory for the current running process [...]. A program that is run in such a modified environment cannot name (and therefore normally not access) files outside the designated directory tree. (from wikipedia)
- 36.
- 37.
- 38. Everything else • KVM •Amazon EC2 • other virtualization
- 39.
- 40. KVM • Create abase image on disk • Clone base image • Boot the cloned image • Do the build and copy built object out. • Delete cloned image when done • Base image is still pristine and can be reused.
- 43. Create builds incleanroom • Avoid contaminating builds with artifacts from previous builds. • chroots help • use mock or pbuilder for RPMs and DEBs • KVM, EC2, or equivalent for everything else • Always create pristine base images and do builds in a copy. • Use SSDs
- 45. Tool problems • git-buildpackagecan’t set changelog distribution field • signing key setup is really painful (especially for RPMs) • deb naming scheme for packages is quite painful • all tools are undocumented and very hard to actually use • recent versions of KVM provided by ubuntu fail to boot VMs sometimes
- 48. two types of linking.... • • dynamiclinking static linking
- 49. static linking • callsto functions in library are resolved at compile time. • code from the library is copied into the resulting binary.
- 50. dynamic linking • calls tofunctions are resolved at runtime. • code for a library lives in it’s own object: • libblah.so.4 • libblah.dll
- 52.
- 54.
- 58. static linking • figure outwhich libraries your app needs • pick a supported major release, if possible • build and package this library • link it statically against your binary during build • you now have fewer stones to turn over when debugging
- 60. static linking • youwill need to track your upstream deps • you will probably want to package your upstream deps • you can then point your chroots and build envs at private deb, rpm, etc repos in your internal infrastructure • merging upstream changes in becomes its own project
- 65.
- 74.
- 75. Use the buildsystem Determine which file to build at compile time.
- 78. Use modules • break upifdef soup into separate files • use the build system to compile the right file at build time • this seems obvious but many C libraries and programs are full of crazy ifdef soup.
- 80. Use modules • veryeasy to fall down a rabbit hole breaking things apart • can make the build process more complicated and harder to debug
- 83.
- 85. Capture debug symbols • DEBand RPM can both output debug packages that contain debug symbols. • output these packages. • store these and make backups. • (or just don’t strip your binary)
- 87. Use googlecoredumper tocatch • you can use google-coredumper segfaults, bus errors, and other bad things. • you can output a coredump when this happens. • you can use this coredump and your debug symbols to figure out what is going on.
- 88. Plan for failure • Havea backup plan • Capture debug symbols during your automated build process. • Store them somewhere safe (and make backups). • Capture coredumps (if possible). • Use coredumps and debug symbols to figure out what happened.
- 90. Plan for failure • cansignificantly increase complexity • google coredumper can’t help if your kernel is buggy • some linux distributions don’t allow ptrace • google coredumper only supports linux
- 96.
- 99. Check things like... • Isthe binary actually statically linked? • Does it get copied to the right path? • Are the right config files autogenerated? • Does the version string the program outputs match the package version? • ....
- 100.
- 101. RSpec is usefulfor this.
- 102. Automated Testing test every •It will be impossible to build and change on every supported platform. • Use your build server to do this for you. • Test things like: • installing/uninstalling the object • object is actually statically linked • correctness testing (RSpec can be useful)
- 104. Automated testing too much •Fine line between not enough and • Windows can be painful, but cygwin can help with scripting • Easy to forget about testing the actual package install/removal • Can be difficult to get working with branch builds
- 105.
- 113.
- 114.