Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Gory Details of Debian packages


Published on

This is the presentation I gave at FSCons in 2008

Published in: Technology

The Gory Details of Debian packages

  1. 1. Debian Packages: the gory details <ul><ul><li>An operetta, of sorts. </li></ul></ul>
  2. 2. Debian GNU/Linux <ul><li>Free, community developed operating system </li></ul><ul><li>Always will be free because of its Social Contract and legally shielded by Software in the Public Interest </li></ul><ul><li>Developed by a group of about 1,000 volunteers, who receive no financial compensation </li></ul><ul><li>Stable </li></ul><ul><ul><li>More info in debian-history.deb </li></ul></ul>
  3. 3. Development reference <ul><li>Tens of thousands of packages (~25,000) </li></ul><ul><li>Security, package and bug tracking </li></ul><ul><li>Gigantic infrastructure </li></ul><ul><li>Serves as sort of a reference platform due to its adherence to Linux Standards Base, File Hierarchy Standard, etc. </li></ul>
  4. 4. Different versions <ul><li>Names taken from Toy Story </li></ul><ul><li>Bruce Perens worked for Pixar and was Debian Project Leader at the time </li></ul><ul><li>Current version: etch </li></ul><ul><li>Next version: lenny (after that its squeeze) </li></ul><ul><li>Scheduled for release in 2008, but could be 2009 (currently ~100 RC bugs.) </li></ul>
  5. 5. What about sid? <ul><li>The kid in Toy Story who broke toys </li></ul><ul><li>This is the unstable version of debian, and will always be unstable </li></ul><ul><li>Periodically testing gets frozen (like it is right now), release critical bugs get fixed, and then released as stable </li></ul><ul><li>So - </li></ul><ul><ul><li>Stable -> 4.0 & 4.1 code name etch </li></ul></ul><ul><ul><li>Testing -> 5.0 code name lenny </li></ul></ul><ul><ul><li>Unstable -> sid (Will never have a version) </li></ul></ul>
  6. 6. How software gets into debian <ul><li>Debian uses “packages” to create a modular system </li></ul><ul><li>The package itself is a binary format called a “deb” </li></ul><ul><li>This means it has to be built (compiled) on a packager's machine before it is uploaded to debian </li></ul><ul><li>A developer finds some software, not already in debian, and files an ITP (Intend To Package) bug, then closes the bug with a new deb </li></ul>
  7. 7. Mentoring / Sponsorship <ul><li>If you have a package that you want to get into debian you can submit it to </li></ul><ul><li>Lots of helpful and experienced debian people to get your package in good shape for debian </li></ul>
  8. 8. First stop, unstable <ul><li>A package gets built from source code </li></ul><ul><li>The packager uploads the resulting binary deb and the pristine source code into unstable (This is a bit of a gloss) </li></ul><ul><li>It sits in unstable for 10 days then moves to testing the package gets built on a variety of architectures </li></ul><ul><li>Once in testing it gets frozen and tested again before release for “release critical” bugs </li></ul>
  9. 9. How do you create a deb? <ul><li>TIMTOWTDI, but debian has excellent tools: </li></ul><ul><ul><li>BTS (Bug Tracking System) </li></ul></ul><ul><ul><li>PTS (Package Tracking System) </li></ul></ul><ul><ul><ul><li>Recently moved to a new server, much more responsive. </li></ul></ul></ul><ul><ul><li>dh-make-perl </li></ul></ul><ul><ul><li>dpkg </li></ul></ul><ul><ul><li>Clean room building with pbuilder or cowbuilder </li></ul></ul>
  10. 10. Closer look at the tools <ul><li>BTS is widely used as a development tool, half-way to a million bugs </li></ul><ul><li>Bugs very often move upstream to perl, ruby, python for example </li></ul><ul><li>Specific types of bugs; start with RFP (Request For Packaging) to get a module into debian </li></ul><ul><li>File an ITP (Intend To Package) bug if you are going to package a module for debian </li></ul><ul><li>Filing packaging bugs is important because it prevents the duplication of work </li></ul>
  11. 11. Package tracking <ul><li>The PTS gives you an overview of every package in debian </li></ul><ul><li>You can look through every debian version (i.e. stable, testing, unstable) </li></ul><ul><li>You can follow bug reports, and package status changes via email from the PTS and on the web </li></ul>
  12. 12. More PTS <ul><li>PTS system has recently been updated </li></ul><ul><li>Hosted in Europe by 1&1 </li></ul><ul><li>Significant traffic: 5.7 Million unique visitors (excluding robots) </li></ul><ul><li>122 Million hits monthly, again excluding robots </li></ul><ul><li>By filing an ITP bug you prevent duplicate work and confusion. </li></ul>
  13. 13. dh_make_perl <ul><li>Powerful perl version of dh_make </li></ul><ul><li>Gobbles up a module from cpan (using the cpan shell) </li></ul><ul><li>Then builds the deb with the --build option </li></ul><ul><li>Lots of options and potential for customization </li></ul><ul><li>Calculates perl dependencies AND operating system dependencies </li></ul>
  14. 14. dh-makin' bacon <ul><li>dh-make-perl will create the files required to build a debian source package out of a perl package. </li></ul><ul><li>This works for most simple packages and is also useful for getting started with packaging perl modules. </li></ul><ul><li>You can build and install the debian package using the --build and --install command line switches. </li></ul><ul><li>Using this program is no excuse for not reading the debian developer documentation, including the Debian policy, the perl policy, the packaging manual and so on. </li></ul>
  15. 15. Flow Source code dh-make Debian directory dpkg Deb
  16. 16. dpkg-buildpackage <ul><li>dpkg is also an extremely powerful build tool. It is a set of tools really </li></ul><ul><li>Responsible for installing and removing packages, gets called by high-level tools like apt-get and aptitude </li></ul><ul><li>dpkg-buildpackage builds your deb as well as the .dsc file you need for uploading to debian </li></ul>
  17. 17. How does dpkg know how to build the package? <ul><li>Because dh-make (alternatively dh-make-perl) built, and populated with boilerplate, the debian directory </li></ul><ul><li>This directory holds files and one executable, the debian/rules script </li></ul><ul><li>debian/rules is a Makefile which calls lots of other small debian scripts called debhelper scripts </li></ul><ul><li>This creates a highly modular build system with lots of opportunity for specific, small changes </li></ul>
  18. 18. Pbuilder and cowbuilder <ul><li>Pbuilder stands for personal builder </li></ul><ul><li>Automatically builds debian packages in a “clean-room” environment </li></ul><ul><li>Helps to verify that a package can be built on the average debian system </li></ul><ul><li>It creates a base chroot image, downloads and installs a complete, but minimal, debian system </li></ul><ul><li>Cowbuilder is pbuilder sped up by using cowdancer, a copy-on-write system in userland </li></ul>
  19. 19. lintian <ul><li>Lintian is the debian policy enforcer </li></ul><ul><li>Chatty, and pretty complete </li></ul><ul><li>Being “lintian clean” is pretty much a requirement for getting your deb into debian and definitely if you want to be a debian developer </li></ul>
  20. 20. What else is in the debian dir? <ul><li>The debian directory also contains files which help to define the package and help in packaging. </li></ul><ul><ul><li>Copyright </li></ul></ul><ul><ul><li>Control </li></ul></ul><ul><ul><li>Changelog </li></ul></ul><ul><ul><li>Watch </li></ul></ul><ul><ul><li>Compat </li></ul></ul>
  21. 21. Changelog <ul><li>This required file lists all the changes </li></ul><ul><li>libyaml-perl (0.39-1) unstable; urgency=low </li></ul><ul><li>* New upstream release. </li></ul><ul><li>* New Maintainer with acknowledgment from Chip. </li></ul><ul><li>* Added the copyright statement to debian/copyright. </li></ul><ul><li>* Updated Standards-Version to 3.6.2. </li></ul><ul><li>* Moved the perl build-dep to build-dep-indep (Closes: #305525). </li></ul><ul><li>* Updated compat version to 4 and upgraded the debhelper dependency </li></ul><ul><li>accordingly. </li></ul><ul><li>-- Florian Ragwitz <> Thu, 1 Dec 2005 15:33:42 +0100 </li></ul>
  22. 22. More changelog <ul><li>The recorded changes are mostly relative to the debian packaging, not the upstream source </li></ul><ul><li>Crucially, this is the canonical location of the package's version information </li></ul><ul><li>Specifies which package fixed which bug (specific syntax allows for automated bug closing from the changelog) </li></ul>
  23. 23. Control <ul><li>Source: libpod-strip-perl </li></ul><ul><li>Section: perl </li></ul><ul><li>Priority: optional </li></ul><ul><li>Build-Depends: debhelper (>= 7), libmodule-build-perl </li></ul><ul><li>Build-Depends-Indep: perl (>= 5.6.10-12) </li></ul><ul><li>Maintainer: Debian Perl Group <> </li></ul><ul><li>Uploaders: Damyan Ivanov <> </li></ul><ul><li>Standards-Version: 3.8.0 </li></ul><ul><li>Homepage: </li></ul><ul><li>Vcs-Svn: svn:// </li></ul><ul><li>Vcs-Browser: </li></ul><ul><li>Package: libpod-strip-perl </li></ul><ul><li>Architecture: all </li></ul><ul><li>Depends: ${perl:Depends}, ${misc:Depends} </li></ul><ul><li>Description: remove POD documentation from Perl code </li></ul><ul><li>Pod::Strip is a subclass of Pod::Simple that removes the POD (plain old </li></ul><ul><li>documentation) from Perl code. The POD may optionally be replaced with </li></ul><ul><li>comments so that line numbers of the code stay the same. </li></ul><ul><li>. </li></ul><ul><li>Pod::Strip is useful in Perl code parsers that don't want to bother about POD. </li></ul>
  24. 24. Copyright <ul><li>Please, please, please, included copyright and license information in your modules on CPAN. </li></ul><ul><li>It may not matter that much to you, it matters a lot to those who package your software for linux (at least for the Free Software distros) </li></ul><ul><li>It may matter a lot to you one day </li></ul><ul><li>It strengthens the license to have a copyright and it appears clear that the licenses DO have legal weight (the artistic license just won a big legal victory recently.) </li></ul>
  25. 25. Debian's copyright file <ul><li>Format-Specification: </li></ul><ul><li>Upstream-Maintainer: Thomas Klausner <> </li></ul><ul><li>Upstream-Source: </li></ul><ul><li>Upstream-Name: Pod-Strip </li></ul><ul><li>Files: * </li></ul><ul><li>Copyright: Copyright 2004, 2005, 2006 Thomas Klausner, All Rights Reserved. </li></ul><ul><li>License-Alias: Perl </li></ul><ul><li>License: Artistic | GPL-1+ </li></ul><ul><li>This program is free software; you can redistribute it and/or modify it under </li></ul><ul><li>the same terms as Perl itself. </li></ul>
  26. 26. More copyright fun <ul><li>Files: debian/* </li></ul><ul><li>Copyright: © 2008, Damyan Ivanov <> </li></ul><ul><li>License: Artistic | GPL-1+ </li></ul><ul><li>Packaging is free; you can redistribute it and/or modify it inder the same terms as Perl itself. </li></ul>
  27. 27. Licensing <ul><li>License: Artistic </li></ul><ul><li>This program is free software; you can redistribute it and/or modify </li></ul><ul><li>it under the terms of the Artistic License, which comes with Perl. </li></ul><ul><li>On Debian GNU/Linux systems, the complete text of the Artistic License </li></ul><ul><li>can be found in /usr/share/common-licenses/Artistic </li></ul><ul><li>License: GPL-1+ </li></ul><ul><li>This program is free software; you can redistribute it and/or modify </li></ul><ul><li>it under the terms of the GNU General Public License as published by </li></ul><ul><li>the Free Software Foundation; either version 1, or (at your option) </li></ul><ul><li>any later version. </li></ul><ul><li>On Debian GNU/Linux systems, the complete text of the GNU General </li></ul><ul><li>Public License can be found in `/usr/share/common-licenses/GPL' </li></ul>
  28. 28. Watch file <ul><li># format version number, currently 3; this line is compulsory! </li></ul><ul><li>version=3 </li></ul><ul><li># URL to the package page followed by a regex to search </li></ul><ul><li> .*/Pod-Strip-v?(d[d_.-]+).(?:tar(?:.gz|.bz2)?|tgz|zip)$ </li></ul><ul><li>Allows for an automated update of source and package when there is a new version on CPAN, via the uscan tool </li></ul>
  29. 29. Uscan <ul><li>This allows us to automatically update the package contents without having to rebuild the entire package scaffolding. $ uscan libspreadsheet-read-perl: Newer version (0.29) available on remote site: (local version is 0.28) libspreadsheet-read-perl: Successfully downloaded updated package Spreadsheet-Read-0.29.tgz and symlinked libspreadsheet-read-perl_0.29.orig.tar.gz to it </li></ul>
  30. 30. How do we know there is a new version on CPAN? <ul><li>Our QA reporting tool: </li></ul><ul><li> </li></ul><ul><li>Part of the debian-perl team's workflow </li></ul><ul><li>Some of this functionality is moving towards CPANTS, maybe other automated debian-centric tests suites perhaps? </li></ul>
  31. 31. Managing packages <ul><li>sudo aptitude install courier-filter-perl </li></ul><ul><li>Reading package lists... Done </li></ul><ul><li>Building dependency tree </li></ul><ul><li>Reading state information... Done </li></ul><ul><li>Reading extended state information </li></ul><ul><li>Initializing package states... Done </li></ul><ul><li>Reading task descriptions... Done </li></ul><ul><li>The following NEW packages will be installed: </li></ul><ul><li>courier-authdaemon{a} courier-authlib{a} courier-authlib-userdb{a} courier-base{a} courier-filter-perl courier-mta{a} expect{a} fam{a} libarchive-zip-perl{a} libclamav-client-perl{a} libfam0{a} libltdl3{a} libmail-spf-perl{a} libnet-rblclient-perl{a} libnetaddr-ip-perl{a} portmap{a} tcl8.4{a} </li></ul><ul><li>The following packages will be REMOVED: </li></ul><ul><li>postfix{a} </li></ul><ul><li>0 packages upgraded, 17 newly installed, 1 to remove and 0 not upgraded. </li></ul><ul><li>Need to get 4012kB of archives. After unpacking 8303kB will be used. </li></ul><ul><li>Do you want to continue? [Y/n/?] </li></ul>
  32. 32. Implications of automated packaging systems <ul><li>Obsoletes debian-perl(?) </li></ul><ul><li>Most automated tools currently come with some form of caveat stating that there is no support, i.e. Security tracking. </li></ul><ul><li>Most likely quite useful if you are SURE no one else will ever use your deb </li></ul><ul><li>Creation of a potentially vast invisible, distributed package repository </li></ul><ul><li>Probably inevitable :-( </li></ul>
  33. 33. How to become a debian developer <ul><li>Maintain packages and understand debian's Social Contract </li></ul><ul><li>Contribute to debian </li></ul><ul><li>Patience </li></ul><ul><li>Sponsorship </li></ul><ul><li>You don't need to be a DD to contribute, only tangible difference is you get a cool [email_address] and you get to vote </li></ul><ul><li>If you don't become a DD, you don't go MIA </li></ul><ul><li>The entire process is improving and becoming faster – there are changes afoot (DM, DME, etc.) Potentially contreversial. </li></ul>
  34. 34. Resources and Thank yous <ul><li> </li></ul><ul><li>Integrating perl in a wider distribution: The debian-perl group, by Gunnar Wolf </li></ul><ul><li>The Debian System, by Martin Krafft </li></ul><ul><li>dam, gregoa, gwolf, tincho, nomeata, sukria, djpig, dom, Florian R., Gabor, Jos, Nadim, joeyh, eloy, ntyni, rra, hertzog </li></ul>