Debian Packages:   the gory details An operetta, of sorts.
Debian GNU/Linux Free, community developed operating system Always will be free because of its Social Contract and legally shielded by Software in the Public Interest Developed by a group of about 1,000 volunteers, who receive no financial compensation Stable More info in debian-history.deb
Development reference Tens of thousands of packages (~25,000) Security, package and bug tracking Gigantic infrastructure  Serves as sort of a reference platform due to its adherence to Linux Standards Base, File Hierarchy Standard, etc.
Different versions Names taken from Toy Story Bruce Perens worked for Pixar and was Debian Project Leader at the time Current version: etch Next version: lenny (after that its squeeze) Scheduled for release in 2008, but could be 2009 (currently ~100 RC bugs.)
What about sid? The kid in Toy Story who broke toys This is the unstable version of debian, and will always be unstable Periodically testing gets frozen (like it is right now), release critical bugs get fixed, and then released as stable So - Stable  -> 4.0 & 4.1 code name etch Testing  -> 5.0 code name lenny Unstable  -> sid (Will never have a version)
How software gets into debian Debian uses “packages” to create a modular system The package itself is a binary format called a “deb”  This means it has to be built (compiled) on a packager's machine before it is uploaded to debian A developer finds some software, not already in debian, and files an ITP (Intend To Package) bug, then closes the bug with a new deb
Mentoring / Sponsorship If you have a package that you want to get into debian you can submit it to  http://mentors.debian.net Lots of helpful and experienced debian people to get your package in good shape for debian
First stop, unstable A package gets built from source code  The packager uploads the resulting binary deb and the pristine source code into unstable (This is a bit of a gloss) It sits in unstable for 10 days then moves to testing the package gets built on a variety of architectures Once in testing it gets frozen and tested again before release for “release critical” bugs
How do you create a deb? TIMTOWTDI, but debian has excellent tools: BTS (Bug Tracking System) PTS (Package Tracking System) Recently moved to a new server, much more responsive. dh-make-perl dpkg Clean room building with pbuilder or cowbuilder
Closer look at the tools BTS is widely used as a development tool, half-way to a million bugs Bugs very often move upstream to perl, ruby, python for example Specific types of bugs; start with RFP (Request For Packaging) to get a module into debian File an ITP (Intend To Package) bug if you are going to package a module for debian Filing packaging bugs is important because it prevents the duplication of work
Package tracking The PTS gives you an overview of every package in debian You can look through every debian version (i.e. stable, testing, unstable) You can follow bug reports, and package status changes via email from the PTS and on the web
More PTS PTS system has recently been updated Hosted in Europe by 1&1 Significant traffic: 5.7 Million unique visitors (excluding robots) 122 Million hits monthly, again excluding robots By filing an ITP bug you prevent duplicate work and confusion.
dh_make_perl Powerful perl version of dh_make Gobbles up a module from cpan (using the cpan shell)  Then builds the deb with the --build option Lots of options and potential for customization Calculates perl dependencies AND operating system dependencies
dh-makin' bacon dh-make-perl will create the files required to build a debian source package out of a perl package. This works for most simple packages and is also useful for getting started with packaging perl modules. You can build and install the debian package using the --build and --install command line switches. Using this program is no excuse for not reading the debian developer documentation, including the Debian policy, the perl policy, the packaging manual and so on.
Flow Source code dh-make Debian directory dpkg Deb
dpkg-buildpackage dpkg is also an extremely powerful build tool. It is a set of tools really Responsible for installing and removing packages, gets called by high-level tools like apt-get and aptitude dpkg-buildpackage builds your deb as well as the .dsc file you need for uploading to debian
How does dpkg know how to build the package? Because dh-make (alternatively dh-make-perl) built, and populated with boilerplate, the debian directory This directory holds files and one executable, the debian/rules script debian/rules is a Makefile which calls lots of other small debian scripts called debhelper scripts This creates a highly modular build system with lots of opportunity for specific, small changes
Pbuilder and cowbuilder Pbuilder stands for personal builder Automatically builds debian packages in a “clean-room” environment Helps to verify that a package can be built on the average debian system It creates a base chroot image, downloads and installs a complete, but minimal, debian system Cowbuilder is pbuilder sped up by using cowdancer, a copy-on-write system in userland
lintian Lintian is the debian policy enforcer Chatty, and pretty complete Being “lintian clean” is pretty much a requirement for getting your deb into debian and definitely if you want to be a debian developer
What else is in the debian dir? The debian directory also contains files which help to define the package and help in packaging. Copyright Control Changelog  Watch  Compat
Changelog This required file lists all the changes libyaml-perl (0.39-1) unstable; urgency=low * New upstream release. * New Maintainer with acknowledgment from Chip. * Added the copyright statement to debian/copyright. * Updated Standards-Version to 3.6.2. * Moved the perl build-dep to build-dep-indep (Closes: #305525). * Updated compat version to 4 and upgraded the debhelper dependency accordingly. -- Florian Ragwitz <rafl@debian.org>  Thu,  1 Dec 2005 15:33:42 +0100
More changelog The recorded changes are mostly relative to the debian packaging, not the upstream source Crucially, this is the canonical location of the package's version information Specifies which package fixed which bug (specific syntax allows for automated bug closing from the changelog)
Control Source: libpod-strip-perl Section: perl Priority: optional Build-Depends: debhelper (>= 7), libmodule-build-perl Build-Depends-Indep: perl (>= 5.6.10-12) Maintainer: Debian Perl Group <pkg-perl-maintainers@lists.alioth.debian.org> Uploaders: Damyan Ivanov <dmn@debian.org> Standards-Version: 3.8.0 Homepage: http://search.cpan.org/dist/Pod-Strip/ Vcs-Svn: svn://svn.debian.org/pkg-perl/trunk/libpod-strip-perl/ Vcs-Browser: http://svn.debian.org/wsvn/pkg-perl/trunk/libpod-strip-perl/ Package: libpod-strip-perl Architecture: all Depends: ${perl:Depends}, ${misc:Depends} Description: remove POD documentation from Perl code Pod::Strip is a subclass of Pod::Simple that removes the POD (plain old documentation) from Perl code. The POD may optionally be replaced with comments so that line numbers of the code stay the same. . Pod::Strip is useful in Perl code parsers that don't want to bother about POD.
Copyright Please, please, please, included copyright and license information in your modules on CPAN. It may not matter that much to you, it matters a lot to those who package your software for linux (at least for the Free Software distros) It may matter a lot to you one day It strengthens the license to have a copyright and it appears clear that the licenses DO have legal weight (the artistic license just won a big legal victory recently.)
Debian's copyright file Format-Specification: http://wiki.debian.org/Proposals/CopyrightFormat?action=recall&rev=196 Upstream-Maintainer: Thomas Klausner <domm@zsi.at> Upstream-Source: http://search.cpan.org/dist/Pod-Strip/ Upstream-Name: Pod-Strip Files: * Copyright: Copyright 2004, 2005, 2006 Thomas Klausner, All Rights Reserved. License-Alias: Perl License: Artistic | GPL-1+ This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
More copyright fun Files: debian/* Copyright: © 2008, Damyan Ivanov <dmn@debian.org> License: Artistic | GPL-1+ Packaging is free; you can redistribute it and/or modify it inder the same terms as Perl itself.
Licensing License: Artistic This program is free software; you can redistribute it and/or modify it under the terms of the Artistic License, which comes with Perl. On Debian GNU/Linux systems, the complete text of the Artistic License can be found in /usr/share/common-licenses/Artistic License: GPL-1+ This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by  the Free Software Foundation; either version 1, or (at your option) any later version. On Debian GNU/Linux systems, the complete text of the GNU General Public License can be found in `/usr/share/common-licenses/GPL'
Watch file # format version number, currently 3; this line is compulsory! version=3 # URL to the package page followed by a regex to search http://search.cpan.org/dist/Pod-Strip/  .*/Pod-Strip-v?(\d[\d_.-]+)\.(?:tar(?:\.gz|\.bz2)?|tgz|zip)$ Allows for an automated update of source and package when there is a new version on CPAN, via the uscan tool
Uscan This allows us to automatically update the package contents without having to rebuild the entire package scaffolding. $ uscan libspreadsheet-read-perl: Newer version (0.29) available on remote site:  http://search.cpan.org/CPAN/authors/id/H/HM/HMBRAND/Spreadsheet-Read-0.29.tgz   (local version is 0.28) libspreadsheet-read-perl: Successfully downloaded updated package Spreadsheet-Read-0.29.tgz and symlinked libspreadsheet-read-perl_0.29.orig.tar.gz to it
How do we know there is a new version on CPAN? Our QA reporting tool: http://pkg-perl.alioth.debian.org/cgi-bin/qareport.cgi Part of the debian-perl team's workflow Some of this functionality is moving towards CPANTS, maybe other automated debian-centric tests suites perhaps?
Managing packages sudo aptitude install courier-filter-perl Reading package lists... Done Building dependency tree  Reading state information... Done Reading extended state information  Initializing package states... Done Reading task descriptions... Done  The following NEW packages will be installed: courier-authdaemon{a} courier-authlib{a} courier-authlib-userdb{a} courier-base{a} courier-filter-perl courier-mta{a} expect{a} fam{a} libarchive-zip-perl{a} libclamav-client-perl{a} libfam0{a} libltdl3{a} libmail-spf-perl{a} libnet-rblclient-perl{a} libnetaddr-ip-perl{a} portmap{a} tcl8.4{a}  The following packages will be REMOVED: postfix{a}  0 packages upgraded, 17 newly installed, 1 to remove and 0 not upgraded. Need to get 4012kB of archives. After unpacking 8303kB will be used. Do you want to continue? [Y/n/?]
Implications of automated packaging systems Obsoletes debian-perl(?) Most automated tools currently come with some form of caveat stating that there is no support, i.e. Security tracking. Most likely quite useful if you are SURE no one else will ever use your deb Creation of a potentially vast invisible, distributed package repository Probably inevitable :-(
How to become a debian developer Maintain packages and understand debian's Social Contract Contribute to debian Patience Sponsorship You don't need to be a DD to contribute, only tangible difference is you get a cool  [email_address]  and you get to vote If you don't become a DD, you don't go MIA The entire process is improving and becoming faster – there are changes afoot (DM, DME, etc.) Potentially contreversial.
Resources and Thank yous http://pkg-perl.alioth.debian.org/ Integrating perl in a wider distribution: The debian-perl group, by Gunnar Wolf The Debian System, by Martin Krafft dam, gregoa, gwolf, tincho, nomeata, sukria, djpig, dom, Florian R., Gabor, Jos, Nadim, joeyh, eloy, ntyni, rra, hertzog

The Gory Details of Debian packages

  • 1.
    Debian Packages: the gory details An operetta, of sorts.
  • 2.
    Debian GNU/Linux Free,community developed operating system Always will be free because of its Social Contract and legally shielded by Software in the Public Interest Developed by a group of about 1,000 volunteers, who receive no financial compensation Stable More info in debian-history.deb
  • 3.
    Development reference Tensof thousands of packages (~25,000) Security, package and bug tracking Gigantic infrastructure Serves as sort of a reference platform due to its adherence to Linux Standards Base, File Hierarchy Standard, etc.
  • 4.
    Different versions Namestaken from Toy Story Bruce Perens worked for Pixar and was Debian Project Leader at the time Current version: etch Next version: lenny (after that its squeeze) Scheduled for release in 2008, but could be 2009 (currently ~100 RC bugs.)
  • 5.
    What about sid?The kid in Toy Story who broke toys This is the unstable version of debian, and will always be unstable Periodically testing gets frozen (like it is right now), release critical bugs get fixed, and then released as stable So - Stable -> 4.0 & 4.1 code name etch Testing -> 5.0 code name lenny Unstable -> sid (Will never have a version)
  • 6.
    How software getsinto debian Debian uses “packages” to create a modular system The package itself is a binary format called a “deb” This means it has to be built (compiled) on a packager's machine before it is uploaded to debian A developer finds some software, not already in debian, and files an ITP (Intend To Package) bug, then closes the bug with a new deb
  • 7.
    Mentoring / SponsorshipIf you have a package that you want to get into debian you can submit it to http://mentors.debian.net Lots of helpful and experienced debian people to get your package in good shape for debian
  • 8.
    First stop, unstableA package gets built from source code The packager uploads the resulting binary deb and the pristine source code into unstable (This is a bit of a gloss) It sits in unstable for 10 days then moves to testing the package gets built on a variety of architectures Once in testing it gets frozen and tested again before release for “release critical” bugs
  • 9.
    How do youcreate a deb? TIMTOWTDI, but debian has excellent tools: BTS (Bug Tracking System) PTS (Package Tracking System) Recently moved to a new server, much more responsive. dh-make-perl dpkg Clean room building with pbuilder or cowbuilder
  • 10.
    Closer look atthe tools BTS is widely used as a development tool, half-way to a million bugs Bugs very often move upstream to perl, ruby, python for example Specific types of bugs; start with RFP (Request For Packaging) to get a module into debian File an ITP (Intend To Package) bug if you are going to package a module for debian Filing packaging bugs is important because it prevents the duplication of work
  • 11.
    Package tracking ThePTS gives you an overview of every package in debian You can look through every debian version (i.e. stable, testing, unstable) You can follow bug reports, and package status changes via email from the PTS and on the web
  • 12.
    More PTS PTSsystem has recently been updated Hosted in Europe by 1&1 Significant traffic: 5.7 Million unique visitors (excluding robots) 122 Million hits monthly, again excluding robots By filing an ITP bug you prevent duplicate work and confusion.
  • 13.
    dh_make_perl Powerful perlversion of dh_make Gobbles up a module from cpan (using the cpan shell) Then builds the deb with the --build option Lots of options and potential for customization Calculates perl dependencies AND operating system dependencies
  • 14.
    dh-makin' bacon dh-make-perlwill create the files required to build a debian source package out of a perl package. This works for most simple packages and is also useful for getting started with packaging perl modules. You can build and install the debian package using the --build and --install command line switches. Using this program is no excuse for not reading the debian developer documentation, including the Debian policy, the perl policy, the packaging manual and so on.
  • 15.
    Flow Source codedh-make Debian directory dpkg Deb
  • 16.
    dpkg-buildpackage dpkg isalso an extremely powerful build tool. It is a set of tools really Responsible for installing and removing packages, gets called by high-level tools like apt-get and aptitude dpkg-buildpackage builds your deb as well as the .dsc file you need for uploading to debian
  • 17.
    How does dpkgknow how to build the package? Because dh-make (alternatively dh-make-perl) built, and populated with boilerplate, the debian directory This directory holds files and one executable, the debian/rules script debian/rules is a Makefile which calls lots of other small debian scripts called debhelper scripts This creates a highly modular build system with lots of opportunity for specific, small changes
  • 18.
    Pbuilder and cowbuilderPbuilder stands for personal builder Automatically builds debian packages in a “clean-room” environment Helps to verify that a package can be built on the average debian system It creates a base chroot image, downloads and installs a complete, but minimal, debian system Cowbuilder is pbuilder sped up by using cowdancer, a copy-on-write system in userland
  • 19.
    lintian Lintian isthe debian policy enforcer Chatty, and pretty complete Being “lintian clean” is pretty much a requirement for getting your deb into debian and definitely if you want to be a debian developer
  • 20.
    What else isin the debian dir? The debian directory also contains files which help to define the package and help in packaging. Copyright Control Changelog Watch Compat
  • 21.
    Changelog This requiredfile lists all the changes libyaml-perl (0.39-1) unstable; urgency=low * New upstream release. * New Maintainer with acknowledgment from Chip. * Added the copyright statement to debian/copyright. * Updated Standards-Version to 3.6.2. * Moved the perl build-dep to build-dep-indep (Closes: #305525). * Updated compat version to 4 and upgraded the debhelper dependency accordingly. -- Florian Ragwitz <rafl@debian.org> Thu, 1 Dec 2005 15:33:42 +0100
  • 22.
    More changelog Therecorded changes are mostly relative to the debian packaging, not the upstream source Crucially, this is the canonical location of the package's version information Specifies which package fixed which bug (specific syntax allows for automated bug closing from the changelog)
  • 23.
    Control Source: libpod-strip-perlSection: perl Priority: optional Build-Depends: debhelper (>= 7), libmodule-build-perl Build-Depends-Indep: perl (>= 5.6.10-12) Maintainer: Debian Perl Group <pkg-perl-maintainers@lists.alioth.debian.org> Uploaders: Damyan Ivanov <dmn@debian.org> Standards-Version: 3.8.0 Homepage: http://search.cpan.org/dist/Pod-Strip/ Vcs-Svn: svn://svn.debian.org/pkg-perl/trunk/libpod-strip-perl/ Vcs-Browser: http://svn.debian.org/wsvn/pkg-perl/trunk/libpod-strip-perl/ Package: libpod-strip-perl Architecture: all Depends: ${perl:Depends}, ${misc:Depends} Description: remove POD documentation from Perl code Pod::Strip is a subclass of Pod::Simple that removes the POD (plain old documentation) from Perl code. The POD may optionally be replaced with comments so that line numbers of the code stay the same. . Pod::Strip is useful in Perl code parsers that don't want to bother about POD.
  • 24.
    Copyright Please, please,please, included copyright and license information in your modules on CPAN. It may not matter that much to you, it matters a lot to those who package your software for linux (at least for the Free Software distros) It may matter a lot to you one day It strengthens the license to have a copyright and it appears clear that the licenses DO have legal weight (the artistic license just won a big legal victory recently.)
  • 25.
    Debian's copyright fileFormat-Specification: http://wiki.debian.org/Proposals/CopyrightFormat?action=recall&rev=196 Upstream-Maintainer: Thomas Klausner <domm@zsi.at> Upstream-Source: http://search.cpan.org/dist/Pod-Strip/ Upstream-Name: Pod-Strip Files: * Copyright: Copyright 2004, 2005, 2006 Thomas Klausner, All Rights Reserved. License-Alias: Perl License: Artistic | GPL-1+ This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
  • 26.
    More copyright funFiles: debian/* Copyright: © 2008, Damyan Ivanov <dmn@debian.org> License: Artistic | GPL-1+ Packaging is free; you can redistribute it and/or modify it inder the same terms as Perl itself.
  • 27.
    Licensing License: ArtisticThis program is free software; you can redistribute it and/or modify it under the terms of the Artistic License, which comes with Perl. On Debian GNU/Linux systems, the complete text of the Artistic License can be found in /usr/share/common-licenses/Artistic License: GPL-1+ This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 1, or (at your option) any later version. On Debian GNU/Linux systems, the complete text of the GNU General Public License can be found in `/usr/share/common-licenses/GPL'
  • 28.
    Watch file #format version number, currently 3; this line is compulsory! version=3 # URL to the package page followed by a regex to search http://search.cpan.org/dist/Pod-Strip/ .*/Pod-Strip-v?(\d[\d_.-]+)\.(?:tar(?:\.gz|\.bz2)?|tgz|zip)$ Allows for an automated update of source and package when there is a new version on CPAN, via the uscan tool
  • 29.
    Uscan This allowsus to automatically update the package contents without having to rebuild the entire package scaffolding. $ uscan libspreadsheet-read-perl: Newer version (0.29) available on remote site: http://search.cpan.org/CPAN/authors/id/H/HM/HMBRAND/Spreadsheet-Read-0.29.tgz (local version is 0.28) libspreadsheet-read-perl: Successfully downloaded updated package Spreadsheet-Read-0.29.tgz and symlinked libspreadsheet-read-perl_0.29.orig.tar.gz to it
  • 30.
    How do weknow there is a new version on CPAN? Our QA reporting tool: http://pkg-perl.alioth.debian.org/cgi-bin/qareport.cgi Part of the debian-perl team's workflow Some of this functionality is moving towards CPANTS, maybe other automated debian-centric tests suites perhaps?
  • 31.
    Managing packages sudoaptitude install courier-filter-perl Reading package lists... Done Building dependency tree Reading state information... Done Reading extended state information Initializing package states... Done Reading task descriptions... Done The following NEW packages will be installed: courier-authdaemon{a} courier-authlib{a} courier-authlib-userdb{a} courier-base{a} courier-filter-perl courier-mta{a} expect{a} fam{a} libarchive-zip-perl{a} libclamav-client-perl{a} libfam0{a} libltdl3{a} libmail-spf-perl{a} libnet-rblclient-perl{a} libnetaddr-ip-perl{a} portmap{a} tcl8.4{a} The following packages will be REMOVED: postfix{a} 0 packages upgraded, 17 newly installed, 1 to remove and 0 not upgraded. Need to get 4012kB of archives. After unpacking 8303kB will be used. Do you want to continue? [Y/n/?]
  • 32.
    Implications of automatedpackaging systems Obsoletes debian-perl(?) Most automated tools currently come with some form of caveat stating that there is no support, i.e. Security tracking. Most likely quite useful if you are SURE no one else will ever use your deb Creation of a potentially vast invisible, distributed package repository Probably inevitable :-(
  • 33.
    How to becomea debian developer Maintain packages and understand debian's Social Contract Contribute to debian Patience Sponsorship You don't need to be a DD to contribute, only tangible difference is you get a cool [email_address] and you get to vote If you don't become a DD, you don't go MIA The entire process is improving and becoming faster – there are changes afoot (DM, DME, etc.) Potentially contreversial.
  • 34.
    Resources and Thankyous http://pkg-perl.alioth.debian.org/ Integrating perl in a wider distribution: The debian-perl group, by Gunnar Wolf The Debian System, by Martin Krafft dam, gregoa, gwolf, tincho, nomeata, sukria, djpig, dom, Florian R., Gabor, Jos, Nadim, joeyh, eloy, ntyni, rra, hertzog