The Gory Details of Debian packages

  • 4,784 views
Uploaded on

This is the presentation I gave at FSCons in 2008

This is the presentation I gave at FSCons in 2008

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
4,784
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
33
Comments
1
Likes
5

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Debian Packages: the gory details
      • An operetta, of sorts.
  • 2. Debian GNU/Linux
    • Free, community developed operating system
    • Always will be free because of its Social Contract and legally shielded by Software in the Public Interest
    • Developed by a group of about 1,000 volunteers, who receive no financial compensation
    • Stable
      • More info in debian-history.deb
  • 3. Development reference
    • Tens of thousands of packages (~25,000)
    • Security, package and bug tracking
    • Gigantic infrastructure
    • Serves as sort of a reference platform due to its adherence to Linux Standards Base, File Hierarchy Standard, etc.
  • 4. Different versions
    • Names taken from Toy Story
    • Bruce Perens worked for Pixar and was Debian Project Leader at the time
    • Current version: etch
    • Next version: lenny (after that its squeeze)
    • Scheduled for release in 2008, but could be 2009 (currently ~100 RC bugs.)
  • 5. What about sid?
    • The kid in Toy Story who broke toys
    • This is the unstable version of debian, and will always be unstable
    • Periodically testing gets frozen (like it is right now), release critical bugs get fixed, and then released as stable
    • So -
      • Stable -> 4.0 & 4.1 code name etch
      • Testing -> 5.0 code name lenny
      • Unstable -> sid (Will never have a version)
  • 6. How software gets into debian
    • Debian uses “packages” to create a modular system
    • The package itself is a binary format called a “deb”
    • This means it has to be built (compiled) on a packager's machine before it is uploaded to debian
    • A developer finds some software, not already in debian, and files an ITP (Intend To Package) bug, then closes the bug with a new deb
  • 7. Mentoring / Sponsorship
    • If you have a package that you want to get into debian you can submit it to http://mentors.debian.net
    • Lots of helpful and experienced debian people to get your package in good shape for debian
  • 8. First stop, unstable
    • A package gets built from source code
    • The packager uploads the resulting binary deb and the pristine source code into unstable (This is a bit of a gloss)
    • It sits in unstable for 10 days then moves to testing the package gets built on a variety of architectures
    • Once in testing it gets frozen and tested again before release for “release critical” bugs
  • 9. How do you create a deb?
    • TIMTOWTDI, but debian has excellent tools:
      • BTS (Bug Tracking System)
      • PTS (Package Tracking System)
        • Recently moved to a new server, much more responsive.
      • dh-make-perl
      • dpkg
      • Clean room building with pbuilder or cowbuilder
  • 10. Closer look at the tools
    • BTS is widely used as a development tool, half-way to a million bugs
    • Bugs very often move upstream to perl, ruby, python for example
    • Specific types of bugs; start with RFP (Request For Packaging) to get a module into debian
    • File an ITP (Intend To Package) bug if you are going to package a module for debian
    • Filing packaging bugs is important because it prevents the duplication of work
  • 11. Package tracking
    • The PTS gives you an overview of every package in debian
    • You can look through every debian version (i.e. stable, testing, unstable)
    • You can follow bug reports, and package status changes via email from the PTS and on the web
  • 12. More PTS
    • PTS system has recently been updated
    • Hosted in Europe by 1&1
    • Significant traffic: 5.7 Million unique visitors (excluding robots)
    • 122 Million hits monthly, again excluding robots
    • By filing an ITP bug you prevent duplicate work and confusion.
  • 13. dh_make_perl
    • Powerful perl version of dh_make
    • Gobbles up a module from cpan (using the cpan shell)
    • Then builds the deb with the --build option
    • Lots of options and potential for customization
    • Calculates perl dependencies AND operating system dependencies
  • 14. dh-makin' bacon
    • dh-make-perl will create the files required to build a debian source package out of a perl package.
    • This works for most simple packages and is also useful for getting started with packaging perl modules.
    • You can build and install the debian package using the --build and --install command line switches.
    • Using this program is no excuse for not reading the debian developer documentation, including the Debian policy, the perl policy, the packaging manual and so on.
  • 15. Flow Source code dh-make Debian directory dpkg Deb
  • 16. dpkg-buildpackage
    • dpkg is also an extremely powerful build tool. It is a set of tools really
    • Responsible for installing and removing packages, gets called by high-level tools like apt-get and aptitude
    • dpkg-buildpackage builds your deb as well as the .dsc file you need for uploading to debian
  • 17. How does dpkg know how to build the package?
    • Because dh-make (alternatively dh-make-perl) built, and populated with boilerplate, the debian directory
    • This directory holds files and one executable, the debian/rules script
    • debian/rules is a Makefile which calls lots of other small debian scripts called debhelper scripts
    • This creates a highly modular build system with lots of opportunity for specific, small changes
  • 18. Pbuilder and cowbuilder
    • Pbuilder stands for personal builder
    • Automatically builds debian packages in a “clean-room” environment
    • Helps to verify that a package can be built on the average debian system
    • It creates a base chroot image, downloads and installs a complete, but minimal, debian system
    • Cowbuilder is pbuilder sped up by using cowdancer, a copy-on-write system in userland
  • 19. lintian
    • Lintian is the debian policy enforcer
    • Chatty, and pretty complete
    • Being “lintian clean” is pretty much a requirement for getting your deb into debian and definitely if you want to be a debian developer
  • 20. What else is in the debian dir?
    • The debian directory also contains files which help to define the package and help in packaging.
      • Copyright
      • Control
      • Changelog
      • Watch
      • Compat
  • 21. Changelog
    • This required file lists all the changes
    • libyaml-perl (0.39-1) unstable; urgency=low
    • * New upstream release.
    • * New Maintainer with acknowledgment from Chip.
    • * Added the copyright statement to debian/copyright.
    • * Updated Standards-Version to 3.6.2.
    • * Moved the perl build-dep to build-dep-indep (Closes: #305525).
    • * Updated compat version to 4 and upgraded the debhelper dependency
    • accordingly.
    • -- Florian Ragwitz <rafl@debian.org> Thu, 1 Dec 2005 15:33:42 +0100
  • 22. More changelog
    • The recorded changes are mostly relative to the debian packaging, not the upstream source
    • Crucially, this is the canonical location of the package's version information
    • Specifies which package fixed which bug (specific syntax allows for automated bug closing from the changelog)
  • 23. Control
    • Source: libpod-strip-perl
    • Section: perl
    • Priority: optional
    • Build-Depends: debhelper (>= 7), libmodule-build-perl
    • Build-Depends-Indep: perl (>= 5.6.10-12)
    • Maintainer: Debian Perl Group <pkg-perl-maintainers@lists.alioth.debian.org>
    • Uploaders: Damyan Ivanov <dmn@debian.org>
    • Standards-Version: 3.8.0
    • Homepage: http://search.cpan.org/dist/Pod-Strip/
    • Vcs-Svn: svn://svn.debian.org/pkg-perl/trunk/libpod-strip-perl/
    • Vcs-Browser: http://svn.debian.org/wsvn/pkg-perl/trunk/libpod-strip-perl/
    • Package: libpod-strip-perl
    • Architecture: all
    • Depends: ${perl:Depends}, ${misc:Depends}
    • Description: remove POD documentation from Perl code
    • Pod::Strip is a subclass of Pod::Simple that removes the POD (plain old
    • documentation) from Perl code. The POD may optionally be replaced with
    • comments so that line numbers of the code stay the same.
    • .
    • Pod::Strip is useful in Perl code parsers that don't want to bother about POD.
  • 24. Copyright
    • Please, please, please, included copyright and license information in your modules on CPAN.
    • It may not matter that much to you, it matters a lot to those who package your software for linux (at least for the Free Software distros)
    • It may matter a lot to you one day
    • It strengthens the license to have a copyright and it appears clear that the licenses DO have legal weight (the artistic license just won a big legal victory recently.)
  • 25. Debian's copyright file
    • Format-Specification: http://wiki.debian.org/Proposals/CopyrightFormat?action=recall&rev=196
    • Upstream-Maintainer: Thomas Klausner <domm@zsi.at>
    • Upstream-Source: http://search.cpan.org/dist/Pod-Strip/
    • Upstream-Name: Pod-Strip
    • Files: *
    • Copyright: Copyright 2004, 2005, 2006 Thomas Klausner, All Rights Reserved.
    • License-Alias: Perl
    • License: Artistic | GPL-1+
    • This program is free software; you can redistribute it and/or modify it under
    • the same terms as Perl itself.
  • 26. More copyright fun
    • Files: debian/*
    • Copyright: © 2008, Damyan Ivanov <dmn@debian.org>
    • License: Artistic | GPL-1+
    • Packaging is free; you can redistribute it and/or modify it inder the same terms as Perl itself.
  • 27. Licensing
    • License: Artistic
    • This program is free software; you can redistribute it and/or modify
    • it under the terms of the Artistic License, which comes with Perl.
    • On Debian GNU/Linux systems, the complete text of the Artistic License
    • can be found in /usr/share/common-licenses/Artistic
    • License: GPL-1+
    • This program is free software; you can redistribute it and/or modify
    • it under the terms of the GNU General Public License as published by
    • the Free Software Foundation; either version 1, or (at your option)
    • any later version.
    • On Debian GNU/Linux systems, the complete text of the GNU General
    • Public License can be found in `/usr/share/common-licenses/GPL'
  • 28. Watch file
    • # format version number, currently 3; this line is compulsory!
    • version=3
    • # URL to the package page followed by a regex to search
    • http://search.cpan.org/dist/Pod-Strip/ .*/Pod-Strip-v?(d[d_.-]+).(?:tar(?:.gz|.bz2)?|tgz|zip)$
    • Allows for an automated update of source and package when there is a new version on CPAN, via the uscan tool
  • 29. Uscan
    • This allows us to automatically update the package contents without having to rebuild the entire package scaffolding. $ uscan libspreadsheet-read-perl: Newer version (0.29) available on remote site: http://search.cpan.org/CPAN/authors/id/H/HM/HMBRAND/Spreadsheet-Read-0.29.tgz (local version is 0.28) libspreadsheet-read-perl: Successfully downloaded updated package Spreadsheet-Read-0.29.tgz and symlinked libspreadsheet-read-perl_0.29.orig.tar.gz to it
  • 30. How do we know there is a new version on CPAN?
    • Our QA reporting tool:
    • http://pkg-perl.alioth.debian.org/cgi-bin/qareport.cgi
    • Part of the debian-perl team's workflow
    • Some of this functionality is moving towards CPANTS, maybe other automated debian-centric tests suites perhaps?
  • 31. Managing packages
    • sudo aptitude install courier-filter-perl
    • Reading package lists... Done
    • Building dependency tree
    • Reading state information... Done
    • Reading extended state information
    • Initializing package states... Done
    • Reading task descriptions... Done
    • The following NEW packages will be installed:
    • courier-authdaemon{a} courier-authlib{a} courier-authlib-userdb{a} courier-base{a} courier-filter-perl courier-mta{a} expect{a} fam{a} libarchive-zip-perl{a} libclamav-client-perl{a} libfam0{a} libltdl3{a} libmail-spf-perl{a} libnet-rblclient-perl{a} libnetaddr-ip-perl{a} portmap{a} tcl8.4{a}
    • The following packages will be REMOVED:
    • postfix{a}
    • 0 packages upgraded, 17 newly installed, 1 to remove and 0 not upgraded.
    • Need to get 4012kB of archives. After unpacking 8303kB will be used.
    • Do you want to continue? [Y/n/?]
  • 32. Implications of automated packaging systems
    • Obsoletes debian-perl(?)
    • Most automated tools currently come with some form of caveat stating that there is no support, i.e. Security tracking.
    • Most likely quite useful if you are SURE no one else will ever use your deb
    • Creation of a potentially vast invisible, distributed package repository
    • Probably inevitable :-(
  • 33. How to become a debian developer
    • Maintain packages and understand debian's Social Contract
    • Contribute to debian
    • Patience
    • Sponsorship
    • You don't need to be a DD to contribute, only tangible difference is you get a cool [email_address] and you get to vote
    • If you don't become a DD, you don't go MIA
    • The entire process is improving and becoming faster – there are changes afoot (DM, DME, etc.) Potentially contreversial.
  • 34. Resources and Thank yous
    • http://pkg-perl.alioth.debian.org/
    • Integrating perl in a wider distribution: The debian-perl group, by Gunnar Wolf
    • The Debian System, by Martin Krafft
    • dam, gregoa, gwolf, tincho, nomeata, sukria, djpig, dom, Florian R., Gabor, Jos, Nadim, joeyh, eloy, ntyni, rra, hertzog