Distro Recipes 2013 : Debian and quality assurance
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Distro Recipes 2013 : Debian and quality assurance

on

  • 2,491 views

https://distro-recipes.org

https://distro-recipes.org

Statistics

Views

Total Views
2,491
Views on SlideShare
2,491
Embed Views
0

Actions

Likes
1
Downloads
6
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Distro Recipes 2013 : Debian and quality assurance Presentation Transcript

  • 1. Debian Quality Assurance Lucas Nussbaum lucas@debian.org lucas@debian.org Debian Quality Assurance 1 / 31
  • 2. Me Assistant professor (Maître de conférences) at Univ. de Lorraine Teach Free Software in a system administration curriculum (Licence Professionnelle Administration Système, Réseaux et Applications à base de Logiciel Libre) Debian involvement: Collaboration between Debian and Ubuntu Ruby packaging Archive rebuilds Ultimate Debian Database lucas@debian.org Debian Quality Assurance 2 / 31
  • 3. DebianWell known and respected for: lucas@debian.org Debian Quality Assurance 3 / 31
  • 4. DebianWell known and respected for: Its long and bumpy release cycles lucas@debian.org Debian Quality Assurance 3 / 31
  • 5. DebianWell known and respected for: Its long and bumpy release cycles Its qualityBut it was not a given: > 1000 Debian Developers Mostly volunteers, with their own agendas > 2000 Package maintainers 18117 software packages Some very popular packages A lot of niche packages lucas@debian.org Debian Quality Assurance 3 / 31
  • 6. Quality factors lucas@debian.org Debian Quality Assurance 4 / 31
  • 7. Quality factors Culture: "We release when it’s ready " Many DDs won’t compromise on this lucas@debian.org Debian Quality Assurance 4 / 31
  • 8. Quality factors Culture: "We release when it’s ready " Many DDs won’t compromise on this Packages ownership: individuals or small teams Many of the maintainers are experts of their packages lucas@debian.org Debian Quality Assurance 4 / 31
  • 9. Quality factors Culture: "We release when it’s ready " Many DDs won’t compromise on this Packages ownership: individuals or small teams Many of the maintainers are experts of their packages Release Critical (RC) bugs Prevent packages from being part of a release Everybody is welcomed to fix them Non-Maintainer Uploads lucas@debian.org Debian Quality Assurance 4 / 31
  • 10. Quality factors Culture: "We release when it’s ready " Many DDs won’t compromise on this Packages ownership: individuals or small teams Many of the maintainers are experts of their packages Release Critical (RC) bugs Prevent packages from being part of a release Everybody is welcomed to fix them Non-Maintainer Uploads Debian Quality Assurance team lucas@debian.org Debian Quality Assurance 4 / 31
  • 11. Debian QA team Goal: Improve the quality of Debian as a whole Not really a team (no strict membership), more like a central place to discuss and work on QA IRC: #debian-qa@irc.debian.org Mailing list: debian-qa@lists.debian.org lucas@debian.org Debian Quality Assurance 5 / 31
  • 12. Debian QA team: Tasks Run archive-wide checks and mass bug filings Maintain QA infrastructure Take care of the dirty areas of Debian lucas@debian.org Debian Quality Assurance 6 / 31
  • 13. Archive-wide checksand Mass bug filings lucas@debian.org Debian Quality Assurance 7 / 31
  • 14. Archive-wide checks and MBF Developers are volunteers: Focus on interesting things Manual testing is boring → not done Need automated tests Other advantage: treats all packages equally Niche packages are very important to some users lucas@debian.org Debian Quality Assurance 8 / 31
  • 15. Archive rebuilds Rebuild all packages from source 7 hours on 40 build nodes (using Amazon EC2) Detect packages that Fail To Build From Source (FTBFS) Bugs in packages Toolchain bugs Failures caused by changes in other packages Ran every 2-3 weeks Scripts to analyze failures and file bugs semi-automatically (about 200 bugs filed per hour) Also used for test rebuilds with new compilers, linkers, . . . → Valuable information for upstream lucas@debian.org Debian Quality Assurance 9 / 31
  • 16. Debcheck Check (statically) that dependencies can be satisfied Not trivial: versioned dependencies, alternate depends, conflicts 2 versions: "Simple one", in PHP http://qa.debian.org/debcheck.php Better one: edos-debcheck (EDOS project) Can also work with RPMs http://edos.debian.net/ lucas@debian.org Debian Quality Assurance 10 / 31
  • 17. Piuparts Testing package installation, upgrade and removal Find bugs in maintainer scripts (Shell) scripts executed during package installation/removal Developers don’t run it on their systems → Archive-wide setup http://piuparts.debian.org lucas@debian.org Debian Quality Assurance 11 / 31
  • 18. Lintian Static analysis of packages Usually run locally by the developer, before uploading Archive-wide setup for continuous testing Packages now rejected at upload time for some errors Some example findings: Use of deprecated packaging features Common packaging mistakes Embedded copies of libraries Spelling errors in binaries usefull visiters treshold targetted superseeded dependancies splitted maintainance explicitely equiped informations mispelled http://lintian.debian.org lucas@debian.org Debian Quality Assurance 12 / 31
  • 19. DEHS - Debian External Health Status Monitor upstream version of packages Detect when the Debian package is outdated HTML scraping + regular expressions Configured in debian/watch lucas@debian.org Debian Quality Assurance 13 / 31
  • 20. And more . . . Scan build logs for dangerous warnings Track security issues Track popularity of packages ... lucas@debian.org Debian Quality Assurance 14 / 31
  • 21. Maintaining infrastructure lucas@debian.org Debian Quality Assurance 15 / 31
  • 22. Infrastructure: mitigating the data hell Debian: aggregation of loosely-connected services No Launchpad! Everyone can start their own service Data everywhere A dozen of places to get interesting data Tools required to gather data in a central place and expose it to the maintainers lucas@debian.org Debian Quality Assurance 16 / 31
  • 23. Package Tracking System (PTS) Source package-centric view http://packages.qa.debian.org/package lucas@debian.org Debian Quality Assurance 17 / 31
  • 24. Developer Packages Overview (DDPO) Developer-centric view http://qa.debian.org/developer.php lucas@debian.org Debian Quality Assurance 18 / 31
  • 25. Ultimate Debian Database (UDD) Gather everything into a single SQL DB Perfect tool for data-mining DebianCurrently imported in UDD:Debian Sources and Packages, bugs, popcon, testing migrations, upload history,orphaned packages, carnivore, lintian, debtags, translations, NEW queue,screenshots, DEHS, ldap, wanna-build, removals, Ubuntu Sources and Packages,Ubuntu bugs, Ubuntu popconPossible questions you can answer with UDD: RC buggy packages in testing, sorted by popcon? Packages for which the last 4 uploads were NMUs? Packages not maintained by official DDs? http://udd.debian.org/ lucas@debian.org Debian Quality Assurance 19 / 31
  • 26. Some UDD queries Packages with the highest number of different lintian errors or warningsselect package, count(distinct tag) as cntfrom lintianwhere tag_type in(’error’,’warning’)group by packageorder by cnt desc limit 15; lucas@debian.org Debian Quality Assurance 20 / 31
  • 27. Some UDD queries Packages with the highest number of different lintian errors or warningsselect package, count(distinct tag) as cntfrom lintianwhere tag_type in(’error’,’warning’)group by packageorder by cnt desc limit 15; package count tsocks 18 pygopherd 18 cyclades-serial-client 17 muddleftpd 17 elib 17 lucas@debian.org Debian Quality Assurance 20 / 31
  • 28. Some UDD queries Who uploaded wheezy’s packages?select changed_by, count(*) from sources s, upload_history uhwhere s.source = uh.source and s.version = uh.versionand s.distribution=’debian’ and s.release = ’wheezy’group by changed_by order by count desc limit 8; lucas@debian.org Debian Quality Assurance 21 / 31
  • 29. Some UDD queries Who uploaded wheezy’s packages?select changed_by, count(*) from sources s, upload_history uhwhere s.source = uh.source and s.version = uh.versionand s.distribution=’debian’ and s.release = ’wheezy’group by changed_by order by count desc limit 8; changed_by count gregor herrmann <gregoa@debian.org> 790 Joachim Breitner <nomeata@debian.org> 237 Christian Perrier <bubulle@debian.org> 229 Alessio Treglia <alessio@debian.org> 222 Salvatore Bonaccorso <carnil@debian.org> 207 Clint Adams <clint@debian.org> 206 Jonas Smedegaard <dr@jones.dk> 195 lucas@debian.org Debian Quality Assurance 21 / 31
  • 30. Taking care of the dirty areas of Debian lucas@debian.org Debian Quality Assurance 22 / 31
  • 31. Taking care of the dirty areas of DebianTwo complementary approaches: Focus on packages neglected by their maintainers Focus on maintainers neglecting their packages lucas@debian.org Debian Quality Assurance 23 / 31
  • 32. Neglected packages Already orphaned: Do minimalistic maintenance Problem: > 700 orphaned packages Not (yet) orphaned: Find them, orphan or remove them Goal: Put maintenance in the hands of people with time lucas@debian.org Debian Quality Assurance 24 / 31
  • 33. Bapase: finding neglected packages Multi-criteria search for neglected packages Based on Ultimate Debian Database http://udd.debian.org/bapase.cgi lucas@debian.org Debian Quality Assurance 25 / 31
  • 34. Inactive maintainers Missing In Action (MIA) team Find them, track them, orphan their packages Mostly based on reporting by users or fellow developers Echelon: records activity of DDs ; Carnivore: tracks identities Collaboration with the Account Managers: possible removal of their account (Unused accounts → possible security problems) lucas@debian.org Debian Quality Assurance 26 / 31
  • 35. Open challenges:following Debian evolutions lucas@debian.org Debian Quality Assurance 27 / 31
  • 36. Co-maintenance lucas@debian.org Debian Quality Assurance 28 / 31
  • 37. Use of VCSes lucas@debian.org Debian Quality Assurance 29 / 31
  • 38. Following Debian evolutionsNeed to improve QA infrastructure Team-aware VCS-aware Abstraction layer? If I’ve learned anything, it’s that OSS projects can’t make a choice. And when you can’t make a choice, what do you do? You support all options! Yay! How do we support all options? An abstraction layer! Yay! Abstraction layers are fun to write! Yay! – – Linux Hater’s Blog Standard workflows? Keep track of sponsorship requests http://mentors.debian.org + sponsorship-requests Track state of packages in VCS Partial solution: PET (http://pet.debian.net/) lucas@debian.org Debian Quality Assurance 30 / 31
  • 39. ConclusionsDebian: A large-scale volunteer-based distribution A continuous effort to organize development and improve quality lucas@debian.org Debian Quality Assurance 31 / 31