Solving the Package Problem


Published on

In the beginning there was RPM (and Debian packages) and it was good. Certainly, Linux packaging has solved many problems and pain points for system admins and developers over the years -- but as software development and deployment have evolved, new pain points have cropped up that have not been solved by traditional packaging.

In this talk, Joe Brockmeier will run through some of the problems that admins and developers have run into, and some of the solutions that organizations should be looking at to solve their issues with developing and deploying software. This includes Software Collections, Docker containers, OStree and rpm-ostree, Platform-as-a-Service, and more.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • In the beginning, all we had was source, and it was a pain to work with. But we didn't know better, and we compiled, and compiled, and compiled...
    Then there were package managers... and software installation became easier, and we looked to a New Hope...
  • For a long time, Linux distributions were – essentially – the center of the universe for getting open source software. You installed Red Hat, Debian, Fedora, or whatever and took the packages that were current because it was the easiest way to consume FOSS projects.
    It also ensured that you'd get updates and bug fixes, and life was good...
  • Surely a variation of “Murphy's Law” must be this:The system version of Python, PHP, Ruby, Perl, Node.js, or whatever is not the version that your application needs or the version you want to use to build a new application. It will invariably be too old or too new, or have some minor bug that breaks your application.
    (e.g., CloudStack and Tomcat bug...)
  • Developers have a lot of choices, not least of which is just grabbing their packages from upstream (e.g., Ruby).
    That way lies madness, and bad things.
    Containers, especially Docker, are the new hotness... but 1) not ready for prime time just yet, 2) may not be suitable for many instances, and 3) where do you get the software for Docker?
    PaaS – see above.
    Software Collections...
  • Let's say you're running Fedora 19 or CentOS 6.x and want to run a different version of LibreOffice. So you go to and decide to download 4.2.1. Then you remove the old packages and install the new packages. Yeah, that's not going to work for system software... so that's not how SCLs work.
  • Software Collections are distributed as a set of several components, which provide their full functionality without overwriting system files.
  • Software Collections make use of a special file system hierarchy to avoid possible conflicts between a single Software Collection and the base system installation.
  • To convert a conventional package to a single Software Collection, you only need to make minor changes to the package spec file.
    With a single spec file, you can build both the conventional package and the Software Collection package.
  • Solving the Package Problem

    1. 1. SSoollvviinngg tthhee PPaacckkaaggee PPrroobblleemm JJooee BBrroocckkmmeeiieerr jjzzbb@@rreeddhhaatt..ccoomm TTwwiitttteerr:: @@jjzzbb hhttttpp::////ddiissssoocciiaatteeddpprreessss..nneett// OOppeenn SSoouurrccee CCoommmmuunniittyy @@ RReedd HHaatt TTwwiitttteerr:: @@RReeddHHaattOOppeenn hhttttpp::////ccoommmmuunniittyy..rreeddhhaatt..ccoomm// 1
    2. 2. 2 WWhhoo''ss tthhiiss gguuyy??
    3. 3. 3 Solving tthhee PPaacckkaaggee PPrroobblleemm ● TThhee PPaacckkaaggiinngg PPrroobblleemm WWee FFaaccee ● SSooffttwwaarree CCoolllleeccttiioonnss ● rrppmm--oossttrreeee ● DDoocckkaahh,, DDoocckkaahh,, DDoocckkaahh ● QQuueessttiioonnss
    4. 4. 4 IInn tthhee BBeeggiinnnniinngg......
    5. 5. 5 DDiissttrriibbuuttiioonnss aass tthhee CCeenntteerr ooff tthhee UUnniivveerrssee
    6. 6. 6 Developers ddoo nnoott wwaanntt ttoo bbee lliimmiitteedd ttoo ssyysstteemm vveerrssiioonnss ooff ssooffttwwaarree
    7. 7. 7 Developers wwaanntt eeaassiieerr wwaayyss ttoo ddeeppllooyy ccoommpplleexx ssooffttwwaarree ffrroomm ddeesskkttoopp ttoo sseerrvveerr
    8. 8. 8 AAuuttoommaattee AALLLL TTHHEE TTHHIINNGGSS
    9. 9. 9 Software Collections, rrppmm--oossttrreeee,, aanndd DDoocckkeerr ((oohh mmyy))
    10. 10. Let's talk about SSooffttwwaarree CCoolllleeccttiioonnss SCALE 12x
    11. 11. 11 Do not rreeqquuiirree cchhaannggeess ttoo RRPPMM
    12. 12. SSooffttwwaarree CCoolllleeccttiioonnss aarree nnoott jjuusstt aa ddiiffffeerreenntt 12 vveerrssiioonn ppaacckkaaggeedd ffoorr yyoouurr OOSS
    13. 13. 13 Do not oovveerrwwrriittee ssyysstteemm ffiilleess
    14. 14. Example: PHP 5.4 OOnn CCeennttOOSS 66..xx PPHHPP 55..44 ppaacckkaaggee iiss php54 TThhiiss ppuullllss iinn:: php54­php­cli. x86_64 php54­php­common. x86_64 php54­php­pear. noarch php54­php­process. x86_64 php54­php­xml. x86_64 php54­runtime. x86_64 LLiivveess iinn:: /opt/rh/php54/root 14
    15. 15. 15 Avoid ccoonnfflliiccttss wwiitthh ssyysstteemm ffiilleess
    16. 16. 16 RReeqquuiirree mmiinnoorr cchhaannggeess ttoo yyoouurr eexxiissttiinngg ssppeecc ffiilleess
    17. 17. Do not conflict wwiitthh uuppddaatteess oonn yyoouurr ssyysstteemm 17
    18. 18. 18 NNiiffttyy:: CCaann ddeeppeenndd oonn ootthheerr SSCCLLss
    19. 19. Let's ttaallkk aabboouutt uussiinngg SSCCLLss SCALE 12x
    20. 20. 20 SSooffttwwaarree CCoolllleeccttiioonnss CCuurrrreennttllyy ● Red Hat Software Collections went GA on 12 Sept 2013 ● CentOS announced SCLs 19 February 2014 ● Fedora has SCLs on Fedora Hosted since 2013:
    21. 21. 21 What's Packaged ffoorr SSooffttwwaarree CCoolllleeccttiioonnss ● Ruby 1.9.3 ● Python 2.7 ● Python 3.3 ● PHP 5.4 ● Perl 5.16.3 ● Tech preview Node.js 0.10 ● MariaDB 5.5 ● MySQL 5.5 ● PostgreSQL 9.2
    22. 22. 22 GGeettttiinngg SSttaarrtteedd ● Assuming using a SLC with CentOS ● yum install centos­release­SCL ● yum install php54 (or whatever...) ● scl enable php54 “application ­­option” ● Your application now uses PHP 5.4 ... the rest of the system ignores it. ● Python & Django with SCL (by Langdon White): ● ● Find packages for CentOS here:
    23. 23. 23 Packaging SCLs ● Grab the necessary ppaacckkaaggeess ((CCeennttOOSS oorr FFeeddoorraa oorr RRHHEELL 66..55)):: yum install scl­utils scl­utils­build ● IInnssttrruuccttiioonnss oonn ccoonnvveerrttiinngg aann eexxiissttiinngg ppaacckkaaggee::­spec­file ● FFoorr CCoonnvveerrssiioonn:: spec2scl ● GGeenneerraall iinnssttrruuccttiioonnss oonn ppaacckkaaggiinngg SSCCLLss::­scls
    24. 24. SSooffttwwaarreeCCoolllleeccttiioonnss..oorrgg SCALE 12x
    25. 25. 25 What is ● Upstream community for development of SCLs. ● Build and hosting services for collections. ● Resources (documentation, forums, mailing lists) for developers/packagers. ● An index of packaged software for users of CentOS, Fedora, RHEL, and other RPM-based distributions.
    26. 26. 26 The Lifecycle of Collections ● SCLs can be used to provide newer software support on older releases, or (going forward) to provide legacy support on newer releases: ● Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7 (when released) using SCL. ● OpenShift leverages SCLs for its cartridges using RHEL supported and non-supported components. ● SCLs can be used inside Docker containers to simplify container deployment.
    27. 27. rrppmm--oossttrreeee SCALE 12x
    28. 28. 28 The Problem with Packages ● RPM (and dpkg) are designed to go one way: forward ● Upgrades are difficult to “roll back” in the event something goes wrong ● Switching between two distinct OSes / versions is more or less impossible
    29. 29. 29 What is rpm-ostree? ● Derived from ostree ● Initially conceived of as a way to parallel install multiple UNIX-like OSes (e.g., Fedora Rawhide and Fedora 20) ● “git for operating system binaries” ● Creates an installable tree from RPMs ● Not a package manager, but does take on some of the role from package managers
    30. 30. 30 What rpm-ostree Enables ● Install one or more operating system trees to a system ● Gives “atomic” updates ● An update is, essentially, one unit – it succeeds or fails ● An update can be rolled back ● Allows switching between “trees” ● Provides tools for creating tree composes
    31. 31. 31 Current Limitations ● Currently, an rpm-ostree “tree” is an immutable system ● Doesn't allow for adding packages to a system w/out rebuilding the tree ● Build tools are still being developed, but moving quickly
    32. 32. So, anybody heard ooff tthhiiss DDoocckkeerr tthhiinngg?? SCALE 12x
    33. 33. 33 The Problem with Packages ● Deploying complex services / applications is difficult with packages ● Packages aren't as portable as we'd like ● Application is developed on CentOS 6, but production is using CentOS 7? ● Packaging guidelines can be ... difficult ● Packages don't provide any solution for running containerized applications...
    34. 34. 34 Docker: It's Like Deluxe Super Awesome Packaging ● Docker is application-centric ● Docker containers are portable ● Supports versioning for an entire container ● Components can be re-used ● Allows for supplying ready-to-run services rather than half-configured packages ● Buzzword compliant
    35. 35. 35 Docker to the Rescue? ● Docker containers: relatively easy to work with ● Provide far more “services” than package systems ● Application isolation ● Image format, sharing, API ● Allows “layering” of applications ● One group provides base image ● Another group provides base image + framework/service (e.g., Apache) ● Another group provides base image + framework + finished application ready to deploy
    36. 36. TThhaannkkss!! JJooee BBrroocckkmmeeiieerr jjzzbb@@rreeddhhaatt.ccoomm TTwwiitttteerr:: @@jjzzbb