Downloaded 16 times
Uploaded byJoe Brockmeier
Solving the Package Problem
The document discusses the challenges of packaging software and presents solutions like Red Hat Software Collections (SCL) and rpm-ostree for managing software versions and deployments. It highlights the need for easier deployment of complex software and introduces Docker as a more effective alternative for application packaging and isolation. Docker provides portability, versioning, and ready-to-run services, streamlining the process of deploying applications across different environments.
More Related Content
Solving the Package Problem
- 1. SSoollvviinngg tthhee PPaacckkaaggeePPrroobblleemm JJooee BBrroocckkmmeeiieerr jjzzbb@@rreeddhhaatt..ccoomm TTwwiitttteerr:: @@jjzzbb hhttttpp::////ddiissssoocciiaatteeddpprreessss..nneett// OOppeenn SSoouurrccee CCoommmmuunniittyy @@ RReedd HHaatt TTwwiitttteerr:: @@RReeddHHaattOOppeenn hhttttpp::////ccoommmmuunniittyy..rreeddhhaatt..ccoomm// 1
- 2. 2 WWhhoo''ss tthhiissgguuyy??
- 3. 3 Solving tthheePPaacckkaaggee PPrroobblleemm ● TThhee PPaacckkaaggiinngg PPrroobblleemm WWee FFaaccee ● SSooffttwwaarree CCoolllleeccttiioonnss ● rrppmm--oossttrreeee ● DDoocckkaahh,, DDoocckkaahh,, DDoocckkaahh ● QQuueessttiioonnss
- 4. 4 IInn tthheeBBeeggiinnnniinngg......
- 5. 5 DDiissttrriibbuuttiioonnss aasstthhee CCeenntteerr ooff tthhee UUnniivveerrssee
- 6. 6 Developers ddoonnoott wwaanntt ttoo bbee lliimmiitteedd ttoo ssyysstteemm vveerrssiioonnss ooff ssooffttwwaarree
- 7. 7 Developers wwaanntteeaassiieerr wwaayyss ttoo ddeeppllooyy ccoommpplleexx ssooffttwwaarree ffrroomm ddeesskkttoopp ttoo sseerrvveerr
- 8. 8 AAuuttoommaattee AALLLLTTHHEE TTHHIINNGGSS
- 9. 9 Software Collections,rrppmm--oossttrreeee,, aanndd DDoocckkeerr ((oohh mmyy))
- 10. Let's talk aboutSSooffttwwaarree CCoolllleeccttiioonnss SCALE 12x
- 11. 11 Do notrreeqquuiirree cchhaannggeess ttoo RRPPMM
- 12. SSooffttwwaarree CCoolllleeccttiioonnss aarreennoott jjuusstt aa ddiiffffeerreenntt 12 vveerrssiioonn ppaacckkaaggeedd ffoorr yyoouurr OOSS
- 13. 13 Do notoovveerrwwrriittee ssyysstteemm ffiilleess
- 14. Example: PHP 5.4 OOnn CCeennttOOSS 66..xx PPHHPP 55..44 ppaacckkaaggee iiss php54 TThhiiss ppuullllss iinn:: php54phpcli. x86_64 php54phpcommon. x86_64 php54phppear. noarch php54phpprocess. x86_64 php54phpxml. x86_64 php54runtime. x86_64 LLiivveess iinn:: /opt/rh/php54/root 14
- 15. 15 Avoid ccoonnfflliiccttsswwiitthh ssyysstteemm ffiilleess
- 16. 16 RReeqquuiirree mmiinnoorrcchhaannggeess ttoo yyoouurr eexxiissttiinngg ssppeecc ffiilleess
- 17. Do not conflictwwiitthh uuppddaatteess oonn yyoouurr ssyysstteemm 17
- 18. 18 NNiiffttyy:: CCaannddeeppeenndd oonn ootthheerr SSCCLLss
- 19. Let's ttaallkk aabboouuttuussiinngg SSCCLLss SCALE 12x
- 20. 20 SSooffttwwaarree CCoolllleeccttiioonnssCCuurrrreennttllyy ● Red Hat Software Collections went GA on 12 Sept 2013 ● CentOS announced SCLs 19 February 2014 ● Fedora has SCLs on Fedora Hosted since 2013: https://fedorahosted.org/SoftwareCollections/
- 21. 21 What's Packagedffoorr SSooffttwwaarree CCoolllleeccttiioonnss ● Ruby 1.9.3 ● Python 2.7 ● Python 3.3 ● PHP 5.4 ● Perl 5.16.3 ● Tech preview Node.js 0.10 ● MariaDB 5.5 ● MySQL 5.5 ● PostgreSQL 9.2
- 22. 22 GGeettttiinngg SSttaarrtteedd ● Assuming using a SLC with CentOS ● yum install centosreleaseSCL ● yum install php54 (or whatever...) ● scl enable php54 “application option” ● Your application now uses PHP 5.4 ... the rest of the system ignores it. ● Python & Django with SCL (by Langdon White): ● http://red.ht/scldjango ● Find packages for CentOS here: http://mirror.metrocast.net/centos/6.5/SCL/x86_64/
- 23. 23 Packaging SCLs ● Grab the necessary ppaacckkaaggeess ((CCeennttOOSS oorr FFeeddoorraa oorr RRHHEELL 66..55)):: yum install sclutils sclutilsbuild ● IInnssttrruuccttiioonnss oonn ccoonnvveerrttiinngg aann eexxiissttiinngg ppaacckkaaggee:: http://bit.ly/sclspecfile ● FFoorr CCoonnvveerrssiioonn:: spec2scl ● GGeenneerraall iinnssttrruuccttiioonnss oonn ppaacckkaaggiinngg SSCCLLss:: http://bit.ly/pkgingscls
- 24.
- 25. 25 What isSoftwareCollections.org? ● Upstream community for development of SCLs. ● Build and hosting services for collections. ● Resources (documentation, forums, mailing lists) for developers/packagers. ● An index of packaged software for users of CentOS, Fedora, RHEL, and other RPM-based distributions.
- 26. 26 The Lifecycleof Collections ● SCLs can be used to provide newer software support on older releases, or (going forward) to provide legacy support on newer releases: ● Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7 (when released) using SCL. ● OpenShift leverages SCLs for its cartridges using RHEL supported and non-supported components. ● SCLs can be used inside Docker containers to simplify container deployment.
- 27.
- 28. 28 The Problemwith Packages ● RPM (and dpkg) are designed to go one way: forward ● Upgrades are difficult to “roll back” in the event something goes wrong ● Switching between two distinct OSes / versions is more or less impossible
- 29. 29 What isrpm-ostree? ● Derived from ostree ● Initially conceived of as a way to parallel install multiple UNIX-like OSes (e.g., Fedora Rawhide and Fedora 20) ● “git for operating system binaries” ● Creates an installable tree from RPMs ● Not a package manager, but does take on some of the role from package managers
- 30. 30 What rpm-ostreeEnables ● Install one or more operating system trees to a system ● Gives “atomic” updates ● An update is, essentially, one unit – it succeeds or fails ● An update can be rolled back ● Allows switching between “trees” ● Provides tools for creating tree composes
- 31. 31 Current Limitations ● Currently, an rpm-ostree “tree” is an immutable system ● Doesn't allow for adding packages to a system w/out rebuilding the tree ● Build tools are still being developed, but moving quickly
- 32. So, anybody heardooff tthhiiss DDoocckkeerr tthhiinngg?? SCALE 12x
- 33. 33 The Problemwith Packages ● Deploying complex services / applications is difficult with packages ● Packages aren't as portable as we'd like ● Application is developed on CentOS 6, but production is using CentOS 7? ● Packaging guidelines can be ... difficult ● Packages don't provide any solution for running containerized applications...
- 34. 34 Docker: It'sLike Deluxe Super Awesome Packaging ● Docker is application-centric ● Docker containers are portable ● Supports versioning for an entire container ● Components can be re-used ● Allows for supplying ready-to-run services rather than half-configured packages ● Buzzword compliant
- 35. 35 Docker tothe Rescue? ● Docker containers: relatively easy to work with ● Provide far more “services” than package systems ● Application isolation ● Image format, sharing, API ● Allows “layering” of applications ● One group provides base image ● Another group provides base image + framework/service (e.g., Apache) ● Another group provides base image + framework + finished application ready to deploy
- 36. TThhaannkkss!! JJooee BBrroocckkmmeeiieerr jjzzbb@@rreeddhhaatt.ccoomm TTwwiitttteerr:: @@jjzzbb
Editor's Notes
- #5 In the beginning, all we had was source, and it was a pain to work with. But we didn't know better, and we compiled, and compiled, and compiled... Then there were package managers... and software installation became easier, and we looked to a New Hope...
- #6 For a long time, Linux distributions were – essentially – the center of the universe for getting open source software. You installed Red Hat, Debian, Fedora, or whatever and took the packages that were current because it was the easiest way to consume FOSS projects. It also ensured that you'd get updates and bug fixes, and life was good...
- #7 Surely a variation of “Murphy's Law” must be this:The system version of Python, PHP, Ruby, Perl, Node.js, or whatever is not the version that your application needs or the version you want to use to build a new application. It will invariably be too old or too new, or have some minor bug that breaks your application. (e.g., CloudStack and Tomcat bug...)
- #10 Developers have a lot of choices, not least of which is just grabbing their packages from upstream (e.g., Ruby). That way lies madness, and bad things. Containers, especially Docker, are the new hotness... but 1) not ready for prime time just yet, 2) may not be suitable for many instances, and 3) where do you get the software for Docker? PaaS – see above. Software Collections...
- #13 Let's say you're running Fedora 19 or CentOS 6.x and want to run a different version of LibreOffice. So you go to LibreOffice.org and decide to download 4.2.1. Then you remove the old packages and install the new packages. Yeah, that's not going to work for system software... so that's not how SCLs work.
- #14 Software Collections are distributed as a set of several components, which provide their full functionality without overwriting system files.
- #16 Software Collections make use of a special file system hierarchy to avoid possible conflicts between a single Software Collection and the base system installation.
- #17 To convert a conventional package to a single Software Collection, you only need to make minor changes to the package spec file. With a single spec file, you can build both the conventional package and the Software Collection package.