Downloaded 22 times
![Behold!
ryan@animatronio ~ $ sudo rpm -‐Uvh http://my.mirror.co/pub/
el/7/x86_64/nano-‐2.3.1-‐10.el7.x86_64.rpm
Retrieving http://my.mirror.co/pub/el/7/x86_64/
nano-‐2.3.1-‐10.el7.x86_64.rpm
Preparing...
################################# [100%]
Updating / installing...
1:nano-‐2.3.1-‐10.el7
################################# [100%]
ryan@animatronio ~ $](https://image.slidesharecdn.com/ures14-141110133449-conversion-gate02/75/Packaging-is-the-Worst-Way-to-Distribute-Software-Except-for-Everything-Else-14-2048.jpg)
More Related Content
Similar to Packaging is the Worst Way to Distribute Software, Except for Everything Else
Packaging is the Worst Way to Distribute Software, Except for Everything Else
- 1. Packaging is theWorst Way to Distribute Software, except for everything else Ryan McKern | Puppet Labs mckern@puppetlabs.com
- 2.
- 3. What do Ido now? Release Engineering at
- 4. Maybe you've used our products?
- 5. What have Idone? System Administration … for 13 years.
- 6. I’ve probably been your customer
- 7.
- 8. "Prejudice is agreat time saver. You can form opinions without having to get the facts." Attributed to E.B. White, Source unknown
- 9. Let's talk aboutsoftware!
- 10. Distributing software sucks Shipping new platforms is so hard Cross-platform packaging is so hard Unpredictable user-space is so hard Moving the packaged bits is so hard
- 11. Everything is sohard
- 12. Who among usknows this pain? sad@roberto Downloads $ wget -‐-‐quiet http:// ftpmirror.gnu.org/gcc/gcc-‐4.9.1/gcc-‐4.9.1.tar.bz2 sad@roberto Downloads $ tar xjf gcc-‐4.9.1.tar.bz2 sad@roberto Downloads $ cd gcc-‐4.9.1/ sad@roberto Downloads $ ./configure ./configure: line 532: sed: command not found ./configure: line 1371: sed: command not found ./configure: line 1920: sed: command not found ./configure: line 2291: sed: command not found configure: error: cannot run /bin/sh ./config.sub ./configure: line 361: sed: command not found ./configure: line 310: sort: command not found
- 13. This was aproblem because the customer's time has value
- 14. Behold! ryan@animatronio ~$ sudo rpm -‐Uvh http://my.mirror.co/pub/ el/7/x86_64/nano-‐2.3.1-‐10.el7.x86_64.rpm Retrieving http://my.mirror.co/pub/el/7/x86_64/ nano-‐2.3.1-‐10.el7.x86_64.rpm Preparing... ################################# [100%] Updating / installing... 1:nano-‐2.3.1-‐10.el7 ################################# [100%] ryan@animatronio ~ $
- 16. What's so great about packages?
- 17. Dependency management calculon~ # apt-‐get install cmake Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: cmake-‐data emacsen-‐common libarchive12 libnettle4 libxmlrpc-‐core-‐c3 The following NEW packages will be installed: cmake cmake-‐data emacsen-‐common libarchive12 libnettle4 libxmlrpc-‐core-‐c3 0 upgraded, 6 newly installed, 0 to remove and 51 not upgraded.
- 18. Verification bender ~# mv /usr/share/man/man8/applydeltarpm.8.gz ~/ bender ~ # rpm -‐V deltarpm missing d /usr/share/man/man8/applydeltarpm.8.gz bender ~ #
- 19. Distribution ryan@tinnytim ~$ gem push erniebert-‐0.1.0.gem Pushing gem to BetterThanRubyGems.org... Successfully registered gem: erniebert (0.1.0) ryan@tinnytim ~ $ gem install erniebert Fetching: ffi-‐1.9.6.gem (100%) Building native extensions. This could take a while... Successfully installed ffi-‐1.9.6 Fetching: erniebert-‐0.1.0.gem (100%) Successfully installed erniebert-‐0.7.1 1 gem installed ryan@tinnytim ~ $
- 20. What could bebetter about packages?
- 21. Sometimes shipping bits really is hard
- 22. Security is oftenboth the joke and the punchline ouch@killbot ~ $ dpkg-‐sig -‐-‐verify puppet_3.7.1-‐1puppetlabs1_all.deb Processing ./puppet_3.7.1-‐1puppetlabs1_all.deb... GOODSIG _gpgbuilder C093A3A56A6E0BEEA2821DD7133957EA11028DF3 1413702159 ouch@killbot ~ $ dpkg-‐sig -‐-‐verify ./puppet_2.7.23-‐1~deb7u3_all.deb Processing ./puppet_2.7.23-‐1~deb7u3_all.deb... ouch@killbot ~ $
- 23. So, so, somany similar-but-different formats
- 24. Let's talk aboutsome popular packaging formats
- 25. .rpm • Managedby the recursively named "RPM Package Manager" & yum • cpio compressed binaries & text files • Post-installation tasks are shell scripts
- 26. .deb • Managedby dpkg & apt, the "Advanced Package Tool" • ar compressed package with two gzipped tarballs & a small text file • Post-installation tasks are shell scripts
- 27. Mac .pkg •Used by Mac OS X, and often delivered in a .dmg (disk image) or a .zip file • xar compressed archive, containing a binary file, two archives, and an XML document • post-installation tasks are still shell scripts
- 28. About all thosepost-install shell scripts Maybe they're not that safe, but the surface area of this problem is big. That doesn't mean we needed "dash"
- 29. Ruby .gem, Python.egg, and Node .npm • These are library managers with delusions of grandeur • Reuses the "download, decompress, configure, build, install" patterns, which hasn't got much spam in it • Constant compilation is a bummer
- 30. What about... ? #realtalk We only have 45 minutes, and I hope you're going to have some questions for me to evade
- 31. What are somealternatives?
- 32.
- 34.
- 36. Full Disclosure •Puppet Labs does use the curl|bash technique as an option for our PE agent installation • If you don't trust your own Puppet Master, who do you trust? • (ALL THE COOL KIDS WERE DOING IT)
- 37. You just wantedRuby but you got Cthulhu ~ $ curl -‐sSL https://get.rvm.io | bash -‐s -‐-‐ -‐-‐fhtagn G̺̞ ̯͔̮̫̥ ͊̌͂ a͍͕͓̦͈̯̟̋ r̘̰̟e̓̓ ̦ C̋͋ͬt̂̅̓ t͇̻̩̲̬ ͇̪̹͔̾ ̟ḧ́΅ͭ ̩̿ͭ ͖̙̤ ͭl̅ͦ̓ ̝̙̭ ̗ ṷ ů̥͖ ͍͎͍ ̦̟ n̠̣̭ ̞̻̱̳ ̬ ̣̗ ̑ ̖͎ͩ hͯ ͐ ̝̤̊ ̞̭̳͚̞ ̘ s ̓ ͔̣ ̺̝͇ l̃ͪ͐̎̍ a̅͋̏̀ ̜̯͉ ͈͇̲ ̓̑ ̭̻l̂ͬ̽ ͮ ̙͇̼͍ r̭̂̋ͦ ̻̺̭ ̗͙̃ ̻̤̳ ̰̤ i̅̿̌ͫͣͪ ̺̙̽_̻͚ ̤ s ̮͇ e͍̞̚ ̿̌ͮ̍ ̝͕̳́̽ ̩̺ͅ ͉ r f͈̱͓͓ ̦̰̬̗ ̗̝̼ m̞̗͎͍̾̈͊ o͈̩ͪ̈ ̟̤̻͉ ̃ͨͬ́̉ͩ̓ ̰̝e͍͎h '̼̬̤̋̉̽lͪ ͣ ̗̼ͥ̉R ̱͚̏ ̫͙͓̰͔ ̃̂̂ ͕͓̲ y .̚ ̣̫ ̞͓͈ ̼̪̠s͔̹̞̟t͈̘͕ H͎̯ ̙̱a͍̟͍ͅ ̘̼ u͙͓̙̟r̥̹̫͇͎ ͚ ̻ͅ ̣ ̙̹ ͍̮ t ̻̳̮ h ̩̜̣_ e̠_̱̣͔ ̼ ̺͉U s͈̰̣̥ p̻ a͕̗̣̺ k ͇e̤͍̯ ̻̹͓̬ ̹̤̳a̠͍̪ b̩_ ̪_̥͖͎͍ e̪̻̣̣ ̙̼ ͈̬s͇̮̞ ḻ͇ ̥͖̠ ̹̩̖ ̙̲ a̺͈̹̤͈͉ͅ ̣̮͕̙ ̗ h ̠̟ ̰̜̜l̬̹̭ l̺̞ ̩̳̮̩̰͕ͅ ̻r̮̥̦͍͍ ͈̫ e̳̠̙̘ ̱ u̠͇r̮̣͓ ̘̬̰ t͔͚̳̹̰ ̰͖ n̗͍ ̥͕ ̥͉f̜͚r̯͍ͅ o͈̯̦ ̖ ̳͓̦͔ ¯ͭ̔ ̻͙̫̪̪̖͈ ͔̬ ̣̌ ̠̟̱̒ ͍m̻̟ ̭ ̼̠ ͍̣t͖h͔͉̞ e̬̫̦ ̋͂ ̖͇̼ ̊ͤ̓̋̄̐͌̾ ̩̝̮ ͓ d̰̼̞̤͕ ̤̘̣̭͍̖ ̻͈ ̟̭ ͡l̴l h͞ow l͢ f̕o ́̃̍̆̂̇̒ͫ k̲͖̻̻̆͋ͬ̑ ̩͍̭̙ ̥ k̙̣͕͔ ̘̮̤̻̜̳ r̯̰̱̬̭ͅ ḁ͙ ͔͔̺ ̠ ̗ s̻̱͎ ̙̦̝̗͍͎ ̞̪t̫͉̟̻ ͖ ạ̫ r͔̺ ͍.͔̖͚̺̹ ̰̫ Ǹ`ya͠rĺath̢ote͡p̢ ͟s̀`h ̨ a ̷reve̢r̀` ͑̎ ň̽̌ ḯ΅ ̣ ̃̄̇ͪ̂͑ ͉͔̙̤̪̜ h ̉ͦ ͕t͂̔ ͨ͋̅̿ ͔̆ͫ̓ͫ ̫͖̻ e ͫ͌͛ ͦ͆ͭ̽ ̊ͩͩ̇ͣ ̗ͅd̂ ā̇ͤ͋ͭͨ ̗̰ ͙̗̝͕̩̥ ̟͍ ř nͮͯ̑̿͒ e̍͒̅̄ͣ̀ ͅͅ ̪̠̗͕̥ ͋̋ ͙̹͎̺̠ s̊̈̽̊̌ s̈̌ͪ ̱̳ .̄̑̎ ͔̙̣ ̤̰̟̦̥ ͉̉ ͙̬
- 38. curl | bashoften assumes • There is no air-gap • Every request is a safe & sane request • That HTTPS is good enough
- 39. curl | bashoften forgets • >100% Broadband coverage • Mirrors exist • HTTPS secures transport, not content
- 40. curl | bashtotally ignores • The benefits of reusability • The fragility of shell scripts • The fragility of shells
- 41. Security is hard • RVM recently introduced hand-rolled GPG signing* • Thread had 48 comments within a week, almost universally about the implementation • Broke semver, automation, and hearts * https://github.com/wayneeseguin/rvm/issues/3105
- 42.
- 43. Isn't that fromChef? • Sure, but so is Test Kitchen • Builds packages while still controlling the entire dependency stack • Lots of love from users with complicated dependency stacks
- 44. Omnibus is oneway to skin the entire cat • Abstracts (instead of removes) dependency management • Only builds packages for the platform it's installed on • You're going to want to know Ruby
- 45.
- 46. Effing Package Managers •General purpose swiss-army knife of package building •Works around a lot of the shortcomings of existing package managers •Jordan Sissel is a SAINT (Shout out to #hugops!)
- 47. "Common packaging patterns,a distaste for existing packaging practices, and some hate-driven development yielded FPM! Add some amazing contributions in code, bugs, features, and support from the community and boom we have modern FPM." Jordan Sissel My inbox, Oct 10 2014
- 48. Effing FPM •Swiss army knives are rarely the best tool for a given job • General purpose in this case means a lot (~150ish) of command line flags • Still infinitely better than curl | bash
- 49. Why so manyalternatives? What went wrong?
- 50. RPM Packaging can be tough • RPM Spec files are weird • Kind-of M4, kind of Shell, all obtuse • Oh, and kind-of Make; only kind-of • Sort-of competing RPM standards
- 51. Deb Packaging canfeels like penance • "debian/" directories are outright hostile to man & beast alike • Debian "Helpers" usually don't • dpatch can use unified diffs (sane) or shell scripts (what?!)
- 52. Conflation of purpose • Some library managers try to install executables, e.g. gem, pip, npm • Remember when I said "delusions of grandeur"? (Google Image Search was kind of useless here)
- 53. But really, Ijust have a hypothesis! • Developers love solving new problems • Sometimes they confuse their problems for the customer's problems • Maybe packaging isn't a solved problem yet, but it's close
- 54. Where do wego from here?
- 55. Sometimes the onlychoices you have are bad ones; but you still have to choose.
- 56. TL;DR: this problemis (mostly) solved Stop writing new installers from scratch Give your customers the best packages possible Don't forget Pareto (any number of 80/20 rules)
- 57. Thank you You'rewonderful. Thank you for letting me rant at you for as long as you did. mckern@puppetlabs.com @the_mckern
- 58.