Recommended
More Related Content
What's hot
What's hot (20)
Similar to Puppet Camp LA 2/19/2015
Similar to Puppet Camp LA 2/19/2015 (20)
Recently uploaded
Recently uploaded (20)
Puppet Camp LA 2/19/2015
- 1. Package Managers and Puppet Joe Damato packagecloud.io
- 3. hi, I’m joe! • i think these things are cool: • computer programs • reproducible builds / infrastructure • automation • configuration management • tahdig* * an rice food
- 4. packagecloud.io • I work on packagecloud.io • packagecloud makes it easy to upload, download, store, and delete software packages • you should use it, it’s cool.
- 7. Why? • Central to maintaining, building, and testing infrastructure. • Packages are a primitive in Puppet. • Understanding where packages come from, and how to store them properly is a requirement for infrastructure of any size. • Packages and packaging are much trickier than they seem!
- 8. Overview • what is a package? • what is a package manager? • ./configure && make && make install pattern • open source tools for package repositories • HOWTO manage repos in your infra with puppet
- 9. What is a package? Beck Gusler, https://flic.kr/p/4A15jm
- 10. What is a package? • A package generally consists of: • metadata (version, architecture, deps, etc) • files to be written to the filesystem (/usr/sbin/ nginx, etc)
- 12. Common package types • RPM packages • Used on CentOS, RHEL, Scientific Linux, Fedora, … • files typically have the “.rpm” file extension • can be inspected, installed, and removed with rpm • are actually a: • header structure (binary data) • CPIO archive
- 13. man 8 rpm
- 15. Common package types • Deb packages: • Used on Ubuntu, Debian, Knoppix, … • files typically have the “.deb” file extension • can be inspected, installed, and removed with dpkg
- 16. Common package types • Deb packages: • are actually an AR archive with: • version file: the debian format version • data.tar.gz: the actual files to write to the filesystem • control.tar.gz: the package metadata • Can be GPG signed, but signatures are never checked!
- 17. man 1 dpkg
- 18. Common package types • There are lots more! (ruby gems, npm, java, python, …) • Some packaging systems also have source packages.
- 19. What is a source package? • A source package consists of: • metadata (version, architecture(s), build deps, etc). • source files (C source, C++ source, py scripts, etc). • Allows you to rebuild a binary package easily.
- 20. Install packages with puppet Use the resource type ‘package’ to install packages: package { 'pygpgme': ensure => latest, }
- 21. Install packages with puppet package { 'pygpgme': ensure => ‘0.3-11’, } Specify the version you want by setting ensure:
- 22. Summary • Packages are a collection of files with metadata. • The metadata usually has info like: • architecture • version • dependency info • and more. • Installation is easy if you don’t have dependencies.
- 24. Dependencies • Installing 1 package is as easy as: • dpkg -i filename.deb • rpm -ivh filename.rpm • Of course, you should use puppet instead :D • But what if your program needs other programs? • For example: nginx depends on libssl, zlib, …
- 26. So, what’s a package manager?
- 27. Package manager • A package manager is a collection of software that allows you to: • install, upgrade, remove packages • query package info from local system or repos • Some tools include more advanced features like mirroring or more advanced caching features.
- 29. • yum (Yellowdog Updater, Modified) • Common on RHEL, CentOS, Fedora, … • Used for installing, removing, configuring, and querying RPM packages and dependencies. Common package managers
- 31. Common package managers • APT (Advanced Package Tool) • Common on Debian, Ubuntu, KNOPPIX, … • Used for installing, removing, configuring, and querying Debian packages and dependencies.
- 32. Install packages with puppet • When you install packages with puppet, puppet will automatically detect which package manager to use. • You won’t need to worry about which command to run, or what options to pass; puppet will take care of that for you!
- 33. Summary • package managers help you install software and associated dependencies • easily remove, upgrade, and query packages • Puppet will automatically detect the system’s package manager when you install a package.
- 35. A problem • You run Ubuntu 10.04 LTS • You want to install redis • Ubuntu 10.04 comes with redis-server 1.2.0-1 • That’s too old! You need 2.8.19! • So, now what?
- 36. Common (not great) solution • A common solution to this sort of problem is building redis (or ruby, or …) from source in your puppet manifest • Like this….
- 37. exec { 'install-redis': command => "make && make install PREFIX=${redis_bin_dir}", cwd => $redis_src_dir, path => '/bin:/usr/bin', unless => "test $(${redis_bin_dir}/bin/redis-server --version | cut -d ' ' -f 1) = ‘Redis'", require => [ Exec['unpack-redis'], Class['gcc'] ], } Common (not great) solution
- 38. Why? • It’s easy! • ./configure && make && make install • It works! • I’m using puppet so it’s reproducible!
- 39. But… • What happens if you need to: • completely remove Redis? • install a security update? • install a new version? • install the same exact Redis on 200 machines?
- 40. The not-so great side • Not all Makefiles have uninstall targets, so you have to remove files manually • Leaving artifacts on the filesystem can cause really, really hard to debug problems later • If the build process changes version to version, it can be painful to rollback
- 41. The not-so great side • Rebuilding the same source does not necessarily get you the same byte-for-byte binary • If the binaries aren’t identical, you can end up with bugs in some of the compiled binaries but not others • Painful to recreate source builds inside of puppet • Makes writing tests for manifests painful
- 42. Make a package • Install the same binary on every machine • When the package is removed, all installed files are removed • Versioning of build process built in (with most tools) • Keep your puppet manifests about config management • Your build steps are “factored out” into the package
- 43. Your new puppet manifest package { 'redis': ensure => latest, }
- 44. Your package • Your build steps get encapsulated in the package itself • Makes iterating on the build more straight forward • Don’t need to apply (potentially) a bunch of manifests to a machine every time you do a build
- 46. “How do I make a package?”
- 48. Use tools! • debbuild • rpmbuild • fpm • mock and pbuilder (more advanced)
- 49. Tradeoffs • Takes time to learn new tools • Takes time to understand packaging • No one ever has enough time
- 50. BUT…
- 51. Tradeoffs • Once you learn how to make packages you can build reproducible infrastructure much more easily • You can use your prod environment in dev and test • You can more easily build tests for your infrastructure with beaker/kitchen.ci
- 53. “How do I store and organize my packages?”
- 54. Package repositories • Major linux distributions keep repositories of packages for users: • EPEL • Ubuntu / Debian official repositories • You can store a package and its dependencies to make it easy to install them all on your infrastructure
- 56. Package repositories • createrepo: creates yum repositories • reprepro: creates apt repositories • Many other free tools available! • Read the documentation carefully. Lots of tricky options. • I’ll show some examples to get you started!
- 58. createrepo • mkdir /var/www/myrepo • cp /path/to/rpms/*.rpm /var/www/myrepo • createrepo /var/www/myrepo • gpg --detach-sign --armor /var/www/my/repo/repomd.xml
- 59. GPG is important • Using GPG to sign the generated repository guarantees that you generated the repository. • This is important. • This means that no one else modified, removed, or inserted a package other than you. • GPG signing the repository is not a very well known security measure, but it is incredibly important! • This is NOT the same as using rpmsign/rpm --sign.
- 60. Secure YUM repos • Sign repository metadata with GPG • Sign packages with GPG (use rpmsign) • Serve repositories over SSL • Enable all the right options for SSL verification, repository GPG checking, AND package GPG checking.
- 61. Wouldn’t it be cool to do all that with Puppet instead? Good news: you can!
- 62. createrepo via puppet Puppet can create YUM repositories for you! $ puppet module install palli-createrepo
- 63. createrepo { 'yumrepo': repository_dir => '/var/yumrepos/yumrepo', repo_cache_dir => '/var/cache/yumrepos/yumrepo' } createrepo via puppet
- 64. You still need to GPG sign the repository yourself :( exec { “gpg_sign_yumrepo”: command => “gpg --detach-sign --armor /var/yumrepos/yumrepo/repodata/repomd.xml“, }
- 65. Once the repository is created, it must be added to the client machines.
- 66. Add YUM repos with puppet yumrepo { 'my_repo': baseurl => "http://myurl.com/repo", gpgcheck => 1, repo_gpgcheck => 1, gpgkey => “http://myurl.com/gpg.pub.key”, sslverify => 1, sslcacert => “/etc/pki/tls/certs/ca-bundle.crt”, enabled => 1, } most people never turn on repo_gpgcheck or sslverify, or set the ssl certificate path, but you should!!
- 67. But that’s not all! • You MUST have the ‘pygpgme’ package installed on the system that will verify the signatures. • Without pygpgme, yum will not be able to verify signatures! • Some versions of CentOS / RHEL do not automatically install pygpgme with yum!!
- 68. Make sure to install pygpgme package { 'pygpgme': ensure => latest, }
- 69. reprepro APT
- 70. reprepro • mkdir /var/www/myrepo • mkdir /var/www/myrepo/conf • Create a file named “distributions” in the conf directory
- 71. reprepro Codename: precise Components: main Architectures: i386 amd64 SignWith: 7ABDB001 /var/www/myrepo/conf/distributions:
- 72. reprepro • You can add more sections if you need more code names (lucid, trusty, etc). • SignWith specifies which GPG key to use for signing repository metadata • You can get your gpg key ID by looking at the output of gpg —list-keys • This is not the same as using debsigs/debsign !!!
- 73. reprepro import your Ubuntu 12.04 packages: reprepro -b /var/www/myrepo/ includedeb precise filename.deb
- 74. Wouldn’t it be cool to do all that with Puppet instead? Good news: you can!
- 75. reprepro via puppet Puppet can create APT repositories for you! $ puppet module install jtopjian-reprepro
- 76. # Base Directory shortcut $basedir = '/var/lib/apt/repo' ! # Main reprepro class class { 'reprepro': basedir => $basedir, } ! # Set up a repository reprepro::repository { 'localpkgs': ensure => present, basedir => $basedir, options => ['basedir .'], } reprepro via puppet
- 77. # Create a distribution within that repository reprepro::distribution { 'precise': basedir => $basedir, repository => 'localpkgs', origin => 'Foobar', label => 'Foobar', suite => 'precise', architectures => 'amd64 i386', components => 'main contrib non-free', description => ‘My repo', sign_with => 'F4D5DAA8', not_automatic => 'No', } reprepro via puppet
- 78. Once the repository is created, it must be added to the client machines.
- 79. Add APT repos with puppet apt::source { 'myrepo': location => ‘http://myurl.com/repo', release => 'precise', repos => 'main', key => '7ABDB001', key_source => ‘http://myurl.com/gpg.pub.key', include_src => true, } $ puppet module install puppetlabs-apt
- 80. But that’s not all! • You MUST have the ‘apt-transport-https’ package installed on the system if your repository is served over HTTPS! • Without apt-transport-https, you can’t install packages over HTTPS. • You definitely want this.
- 81. Make sure to install apt-transport-https package { ‘apt-transport-https‘: ensure => latest, }
- 83. Success • You can now use beaker/kitchen.ci/etc to test your infrastructure. • Determine if the packages you need are actually installed after your manifests are applied. • Determine if the repositories you added are actually added after your manifests are applied. • Don’t need to wait forever for Ruby, redis, et al to build during a test run.
- 84. BEST OF ALL !!!! • You can now run Puppet on your development VM using the same manifests you use in production • The manifests are applied and you are running the same exact binaries you run in production • Won’t catch ALL production bugs, but getting closer to production during development is super useful
- 85. Summary • Creating package repositories can be tricky. Make sure to GPG sign repository metadata. • 99% of package repositories get this wrong. • Carefully read the documentation of createrepo and reprepro. • Make sure to install necessary libraries for verifying signatures and accessing repositories via HTTPS. • Always serve up your repositories over HTTPS.
- 86. Use puppet to automate this.