Wrangling 3rd Party Installers from Puppet

1,385 views
1,224 views

Published on

"Wrangling 3rd Party installers from Puppet" by Spencer Krum
and Mike Kinney of UTI Worldwide at Puppet Camp Portland 2014.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,385
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
22
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Wrangling 3rd Party Installers from Puppet

  1. 1. Wrangling 3rd Party Installers with Puppet Puppet Camp Portland 2014 January 14, 2014
  2. 2. Overview ● ● ● ● ● About us Definitions Our situation Overall steps What we get and what is missing from installers ● Requirements ● ● ● ● ● ● ● ● Installer “state” Initial scripting v2.0 using Puppet Our solution Testing Challenges Future Q&A
  3. 3. About us Mike Kinney Spencer Krum mike.kinney@gmail.com krum.spencer@gmail.com github.com/mkinney github.com/nibalizer ● UTi Worldwide ● Mentioned in the “Book” ● UTi Worldwide ● PSU CAT ● Co-author of the “Book” 2nd Edition
  4. 4. About us Mike Kinney ● Automating installations for few years ● New to puppet Spencer Krum ● New to these 3rd party tools ● Puppet expert
  5. 5. A few words about vendors ● We use a few vendors. ● We will try to keep their names out of it, though it might slip out once or twice. ● The focus of this talk is how we work around not being able to use a package management tool. (like apt/rpm) ● These patterns should be universal.
  6. 6. Definitions (1 of 4) ● 3rd Party software: ○ Software that is bought from a software vendor, almost always closed source. ● Installer: ○ The installation method described by the vendor for the 3rd party software. Often this is a binary file like “installer.bin”.
  7. 7. Definitions (2 of 4) ● Quality Assurance (QA): ○ Process that validates functionality of a release. ● Silent installation template file: ○ File with the all installation options and values. Sample file: <?xml version="1.0"?> <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd"> <properties> <entry key="acceptLicense">true</entry> </properties>
  8. 8. Definitions (3 of 4) ● Version upgrade: When either the x or y in version x.y.z is higher than the currently installed version. When the product has bugs or errors, sometimes we will get out of cycle patches from the vendor: ● Engineering build: (no applicable version) ○ quick (little or no) QA before releasing version ● Hotfix: (the “z” in version x.y.z) ○ full QA before releasing version Note: Some vendors use x.y.z.a
  9. 9. Definitions (4 of 4) ● idempotent: Only make changes if they are required. ● hiera: Helps to separate data from code (see Chapter 12 of the “Book”)
  10. 10. Our situation ● most of our stack uses Enterprise tools (lots of 3rd party installers) ● mostly linux (OEL 5) ● lack of “root” has its challenges ● lots of environments ● each technology on a separate virtual machine (means lots of VMs) ● many vendor/consultants/partners
  11. 11. Overall Installation steps ● ● ● ● ● ● Download installation file (zip/tgz) Review requirements Pre-install steps Run installer Installation options Post-install steps
  12. 12. 3rd Party Installers Most have: ● sample silent template file ● installation binary ● installation files ● installation documentation Most are missing: ● service script (start/stop/status) ● any verification methods (post installation)
  13. 13. Pre-install Requirements ● Most require java (run-time) ○ cannot use openjdk! ○ some products bring own jre ● Most require another specific version of another 3rd party installation (example: product B v4.9.0 requires product A v8.3+)
  14. 14. Installation User Interfaces ● Interactive graphical user interface (xterm window) ○ often the options are saved in a file and can be converted into a silent installation file ● Interactive text mode ● Command line using “silent” installation option: ./installer -silent options.txt
  15. 15. 3rd Party Installer “state” ● Different installers store installation status in different places; typically in home dir and or in the destination directory ● If vendor support wants to know about system, they often ask for files/directories showing this state (for what is installed)
  16. 16. Scripting options A. Re-package: ○ install once - preserve files post-install ○ the internal “state” would have to be reverse engineered ○ vendor would have an excuse “you did not install per instructions” B. Script using the “silent” installation
  17. 17. Initial installation scripting (v1.0) ● ● ● ● ● ● Initial scripting (ant/shell scripts) One configuration file per installation per VM Lots of redundant config Hard to maintain Hard to train Put the binary installers in version control system (sounds bad - but it works for us) ● Shared /software source (home of tarballs)
  18. 18. v2.0 - Using puppet ● ● ● ● <environment>/site.pp rootless package /data/env/<envname> (hiera) Combine product installs into one class
  19. 19. v2.0 - Using puppet (continued) ● Vendor install class (defined type) ● Two stages to install: 1. extract software to /tmp and run installation (see sufact hack) 2. configuration files, cron, aliases, etc.
  20. 20. Example of a Puppet Package package { 'apache2': ensure => '2.4', }
  21. 21. Resource abstraction layer Pretty Picture Here
  22. 22. Resource abstraction layer
  23. 23. Puppet is a Programming Language ● Contains and hides complexity (API) ● You do not really want to see how the sausage gets made. Note: ● We are creating “anti-patterns”. ● Need to ensure code is idempotent.
  24. 24. Example of API class { 'ssh': root_login => 'without-password', }
  25. 25. Another API example class { 'apache': mods => ['index', 'rewrite'], }
  26. 26. Example of Vendor::Software class { 'vendorSoft::someSoftware': version => 2.1, }
  27. 27. Sample Vendor/Product As an example, we are going to install Acme’s product Foo version 4.9.0. ● In the <environment>/site.pp: class { 'acme::foo490': }
  28. 28. Two types of Vendor Installs 1. 'untar' installs Install consists of untarring or unziping files to the correct locations. 2. 'installer' installs Install consists of running the vendor's installation binary with flags and a silent file. Note: Both of these will have the same interface from puppet.
  29. 29. 'untar' installs: Puppet class ● Ensure all dependencies ● Unzip zipfile/tarball from network share to installation location ● Set values in configuration files
  30. 30. 'install' installs: Puppet class ● Unzip zipfile/tarball from network share to temporary location ● Set values in configuration files or “silent” file ● Run installer ● Clean up temporary installation Note: We were not going for how many Puppet Style guide infractions we could make, it just happened that way.
  31. 31. Idempotency How do we make these idempotent?
  32. 32. Idempotency: untars rootless::tardir { '/opt/app/other_place/folder': ensure => present, tarfile => '/big/nfs/mount/tar.tar.gz' }
  33. 33. Idempotency: untars exec { 'untar-directory': provider => shell, command => '/bin/tar -xvzf /big/file.tgz /the/location', unless => 'stat /the/location/tardir', }
  34. 34. Sample acme::acme_install (1 of 5) define acme::acme_install( $creates_file, $installationFile, $installationRoot, $installer, $product, $tmpdir, $version, $installer_content, $ensure = true, $group = $::puppet_user, $owner = $::puppet_group, $source_dir = "/software/install/installers", $create_tmpdir = true, ){
  35. 35. Sample acme::acme_install (2 of 5) if $ensure { # install if $create_tmpdir { file { "${tmpdir}-create": path => $tmpdir, ensure => directory, } } anchor { "$name-unpack::begin": } anchor { "$name-unpack::end": } <snip - unzip/untar > Anchor pattern
  36. 36. Sample acme::acme_install (3 of 5) file { "${tmpdir}/silent": ensure => file, content => "$installer_content", Silent file from template require => Anchor["$name-unpack::end"], } file { "${tmpdir}/${installer}": ensure => file, mode => '0755', require => Anchor["$name-unpack::end"], } Ensure installer is executable
  37. 37. Question What is the return value of a zero byte file that has the “execute” bit set? $ echo > /tmp/nobytes $ ls -al /tmp/nobytes -rw-r--r-- 1 user wheel 1 Jan 14 12:06 /tmp/nobytes $ chmod +x /tmp/nobytes $ ./nobytes $ echo $? What is the return value from “./nobytes”?
  38. 38. Sample acme::acme_install (4 of 5) # verify that the file we are about to execute is *not* a 0 byte "executable" exec { "verify-installer-${acme_product}": command => "/usr/bin/[ -s ${tmpdir}/${installer} ]", provider => 'shell', cwd => $tmpdir, require Check => File["${tmpdir}/${installer}"], } exec { "install-${acme_product}": command => "source ~/.bashrc && ${tmpdir}/${installer} -silent -V responseFile=${tmpdir}/silent", provider => 'shell', timeout => 30000, cwd => $tmpdir, Do installation require => [File["${tmpdir}/silent"], File["${tmpdir}/${installer}"], Exec["verifyinstaller-${product}"], ], creates } => "${installationRoot}/${creates_file}",
  39. 39. Sample acme::acme_install (5 of 5) # Check the logs for errors exec { "${acme_product}-log-check": command => "! /bin/grep -R -E "::ERROR::|err.X" ${::puppet_user_home}/.ACME", provider => 'shell', require => Exec["install-${acme_product}"] Any errors? } # need to clean up tmpdir if $clean_up_tmpdir { exec { "${tmpdir}-remove": Clean up command => "/bin/rm -fr ${tmpdir}", provider => 'shell', require => [ Exec["install-${product}"], Exec["${product}-log-check"], ], } } } else { # uninstall <snip>
  40. 40. Problem is (still) idempotency ● Multiple resources cannot be chained to an unless clause of an exec ● Anchors are a part of this ● Maybe 3.4 contains() will help fix this, we have not had time to evaluate
  41. 41. Sufact (su fact) ● Puppet module ● Modest ambitions ● Turns out to be really awesome https://github.com/TheDarren/fact
  42. 42. Sufact (su fact) ● fact { "su_support": value => "IDG" } ● Then: ○ $su_support == "IDG" We use this to flag and pin versions of vendor software.
  43. 43. Puppet ● Built each version of each installer as separate module acme::yyy123 where yyy is a short name of the product and 123 is the “version” Write facts to set these values, then check
  44. 44. Sample foo490 (1 of 3) class acme::foo490( $owner = $::puppet_user, $group = $::puppet_group, $install_root = "/opt/app/${::puppet_user}", $source_dir = '/software/install/installers', $tmpdir = "/tmp/tmpfoo490-${::puppet_user}", $installationFile = 'foo_4.9.0_linux26gl23_x86_64.zip', $installer = 'Installer-lnx-x86.bin', $version = '4.9.0', # these values are in the silent file $silent_environmentName $feature_acme $ensure = true, Silent values = hiera('acme_feature_xxx', “blargh”, ){ = 'FOO', Hiera pattern
  45. 45. Sample foo490 (2 of 3) $installationRoot="${install_root}/somedir" $product = 'acme490' anchor { "${product}_installed": } if $ensure { Anchor pattern Ensure pattern # Note: Config files/cron entries, etc. here if $acme490_installed == undef { $installer_content = template("acme/${product}/silent.erb") acme::acme_install { $product: installationFile => $installationFile, installationRoot => $installationRoot, installer => $installer, product => $product, tmpdir => $tmpdir, Call our defined type
  46. 46. Sample foo490 (3 of 3) version => $version, creates_file => "foo${product}", ensure => true, group => $group, owner => $owner, source_dir => $source_dir, installer_content => $installer_content, before => Anchor["${product}_installed"], Anchor pattern } fact { "${product}_installed": value => $version, require => Acme::Acme_install[$product], } } } else { # uninstall Fact pattern
  47. 47. Combining Vendor products class acme::foo12 ( $installationRoot = "/opt/app/${::puppet_user}/somedir", ){ anchor { 'acme::foo12::begin': } anchor { 'acme::foo12::end': } class { 'acme::foo123': #parameters } Anchor [‘acme::foo12::begin’] -> Class [‘acme::bar123’] -> Class [‘acme::baz124’] -> Anchor [‘acme::foo12::end’ <snip> Dependency chaining pattern
  48. 48. Testing the Puppet installations ● ● ● ● ● Similar to Rspec-system/Beaker Totally home-grown Written in shell Integration testing Linux Container (LXC) Based
  49. 49. Testing the Puppet installations ● ● ● ● ● ● Fires up LXC Builds ssh bridge to LXC Run “puppet apply” Runs suite of 'checks' Teardown LXC Repeat for ever suite of checks
  50. 50. Testing the Puppet installations ● Work in progress ● Plan to leverage this for refactoring and beautification project ● Plan to use Jenkins for reporting
  51. 51. Testing the Puppet installations #!/bin/bash testname=foo610 if [ -d "/somedir/config" ];then : else exit 1 fi
  52. 52. Challenges ● ● ● ● ● ● ● Often used the “dot-uh-oh” versions Large and many installers Many different versions Testing Puppet development environment Installation validation Orchestration issues (queue creation, stop commands before upgrading files, start & wait, etc.)
  53. 53. Future ● Automation of pre-installation (checks) ● Automatically add the monitoring/syslog configs (both are done manually now) ● Possibly pull post-processing into puppet ● md5sum checking of installed files ● Possibly look into app deployments using puppet (need orchestration in a big way)
  54. 54. Questions Any questions?

×