tp install anything [anywhere(*)]
by Alessandro Franceschi @alvagante

Founder and CTO @example42

at #cfgmgmtcamp 2017
1
Tiny Puppet needs Puppet 4 (**)
• If Puppet is not installed, just feel reckless and run:

wget -O - https://bit.ly/installpuppet |
sudo bash
• Script removes old versions, detects OS, installs official
Puppet repos and then Puppet 4 agent
• (**) tp is optimised for Puppet4 but has defines compatible with Puppet 3:
tp::install3, tp::conf3, tp::dir3
2
"Piping the internet to my shell !?!?!"
• The https://bit.ly/installpuppet URL
redirects to a script in example42 control-repo:



https://raw.githubusercontent.com/example42/control-
repo/production/bin/puppet_install.sh
• Take your time to review it or just fire and forget




.



3
Install Tiny Puppet
• Usage from shell

puppet module install example42/tp

puppet tp setup

tp

• In Puppetfile for usage in manifests:

mod "example42/tp"

mod "example42/tinydata"
4
Yet Another Universal Installer
• al@localhost$ [sudo] tp install app
• It supports any application

on any operating system (*)
• It manages package repositories and other
dependencies
5
(*) Any Opeating System?
• RedHat and Debian derivatives always supported
• Suse, Alpine and other Linux distros need more tiny data
• Solaris, BSD, Darwin support partial but possible (may
miss data, tests and some features)
• Windows: WIP. Currently not supported
6
Some supported apps
• tp install docker-engine
• tp install sysdig
• tp install gitlab-ce
• tp install gitlab-runner
• tp install zabbix-agent
• tp install virtualbox
• tp install puppetserver
• tp install puppetdb
• tp install awscli
• tp install elasticsearch
7
All the supported apps
• By default TP uses tinydata as source for data about
applications
• Missing support for apps or OS is just a matter of

missing data
• github.com/example42/tinydata/
• You can use your own data module
8
Usage on cli: tp test to show apps status
• al@localhost$ tp test [app]
• Show status of all the applications managed by tp
• Can be automated in CI and testing
• Custom checks can be easily created
• If any of the apps fails, the global exit code is 1

9
Usage on cli: tp log to show apps logs
• al@localhost$ tp log [app]
• tail -f the log[s] of all or the written application
• Useful for quick debugging when unaware of log paths
• Customisations:
tp::install { 'redis':
settings_hash => {

log_file_path => '/opt/redis/redis.logs',

log_dir_path => '/opt/redis'

},
}

10
Usage on cli: tp list - tp uninstall
• al@localhost$ tp uninstall <app>
• Removes package and eventually repo file of an app
• al@localhost$ tp list
• List all the applications tp can install



11
Usage on manifests: tp::install
• Different options for tp::install. Here shown default values.
tp::install { 'postgresql':
ensure => present,
repo => undef,
auto_repo => true,
auto_prerequisites => false,
auto_conf => true,
options_hash => {},
settings_hash => {},
cli_enable => true,
test_enable => false,
puppi_enable => false,
}
12
Usage on manifests: tp::conf
• Manage configuration files of an app
tp::conf { 'nginx::example.com.conf':
ensure => present,
base_dir => 'confd',
$source => undef,
$template => undef,
$epp => undef,
$content => undef,
$base_dir => 'config',
$base_file => 'config',
$path => undef,
$config_file_notify => true,
$config_file_require => true,
$options_hash => { },
$settings_hash => { },
$data_module => tinydata
}
13
Usage on manifests: Test installed apps
• Testing of application can be enable by either one of:
tp::test { 'redis': }

tp::install { 'redis':
test_enable => true,

test_template => 'site/test/nginx.erb',
}
• A custom test script is created in /etc/tp/test/$app

Default tests use data from /etc/tp/app/$app
14
Essential acceptance tests
• TP be able to test an app (alternatives)


git clone https://github.com/example42/tp-acceptance
cd tp-acceptance
r10k puppetfile install
vagrant status
# Test munin on all supported OS
bin/test_app.sh munin all acceptance
# Test all apps on a specific OS
bin/test_app.sh all Centos7 acceptance
# Test all appas on all supporte OS

bin/test_app.sh all all acceptance
Compatibility Matrix
15
It's all about Tiny Data
• Data to manage apps on different OS is on separated tinydata
module
• Currently 149 applications supported (ls -1 data/ | wc -l)
• For each application a data/$app/hiera.yaml file define the
hierarchy to use for data lookup (erm, no, Hiera is not actually used):


---
:hierarchy:
- "%{title}/operatingsystem/%{operatingsystem}-%{operatingsystemmajrelease}"
- "%{title}/operatingsystem/%{operatingsystem}"
- "%{title}/osfamily/%{osfamily}"
- "%{title}/default"
- default
16
It's all about Tiny Data
• Eash data file, in YAML format, contains the info tp uses
to manage applications:

---
redis::settings:
package_name: 'redis'
service_name: 'redis'
config_file_path: '/etc/redis/redis.conf'
config_dir_path: '/etc/redis'
tcp_port: '6379'
pid_file_path: '/var/run/redis.pid'
log_file_path: '/var/log/redis.log'
log_dir_path: ''
process_user: 'redis'
process_group: 'redis'
17
Use cases - In manifests
• In local site profiles, when we have to configure
applications we know how to configure
• In component modules, with local tiny data
• tp can replace or coexist with component modules
18
Use cases - On the command line
• Quickly install applications
• Check if installed apps are working correctly:

monitoring, CI, runtime tests.
• Troubleshoot locally problems by quickly accessing logs
19
Thank you
• http://www.tiny-puppet.com
• @alvagante @example42
• Slides published on http://slideshare.net/alvagante
20

Tp install anything

  • 1.
    tp install anything[anywhere(*)] by Alessandro Franceschi @alvagante
 Founder and CTO @example42 at #cfgmgmtcamp 2017 1
  • 2.
    Tiny Puppet needsPuppet 4 (**) • If Puppet is not installed, just feel reckless and run:
 wget -O - https://bit.ly/installpuppet | sudo bash • Script removes old versions, detects OS, installs official Puppet repos and then Puppet 4 agent • (**) tp is optimised for Puppet4 but has defines compatible with Puppet 3: tp::install3, tp::conf3, tp::dir3 2
  • 3.
    "Piping the internetto my shell !?!?!" • The https://bit.ly/installpuppet URL redirects to a script in example42 control-repo:
 
 https://raw.githubusercontent.com/example42/control- repo/production/bin/puppet_install.sh • Take your time to review it or just fire and forget 
 
 .
 
 3
  • 4.
    Install Tiny Puppet •Usage from shell
 puppet module install example42/tp
 puppet tp setup
 tp
 • In Puppetfile for usage in manifests:
 mod "example42/tp"
 mod "example42/tinydata" 4
  • 5.
    Yet Another UniversalInstaller • al@localhost$ [sudo] tp install app • It supports any application
 on any operating system (*) • It manages package repositories and other dependencies 5
  • 6.
    (*) Any OpeatingSystem? • RedHat and Debian derivatives always supported • Suse, Alpine and other Linux distros need more tiny data • Solaris, BSD, Darwin support partial but possible (may miss data, tests and some features) • Windows: WIP. Currently not supported 6
  • 7.
    Some supported apps •tp install docker-engine • tp install sysdig • tp install gitlab-ce • tp install gitlab-runner • tp install zabbix-agent • tp install virtualbox • tp install puppetserver • tp install puppetdb • tp install awscli • tp install elasticsearch 7
  • 8.
    All the supportedapps • By default TP uses tinydata as source for data about applications • Missing support for apps or OS is just a matter of
 missing data • github.com/example42/tinydata/ • You can use your own data module 8
  • 9.
    Usage on cli:tp test to show apps status • al@localhost$ tp test [app] • Show status of all the applications managed by tp • Can be automated in CI and testing • Custom checks can be easily created • If any of the apps fails, the global exit code is 1
 9
  • 10.
    Usage on cli:tp log to show apps logs • al@localhost$ tp log [app] • tail -f the log[s] of all or the written application • Useful for quick debugging when unaware of log paths • Customisations: tp::install { 'redis': settings_hash => {
 log_file_path => '/opt/redis/redis.logs',
 log_dir_path => '/opt/redis'
 }, }
 10
  • 11.
    Usage on cli:tp list - tp uninstall • al@localhost$ tp uninstall <app> • Removes package and eventually repo file of an app • al@localhost$ tp list • List all the applications tp can install
 
 11
  • 12.
    Usage on manifests:tp::install • Different options for tp::install. Here shown default values. tp::install { 'postgresql': ensure => present, repo => undef, auto_repo => true, auto_prerequisites => false, auto_conf => true, options_hash => {}, settings_hash => {}, cli_enable => true, test_enable => false, puppi_enable => false, } 12
  • 13.
    Usage on manifests:tp::conf • Manage configuration files of an app tp::conf { 'nginx::example.com.conf': ensure => present, base_dir => 'confd', $source => undef, $template => undef, $epp => undef, $content => undef, $base_dir => 'config', $base_file => 'config', $path => undef, $config_file_notify => true, $config_file_require => true, $options_hash => { }, $settings_hash => { }, $data_module => tinydata } 13
  • 14.
    Usage on manifests:Test installed apps • Testing of application can be enable by either one of: tp::test { 'redis': }
 tp::install { 'redis': test_enable => true,
 test_template => 'site/test/nginx.erb', } • A custom test script is created in /etc/tp/test/$app
 Default tests use data from /etc/tp/app/$app 14
  • 15.
    Essential acceptance tests •TP be able to test an app (alternatives) 
 git clone https://github.com/example42/tp-acceptance cd tp-acceptance r10k puppetfile install vagrant status # Test munin on all supported OS bin/test_app.sh munin all acceptance # Test all apps on a specific OS bin/test_app.sh all Centos7 acceptance # Test all appas on all supporte OS
 bin/test_app.sh all all acceptance Compatibility Matrix 15
  • 16.
    It's all aboutTiny Data • Data to manage apps on different OS is on separated tinydata module • Currently 149 applications supported (ls -1 data/ | wc -l) • For each application a data/$app/hiera.yaml file define the hierarchy to use for data lookup (erm, no, Hiera is not actually used): 
 --- :hierarchy: - "%{title}/operatingsystem/%{operatingsystem}-%{operatingsystemmajrelease}" - "%{title}/operatingsystem/%{operatingsystem}" - "%{title}/osfamily/%{osfamily}" - "%{title}/default" - default 16
  • 17.
    It's all aboutTiny Data • Eash data file, in YAML format, contains the info tp uses to manage applications:
 --- redis::settings: package_name: 'redis' service_name: 'redis' config_file_path: '/etc/redis/redis.conf' config_dir_path: '/etc/redis' tcp_port: '6379' pid_file_path: '/var/run/redis.pid' log_file_path: '/var/log/redis.log' log_dir_path: '' process_user: 'redis' process_group: 'redis' 17
  • 18.
    Use cases -In manifests • In local site profiles, when we have to configure applications we know how to configure • In component modules, with local tiny data • tp can replace or coexist with component modules 18
  • 19.
    Use cases -On the command line • Quickly install applications • Check if installed apps are working correctly:
 monitoring, CI, runtime tests. • Troubleshoot locally problems by quickly accessing logs 19
  • 20.
    Thank you • http://www.tiny-puppet.com •@alvagante @example42 • Slides published on http://slideshare.net/alvagante 20