RPM experiments

Julien Pivotto
Julien PivottoOpen Source Consultant at Inuits
RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016
Julien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien Pivotto
Inuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTT
February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016
whoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoami
• JJJJJJJJJJJJJJJJJulien Pivotto
• rrrrrrrrrrrrrrrrroidelapluie
• SSSSSSSSSSSSSSSSSysadmin
• FFFFFFFFFFFFFFFFFLOSS since 2004
• IIIIIIIIIIIIIIIIInuits since 2011
• CCCCCCCCCCCCCCCCCentOS CfgMgmtSig
Native packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packages
• RRRRRRRRRRRRRRRRReally? It is still needed?
• PPPPPPPPPPPPPPPPPypi/Rubygems/npm/pear…
• OOOOOOOOOOOOOOOOOmnibus/FPM/…
No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .
It is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easy
gem install fpm
ERROR: Could not find a valid gem `fpm'
(>= 0), here is why:
Unable to download data from https://
rubygems.org/ − Errno::ECONNREFUSED:
Connection refused − connect(2) for "
api.rubygems.org" port 443 (https://api
.rubygems.org/latest_specs.4.8.gz)
I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)
• /////////////////usr/bin/fpm
• ./vendor/bin/fpm
• /////////////////usr/local/bin/fpm
• ~~~~~~~~~~~~~~~~~/.ruby/2.4/gems/rubygems/fpm/bin/fpm
Now I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prod
• DDDDDDDDDDDDDDDDDev: version 1.1
• UUUUUUUUUUUUUUUUUAT (deployed next day): 1.2
• PPPPPPPPPPPPPPPPProd (a week after): 2.0. broken.
Open questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questions
• WWWWWWWWWWWWWWWWWhat is installed?
• WWWWWWWWWWWWWWWWWhere does the file come from?
• CCCCCCCCCCCCCCCCCompilation at install time?
RPMRPMRPMRPMRPMRPMRPMRPMRPMRPMRPMRPMRPMRPMRPMRPMRPM
How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011
• hhhhhhhhhhhhhhhhhttps://github.com/roidelapluie/vagrant-
build-mapnik
• /////////////////usr/bin/wget -O /tmp/mapnik.tar.bz2
https://github.com/mapnik-2.0.0.tar.bz2
• /////////////////usr/bin/screen -d -m
/usr/local/bin/build_mapnik.sh
• AAAAAAAAAAAAAAAAAll of that in puppet manifests
I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…
After vagrant provision , wait until the
compilation is done.
You can see the compilations process:
vagrant ssh
sudo screen −r
The packages will be located in the "rpms"
folder.
Building in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VM
cd /opt/mapnik −2.0.0
python scons/scons.py configure PREFIX=${
target} PYTHON_PREFIX=${target}
python scons/scons.py
python scons/scons.py install
FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…
fpm −s dir −n mapnik −v 2.0.0 −−iteration "${
start_date}" 
−C "${target}" −t rpm −−prefix /usr −−url
http://mapnik.org/ 
−−description "Mapnik is a Free Toolkit for
developing mapping applications." 
−−exclude include
Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…
fpm −s dir −n mapnik−devel −v 2.0.0 −−
iteration "${start_date}" 
−C "${target}/include" −t rpm −−prefix /usr/
include −−url http://mapnik.org/ 
−−description "The mapnik−devel package
contains header files for developing
programs using the Mapnik library." 
−−depends "mapnik = 2.0.0"
How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014
• hhhhhhhhhhhhhhhhhttps://github.com/roidelapluie/collectd-
rpm
• SSSSSSSSSSSSSSSSSpec file in collectd upstream
• wwwwwwwwwwwwwwwwwget sources
• iiiiiiiiiiiiiiiiinstall build deps
• rrrrrrrrrrrrrrrrrpmbuild -bb
install build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build deps
egrep "^BuildRequires" collectd −5.5.0/contrib/
redhat/collectd.spec | cut −d `:' −f 2 | tr
−d ` t' | sed `s/,/n/' | uniq | tr −d `
n' | tr `,' ` '|xargs yum install −y
And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?
PackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglove
• hhhhhhhhhhhhhhhhhttp://packaginglove.github.io/packaging-
book/
• CCCCCCCCCCCCCCCCCurrently limited to RPM
• AAAAAAAAAAAAAAAAA pragmatic approach
• DDDDDDDDDDDDDDDDDeb coming
• PPPPPPPPPPPPPPPPPlz contribute
Pick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better tools
• dddddddddddddddddocker instead of vagrant
• yyyyyyyyyyyyyyyyyum/rpm helpers commands
• rrrrrrrrrrrrrrrrreuse/rebuild fedora packages
• ttttttttttttttttthe Cloud ©: copr/travis
RPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpers
• yyyyyyyyyyyyyyyyyum-builddeps install build dependencies
• ssssssssssssssssspectool download the sources
Reuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packages
• ccccccccccccccccclone spec file
• fffffffffffffffffix if needed
dockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdocker
• dddddddddddddddddocker runs on travis
• FFFFFFFFFFFFFFFFFaster than vagrant
• uuuuuuuuuuuuuuuuuseful run options: --rm -v -w -t -i --env
Docker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" image
FROM centos:7
MAINTAINER Julien Pivotto <roidelapluie@inuits
.eu>
RUN yum install −y /usr/bin/wget
RUN yum install −y /usr/bin/spectool
RUN yum install −y /usr/bin/rpmbuild
RUN yum install −y /usr/bin/yum−builddep
RUN yum install −y /usr/bin/make
RUN yum install −y epel−release
Docker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" image
docker build −t el7−build .
Docker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker run
docker run −e XUID="$(id −u)" −−rm −ti −v $PWD
:/work −w /work el7−build ./el7−build.sh
• ------------------rm: do not fill my disk
• -----------------t: allocate a tty
• -----------------i: interactive
• -----------------v $PWD:/work mount current dir on host to
/work in container
• -----------------w /work: use /work as working dir
el7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.sh
useradd −u ${XUID:−1000} rpm
mkdir −p /home/rpm/rpmbuild/SOURCES
spectool −C /home/rpm/rpmbuild/SOURCES −g test
.spec
yum−builddep −y facter.spec
chown −R rpm: /home/rpm
su − rpm −c "rpmbuild −bb /work/facter.spec"
cp −a /home/rpm/rpmbuild/SRPMS/*.src.rpm .
cp −a /home/rpm/rpmbuild/RPMS/*/*.rpm .
Cloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud tools
travis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secrets
• TTTTTTTTTTTTTTTTTravis "secrets" = encrypted variables
• UUUUUUUUUUUUUUUUUse for password, credentials
• VVVVVVVVVVVVVVVVVariables or files
• TTTTTTTTTTTTTTTTThe travis gem
• PPPPPPPPPPPPPPPPPublish to github releases
travis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gem
travis setup releases
travis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.yml
sudo: required
services:
− docker
script:
− make
deploy:
provider: releases
api_key:
secure: Some encrypted stuff
skip_cleanup: true
file_glob: true
file: python−*.rpm
on:
tags: true
coprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcopr
•
hhhhhhhhhhhhhhhhhttps://copr.fedorainfracloud.org/coprs/roidelapl
• fffffffffffffffffedora project
• IIIIIIIIIIIIIIIIInfra to build RPM
• WWWWWWWWWWWWWWWWWeb interface
• pppppppppppppppppublic builds
• llllllllllllllllless limitations than travis
RPM experiments
RPM experiments
RPM experiments
RPM experiments
copr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and travis
travis encrypt−file copr−config −−add
echo copr−config > .gitignore
copr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and travis
sudo: required
services:
− docker
script:
− make
before_install:
− openssl aes−256−cbc −K
$encrypted_2ec4d63b6867_key −iv
$encrypted_2ec4d63b6867_iv
−in copr−config.enc −out copr−config −d
MakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefile
el7:
docker run −e XUID="$(shell id −u)" −−rm −v
$(shell pwd):/work −w /work −ti fedora:23
./dockerscript.sh
dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)
#!/bin/bash
set −x;set −e
dnf install −y /usr/bin/spectool /usr/bin/
rpmbuild /usr/bin/copr−cli /usr/bin/python3
−config
useradd −u ${XUID:−1000} bob
mkdir −p /home/bob/rpmbuild/SOURCES
find . −maxdepth 1 −type f −exec cp −v '{}' /
home/bob/rpmbuild/SOURCES ';'
spectool −C /home/bob/rpmbuild/SOURCES −g /
work/boost.spec
chown −R bob: /home/bob
dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)
su − bob −c "rpmbuild −bs /work/boost.spec"
copr−cli −−debug −−config ./copr−config build
boost /home/bob/rpmbuild/SRPMS/*.src.rpm
• rrrrrrrrrrrrrrrrrpmbuild -bs: build only SRPM
• cccccccccccccccccopr-cli --debug is to avoid the 10 minutes
limitation in travis-ci
ConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusion
ConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusion
• fffffffffffffffffedora: source of pkgs
• tttttttttttttttttravis/copr: Public builds
• dddddddddddddddddocker: reproducible locally
1 of 42

More Related Content

What's hot(20)

An introduction to the Pulp projectAn introduction to the Pulp project
An introduction to the Pulp project
Julien Pivotto9.6K views
DNSSecDNSSec
DNSSec
Julien Pivotto1.9K views
Monitoring at Cloud ScaleMonitoring at Cloud Scale
Monitoring at Cloud Scale
Julien Pivotto8.5K views
Dive into Puppet 4Dive into Puppet 4
Dive into Puppet 4
Julien Pivotto2.7K views
Coworking with gitCoworking with git
Coworking with git
Julien Pivotto2.7K views
Postgresql 9.0 HA at RMLL 2012Postgresql 9.0 HA at RMLL 2012
Postgresql 9.0 HA at RMLL 2012
Julien Pivotto4K views
Linux HA anno 2014Linux HA anno 2014
Linux HA anno 2014
Julien Pivotto6.3K views
Testing your puppet codeTesting your puppet code
Testing your puppet code
Julien Pivotto5.3K views
Making Py Pi Sux Less KeyMaking Py Pi Sux Less Key
Making Py Pi Sux Less Key
Andreas Jung1.8K views
Logstash and friendsLogstash and friends
Logstash and friends
Julien Pivotto13.1K views
Dependency Management With PintoDependency Management With Pinto
Dependency Management With Pinto
Jeffrey Ryan Thalhammer2.2K views
Another 7 tools for your #devops stackAnother 7 tools for your #devops stack
Another 7 tools for your #devops stack
Kris Buytaert1.6K views
Puppet Camp LA 2/19/2015Puppet Camp LA 2/19/2015
Puppet Camp LA 2/19/2015
ice799866 views
VCS for Teamwork - GIT WorkshopVCS for Teamwork - GIT Workshop
VCS for Teamwork - GIT Workshop
Anis Ahmad176 views

More from Julien Pivotto(20)

The O11y ToolkitThe O11y Toolkit
The O11y Toolkit
Julien Pivotto35 views
What's new in Prometheus?What's new in Prometheus?
What's new in Prometheus?
Julien Pivotto15 views
Introduction to Grafana LokiIntroduction to Grafana Loki
Introduction to Grafana Loki
Julien Pivotto187 views
Why you should revisit mgmtWhy you should revisit mgmt
Why you should revisit mgmt
Julien Pivotto10 views
Prometheus and TLS - an IntroductionPrometheus and TLS - an Introduction
Prometheus and TLS - an Introduction
Julien Pivotto15 views
Powerful graphs in GrafanaPowerful graphs in Grafana
Powerful graphs in Grafana
Julien Pivotto12 views
YAML MagicYAML Magic
YAML Magic
Julien Pivotto4.3K views
HAProxy as Egress ControllerHAProxy as Egress Controller
HAProxy as Egress Controller
Julien Pivotto2.8K views
SIngle Sign On with KeycloakSIngle Sign On with Keycloak
SIngle Sign On with Keycloak
Julien Pivotto10K views
Incident Resolution as CodeIncident Resolution as Code
Incident Resolution as Code
Julien Pivotto818 views
An introduction to AnsibleAn introduction to Ansible
An introduction to Ansible
Julien Pivotto581 views

Recently uploaded(20)

ThroughputThroughput
Throughput
Moisés Armani Ramírez31 views
METHOD AND SYSTEM FOR PREDICTING OPTIMAL LOAD FOR WHICH THE YIELD IS MAXIMUM ...METHOD AND SYSTEM FOR PREDICTING OPTIMAL LOAD FOR WHICH THE YIELD IS MAXIMUM ...
METHOD AND SYSTEM FOR PREDICTING OPTIMAL LOAD FOR WHICH THE YIELD IS MAXIMUM ...
Prity Khastgir IPR Strategic India Patent Attorney Amplify Innovation24 views
The Research Portal of Catalonia: Growing more (information) & more (services)The Research Portal of Catalonia: Growing more (information) & more (services)
The Research Portal of Catalonia: Growing more (information) & more (services)
CSUC - Consorci de Serveis Universitaris de Catalunya59 views

RPM experiments

  • 1. RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016 Julien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien Pivotto Inuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTT February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016
  • 2. whoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoami • JJJJJJJJJJJJJJJJJulien Pivotto • rrrrrrrrrrrrrrrrroidelapluie • SSSSSSSSSSSSSSSSSysadmin • FFFFFFFFFFFFFFFFFLOSS since 2004 • IIIIIIIIIIIIIIIIInuits since 2011 • CCCCCCCCCCCCCCCCCentOS CfgMgmtSig
  • 3. Native packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packages • RRRRRRRRRRRRRRRRReally? It is still needed? • PPPPPPPPPPPPPPPPPypi/Rubygems/npm/pear… • OOOOOOOOOOOOOOOOOmnibus/FPM/…
  • 4. No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .
  • 5. It is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easy gem install fpm ERROR: Could not find a valid gem `fpm' (>= 0), here is why: Unable to download data from https:// rubygems.org/ − Errno::ECONNREFUSED: Connection refused − connect(2) for " api.rubygems.org" port 443 (https://api .rubygems.org/latest_specs.4.8.gz)
  • 6. I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/) • /////////////////usr/bin/fpm • ./vendor/bin/fpm • /////////////////usr/local/bin/fpm • ~~~~~~~~~~~~~~~~~/.ruby/2.4/gems/rubygems/fpm/bin/fpm
  • 7. Now I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prod • DDDDDDDDDDDDDDDDDev: version 1.1 • UUUUUUUUUUUUUUUUUAT (deployed next day): 1.2 • PPPPPPPPPPPPPPPPProd (a week after): 2.0. broken.
  • 8. Open questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questions • WWWWWWWWWWWWWWWWWhat is installed? • WWWWWWWWWWWWWWWWWhere does the file come from? • CCCCCCCCCCCCCCCCCompilation at install time?
  • 10. How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011 • hhhhhhhhhhhhhhhhhttps://github.com/roidelapluie/vagrant- build-mapnik • /////////////////usr/bin/wget -O /tmp/mapnik.tar.bz2 https://github.com/mapnik-2.0.0.tar.bz2 • /////////////////usr/bin/screen -d -m /usr/local/bin/build_mapnik.sh • AAAAAAAAAAAAAAAAAll of that in puppet manifests
  • 11. I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young… After vagrant provision , wait until the compilation is done. You can see the compilations process: vagrant ssh sudo screen −r The packages will be located in the "rpms" folder.
  • 12. Building in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VM cd /opt/mapnik −2.0.0 python scons/scons.py configure PREFIX=${ target} PYTHON_PREFIX=${target} python scons/scons.py python scons/scons.py install
  • 13. FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things… fpm −s dir −n mapnik −v 2.0.0 −−iteration "${ start_date}" −C "${target}" −t rpm −−prefix /usr −−url http://mapnik.org/ −−description "Mapnik is a Free Toolkit for developing mapping applications." −−exclude include
  • 14. Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package… fpm −s dir −n mapnik−devel −v 2.0.0 −− iteration "${start_date}" −C "${target}/include" −t rpm −−prefix /usr/ include −−url http://mapnik.org/ −−description "The mapnik−devel package contains header files for developing programs using the Mapnik library." −−depends "mapnik = 2.0.0"
  • 15. How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014 • hhhhhhhhhhhhhhhhhttps://github.com/roidelapluie/collectd- rpm • SSSSSSSSSSSSSSSSSpec file in collectd upstream • wwwwwwwwwwwwwwwwwget sources • iiiiiiiiiiiiiiiiinstall build deps • rrrrrrrrrrrrrrrrrpmbuild -bb
  • 16. install build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build deps egrep "^BuildRequires" collectd −5.5.0/contrib/ redhat/collectd.spec | cut −d `:' −f 2 | tr −d ` t' | sed `s/,/n/' | uniq | tr −d ` n' | tr `,' ` '|xargs yum install −y
  • 17. And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?
  • 19. Pick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better tools • dddddddddddddddddocker instead of vagrant • yyyyyyyyyyyyyyyyyum/rpm helpers commands • rrrrrrrrrrrrrrrrreuse/rebuild fedora packages • ttttttttttttttttthe Cloud ©: copr/travis
  • 20. RPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpers • yyyyyyyyyyyyyyyyyum-builddeps install build dependencies • ssssssssssssssssspectool download the sources
  • 21. Reuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packages • ccccccccccccccccclone spec file • fffffffffffffffffix if needed
  • 22. dockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdocker • dddddddddddddddddocker runs on travis • FFFFFFFFFFFFFFFFFaster than vagrant • uuuuuuuuuuuuuuuuuseful run options: --rm -v -w -t -i --env
  • 23. Docker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" image FROM centos:7 MAINTAINER Julien Pivotto <roidelapluie@inuits .eu> RUN yum install −y /usr/bin/wget RUN yum install −y /usr/bin/spectool RUN yum install −y /usr/bin/rpmbuild RUN yum install −y /usr/bin/yum−builddep RUN yum install −y /usr/bin/make RUN yum install −y epel−release
  • 24. Docker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" image docker build −t el7−build .
  • 25. Docker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker run docker run −e XUID="$(id −u)" −−rm −ti −v $PWD :/work −w /work el7−build ./el7−build.sh • ------------------rm: do not fill my disk • -----------------t: allocate a tty • -----------------i: interactive • -----------------v $PWD:/work mount current dir on host to /work in container • -----------------w /work: use /work as working dir
  • 26. el7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.sh useradd −u ${XUID:−1000} rpm mkdir −p /home/rpm/rpmbuild/SOURCES spectool −C /home/rpm/rpmbuild/SOURCES −g test .spec yum−builddep −y facter.spec chown −R rpm: /home/rpm su − rpm −c "rpmbuild −bb /work/facter.spec" cp −a /home/rpm/rpmbuild/SRPMS/*.src.rpm . cp −a /home/rpm/rpmbuild/RPMS/*/*.rpm .
  • 27. Cloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud tools
  • 28. travis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secrets • TTTTTTTTTTTTTTTTTravis "secrets" = encrypted variables • UUUUUUUUUUUUUUUUUse for password, credentials • VVVVVVVVVVVVVVVVVariables or files • TTTTTTTTTTTTTTTTThe travis gem • PPPPPPPPPPPPPPPPPublish to github releases
  • 29. travis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gem travis setup releases
  • 31. coprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcopr • hhhhhhhhhhhhhhhhhttps://copr.fedorainfracloud.org/coprs/roidelapl • fffffffffffffffffedora project • IIIIIIIIIIIIIIIIInfra to build RPM • WWWWWWWWWWWWWWWWWeb interface • pppppppppppppppppublic builds • llllllllllllllllless limitations than travis
  • 36. copr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and travis travis encrypt−file copr−config −−add echo copr−config > .gitignore
  • 37. copr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and travis sudo: required services: − docker script: − make before_install: − openssl aes−256−cbc −K $encrypted_2ec4d63b6867_key −iv $encrypted_2ec4d63b6867_iv −in copr−config.enc −out copr−config −d
  • 38. MakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefile el7: docker run −e XUID="$(shell id −u)" −−rm −v $(shell pwd):/work −w /work −ti fedora:23 ./dockerscript.sh
  • 39. dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2) #!/bin/bash set −x;set −e dnf install −y /usr/bin/spectool /usr/bin/ rpmbuild /usr/bin/copr−cli /usr/bin/python3 −config useradd −u ${XUID:−1000} bob mkdir −p /home/bob/rpmbuild/SOURCES find . −maxdepth 1 −type f −exec cp −v '{}' / home/bob/rpmbuild/SOURCES ';' spectool −C /home/bob/rpmbuild/SOURCES −g / work/boost.spec chown −R bob: /home/bob
  • 40. dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2) su − bob −c "rpmbuild −bs /work/boost.spec" copr−cli −−debug −−config ./copr−config build boost /home/bob/rpmbuild/SRPMS/*.src.rpm • rrrrrrrrrrrrrrrrrpmbuild -bs: build only SRPM • cccccccccccccccccopr-cli --debug is to avoid the 10 minutes limitation in travis-ci