Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenkins, r10k and Git (Intermediate)

1.
Continuously deliver yourpuppet code with jenkins, r10k and git Toni Schmidbauer October 14, 2014

2.
whoami I SysAdmin@s-itsolutions.at I toni@stderr.at I stderr@jabber.org I http://stderr.at I http://github.com/tosmi

3.
Agenda I Ashort story about con

4.
guration management IWhat is continuous delivery I Tools used to achieve continuous delivery I DEMO I Things to improve

5.
A short storyabout con

6.
guration management (CM) I We manage a very diverse environment of UNIX/Linux Systems (Solaris 10/11 SPARC/i386, AIX, RHEL/CentOS 5/6/7)

7.

8.
guration management (CM) I We manage a very diverse environment of UNIX/Linux Systems (Solaris 10/11 SPARC/i386, AIX, RHEL/CentOS 5/6/7) I We've got around 1000 nodes in total

9.

10.
guration management (CM) I We manage a very diverse environment of UNIX/Linux Systems (Solaris 10/11 SPARC/i386, AIX, RHEL/CentOS 5/6/7) I We've got around 1000 nodes in total I Before CM we had strict standards on how to manage these systems

11.

12.
guration management (CM) I We manage a very diverse environment of UNIX/Linux Systems (Solaris 10/11 SPARC/i386, AIX, RHEL/CentOS 5/6/7) I We've got around 1000 nodes in total I Before CM we had strict standards on how to manage these systems I The problem: count(team members) == count(standards)

13.

14.
guration management (CM) I We manage a very diverse environment of UNIX/Linux Systems (Solaris 10/11 SPARC/i386, AIX, RHEL/CentOS 5/6/7) I We've got around 1000 nodes in total I Before CM we had strict standards on how to manage these systems I The problem: count(team members) == count(standards) I So con

15.
guration management isthe solution to all our problems

16.
The solution toall our problems

17.
The solution toall our problems I Broke our systems

18.
WHY????

19.
Problems with ourold CM system I Deployments sucked

20.
Problems with ourold CM system I Deployments sucked I Deployment via manual tagging and checkout, so mistakes happened I Deployment in stages, but we always had to cross our

21.
ngers

22.

23.
ngers I Testingsucked

24.

25.
ngers I Testingsucked I No Unittests I No acceptance tests

26.

27.
ngers I Testingsucked I No Unittests I No acceptance tests I No immediate feedback if things where OK or not

28.

29.
ngers I Testingsucked I No Unittests I No acceptance tests I No immediate feedback if things where OK or not I Systems installed without CM are hard to bring under CM control

30.

31.
ngers I Testingsucked I No Unittests I No acceptance tests I No immediate feedback if things where OK or not I Systems installed without CM are hard to bring under CM control I Every system was a special case

32.
So what's oursolution?

33.
Continuous delivery Iis a pattern for getting software from development to release 1 1Continuous Delivery: Jez Humble, David Farley

34.
Continuous delivery Iis a pattern for getting software from development to release I this pattern is called the deployment pipeline 1 1Continuous Delivery: Jez Humble, David Farley

35.
The deployment pipeline Commit Stage Automated unit testing Automated acceptance testing Tag release Deployment

36.
Why are weusing continuous delivery I When you automate your deployment,

37.
Why are weusing continuous delivery I When you automate your deployment, I less mistakes will happen and the same mistake will not happen twice

38.
Why are weusing continuous delivery I When you automate your deployment, I less mistakes will happen and the same mistake will not happen twice I less mistakes means less stress when deploying

39.
Why are weusing continuous delivery I When you automate your deployment, I less mistakes will happen and the same mistake will not happen twice I less mistakes means less stress when deploying I less stress means you are going to deploy more often

40.
Why are weusing continuous delivery I When you automate your deployment, I less mistakes will happen and the same mistake will not happen twice I less mistakes means less stress when deploying I less stress means you are going to deploy more often I more deployments means a more exible environment

42.
Tools to builda deployment pipeline

43.
Jenkins I Jenkinsis an Open Source continuous integration server I It's purpose is to execute and monitor jobs I Jobs are shell scripts or any other thing that's executable and returns 0 on success I Many plugins available to extend Jenkins (e.g. git, build-pipeline, monitor) I You can link jobs together, thats our pipeline

44.
Jenkins II

45.
Monitoring with Jenkins

46.
GIT I gitpush triggers the deployment pipeline

47.
GIT I gitpush triggers the deployment pipeline I one central repository for internal modules I gitolite for access control I 3 main branches I development I testing I production I feature branches for new site local modules I hiera data is in the same repository

48.
GIT repository layout I modules/: I where r10k stores external (forge, github) modules I site/: I site local modules, we do not want to share I hiera/: I our hiera yaml

49.
les I Puppetfile: I con

51.
le for r10kthat speci

52.
es which externalmodules we need I Vagrantfile: I To boot a development puppet environment on your local workstation

53.
GIT work ow Puppetdeveloper Development Puppet Master Testing Puppet Master Production Puppet Master Feature Branch Development Branch Testing Branch Production Branch 1 2 3 4 Dev Nodes (7−8) Testing Nodes (~40) Production Nodes 1 2 3 4 1 Features Branches get automatically created on Puppet Master (Dynamic Environments) 2 Development Branch gets deployed on commit via Jenkins 3 Testing Branch gets deployed via GIT tag 4 GIT Jenkins pushing to testing triggers a deployment Production Branch gets deployed via GIT tag pushing to production triggers a deployment It’s all the same for Hiera yaml files!

54.
r10k I atool to deploy puppet environments and modules I every git branch gets deployed to a corresponding puppet environment I it also downloads and installs modules from puppetforge or github I in the current version (1.3.2) dependencies have to be managed manually

55.
Example Puppet

56.
le 1 fo r g e ' f o r g e . p u p p e t l a b s . com' 3 mod ' p u p p e t l a b s / ntp ' , ' 3 . 1 . 2 ' mod ' p u p p e t l a b s / p o s t g r e s q l ' , ' 3 . 4 . 2 ' 5 mod ' p u p p e t l a b s / s t d l i b ' , ' 4 . 3 . 2 ' mod ' p u p p e t l a b s / f i r e w a l l ' , ' 1 . 1 . 3 ' 7 mod ' p u p p e t l a b s / apache ' , ' 1 . 1 . 1 ' mod ' p u p p e t l a b s /lvm ' , ' 0 . 3 . 2 ' 9 mod ' n o s o l u t i o n s /tsm ' , ' 0 . 2 . 2 ' mod ' s a z / sudo ' , ' 3 . 0 . 6 ' 11 mod ' s p i e t t e / s e l i n u x ' , ' 0 . 5 . 4 ' 13 mod ' concat ' , : g i t => ' h t t p s : / / g i t h u b . com/ p u p p e t l a b s / puppe t l absconcat ' , 15 : commit = ' feba3096c99502219043b8161bde299ba65e7b8a ' You are able to pin to a git tag / branch / commit hash

57.
a word ontesting I you must have unit tests for your puppet code: rspec-puppet I for acceptance tests there's puppetlabs/beaker I you need to test everything to get most out of the build pipeline I we test I internal puppet modules I hiera data I puppet con

58.
guration I allinternal modules are required to have rspec tests

59.
rspec-puppet example samplemodule/manifests/init.pp 1 c l a s s samplemodule ( $mes sage = ' d e f a u l tme s s a g e ' ) f n o t i f y f ' samplemes sage ' : 3 mes sage = Thi s i s the sample module , my mes sage i s : $mes sage , g 5 g samplemodule/spec/classes/samplemodules spec.rb 1 r e q u i r e ' s p e c h e l p e r ' 3 d e s c r i b e ' samplemodule ' , : t ype = : c l a s s do c o n t e x t ' wi th d e f a u l t parame t e r s ' do 5 i t f s h o u l d c o n t a i n n o t i f y ( ' samplemes sage ' ) g end 7 end

60.
beaker example 1r e q u i r e ' s p e c h e l p e r a c c e p t a n c e ' 3 d e s c r i b e ' p r o f i l e s : : o s s b a s e c l a s s ' do i t ' s h o u l d work wi th no e r r o r s ' do 5 a p p l y ma n i f e s t ( ' i n c l u d e p r o f i l e s : : os sba s e ' , : c a t c h f a i l u r e s = t r u e ) end 7 end

61.
DEMO

62.
Do try thisat home I You need: I Vagrant from http://vagrantup.com I Virtualbox I Git client I You have to run: I git clone https://github.com/tosmi/puppetcamp2014.git I cd puppetcamp2014 I vagrant up I vagrant ssh

63.
Things we haveto improve I We need more test Systems (CentOS/RHEL/Solaris/AIX?) I We need more acceptance tests I Once again use stages in production I Our documentation

64.
Things puppetlabs shouldimprove I Puppetlabs should package beaker as a rpm/deb whatever, gems suck in production

65.
Summary I ContinuousDelivery is implemented via a deployment pipeline

66.
Summary I ContinuousDelivery is implemented via a deployment pipeline I Within the pipeline everything is automated

67.
Summary I ContinuousDelivery is implemented via a deployment pipeline I Within the pipeline everything is automated I When everything is automated you need to test everything

68.
Summary I ContinuousDelivery is implemented via a deployment pipeline I Within the pipeline everything is automated I When everything is automated you need to test everything I When everything is automated and well tested deploying becomes easy

69.
Summary I ContinuousDelivery is implemented via a deployment pipeline I Within the pipeline everything is automated I When everything is automated you need to test everything I When everything is automated and well tested deploying becomes easy I Tools we are using to implement a deployment pipeline I Jenkins: to execute jobs I GIT: version control I Gitolite: access control and authorization for GIT I r10k: install puppet modules / create puppet environments from branches I rspec-puppet: unittests for puppet I puppetlabs/beaker: acceptance tests for puppet

70.
Thanks for yourattention!

Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenkins, r10k and Git (Intermediate)

More Related Content

What's hot

Viewers also liked

Similar to Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenkins, r10k and Git (Intermediate)

More from Puppet

Recently uploaded

Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenkins, r10k and Git (Intermediate)