One Click Deployment with Jenkins

One Click Deployment with JenkinsThorsten Rinne I 12th October 201 Mainz, Germany 1, © 201 Mayﬂower GmbH 1

Team Lead / Scrum MasterHead of Open Source Labs@ThorstenRinne Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 2

Who are you? Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 3

Developers? Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 4

Administrators? Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 5

DevOps? Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 6

Continuous Integration? Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 7

How much time do you need for a release? Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 8

More than a year? Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 9

6 - 12 months? Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 10

1 - 6 months? Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 11

2 - 4 weeks? Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 12

1 - 14 days? Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 13

1 - 24 hours? Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 14

Less than 15 minutes? Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 15

15 Minutes! Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 16

Why One ClickDeployment? Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 17

Everything is getting faster. Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 18

Faster feedback fromyour customer / user Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 19

Faster ROI(return on invest) Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 20

Faster responseif the market changes Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 21

Developers are lazy. Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 22

Basic requirements Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 23

Agile software development Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 24

Scrum or Kanban Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 25

Test Driven Development Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 26

Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 27

Continuous Integration Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 28

Continuous Inspection Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 29

Refactoring Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 30

ScrumKanban Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 31

Scrum ContinuousKanban Integration Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 31

Scrum Continuous ContinuousKanban Integration Inspection Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 31

Scrum Continuous ContinuousKanban Integration InspectionContinuous Deployment Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 31

Things you don‘t want. Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 32

Having your SVN repo inyour webserver‘s docroot. Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 33

Using SSH and vi and your root account on the live server. Never. Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 34

Who‘s still doing this? :-) Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 35

What do we need? Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 37

First: a central build server Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 38

Continous Testing with PHPUnit Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 40

Advantages Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 41

Test results at one central page Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 42

Software metrics at one central page Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 43

Errors can be identified fast and then fixed Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I 44

Awareness Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 45

Awareness! Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 46

Continuous Integration in PHP projects Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 47

PHPUnitDocBlox Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 48

Continuous Inspection in PHP projects Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 49

PHPCSPHP Depend PHPMD phpcpd jslint Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 50

jslint? yes! Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 51

Infrastructure Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 52

http://www.ﬂickr.com/photos/nirak/644337888/Building a Cloud-based social network I Mayﬂower GmbH I 1 October 201 I 53 1th 1

One Ubuntu package to rule them all. Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 54

Development VMsJenkins CI Server Selenium ServerMonitoring Server Testing Server Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 55

http://www.ﬂickr.com/photos/nirak/644337888/ Only on production: Frontend Nodes 11.04Backend Nodes 10.04 LTS Building a Cloud-based social network I Mayﬂower GmbH I 1 October 201 I 56 1th 1

http://www.flickr.com/photos/nirak/644337888/Mayflower DMZ amazon AWSEucalyptus Cloud Building a Cloud-based social network I Mayflower GmbH I 1 October 201 I 57 1th 1

http://www.flickr.com/photos/nirak/644337888/ Mayflower DMZ Developer Laptopamazon AWS with Ubuntu VM(Git repo + complete Eucalyptus Cloud software stack) Building a Cloud-based social network I Mayflower GmbH I 1 October 201 I 58 1th 1

http://www.flickr.com/photos/nirak/644337888/Mayflower DMZ Gitorious Server amazon AWS Jenkins CI Server Puppet MasterEucalyptus Cloud Selenium Server Building a Cloud-based social network I Mayflower GmbH I 1 October 201 I 59 1th 1

http://www.flickr.com/photos/nirak/644337888/ 2 Frontnodes 3 BackendnodesMayflower DMZ Development Monitor Staging Monitor amazon AWSEucalyptus Cloud Building a Cloud-based social network I Mayflower GmbH I 1 October 201 I 60 1th 1

http://www.flickr.com/photos/nirak/644337888/ Mayflower DMZ 2 Frontnodes amazon AWS2 Backendnodes Drupal Cloud Eucalyptus Live Monitor Building a Cloud-based social network I Mayflower GmbH I 1 October 201 I 61 1th 1

http://www.flickr.com/photos/nirak/644337888/ ConfigurationManagement done with Building a Cloud-based social network I Mayflower GmbH I 1 October 201 I 62 1th 1

http://www.flickr.com/photos/nirak/644337888/Configuration changes areon every VM in 5 Minutes. #WIN Building a Cloud-based social network I Mayflower GmbH I 1 October 201 I 63 1th 1

Frontnode 1 Frontnode 2 Varnish Varnish nginx nginx PHP 5.3 PHP 5.3Backendnode 1 Backendnode 2 Backendnode 3MySQL Master MySQL Slave RabbitMQ MogileFS MogileFS Apache Solr Memcached Memcached PHP 5.3 PHP 5.3 PHP 5.3 Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 64

Development workﬂow Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 65

1. git workﬂow$ git add Foo.php$ git commit -m ″Bar″$ ant jslint$ ant phpunit ...$ git push Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 66

2. Post commit hookBuild triggered on Jenkins Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 67

Jenkins build checks Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 68

1. PHPUnit Build: UnittestsController Tests (Zend_Test) Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 69

2. Debian Package Build Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 70

3. Deploy to staging4. Deploy staging to live Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 71

At midnight: DocBloxStatic code analysis Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 72

What is a build process? Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 73

ant sync ant cleanant db-update parallelTasks: ant phpunit ant phpcb ant deb Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 74

ant phpunit<target name="phpunit" depends="db-test" description="Run all tests"> <exec executable="phpunit" failonerror="true"> <arg line="--configuration ${basedir}/tests/phpunit-verbose.xml ${basedir}/tests/AllTests.php" /> </exec></target> Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 75

parallelTasks:php_depend phpmd phpunit phpcpd phpcs jslint Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 76

Build Debian Packageant autoloader ant phpunit ant clean ant compilejsant compilecss Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 77

ant deb (I)<target name="deb" depends="autoloader,phpunit-coverage,clean,autoloader,compilejs,compilecss" description="Generate Debian package"> <exec executable="gpg"> <arg value="--armor" /> <arg value="--import" /> <arg value="scripts/build/deb/mapolis.mayflower.de.pgp" /> </exec> <buildnumber /> Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 78

ant deb (II)<exec executable="git" outputproperty="githash"> <arg value="describe" /> <arg value="--always" /> <arg value="--match" /> <arg value="" /></exec><touch file="debian/changelog" /><move file="debian/changelog" tofile="debian/changelog.old" /> Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 79

ant deb (III) <exec executable="git" output="debian/changelog.new"> <arg value="log" /> <arg value="-n" /> <arg value="1" /> <arg value=--format=format:mapolis-web (${build.number}-${githash})unstable; urgency=low%n%n * %s%n%n --%an <%ae> %aD%n%n /> </exec> Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 80

ant deb (IV)<concat destfile="debian/changelog"> <fileset file="debian/changelog.new" /> <fileset file="debian/changelog.old" /></concat><delete file="debian/changelog.new" /><delete file="debian/changelog.old" /> Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 81

ant deb (V) <exec executable="dpkg-buildpackage"> <arg value="-b" /> <arg value="-v${build.number}-${githash}" /> <arg value="-Zlzma" /> <arg value="-z9" /> <arg value="-k78532445" /> <arg value="-pgpg" /> <arg value="-sgpg" /> <arg value="-mMayflower Mapolis Team<mapolis@mayflower.de>" /> </exec> Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 82

ant deb (VI) <move todir="build/deb/"> <fileset dir="${basedir}/../"> <include name="mapolis*.deb" /> <include name="mapolis*.changes" /> </fileset> </move> <exec executable="dpkg-buildpackage"> <arg value="-Tclean" /> </exec></target> Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 83

Deploy Staging ant clean ant deb-signant db-staging-upscp to all hosts Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 84

Deploy Staging to livescp to all hosts Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 85

Disadvantages and problems Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 86

Build time15 min on harddisk massive disk i/o4 min on ramdisk :-) Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 87

Complexity beginning such a infrastructure Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 88

Jenkins:Single point of failure! Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 89

Cool stuffwhich is now possible Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 90

Deploy by customer. :-) Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 91

Jenkins Chuck Norris Plugin Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 92

git bisect start $BAD $GOOD Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 93

Minimum time frombugreport to deploy: 30 minutes Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 94

$ git branch hotfix-2011-08-22$ git checkout hotfix-2011-08-22$ git reset --hard "last stable sha-1"$ git cherry-pick "hotfix sha-1"$ echo build.number=23 > build.number$ ant deb$ scp build/deb/*.deb xyz.amazonaws.com:/opt/repository/$ git checkout master$ git push origin hotfix-2011-08-10 Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 95

LiveDemo

Questions? Continuous Improvement in PHP Projects I Mayﬂower GmbH I 25th February I 97

Thank you very much for your attention!Please rate me @ http://joind.in/talk/view/3904Contact Thorsten Rinne thorsten.rinne@mayflower.de +49 89 242054-31 Mayflower GmbH Mannhardtstr. 6 80538 München Germany © 201 Mayflower GmbH 1

One Click Deployment with Jenkins

by Mayflower GmbH on Oct 12, 2011

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 2

Statistics

One Click Deployment with Jenkins — Presentation Transcript