0
Jenkins User Conference Boston #jenkinsconf
Jenkins + Gitlab +
RabbitMQ + Symfony2
+ Phing
Brandon Mueller
K12 Services, I...
Jenkins User Conference Boston #jenkinsconf
About Me
• Web Developer
• Lead Engineer @ K12 Services Inc
• Total Jenkins Ne...
Jenkins User Conference Boston #jenkinsconf
Jenkins User Conference Boston #jenkinsconf
Jenkins User Conference Boston #jenkinsconf
How I Sold CI to Executives
• We are humans, we make mistakes
• Deploying upda...
Jenkins User Conference Boston #jenkinsconf
Why Jenkins?
• Open Source; free as in freedom!
• Battle tested
• Huge communi...
Jenkins User Conference Boston #jenkinsconf
GitLab
https://gitlab.com/gitlab-org/gitlab-ce/blob/master/README.md
Jenkins User Conference Boston #jenkinsconf
What Is
Open source software to collaborate on code
• Manage Git repositories ...
Jenkins User Conference Boston #jenkinsconf
Why GitLab?
• Open Source; Free as in freedom!
• Runs on the Linux + Nginx + M...
Jenkins User Conference Boston #jenkinsconf
Why GitLab? Why Not GitHub or BitBucket?
• GitLab could be called a GitHub clo...
Jenkins User Conference Boston #jenkinsconf
Jenkins + GitLab
GitLab Merge Request Builder Plugin
Jenkins User Conference Boston #jenkinsconf
Jenkins User Conference Boston #jenkinsconf
Jenkins User Conference Boston #jenkinsconf
Jenkins + GitLab
GitLab Build Now Plugin
Jenkins User Conference Boston #jenkinsconf
Jenkins User Conference Boston #jenkinsconf
Jenkins + GitLab Workflow
• Dev creates a feature branch following a
feature/i...
Jenkins User Conference Boston #jenkinsconf
Jenkins + GitLab Workflow
app_dev Jenkins Project
(develop & master branch)
ap...
Jenkins User Conference Boston #jenkinsconf
What is Phing?
PHing Is Not GNU make; it's a PHP project build
system or build...
Jenkins User Conference Boston #jenkinsconf
In other words
• PHING in a build tool for automating tasks.
• Uses a build XM...
Jenkins User Conference Boston #jenkinsconf
Phing in Dev
• All the things that Jenkins does for the build, we
can do local...
Jenkins User Conference Boston #jenkinsconf
Jenkins User Conference Boston #jenkinsconf
Jenkins User Conference Boston #jenkinsconf
Jenkins + Phing
Phing Plugin
• It works ;)
• Jenkins executes Phing targets as...
Jenkins User Conference Boston #jenkinsconf
Jenkins + Phing
•
Jenkins User Conference Boston #jenkinsconf
Code Quality
Jenkins User Conference Boston #jenkinsconf
PHP Code Sniffer
• Reports on coding standards (PSR-2)
• Outputs standard chec...
Jenkins User Conference Boston #jenkinsconf
CSSlint & JShint
• Same thing, only for CSS and JS
Jenkins User Conference Boston #jenkinsconf
Build Summary
• Make env/build specific config files
• Composer
• Bower
• Twig...
Jenkins User Conference Boston #jenkinsconf
Jenkins User Conference Boston #jenkinsconf
What About Deployment?
• Publish Over SSH Plugin
• This gets the files where t...
Jenkins User Conference Boston #jenkinsconf
RabbitMQ
• Post build task triggers a bash script that sends
a RabbitMQ messag...
Jenkins User Conference Boston #jenkinsconf
A brief explanation of how RabbitMQ works
• A producer posts messages to excha...
Jenkins User Conference Boston #jenkinsconf
Types of exchanges
Direct
• Message goes directly to queue
Fanout
• Every cons...
Jenkins User Conference Boston #jenkinsconf
Message Example
Exchange ci
Routing Key ci.k12fs2_dev.bug_216_login
Payload
{
...
Jenkins User Conference Boston #jenkinsconf
Topic
Exchange
Develop
Branch
Queue
Feature
Branch
Branch
Queue
Dev App
Server...
Jenkins User Conference Boston #jenkinsconf
Symfony2 + RabbitMQ
• High quality bundle called OldSoundRabbitMq
Bundle
• Con...
Jenkins User Conference Boston #jenkinsconf
OldSouundRabbitMqBundle Configz
Jenkins User Conference Boston #jenkinsconf
Consumer Codez
Jenkins User Conference Boston #jenkinsconf
Bringing It All Together
Application Server
Vagrant
Jenkins
<Phing>
Vagrant
<P...
Jenkins User Conference Boston #jenkinsconf
All The Jenkins Things
• Environment Injector Plugin
• Checkstyle Plugin
• Git...
Jenkins User Conference Boston #jenkinsconf
Things I Feel Are missing
• RabbitMQ Post Build Producer plugin
• Better Notif...
Jenkins User Conference Boston #jenkinsconf
All (most of) the things
• https://www.gitlab.com/gitlab-ce/
• http://www.phin...
Jenkins User Conference Boston #jenkinsconf
Thank You To Our Sponsors
Platinum Gold
Silver
Upcoming SlideShare
Loading in...5
×

Juc boston2014.pptx

1,397

Published on

Chaining together Jenkins, GitLab, RabbitMQ, Symfony2 & PHING to create a continuous deployment toolchain

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,397
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
14
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Transcript of "Juc boston2014.pptx"

  1. 1. Jenkins User Conference Boston #jenkinsconf Jenkins + Gitlab + RabbitMQ + Symfony2 + Phing Brandon Mueller K12 Services, Inc @fatmuemoo June 18, 2014 #jenkinsconf
  2. 2. Jenkins User Conference Boston #jenkinsconf About Me • Web Developer • Lead Engineer @ K12 Services Inc • Total Jenkins Newbie (~6 months)
  3. 3. Jenkins User Conference Boston #jenkinsconf
  4. 4. Jenkins User Conference Boston #jenkinsconf
  5. 5. Jenkins User Conference Boston #jenkinsconf How I Sold CI to Executives • We are humans, we make mistakes • Deploying updates manually takes time and it is error prone • Tests and code quality assurance is inconsistent without CI • Automagically test and inspect before every update • Upfront cost can be significant, especially for a newbie, but pays off in the long run
  6. 6. Jenkins User Conference Boston #jenkinsconf Why Jenkins? • Open Source; free as in freedom! • Battle tested • Huge community • Lots of plugins • In the Ubuntu repository • Easy to deploy and update • Other CI servers are either closed source or too immature
  7. 7. Jenkins User Conference Boston #jenkinsconf GitLab https://gitlab.com/gitlab-org/gitlab-ce/blob/master/README.md
  8. 8. Jenkins User Conference Boston #jenkinsconf What Is Open source software to collaborate on code • Manage Git repositories with fine grained access controls that keep your code secure • Perform code reviews and enhance collaboration with merge requests • Each project can also have an issue tracker and a wiki • Used by more than 100,000 organizations, GitLab is the most popular solution to manage Git repositories on- premises • Completely free and open source (MIT Expat license) • Powered by Ruby on Rails source: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/README.md
  9. 9. Jenkins User Conference Boston #jenkinsconf Why GitLab? • Open Source; Free as in freedom! • Runs on the Linux + Nginx + MySql + Ruby stack • Easy to deploy and update; Digital Ocean has a push button install • Predictable release schedule • Awesome access controls
  10. 10. Jenkins User Conference Boston #jenkinsconf Why GitLab? Why Not GitHub or BitBucket? • GitLab could be called a GitHub clone; Fairly intuitive UI and workflow • We have control of the service; no surprises • GitHub can get pricey when you have more than a few repositories • We still use a free BitBucket account for a third tier backups • We still use GitHub for Open Source code https://github.com/K12Labs
  11. 11. Jenkins User Conference Boston #jenkinsconf Jenkins + GitLab GitLab Merge Request Builder Plugin
  12. 12. Jenkins User Conference Boston #jenkinsconf
  13. 13. Jenkins User Conference Boston #jenkinsconf
  14. 14. Jenkins User Conference Boston #jenkinsconf Jenkins + GitLab GitLab Build Now Plugin
  15. 15. Jenkins User Conference Boston #jenkinsconf
  16. 16. Jenkins User Conference Boston #jenkinsconf Jenkins + GitLab Workflow • Dev creates a feature branch following a feature/issue_desc naming schema • ex: feature/124_my_awsome_thing • Dev writes some awesome code, sends a merge request when ready • Merge Request Builder plugin builds the project when MR is opened and when the MR is updated • When a MR is accepted, Jenkins triggers a build b/c code was pushed to the develop branch
  17. 17. Jenkins User Conference Boston #jenkinsconf Jenkins + GitLab Workflow app_dev Jenkins Project (develop & master branch) app_test Jenkins Project (feature branches) MR is created or updated MR is accepted in GitLab
  18. 18. Jenkins User Conference Boston #jenkinsconf What is Phing? PHing Is Not GNU make; it's a PHP project build system or build tool based on ​Apache Ant. You can do anything with it that you could do with a traditional build system like GNU make, and its use of simple XML build files and extensible PHP "task" classes make it an easy-to-use and highly flexible build framework. source: phing.info
  19. 19. Jenkins User Conference Boston #jenkinsconf In other words • PHING in a build tool for automating tasks. • Uses a build XML file. • Lots of pre made PHING targets. • You can tie together pre made string targets • You can easily execute shell commands • You can write more complex phing targets in PHP • PHING is Awesome. Use PHING.
  20. 20. Jenkins User Conference Boston #jenkinsconf Phing in Dev • All the things that Jenkins does for the build, we can do locally • No surprises if a build fails (usually)
  21. 21. Jenkins User Conference Boston #jenkinsconf
  22. 22. Jenkins User Conference Boston #jenkinsconf
  23. 23. Jenkins User Conference Boston #jenkinsconf Jenkins + Phing Phing Plugin • It works ;) • Jenkins executes Phing targets as part of the build process • Passes in environment variables
  24. 24. Jenkins User Conference Boston #jenkinsconf Jenkins + Phing •
  25. 25. Jenkins User Conference Boston #jenkinsconf Code Quality
  26. 26. Jenkins User Conference Boston #jenkinsconf PHP Code Sniffer • Reports on coding standards (PSR-2) • Outputs standard checkstyle xml formats (among other formats)
  27. 27. Jenkins User Conference Boston #jenkinsconf CSSlint & JShint • Same thing, only for CSS and JS
  28. 28. Jenkins User Conference Boston #jenkinsconf Build Summary • Make env/build specific config files • Composer • Bower • Twig.js • Assetic • Run Analytics/Tests • Symfony Cache
  29. 29. Jenkins User Conference Boston #jenkinsconf
  30. 30. Jenkins User Conference Boston #jenkinsconf What About Deployment? • Publish Over SSH Plugin • This gets the files where they need to go, but I need more than that • Deploy Plugin • Java Specific Platforms • Bash Script • HUH?
  31. 31. Jenkins User Conference Boston #jenkinsconf RabbitMQ • Post build task triggers a bash script that sends a RabbitMQ message • Jenkins no longer cares about where the app is being deployed • Many consumers can receive the message - deploy the app on as many servers as needed
  32. 32. Jenkins User Conference Boston #jenkinsconf A brief explanation of how RabbitMQ works • A producer posts messages to exchanges. • Exchanges in RabbitMQ route the message to where it needs to go. • A consumer connects to a queue to consume a message. • There are different types of exchanges
  33. 33. Jenkins User Conference Boston #jenkinsconf Types of exchanges Direct • Message goes directly to queue Fanout • Every consumer connected to queue gets the message Topic • The exchange routes to different queues (or different exchanges) based on a routing key http://www.rabbitmq.com/getstarted.html
  34. 34. Jenkins User Conference Boston #jenkinsconf Message Example Exchange ci Routing Key ci.k12fs2_dev.bug_216_login Payload { status : "complete", gitlabMergeRequestId : "193", gitlabSourceBranch : "bug/216_login", gitlabSourceName : "k12fs2/app_k12fs2", gitlabSourceRepository : "", gitlabTargetBranch : "develop", APP_VERSION : "2.0.2.233", BUILD_CAUSE : "GITLABCAUSE", BUILD_DISPLAY_NAME : "#348", BUILD_ID : "2014-06-06_14-15-52", BUILD_NUMBER : "348", BUILD_TAG : "jenkins-k12fs2_dev-348", BUILD_URL : "http://example.com/job/k12fs2_dev/348/", GIT_BRANCH : "origin/develop", GIT_COMMIT : "351790406a56aad54b2afe8fdf6a60b3737441d8", GIT_PREVIOUS_COMMIT: "351790406a56aad54b2afe8fdf6a60b3737441d8", GIT_URL : "http://example.com/", HUDSON_URL : "http://example.com/", JOB_NAME : "k12fs2_dev", JOB_URL : "http://example.com/job/k12fs2_dev/" }
  35. 35. Jenkins User Conference Boston #jenkinsconf Topic Exchange Develop Branch Queue Feature Branch Branch Queue Dev App Server Prod App Server Staging/QA App Server Master Branch Fanout Exchange Prod App Server Prod App Server routing key: ci.project_name.branch_name app_dev.origin_develop app_dev.origin_master
  36. 36. Jenkins User Conference Boston #jenkinsconf Symfony2 + RabbitMQ • High quality bundle called OldSoundRabbitMq Bundle • Consumer code for deploying the application lives in our Symfony app • Consumer code drops down to the command line to execute PHING targets etc
  37. 37. Jenkins User Conference Boston #jenkinsconf OldSouundRabbitMqBundle Configz
  38. 38. Jenkins User Conference Boston #jenkinsconf Consumer Codez
  39. 39. Jenkins User Conference Boston #jenkinsconf Bringing It All Together Application Server Vagrant Jenkins <Phing> Vagrant <Phing> Vagrant <Phing> Dev machines use Phing to deploy the application internally and run analytics Application server(s) consume message from Jenkins and use Phing to deploy the application GitLab GitLab notifies Jenkins of push. Jenkins Pull down repository <Phing> Jenkins uses Phing to build the application I’m done building! Message Consumer <Phing> Application Server Consumer <Phing> Application Server Consumer <Phing>
  40. 40. Jenkins User Conference Boston #jenkinsconf All The Jenkins Things • Environment Injector Plugin • Checkstyle Plugin • Gitlab Build Now Plugin • Gitlab Merge Request Builder Plugin • Hudson Post Build Task
  41. 41. Jenkins User Conference Boston #jenkinsconf Things I Feel Are missing • RabbitMQ Post Build Producer plugin • Better Notifications from Jenkins to GitLab CE (Coming Soon?!?) • Artifact publishing inside of Jenkins • Dynamic build plan for feature branches or I’m too incompetent to develop myself or the solution exists and I just couldn’t figure it out it
  42. 42. Jenkins User Conference Boston #jenkinsconf All (most of) the things • https://www.gitlab.com/gitlab-ce/ • http://www.phing.info/ • https://github.com/squizlabs/PHP_CodeSniffer • https://github.com/CSSLint/csslint • http://www.jshint.com/ • https://getcomposer.org/ • http://bower.io/ • http://www.rabbitmq.com/ • https://github.com/videlalvaro/rabbitmqbundle • http://www.rabbitmq.com/getstarted.html
  43. 43. Jenkins User Conference Boston #jenkinsconf Thank You To Our Sponsors Platinum Gold Silver
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×