Presentation about open source tools to set up continuous integration and continuous deployment. Covers Git, Gitlab, Chef, Vagrant, Jenkins, Gatling, Dashing, TYPO3 Surf and some other tools. Shows some best practices for testing with Behat and Functional Testing.

1. Continuous Integration
& Continuous Delivery
with OpenSource Tools
PHP Usergroup Frankfurt, 2014-11-20

2. Annotated Version
• The grey slides are additional slides to the presentation
to make it easier to understand the presentation on
Slideshare etc.

3. @kaktusmimi

4. Inspired by…
Jo
Dani
Paddy
Seb
Daniel
Christiane

5. Inspiration
• This talk was inspired a lot by the work of my
colleagues at punkt.de
• And a presentation of Sebastian Helzle at FOSSASIA
2014
• http://www.slideshare.net/Sebobo/continuous-delivery-
with-open-source-tools

6. Motivation
Version Control
Tests
Jenkins
Deployment
Configuration Management
Monitoring
Summary & Further Reading

7. Are you
agile?

8. Agile Requirements
• From a Developers Point of View
• Delivering Increments often
• Being able to do that very fast
• Being able to be sure, the Increments work

9. Feedback

10. Improving through
Feedback
• Feedback is great tool for learning
• Continuous Integration can give developers
feedback on their code „automatically“
• Can come from tests, metrics, linting
• The more specific the longer it takes
• Feedback from different stages of CI

11. Safety

12. Safe Delivery
• Continuous Integration assures that your code is
working as expected
• We don’t see bugs only in production, but during the
CI cycle
• Let’s you deploy your code „with a better feeling“

13. ① Git

14. First Step Towards Continuous Integration
!
if you don’t use it yet
USE GIT

15. Source: http://nvie.com/posts/a-successful-git-branching-model/

16. Git Workflows
• There are several Workflows for git
• Centralized Workflow
• Feature Branch Workflow
• Gitflow Workflow
• Forking Workflow
• Overview: https://www.atlassian.com/git/tutorials/
comparing-workflows/

18. Git GUIs
• „Not Nerdy“
• If you cannot manage to handle Git on a command line
• Use a GUI!
• Better, more concise commits
• Better overview of project history
• Eg. SourceTree, Tower, TortoiseGit

19. git pull --rebase
Image Source: Atlassian

20. Rebase instead of merge
• Gives you a clean Git commit history
• Sometimes it makes sense to still keep your feature
branches and only rebase inside a branch and not
when merging

22. Gitlab
• OpenSource Git Server
• Written in Ruby
• Available as a ready-to-run virtual machine on Bitnami
• https://bitnami.com/stack/gitlab/virtual-machine

23. Mind the Seb!

24. Code Review with Gitlab
and RSS Feeds
• See blog post of Seb
• http://www.mind-the-seb.de/blog/codereview-made-
simple.html

25. Gitlab Server
git pull / push
Developer
Reviewer
RSS Feeds

26. ② Tests

27. TDD is dead. Long live testing.
!
Test-first fundamentalism is like abstinence-only sex ed: An
unrealistic, ineffective morality campaign for self-loathing and
shaming.

28. Use the right Tests
• Unit Tests are a great tool for business logic
• Unit Tests don’t work for testing legacy code
• Functional Tests or testing „from outside“ might give
you a lot more feedback with less effort

29. Functional Testing
• Test that your code is working
• Not how it is working
• More black-box testing
• Incorporate your Database
• End-to-End testing

30. /** @test */
public function createCustomerCreatesExpectedCustomer() {
$this->customerService->createCustomer('4711', 'MickeyMouse');
$this->persistenceManager->persistAll();
$this->assertSame(
'4711',
$this->customerRepository->findAll()->getFirst()->getId()
);
}
!

32. Behat
• Behat can be used for different kinds of tests
• Unit-Tests, Functional Tests, UI Tests
• Enables non-technical people to write tests
• Let’s developers implement the required logic
• Uses Gherkin Language
• Easily extendible using Contexts

33. Feature: Language menu
In order to switch the language on the website
As a website user
I need to be able to select the language in a menu
Background:
Given I am in "desktop" layout
And I am on "/"
Scenario: Switching the language from english to
german
When I follow "Language"
And I wait for 500 milliseconds
And I follow "Deutsch"
Then I should be on "/de.html"

34. /**
* Given I am in "desktop" layout
*
* @When I am in :layout layout
*/
public function iResizeTheWindowToLayout($layout) {
if (array_key_exists($layout, $this->screenSizes)) {
$currentLayout = $this->screenSizes[$layout];
$this->getSession()->getDriver()->resizeWindow(
$currentLayout['width'], $currentLayout['height'], 'current');
return TRUE;
}
throw new Exception(sprintf('Layout "%s" not defined', $layout));
}

35. ③ Jenkins

36. Jenkins
• Java Application
• Can be deployed into Tomcat or Standalone
• There are good reasons for Standalone!
• Many Plugins available
• Basically a Task Runner

38. Deployment Stage!
!
* Demo
Deployment
* Production
Deployment
Acceptance Stage!
!
* Functional Tests
* Frontend Tests
Triggered
by
Commit
Commit Stage!
!
* Clone Repository
* Build Project
* Unit Tests
green green
Triggered by
Scheduler

39. Version Everything!
• Even Jenkins Configuration
• And most of all: Jenkins Jobs!!!
• Make them part of your project!

40. ④ Surf

41. TYPO3 Surf
• A Remote Server Automation and Deployment Tool
• Written in PHP
• Based on the TYPO3 Flow Framework
• Can be deployed as a Flow Package or Standalone

42. Simple Deployment
• Build Locally (e.g. on Jenkins)
• Ship as Package / Files (e.g. with rsync)
• Don’t use git remotely
• Too many things can go wrong!
• Make sure to have a Rollback!

43. More sophisticated…
• Build Locally
• Run Tests
• Ship / Transfer
• Run Smoketests
• Only switch Release if Tests pass

44. Surf Concepts
$workflow !! ! set up HOW to deploy!
$node !! ! ! ! set up WHERE to deploy!
$application ! set up WHAT to deploy
$deployment !! glue it all together

45. <?php
$workflow = new TYPO3SurfDomainModelSimpleWorkflow();
$node = new TYPO3SurfDomainModelNode('staging');
$node->setOptions(array(
'username' => '<username>',
'composerCommandPath' => '/usr/local/bin/composer'
));
$node->setHostname($host);
$application = new TYPO3SurfApplicationTYPO3Neos('<project_name>');
$application->setOptions(array(
'repositoryUrl' => '<git_remote_url>',
'keepReleases' => 5,
'packageMethod' => 'git',
'transferMethod' => 'rsync',
'updateMethod' => NULL
));
$application->setDeploymentPath('<deployment_path>');
$application->addNode($node);
$deployment->addApplication($application);
$deployment->onInitialize(function() use ($workflow, $application, $project) {
$workflow->afterStage('migrate', 'codecoon:importContent');
});

46. /var/apache/XXXXXX/staging/XXXXX/releases$ ls!
!
20141105100124!
20141105102935!
20141112095924!
20141118055114!
20141118072225!
20141119041835!
20141120045634!
current -> ./20141119041835!
previous -> ./20141118072225!
next -> ./20141120045634!
!
!
/var/apache/XXXXXX$ ls -la!
!
htdocs -> staging/XXXXXX/releases/current/htdocs

47. ⑤ Chef
&Vagrant

48. chef-solo
Developer Machine / Laptop
ssh
vagrant up
Workspace
Project 1
Virtual Machine
Gitlab Server
git
pull / push
ssh
mysql
samba
nfs
samba
nfs
git
ssh
http
http
chef run

49. Our Vagrant Approach
* We have VirtualBox and Vagrant running on our laptop
* We start a virtual machine and run Chef inside this machine
* Chef sets up
* Our services (Apache, PHP, MySQL, …)
* Our projects / webspaces
* We can now use our familiar tools to work on our projects
* It feels like „working locally“ although we have a Sandbox
* We cannot crash the host OS when crashing the Dev-Environment
* Think about packages
* Think about different software versions for different projects
* Think about how long it takes to re-install your laptop…

50. Workspace
Project 1
Provision Jenkins Server
Configure Apache
Configure PHP
& MySQL
Provision Projects
Workspace
Project 2
Workspace
Project 2

51. Workspace
Project 1
Workspace
Project 2
Workspace
Project 2
Provision Projects
Provision Jenkins Server
Configure Apache
Configure PHP
& MySQL
We set up our Projects
on Jenkins

52. Workspace
Project 1
Workspace
Project 2
Workspace
Project 2
Provision Projects
Provision Jenkins Server
Configure Apache
Configure PHP
& MySQL
We set up our Projects
on Jenkins
Automatically!!!

53. Chef & Jenkins
There shall be 2 Chef runs
1. The one that provisions our Jenkins Server [not yet finished…]
2. The one that provisions the projects inside our Jenkins Server
Those Projects are „publicly“ available
* We can use them as „normal“ Website (e.g. for Review and Manual Testing)
* We can run UI tests on them
!
In the future we want to change this to a master/slave approach.

54. Teaser

55. More on Chef & Vagrant
• There is another presentation on Chef & Vagrant
• http://de.slideshare.net/mimiknoll/vagrant-fossasia-
2014

56. ⑥ Monitoring

57. Monitoring Stage
• Check for Website to be alive
• Use simple Tools
• I.e. Selenium has many Hiccups —> False Positives!
• Send Notifications in Case of Emergency
• Email, Jabber, SMS
• Ops: Nagios — Devs: Jenkins
• Devs can do more sophisticated Things with Tests

58. Performance Stage
• Check Realtime Performance of your Website
• Gatling is a great Tool for writing Performance Tests
• Use a Dashboard to visualize your Results
• Dashing can help you to create Dashboards
• Raspberry Pi is great Hardware to display Dashboards

61. Summary

62. gitlab triggers
Version Control Build
build succeeds
Commit Stage Unit Tests
UI Tests
tests pass
Acceptance Stage
DB Tests
tests pass
tests pass
build succeeds
Deployment Stage
Build locally
Demo Stage
Production Stage
rsync
rsync
git push
Developer
Feedback

63. git shell / ssh
Gitlab Server
Jenkins Server
Workspace
Project 1
Workspace
Project 2
Workspace
Project 2
provision
Selenium Server
http / RESTful Services
Repository
Project 1
Workspace
Project 2
Workspace
Project 3
http / UI Testing
git clone
Webspace
Project 1
Production 1
Webspace
Project 2
Production 2
Webspace
Project 3
Production 3
ssh / rsync

65. Check it out
www.codecoon.com
Make it fun to code again

66. Questions

67. Some Questions
• „Do you still have time to write code?“
• Yes - this toolchain makes it a lot easier for us
• „This is to complex for me!“
• You can pick out modules that work for you

68. More about Chef?!?
DevOps Meetup
Frankfurt, 2014-12-08
„Testing Infrastructure
Code with Chef“

69. https://about.gitlab.com/
http://jenkins-ci.org/
https://phpunit.de/
https://www.getchef.com/
https://www.vagrantup.com/
http://www.seleniumhq.org/
http://typo3.org/additional-products/surf

70. http://www.sourcetreeapp.com/download/
http://dashing.io/
http://gatling.io/

71. http://www.martinfowler.com/articles/
continuousIntegration.html
http://www.thoughtworks.com/insights
http://david.heinemeierhansson.com/2014/tdd-is-dead-long-
live-testing.html
http://www.rbcs-us.com/documents/Why-Most-Unit-
Testing-is-Waste.pdf
http://www.mind-the-seb.de/blog/codereview-made-simple.
html

72. https://www.atlassian.com/git/tutorials/comparing-workflows/
forking-workflow
http://nvie.com/posts/a-successful-git-branching-model/