Yet Another Continuous Integration Story
Upcoming SlideShare
Loading in...5
×
 

Yet Another Continuous Integration Story

on

  • 1,543 views

PHP User Group Minsk #4

PHP User Group Minsk #4

Statistics

Views

Total Views
1,543
Slideshare-icon Views on SlideShare
1,534
Embed Views
9

Actions

Likes
3
Downloads
7
Comments
0

2 Embeds 9

http://www.linkedin.com 5
https://twitter.com 4

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Yet Another Continuous Integration Story Yet Another Continuous Integration Story Presentation Transcript

    • Yet Another Continuous Integration Story
    • Who I Am ● Anton Serdyuk ● Technical Leader at Startup Labs, Inc. ● anton.serdyuk@gmail.com ● anton.serdyuk@itstartuplabs.com ● @anton_serdyuk ● Skype: serdyuk.anton
    • Disclaimer
    • Disclaimer Projects are different ● Technical requirements ● Business requirements ● Length ● Budget ● Load ● Data size ● etc
    • Disclaimer Teams are different ● Level ● Size ● Structure ● etc
    • Disclaimer There is no silver bullet ● Worked for us != will work for you ● Adopt patterns and principles rather than the full process
    • Who We Are
    • Who We Are Project ● b2b system ● Unexpected downtime is very expensive ● Bugs in production are also very expensive because of the reputation loss ● Small amount of traffic (But expensive) ● 1.5 years ● 4-15 developers (+ layout, + qa) ● 15 Gb Database (Grows not very fast)
    • Who We Are Technologies ● PHP 5.4 ● Java 7 ● PostgreSQL ● RabbitMQ
    • Who We Are Frameworks ● Hibernate ● JUnit ● Symfony2 ● Silex ● Behat ● PHPUnit
    • Goals and Principles
    • Goals and Principles Agile ● Small changes ● Frequent releases ● Early feedback
    • What Is NOT Our Way
    • What Is NOT Our Way Git Flow
    • What Is NOT Our Way Feature Branches feature1 master feature2
    • What Is NOT Our Way Feature Branches Active feature branches usage example
    • What Is NOT Our Way Release Branches 1.1.x 1.0.x master
    • What Is NOT Our Way master/staging/develop branches develop master
    • What Is NOT Our Way Release Tags v1.1.1 v1.1.2 master
    • What IS Our Way
    • What IS Our Way Continuous Integration ● Push every day ● Build every commit ● Run all tests on every build ● Fix red builds quickly
    • What IS Our Way One Branch master
    • What IS Our Way Branch by Abstraction ● Commit switch-off for this feature ● Develop, integrate ● … your code in production now, but switched-off ● Develop, integrate ● Test ● Switch-on this feature in production
    • What IS Our Way Autotests ● QAs write behat scenarios ● Developers implement ● Every feature covered by behat scenarios ● Great regression tests suite - easy refactoring, less bugs ● + ~100% development cost
    • What IS Our Way Every Build Is Release Candidate ● Do not push if state after commit is not suitable for production
    • What IS Our Way Some Numbers ● ~1000 behat scenarios (unit tests are not counted) ● ~15 minutes build duration ● 2 build servers (Intel Xeon, 32 Gb RAM, SSD) ● 5-6 build agents for tests (VMs) ● 3 build agents for build phase (VMs) ● ~15 builds per day ● 2 releases per week
    • Build Process
    • Build Process Overview build automatic tests manual tests load and prod migrate tests production
    • Build Process Overview bamboo pull build git repository push developers build build agent agents test test test test agents test agents agents agents QA server pre prod prod artifacts apt repository puppet master tune release engineer (architect/admin)
    • Build Process Use The Same Artifacts build build agent agents artifacts apt repository test test test agents test agents agents agents QA server pre prod prod
    • Build Process Autoconfigure All Environments test test test agents test agents agents agents QA server puppet master pre prod prod tune release engineer (architect/admin)
    • Build Process Deploy by Pressing a Button bamboo test test test agents test agents agents agents QA server pre prod prod
    • Build Phase
    • Build Phase Goals ● Create artifacts which will be used in all phases ● Run unit tests
    • Build Phase Technologies ● Phing ● Maven ● deb-packages (dpkg-buildpackage) ● PHPUnit ● JUnit
    • Build Phase Example
    • Automatic Test Phase
    • Automatic Test Phase Goals ● Run functional tests
    • Automatic Test Phase Deploy to Completely Clean System
    • Automatic Test Phase Run Tests in Parallel agent 1 behat process 1 application 1 agent 2 behat process 2 application 2 ... agent 5 behat process 5 application 5
    • Automatic Test Phase Run Tests in Parallel feature 1 feature 2 job 01 agent 1 feature 3 feature 4 feature 5 job 02 feature 6 agent 2 feature 7 feature 8 job 03 feature 9 ... ... feature 50 feature 51 Job 15 agent 3
    • Automatic Test Phase Run Tests in Parallel github: StartupLabs/behat-partial-runner ● bin/behat --count 15 --current 1 ● bin/behat --count 15 --current 2 ● bin/behat --count 15 --current 3 ● bin/behat --count 15 --current 4 ● bin/behat --count 15 --current 5 ● … ● bin/behat --count 15 --current 15
    • Automatic Test Phase Write Fast Tests ● Continuously speed-up your tests ● Use curl instead of Selenium ● Put initial data directly to DB ● etc
    • Automatic Test Phase Write Fast Tests Merge scenarios which do not modify data Feature: Posts pagination Feature: Posts pagination Background: Background: Given I am logged in as admin Given I am logged in as admin And there is many posts And there is many posts Scenario: Pagination works Scenario: All pagination tests Given I am at the posts list # Scenario: Pagination works Then I should see pagination Given I am at the posts list Then I should see pagination Scenario: Link “last” works Given I am at the posts list # Scenario: Link “last” works When I click at the “last” link When I click at the “last” link Then I should see last page Then I should see last page
    • Automatic Test Phase Write Fast Tests Translate outline to single scenarios with TableNode Feature: Post validation Feature: Post validation Background: Background: Given I am logged in as admin Given I am logged in as admin And I am on post add page And I am on post add page Scenario Outline: Validation works Scenario: Validation works When I enter <value> into <field> Then I should see such errors with such input Then I should see error <error> | field | value | error Examples: | field | value | error | | title | | Can not be empty | | tags | Should be letters | | %%%% | | title | | | Can not be empty | tags | %%%% | Should be letters |
    • Automatic Test Phase Write Fast Tests Parallel test execution @parallel Feature: Login server behat process 1 Background: Given I am registered user alex = alex1 alex = alex2 application behat process 2 | username | alex | | password | 123 | Scenario: I can login Given I am at the login page When I fill form with ● Many problems with race conditions | username | alex | | password | 123 | Then I should be logged in ● Not worked for us :(
    • Manual Test Phase
    • Manual Test Phase Goals ● Run manual tests ● Place where QA see new functionality
    • Manual Test Phase ● Deploy to completely clean system
    • Load and Prod Migrate Tests
    • Load and Prod Migrate Tests Load Tests Goals ● Catch race conditions ● Catch serious performance degradation when database grows or after adding new functionality
    • Load and Prod Migrate Tests Prod Migrate Tests Goals ● Test migrations on real data ● Test migrations time on real data
    • Load and Prod Migrate Tests ● Migrate load test env to new version ● Rollback migrate test env to production snapshot
    • Deploy to Production
    • Deploy to Production ● nb-prod - small production copy used by other teams for integration purposes
    • Problems
    • Problems Red Builds for a Long Time ● QA can not deploy new version to staging ● New commits make new tests red like a snowball ● Nobody knows who are responsible for all those failed tests Solutions: ● If build can not be fixed in 15 minutes - revert commit ● Autoreverts on red builds (?)
    • Problems Long Builds ● Nobody wants to wait green build ● Nobody cares about green build after commit ● Red builds for a long time Solutions: ● Speedup tests ● Parallel build ● Add more build servers
    • Problems Blinking Failures ● Failures are trustless - “it is not my fault - it is blinking, just do rebuild” ● Time consuming Solutions: ● Eliminate them ASAP ● Do not just run “Rebuild failed jobs”
    • Key Points, Again
    • Key Points, Again Autotests ● There is no sense in CI without tests ● Easy refactoring ● Less bugs
    • Key Points, Again Fast Builds ● Long builds => More changes => Red builds ● Fast tests ● Parallel run ● Hacks in tests ● etc
    • Key Points, Again One branch ● There are no builds from other branches ● There are no hotfixes ● Easy deploy and build pipeline
    • Todo
    • Todo ● Rollback strategy ● Use Bamboo deployment
    • Inspired By ● Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation (Jez Humble, David Farley) ● http://timothyfitz.com/
    • Questions?