Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Practical continuous
quality gates for
development process
Andrii Soldatenko
26-27 February 2016
@a_soldatenko
About me:
• Software Engineer at DataRobot
• Speaker at many PyCon’s
• Speaker at Selenium Camp’15
Motivation
History of CI
Illustration by Andrew Rae
CI 101
Martin Fowler
CI 101
It was first named and proposed by Grady Booch in 1991
CI 101
"At regular intervals, the process of
"continuous integration" yields
executable releases that grow in
functionalit...
The role of automated
testing in CI
Quick Start CI
Run Jenkins
docker run -p 8080:8080 -p 50000:50000  

-v /$USER/home:/var/jenkins_home jenkins
Jenkins 

Plugin Manager
Tip: use plugins.txt
cat Dockerfile
FROM jenkins
COPY plugins.txt /usr/share/jenkins/ref/
RUN /usr/local/bin/plugins.sh /u...
Tip: use plugins.txt
$ cat plugins.txt
credentials:1.18
github-oauth:0.22.2
shiningpanda:0.22
xunit:1.100
postbuild-task:1...
Wuala we see plugins
$ cat plugins.txt
credentials:1.18
github-oauth:0.22.2
shiningpanda:0.22
xunit:1.100
postbuild-task:1...
Create New Jobs
Create New Jobs
1. Manually Create new job
2. Jenkins Remote access API
Manually create new job
Manually create new job
2. Jenkins REST API
2. What can you do with
Jenkins REST API?
• retrieve information from Jenkins for
programmatic consumption.
• trigger a ne...
Create/copy jobs
<project>
<actions/>
<description/>
<keepDependencies>false</keepDependencies>
<properties/>
<scm class="...
XML
Jenkins Job Builder
YAML
Install JJB
git clone https://git.openstack.org/
openstack-infra/jenkins-job-builder
cd jenkins-job-builder
python setup.p...
JJB Tip #1:

Configure settings
cat /etc/jenkins_jobs/jenkins_jobs.ini-
sample
[job_builder]
ignore_cache=True
[jenkins]
u...
JJB Tip #2:

Test a job definition
jenkins-jobs test templates002.yaml
JJB Tip #3:

Updating/Delete Jenkins Jobs
# Update Jenkins Job

jenkins-jobs --conf jenkins_jobs.ini 

update templates002...
JJB Example
yaml template:
- job:
name: regression_testing
parameters:
- string:
name: one_param
default: origin/master
bu...
Pros and Cons JJB:
Cons JJB:
publishers:
- raw:
xml: |
<hudson.tasks.junit.JUnitResultArchiver>
<testResults>reports/*.xml</testResults>
<kee...
Integration with Github
Integration with Slack
Slack and JJB
- project:
name: foo
properties:
- slack:
notify-start: true
include-test-summary: true
show-commit-list: tr...
Avoiding Flaky Tests
Test Result Analyzer
Flaky Test Analyzer
Parse build logs
Get all failed
build ids
Return list of 

flaky tests
https://goo.gl/OZm0rA
Workspace Cleanup
Example: npm install
Thanks @gavinjoyce
Final Thoughts
• to be continuous integration
• use yaml, git and jjb if possible
• use more practical tips
• read more bo...
Future
P.S. books to read:
Thank You
Your feedback is very
valuable and will help me
to continuously improve
my talks

https://asoldatenko.com
We are hiring
Please send you CV to

andrii.soldatenko@datarobot.com
Questions
?
Upcoming SlideShare
Loading in …5
×

Practical continuous quality gates for development process

5,517 views

Published on

There are a lot of books and publications about the continuous integration in the world. But in my experience it’s difficult to find information about how to open quality gates between automated tests and to continuous integration practice to in your current project. After reading several articles and even a couple of books you will understand how to work with it. But what next? I will share with you practical tips and tricks on how to lift iron curtain to your automated tests before a continuous quality practice today. It is for this reason why I am pleased to share with you my acquired experience in my presentation.

Published in: Internet

Practical continuous quality gates for development process

  1. 1. Practical continuous quality gates for development process Andrii Soldatenko 26-27 February 2016 @a_soldatenko
  2. 2. About me: • Software Engineer at DataRobot • Speaker at many PyCon’s • Speaker at Selenium Camp’15
  3. 3. Motivation
  4. 4. History of CI Illustration by Andrew Rae
  5. 5. CI 101 Martin Fowler
  6. 6. CI 101 It was first named and proposed by Grady Booch in 1991
  7. 7. CI 101 "At regular intervals, the process of "continuous integration" yields executable releases that grow in functionality at every release. ... It is through these milestones that management can measure progress and quality, and hence anticipate, identify, and then actively attach risks on an ongoing basis." --
  8. 8. The role of automated testing in CI
  9. 9. Quick Start CI
  10. 10. Run Jenkins docker run -p 8080:8080 -p 50000:50000 
 -v /$USER/home:/var/jenkins_home jenkins
  11. 11. Jenkins 
 Plugin Manager
  12. 12. Tip: use plugins.txt cat Dockerfile FROM jenkins COPY plugins.txt /usr/share/jenkins/ref/ RUN /usr/local/bin/plugins.sh /usr/share/ jenkins/ref/plugins.txt
  13. 13. Tip: use plugins.txt $ cat plugins.txt credentials:1.18 github-oauth:0.22.2 shiningpanda:0.22 xunit:1.100 postbuild-task:1.8 ghprb:1.30.4 jenkins-multijob-plugin:1.20
  14. 14. Wuala we see plugins $ cat plugins.txt credentials:1.18 github-oauth:0.22.2 shiningpanda:0.22 xunit:1.100 postbuild-task:1.8 ghprb:1.30.4 jenkins-multijob-plugin:1.20
  15. 15. Create New Jobs
  16. 16. Create New Jobs 1. Manually Create new job 2. Jenkins Remote access API
  17. 17. Manually create new job
  18. 18. Manually create new job
  19. 19. 2. Jenkins REST API
  20. 20. 2. What can you do with Jenkins REST API? • retrieve information from Jenkins for programmatic consumption. • trigger a new build • create/copy jobs
  21. 21. Create/copy jobs <project> <actions/> <description/> <keepDependencies>false</keepDependencies> <properties/> <scm class="hudson.scm.NullSCM"/> <canRoam>true</canRoam> <disabled>false</disabled> <blockBuildWhenDownstreamBuilding>false</ blockBuildWhenDownstreamBuilding> <blockBuildWhenUpstreamBuilding>false</ blockBuildWhenUpstreamBuilding> <triggers/> <concurrentBuild>false</concurrentBuild> <builders/> <publishers/> <buildWrappers/> </project>
  22. 22. XML
  23. 23. Jenkins Job Builder YAML
  24. 24. Install JJB git clone https://git.openstack.org/ openstack-infra/jenkins-job-builder cd jenkins-job-builder python setup.py install jenkins-jobs --help usage: jenkins-jobs …
  25. 25. JJB Tip #1:
 Configure settings cat /etc/jenkins_jobs/jenkins_jobs.ini- sample [job_builder] ignore_cache=True [jenkins] user=andriisoldatenko password=password url=192.168.99.100:8080
  26. 26. JJB Tip #2:
 Test a job definition jenkins-jobs test templates002.yaml
  27. 27. JJB Tip #3:
 Updating/Delete Jenkins Jobs # Update Jenkins Job
 jenkins-jobs --conf jenkins_jobs.ini 
 update templates002.yaml
 
 # Delete Jenkins Job
 jenkins-jobs --conf jenkins_jobs.ini 
 delete simple
  28. 28. JJB Example yaml template: - job: name: regression_testing parameters: - string: name: one_param default: origin/master builders: - shining-panda: publishers: - email: recipients: andrii@test.io
  29. 29. Pros and Cons JJB:
  30. 30. Cons JJB: publishers: - raw: xml: | <hudson.tasks.junit.JUnitResultArchiver> <testResults>reports/*.xml</testResults> <keepLongStdio>false</keepLongStdio> <testDataPublishers> <hudson.plugins.sauce__ondemand.SauceOnDemandReportPublish er /> </testDataPublishers> <healthScaleFactor>1.0</healthScaleFactor> </hudson.tasks.junit.JUnitResultArchiver>
  31. 31. Integration with Github
  32. 32. Integration with Slack
  33. 33. Slack and JJB - project: name: foo properties: - slack: notify-start: true include-test-summary: true show-commit-list: true room: '#jenkins' token: secret team-domain: example.com include-custom-message: true custom-message: message publishers: - slack: team-domain: example.com auth-token: secret build-server-url: https://jenkins.example.com room: '#jenkins' https://goo.gl/F513DU
  34. 34. Avoiding Flaky Tests
  35. 35. Test Result Analyzer
  36. 36. Flaky Test Analyzer Parse build logs Get all failed build ids Return list of 
 flaky tests https://goo.gl/OZm0rA
  37. 37. Workspace Cleanup
  38. 38. Example: npm install Thanks @gavinjoyce
  39. 39. Final Thoughts • to be continuous integration • use yaml, git and jjb if possible • use more practical tips • read more books
  40. 40. Future
  41. 41. P.S. books to read:
  42. 42. Thank You Your feedback is very valuable and will help me to continuously improve my talks
 https://asoldatenko.com
  43. 43. We are hiring Please send you CV to
 andrii.soldatenko@datarobot.com
  44. 44. Questions ?

×