SlideShare a Scribd company logo
CI with Docker and
Jenkins
Francesco Bruni
Club degli sviluppatori - Bari, May ‘19
@brunifrancesco
Who am I
•MSc Telecommunication Engineer @ Poliba
•Despite a Java background, I prefer Python whenever
possible
•I'm not a computer scientist

Continuous Integration
•Keep integrating multiple times a day the central code
repository;
•Run as more tests as you can to ensure nothing broke
down;
•Be ready to replicate environment.
Why CI matters
•Continuous testing/building for all teammates;
•Let someone do trivial jobs.
CI advantage
“Continuous Integration doesn’t get rid
of bugs, but it does make them
dramatically easier to find and remove”
M. Fowler
CI requirements
•Single source repo;
•Automated building;
•Self testing build;
•Build fast;
•Run tests in a production env clone;
•Automate deployment.
CI cycle
Develop Feature
Integrate in the
main code base
Test/build/label
Checkout code
Correct detected
iussues
CI and TDD
TDD
CI
CI cycle
•Integrating code should be automated;
•Multiple levels testing;
•Devs must not checkout bugged code;
•Devs must not deploy untested code;
CI drawbacks
•Setting files;
•Setting all the stuff up requires time and patience.
CI natural evolution:

Continuous Delivery
•If a commit builds (all tests succeed), deliver it;
•Keep delivery time at minimum;
•Multiple deliveries for multiple versions.
Know your tools
•Test everything;
•Keep your sources/setting files versioned;
•Containerize everything;
•Automate as much as you can.
Test everything
•Test your sources, checking coverage;
•Test for writing new features;
•Test your application settings;
•If you want to deliver software, check external
requirements.
Smarter use of VCs
•Keep code versioned: application settings/sources/dep
indexes;
•Keep env settings versioned: scripts and configuration
files;
•Enjoy version control well defined ‘flows’ :)
Be containerized
•Develop solutions in well separated envs;
•Ship software as main part of env;
•Don’t play with global deps.
•‘Restore’ basic envs to speed up env creation process.
What’s Docker
•Wrap many services in many
containers;
•Create, play with and destroy
containers;
•Save and easily replicate
isolated containers.
Dockerfiles
# pull base image.
FROM java:8
# maintainer details
MAINTAINER Francesco Bruni "francesco.bruni@stasbranger.com"
# update packages and install maven
RUN 
export DEBIAN_FRONTEND=noninteractive && 
sed -i 's/# (.*multiverse$)/1/g' /etc/apt/sources.list && 
apt-get update && 
apt-get -y upgrade && 
apt-get install -y vim wget curl git maven ssh nano python-pip
# attach volumes
VOLUME /volume/git
# create working directory and set some stuff up
RUN mkdir -p /local/git
WORKDIR /local/git
RUN mkdir -p /root/.ssh
ADD id_rsa /root/.ssh/id_rsa
RUN chmod 700 /root/.ssh/id_rsa
RUN echo "Host <ip>ntStrictHostKeyChecking non" >> /root/.ssh/config
COPY .m2 /root/.m2
COPY run.sh run.sh
RUN chmod a+x run.sh
# run this when container starts
CMD ["./run.sh"]
Automate procedures
•Repetitive tasks should be performed by automated
agents;
•Once you teach the agent what to do, it’s up to him to
fulfill the job.
What’s Jenkins
•Automated procedures worker;
•Run tasks, report final status,
log activities;
•Multiple plugin for multiple
features;
•Use it just for playing too :)
Deploy w/ Docker w/o
Jenkins
•Test code (or at least run a set of tests);
•Push code to repo;
•SSH in the server, enter in container;
•Pull changes and restart required services.
•Check if it works! :)
Practical session
•Integrate new code into the
main branch;
•Test all the stuff;
•Deploy it :)
What we’ll need
•A GIT web hook;
•Set private “credentials”;
•A tool to test configuration settings;
•A container to run the test suite;
•A container to run the new application;
•A pipeline.
GIT web hook
• Bind an action (tipically run a script) to GIT events;
• Notify Jenkins new code has been pushed over the
‘develop’ branch.
#! /bin/bash
read oldrev newrev _branch
tail=$(git log -1 --pretty=format:'%h %cn: %s%b' $newrev)
curl http://<ip>:8080/job/QW/build?
token=vH3YKmrRR5KOT4aeAOAV&commitMessage=
$tail&cause=JenkinsDeploy
Merge/Test
configurations
import os



main_path = os.path.join("src",
"main","resources","application.properties")

test_path = os.path.join("src",
"test","resources","application.properties")



for element in (main_path, test_path,):

with open(element) as old_main:

data = map(lambda line: line.strip(), old_main.readlines())

data[6] = data[6].replace("192.168.1.75", "mysql")

data[6] = data[6].replace("<myIp>", "mysql")





with open(element, "wb") as new_main:

new_main.write(“n".join(data))
Connection().quick_connect("user", "password", “db_test", "<myIp>")
Connection().quick_connect("user", "password", "db", "mysql")



Create the container
to run test suite
Run tests and see if they fails :)
Deploy the new
version
• Create the new container and run the new version;
• The new container should replace an existing one.
Going on production
No way, do some manual testing and start a new CD
build
All togheter
node {
stage 'Stage 1'
sh 'docker run --rm --link mysqlC:mysql --name sms-checklink-container
sms-checklink'
stage 'Stage 2'
sh 'docker rm -f sms-test-container || echo 'no container to delete''
sh 'docker run --rm --name sms-test-container --link mysqlC:mysql sms-
test'
stage 'Stage 3'
sh 'docker run --rm --name sms-merge-container sms-merge'
stage 'Stage 4'
sh 'docker rm -f sms-staging-container || echo 'no container to delete''
sh 'docker run -d -p 8433:8080 --name sms-staging-container --link
mysqlC:mysql sms-staging'
}
Conclusions
• Improve configuration settings handling;
• Reduce testing time;
• Bind Jenkins job to commit messages;
• Improve git-flow integration;
• Docker Compose :(
• Deploy project on production via manual tests.
References
Continuous Delivery

Reliable Software Releases through Build, Test,
and Deployment Automation
by Jez Humble and David Farley
Continuous Integration
by Martin Fowler

http://martinfowler.com/articles/continuousIntegration.html
References
Docker for beginners
https://scotch.io/tutorials/getting-started-with-docker
Meet Jenkins



https://wiki.jenkins-ci.org/display/JENKINS/Meet+Jenkins
Docker Explained: Using Dockerfiles to Automate
Building of Images
https://www.digitalocean.com/community/tutorials/
docker-explained-using-dockerfiles-to-automate-
building-of-images
Let’s see it @ work
Questions?
@brunifrancesco

More Related Content

What's hot

CI, CD with Docker, Jenkins and Tutum
CI, CD with Docker, Jenkins and TutumCI, CD with Docker, Jenkins and Tutum
CI, CD with Docker, Jenkins and Tutum
Sreenivas Makam
 
Building Jenkins Pipelines at Scale
Building Jenkins Pipelines at ScaleBuilding Jenkins Pipelines at Scale
Building Jenkins Pipelines at Scale
Julien Pivotto
 
Automate App Container Delivery with CI/CD and DevOps
Automate App Container Delivery with CI/CD and DevOpsAutomate App Container Delivery with CI/CD and DevOps
Automate App Container Delivery with CI/CD and DevOps
Daniel Oh
 
Simply your Jenkins Projects with Docker Multi-Stage Builds
Simply your Jenkins Projects with Docker Multi-Stage BuildsSimply your Jenkins Projects with Docker Multi-Stage Builds
Simply your Jenkins Projects with Docker Multi-Stage Builds
Eric Smalling
 
7 Habits of Highly Effective Jenkins Users
7 Habits of Highly Effective Jenkins Users7 Habits of Highly Effective Jenkins Users
7 Habits of Highly Effective Jenkins Users
Jules Pierre-Louis
 
Jenkins & IaC
Jenkins & IaCJenkins & IaC
Jenkins & IaC
HungWei Chiu
 
DockerCon SF 2015: Enabling Microservices @Orbitz
DockerCon SF 2015: Enabling Microservices @OrbitzDockerCon SF 2015: Enabling Microservices @Orbitz
DockerCon SF 2015: Enabling Microservices @Orbitz
Docker, Inc.
 
Javaone 2014 - Git & Docker with Jenkins
Javaone 2014 - Git & Docker with JenkinsJavaone 2014 - Git & Docker with Jenkins
Javaone 2014 - Git & Docker with Jenkins
Andy Pemberton
 
Testing with Docker
Testing with DockerTesting with Docker
Testing with Docker
toffermann
 
Seven Habits of Highly Effective Jenkins Users (2014 edition!)
Seven Habits of Highly Effective Jenkins Users (2014 edition!)Seven Habits of Highly Effective Jenkins Users (2014 edition!)
Seven Habits of Highly Effective Jenkins Users (2014 edition!)
Andrew Bayer
 
Jenkins days workshop pipelines - Eric Long
Jenkins days workshop  pipelines - Eric LongJenkins days workshop  pipelines - Eric Long
Jenkins days workshop pipelines - Eric Long
ericlongtx
 
Developer South Coast 2018: Docker on Windows - The Beginner's Guide
Developer South Coast 2018: Docker on Windows - The Beginner's GuideDeveloper South Coast 2018: Docker on Windows - The Beginner's Guide
Developer South Coast 2018: Docker on Windows - The Beginner's Guide
Elton Stoneman
 
Ci with jenkins docker and mssql belgium
Ci with jenkins docker and mssql belgiumCi with jenkins docker and mssql belgium
Ci with jenkins docker and mssql belgium
Chris Adkin
 
Automated Deployment Pipeline using Jenkins, Puppet, Mcollective and AWS
Automated Deployment Pipeline using Jenkins, Puppet, Mcollective and AWSAutomated Deployment Pipeline using Jenkins, Puppet, Mcollective and AWS
Automated Deployment Pipeline using Jenkins, Puppet, Mcollective and AWS
Bamdad Dashtban
 
An Open-Source Chef Cookbook CI/CD Implementation Using Jenkins Pipelines
An Open-Source Chef Cookbook CI/CD Implementation Using Jenkins PipelinesAn Open-Source Chef Cookbook CI/CD Implementation Using Jenkins Pipelines
An Open-Source Chef Cookbook CI/CD Implementation Using Jenkins Pipelines
Steffen Gebert
 
Next-gen DevOps engineering with Docker and Kubernetes by Antons Kranga
Next-gen DevOps engineering with Docker and Kubernetes by Antons KrangaNext-gen DevOps engineering with Docker and Kubernetes by Antons Kranga
Next-gen DevOps engineering with Docker and Kubernetes by Antons Kranga
JavaDayUA
 
(Declarative) Jenkins Pipelines
(Declarative) Jenkins Pipelines(Declarative) Jenkins Pipelines
(Declarative) Jenkins Pipelines
Steffen Gebert
 
Jenkins tutorial
Jenkins tutorialJenkins tutorial
Jenkins tutorial
Mamun Rashid, CCDH
 
Building kubectl plugins with Quarkus | DevNation Tech Talk
Building kubectl plugins with Quarkus | DevNation Tech TalkBuilding kubectl plugins with Quarkus | DevNation Tech Talk
Building kubectl plugins with Quarkus | DevNation Tech Talk
Red Hat Developers
 
Developer South Coast 2018: Modernizing .NET Apps with Docker
Developer South Coast 2018: Modernizing .NET Apps with DockerDeveloper South Coast 2018: Modernizing .NET Apps with Docker
Developer South Coast 2018: Modernizing .NET Apps with Docker
Elton Stoneman
 

What's hot (20)

CI, CD with Docker, Jenkins and Tutum
CI, CD with Docker, Jenkins and TutumCI, CD with Docker, Jenkins and Tutum
CI, CD with Docker, Jenkins and Tutum
 
Building Jenkins Pipelines at Scale
Building Jenkins Pipelines at ScaleBuilding Jenkins Pipelines at Scale
Building Jenkins Pipelines at Scale
 
Automate App Container Delivery with CI/CD and DevOps
Automate App Container Delivery with CI/CD and DevOpsAutomate App Container Delivery with CI/CD and DevOps
Automate App Container Delivery with CI/CD and DevOps
 
Simply your Jenkins Projects with Docker Multi-Stage Builds
Simply your Jenkins Projects with Docker Multi-Stage BuildsSimply your Jenkins Projects with Docker Multi-Stage Builds
Simply your Jenkins Projects with Docker Multi-Stage Builds
 
7 Habits of Highly Effective Jenkins Users
7 Habits of Highly Effective Jenkins Users7 Habits of Highly Effective Jenkins Users
7 Habits of Highly Effective Jenkins Users
 
Jenkins & IaC
Jenkins & IaCJenkins & IaC
Jenkins & IaC
 
DockerCon SF 2015: Enabling Microservices @Orbitz
DockerCon SF 2015: Enabling Microservices @OrbitzDockerCon SF 2015: Enabling Microservices @Orbitz
DockerCon SF 2015: Enabling Microservices @Orbitz
 
Javaone 2014 - Git & Docker with Jenkins
Javaone 2014 - Git & Docker with JenkinsJavaone 2014 - Git & Docker with Jenkins
Javaone 2014 - Git & Docker with Jenkins
 
Testing with Docker
Testing with DockerTesting with Docker
Testing with Docker
 
Seven Habits of Highly Effective Jenkins Users (2014 edition!)
Seven Habits of Highly Effective Jenkins Users (2014 edition!)Seven Habits of Highly Effective Jenkins Users (2014 edition!)
Seven Habits of Highly Effective Jenkins Users (2014 edition!)
 
Jenkins days workshop pipelines - Eric Long
Jenkins days workshop  pipelines - Eric LongJenkins days workshop  pipelines - Eric Long
Jenkins days workshop pipelines - Eric Long
 
Developer South Coast 2018: Docker on Windows - The Beginner's Guide
Developer South Coast 2018: Docker on Windows - The Beginner's GuideDeveloper South Coast 2018: Docker on Windows - The Beginner's Guide
Developer South Coast 2018: Docker on Windows - The Beginner's Guide
 
Ci with jenkins docker and mssql belgium
Ci with jenkins docker and mssql belgiumCi with jenkins docker and mssql belgium
Ci with jenkins docker and mssql belgium
 
Automated Deployment Pipeline using Jenkins, Puppet, Mcollective and AWS
Automated Deployment Pipeline using Jenkins, Puppet, Mcollective and AWSAutomated Deployment Pipeline using Jenkins, Puppet, Mcollective and AWS
Automated Deployment Pipeline using Jenkins, Puppet, Mcollective and AWS
 
An Open-Source Chef Cookbook CI/CD Implementation Using Jenkins Pipelines
An Open-Source Chef Cookbook CI/CD Implementation Using Jenkins PipelinesAn Open-Source Chef Cookbook CI/CD Implementation Using Jenkins Pipelines
An Open-Source Chef Cookbook CI/CD Implementation Using Jenkins Pipelines
 
Next-gen DevOps engineering with Docker and Kubernetes by Antons Kranga
Next-gen DevOps engineering with Docker and Kubernetes by Antons KrangaNext-gen DevOps engineering with Docker and Kubernetes by Antons Kranga
Next-gen DevOps engineering with Docker and Kubernetes by Antons Kranga
 
(Declarative) Jenkins Pipelines
(Declarative) Jenkins Pipelines(Declarative) Jenkins Pipelines
(Declarative) Jenkins Pipelines
 
Jenkins tutorial
Jenkins tutorialJenkins tutorial
Jenkins tutorial
 
Building kubectl plugins with Quarkus | DevNation Tech Talk
Building kubectl plugins with Quarkus | DevNation Tech TalkBuilding kubectl plugins with Quarkus | DevNation Tech Talk
Building kubectl plugins with Quarkus | DevNation Tech Talk
 
Developer South Coast 2018: Modernizing .NET Apps with Docker
Developer South Coast 2018: Modernizing .NET Apps with DockerDeveloper South Coast 2018: Modernizing .NET Apps with Docker
Developer South Coast 2018: Modernizing .NET Apps with Docker
 

Similar to Continuous Integration/Deployment with Docker and Jenkins

Dependencies Managers in C/C++. Using stdcpp 2014
Dependencies Managers in C/C++. Using stdcpp 2014Dependencies Managers in C/C++. Using stdcpp 2014
Dependencies Managers in C/C++. Using stdcpp 2014
biicode
 
CICD_1670665418.pdf
CICD_1670665418.pdfCICD_1670665418.pdf
CICD_1670665418.pdf
edsonJeancarloRuedaS
 
Continuous Integration with Open Source Tools - PHPUgFfm 2014-11-20
Continuous Integration with Open Source Tools - PHPUgFfm 2014-11-20Continuous Integration with Open Source Tools - PHPUgFfm 2014-11-20
Continuous Integration with Open Source Tools - PHPUgFfm 2014-11-20
Michael Lihs
 
Jenkins Pipelines
Jenkins PipelinesJenkins Pipelines
Jenkins Pipelines
Steffen Gebert
 
Webinar: Creating an Effective Docker Build Pipeline for Java Apps
Webinar: Creating an Effective Docker Build Pipeline for Java AppsWebinar: Creating an Effective Docker Build Pipeline for Java Apps
Webinar: Creating an Effective Docker Build Pipeline for Java Apps
Codefresh
 
Gianluca Varisco - DevOoops (Increase awareness around DevOps infra security)
Gianluca Varisco - DevOoops (Increase awareness around DevOps infra security)Gianluca Varisco - DevOoops (Increase awareness around DevOps infra security)
Gianluca Varisco - DevOoops (Increase awareness around DevOps infra security)
Codemotion
 
Dev ops meetup
Dev ops meetupDev ops meetup
Dev ops meetup
Bigdata Meetup Kochi
 
Road to Opscon (Pisa '15) - DevOoops
Road to Opscon (Pisa '15) - DevOoopsRoad to Opscon (Pisa '15) - DevOoops
Road to Opscon (Pisa '15) - DevOoops
Gianluca Varisco
 
Comment améliorer le quotidien des Développeurs PHP ?
Comment améliorer le quotidien des Développeurs PHP ?Comment améliorer le quotidien des Développeurs PHP ?
Comment améliorer le quotidien des Développeurs PHP ?
AFUP_Limoges
 
ASP.NET 5 auf Raspberry PI & docker
ASP.NET 5 auf Raspberry PI & dockerASP.NET 5 auf Raspberry PI & docker
ASP.NET 5 auf Raspberry PI & docker
Jürgen Gutsch
 
Continuous Integration & Development with Gitlab
Continuous Integration & Development with GitlabContinuous Integration & Development with Gitlab
Continuous Integration & Development with Gitlab
Ayush Sharma
 
Automating Software Development Life Cycle - A DevOps Approach
Automating Software Development Life Cycle - A DevOps ApproachAutomating Software Development Life Cycle - A DevOps Approach
Automating Software Development Life Cycle - A DevOps Approach
Akshaya Mahapatra
 
Test-Driven Infrastructure with Chef
Test-Driven Infrastructure with ChefTest-Driven Infrastructure with Chef
Test-Driven Infrastructure with Chef
Michael Lihs
 
Prescriptive System Security with InSpec
Prescriptive System Security with InSpecPrescriptive System Security with InSpec
Prescriptive System Security with InSpec
All Things Open
 
Prescriptive Security with InSpec - All Things Open 2019
Prescriptive Security with InSpec - All Things Open 2019Prescriptive Security with InSpec - All Things Open 2019
Prescriptive Security with InSpec - All Things Open 2019
Mandi Walls
 
Deployment Tactics
Deployment TacticsDeployment Tactics
Deployment Tactics
Ian Barber
 
Using Chef InSpec for Infrastructure Security
Using Chef InSpec for Infrastructure SecurityUsing Chef InSpec for Infrastructure Security
Using Chef InSpec for Infrastructure Security
Mandi Walls
 
DCSF 19 Building Your Development Pipeline
DCSF 19 Building Your Development Pipeline  DCSF 19 Building Your Development Pipeline
DCSF 19 Building Your Development Pipeline
Docker, Inc.
 
InSpec For DevOpsDays Amsterdam 2017
InSpec For DevOpsDays Amsterdam 2017InSpec For DevOpsDays Amsterdam 2017
InSpec For DevOpsDays Amsterdam 2017
Mandi Walls
 
The Future of Security and Productivity in Our Newly Remote World
The Future of Security and Productivity in Our Newly Remote WorldThe Future of Security and Productivity in Our Newly Remote World
The Future of Security and Productivity in Our Newly Remote World
DevOps.com
 

Similar to Continuous Integration/Deployment with Docker and Jenkins (20)

Dependencies Managers in C/C++. Using stdcpp 2014
Dependencies Managers in C/C++. Using stdcpp 2014Dependencies Managers in C/C++. Using stdcpp 2014
Dependencies Managers in C/C++. Using stdcpp 2014
 
CICD_1670665418.pdf
CICD_1670665418.pdfCICD_1670665418.pdf
CICD_1670665418.pdf
 
Continuous Integration with Open Source Tools - PHPUgFfm 2014-11-20
Continuous Integration with Open Source Tools - PHPUgFfm 2014-11-20Continuous Integration with Open Source Tools - PHPUgFfm 2014-11-20
Continuous Integration with Open Source Tools - PHPUgFfm 2014-11-20
 
Jenkins Pipelines
Jenkins PipelinesJenkins Pipelines
Jenkins Pipelines
 
Webinar: Creating an Effective Docker Build Pipeline for Java Apps
Webinar: Creating an Effective Docker Build Pipeline for Java AppsWebinar: Creating an Effective Docker Build Pipeline for Java Apps
Webinar: Creating an Effective Docker Build Pipeline for Java Apps
 
Gianluca Varisco - DevOoops (Increase awareness around DevOps infra security)
Gianluca Varisco - DevOoops (Increase awareness around DevOps infra security)Gianluca Varisco - DevOoops (Increase awareness around DevOps infra security)
Gianluca Varisco - DevOoops (Increase awareness around DevOps infra security)
 
Dev ops meetup
Dev ops meetupDev ops meetup
Dev ops meetup
 
Road to Opscon (Pisa '15) - DevOoops
Road to Opscon (Pisa '15) - DevOoopsRoad to Opscon (Pisa '15) - DevOoops
Road to Opscon (Pisa '15) - DevOoops
 
Comment améliorer le quotidien des Développeurs PHP ?
Comment améliorer le quotidien des Développeurs PHP ?Comment améliorer le quotidien des Développeurs PHP ?
Comment améliorer le quotidien des Développeurs PHP ?
 
ASP.NET 5 auf Raspberry PI & docker
ASP.NET 5 auf Raspberry PI & dockerASP.NET 5 auf Raspberry PI & docker
ASP.NET 5 auf Raspberry PI & docker
 
Continuous Integration & Development with Gitlab
Continuous Integration & Development with GitlabContinuous Integration & Development with Gitlab
Continuous Integration & Development with Gitlab
 
Automating Software Development Life Cycle - A DevOps Approach
Automating Software Development Life Cycle - A DevOps ApproachAutomating Software Development Life Cycle - A DevOps Approach
Automating Software Development Life Cycle - A DevOps Approach
 
Test-Driven Infrastructure with Chef
Test-Driven Infrastructure with ChefTest-Driven Infrastructure with Chef
Test-Driven Infrastructure with Chef
 
Prescriptive System Security with InSpec
Prescriptive System Security with InSpecPrescriptive System Security with InSpec
Prescriptive System Security with InSpec
 
Prescriptive Security with InSpec - All Things Open 2019
Prescriptive Security with InSpec - All Things Open 2019Prescriptive Security with InSpec - All Things Open 2019
Prescriptive Security with InSpec - All Things Open 2019
 
Deployment Tactics
Deployment TacticsDeployment Tactics
Deployment Tactics
 
Using Chef InSpec for Infrastructure Security
Using Chef InSpec for Infrastructure SecurityUsing Chef InSpec for Infrastructure Security
Using Chef InSpec for Infrastructure Security
 
DCSF 19 Building Your Development Pipeline
DCSF 19 Building Your Development Pipeline  DCSF 19 Building Your Development Pipeline
DCSF 19 Building Your Development Pipeline
 
InSpec For DevOpsDays Amsterdam 2017
InSpec For DevOpsDays Amsterdam 2017InSpec For DevOpsDays Amsterdam 2017
InSpec For DevOpsDays Amsterdam 2017
 
The Future of Security and Productivity in Our Newly Remote World
The Future of Security and Productivity in Our Newly Remote WorldThe Future of Security and Productivity in Our Newly Remote World
The Future of Security and Productivity in Our Newly Remote World
 

More from Francesco Bruni

pyconus22_complete.pdf
pyconus22_complete.pdfpyconus22_complete.pdf
pyconus22_complete.pdf
Francesco Bruni
 
Four key factors to design a web of things based architecture
Four key factors to design a web of things based architectureFour key factors to design a web of things based architecture
Four key factors to design a web of things based architecture
Francesco Bruni
 
Sparking pandas: an experiment
Sparking pandas: an experimentSparking pandas: an experiment
Sparking pandas: an experiment
Francesco Bruni
 
Basic NLP with Python and NLTK
Basic NLP with Python and NLTKBasic NLP with Python and NLTK
Basic NLP with Python and NLTK
Francesco Bruni
 
Rethink programming: a functional approach
Rethink programming: a functional approachRethink programming: a functional approach
Rethink programming: a functional approach
Francesco Bruni
 
Introduction to Functional Programming
Introduction to Functional ProgrammingIntroduction to Functional Programming
Introduction to Functional Programming
Francesco Bruni
 

More from Francesco Bruni (6)

pyconus22_complete.pdf
pyconus22_complete.pdfpyconus22_complete.pdf
pyconus22_complete.pdf
 
Four key factors to design a web of things based architecture
Four key factors to design a web of things based architectureFour key factors to design a web of things based architecture
Four key factors to design a web of things based architecture
 
Sparking pandas: an experiment
Sparking pandas: an experimentSparking pandas: an experiment
Sparking pandas: an experiment
 
Basic NLP with Python and NLTK
Basic NLP with Python and NLTKBasic NLP with Python and NLTK
Basic NLP with Python and NLTK
 
Rethink programming: a functional approach
Rethink programming: a functional approachRethink programming: a functional approach
Rethink programming: a functional approach
 
Introduction to Functional Programming
Introduction to Functional ProgrammingIntroduction to Functional Programming
Introduction to Functional Programming
 

Recently uploaded

一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理
dakas1
 
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
Luigi Fugaro
 
WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
Patrick Weigel
 
Penify - Let AI do the Documentation, you write the Code.
Penify - Let AI do the Documentation, you write the Code.Penify - Let AI do the Documentation, you write the Code.
Penify - Let AI do the Documentation, you write the Code.
KrishnaveniMohan1
 
Voxxed Days Trieste 2024 - Unleashing the Power of Vector Search and Semantic...
Voxxed Days Trieste 2024 - Unleashing the Power of Vector Search and Semantic...Voxxed Days Trieste 2024 - Unleashing the Power of Vector Search and Semantic...
Voxxed Days Trieste 2024 - Unleashing the Power of Vector Search and Semantic...
Luigi Fugaro
 
All you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVMAll you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVM
Alina Yurenko
 
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
kalichargn70th171
 
Orca: Nocode Graphical Editor for Container Orchestration
Orca: Nocode Graphical Editor for Container OrchestrationOrca: Nocode Graphical Editor for Container Orchestration
Orca: Nocode Graphical Editor for Container Orchestration
Pedro J. Molina
 
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
dakas1
 
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
kgyxske
 
The Role of DevOps in Digital Transformation.pdf
The Role of DevOps in Digital Transformation.pdfThe Role of DevOps in Digital Transformation.pdf
The Role of DevOps in Digital Transformation.pdf
mohitd6
 
Transforming Product Development using OnePlan To Boost Efficiency and Innova...
Transforming Product Development using OnePlan To Boost Efficiency and Innova...Transforming Product Development using OnePlan To Boost Efficiency and Innova...
Transforming Product Development using OnePlan To Boost Efficiency and Innova...
OnePlan Solutions
 
ppt on the brain chip neuralink.pptx
ppt  on   the brain  chip neuralink.pptxppt  on   the brain  chip neuralink.pptx
ppt on the brain chip neuralink.pptx
Reetu63
 
The Rising Future of CPaaS in the Middle East 2024
The Rising Future of CPaaS in the Middle East 2024The Rising Future of CPaaS in the Middle East 2024
The Rising Future of CPaaS in the Middle East 2024
Yara Milbes
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdfThe Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
kalichargn70th171
 
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery FleetStork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
Vince Scalabrino
 
Manyata Tech Park Bangalore_ Infrastructure, Facilities and More
Manyata Tech Park Bangalore_ Infrastructure, Facilities and MoreManyata Tech Park Bangalore_ Infrastructure, Facilities and More
Manyata Tech Park Bangalore_ Infrastructure, Facilities and More
narinav14
 
Enhanced Screen Flows UI/UX using SLDS with Tom Kitt
Enhanced Screen Flows UI/UX using SLDS with Tom KittEnhanced Screen Flows UI/UX using SLDS with Tom Kitt
Enhanced Screen Flows UI/UX using SLDS with Tom Kitt
Peter Caitens
 
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdfBaha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid
 

Recently uploaded (20)

一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理
 
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
 
WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
 
Penify - Let AI do the Documentation, you write the Code.
Penify - Let AI do the Documentation, you write the Code.Penify - Let AI do the Documentation, you write the Code.
Penify - Let AI do the Documentation, you write the Code.
 
Voxxed Days Trieste 2024 - Unleashing the Power of Vector Search and Semantic...
Voxxed Days Trieste 2024 - Unleashing the Power of Vector Search and Semantic...Voxxed Days Trieste 2024 - Unleashing the Power of Vector Search and Semantic...
Voxxed Days Trieste 2024 - Unleashing the Power of Vector Search and Semantic...
 
All you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVMAll you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVM
 
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
 
Orca: Nocode Graphical Editor for Container Orchestration
Orca: Nocode Graphical Editor for Container OrchestrationOrca: Nocode Graphical Editor for Container Orchestration
Orca: Nocode Graphical Editor for Container Orchestration
 
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
 
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
 
The Role of DevOps in Digital Transformation.pdf
The Role of DevOps in Digital Transformation.pdfThe Role of DevOps in Digital Transformation.pdf
The Role of DevOps in Digital Transformation.pdf
 
Transforming Product Development using OnePlan To Boost Efficiency and Innova...
Transforming Product Development using OnePlan To Boost Efficiency and Innova...Transforming Product Development using OnePlan To Boost Efficiency and Innova...
Transforming Product Development using OnePlan To Boost Efficiency and Innova...
 
ppt on the brain chip neuralink.pptx
ppt  on   the brain  chip neuralink.pptxppt  on   the brain  chip neuralink.pptx
ppt on the brain chip neuralink.pptx
 
The Rising Future of CPaaS in the Middle East 2024
The Rising Future of CPaaS in the Middle East 2024The Rising Future of CPaaS in the Middle East 2024
The Rising Future of CPaaS in the Middle East 2024
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
 
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdfThe Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
 
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery FleetStork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
 
Manyata Tech Park Bangalore_ Infrastructure, Facilities and More
Manyata Tech Park Bangalore_ Infrastructure, Facilities and MoreManyata Tech Park Bangalore_ Infrastructure, Facilities and More
Manyata Tech Park Bangalore_ Infrastructure, Facilities and More
 
Enhanced Screen Flows UI/UX using SLDS with Tom Kitt
Enhanced Screen Flows UI/UX using SLDS with Tom KittEnhanced Screen Flows UI/UX using SLDS with Tom Kitt
Enhanced Screen Flows UI/UX using SLDS with Tom Kitt
 
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdfBaha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
 

Continuous Integration/Deployment with Docker and Jenkins

  • 1. CI with Docker and Jenkins Francesco Bruni Club degli sviluppatori - Bari, May ‘19 @brunifrancesco
  • 2. Who am I •MSc Telecommunication Engineer @ Poliba •Despite a Java background, I prefer Python whenever possible •I'm not a computer scientist

  • 3. Continuous Integration •Keep integrating multiple times a day the central code repository; •Run as more tests as you can to ensure nothing broke down; •Be ready to replicate environment.
  • 4. Why CI matters •Continuous testing/building for all teammates; •Let someone do trivial jobs.
  • 5. CI advantage “Continuous Integration doesn’t get rid of bugs, but it does make them dramatically easier to find and remove” M. Fowler
  • 6. CI requirements •Single source repo; •Automated building; •Self testing build; •Build fast; •Run tests in a production env clone; •Automate deployment.
  • 7. CI cycle Develop Feature Integrate in the main code base Test/build/label Checkout code Correct detected iussues
  • 9. CI cycle •Integrating code should be automated; •Multiple levels testing; •Devs must not checkout bugged code; •Devs must not deploy untested code;
  • 10. CI drawbacks •Setting files; •Setting all the stuff up requires time and patience.
  • 11. CI natural evolution:
 Continuous Delivery •If a commit builds (all tests succeed), deliver it; •Keep delivery time at minimum; •Multiple deliveries for multiple versions.
  • 12. Know your tools •Test everything; •Keep your sources/setting files versioned; •Containerize everything; •Automate as much as you can.
  • 13. Test everything •Test your sources, checking coverage; •Test for writing new features; •Test your application settings; •If you want to deliver software, check external requirements.
  • 14. Smarter use of VCs •Keep code versioned: application settings/sources/dep indexes; •Keep env settings versioned: scripts and configuration files; •Enjoy version control well defined ‘flows’ :)
  • 15. Be containerized •Develop solutions in well separated envs; •Ship software as main part of env; •Don’t play with global deps. •‘Restore’ basic envs to speed up env creation process.
  • 16. What’s Docker •Wrap many services in many containers; •Create, play with and destroy containers; •Save and easily replicate isolated containers.
  • 17. Dockerfiles # pull base image. FROM java:8 # maintainer details MAINTAINER Francesco Bruni "francesco.bruni@stasbranger.com" # update packages and install maven RUN export DEBIAN_FRONTEND=noninteractive && sed -i 's/# (.*multiverse$)/1/g' /etc/apt/sources.list && apt-get update && apt-get -y upgrade && apt-get install -y vim wget curl git maven ssh nano python-pip # attach volumes VOLUME /volume/git # create working directory and set some stuff up RUN mkdir -p /local/git WORKDIR /local/git RUN mkdir -p /root/.ssh ADD id_rsa /root/.ssh/id_rsa RUN chmod 700 /root/.ssh/id_rsa RUN echo "Host <ip>ntStrictHostKeyChecking non" >> /root/.ssh/config COPY .m2 /root/.m2 COPY run.sh run.sh RUN chmod a+x run.sh # run this when container starts CMD ["./run.sh"]
  • 18. Automate procedures •Repetitive tasks should be performed by automated agents; •Once you teach the agent what to do, it’s up to him to fulfill the job.
  • 19. What’s Jenkins •Automated procedures worker; •Run tasks, report final status, log activities; •Multiple plugin for multiple features; •Use it just for playing too :)
  • 20. Deploy w/ Docker w/o Jenkins •Test code (or at least run a set of tests); •Push code to repo; •SSH in the server, enter in container; •Pull changes and restart required services. •Check if it works! :)
  • 21. Practical session •Integrate new code into the main branch; •Test all the stuff; •Deploy it :)
  • 22. What we’ll need •A GIT web hook; •Set private “credentials”; •A tool to test configuration settings; •A container to run the test suite; •A container to run the new application; •A pipeline.
  • 23. GIT web hook • Bind an action (tipically run a script) to GIT events; • Notify Jenkins new code has been pushed over the ‘develop’ branch. #! /bin/bash read oldrev newrev _branch tail=$(git log -1 --pretty=format:'%h %cn: %s%b' $newrev) curl http://<ip>:8080/job/QW/build? token=vH3YKmrRR5KOT4aeAOAV&commitMessage= $tail&cause=JenkinsDeploy
  • 24. Merge/Test configurations import os
 
 main_path = os.path.join("src", "main","resources","application.properties")
 test_path = os.path.join("src", "test","resources","application.properties")
 
 for element in (main_path, test_path,):
 with open(element) as old_main:
 data = map(lambda line: line.strip(), old_main.readlines())
 data[6] = data[6].replace("192.168.1.75", "mysql")
 data[6] = data[6].replace("<myIp>", "mysql")
 
 
 with open(element, "wb") as new_main:
 new_main.write(“n".join(data)) Connection().quick_connect("user", "password", “db_test", "<myIp>") Connection().quick_connect("user", "password", "db", "mysql")
 

  • 25. Create the container to run test suite Run tests and see if they fails :)
  • 26. Deploy the new version • Create the new container and run the new version; • The new container should replace an existing one.
  • 27. Going on production No way, do some manual testing and start a new CD build
  • 28. All togheter node { stage 'Stage 1' sh 'docker run --rm --link mysqlC:mysql --name sms-checklink-container sms-checklink' stage 'Stage 2' sh 'docker rm -f sms-test-container || echo 'no container to delete'' sh 'docker run --rm --name sms-test-container --link mysqlC:mysql sms- test' stage 'Stage 3' sh 'docker run --rm --name sms-merge-container sms-merge' stage 'Stage 4' sh 'docker rm -f sms-staging-container || echo 'no container to delete'' sh 'docker run -d -p 8433:8080 --name sms-staging-container --link mysqlC:mysql sms-staging' }
  • 29. Conclusions • Improve configuration settings handling; • Reduce testing time; • Bind Jenkins job to commit messages; • Improve git-flow integration; • Docker Compose :( • Deploy project on production via manual tests.
  • 30. References Continuous Delivery
 Reliable Software Releases through Build, Test, and Deployment Automation by Jez Humble and David Farley Continuous Integration by Martin Fowler
 http://martinfowler.com/articles/continuousIntegration.html
  • 31. References Docker for beginners https://scotch.io/tutorials/getting-started-with-docker Meet Jenkins
 
 https://wiki.jenkins-ci.org/display/JENKINS/Meet+Jenkins Docker Explained: Using Dockerfiles to Automate Building of Images https://www.digitalocean.com/community/tutorials/ docker-explained-using-dockerfiles-to-automate- building-of-images
  • 32. Let’s see it @ work