SlideShare a Scribd company logo
On the usage, co-usage and
migration of CI/CD tools:
A qualitative analysis
P. Rostami Mazrae, T. Mens, M. Golzadeh, A. Decan
Software Engineering Lab
1
This work is supported by the ARC-21/25 UMONS3 Action de Recherche Concertée financée par le Ministère de la Communauté française – Direction générale de l’Enseignement non obligatoire et
de la Recherche scientifique, as well as by the Fonds de la Recherche Scien4fique - FNRS under grant numbers O.0157.18F-RG43, T.0017.18 and T.0022.13.
ASE 2023
Journal First
presentation
2
CI/CD
Collaborative software development
Quality
analysis
Build Test Deploy
3
Goals of the Qualitative Study
How and why do
experienced developers in
software projects rely on
specific CI/CD tools?
What are the reasons for using multiple
CI/CD tools simultaneously, and migrating
from one CI/CD tool to another?
4
Methodology
5
• Around 30 questions about CI/CD usage, co-usage and migration
Interview questionnaire
• through social media (Twitter, LinkedIn), email, direct messaging
• Colleagues' referrals (snowballing)
Selection of respondents
• Using online video conferencing tool
Interview method
• Actively contributed to, or having been responsible for a software project relying on CI
• Sufficient knowledge about which CI/CD tool is used in the software project and how
• Having been involved in setting up or maintaining the CI/CD process of the project
Inclusion Criteria
Demographics
6
16
4 2
22 interviewees
Average of
12 years and 4 months
software development
experience
12 industry
and
open source
7 only
industry
2 only
open
source
7
CI/CD tools
being used
31 CI/CD
tools
reported
6 CI/CD
reported
>10 times
8 CI/CD
reported
several
times
14 CI/CD
reported
only once
3 cases of
custom-built
in-house
CI/CD
solutions
GHA, Jenkins, Travis,
GitLab CI/CD, CircleCI,
Azure DevOps
AppVeyor,
Hudson,
TeamCity,
Cruise Control,
Drone,
BitBucket Pipeline,
Netlify,
Bamboo
Activities
automated by
CI/CD tools
Basic tasks:
• build automation
• automated unit testing
• code coverage analysis
Advanced tasks:
• security analysis
• code quality analysis
• dependency management
• packaging
• deployment
8
Most valuable features of CI/CD tools
Good
integration
with hosting
platform
GHA
Azure
DevOps
Travis
GitLab
CI/CD
9
Good
free tier
GHA
Travis
Good
plugin
support
GHA
Azure
DevOps
Travis
Jenkins Specific
architecture/OS
GHA
Azure
DevOps
CircleCI
AppVeyor
Jenkins
Reported shortcoming of CI/CD tools
10
Hard to
configure
Slow
Unsatisfactory
user experince
Restriction of
free tier
Security issues
No support for
specific
architecture/OS
Lack of reliability
(specific to Travis)
Most popular
CI/CD co-usages
• Travis + AppVeyor
• CircleCI + AppVeyor
• Azure DevOps + AppVeyor
• Github Actions + CircleCI
11
Reasons behind co-usage
12
SUPPORTING
MULTIPLE OS
(5 RESPONSES)
COMPLEMENTARY
FUNCTIONALITY
(3 RESPONSES)
BACKUP CI TOOL
(3 RESPONSES)
COUNTERING
RESOURCE
LIMITATION
(3 RESPONSES)
SUPPORTING
SPECIFIC HARDWARE
ARCHITECTURES
(2 RESPONSES)
TESTING A CI FOR
POTENTIAL
MIGRATION
(1 RESPONSE)
CI/CD
migration
13
Reasons for
CI migration
14
Having less restriction on the free tier (all moving from Travis)
Using a more reliable CI/CD tool (all moving from Travis)
Better integration with hosting platform (mostly moving to GHA)
Better support of multiple platform (mostly moving to GHA)
Decrease the amount of CI/CD tools being co-used
Difficulties in CI migration
• Learning curve
• Fundamental differences between the
source and target of the migration
• Trial-and-error nature of configuring a new
CI/CD tool
• Lack of familiarity with the new CI/CD tool
• Important missing features
15
Summary
16

More Related Content

Similar to On the usage, co-usage and migration of CI:CD tools A qualitative analysis.pptx

Training Bootcamp - MainframeDevOps.pptx
Training Bootcamp - MainframeDevOps.pptxTraining Bootcamp - MainframeDevOps.pptx
Training Bootcamp - MainframeDevOps.pptx
Nashet Ali
 
Making software development processes to work for you
Making software development processes to work for youMaking software development processes to work for you
Making software development processes to work for you
Ambientia
 
NYIT DSC/ Spring 2021 - Introduction to DevOps (CI/CD)
NYIT DSC/ Spring 2021 - Introduction to DevOps (CI/CD)NYIT DSC/ Spring 2021 - Introduction to DevOps (CI/CD)
NYIT DSC/ Spring 2021 - Introduction to DevOps (CI/CD)
Hui (Henry) Chen
 
A Roadmap to Enterprise Quality
A Roadmap to Enterprise QualityA Roadmap to Enterprise Quality
A Roadmap to Enterprise Quality
Jeff Bramwell
 
Bridging the Gap: from Data Science to Production
Bridging the Gap: from Data Science to ProductionBridging the Gap: from Data Science to Production
Bridging the Gap: from Data Science to Production
Florian Wilhelm
 
Programming languages and techniques for today’s embedded andIoT world
Programming languages and techniques for today’s embedded andIoT worldProgramming languages and techniques for today’s embedded andIoT world
Programming languages and techniques for today’s embedded andIoT world
Rogue Wave Software
 
DevOps: Age Of CI/CD
DevOps: Age Of CI/CDDevOps: Age Of CI/CD
DevOps: Age Of CI/CD
MoogleLabs default
 
Streamlining Mobile App Development with CI/CD A Pathway to Efficiency and ...
Streamlining Mobile App Development with CI/CD   A Pathway to Efficiency and ...Streamlining Mobile App Development with CI/CD   A Pathway to Efficiency and ...
Streamlining Mobile App Development with CI/CD A Pathway to Efficiency and ...
Elina619459
 
Chris Munns, DevOps @ Amazon: Microservices, 2 Pizza Teams, & 50 Million Depl...
Chris Munns, DevOps @ Amazon: Microservices, 2 Pizza Teams, & 50 Million Depl...Chris Munns, DevOps @ Amazon: Microservices, 2 Pizza Teams, & 50 Million Depl...
Chris Munns, DevOps @ Amazon: Microservices, 2 Pizza Teams, & 50 Million Depl...
TriNimbus
 
Aligning Software Testing With Modern Age Development Practices
Aligning Software Testing With Modern Age Development PracticesAligning Software Testing With Modern Age Development Practices
Aligning Software Testing With Modern Age Development Practices
Aspire Systems
 
Continuous integration for se group meeting
Continuous integration for se group meetingContinuous integration for se group meeting
Continuous integration for se group meeting
Sergii Shmarkatiuk
 
DevOps State of the Union 2015
DevOps State of the Union 2015DevOps State of the Union 2015
DevOps State of the Union 2015
Ernest Mueller
 
Building and Scaling High Performing Technology Organizations by Jez Humble a...
Building and Scaling High Performing Technology Organizations by Jez Humble a...Building and Scaling High Performing Technology Organizations by Jez Humble a...
Building and Scaling High Performing Technology Organizations by Jez Humble a...
Agile India
 
Software Security Assurance for DevOps - Hewlett Packard Enterprise + Black Duck
Software Security Assurance for DevOps - Hewlett Packard Enterprise + Black DuckSoftware Security Assurance for DevOps - Hewlett Packard Enterprise + Black Duck
Software Security Assurance for DevOps - Hewlett Packard Enterprise + Black Duck
Black Duck by Synopsys
 
DevSecOps: The Open Source Way
DevSecOps: The Open Source WayDevSecOps: The Open Source Way
DevSecOps: The Open Source Way
Black Duck by Synopsys
 
Open-DO: Towards a Lean Approach for Certification (Cyrille Comar)
Open-DO: Towards a Lean Approach for Certification  (Cyrille Comar)Open-DO: Towards a Lean Approach for Certification  (Cyrille Comar)
Open-DO: Towards a Lean Approach for Certification (Cyrille Comar)
AdaCore
 
DevOps and SF.pdf
DevOps and SF.pdfDevOps and SF.pdf
DevOps and SF.pdf
PatrickYANG48
 
SCALABLE CI CD DEVOPS
SCALABLE CI CD DEVOPSSCALABLE CI CD DEVOPS
SCALABLE CI CD DEVOPS
G R VISHAL
 
DevOps CTO Masterclass | Webinar Oct. 2020
DevOps CTO Masterclass | Webinar Oct. 2020DevOps CTO Masterclass | Webinar Oct. 2020
DevOps CTO Masterclass | Webinar Oct. 2020
Cyber-Duck
 
Linking Upstream and Downstream Agile
Linking Upstream and Downstream AgileLinking Upstream and Downstream Agile
Linking Upstream and Downstream Agile
CollabNet
 

Similar to On the usage, co-usage and migration of CI:CD tools A qualitative analysis.pptx (20)

Training Bootcamp - MainframeDevOps.pptx
Training Bootcamp - MainframeDevOps.pptxTraining Bootcamp - MainframeDevOps.pptx
Training Bootcamp - MainframeDevOps.pptx
 
Making software development processes to work for you
Making software development processes to work for youMaking software development processes to work for you
Making software development processes to work for you
 
NYIT DSC/ Spring 2021 - Introduction to DevOps (CI/CD)
NYIT DSC/ Spring 2021 - Introduction to DevOps (CI/CD)NYIT DSC/ Spring 2021 - Introduction to DevOps (CI/CD)
NYIT DSC/ Spring 2021 - Introduction to DevOps (CI/CD)
 
A Roadmap to Enterprise Quality
A Roadmap to Enterprise QualityA Roadmap to Enterprise Quality
A Roadmap to Enterprise Quality
 
Bridging the Gap: from Data Science to Production
Bridging the Gap: from Data Science to ProductionBridging the Gap: from Data Science to Production
Bridging the Gap: from Data Science to Production
 
Programming languages and techniques for today’s embedded andIoT world
Programming languages and techniques for today’s embedded andIoT worldProgramming languages and techniques for today’s embedded andIoT world
Programming languages and techniques for today’s embedded andIoT world
 
DevOps: Age Of CI/CD
DevOps: Age Of CI/CDDevOps: Age Of CI/CD
DevOps: Age Of CI/CD
 
Streamlining Mobile App Development with CI/CD A Pathway to Efficiency and ...
Streamlining Mobile App Development with CI/CD   A Pathway to Efficiency and ...Streamlining Mobile App Development with CI/CD   A Pathway to Efficiency and ...
Streamlining Mobile App Development with CI/CD A Pathway to Efficiency and ...
 
Chris Munns, DevOps @ Amazon: Microservices, 2 Pizza Teams, & 50 Million Depl...
Chris Munns, DevOps @ Amazon: Microservices, 2 Pizza Teams, & 50 Million Depl...Chris Munns, DevOps @ Amazon: Microservices, 2 Pizza Teams, & 50 Million Depl...
Chris Munns, DevOps @ Amazon: Microservices, 2 Pizza Teams, & 50 Million Depl...
 
Aligning Software Testing With Modern Age Development Practices
Aligning Software Testing With Modern Age Development PracticesAligning Software Testing With Modern Age Development Practices
Aligning Software Testing With Modern Age Development Practices
 
Continuous integration for se group meeting
Continuous integration for se group meetingContinuous integration for se group meeting
Continuous integration for se group meeting
 
DevOps State of the Union 2015
DevOps State of the Union 2015DevOps State of the Union 2015
DevOps State of the Union 2015
 
Building and Scaling High Performing Technology Organizations by Jez Humble a...
Building and Scaling High Performing Technology Organizations by Jez Humble a...Building and Scaling High Performing Technology Organizations by Jez Humble a...
Building and Scaling High Performing Technology Organizations by Jez Humble a...
 
Software Security Assurance for DevOps - Hewlett Packard Enterprise + Black Duck
Software Security Assurance for DevOps - Hewlett Packard Enterprise + Black DuckSoftware Security Assurance for DevOps - Hewlett Packard Enterprise + Black Duck
Software Security Assurance for DevOps - Hewlett Packard Enterprise + Black Duck
 
DevSecOps: The Open Source Way
DevSecOps: The Open Source WayDevSecOps: The Open Source Way
DevSecOps: The Open Source Way
 
Open-DO: Towards a Lean Approach for Certification (Cyrille Comar)
Open-DO: Towards a Lean Approach for Certification  (Cyrille Comar)Open-DO: Towards a Lean Approach for Certification  (Cyrille Comar)
Open-DO: Towards a Lean Approach for Certification (Cyrille Comar)
 
DevOps and SF.pdf
DevOps and SF.pdfDevOps and SF.pdf
DevOps and SF.pdf
 
SCALABLE CI CD DEVOPS
SCALABLE CI CD DEVOPSSCALABLE CI CD DEVOPS
SCALABLE CI CD DEVOPS
 
DevOps CTO Masterclass | Webinar Oct. 2020
DevOps CTO Masterclass | Webinar Oct. 2020DevOps CTO Masterclass | Webinar Oct. 2020
DevOps CTO Masterclass | Webinar Oct. 2020
 
Linking Upstream and Downstream Agile
Linking Upstream and Downstream AgileLinking Upstream and Downstream Agile
Linking Upstream and Downstream Agile
 

Recently uploaded

8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
kalichargn70th171
 
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
Green Software Development
 
SMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API ServiceSMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API Service
Yara Milbes
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
Green Software Development
 
Mobile app Development Services | Drona Infotech
Mobile app Development Services  | Drona InfotechMobile app Development Services  | Drona Infotech
Mobile app Development Services | Drona Infotech
Drona Infotech
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
Drona Infotech
 
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language ProcessorsFundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
Rakesh Kumar R
 
All you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVMAll you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVM
Alina Yurenko
 
Top 9 Trends in Cybersecurity for 2024.pptx
Top 9 Trends in Cybersecurity for 2024.pptxTop 9 Trends in Cybersecurity for 2024.pptx
Top 9 Trends in Cybersecurity for 2024.pptx
devvsandy
 
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
mz5nrf0n
 
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
mz5nrf0n
 
Lecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptxLecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptx
TaghreedAltamimi
 
socradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdfsocradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdf
SOCRadar
 
Using Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional SafetyUsing Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional Safety
Ayan Halder
 
How to write a program in any programming language
How to write a program in any programming languageHow to write a program in any programming language
How to write a program in any programming language
Rakesh Kumar R
 
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CDKuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
rodomar2
 
Microservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we workMicroservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we work
Sven Peters
 
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
XfilesPro
 
Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !
Marcin Chrost
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
Peter Muessig
 

Recently uploaded (20)

8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
 
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
 
SMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API ServiceSMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API Service
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
 
Mobile app Development Services | Drona Infotech
Mobile app Development Services  | Drona InfotechMobile app Development Services  | Drona Infotech
Mobile app Development Services | Drona Infotech
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
 
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language ProcessorsFundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
 
All you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVMAll you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVM
 
Top 9 Trends in Cybersecurity for 2024.pptx
Top 9 Trends in Cybersecurity for 2024.pptxTop 9 Trends in Cybersecurity for 2024.pptx
Top 9 Trends in Cybersecurity for 2024.pptx
 
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
 
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
 
Lecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptxLecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptx
 
socradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdfsocradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdf
 
Using Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional SafetyUsing Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional Safety
 
How to write a program in any programming language
How to write a program in any programming languageHow to write a program in any programming language
How to write a program in any programming language
 
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CDKuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
 
Microservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we workMicroservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we work
 
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
 
Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
 

On the usage, co-usage and migration of CI:CD tools A qualitative analysis.pptx

  • 1. On the usage, co-usage and migration of CI/CD tools: A qualitative analysis P. Rostami Mazrae, T. Mens, M. Golzadeh, A. Decan Software Engineering Lab 1 This work is supported by the ARC-21/25 UMONS3 Action de Recherche Concertée financée par le Ministère de la Communauté française – Direction générale de l’Enseignement non obligatoire et de la Recherche scientifique, as well as by the Fonds de la Recherche Scien4fique - FNRS under grant numbers O.0157.18F-RG43, T.0017.18 and T.0022.13.
  • 4. Goals of the Qualitative Study How and why do experienced developers in software projects rely on specific CI/CD tools? What are the reasons for using multiple CI/CD tools simultaneously, and migrating from one CI/CD tool to another? 4
  • 5. Methodology 5 • Around 30 questions about CI/CD usage, co-usage and migration Interview questionnaire • through social media (Twitter, LinkedIn), email, direct messaging • Colleagues' referrals (snowballing) Selection of respondents • Using online video conferencing tool Interview method • Actively contributed to, or having been responsible for a software project relying on CI • Sufficient knowledge about which CI/CD tool is used in the software project and how • Having been involved in setting up or maintaining the CI/CD process of the project Inclusion Criteria
  • 6. Demographics 6 16 4 2 22 interviewees Average of 12 years and 4 months software development experience 12 industry and open source 7 only industry 2 only open source
  • 7. 7 CI/CD tools being used 31 CI/CD tools reported 6 CI/CD reported >10 times 8 CI/CD reported several times 14 CI/CD reported only once 3 cases of custom-built in-house CI/CD solutions GHA, Jenkins, Travis, GitLab CI/CD, CircleCI, Azure DevOps AppVeyor, Hudson, TeamCity, Cruise Control, Drone, BitBucket Pipeline, Netlify, Bamboo
  • 8. Activities automated by CI/CD tools Basic tasks: • build automation • automated unit testing • code coverage analysis Advanced tasks: • security analysis • code quality analysis • dependency management • packaging • deployment 8
  • 9. Most valuable features of CI/CD tools Good integration with hosting platform GHA Azure DevOps Travis GitLab CI/CD 9 Good free tier GHA Travis Good plugin support GHA Azure DevOps Travis Jenkins Specific architecture/OS GHA Azure DevOps CircleCI AppVeyor Jenkins
  • 10. Reported shortcoming of CI/CD tools 10 Hard to configure Slow Unsatisfactory user experince Restriction of free tier Security issues No support for specific architecture/OS Lack of reliability (specific to Travis)
  • 11. Most popular CI/CD co-usages • Travis + AppVeyor • CircleCI + AppVeyor • Azure DevOps + AppVeyor • Github Actions + CircleCI 11
  • 12. Reasons behind co-usage 12 SUPPORTING MULTIPLE OS (5 RESPONSES) COMPLEMENTARY FUNCTIONALITY (3 RESPONSES) BACKUP CI TOOL (3 RESPONSES) COUNTERING RESOURCE LIMITATION (3 RESPONSES) SUPPORTING SPECIFIC HARDWARE ARCHITECTURES (2 RESPONSES) TESTING A CI FOR POTENTIAL MIGRATION (1 RESPONSE)
  • 14. Reasons for CI migration 14 Having less restriction on the free tier (all moving from Travis) Using a more reliable CI/CD tool (all moving from Travis) Better integration with hosting platform (mostly moving to GHA) Better support of multiple platform (mostly moving to GHA) Decrease the amount of CI/CD tools being co-used
  • 15. Difficulties in CI migration • Learning curve • Fundamental differences between the source and target of the migration • Trial-and-error nature of configuring a new CI/CD tool • Lack of familiarity with the new CI/CD tool • Important missing features 15