CI/CD practices play a significant role in collaborative software development. The GitHub social coding platform introduced GitHub Actions as a way to automate different aspects of software production such as testing, building, quality checking, dependency and security management. We report on preliminary findings of a quantitative analysis on how GitHub Actions workflows are being changed over time. The study is based on a dataset of 22,733 GitHub repositories. We extracted over 4 million weekly snapshots of workflow files from November 2019 to September 2022. First, we analyse the coarse-grained changes being made to workflows, including when repositories start using them, when they are being added, modified, renamed and removed. Second, we analyse changes made to the workflow contents, including how many lines are changed and what types of changes are being made to them. The findings of this quantitative analysis provide preliminary insights on how GitHub Actions workflows are being changed over time by developers, and pave the way for further analysis on the type of modifications they apply on workflows to cover their needs in software projects.

1. A preliminary study of
GitHub Actions workflow changes
Pooya Rostami Mazrae
Alexandre Decan, Tom Mens
University of Mons, Belgium
This work is supported by the ARC-21/25 UMONS3 Action de Recherche Concertée financée par le Ministère de la Communauté française
Direction générale de l’Enseignement non obligatoire et de la Recherche scientifique, and by the Fonds de la Recherche Scientifique - FNRS under grant number F.4515.23.
Mairieli Wessel
Radboud University, Netherlands
SATToSE 2023

2. Collaborative software development
Quality
analysis
Trigger a build Run test Deploy
CI/CD
2

3. CI/CD usage in 90K GitHub repositories
3
On the rise and fall of CI services in GitHub, SANER 2022, 10.1109/SANER53432.2022.00084

4. GitHub Actions 4

5. Workflow file change example
5

6. Laws of Software
Evolution
• Continuing Change
• Continuing Growth
• Increasing Complexity
• Declining Quality
• …
6
Lehman, M. M. Laws of software evolution revisited.
In Software Process Technology, EWSPT’96, October 9–11, 1996 (pp. 108-124).

7. Data extraction
• Repositories active in 2022
• At least 100 commits and 100 stars
• Dataset:
• 22,733 repositories use GHA (out of 62,673)
• 65,067 workflow files
• 4,127,760 workflow file snapshots (weekly)
• November 2019 to September 2022
7

8. Goals
• Goal 1: Characterise and quantify the coarse-grained changes being made to
GHA workflow files over their lifetime
• Goal 2: Characterise and quantify the fine-grained changes being made to
GHA workflow file contents over their lifetime
8

9. Goal1: Characterise and quantify the coarse-grained
changes being made to GHA workflow files
• RQ1.1: When do repositories start using GitHub Actions?
• RQ1.2: Which types of coarse-grained changes are workflow files
subject to?
• RQ1.3: When do the different types of coarse-grained changes
occur?
9

10. RQ1.1: When do
repositories start using
GitHub Actions?
10
The majority of the repositories that were
created after GHA’s release adopted it within a
few months.
It took more than one year since GHA’s release
for most of the older repositories to adopt GHA.

11. RQ1.2: Which types of
coarse-grained changes
are workflow files
subject to?
• Four types of changes:
• addition
• modification
• renaming
• removal
11
This suggests that workflow files are subject to continuing change
and continuing growth.
proportion
of
repositories
exhibiting
a
change
in
a
workflow
file

12. RQ1.3: When do different
types of coarse-grained
changes occur?
12
This confirms that workflow files are subject to
continuing change and continuing growth.

13. Goal 2: Quantify fine-grained workflow changes by
analysing the contents of the workflow files
• RQ2.1: Which types of line-based changes are workflow file
contents subject to?
• RQ2.2: When do different types of line-based changes occur?
13

14. RQ2.1: Which types of line-
based changes are workflow
files subject to?
• CLOC command-line tool
• Lines added, modified, removed
14
• Modifying and adding lines are the most frequent
operations made to workflow file content.
• 9 out of 10 lines added, removed or modified are
related to Code.
added modified
removed
18.86%
3.06% 6.09%
13.63%
40.11%
14.58%
3.45%
Proportion of changes

15. RQ2.2: When do different types of line-based
changes occur?
15
Proportion of lines of code touched
during workflow changes.
Contents of workflow files are subject to
continuing change and continuing growth.
Number of lines of code
modified during workflow changes.
Number of lines of code
added during workflow changes.

16. Future work
16
Line-based changes in
workflows
syntactic changes in
workflows
• What kind of changes are we facing?
• When do the changes happen?
• Who makes these changes?
coarse-grained changes in
workflows