Jenkins to Gitlab - Intelligent Build-Pipelines

Intelligent Build Pipelines
from Jenkins to Gitlab
netz98 GmbH - Christian Münch / @cmuench 1

Jenkins Job List

Jenkins Job

Current Jenkins Setup
4 1 Job for every Magento Module (sometimes two CE/EE)
4 1 Magento Installation for every Magento Module
4 1 Database for every Magento Module
4 1 extra database for Integration tests
4 Manually compiled PHP versions.
4 Software is directly installed on host
4 Only few commits in most M1 repos

Disk Space! -> Magento 1 Modules
4 1x Magento 1 Module
4 Webroot: 175 MB
4 Database: 62 MB
4 Total with Log-Data: ~200 MB
4 50x Magento 1 Module -> 10 GB

Disk Space! -> Magento 2 Modules
4 1x Magento 2 Module
4 Webroot: 1,1 GB
4 Database: 82 MB
4 Total with Log-Data: ~1.200 MB
4 200 planned Modules: 240 GB

Simple CI Workflow
4 Configuration only with a single file
4 .gitlab-ci.yml

Simple Job Example
job1:
script: ls -l

Multiple Script Commands
job1:
script:
- cd myproject
- vendor/bin/phpunit

Multi Stages
stages:
- test
- deploy
run_tests:
stage: test
script: php run-some-tests.php
deploy_to_server:
stage: deploy
script: php mageDeploy2.php

Pipeline View

Same Jobs in Docker Containers
stages:
- test
- deploy
image: php:5.6
run_tests:
stage: test
script: php run-some-tests.php
deploy_to_server:
stage: deploy
image: php:7.0
script: php mageDeploy2.php

Install Special Tools
before_script:
- apt-get install swaks
job1:
script:
- swaks --from foo@example.com --to bar@example.com
- ...

Artifacts
4 Runner sends generated artifacts to Gitlab Instance
via REST Api.
job1:
script: php generate_artifact.php
artifacts:
when: on_success
paths:
- path_to_artifact/
expire_in: 1 week

Sphinx Doc
stages:
- build
documentation_pdf:
stage: build
image: n98-sphinx-doc:1.4.6
script:
- cd doc
- make clean
- make latexpdf
retry: 2
only:
- master
artifacts:
when: on_success
paths:
- doc/_build/latex/*.pdf
expire_in: 1 week

Gitlab-Runner / Docker Execution

Test Build-Pipeline
Locally
brew install gitlab-runner # or use apt-get, yum, ....
cd <project-with-gitlab-ci-file>
gitlab-runner exec docker <job_in_gitlab_ci_config>

Docker Image Hierarchy

Image-Generation with CI-Pipeline

Technical Setup (Current) -> Unix Socket
Mounting

Technical Setup (Future) -> Docker in Docker

Testing PHP Library
stages:
- test
ci_run_php_5:
stage: test
image: n98-php:5.6
script:
- composer install --prefer-dist
- bash build/gitlab/php_lint.sh
- vendor/bin/robo run:ci
coverage: '/^s*Lines:s*d+.d+%/'
artifacts:
when: on_success
paths:
- build/output/phpunit
expire_in: 1 week

Cache Vendor Directory
cache:
paths:
- vendor
untracked: true

XDebug
4 Installed but disabled by default
4 Activate during runtime
install_special_project_tools:
stage: setup
image: n98-php:7.0
script:
- docker-php-ext-enable xdebug
- vendor/bin/phpunit --colors --debug --coverage-html coverage-html-report --coverage-text
- coverage: '/^s*Lines:s*d+.d+%/'

Testing M2 Modules
stages:
- test
before_script:
- mysql_start
.job_template: &job_definition
script:
- install-module
- cd $MAGENTO_ROOT
- vendor/bin/phpunit -c dev/tests/unit/phpunit.xml.dist $CI_PROJECT_DIR/src/Test
EE-2.1:
image: n98-magento-ee:2.1
<<: *job_definition
EE-2.2:
<<: *job_definition

Test M2 Theme
stages:
- test
before_script:
- mysql_start
.job_template: &job_definition
script:
- install-theme
CE-2.2:
image: n98-magento-ce:2.2
<<: *job_definition
EE-2.2:
<<: *job_definition

Gitflow Support
Branches
4 Develop branch contains the
"bleeding edge"
4 Test only against latest
supported Magento versions.
4 Every Major Break -> New
Support Branch

Gitflow Support
Branches
4 One or more support branches
are available for older Magento
versions.
4 Needs backports for bugfixes,
features
4 Allows different CI configs
4 Needs strict Semantic

netz98 Gitlab CI Status
✔ PHP Images
✔ Magento Images
✔ Ansible, Sphinx-Doc Images, ...
✔ Git Mirror Sync
✔ Test PHP Libraries
✔ Test Magento 1 Modules
✔ Test Magento 2 Modules
◻ Project Builds (with docker-compose support)
◻ Deployments with specific versions (prototype available)

Links
4 https://store.fooman.co.nz/blog/harnessing-the-power-of-
composer-in-magento-2.html
4 https://docs.gitlab.com/ee/ci/docker/usingdockerbuild.html
4 https://media.ccc.de/v/froscon2017-1948-gitlab-
ciunddocker_registry
4 https://github.com/oleg-fiksel/SettingupGitLab-
CIundDocker_Registry

Jenkins to Gitlab - Intelligent Build-Pipelines

More Related Content

What's hot

Similar to Jenkins to Gitlab - Intelligent Build-Pipelines

More from Christian Münch

Recently uploaded

Jenkins to Gitlab - Intelligent Build-Pipelines