Slides of a meetup I gave: https://www.meetup.com/full-stack-developer-il/events/245611354/ The talk will introduced the goal to providing a better usage of Jenkins pipeline framework by: - Walk thought an implementation of a project-pipeline solution. - Jenkins pipeline touch base. - Explore advanced Jenkins pipeline features. - Provide best practices and tips. - Aim toward DevOps-as-a-code solution.

1. Tikal Knowledge
JENKINS PIPELINE
The Jenkins Pipeline Force Awakens
Yoram Michaeli
https://www.youtube.com/watch?v=lPJbTQyiKPA

2. Tikal Knowledge
I’M JUST A SIMPLE MAN
TRYING TO MAKE MY WAY
IN THE UNIVERSE.
Jango Fett

3. Tikal KnowledgeTikal Knowledge
ABOUT ME
v @YORAMMI
v DEVOPS TECH. LEADER
v COMMUNITY ORIENTED
v JEDI MASTER

4. Tikal KnowledgeTikal Knowledge
ABOUT TIKAL
v VALUABLE COMPANY
v COMMUNITY LEADERS
v UNIQUE ROUTINES
v JEDI MASTERS

5. Tikal Knowledge
DO. OR DO NOT.
THERE IS NO TRY.
Yoda

6. Tikal KnowledgeTikal Knowledge
MISSION CHECKLIST
q MOVE ALL ACTIVE JOBS FROM JENKINS 1.X
SERVER TO JENKINS 2.X SERVER
q REDUCE THE NUMBER OF JOBS WHEN
POSSIBLE
q INSTALL NOTHING ON THE JENKINS SERVER
q CONTINUE THE SEQUENCED BUILD NUMBERS
WHEN NEEDED
q FASTER JOBS (TOWARD CI)
q USE JENKINS PIPELINE FOR CODING THE
JOBS
q USE DOCKER AS THE PLATFORM TO RUN THE
JOBS ON
q USE SLACK NOTIFICATIONS
q UNIFIED VIEWS, JOBS, AND
NOTIFICATIONS
q REUSABLE CODE
q USE OF SHARED LIBRARIES
q BASIC FLOWS IMPLEMENTATION
q DEVELOPERS CAN WRITE NEW PIPELINES
EASILY
q BETTER NIGHTLY CI JOB

7. Tikal Knowledge
I FOUND YOUR
LACK OF FAITH
DISTURBING.
Darth Vadar

8. Tikal Knowledge
JENKINS TOUCH BASE

9. Tikal KnowledgeTikal Knowledge
ABOUT JENKINS PIPELINE
v INTRODUCED IN VER. 2.X
v PROCESS-AS-A-CODE
v GROOVY PROGRAMMING
v USE DECLARATIVE PIPELINE

10. Tikal Knowledge
I SPEAK FLUENTLY OVER SIX MILLION
LANGUAGES…
WHY GROOVY???

11. Tikal KnowledgeTikal Knowledge
BASIC PIPELINES
JOB CONFIGURATION
JOB PARAMETERS
WHERE TO RUN
JOB MAIN FLOW
JOB POST-FLOW
SCRIPTED CODE

12. Tikal KnowledgeTikal Knowledge
PIPELINE JOB FORM
IN-LINE SCRIPT SCRIPT FROM SCM

13. Tikal Knowledge
START
END
LOAD/REFRESH
JOB PAGE
JOB TRIGGERS
LOAD FORM WITH
CONFIG.XML PARAMETERS
LOAD PIPELINE
UPDATE
CONFIG.XML
RUN
FLOW
CONFIG.XML
CONTAINS
PARAMETERS
JOB CONFIG.XML
PIPELINE JOB FLOW
YES
NO

14. Tikal KnowledgeTikal Knowledge
AGENT TYPES

15. Tikal KnowledgeTikal Knowledge
DOCKER & PIPELINE

16. Tikal KnowledgeTikal Knowledge
PIPELINE SYNTAX
https://www.tikalk.com/devops/how-to-add-pipeline-dsl-support-to-intellij

17. Tikal Knowledge
PROJECT COMMON FLOWS

18. Tikal Knowledge
GIT CHECKOUT
INSTALL PYTHON MODULES
RUN PYTHON SCRIPT
GIT URL
BRANCH
CREDENTIALS
PYTHON
MODULES TO
INSTALL
PYTHON
SCRIPT TO
RUN

19. Tikal Knowledge
JOB PARAMETERS
PYTHON DOCKER AGENT
CHECKOUT GIT REPOSITORY
INSTALL PYTHON MODULES
RUN PYTHON SCRIPT
DOWNSTREAM JOB

20. Tikal Knowledge
RUN SPECIFIC PYTHON SCRIPT
UPSTREAM JOB

21. Tikal Knowledge
UPSTREAM & DOWNSTREAM

22. Tikal Knowledge
POWER.
UNLIMITED POWER!
Darth Sidious

23. Tikal KnowledgeTikal Knowledge
SHARED LIBRARY
Shared Library repositoryJenkins server configuration form
library identifier: 'custom-lib@master', retriever: modernSCM(
[$class: 'GitSCMSource',
remote: 'git@git.mycorp.com:my-jenkins-utils.git',
credentialsId: 'my-private-key'])

24. Tikal KnowledgeTikal Knowledge
CUSTOM STEPS
CUSTOM ACTION

25. Tikal KnowledgeTikal Knowledge
CUSTOM STEPS

26. Tikal KnowledgeTikal Knowledge
SMART SLACK NOTIFICATIONS
v STATUS-BASED BEHAVIOUR AND MESSAGES
v AUTO CHANNEL DETECTION
v TESTS DETECTION MESSAGES
v INFORMATIVE LINKS AND HIGHLIGHTED MESSAGES
v ADVANCED STEP PARAMETERS

27. Tikal Knowledge
SO THIS IS HOW LIBERTY DIES.
WITH THUNDEROUS APPLAUSE.
Padmé Amidala

28. Tikal KnowledgeTikal Knowledge
COMPLEX FLOWS

29. Tikal Knowledge
GIT CHECKOUT
INSTALL PYTHON MODULES
RUN PYTHON SCRIPT
GIT URL
BRANCH
CREDENTIALS
PYTHON
MODULES TO
INSTALL
PYTHON
SCRIPT TO
RUN

30. Tikal Knowledge COMPLEX FLOWS
MAIN FLOW
BLUEPRINT

31. Tikal Knowledge

32. Tikal Knowledge
SEQUENTIAL TESTS RUN
RUN ALL TESTS

33. Tikal Knowledge
PARALLEL TESTS RUN
RUN EACH MODULE FILE IN PARALLEL

34. Tikal KnowledgeTikal Knowledge
PARALLEL TESTS RUN
v REDUCE OVERALL TESTS RUN
v ENABLE RUN OF FILTERED TESTS SUITES
v BETTER TESTS REPORTS
v BETTER CI
SELENIUM TESTS
API TESTS

35. Tikal Knowledge
YOU CAN’T STOP THE CHANGE,
ANY MORE THAN YOU CAN STOP
THE SUNS FROM SETTING.
Shmi Skywalker

36. Tikal Knowledge
TO BE OR NOT TO BE(IN THE SAME REPOSITORY)

37. Tikal KnowledgeTikal Knowledge
JENKINS INSIDE CODE REPOSITORY
q SHIFT-LEFT APPROACH
q CODE/BUILD DEPENDENCY CASES
q REDUCE NUMBER OF REPOSITORIES
q KEEP BUILD-CODE CLOSE TO DEV
q JENKINS CODE TEND TO BECOME STEADY
q RARE CODE/BUILD DEPENDENCY CASES
q DIFFERENT LIFE-CYCLE AND VERSIONING
q LIBRARY HIERARCHY IN SAME PLACE
q DIFFERENT VALIDATION LOGIC
q NON-EASY CROSS-REPOSITORIES REFACTORING
PROS CONS

38. Tikal Knowledge
TIPS

39. Tikal KnowledgeTikal Knowledge
MAKE SOME ORDER HERE
v VIEW PER TEAM/SUBJECT/CONTEXT
v VIEW JOBS WITH SAME PREFIX IN
NAME
v MATCHING SLACK CHANNEL PER VIEW
v (POSSIBLE) SECURITY PER VIEW
v POSSIBLE DUPLICATE INSTANCES OF
SAME PIPELINE IN DIFFERENT VIEWS

40. Tikal KnowledgeTikal Knowledge
SET NEXT BUILD NUMBER
1
2
3
4

41. Tikal KnowledgeTikal Knowledge
THE TEST FILES RIDDLE
WHAT METHOD SHOULD BE USED
FOR LISTING ALL PYTHON TEST FILES?
OR

42. Tikal KnowledgeTikal Knowledge
ENVIRONMENT VARIABLE
USE ENVIRONMENT
VARIABLES FOR
GLOBAL USAGE
VARIABLES

43. Tikal KnowledgeTikal Knowledge
ROOT DOCKER ISSUE
ROOT DOCKER ISSUE

44. Tikal Knowledge
NEVER
TELL ME THE ODDS!
Han Solo

45. Tikal KnowledgeTikal Knowledge
MISSION STATUS
q MOVE ALL ACTIVE JOBS FROM JENKINS 1.X
SERVER TO JENKINS 2.X SERVER
ü REDUCE THE NUMBER OF JOBS WHEN
POSSIBLE
ü INSTALL NOTHING ON THE JENKINS SERVER
ü CONTINUE THE SEQUENCED BUILD NUMBERS
WHEN NEEDED
ü FASTER JOBS (TOWARD CI)
ü USE JENKINS PIPELINE FOR CODING THE
JOBS
ü USE DOCKER AS THE PLATFORM TO RUN THE
JOBS ON
ü USE SLACK NOTIFICATIONS
ü UNIFIED VIEWS, JOBS, AND
NOTIFICATIONS
ü REUSABLE CODE
ü USE OF SHARED LIBRARIES
ü BASIC FLOWS IMPLEMENTATION
ü DEVELOPERS CAN WRITE NEW PIPELINES
EASILY
ü BETTER NIGHTLY CI JOB

46. Tikal KnowledgeTikal Knowledge
MISSION NEXT GOALS
q BETTER JENKINS-JOBS SECURITY
q IMPROVED CI
q BETTER VERSIONING
q HANDOVER

47. Tikal Knowledge
THERE’S ALWAYS
A BIGGER FISH.
Qui-Gon Jinn

48. Tikal KnowledgeTikal Knowledge
BLUE OCEAN
v BETTER FLOW VIEW
v PIPELINE CREATION GUI
v SLOW LOAD

49. Tikal KnowledgeTikal Knowledge
JENKINS PIPELINE UNIT
https://github.com/jenkinsci/JenkinsPipelineUnit
v TEST YOUR PIPELINE WITH MOCKS
v TEST SHARED LIBRARIES
v TESTS-AS-CODE

50. Tikal KnowledgeTikal Knowledge
v CONFIGURE JENKINS SERVER AS CODE
v VERY NEW AND INTERESTING PROJECT
v USE HTTPS://GITHUB.COM/SHELLEG/ANSIBLE-
ROLE-CUSTOM-JENKINS FOR NOW
CONFIGURATION AS CODE PLUGIN
https://github.com/jenkinsci/configuration-as-code-plugin

51. Tikal KnowledgeTikal Knowledge
v CONVERT SINGLE
FREESTYLE JOB TO
PIPELINE
v CONVERT CHAIN OF
FREESTYLE JOBS TO
SINGLE PIPELINE
AUTO-CONVERT JOB TO PIPELINE
https://jenkins.io/blog/2017/12/15/auto-convert-freestyle-jenkins-jobs-to-coded-pipeline/

52. Tikal Knowledge
THE FORCE
WILL BE WITH YOU.
ALWAYS.
Obi-Wan Kenobi

53. Tikal KnowledgeTikal Knowledge
TIKAL SHARED LIBRARY
v HTTPS://GITHUB.COM/TIKALK/TIKAL-PIPELIB
v SIMPLE STEPS
v COMPLEX FLOWS
v SHARED WITH THE COMMUNITY

54. Tikal KnowledgeTikal Knowledge
TIKAL SHARED PIPELINES
v HTTPS://GITHUB.COM/TIKALK/TIKAL-PIPELIB-TOUCHSTONE
v PIPELINE EXAMPLES
v TESTS FOR TIKAL-PIPELIB
v SHARED WITH THE COMMUNITY

55. Tikal KnowledgeTikal Knowledge
JENKINS PIPELINE HANDS-ON

56. Tikal Knowledge
YOU KNOW, NO MATTER HOW
MUCH WE FORGET, I’VE ALWAYS
HATED WATCHING YOU LEAVE.
Leia Organa