Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Git for jenkins faster and better

53 views

Published on

Techniques to improve performance and disc use of Jenkins jobs that store their source code in git repositories. Presented as a lightning talk at Jenkins World 2019, August 14, 2019 by Mark Waite

Published in: Software
  • Be the first to comment

  • Be the first to like this

Git for jenkins faster and better

  1. 1. Git for Jenkins Faster and Better Reducing Clone Time and Disc Use
  2. 2. Git for Jenkins Faster and Better Reducing Clone Time and Disc UseLightning Talk
  3. 3. © 2019 All Rights Reserved. 3 Git for Jenkins – Faster and Better Reducing Clone Time and Disc Use Git • Repo Master • Polling • Pipeline Agents • Workspace • Pipeline
  4. 4. © 2019 All Rights Reserved. 4 Help the Remote • Remote Repository Content & Tasks • Includes all history • Sends requested history • Includes all large files • Sends requested large files • Techniques to Help the Remote • Reference repository • Narrow refspec • Shallow clone • Large file support Git • Repo Master • Polling • Pipeline Agents • Workspace • Pipeline This Photo by Unknown Author is licensed under CC BY-SA
  5. 5. © 2019 All Rights Reserved. 5 Help the Master • Master Content & Tasks • Needs all history • Requests most recent large files • Scans repositories for changes • Computes changes • Clones and executes pipelines and their shared libraries • Techniques to Help the Master • Reference repository • Large file support Git • Repo Master • Polling • Pipeline Agents • Workspace • Pipeline
  6. 6. © 2019 All Rights Reserved. 6 Help the Agents • Agent Content & Tasks • Needs most recent files • Needs most recent large files • Workspace checkout • Techniques to Help the Agent • Narrow refspec • Shallow clone • Reference repository • Large file support • Sparse checkout Git • Repo Master • Polling • Pipeline Agents • Workspace • Pipeline
  7. 7. © 2019 All Rights Reserved. 7 Reference Repository • New repositories can “reference” another repository Avoid download and storage of existing data • Advantage • Reduce network data transfer • Reduce local workspace storage • Disadvantage • Reference repositories not automatically update • Destroying reference copy damages all referring workspaces Git • Repo Master • Polling • Pipeline Agents • Workspace • Pipeline
  8. 8. © 2019 All Rights Reserved. 8 Narrow Refspec • Refspec limits history retrieval (breadth of history – request a subset of branches) • Advantage • Retrieve only the required branches • Reduce local repository storage • Disadvantage • Copies history only for the requested subset • Refspec patterns are limited Git • Repo Master • Polling • Pipeline Agents • Workspace • Pipeline
  9. 9. © 2019 All Rights Reserved. 9 Shallow Clone • Shallow clone limits history retrieval depth • Advantage • Reduce workspace storage by subset clone • Disadvantage • Breaks merge workflows • Incomplete change history reports • Command line git only (no JGit support) Git • Repo Master • Polling • Pipeline Agents • Workspace • Pipeline
  10. 10. © 2019 All Rights Reserved. 10 Large File Support • Extension stores large files outside the repository • Advantage • High performance (parallel download, https transport, …) • Very active development • Reduces local repository storage • Disadvantage • Requires LFS extension on agents and master • Requires extra support from hosting provider • Requires authentication and remote access for merge Git • Repo Master • Polling • Pipeline Agents • Workspace • Pipeline
  11. 11. © 2019 All Rights Reserved. 11 Sparse Checkout • Sparse checkout defines directories for checkout (workspace breadth) • Advantage • Reduce workspace storage by subset checkout • Disadvantage • Reduced workspace does not include all workspace files Git • Repo Master • Polling • Pipeline Agents • Workspace • Pipeline
  12. 12. © 2019 All Rights Reserved. 12 Pipeline Specifics • Use the Pipeline Syntax helper for checkout syntax • Use reference repositories • If using Git as branch source, define narrow refspec to specific branch • Narrow refspec for specific branch is automatic with higher level branch sources • Use GitHub rather than Git as branch source • Use Bitbucket rather than Git as branch source • Use Gitea rather than Git as branch source Git • Repo Master • Polling • Pipeline Agents • Workspace • Pipeline
  13. 13. DevOptics Software Delivery Visibility & Insights Core Unified Software Delivery & Governance CodeShip CI/CD as a Service Flow Adaptive Release Orchestration DevOps ExcellenceJenkins CloudBees Jenkins Distribution CloudBees Jenkins X Distribution 24x7 Technical Support Assisted Updates Support Accelerator Training Customer Success Managers DevOps Consultants Rollout Feature Flag Management Continuous Delivery Products and Services

×