Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Git in-the-large

1,042 views

Published on

Slides from Mark Waite's Jenkins World 2017 talk on managing large git repositories in a Jenkins environment. Includes reference repositories, shallow clones, large file support, and other techniques

Published in: Software
  • Be the first to like this

Git in-the-large

  1. 1. Git in the Large – Tips & Tricks Mark Waite
  2. 2. Git in the Large – Tips & Tricks Mark Waite – mark.earl.waite@gmail.com
  3. 3. Remote • Central repo Master • Pipeline cache repo • Polling repo Agent • Pipeline cache repo • Local repo • Workspace Git in the Large - Concepts © 2017 CloudBees, Inc. All Rights Reserved
  4. 4. • Reduce remote work and network transfer with: • Reference repository • Narrow refspec • Shallow clone • Large file support Git in the Large – Help the Remote © 2017 CloudBees, Inc. All Rights Reserved • Remote repository • Includes all history • Sends requested history • Includes all large files • Sends requested large files
  5. 5. Reference Repository • New repositories can “reference” an existing repository rather than downloading the referenced data • Positive – Reduce network data transfer – Reduce local repository storage • Negative – Reference copies not automatically updated – Destroying reference repository damages all references © 2017 CloudBees, Inc. All Rights Reserved
  6. 6. Narrow Refspec • Refspec defines history retrieval start points (breadth) • Positive – If you only need one branch, narrow refspec retrieves one – Reduce local repository storage • Negative – Copies history only for matching branches – Refspec patterns severely limited © 2017 CloudBees, Inc. All Rights Reserved
  7. 7. Shallow Clone • Shallow clone defines history retrieval depth • Positive – If you only need most recent commit, use shallow clone – Reduce local repository storage • Negative – Breaks merge workflows – Change reports can be incomplete – Command line git only (no JGit) © 2017 CloudBees, Inc. All Rights Reserved
  8. 8. Large File Support • Git extension that stores large files outside repository • Positive – High performance – Very active development – Reduces local repository storage • Negative – Requires LFS extension – Requires extra support from hosting provider – No ssh support (http / https only) – No submodule support in the git plugin (yet) – Large file history requires separate request (LFS commands) © 2017 CloudBees, Inc. All Rights Reserved
  9. 9. Git in the Large – Help the Master © 2017 CloudBees, Inc. All Rights Reserved • Master • Scans repositories • Detects changes • Computes changes • Reduce master work and disc use with: • Reference repository • Large file support
  10. 10. Git in the Large – Help the Agent © 2017 CloudBees, Inc. All Rights Reserved • Agent • Fills workspace • Builds the job • Reduce agent work and disc space with: • Narrow refspec • Shallow clone • Reference repository • Large file support • Sparse checkout
  11. 11. Sparse Checkout • Sparse checkout defines directories for checkout (workspace breadth) • Positive – If you only need a subtree, sparse checkout creates it – Reduces local working directory storage • Negative – Subtree checkout is not the full tree © 2017 CloudBees, Inc. All Rights Reserved

×