Download as PDF, PPTX
![@kannonboy
$ cat ~/.gitconfig
[filter "lfs"]
clean = git-lfs clean %f
smudge = git-lfs smudge %f
required = true](https://image.slidesharecdn.com/trackinghugefileswithgitlfs-linuxconjp-160715065720/85/Tracking-huge-files-with-Git-LFS-LinuxCon-2016-25-320.jpg){kind=tracking}
![@kannonboy
POST .../repo.git
I need object cafebabe
@kannonboy
/info/lfs/objects/batch
{“objects”:[
{
“oid”: “cafebabe...”,
“size”: 40689401
}, ...
],
“operation”: “download”}](https://image.slidesharecdn.com/trackinghugefileswithgitlfs-linuxconjp-160715065720/85/Tracking-huge-files-with-Git-LFS-LinuxCon-2016-33-320.jpg){kind=tracking}
![@kannonboy
$ git lfs lock <path>
$ git lfs unlock <path>
$ git lfs locks [-i id] [-p path]](https://image.slidesharecdn.com/trackinghugefileswithgitlfs-linuxconjp-160715065720/85/Tracking-huge-files-with-Git-LFS-LinuxCon-2016-76-320.jpg){kind=tracking}
More Related Content
Similar to Tracking huge files with Git LFS - LinuxCon 2016
Tracking huge files with Git LFS - LinuxCon 2016
- 1. @kannonboy Photo: Le Mondeen Vidéo @kannonboy
- 2.
- 3. @kannonboy Git LFS! Git LOB! @kannonboy Photo:Le Monde en Vidéo
- 4. @kannonboy ok cool @kannonboy Photo: LeMonde en Vidéo Git LFS!
- 5. TIM PETTERSEN •SENIOR DEVELOPER • ATLASSIAN • @KANNONBOY Tracking huge files with Git LFS
- 6. @kannonboy GIT LFS THE PROBLEMWITH BIG FILES Agenda CONVERTING YOUR REPO TIPS FOR TEAMS
- 7.
- 8.
- 9. @kannonboy 434bb..tree bab1e..parent Tim P <kannonboy@…>1455209277 -0800committer Tim P <kannonboy@…> 1455209277 -0800author My life is my commit message. 98ca9.. bab1e.. fad3d.. git cat-file -p 98ca9$
- 10. @kannonboy git cat-file -p434bb$ 434bb.. 98ca9.. bab1e.. fad3d.. ace23..100644 blob .gitignore dbdbd..100644 blob README.md a0bc3..040000 tree app 33d33..040000 tree config b1de7..100755 blob deploy-prod.sh 7011e..100755 blob deploy-staging.sh typefilemode SHA-1
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24. @kannonboy $ brew installgit-lfs $ git lfs install
- 25. @kannonboy $ cat ~/.gitconfig [filter"lfs"] clean = git-lfs clean %f smudge = git-lfs smudge %f required = true
- 26. @kannonboy $ git lfstrack “*.mp4” $ cat .gitattributes *.mp4 filter=lfs diff=lfs merge=lfs -text
- 27. @kannonboy massive_video.mp4 Work tree dev .git/lfs/objects Clean filter (git-lfsclean) ☞ Index massive_video.mp4 $ .git/objects git add
- 28. @kannonboy $ dev .git/lfs/objects Smudge filter (git-lfs smudge) Worktree massive_video.mp4 ☞ Commit tree massive_video.mp4 .git/objects LFS Store git checkout
- 29.
- 30.
- 31. @kannonboy $ git push GitLFS: (12 of 13 files, 1 skipped) 168.75 MB / 180.87 MB, 12.12 skipped Counting objects: 22, done. ...
- 32. @kannonboy $ git pull remote:Counting objects: 3, done. ... Downloading massive_video.mp4 (38.79 MB) ... 1 file changed, 2 insertions(+)
- 33. @kannonboy POST .../repo.git I needobject cafebabe @kannonboy /info/lfs/objects/batch {“objects”:[ { “oid”: “cafebabe...”, “size”: 40689401 }, ... ], “operation”: “download”}
- 34. @kannonboy 200 OK {“objects”:[ {“oid”: “cafebabe…”,“size”: 40689401, “actions”: { “download”: { “href”: “https://…/lfs/cafebabe…”, @kannonboy cafebabe is over there } } ...
- 35. @kannonboy 200 OK {“objects”:[ {“oid”: “cafebabe…”,“size”: 40689401, “actions”: { “download”: { “href”: “https://…/lfs/cafebabe…”, @kannonboy cafebabe is over there “header”: { “Authorization”: “JWT eyJ0eXA…”, } } } ...
- 36.
- 37.
- 38.
- 39. @kannonboy git filter-branch $ gitfilter-branch --force --index-filter 'git rm --cached --ignore-unmatch big_video.mp4’ --prune-empty --tag-name-filter cat -- --all DON’T DO THIS!
- 40. @kannonboy $ git filter—branch--prune-empty --tree-filter ' git config -f .gitconfig lfs.url “https://bitbucket.example.com/team/repo.git” git lfs track "*.mp4" git add .gitattributes .gitconfig for file in $(git ls-files | xargs git check-attr filter | grep "filter: lfs" | sed -r "s/(.*): filter: lfs/1/"); do git rm -f --cached ${file} git add ${file} done' --tag-name-filter cat -- --all @kannonboy DON’T DO THIS EITHER!
- 41.
- 42. @kannonboy BFG Repo-Cleaner @kannonboy 10-720x faster thanfilter-branch built to kill history Git LFS support by @rtyley
- 43. @kannonboy $ git filter—branch--prune-empty --tree-filter ' git config -f .gitconfig lfs.url “https://bitbucket.example.com/team/repo.git” git lfs track "*.mp4" git add .gitattributes .gitconfig for file in $(git ls-files | xargs git check-attr filter | grep "filter: lfs" | sed -r "s/(.*): filter: lfs/1/"); do git rm -f --cached ${file} git add ${file} done' --tag-name-filter cat -- --all @kannonboy DON’T DO THIS EITHER!
- 44. @kannonboy $ brew installbfg $ bfg —-convert-to-git-lfs ‘*.{zip,mp4}’ --no-blob-protection
- 45.
- 46.
- 47. @kannonboy $ a295ef4… 102437 95372 2cc7063…152171 140443 blob SHA size on disk average blob size generate-larger-than 50000 Identifying large objects
- 48. @kannonboy a295ef4… 102437 95372 2cc7063…152171 140443 , PNG, 1148 x 482 , PNG, 1101 x 800 $ generate-larger-than 50000 | add-file-info Identifying large objects
- 49. @kannonboy $ generate-larger-than 50000 | add-file-info 2cc7063… 152171 140443, PNG, 1101 x 800 a295ef4… 102437 95372, PNG, 1148 x 482 order by average blob size | sort -k3nr Identifying large objects
- 50. @kannonboy $ generate-larger-than 50000 | add-file-info | sort -k3nr $ report-on-large-objects big-stuff.txt logo-hdpi.png 2cc7063… 152171 140443, PNG… logo-mdpi.png a295ef4… 102437 95372, PNG… paths > big-stuff.txt Identifying large objects
- 51. @kannonboy $ bfg —-convert-to-git-lfs‘logo-*.png’ --no-blob-protection Identifying large objects
- 52.
- 53.
- 54.
- 55. @kannonboyBeware merge conflicts@kannonboy
- 56.
- 57.
- 58.
- 59.
- 60.
- 61.
- 62. @kannonboy master feature0 feature1 File locking (singlebranch model) LAME
- 63.
- 64.
- 65.
- 66.
- 67.
- 68.
- 69.
- 70.
- 71.
- 72.
- 73.
- 74.
- 75.
- 76. @kannonboy $ git lfslock <path> $ git lfs unlock <path> $ git lfs locks [-i id] [-p path]
- 77.
- 78.
- 79. @kannonboy $ git lfsfetch master feature0 feature1
- 80. @kannonboy $ git lfsfetch $ git config lfs.fetchrecentalways “true” master feature0 feature1 --recent T-minus 7 days
- 81. @kannonboy $ git configlfs.fetchrecentalways “true” lfs.fetchrecentrefsdays lfs.fetchrecentremoterefs lfs.fetchrecentcommitsdays (default = 7) (default = 0) $ git lfs fetch --recent
- 82.
- 83.
- 84. @kannonboy lfs.pruneoffsetdays lfs.pruneverifyremotealways (default = 3) Setthis! $ git lfs prune
- 85. @kannonboyFetch the barenecessitiesFetch the bare necessities @kannonboy © Disney
- 86. @kannonboy # for abuild that just runs the unit tests $ git lfs fetch --exclude Assets/** # for an audio engineer $ git lfs fetch --include Assets/Audio/**
- 87. @kannonboy # for abuild that just runs the unit tests $ git lfs fetch --exclude Assets/** # for an audio engineer $ git lfs fetch --include Assets/Audio/** $ git config lfs.fetchexclude Assets/** $ git config lfs.fetchinclude Assets/Audio/**
- 88. @kannonboy coming very soon! BitbucketCloud git-lfs.github.com docs github.com/github/git-lfs source atlassian.com/bitbucket Bitbucket Server Looking for more?
- 89. @kannonboy coming very soon! BitbucketCloud git-lfs.github.com docs github.com/github/git-lfs source atlassian.com/bitbucket Bitbucket Server Looking for more?
- 90. @kannonboy coming very soon! BitbucketCloud git-lfs.github.com docs github.com/github/git-lfs source atlassian.com/bitbucket Bitbucket Server Looking for more? Follow me for occasional Git, Bitbucket & JIRA trivia