Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Jedi Mind Tricks for Git

1,377 views

Published on

Slides for a pre-conference workshop I delivered together with Johan Abildskov (@randomsort) at Git Merge 2017 in Brussels.

In the workshop we covered fun things to do with Git hooks, Git attributes and custom drivers.
In the first half, we demonstrate how you can implement a fully local continuous integration workflow using git hooks.
In the second half, we cover cool and creative ways to diff binary files and custom filters for modifying file content while commit'ing.

Published in: Technology
  • Be the first to comment

Jedi Mind Tricks for Git

  1. 1. Jedi Mind Tricks in Git Help me Obi-Git, you’re my only hope
  2. 2. Who are we? @jankrag - @randomsort Git Trainers
  3. 3. “Oh my. Software development sound rather perilous. I can assure you they will never get me on one of those dreadful Computers” - C3PO
  4. 4. this -is a command This is the console output of a command It can be coloured as well this completed successfully. #!/foo/bar This is the content of a script [config] Or the content of a config file hello=git merge Conventions
  5. 5. Working with git hooks
  6. 6. What are git hooks? We can jam a hook into Git’s normal flow
  7. 7. Git hook control flow CompleteFinalizeWrite MsgCommitStage precommit Prepare commit msg Commit msg postcommit Notification Actionable
  8. 8. Yes - but how do I get started? git init
  9. 9. Making Git hooks ls .git/hooks applypatch-msg.sample commit-msg.sample post-update.sample pre-commit.sample prepare-commit-msg.sample pre- rebase.sample pre-applypatch.sample pre-push.sample update.sample
  10. 10. Client side vs server side hooks precommit prepare-commit-msg commit-msg post-commit post-checkout pre-rebase pre-receive update post-receive
  11. 11. The truth is out there... Specifically on master
  12. 12. pre-commit hook ● First thing that happens after git commit ● Able to abandon the commit ● We have not got any arguments
  13. 13. Where are we at? git symbolic-ref --short HEAD master
  14. 14. pre-commit hook #!/bin/bash # Check where our HEAD is at if [ "$(git symbolic-ref --short HEAD)" == "master" ]; then echo "This is not the branch you are looking for" exit 1 fi exit 0
  15. 15. pre-commit hook #!/bin/bash # Check where our HEAD is at if [ "$(git symbolic-ref --short HEAD)" == "master" ]; then echo "This is not the branch you are looking for" exit 1 fi exit 0
  16. 16. pre-commit hook #!/bin/bash # Check where our HEAD is at if [ "$(git symbolic-ref --short HEAD)" == "master" ]; then echo "This is not the branch you are looking for" exit 1 fi exit 0
  17. 17. cp pre-commit .git/hooks/pre-commit git checkout master git commit -am “I want to commit this” This is not the branch you’re looking for
  18. 18. Well, I should be able to commit on master git checkout master git commit -n -am “I want to commit this” [master 5d5308b] I want to commit this 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 foo.md
  19. 19. The mission ● I want to reference an issue in my commit messages ● If I do not reference an issue in my repository - abandon my commit
  20. 20. Where are our issues? Git remote show origin * remote origin Fetch URL: git@github.com:RandomSort/hooks Push URL: git@github.com:RandomSort/hooks HEAD branch: master Remote branch: master tracked Local branch configured for 'git pull': master merges with remote master Local ref configured for 'git push': master pushes to master (up to date)
  21. 21. I <3 APIS curl -s https://api.github.com/repos/$repo/issues/$issue_number { "url": "https://api.github.com/repos/RandomSort/hooks/issues/1", "repository_url": "https://api.github.com/repos/RandomSort/hooks", "labels_url": "https://api.github.com/repos/RandomSort/hooks/issues/1/labels {/name}", .... TONS OF JSON :D
  22. 22. commit-msg hook #!/bin/bash # $1 is the temp file containing the commit message issue_number=`cat $1 | xargs | sed 's/.*#([0-9]*).*/1/'` repo="`git remote show origin | grep "Push" | xargs | sed 's/.*:(.*)/1/'`" curl -s https://api.github.com/repos/$repo/issues/$issue_number | grep -q ""Not Found"" > /dev/null exit $((1 -$?))
  23. 23. commit-msg hook #!/bin/bash # $1 is the temp file containing the commit message issue_number=`cat $1 | xargs | sed 's/.*#([0-9]*).*/1/'` repo="`git remote show origin | grep "Push" | xargs | sed 's/.*:(.*)/1/'`" curl -s https://api.github.com/repos/$repo/issues/$issue_number | grep -q ""Not Found"" > /dev/null exit $((1 -$?))
  24. 24. commit-msg hook #!/bin/bash # $1 is the temp file containing the commit message issue_number=`cat $1 | xargs | sed 's/.*#([0-9]*).*/1/'` repo="`git remote show origin | grep "Push" | xargs | sed 's/.*:(.*)/1/'`" curl -s https://api.github.com/repos/$repo/issues/$issue_number | grep -q ""Not Found"" > /dev/null exit $((1 -$?))
  25. 25. commit-msg hook #!/bin/bash # $1 is the temp file containing the commit message issue_number=`cat $1 | xargs | sed 's/.*#([0-9]*).*/1/'` repo="`git remote show origin | grep "Push" | xargs | sed 's/.*:(.*)/1/'`" curl -s https://api.github.com/repos/$repo/issues/$issue_number | grep -q ""Not Found"" > /dev/null exit $((1 -$?))
  26. 26. So now we’ve done something sane .. ● Git hooks can do useful stuff ● Everything in moderation ● (bust the myth, then go all in)
  27. 27. Trust your workflow "Fear is the path to the dark side"
  28. 28. Branch based delivery C1 C2 C3 C4 ready/feature master HEAD git push origin feature:ready/feature
  29. 29. #serverless These are not the servers you are looking for
  30. 30. Going native
  31. 31. The mission ● I want to work on a feature branch never on master ● I want to commit work referencing issues ● I want to be able to run my Continuous Integration setup ● I deliver through ready branches ● Only allow integration if ○ Tests succeed ○ it’s a fast forward merge
  32. 32. Where are we at (again)? git symbolic-ref --short HEAD ready/foo git rev-parse --show-toplevel /home/randomsort/repos/hooks .. And where’s our stuff?
  33. 33. We only want to do fast forward merges ● It’s prettiest ● We prefer to have our automation engine only do trivial merges C1 C2 C3 C4 Feature#79 master HEAD
  34. 34. We only want to do fast forward merges ● It’s prettiest ● We prefer to have our automation engine only do trivial merges C1 C2 C3 C4 Feature#79 master HEAD
  35. 35. How fast can we go? git rev-list --maxcount 1 master..ready/foo C1 C2 C3 C4 ready/foo master HEAD
  36. 36. Putting it together #!bin/bash if [ not on ready branch ] do nothing fi if [ not fast forwardable ] cleanup fi run tests if [ successful ] merge cleanup else cleanup
  37. 37. Putting it together #!bin/bash if [ not on ready branch ] do nothing fi if [ not fast forwardable ] cleanup fi run tests if [ successful ] merge cleanup else cleanup
  38. 38. Putting it together #!bin/bash if [ not on ready branch ] do nothing fi if [ not fast forwardable ] cleanup fi run tests if [ successful ] merge cleanup else cleanup
  39. 39. Cleaning up fails #We can't do a fast forward merge so let's abort this and move some stuff around failbranch=`echo $branch | xargs | sed 's/ready/failed/'` echo "Unable to fast forward master to $branch leaving state on $failbranch" git checkout -b $failbranch git branch -D $branch
  40. 40. Cleaning up fails #We can't do a fast forward merge so let's abort this and move some stuff around failbranch=`echo $branch | xargs | sed 's/ready/failed/'` echo "Unable to fast forward master to $branch leaving state on $failbranch" git checkout -b $failbranch git branch -D $branch
  41. 41. Running git checkout -b ready/feature Switched to a new branch 'ready/feature' Running tests Testing has failed, leaving state on failed/feature Switched to a new branch 'failed/feature' Deleted branch ready/feature (was 462b135).
  42. 42. Running git checkout -b ready/feature Switched to a new branch 'ready/feature' Running tests Testing has failed, leaving state on failed/feature Switched to a new branch 'failed/feature' Deleted branch ready/feature (was 462b135).
  43. 43. Running git checkout -b ready/feature Switched to a new branch 'ready/feature' Running tests Testing has failed, leaving state on failed/feature Switched to a new branch 'failed/feature' Deleted branch ready/feature (was 462b135).
  44. 44. Running git checkout -b ready/feature Switched to a new branch 'ready/feature' Running tests Testing has failed, leaving state on failed/feature Switched to a new branch 'failed/feature' Deleted branch ready/feature (was 462b135).
  45. 45. Running with successful tests git checkout -b ready/feature Switched to a new branch 'ready/feature1' Running tests Switched to branch 'master' Updating e28500e..06a16b1 Fast-forward temp.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 temp.md Deleted branch ready/feature1 (was 06a16b1).
  46. 46. It works on my machine "I find your lack of faith disturbing"
  47. 47. Tatooine Jan
  48. 48. TATOOINE - LARS OWENS HOMESTEAD OWEN: I have no need for a protocol droid. C3P0: Sir -- not in an environment such as this -- that's why I've also been programmed for over thirty secondary functions that... OWEN: What I really need is a droid that understands the binary language of moisture vaporators.
  49. 49. ... secondary functions "I have no need for a protocol droid."
  50. 50. Binary is annoying... git diff wordfile.docx diff --git a/wordfile.docx b/wordfile.docx index d24c436..3c4aa59 100644 Binary files a/wordfile.docx and b/wordfile.docx differ No idea what changed...
  51. 51. Teaching git new tricks Combining two features: Git attributes Assign behaviour per file or path Custom drivers Define new behaviour
  52. 52. Git attributes Each line in an attributes definition is of form: pattern attr1 attr2 ... Where a attribute can be set, unset or assigned a value: *.txt text *.jpg -text *.sh text eol=lf
  53. 53. Git attributes - global Default location: $XDG_CONFIG_HOME/git/attributes. Fallback: $HOME/.config/git/attributes If you want to set the path you can use: git config --global core.attributesfile <path> git config --global core.attributesfile ~/.gitattributes e.g.
  54. 54. Git attributes - local Just add definitions to a local .gitattributes file. Can be set in root folder and in subfolders
  55. 55. Git attributes commonly used for changing (force'ing): CRLF behaviour text / binary behaviour But that is not Jedi enough ...... so we will skip those
  56. 56. Diff driver Set as a config parameter like diff.foo [diff "hexify"] binary = true textconv = hexdump -v -C
  57. 57. Using the new driver is easy In .gitattributes: *.bin diff=hexify Note - we need both: Driver definition in config Path attribute that tells diff to use it
  58. 58. But hexdump is still pretty useless 00000000 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 |.PNG........IHDR| 00000010 00 00 03 20 00 00 02 80 08 02 00 00 00 eb 79 8b |... ..........y.| 00000020 65 00 00 00 09 70 48 59 73 00 00 0b 13 00 00 0b |e....pHYs.......| 00000030 13 01 00 9a 9c 18 00 00 00 06 62 4b 47 44 00 ff |..........bKGD..| 00000040 00 ff 00 ff a0 bd a7 93 00 00 8a b5 49 44 41 54 |............IDAT| 00000050 78 da ec bd 09 5b 53 57 f7 fe df f7 55 32 9d cc |x....[SW....U2..| 00000060 33 09 99 e7 79 20 09 19 50 91 41 90 41 26 41 44 |3...y ..P.A.A&AD| 00000070 44 14 11 41 04 04 ad b5 b5 b6 b5 b5 56 ed ef 85 |D..A........V...| 00000080 fd 17 a5 7f bf 3e 4a 4e 42 38 c9 39 27 e7 be ae |.....>JNB8.9'...| 00000090 cf d5 cb c7 47 25 39 67 ef b5 ee bd f7 da f7 fa |....G%9g........| 000000a0 ae a7 e7 7b 00 00 00 00 00 c0 21 df e1 11 00 00 |...{......!.....| 000000b0 00 00 00 40 60 01 00 00 00 00 40 60 01 00 00 00 |...@`.....@`....| 000000c0 00 40 60 01 00 00 00 00 00 08 2c 00 00 00 00 00 |.@`.......,.....| 000000d0 08 2c 00 00 00 00 00 08 2c 00 00 00 00 00 00 81 |.,......,.......| 000000e0 05 00 00 00 00 00 81 05 00 00 00 00 00 81 05 00 |................| 000000f0 00 00 00 00 20 b0 00 00 00 00 00 20 b0 00 00 00 |.... ...... ....| 00000100 00 00 20 b0 00 00 00 00 00 20 b0 00 00 00 00 00 |.. ...... ......| 00000110 00 04 16 00 00 00 00 00 04 16 00 00 00 00 00 04 |................| <pages and pages without end>
  59. 59. Recipe for success For each binary type: 1. Find a tool that extracts useful info from a file 2. Set up a diff driver to use it 3. Profit
  60. 60. Word files
  61. 61. My brain parses markdown quite fine... *.docx diff=pandoc2md [diff "pandoc2md"] textconv=pandoc --to=markdown brew install pandoc For reference: .gitattributes config
  62. 62. git diff wordfile.docx diff --git a/wordfile.docx b/wordfile.docx index d24c436..48249e3 100644 --- a/wordfile.docx +++ b/wordfile.docx @@ -1,8 +1,8 @@ -Headline +Hi Git Merge This is some simple text -*and another important paragraph in italic* +*and another* **important** *paragraph in italic* Word to MarkDown
  63. 63. git diff --word-diff=color wordfile.docx diff --git a/wordfile.docx b/wordfile.docx index d24c436..48249e3 100644 --- a/wordfile.docx +++ b/wordfile.docx @@ -1,8 +1,8 @@ HeadlineHi Git Merge This is some simple text *and another* **important** *paragraph in italic* Word to MarkDown
  64. 64. Other ideas?
  65. 65. PDF files *.pdf diff=pdfconv [diff "pdfconv"] textconv=pdftohtml -stdout brew install pdftohtml <BODY bgcolor="#A0A0A0" vlink="blue" link="blue"> <A name=1></a>Headline<br> This is some simple text<br> -and another paragraph<br> -and stuff<br> +and another important paragraph in italic<br> +stuff goes here<br> +Greetings from Denmark<br> <hr> </BODY>
  66. 66. Maybe even for non-binaries?
  67. 67. Syntax highlighting *.py diff=color pip install Pygments For reference: .gitattributes config [diff "color"] textconv=pygmentize
  68. 68. git diff --cached primes.py ... @@ -0,0 +1,22 @@ +def primes(n): + """ + Compute the sequence of the first n primes. + """ + p = [] + if n > 0: + p = [2, ] + for x in range(1, n): + q = p[-1] + 1 + while True: + for y in p: + if not (q % y): + break + else: + p.append(q) Syntax highlighted code in diff's My brain parses coloured python better Snippet from: https://github.com/cbcunc/primer (GPL)
  69. 69. Reformatting *.md diff=wrap brew install fmt For reference: .gitattributes config [diff "wrap"] textconv=fmt
  70. 70. git diff autostash.md >"Why can't I pull when I have a dirty workspace, when Mercurial can do this out of the box?" -I gave the immediate answer that this is just Git's way of protecting the user from possibly harmful and, more importantly, irreversible changes. Git by default takes a very paranoid approach to any operations that change dirty files in your file system, when Git itself can't get you out of those changes again. _This is normally considered a feature_. The known "workaround", or possible workflow, is to stash any changes before doing a pull (with `git stash save`, and then unstash them again (`git stash pop`) when done. It seems obvious that it should be easy to automate this with a git alias, but it turns out that this isn't trivial, as git stash doesn't fail gracefully when there are no local changes. +I gave the immediate answer that this is just Git's way of protecting the user from possibly harmful and, more importantly, irreversible changes. Git by default takes a very paranoid approach to any operations that change dirty files in your file system, when **Git** itself can't get you out of those changes again. _This is normally considered a feature_. The known "workaround", or possible workflow, is to stash any changes before doing a pull (with `git stash save`, and then unstash them again (`git stash pop`) when done. It seems obvious that it should be easy to automate this with a git alias but it turns out that this isn't trivial. Git stash does not fail gracefully when there are no local changes. .md - Before reformatting
  71. 71. git diff autostash.md @@ -13,13 +13,13 @@ can do this out of the box?" I gave the immediate answer that this is just Git's way of protecting the user from possibly harmful and, more importantly, irreversible changes. Git by default takes a very paranoid approach to any -operations that change dirty files in your file system, when Git +operations that change dirty files in your file system, when **Git** itself can't get you out of those changes again. _This is normally considered a feature_. The known "workaround", or possible workflow, is to stash any changes before doing a pull (with `git stash save`, and then unstash them again (`git stash pop`) when done. It seems -obvious that it should be easy to automate this with a git alias, -but it turns out that this isn't trivial, as git stash doesn't fail +obvious that it should be easy to automate this with a git alias +but it turns out that this isn't trivial. Git stash does not fail gracefully when there are no local changes. .md - after formatting with fmt
  72. 72. Images
  73. 73. Extract meta data Define a useful driver: [diff "exif"] textconv=exiftool *.jpg diff=exif *.jpeg diff=exif *.png diff=exif *.gif diff=exif
  74. 74. https://openclipart.org/detail/227445/yoda Let's resize Yoda
  75. 75. git diff Yoda-800px.png diff --git a/Yoda-800px.png b/Yoda-800px.png index fc8ee0f..8ed738c 100644 --- a/Yoda-800px.png +++ b/Yoda-800px.png @@ -1,24 +1,26 @@ -File Size : 35 kB -File Modification Date/Time : 2017:01:27 17:29:06+01:00 +File Size : 85 kB +File Modification Date/Time : 2017:01:30 18:52:20+01:00 File Type : PNG -Image Width : 800 -Image Height : 640 +Image Width : 600 +Image Height : 480 Bit Depth : 8 -Color Type : RGB +Color Type : RGB with Alpha Much more useful Note: manually abbreviated for slide But do I know what changed?
  76. 76. Can we see the difference? cachetextconv caches the result [diff "jpg"] textconv=jp2a --width=80 cachetextconv = true *.jpg diff=jpg *.jpeg diff=jpg Can't quite do holograms yet, but... brew install jp2a
  77. 77. git add Yoda.jpg git diff --cached diff --git a/Yoda.jpg b/Yoda.jpg new file mode 100644 index 0000000..cf0300a --- /dev/null +++ b/Yoda.jpg @@ -0,0 +1,32 @@ +............................................................................... . +......................................''''..................................... . +................................,:ldxkkOOkkxdl:,............................... . +.............................,lxOOdlllxOOxllldOkxc'............................ . +....',:ccllllollc:,'.......'lkOxllllllxOOxllllllkOkl'.......',:cllooollcc:,'... . +..;okOOkccccclllodxkxoc;'.:xOOOkkxddddkOOkodddxkkOOOx;.';coxkxdollccccclkOOko;. . +....':dOxl;;:::::;;;:ldx:oOxolllloddxxkOOkxxddollllokklcxdl:;;;:::::;:okkd:'... .
  78. 78. https://openclipart.org/detail/227445/yoda Let's give him eyes
  79. 79. diff --git a/Yoda.jpg b/Yoda.jpg index cf0300a..c16b7de 100644 --- a/Yoda.jpg +++ b/Yoda.jpg @@ -9,8 +9,8 @@ ........'okOOd;;:::::,okOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOkl,:::::;;dOOkl'........ ..........;dOOkl;;:;:xOOOOOOOkkOOOOOOOOOOOOOOOOOOkkOOOOOOOx;;:;;okOkd,.......... ............;okOkd,ckOOOOOkc,..,oOOOOOOOOOOOOOkl,..,lkOOOOOk;;dkOkl,............ - ..............';lo;kOOOOOOl......xOOOOOOOOOOOOd......oOOOOOOk;dl;............... - ..................lOOOOOOOk;...'ckOOOOOOOOOOOOk:'...:kOOOOOOOc.................. +..............';lo;kOOOOOOl'oxc..xOOOOOOOOOOOOd,dd:..oOOOOOOk;dl;.............. . +..................lOOOOOOOk:odc'ckOOOOOOOOOOOOkldd:.:kOOOOOOOc................. . git diff Yoda.jpg
  80. 80. Even in gitk
  81. 81. Other quick ones MP3 file meta data: exiftool can do those too, or try mp3info: [diff "mp3"] textconv=mp3info -x Excel files: [diff "xlsconv"] textconv=xls2csv Zip files: [diff "zipshow"] textconv=unzip -c -a [diff "ziplist"] textconv=unzip -l
  82. 82. Length Date Time Name --------- ---------- ----- ---- 0 01-25-2017 12:52 stuff/ 0 01-25-2017 12:53 stuff/foo/ 7 01-25-2017 12:53 stuff/foo/bar 12 01-25-2017 12:52 stuff/hello.world - 9 01-25-2017 12:52 stuff/hi.txt --------- ------- - 28 5 files + 19 4 files git diff stuff.zip
  83. 83. And we haven't even left Tatooine yet!
  84. 84. Recap - Git objects https://github.com/pluralsight/git-internals-pdf
  85. 85. Filter drivers Process blobs on save or checkout [filter "name"] clean = <command> smudge = <command> Let's have some fun with them too...
  86. 86. There is no try... Yapf = Yet Another Python Formatter ( pip install --user yapf ) [filter "cleanpython"] clean = yapf smudge = cat *.py filter=cleanpython
  87. 87. ugly.py x = { 'a':37,'b':42, 'c':927} y = 'hello ''world' z = 'hello '+'world' a = 'hello {}'.format('world' ) class foo ( object ): def f (self ): return 37*-+2 def g(self, x,y=42): return y def f ( a ) : return 37+-+a[42-x : y**3] git add ugly.py git commit .... git push
  88. 88. ... and behold.
  89. 89. Caveat That was a conceptual demo - probably not production ready. Git diff gets a bit confused: git status now shows ugly.py as modified git add - and it disappears :-)
  90. 90. “Your eyes can deceive you. Don’t trust them.” – Obi-Wan For demo: simple rot13 'encryption' - only affects letters [filter "secret"] clean = ruby -ne 'print $_.tr( "A-Za-z", "N-ZA-Mn-za- m") ' smudge = perl -pe 'tr/N-ZA-Mn-za-m/A-Za-z/' *.java filter=secret
  91. 91. public static void sort(int[] numbers) { int n = numbers.length; int temp = 0; for (int i = 0; i < n; i++) { for (int j = 1; j < (n - i); j++) { if (numbers[j - 1] > numbers[j]) { //comment temp = numbers[j - 1]; numbers[j - 1] = numbers[j]; numbers[j] = temp; } } } } sort.java
  92. 92. git ls-tree HEAD 100644 blob cf0300a887b362db6e26891afb6ad86757e00f72 Yoda.jpg 100644 blob 60f95e16c59e6e0ef7dc5542cc0f5c15a35e9df4 primes.py 100644 blob 887177cdd3976c695ec5303e6a6d2c1832b458a9 sort.java 100644 blob 809ec4f968232287368681257a0b5ad5c726c08a ugly.py git show 887177cdd choyvp fgngvp ibvq fbeg(vag[] ahzoref) { vag a = ahzoref.yratgu; vag grzc = 0; sbe (vag v = 0; v < a; v++) { sbe (vag w = 1; w < (a - v); w++) { vs (ahzoref[w - 1] > ahzoref[w]) { grzc = ahzoref[w - 1];
  93. 93. And on GitHub too, obviously
  94. 94. Now just imagine we had used GPG or AES-256 with private key *.cred filter=secret Publish your private content to public repos :-)
  95. 95. The End "Already know you that which you need." – Yoda

×