Advanced Continuous Integration - PyGrunn 2014 - Dirk Zittersteyn
•Download as PPTX, PDF•
2 likes•2,131 views
Presenting a short introduction to version control, and how to set up your tools to make sure you can always deploy. Download the version with annotations here: https://www.dropbox.com/s/nweywk72ztldphf/Advanced%20CI%20Pygrunn%20-%20standalone.pptx Reinout van Rees wrote a summary of the presentation, which can be found on his blog: http://reinout.vanrees.org/weblog/2014/05/09/continuous-integration.html
Recommended
More Related Content
Similar to Advanced Continuous Integration - PyGrunn 2014 - Dirk Zittersteyn
Similar to Advanced Continuous Integration - PyGrunn 2014 - Dirk Zittersteyn (20)
Recently uploaded
Recently uploaded (20)
Advanced Continuous Integration - PyGrunn 2014 - Dirk Zittersteyn
- 1. Advanced CI
- 2. I’m Dirk Zittersteyn, Software engineer @ since mid 2013
- 3. Did some stuff @ before that
- 4. The wonderful world of Continuous Integration.
- 8. Is your team larger than 1 person? Use version control.
- 9. Is your team 1 person? Use version control.
- 10. Big (open source) players
- 11. Version control can look overly complicated
- 12. Codebase
- 13. More awesome
- 18. Mainline branch
- 19. Mainline branch The mainline stays deliverable
- 20. Mainline branch Branch “New feature” $ git branch $ hg branch $ svn copy
- 21. Mainline branch Branch “New feature”
- 22. Mainline branch Branch “New feature”
- 23. Mainline branch Branch “New feature” Integrate your changes
- 24. Concurrent work!
- 25. Concurrent work Mainline branch Branch “New feature” Branch “Other feature”
- 28. So, are we done?
- 30. Meet Charlie.
- 37. You better check yo build Before you wreck yo build Ice Cube
- 38. ! Mainline should be green
- 39. Agreement ≠ Reality Missing testcases “It’s so simple, it won’t break”
- 40. Agreement ≠ Reality ‘Bad’ commits $ git commit -m ’removed debug statement’
- 41. Agreement ≠ Reality Quick fixes w/o running tests $ git commit -m ’server no longer explodes’
- 42. But wait, there’s more!
- 43. + = !+ = ! Mainline + Branch = New Mainline
- 44. !+ != ?
- 45. Yes! !+ !=
- 46. Yes(ish)! !+ !=
- 47. Assuming everything is okay. e.g. all environments are the same, nobody is removing tests, nobody is setting this up to happen !+ !=
- 48. + = ?
- 49. Mainline: add(a, b): result = a + b return result test_add(): assert add(1,1) == 2 + = ?
- 50. Developer A: add(a, b): + g.adds += 1 result = a + b return result test_add(): + g.adds = 0 assert add(1,1) == 2 + assert g.adds == 1 Developer B: add(a, b): result = a + b + g.adds += 1 return result test_add(): + g.adds = 0 assert add(1,1) == 2 + assert g.adds == 1
- 51. Merge A’s changes with mainline add(a, b): + g.adds += 1 result = a + b return result test_add(): + g.adds = 0 assert add(1,1) == 2 + assert g.adds == 1
- 52. Merge B’s changes with mainline add(a, b): g.adds += 1 result = a + b + g.adds += 1 return result test_add(): g.adds = 0 assert add(1,1) == 2 assert g.adds == 1
- 53. add(a, b): g.adds += 1 result = a + b g.adds += 1 return result test_add(): g.adds = 0 assert add(1,1) == 2 ! assert g.adds == 1 !+ =
- 56. if ((err = SSLFreeBuffer(&hashCtx)) != 0) goto fail; if ((err = ReadyHash(&SSLHashSHA1, &hashCtx)) != 0) goto fail; if ((err = SSLHashSHA1.update(&hashCtx, &clientRandom)) != 0) goto fail; if ((err = SSLHashSHA1.update(&hashCtx, &serverRandom)) != 0) goto fail; if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0) goto fail; ! goto fail; if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0) goto fail; At least, that’s what the NSA wants you to think
- 57. Just because you're paranoid doesn't mean that they're not after you Joseph Heller
- 58. Developers shouldn’t change the mainline
- 59. Every merge is a change!
- 66. Oh merciful Gatekeeper, please merge my code!
- 69. ✓
- 70. !
- 72. !
- 73. Calm down, Try again !
- 74. Calm down, Try again !
- 75. What do we use? Our judge, jury and execut(ion)er Jenkins-CI.org • Test execution: py.test • Code merging Gatekeeper plugin • Developer notification Fogbugz plugin .com/ /paylogic-jenkins-plugins
- 98. So, are we done?
Editor's Notes
- Thanks for the intro I’m Dirk Zittersteyn I’ll be talking about continuous integration
- But you’re not here to hear about me
- This is a screenshot of a build of mine on paylogic’s CI server The green light is so satisfying
- It’s like your code is saying: Well done! So how do you get there?
- First off, Version control.
- Still use version control.
- Which one? Git is cool, Mercurial is python, subversion is a wise old granddad
- Now we have some very powerful tools. But they seem to do things very complicated! They don’t magically make things awesome!
- You have some codebase, and you start implementing a sweet new feature
- Clearly your project is now better
- What if you break your build? Can’t deploy! Can’t do anything before fixing the bug! This workflow breaks down with multiple people.
- Also, you can’t deploy quickly.
- You’ve gone from something that’s pretty useless
- To something that’s way more advanced, but still useless
- Let’s solve this. This mainline can be your ‘base’ branch, In Git: Master In Mercurial: Default In SVN: Trunk (kinda) Or any other branch, by convention
- Deliverable means: it works.
- You create a branch, which is a copy of the mainline
- You work on your copy, and you commit Nothing happens in the mainline
- You can break your branch, but not the mainline!
- And when you’re convinced your feature is done, you merge!
- You can now do more than one thing at the same time!
- You decide to work on two features, at the same time!
- Integrate Often! Why? Glengarry GlenRoss
- The longer you wait to integrate, the harder it is.
- That would make for a really short talk We aren’t.
- Not just in algorithms, also in development!
- Charlie is a programmer. He thinks he’s quite clever.
- Now obviously, I would never commit broken code, But charlie does!
- Now your mainline is broken!
- And it breaks for everyone!
- Finger pointing results And finger pointing can be very wrong.
- My code might be buggy as well, but hidden by the other bug! This situation is so common…
- Someone made the effort to make this. How do we solve this?
- Don’t merge broken code! Never!
- You make an agreement with your devs: Don’t commit broken code.
- It’s so easy, it won’t break anything
- For example accidentally leaving debug statements in your code
- But even when you disregard this
- Even when you do everything right, we’re making an assumption
- Merging good code in a good mainline results in a good new mainline
- And we want to count the number of adds we do.
- Well, maybe?
- Ofcourse, there are plenty of people that say this is an intentional vulnerability, Used by the nsa to spy on us. Joseph heller said the following in the book catch-22
- This not only works for Surveilance agencies, but also for keeping your mainline safe.
- Ergo, developers shouldn’t do the merging
- We start with a mainline
- You branch
- And somebody makes a change in the mainline (doesn’t matter how)
- You test your build
- You test your build
- And the build is successful
- You request
- The Gatekeeper merges the code into the mainline, but doesn’t commit yet! First the build is tested
- If the build succeeds?
- The changes are commited to the mainline Much rejoicing
- The merge is discarded, And the developer is notified
- Developer f
- And ask for forgiveness
- With a very pretty please.
- For bare metal we use our ‘old’ hardware, using Wake-on-LAN to conserve power When the bare metal is fully used, and there’s still more, we spin up a couple of DigitalOcean droplets Tasks are distributed over multiple nodes with py.test xdist
- For bare metal we use our ‘old’ hardware, using Wake-on-LAN to conserve power When the bare metal is fully used, and there’s still more, we spin up a couple of DigitalOcean droplets Tasks are distributed over multiple nodes with py.test xdist
- And the cycle continues!
- That would make for a really short talk We aren’t.