Extending Continuous Integration

911 views

Published on

How to automate a release tool chain

Published in: Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
911
On SlideShare
0
From Embeds
0
Number of Embeds
32
Actions
Shares
0
Downloads
50
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Extending Continuous Integration

  1. 1. Extending Continuous Integration
  2. 2. About the speaker <ul><li>Johannes Brodwall </li></ul><ul><li>Lame duck Lead software architect at BBS </li></ul><ul><li>Organizer Oslo XP meetup </li></ul><ul><li>Organizer Smidig 2008 </li></ul><ul><li>Blog: http://brodwall.com/johannes/blog </li></ul>.meetup.com/13
  3. 3. What’s the point of Continuous Integration?
  4. 4. <ul><li>Pay now to insure against defects </li></ul><ul><li>Pay less to deploy to production </li></ul><ul><li>Release whenever you want </li></ul>
  5. 5. What is Continuous Integration?
  6. 9. <ul><li>Unit test </li></ul><ul><li>Functional tests </li></ul><ul><li>System run in test harness </li></ul>
  7. 10. Result:
  8. 11. Every check in is is now tested
  9. 13. Will this find all our defects?
  10. 14. No!
  11. 15. <ul><li>Limited to our test harness </li></ul><ul><li>Limited to our imagination </li></ul><ul><li>And then there’s performance </li></ul>
  12. 16. So what should we do?
  13. 17. (Hint: CI is about paying now to avoid paying later)
  14. 18. Pay more now!
  15. 19. Our approach: Automated system test
  16. 20. <ul><li>Realistic setup </li></ul><ul><li>Realistic load </li></ul><ul><li>Realistic variation </li></ul>
  17. 22. How to automate system tests <ul><li>Automated build (and test) </li></ul><ul><li>Scrap old data </li></ul><ul><li>Install latest snapshot </li></ul><ul><li>Download production data </li></ul><ul><li>Replay production data </li></ul><ul><li>Verify result </li></ul><ul><li>Send results and logs via email </li></ul>
  18. 23. ” Did you meet any problems?”
  19. 24. No
  20. 25. No, Only challenges!
  21. 26. The hard bits <ul><li>Installation </li></ul><ul><li>Integration </li></ul><ul><li>Simulation </li></ul><ul><li>Verification </li></ul>
  22. 27. Installation
  23. 28. How to automate installation <ul><li>Easy, scripted, always working install </li></ul><ul><li>Simplify </li></ul><ul><ul><li>(Replace WebSphere with Jetty) </li></ul></ul><ul><ul><li>(Combine components) </li></ul></ul><ul><li>Reduce integration </li></ul><ul><li>All nodes look the same </li></ul>
  24. 29. (Side effects) <ul><li>Easy installation </li></ul><ul><li>Simpler design </li></ul><ul><li>Simpler architecture </li></ul>
  25. 30. Integration
  26. 31. Integration Dealing with dependencies
  27. 32. Dealing with integration <ul><li>Don’t integrate - Do it yourself! </li></ul><ul><li>Simulate other system </li></ul><ul><ul><li>Fake responses </li></ul></ul><ul><ul><li>Keep canned data (data centric) </li></ul></ul><ul><li>Integrate with test version </li></ul>
  28. 33. Simulation
  29. 34. Simulation Put realistic load on the system
  30. 35. Simulation Depends on what you system does
  31. 36. How to simulate production <ul><li>In our case: Files </li></ul><ul><li>Crawler (Dyrkorn & Watne) </li></ul><ul><li>Load generator (D&W) </li></ul><ul><li>Record HTTP requests </li></ul>
  32. 37. Verification
  33. 38. Verification Finding out if it worked
  34. 39. How to verify results <ul><li>Compare with production </li></ul><ul><li>Verify integrity </li></ul><ul><li>Check logs </li></ul>
  35. 40. Comparing <ul><li>Store test result in database </li></ul><ul><li>Store production result in database </li></ul><ul><li>Full outer join on key fields </li></ul><ul><li>Find missing or mismatched status </li></ul><ul><li>Filter out know deviations </li></ul>
  36. 42. Date
  37. 43. Number of files
  38. 44. Okay
  39. 45. Missing
  40. 46. Extra
  41. 47. Extra Mismatched
  42. 48. Extra Extra Extra Known deviations
  43. 49. Consistency checks <ul><li>SQL expressions that pick out things that are weird </li></ul>
  44. 50. Logging <ul><li>It matters! </li></ul><ul><li>Error logs should be empty if everything is okay </li></ul>
  45. 51. Result:
  46. 52. Every build is now system tested
  47. 54. Will this find all our defects?
  48. 55. No!
  49. 56. <ul><li>Limited integration </li></ul><ul><li>Limited stability </li></ul>
  50. 57. No! Automated staging
  51. 58. No! Automated staging
  52. 59. Automated staging <ul><li>” Next” version </li></ul><ul><li>Lock-step with production </li></ul><ul><li>Promote after a week </li></ul><ul><li>Monitor 9:00-16:00 </li></ul>
  53. 62. Only when you can think as an operator, can you master your system
  54. 63. Result:
  55. 64. Every release is hardened
  56. 66. Will this find all our defects?
  57. 67. No!
  58. 68. <ul><li>Wrong requirements </li></ul><ul><li>Poor solutions </li></ul><ul><li>New user behavior exposes bugs </li></ul><ul><li>Bugs we didn’t care enough about </li></ul>
  59. 69. The goal: Release after every iteration
  60. 71. The sad reality: Pilot release after every iteration
  61. 72. Pilot production
  62. 73. Why releases every iteration?
  63. 74. <ul><li>Find more bugs </li></ul><ul><li>Try the easy solution first </li></ul><ul><li>Find new requirement faster </li></ul>Exploit opportunity
  64. 75. Result:
  65. 76. Find all the bugs cheaply!
  66. 77. Make sure it always works
  67. 78. Pay more now to pay less later
  68. 79. The goal: Release after every iteration
  69. 80. (And throw away the bugtracker)
  70. 81. Thank you!

×