Mozilla: Continuous Deploment on SUMO

1,988 views
1,871 views

Published on

An overview of how Mozilla is doing Continuous Deployment for support.mozilla.org

Published in: Technology, News & Politics
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,988
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
11
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Mozilla: Continuous Deploment on SUMO

  1. 1. MOZILLA Continuous Deployment on Sumo support.mozilla.com Matt Brandt, QA Engineer mbrandt@mozilla.comTuesday, November 22, 2011
  2. 2. [Nearly] MOZILLA Continuous Deployment on Sumo support.mozilla.com Matt Brandt, QA Engineer mbrandt@mozilla.comTuesday, November 22, 2011
  3. 3. Bring questionsTuesday, November 22, 2011
  4. 4. Bring questionsTuesday, November 22, 2011
  5. 5. Bring questions What’s wrong with our current small iterations?Tuesday, November 22, 2011
  6. 6. Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!!Tuesday, November 22, 2011
  7. 7. Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! What happens to our Selenium test suite?Tuesday, November 22, 2011
  8. 8. Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! What happens to our Selenium test suite? We’ll be relegated to manual testers?Tuesday, November 22, 2011
  9. 9. Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! What happens to our Selenium test suite? We’ll be relegated to manual testers? How do we measure quality?Tuesday, November 22, 2011
  10. 10. But first a little history...Tuesday, November 22, 2011
  11. 11. But first a little history...Tuesday, November 22, 2011
  12. 12. Short IterationsTuesday, November 22, 2011
  13. 13. Simplified What the team did before Trunk/ Stage Prod DevTuesday, November 22, 2011
  14. 14. Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •Is this the right thing? •AutomateTuesday, November 22, 2011
  15. 15. Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •Is this the right thing? •AutomateTuesday, November 22, 2011
  16. 16. Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •IT pushes •Is this the right thing? •Regression test •Automate •Exploratory Testing •Is this the right thing?Tuesday, November 22, 2011
  17. 17. Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •IT pushes •Is this the right thing? •Regression test •Automate •Exploratory Testing •Is this the right thing?Tuesday, November 22, 2011
  18. 18. Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •IT pushes •IT pushes •Is this the right thing? •Regression test •Verify •Automate •Exploratory Testing •Wahoo! •Is this the right thing?Tuesday, November 22, 2011
  19. 19. The brave new world Continuous deployment http://cliptank.com/funny/dog-costumes.phpTuesday, November 22, 2011
  20. 20. Tools & attitudeTuesday, November 22, 2011
  21. 21. Discuss what is acceptable risk https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_ToolsTuesday, November 22, 2011
  22. 22. Discuss what is acceptable risk Risk: Less critical areas will not be covered by automation https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_ToolsTuesday, November 22, 2011
  23. 23. Discuss what is acceptable risk Risk: Less critical areas will not be covered by automation •If the feature breaks, it is acceptable to wait up to an hour for a fix. https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_ToolsTuesday, November 22, 2011
  24. 24. Discuss what is acceptable risk Risk: Less critical areas will not be covered by automation •If the feature breaks, it is acceptable to wait up to an hour for a fix. • Example: the ability to answer a question. https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_ToolsTuesday, November 22, 2011
  25. 25. Acceptable riskTuesday, November 22, 2011
  26. 26. Acceptable riskTuesday, November 22, 2011
  27. 27. What’s our manual strategy?Tuesday, November 22, 2011
  28. 28. What’s our manual strategy?Tuesday, November 22, 2011
  29. 29. Manual strategyTuesday, November 22, 2011
  30. 30. Manual strategy Feature verification & exploratory testingTuesday, November 22, 2011
  31. 31. Manual strategy Feature verification & exploratory testing Testing can occur in productionTuesday, November 22, 2011
  32. 32. Manual strategy Feature verification & exploratory testing Testing can occur in production Community involvement on new featuresTuesday, November 22, 2011
  33. 33. Automation is fun https://github.com/mozilla/sumo-testsTuesday, November 22, 2011
  34. 34. Automation is fun https://github.com/mozilla/sumo-testsTuesday, November 22, 2011
  35. 35. Automation strategyTuesday, November 22, 2011
  36. 36. Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnitTuesday, November 22, 2011
  37. 37. Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Qualify as deployment blockers: run in dev & staging envsTuesday, November 22, 2011
  38. 38. Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Qualify as deployment blockers: run in dev & staging envs Non-volatile production tests verify env & services workTuesday, November 22, 2011
  39. 39. Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Qualify as deployment blockers: run in dev & staging envs Non-volatile production tests verify env & services work [All] Tests will be deleted if they don’t cover critical areasTuesday, November 22, 2011
  40. 40. Waffling Failure is always an optionTuesday, November 22, 2011
  41. 41. Waffling Failure is always an optionTuesday, November 22, 2011
  42. 42. ControlTuesday, November 22, 2011
  43. 43. Gathering dataTuesday, November 22, 2011
  44. 44. Gathering dataTuesday, November 22, 2011
  45. 45. Now Trunk/ Stage Prod Dev •Automation runs •Fast iterations •Early stage testingTuesday, November 22, 2011
  46. 46. Now Trunk/ Stage Prod Dev •Automation runs •Auto-updated •Fast iterations •No post-commit hooks •Early stage testing •Bug & feature testingTuesday, November 22, 2011
  47. 47. Now Trunk/ Stage Prod Dev •Not push-button •Automation runs •Auto-updated •Automation •Fast iterations •No post-commit hooks •Monitoring •Early stage testing •Bug & feature testing •Feedback tool •Wahoo!Tuesday, November 22, 2011
  48. 48. QuestionsTuesday, November 22, 2011
  49. 49. References •https://support.mozilla.com •https://wiki.mozilla.org/QA/Execution/Web_Testing/ •http://docs.jquery.com/QUnit Continuous_Deployment •https://github.com/mozilla/sumo-tests •https://github.com/jsocol/django-waffle •http://coffeeonthekeyboard.com/the-future-of-sumo- •http://coffeeonthekeyboard.com/introducing-waffle-for- django-541/ development-511/ •http://coffeeonthekeyboard.com/acronyms-you-should-know- •https://github.com/etsy/statsd mttd-and-mttr-597/ •http://codeascraft.etsy.com/2011/02/15/measure-anything- measure-everything/ •http://www.slideshare.net/stephendonner/continuous- deployment-9363895Tuesday, November 22, 2011

×