SlideShare a Scribd company logo

Mozilla: Continuous Deploment on SUMO

•
•
Matt Brandt
Matt Brandt

An overview of how Mozilla is doing Continuous Deployment for support.mozilla.org

TechnologyNews & Politics
1 of 49
Download to read offline
MOZILLA Continuous Deployment on Sumo support.mozilla.com Matt Brandt, QA Engineer mbrandt@mozilla.com Tuesday, November 22, 2011
[Nearly] MOZILLA Continuous Deployment on Sumo support.mozilla.com Matt Brandt, QA Engineer mbrandt@mozilla.com Tuesday, November 22, 2011
Bring questions Tuesday, November 22, 2011
Bring questions Tuesday, November 22, 2011
Bring questions What’s wrong with our current small iterations? Tuesday, November 22, 2011
Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! Tuesday, November 22, 2011
Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! What happens to our Selenium test suite? Tuesday, November 22, 2011
Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! What happens to our Selenium test suite? We’ll be relegated to manual testers? Tuesday, November 22, 2011
Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! What happens to our Selenium test suite? We’ll be relegated to manual testers? How do we measure quality? Tuesday, November 22, 2011
But first a little history... Tuesday, November 22, 2011
But first a little history... Tuesday, November 22, 2011
Short Iterations Tuesday, November 22, 2011
Simplified What the team did before Trunk/ Stage Prod Dev Tuesday, November 22, 2011
Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •Is this the right thing? •Automate Tuesday, November 22, 2011
Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •Is this the right thing? •Automate Tuesday, November 22, 2011
Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •IT pushes •Is this the right thing? •Regression test •Automate •Exploratory Testing •Is this the right thing? Tuesday, November 22, 2011
Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •IT pushes •Is this the right thing? •Regression test •Automate •Exploratory Testing •Is this the right thing? Tuesday, November 22, 2011
Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •IT pushes •IT pushes •Is this the right thing? •Regression test •Verify •Automate •Exploratory Testing •Wahoo! •Is this the right thing? Tuesday, November 22, 2011
The brave new world Continuous deployment http://cliptank.com/funny/dog-costumes.php Tuesday, November 22, 2011
Tools & attitude Tuesday, November 22, 2011
Discuss what is acceptable risk https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_Tools Tuesday, November 22, 2011
Discuss what is acceptable risk Risk: Less critical areas will not be covered by automation https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_Tools Tuesday, November 22, 2011
Discuss what is acceptable risk Risk: Less critical areas will not be covered by automation •If the feature breaks, it is acceptable to wait up to an hour for a fix. https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_Tools Tuesday, November 22, 2011
Discuss what is acceptable risk Risk: Less critical areas will not be covered by automation •If the feature breaks, it is acceptable to wait up to an hour for a fix. • Example: the ability to answer a question. https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_Tools Tuesday, November 22, 2011
Acceptable risk Tuesday, November 22, 2011
Acceptable risk Tuesday, November 22, 2011
What’s our manual strategy? Tuesday, November 22, 2011
What’s our manual strategy? Tuesday, November 22, 2011
Manual strategy Tuesday, November 22, 2011
Manual strategy Feature verification & exploratory testing Tuesday, November 22, 2011
Manual strategy Feature verification & exploratory testing Testing can occur in production Tuesday, November 22, 2011
Manual strategy Feature verification & exploratory testing Testing can occur in production Community involvement on new features Tuesday, November 22, 2011
Automation is fun https://github.com/mozilla/sumo-tests Tuesday, November 22, 2011
Automation is fun https://github.com/mozilla/sumo-tests Tuesday, November 22, 2011
Automation strategy Tuesday, November 22, 2011
Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Tuesday, November 22, 2011
Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Qualify as deployment blockers: run in dev & staging envs Tuesday, November 22, 2011
Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Qualify as deployment blockers: run in dev & staging envs Non-volatile production tests verify env & services work Tuesday, November 22, 2011
Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Qualify as deployment blockers: run in dev & staging envs Non-volatile production tests verify env & services work [All] Tests will be deleted if they don’t cover critical areas Tuesday, November 22, 2011
Waffling Failure is always an option Tuesday, November 22, 2011
Waffling Failure is always an option Tuesday, November 22, 2011
Control Tuesday, November 22, 2011
Gathering data Tuesday, November 22, 2011
Gathering data Tuesday, November 22, 2011
Now Trunk/ Stage Prod Dev •Automation runs •Fast iterations •Early stage testing Tuesday, November 22, 2011
Now Trunk/ Stage Prod Dev •Automation runs •Auto-updated •Fast iterations •No post-commit hooks •Early stage testing •Bug & feature testing Tuesday, November 22, 2011
Now Trunk/ Stage Prod Dev •Not push-button •Automation runs •Auto-updated •Automation •Fast iterations •No post-commit hooks •Monitoring •Early stage testing •Bug & feature testing •Feedback tool •Wahoo! Tuesday, November 22, 2011
Questions Tuesday, November 22, 2011
References •https://support.mozilla.com •https://wiki.mozilla.org/QA/Execution/Web_Testing/ •http://docs.jquery.com/QUnit Continuous_Deployment •https://github.com/mozilla/sumo-tests •https://github.com/jsocol/django-waffle •http://coffeeonthekeyboard.com/the-future-of-sumo- •http://coffeeonthekeyboard.com/introducing-waffle-for- django-541/ development-511/ •http://coffeeonthekeyboard.com/acronyms-you-should-know- •https://github.com/etsy/statsd mttd-and-mttr-597/ •http://codeascraft.etsy.com/2011/02/15/measure-anything- measure-everything/ •http://www.slideshare.net/stephendonner/continuous- deployment-9363895 Tuesday, November 22, 2011

Recommended

Quality Assurance in a DevOps World
Quality Assurance in a DevOps WorldQuality Assurance in a DevOps World
Quality Assurance in a DevOps World
Kit Plummer
 
Bonfire... How'd You Do That?! - AtlasCamp 2011
Bonfire... How'd You Do That?! - AtlasCamp 2011Bonfire... How'd You Do That?! - AtlasCamp 2011
Bonfire... How'd You Do That?! - AtlasCamp 2011
Atlassian
 
Alternative Software Development Methodology
Alternative Software Development MethodologyAlternative Software Development Methodology
Alternative Software Development Methodology
ทวิร พานิชสมบัติ
 
Test Driven Sysadmin
Test Driven SysadminTest Driven Sysadmin
Test Driven Sysadmin
johanku
 
obs-tdd-intro
obs-tdd-introobs-tdd-intro
obs-tdd-intro
Amir Barylko
 
3D in the Browser via WebGL: It's Go Time
3D in the Browser via WebGL: It's Go Time 3D in the Browser via WebGL: It's Go Time
3D in the Browser via WebGL: It's Go Time
Pascal Rettig
 
Agile2012 soccer witha_basketballteam
Agile2012 soccer witha_basketballteamAgile2012 soccer witha_basketballteam
Agile2012 soccer witha_basketballteam
drewz lin
 
Monitoring is easy, why are we so bad at it presentation
Monitoring is easy, why are we so bad at it presentationMonitoring is easy, why are we so bad at it presentation
Monitoring is easy, why are we so bad at it presentation
Theo Schlossnagle
 

Recommended

Quality Assurance in a DevOps World
Quality Assurance in a DevOps WorldQuality Assurance in a DevOps World
Quality Assurance in a DevOps World
Kit Plummer
 
Bonfire... How'd You Do That?! - AtlasCamp 2011
Bonfire... How'd You Do That?! - AtlasCamp 2011Bonfire... How'd You Do That?! - AtlasCamp 2011
Bonfire... How'd You Do That?! - AtlasCamp 2011
Atlassian
 
Alternative Software Development Methodology
Alternative Software Development MethodologyAlternative Software Development Methodology
Alternative Software Development Methodology
ทวิร พานิชสมบัติ
 
Test Driven Sysadmin
Test Driven SysadminTest Driven Sysadmin
Test Driven Sysadmin
johanku
 
obs-tdd-intro
obs-tdd-introobs-tdd-intro
obs-tdd-intro
Amir Barylko
 
3D in the Browser via WebGL: It's Go Time
3D in the Browser via WebGL: It's Go Time 3D in the Browser via WebGL: It's Go Time
3D in the Browser via WebGL: It's Go Time
Pascal Rettig
 
Agile2012 soccer witha_basketballteam
Agile2012 soccer witha_basketballteamAgile2012 soccer witha_basketballteam
Agile2012 soccer witha_basketballteam
drewz lin
 
Monitoring is easy, why are we so bad at it presentation
Monitoring is easy, why are we so bad at it presentationMonitoring is easy, why are we so bad at it presentation
Monitoring is easy, why are we so bad at it presentation
Theo Schlossnagle
 
Donating a mature project to Eclipse
Donating a mature project to EclipseDonating a mature project to Eclipse
Donating a mature project to Eclipse
glynnormington
 
ebay @ Hasso Plattner Institut Potsdam
ebay @ Hasso Plattner Institut Potsdamebay @ Hasso Plattner Institut Potsdam
ebay @ Hasso Plattner Institut Potsdam
Michael Palotas
 
Agile Software Development Process Practice in Thai Culture
Agile Software Development Process Practice in Thai CultureAgile Software Development Process Practice in Thai Culture
Agile Software Development Process Practice in Thai Culture
Wee Witthawaskul
 
Building Mobile (app) Masterpiece with Distributed Agile
Building Mobile (app) Masterpiece with Distributed AgileBuilding Mobile (app) Masterpiece with Distributed Agile
Building Mobile (app) Masterpiece with Distributed Agile
Wee Witthawaskul
 
Get your Project back in Shape!
Get your Project back in Shape!Get your Project back in Shape!
Get your Project back in Shape!
Joachim Tuchel
 
Making software development processes to work for you
Making software development processes to work for youMaking software development processes to work for you
Making software development processes to work for you
Ambientia
 
Agile Software Development in Practice - A Developer Perspective
Agile Software Development in Practice - A Developer PerspectiveAgile Software Development in Practice - A Developer Perspective
Agile Software Development in Practice - A Developer Perspective
Wee Witthawaskul
 
Asteroïden
AsteroïdenAsteroïden
Asteroïden
ThomasBos
 
Webprog lecture1
Webprog lecture1Webprog lecture1
Webprog lecture1
teoper1205
 
Appnostic
AppnosticAppnostic
Appnostic
Veronika Kormanyos
 
Rtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortRtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effort
Veronika Kormanyos
 
Think tank – paris 27 apr
Think tank – paris 27 aprThink tank – paris 27 apr
Think tank – paris 27 apr
Veronika Kormanyos
 
Presentation zahid 2
Presentation zahid 2Presentation zahid 2
Presentation zahid 2
twi8er
 
Rtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortRtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effort
Veronika Kormanyos
 
Rtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortRtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effort
Veronika Kormanyos
 
Think tank – paris 27 apr
Think tank – paris 27 aprThink tank – paris 27 apr
Think tank – paris 27 apr
Veronika Kormanyos
 
Think tank – paris 27 apr
Think tank – paris 27 aprThink tank – paris 27 apr
Think tank – paris 27 apr
Veronika Kormanyos
 
Wifi work of july 19, paris
Wifi work of july 19, parisWifi work of july 19, paris
Wifi work of july 19, paris
Veronika Kormanyos
 
Dreamitalive
DreamitaliveDreamitalive
Dreamitalive
Veronika Kormanyos
 
Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...
Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...
Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...
bradyjay
 
PCMSO - Programa de Controle Médico de Saúde Ocupacional
PCMSO - Programa de Controle Médico de Saúde OcupacionalPCMSO - Programa de Controle Médico de Saúde Ocupacional
PCMSO - Programa de Controle Médico de Saúde Ocupacional
Renato Sfolia
 
ExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias Sociais
ExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias SociaisExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias Sociais
ExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias Sociais
DOGSCAMP Summit
 

More Related Content

What's hot

ebay @ Hasso Plattner Institut Potsdam
ebay @ Hasso Plattner Institut Potsdamebay @ Hasso Plattner Institut Potsdam
ebay @ Hasso Plattner Institut Potsdam
Michael Palotas
 

What's hot (7)

Donating a mature project to Eclipse
Donating a mature project to EclipseDonating a mature project to Eclipse
Donating a mature project to Eclipse
 
ebay @ Hasso Plattner Institut Potsdam
ebay @ Hasso Plattner Institut Potsdamebay @ Hasso Plattner Institut Potsdam
ebay @ Hasso Plattner Institut Potsdam
 
Agile Software Development Process Practice in Thai Culture
Agile Software Development Process Practice in Thai CultureAgile Software Development Process Practice in Thai Culture
Agile Software Development Process Practice in Thai Culture
 
Building Mobile (app) Masterpiece with Distributed Agile
Building Mobile (app) Masterpiece with Distributed AgileBuilding Mobile (app) Masterpiece with Distributed Agile
Building Mobile (app) Masterpiece with Distributed Agile
 
Get your Project back in Shape!
Get your Project back in Shape!Get your Project back in Shape!
Get your Project back in Shape!
 
Making software development processes to work for you
Making software development processes to work for youMaking software development processes to work for you
Making software development processes to work for you
 
Agile Software Development in Practice - A Developer Perspective
Agile Software Development in Practice - A Developer PerspectiveAgile Software Development in Practice - A Developer Perspective
Agile Software Development in Practice - A Developer Perspective
 

Viewers also liked

Webprog lecture1
Webprog lecture1Webprog lecture1
Webprog lecture1
teoper1205
 
Presentation zahid 2
Presentation zahid 2Presentation zahid 2
Presentation zahid 2
twi8er
 
Rtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortRtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effort
Veronika Kormanyos
 
PCMSO - Programa de Controle Médico de Saúde Ocupacional
PCMSO - Programa de Controle Médico de Saúde OcupacionalPCMSO - Programa de Controle Médico de Saúde Ocupacional
PCMSO - Programa de Controle Médico de Saúde Ocupacional
Renato Sfolia
 

Viewers also liked (14)

Asteroïden
AsteroïdenAsteroïden
Asteroïden
 
Webprog lecture1
Webprog lecture1Webprog lecture1
Webprog lecture1
 
Appnostic
AppnosticAppnostic
Appnostic
 
Rtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortRtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effort
 
Think tank – paris 27 apr
Think tank – paris 27 aprThink tank – paris 27 apr
Think tank – paris 27 apr
 
Presentation zahid 2
Presentation zahid 2Presentation zahid 2
Presentation zahid 2
 
Rtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortRtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effort
 
Rtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortRtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effort
 
Think tank – paris 27 apr
Think tank – paris 27 aprThink tank – paris 27 apr
Think tank – paris 27 apr
 
Think tank – paris 27 apr
Think tank – paris 27 aprThink tank – paris 27 apr
Think tank – paris 27 apr
 
Wifi work of july 19, paris
Wifi work of july 19, parisWifi work of july 19, paris
Wifi work of july 19, paris
 
Dreamitalive
DreamitaliveDreamitalive
Dreamitalive
 
Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...
Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...
Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...
 
PCMSO - Programa de Controle Médico de Saúde Ocupacional
PCMSO - Programa de Controle Médico de Saúde OcupacionalPCMSO - Programa de Controle Médico de Saúde Ocupacional
PCMSO - Programa de Controle Médico de Saúde Ocupacional
 

Similar to Mozilla: Continuous Deploment on SUMO

PyconUA - How to build ERP application having fun?
PyconUA - How to build ERP application having fun?PyconUA - How to build ERP application having fun?
PyconUA - How to build ERP application having fun?
Andrew Mleczko
 
Continuous Deployment at Disqus (Pylons Minicon)
Continuous Deployment at Disqus (Pylons Minicon)Continuous Deployment at Disqus (Pylons Minicon)
Continuous Deployment at Disqus (Pylons Minicon)
zeeg
 
JavaSE - The road forward
JavaSE - The road forwardJavaSE - The road forward
JavaSE - The road forward
eug3n_cojocaru
 
SecurityBSides las vegas - Agnitio
SecurityBSides las vegas - AgnitioSecurityBSides las vegas - Agnitio
SecurityBSides las vegas - Agnitio
Security Ninja
 
Clouds against the Floods (RubyConfBR2011)
Clouds against the Floods (RubyConfBR2011) Clouds against the Floods (RubyConfBR2011)
Clouds against the Floods (RubyConfBR2011)
Leonardo Borges
 

Similar to Mozilla: Continuous Deploment on SUMO (20)

ExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias Sociais
ExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias SociaisExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias Sociais
ExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias Sociais
 
Agile Deployments with Puppet
Agile Deployments with PuppetAgile Deployments with Puppet
Agile Deployments with Puppet
 
PyconUA - How to build ERP application having fun?
PyconUA - How to build ERP application having fun?PyconUA - How to build ERP application having fun?
PyconUA - How to build ERP application having fun?
 
Pitfalls of Continuous Deployment
Pitfalls of Continuous DeploymentPitfalls of Continuous Deployment
Pitfalls of Continuous Deployment
 
Agile xptdd@gosoft
Agile xptdd@gosoftAgile xptdd@gosoft
Agile xptdd@gosoft
 
Agile xp tdd@gosoft
Agile xp tdd@gosoftAgile xp tdd@gosoft
Agile xp tdd@gosoft
 
Ruby can haz Everyone
Ruby can haz EveryoneRuby can haz Everyone
Ruby can haz Everyone
 
Hudson
HudsonHudson
Hudson
 
Building an experimentation framework
Building an experimentation frameworkBuilding an experimentation framework
Building an experimentation framework
 
Continuous Deployment at Disqus (Pylons Minicon)
Continuous Deployment at Disqus (Pylons Minicon)Continuous Deployment at Disqus (Pylons Minicon)
Continuous Deployment at Disqus (Pylons Minicon)
 
Gradle - The hackable build-tool
Gradle - The hackable build-toolGradle - The hackable build-tool
Gradle - The hackable build-tool
 
Recognizing Software Debt - Beyond Agile Puget Sound
Recognizing Software Debt - Beyond Agile Puget SoundRecognizing Software Debt - Beyond Agile Puget Sound
Recognizing Software Debt - Beyond Agile Puget Sound
 
every-day-automation
every-day-automationevery-day-automation
every-day-automation
 
JavaSE - The road forward
JavaSE - The road forwardJavaSE - The road forward
JavaSE - The road forward
 
SecurityBSides las vegas - Agnitio
SecurityBSides las vegas - AgnitioSecurityBSides las vegas - Agnitio
SecurityBSides las vegas - Agnitio
 
Android 1.5 to 3.0: a compatibility journey
Android 1.5 to 3.0: a compatibility journeyAndroid 1.5 to 3.0: a compatibility journey
Android 1.5 to 3.0: a compatibility journey
 
Is Advanced Verification for FPGA based Logic needed
Is Advanced Verification for FPGA based Logic neededIs Advanced Verification for FPGA based Logic needed
Is Advanced Verification for FPGA based Logic needed
 
Clouds against the Floods (RubyConfBR2011)
Clouds against the Floods (RubyConfBR2011) Clouds against the Floods (RubyConfBR2011)
Clouds against the Floods (RubyConfBR2011)
 
Plugin Testing
Plugin TestingPlugin Testing
Plugin Testing
 
Integrating Quality into Project Portfolio Management
Integrating Quality into Project Portfolio ManagementIntegrating Quality into Project Portfolio Management
Integrating Quality into Project Portfolio Management
 

Recently uploaded

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Miguel Araújo
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
sammart93
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Drew Madelung
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
Igalia
 

Recently uploaded (20)

FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot ModelNavi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 

Mozilla: Continuous Deploment on SUMO

  • 1. MOZILLA Continuous Deployment on Sumo support.mozilla.com Matt Brandt, QA Engineer mbrandt@mozilla.com Tuesday, November 22, 2011
  • 2. [Nearly] MOZILLA Continuous Deployment on Sumo support.mozilla.com Matt Brandt, QA Engineer mbrandt@mozilla.com Tuesday, November 22, 2011
  • 5. Bring questions What’s wrong with our current small iterations? Tuesday, November 22, 2011
  • 6. Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! Tuesday, November 22, 2011
  • 7. Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! What happens to our Selenium test suite? Tuesday, November 22, 2011
  • 8. Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! What happens to our Selenium test suite? We’ll be relegated to manual testers? Tuesday, November 22, 2011
  • 9. Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! What happens to our Selenium test suite? We’ll be relegated to manual testers? How do we measure quality? Tuesday, November 22, 2011
  • 10. But first a little history... Tuesday, November 22, 2011
  • 11. But first a little history... Tuesday, November 22, 2011
  • 13. Simplified What the team did before Trunk/ Stage Prod Dev Tuesday, November 22, 2011
  • 14. Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •Is this the right thing? •Automate Tuesday, November 22, 2011
  • 15. Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •Is this the right thing? •Automate Tuesday, November 22, 2011
  • 16. Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •IT pushes •Is this the right thing? •Regression test •Automate •Exploratory Testing •Is this the right thing? Tuesday, November 22, 2011
  • 17. Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •IT pushes •Is this the right thing? •Regression test •Automate •Exploratory Testing •Is this the right thing? Tuesday, November 22, 2011
  • 18. Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •IT pushes •IT pushes •Is this the right thing? •Regression test •Verify •Automate •Exploratory Testing •Wahoo! •Is this the right thing? Tuesday, November 22, 2011
  • 19. The brave new world Continuous deployment http://cliptank.com/funny/dog-costumes.php Tuesday, November 22, 2011
  • 20. Tools & attitude Tuesday, November 22, 2011
  • 21. Discuss what is acceptable risk https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_Tools Tuesday, November 22, 2011
  • 22. Discuss what is acceptable risk Risk: Less critical areas will not be covered by automation https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_Tools Tuesday, November 22, 2011
  • 23. Discuss what is acceptable risk Risk: Less critical areas will not be covered by automation •If the feature breaks, it is acceptable to wait up to an hour for a fix. https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_Tools Tuesday, November 22, 2011
  • 24. Discuss what is acceptable risk Risk: Less critical areas will not be covered by automation •If the feature breaks, it is acceptable to wait up to an hour for a fix. • Example: the ability to answer a question. https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_Tools Tuesday, November 22, 2011
  • 27. What’s our manual strategy? Tuesday, November 22, 2011
  • 28. What’s our manual strategy? Tuesday, November 22, 2011
  • 30. Manual strategy Feature verification & exploratory testing Tuesday, November 22, 2011
  • 31. Manual strategy Feature verification & exploratory testing Testing can occur in production Tuesday, November 22, 2011
  • 32. Manual strategy Feature verification & exploratory testing Testing can occur in production Community involvement on new features Tuesday, November 22, 2011
  • 33. Automation is fun https://github.com/mozilla/sumo-tests Tuesday, November 22, 2011
  • 34. Automation is fun https://github.com/mozilla/sumo-tests Tuesday, November 22, 2011
  • 36. Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Tuesday, November 22, 2011
  • 37. Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Qualify as deployment blockers: run in dev & staging envs Tuesday, November 22, 2011
  • 38. Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Qualify as deployment blockers: run in dev & staging envs Non-volatile production tests verify env & services work Tuesday, November 22, 2011
  • 39. Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Qualify as deployment blockers: run in dev & staging envs Non-volatile production tests verify env & services work [All] Tests will be deleted if they don’t cover critical areas Tuesday, November 22, 2011
  • 40. Waffling Failure is always an option Tuesday, November 22, 2011
  • 41. Waffling Failure is always an option Tuesday, November 22, 2011
  • 45. Now Trunk/ Stage Prod Dev •Automation runs •Fast iterations •Early stage testing Tuesday, November 22, 2011
  • 46. Now Trunk/ Stage Prod Dev •Automation runs •Auto-updated •Fast iterations •No post-commit hooks •Early stage testing •Bug & feature testing Tuesday, November 22, 2011
  • 47. Now Trunk/ Stage Prod Dev •Not push-button •Automation runs •Auto-updated •Automation •Fast iterations •No post-commit hooks •Monitoring •Early stage testing •Bug & feature testing •Feedback tool •Wahoo! Tuesday, November 22, 2011
  • 49. References •https://support.mozilla.com •https://wiki.mozilla.org/QA/Execution/Web_Testing/ •http://docs.jquery.com/QUnit Continuous_Deployment •https://github.com/mozilla/sumo-tests •https://github.com/jsocol/django-waffle •http://coffeeonthekeyboard.com/the-future-of-sumo- •http://coffeeonthekeyboard.com/introducing-waffle-for- django-541/ development-511/ •http://coffeeonthekeyboard.com/acronyms-you-should-know- •https://github.com/etsy/statsd mttd-and-mttr-597/ •http://codeascraft.etsy.com/2011/02/15/measure-anything- measure-everything/ •http://www.slideshare.net/stephendonner/continuous- deployment-9363895 Tuesday, November 22, 2011