Some Facts & Figures*
High performing organizations:
Deploy code 30 times more frequently.
– 8000 times faster than their peers
– Deploy multiple times a day, versus an average of once a month.
Have 50 percent fewer failures.
Restore service 12 times faster than their peers.
– Fewer failures and faster recovery mean less risk to the business when
changes are deployed.
Automated Testing Does More Than Find Bugs
Its Helps Teams Build a Better Product
Bug are Common, Difficult
to Eliminate & Very Costly
Using software to test software
Automation: Reduces the Cognitive Feedback Loop
Waiting for tests to run
is no fun!
What Makes for a Good Automated Test?
• Eliminate repetitive manual tests
• Yield reproducible results
• Cover common and edge/error cases
• Ideally either pure unit or integration test
• Strive for clarity even when code is gnarly
• Test for business requirements
Why Not Automate Everything?
• Writing tests does have a cost
– May not be appropriate for experimental code
– May be fragile – UI/mobile automated testing is evolving
– Difficult or impossible to express design and human factors
• Thorough testing requires commitment and discipline
– Writing tests is a short-term pain for a specific individual
– Test suite as a whole is valuable to the team over time
Continuous Integration: A Discipline for Testing Every
Typical CI Setup:
Components that Make for Effective CI:
Continuous Deployment: Automatically Release Validated
• Goal: make release a non-event
– Popularized by Web 2.0, but much more widely applicable
– Final decision to deploy may still be manual
• Valuable to business and engineering sides of the house:
– Selectively enabled features – feature flags
– Consistent, widespread measurement
• Key Ingredients:
– Small changes, continuously integrated
– Automated infrastructure (DevOps)
– Staging/test environment accurately models production