Pivotal Labs Open View Presentation Quality Assurance And Developer Testing

In practice, we want... • Ability to deploy new releases frequently o at a bare minimum, once per iteration/sprint o ideally, as each new feature is finished • Confidence that everything we build has a very low defect rate • Confidence that adding features/performing refactorings - won't introduce defects • Minimal firefighting/bug fixing - focus on new features • Keep code flexible enough to keep feature cost constant • Sleep well at night Copyright © 2008 Pivotal Labs, Inc. All rights reserved.

The Old Way • Write code • Deploy to QA • QA finds and reports bugs • Development fixes bugs • Deploy to QA • QA finds bugs • Development fixes bugs, repeat • QA doesn't find anymore bugs • Deploy to production Copyright © 2008 Pivotal Labs, Inc. All rights reserved.

Who’s responsible for Quality? •QA’ers are responsible •Quality tested after development is complete •Developer-QA team relationship is loose & adversarial •Automated tests are an afterthought Copyright © 2008 Pivotal Labs, Inc. All rights reserved.

The Result • Ever-increasing percentage of development time spent on bug fixing and firefighting • Ever-decreasing percentage available for feature development • Increasing fear that new features will inject bugs • Architecture changes become prohibitively expensive • Performance concerns conflict with stability concerns • Eventually, feature production grinds to a halt Copyright © 2008 Pivotal Labs, Inc. All rights reserved.

Rethinking our attitudes torward testing • Everyone is responsible for quality o Developers responsible for the quality of their code o 'Customer' responsible for accepting work o QA operates at a higher level of abstraction • Reduce the cost of defects o Move defect detection as early in lifecycle as possible • Make quality transparent o Make bug detection easy Copyright © 2008 Pivotal Labs, Inc. All rights reserved.

Quality as the focal point • Tests are the core of the development process • Test all the time, not just at the end • Automate everything • Invest in making it fast and easy to run tests • Write automated tests before we write the code • Make build status visible to the entire team • Run regression suites at every check-in Copyright © 2008 Pivotal Labs, Inc. All rights reserved.

Kind of Tests • Unit Tests • Functional Tests • Acceptance Tests • Smoke Tests • Integration Tests • End-to-End tests • Scenario Tests • System Tests • UI Tests • Manual tests • Exploratory Tests • Regression Tests Copyright © 2008 Pivotal Labs, Inc. All rights reserved.

Test Isolation

Where’s the Bug? Test Fixture Client View Controller Model Database

Where’s the Bug? Client View Controller Test Fixture Model Database

Where’s the Bug? Test Fixture Client View Controller Test Fixture Model Database

Isolation •Tests at different granularities help you more quickly identify the source of the problem •Good test coverage tells you what’s wrong as quickly as possible

The Food Pyramid Confidence in System as a Whole Increases Test Isolation, Focus, Speed Increases Scenario Tests Functional Tests Integration Tests Unit Tests Copyright © 2008 Pivotal Labs, Inc. All rights reserved.

Testing as part of Development •Test-Driven Development •Red-Green-Refactor •No code before a red test demands it •Outside-in vs. Inside-out •Fast test run frequently •Continuous Integration Copyright © 2008 Pivotal Labs, Inc. All rights reserved.

A Typical Day at Pivotal • Standup - team communication • Pair-up • Pair pops next story from stack in Tracker • Update from RCS • Test-drive story [repeat] Copyright © 2008 Pivotal Labs, Inc. All rights reserved.

Test-Driving a Story • Write functional test (test is red) • Write unit tests [repeat]: • Write unit test (test is red) • Fix unit test by writing code (test is green) • Refactor (tests are green) • Rerun functional test (test is green) • Run all tests (tests are green) • Update from RCS • Run all tests (tests are green) • Check in • Mark story finished Copyright © 2008 Pivotal Labs, Inc. All rights reserved.

Stop the bleeding •Write a test every time a new defect is detected, to isolate the test •Test all new code •Refactor touch points with old code for testability Copyright © 2008 Pivotal Labs, Inc. All rights reserved.

Retrofit coverage •Make retrofitting tests part of everyday work •Don’t “stop” and cover all your old code at once •Measure coverage as you go •Have goals and measure progress •Understand what coverage you have Copyright © 2008 Pivotal Labs, Inc. All rights reserved.

Maintaining Testability •...requires vigilance •It’s easy to slip into old habits... •when there’s time pressure •when you’re tired •when you’re soloing •When it’s hard to write tests, do it more Copyright © 2008 Pivotal Labs, Inc. All rights reserved.

Maintaining Testability •Keep code amenable to testing •Keep coverage high, and focused •Keep the test corpus small •Less is more, assuming your coverage is complete •Don’t punt on hard testing problems Copyright © 2008 Pivotal Labs, Inc. All rights reserved.

The role of QA on an Agile team • Different pair of eyes • Can focus on exploratory testing and test planning • Catch gaps in requirements and coverage • Should write additional coverage • Less monkey work, more brain work • Any bug discovered by QA must have an automated regression test case written Copyright © 2008 Pivotal Labs, Inc. All rights reserved.

Pivotal Labs Open View Presentation Quality Assurance And Developer Testing

0 comments

Notes on slide 1

1 Favorite