• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Pivotal Labs Open View Presentation Quality Assurance And Developer Testing
 

Pivotal Labs Open View Presentation Quality Assurance And Developer Testing

on

  • 2,910 views

 

Statistics

Views

Total Views
2,910
Views on SlideShare
2,882
Embed Views
28

Actions

Likes
6
Downloads
0
Comments
0

5 Embeds 28

http://pivotallabs.com 11
http://blog.ennuyer.net 10
http://www.slideshare.net 5
http://static.slideshare.net 1
http://webcache.googleusercontent.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

Pivotal Labs Open View Presentation Quality Assurance And Developer Testing Pivotal Labs Open View Presentation Quality Assurance And Developer Testing Presentation Transcript

  • Quality Assurance and Developer Testing Edward Hieatt, VP Engineering Ian McFarland, VP Technology Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • What’s the goal? • To develop software quickly, cheaply, with the lowest defect rates possible Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • In practice, we want... • Ability to deploy new releases frequently o at a bare minimum, once per iteration/sprint o ideally, as each new feature is finished • Confidence that everything we build has a very low defect rate • Confidence that adding features/performing refactorings - won't introduce defects • Minimal firefighting/bug fixing - focus on new features • Keep code flexible enough to keep feature cost constant • Sleep well at night Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • The Old Way • Write code • Deploy to QA • QA finds and reports bugs • Development fixes bugs • Deploy to QA • QA finds bugs • Development fixes bugs, repeat • QA doesn't find anymore bugs • Deploy to production Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Who’s responsible for Quality? •QA’ers are responsible •Quality tested after development is complete •Developer-QA team relationship is loose & adversarial •Automated tests are an afterthought Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • The Result • Ever-increasing percentage of development time spent on bug fixing and firefighting • Ever-decreasing percentage available for feature development • Increasing fear that new features will inject bugs • Architecture changes become prohibitively expensive • Performance concerns conflict with stability concerns • Eventually, feature production grinds to a halt Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Rethinking our attitudes torward testing • Everyone is responsible for quality o Developers responsible for the quality of their code o 'Customer' responsible for accepting work o QA operates at a higher level of abstraction • Reduce the cost of defects o Move defect detection as early in lifecycle as possible • Make quality transparent o Make bug detection easy Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Quality as the focal point • Tests are the core of the development process • Test all the time, not just at the end • Automate everything • Invest in making it fast and easy to run tests  • Write automated tests before we write the code • Make build status visible to the entire team • Run regression suites at every check-in Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Kind of Tests Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Kind of Tests • Unit Tests • Functional Tests • Acceptance Tests • Smoke Tests • Integration Tests • End-to-End tests • Scenario Tests • System Tests • UI Tests • Manual tests • Exploratory Tests • Regression Tests Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Kind of Tests • Unit Tests • Functional Tests • Acceptance Tests • Smoke Tests • Integration Tests • End-to-End tests • Scenario Tests • System Tests • UI Tests • Manual tests • Exploratory Tests • Regression Tests Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Testing Frameworks •Domain Specific Languages (DSLs) •FIT and FITnesse •DSLs and FIT let PMs, analysts and QA write more and better tests Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Test Isolation
  • Where’s the Bug? Test Fixture Client View Controller Model Database
  • Where’s the Bug? Client View Controller Test Fixture Model Database
  • Where’s the Bug? Test Fixture Client View Controller Test Fixture Model Database
  • Isolation •Tests at different granularities help you more quickly identify the source of the problem •Good test coverage tells you what’s wrong as quickly as possible
  • The Food Pyramid Confidence in System as a Whole Increases Test Isolation, Focus, Speed Increases Scenario Tests Functional Tests Integration Tests Unit Tests Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Testing as part of Development •Test-Driven Development •Red-Green-Refactor •No code before a red test demands it •Outside-in vs. Inside-out •Fast test run frequently •Continuous Integration Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • TDD Example Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • A Typical Day at Pivotal • Standup - team communication • Pair-up • Pair pops next story from stack in Tracker • Update from RCS • Test-drive story [repeat] Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Test-Driving a Story • Write functional test (test is red) • Write unit tests [repeat]: • Write unit test (test is red) • Fix unit test by writing code (test is green) • Refactor (tests are green) • Rerun functional test (test is green) • Run all tests (tests are green) • Update from RCS • Run all tests (tests are green) • Check in • Mark story finished Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • How do we get there? Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Action Plan •Stop the bleeding •Transition to testability •Retrofit coverage where needed Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Stop the bleeding •Write a test every time a new defect is detected, to isolate the test •Test all new code •Refactor touch points with old code for testability Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Transition to Testability •Characterization/black box tests first •Refactor for testability •Gradually work down to unit tests in legacy code Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Retrofit coverage •Make retrofitting tests part of everyday work •Don’t “stop” and cover all your old code at once •Measure coverage as you go •Have goals and measure progress •Understand what coverage you have Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Steady State •Maintain Testability •Continuously improve test suite Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Maintaining Testability •...requires vigilance •It’s easy to slip into old habits... •when there’s time pressure •when you’re tired •when you’re soloing •When it’s hard to write tests, do it more Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Maintaining Testability •Keep code amenable to testing •Keep coverage high, and focused •Keep the test corpus small •Less is more, assuming your coverage is complete •Don’t punt on hard testing problems Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • Continuous Improvement •Test Speed •Test Clarity •Test Quality •Test Readability •Remove stale tests Copyright © 2008 Pivotal Labs, Inc. All rights reserved.
  • The role of QA on an Agile team • Different pair of eyes • Can focus on exploratory testing and test planning • Catch gaps in requirements and coverage • Should write additional coverage • Less monkey work, more brain work • Any bug discovered by QA must have an automated regression test case written Copyright © 2008 Pivotal Labs, Inc. All rights reserved.