Agile Testing, Risk,
             Uncertainty, and Why It
             All Works
             (the updated for AGILEEE,
             almost all new edition)
             Elisabeth Hendrickson
             Quality Tree Software, Inc.
             www.qualitytree.com
             esh@qualitytree.com




                                Last updated September 22, 2011
Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
What Does Agile Really Mean?

Agile software teams…




                                    Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
…Deliver value in the form of
 releasable software at frequent
 regular intervals (at least
 monthly)…
…At a sustainable pace…
…While adapting to the changing
 needs of the business.
Increasing Complexity




                                             Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
Source: Wikimedia Commons. Courtesy of Ray
  Kurzweil and Kurzweil Technologies, Inc.
Speculation
                                                                      Speculation Buildup




      Analyze Design Implement Stabilize
    Release
    Big Bang




Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
Speculation, Lack of Alignment, and Risk




                                           Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
 Intentions         ?
 ?                    Implementation

Actual Need         ?
Speculation
                                                                      Agile eliminates speculation buildup…




Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
…only if we eliminate the speculation




Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
Speculation




       Iterations…
                                                                “frAgile”!


                     Stabilize




Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
speculation!
        Every. Single. Time.
                                               Evidence
                                              Empirical




Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
are the ones that
                                                     positioned to succeed



                          LEARN
                                                     The organizations best




Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
Learn
                                                   Build

                                   Measure




Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
How do we check alignment?
            Intentions




                                                    Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
          ?                 Implementation

          Actual Need


            Get out of
           the building!



Steve Blank, author of Four Steps to the Epiphany
How do we check alignment?




                                            Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
              Begin with the End in Mind.
                Capture examples with
                expectations. Automate
               checking of expectations.

          ?
 Intentions
               Explore to discover risks.



              Implementation

Actual Need
The Letter and Spirit of a Story




                                            Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  Explicit
Expectations




      Implicit
    Expectations
Check that all previously met expectations
are still met even as the solution grows in




                                              Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
          capability and complexity.
How do we check alignment?




                                         Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
 Intentions

              Implementation

Actual Need
              ?          Demo.
                   Rehearse delivery.
                   Deliver frequently.
                    Explore with the
                   customer or proxy.
Latency
                                                                      Reducing Feedback Latency




Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
What Does Agile Really Mean?

Agile software teams…




                                    Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
…Deliver value in the form of
 releasable software at frequent
 regular intervals (at least
 monthly)…
…At a sustainable pace…
…While adapting to the changing
 needs of the business.
Intentions




              Actual Need
              ✔
                                                     ✔
                                   Implementation




Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
(OK, now let’s talk about “testing.”)




   Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
any activity that yields
                     about
the extent to which our
           , our
                 , and the
                       are
       .
Intentions




            Actual Need
                                                                      Checking Alignment




                                 Implementation




Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
speculation!
        Every. Single. Time.
                                               Evidence
                                              Empirical




Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
The Whole Team Succeeds or Fails TOGETHER




                                                  Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
                        “Hey! There’s a
                        hole in your side
                         of the boat.”




                     As Alistair Cockburn says:

There is no “Us” and “Them.”
  There is only Us.
Nine Key Technical Testing-Related
              Agile Practices




                                               Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
                                 Exploratory
   ATDD            TDD
                                   Testing



 Automated      Automated        Collective
System Tests    Unit Tests       Ownership



Continuous      Automated         Rehearse
Integration       Deploy          Delivery
Test-First Mindset:
  Begin with the End in Mind




                                Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
Strategy
                 How will I
Initiative
                 know my
 Code            efforts have
                 the result I
                 intended?
Acceptance-Test Driven Development (ATDD)




                                                       Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
            Agile-Friendly Test Automation Framework

User Story Workshop             Implementation
Continuous Integration (CI)




                                                Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
       CI tools do
automated builds,
execute tests, and
report the results


                     Developers practicing CI
                     merge their changes
                     locally & execute tests
                     before checking in
Version ALL Technical Assets with the Code




                                              Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
           Source Control Repository


  Production     Acceptance       (Plus All
     Code          Tests            other
                                  technical
                                   assets)
  Unit Tests       Test
                 “Fixture”
                   Code
Increase Visibility




Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
Automated Deploy




Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
Fail Early, Fail Fast




                                                  Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
         Exploration
Automated Regression
             CI Build
                        Secs- Mins   Mins- Hrs-
     Local Unit Tests   Mins         Hrs   Days




    Failing early & fast gives us time
           to fix the problems.
Red Build Means Stop the Line




                                                       Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
     We can just throw           But that will
    that bug on the pile      increase technical
      with the others.           debt & slow
                                   velocity.




                           Yuck.
If a previously passing expectation fails, there’s a
     bug. Bugs slow everything down. To keep
          sustainable pace, fix bugs fast.
Exploratory Testing




                                                      Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
              Simultaneously…
              …learning about the software
              …designing tests
              …executing tests
              using feedback from the last
               test to inform the next



(See Jon and James Bach’s work on Session-Based ET)
+ Explored
                           Implemented



                               == DONE
                             + Checked
                                                                      Done Includes Tested




Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
speculation!
        Every. Single. Time.
                                               Evidence
                                              Empirical




Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
Intentions




                                 Actual Need
                          TEST
                                                      Implementation




LEARN
 Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.

AGILEEE Friday 17:15 Talk

  • 1.
    Agile Testing, Risk, Uncertainty, and Why It All Works (the updated for AGILEEE, almost all new edition) Elisabeth Hendrickson Quality Tree Software, Inc. www.qualitytree.com esh@qualitytree.com Last updated September 22, 2011 Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  • 2.
    What Does AgileReally Mean? Agile software teams… Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. …Deliver value in the form of releasable software at frequent regular intervals (at least monthly)… …At a sustainable pace… …While adapting to the changing needs of the business.
  • 3.
    Increasing Complexity Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. Source: Wikimedia Commons. Courtesy of Ray Kurzweil and Kurzweil Technologies, Inc.
  • 4.
    Speculation Speculation Buildup Analyze Design Implement Stabilize Release Big Bang Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  • 5.
    Speculation, Lack ofAlignment, and Risk Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. Intentions ? ? Implementation Actual Need ?
  • 6.
    Speculation Agile eliminates speculation buildup… Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  • 7.
    …only if weeliminate the speculation Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  • 8.
    Speculation Iterations… “frAgile”! Stabilize Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  • 9.
    speculation! Every. Single. Time. Evidence Empirical Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  • 10.
    are the onesthat positioned to succeed LEARN The organizations best Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  • 11.
    Learn Build Measure Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  • 12.
    How do wecheck alignment? Intentions Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. ? Implementation Actual Need Get out of the building! Steve Blank, author of Four Steps to the Epiphany
  • 13.
    How do wecheck alignment? Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. Begin with the End in Mind. Capture examples with expectations. Automate checking of expectations. ? Intentions Explore to discover risks. Implementation Actual Need
  • 14.
    The Letter andSpirit of a Story Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. Explicit Expectations Implicit Expectations
  • 15.
    Check that allpreviously met expectations are still met even as the solution grows in Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. capability and complexity.
  • 16.
    How do wecheck alignment? Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. Intentions Implementation Actual Need ? Demo. Rehearse delivery. Deliver frequently. Explore with the customer or proxy.
  • 17.
    Latency Reducing Feedback Latency Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  • 18.
    What Does AgileReally Mean? Agile software teams… Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. …Deliver value in the form of releasable software at frequent regular intervals (at least monthly)… …At a sustainable pace… …While adapting to the changing needs of the business.
  • 19.
    Intentions Actual Need ✔ ✔ Implementation Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  • 20.
    (OK, now let’stalk about “testing.”) Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  • 21.
    Copyright © 2011Elisabeth Hendrickson, Quality Tree Software, Inc. any activity that yields about the extent to which our , our , and the are .
  • 22.
    Intentions Actual Need Checking Alignment Implementation Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  • 23.
    speculation! Every. Single. Time. Evidence Empirical Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  • 24.
    The Whole TeamSucceeds or Fails TOGETHER Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. “Hey! There’s a hole in your side of the boat.” As Alistair Cockburn says: There is no “Us” and “Them.” There is only Us.
  • 25.
    Nine Key TechnicalTesting-Related Agile Practices Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. Exploratory ATDD TDD Testing Automated Automated Collective System Tests Unit Tests Ownership Continuous Automated Rehearse Integration Deploy Delivery
  • 26.
    Test-First Mindset: Begin with the End in Mind Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. Strategy How will I Initiative know my Code efforts have the result I intended?
  • 27.
    Acceptance-Test Driven Development(ATDD) Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. Agile-Friendly Test Automation Framework User Story Workshop Implementation
  • 28.
    Continuous Integration (CI) Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. CI tools do automated builds, execute tests, and report the results Developers practicing CI merge their changes locally & execute tests before checking in
  • 29.
    Version ALL TechnicalAssets with the Code Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. Source Control Repository Production Acceptance (Plus All Code Tests other technical assets) Unit Tests Test “Fixture” Code
  • 30.
    Increase Visibility Copyright ©2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  • 31.
    Automated Deploy Copyright ©2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  • 32.
    Fail Early, FailFast Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. Exploration Automated Regression CI Build Secs- Mins Mins- Hrs- Local Unit Tests Mins Hrs Days Failing early & fast gives us time to fix the problems.
  • 33.
    Red Build MeansStop the Line Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. We can just throw But that will that bug on the pile increase technical with the others. debt & slow velocity. Yuck. If a previously passing expectation fails, there’s a bug. Bugs slow everything down. To keep sustainable pace, fix bugs fast.
  • 34.
    Exploratory Testing Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc. Simultaneously… …learning about the software …designing tests …executing tests using feedback from the last test to inform the next (See Jon and James Bach’s work on Session-Based ET)
  • 35.
    + Explored Implemented == DONE + Checked Done Includes Tested Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  • 36.
    speculation! Every. Single. Time. Evidence Empirical Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.
  • 37.
    Intentions Actual Need TEST Implementation LEARN Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software, Inc.