SlideShare a Scribd company logo
Maintainable Acceptance Tests
                               Badrinath Janakiraman
                               Jez Humble
                               Agile 2012 Dallas

                               @badrij | badri@thoughtworks.com
                               @jezhumble | jez@thoughtworks.com




                          http://thoughtworks-studios.com/
Thursday, August 16, 12
what to expect
        • Creating high quality acceptance tests
        • How to structure a maintainable acceptance test
          suite
        • Patterns for effective teamwork
        • Managing test data




Thursday, August 16, 12
what to take away
        • Quality is everybody’s responsibility
        • High quality test suites are continuously curated
          - by testers and developers working together
        • Test code needs to receive the same care as
          production code
        • Exhaustive story-level testing is not a good
          basis for maintainable acceptance suites




Thursday, August 16, 12
different kinds of tests


                                                                  Business facing
                                                     AUTOMATED                       MANUAL

                                                                                Showcases
                          Support programming




                                                Functional acceptance
                                                                              Usability testing
                                                        tests




                                                                                                           Critique project
                                                                             Exploratory testing



                                                       Unit tests               Non-functional
                                                   Integration tests          acceptance tests
                                                     System tests         (performance, scaling, ...)
                                                     AUTOMATED               MANUAL / AUTOMATED

                                                                 Technology facing



                                                                                                        Diagram invented by Brian Marick

Thursday, August 16, 12
UI
                                    Mike Cohn:
                                    Succeeding with Agile
                          Service


                            Unit

Thursday, August 16, 12
End to End
                             Business Facing




                             Localized
                          Technology Facing

Thursday, August 16, 12
principle 0


        Writing good acceptance tests is
        hard.

        (good: when the tests are green,
        we know the software works)

Thursday, August 16, 12
mingle

                 2006             2012
                 20 tests         3000 tests
                 500 LOC          50k LOC
                 2 minutes        12 hours

              • actual running time: 55 minutes
              • 7-10 times a day
              • for 6 years, across 4 offices now

Thursday, August 16, 12
why do test suites decay?

   for the same reasons code does



   we don’t pay enough attention to expressing intent




   only testers care about maintaining tests

Thursday, August 16, 12
principles




Thursday, August 16, 12
principle 1


        Tests are first-class citizens of
        your project




Thursday, August 16, 12
preventing decay in test code

   treat test code as production c0de

   refactor relentlessly

   don’t repeat yourself

   don’t repeat yourself

   use record-playback tools to build your suite

Thursday, August 16, 12
preventing decay of intention


   given-when-then is insufficient


    separate intention from mechanics



   express the test as steps of a user's journey


Thursday, August 16, 12
a solution

    use natural language to express intentions



    use a general purpose programming language to
    express test mechanics


    use a tool that allows you to operate in either
    domain transparently

Thursday, August 16, 12
Thursday, August 16, 12
page object
                                                                         https://gist.github.com/3345556


    public class LoginPage {

        private final SeleniumSession browser;

        public LoginPage(Selenium browser){
          this.browser = browser;
        }

        public HomePage loginAs(String user, String password){
          browser.type('#login', login);
          browser.type('#password', password);
          browser.submit('#login-form');
          return new HomePage(this.browser);
        }

        public HomePage loginExpectingError(String user, String password){
          browser.type('#login', login);
          browser.type('#password', password);
          browser.submit('#login-form');
          return new LoginPage(this.browser);
        }
    }




Thursday, August 16, 12
Acceptance Criteria
                                            Customer    Tester




                      Test implementation
                                            Developer   Tester




Thursday, August 16, 12
tester / quality analyst
   ...is a role, not a person

   ...is not a failed developer

   ...advocates for the user and makes the quality of the
   system transparent
   ...should not be primarily working on manual
   regression testing
   ...should be focussed on exploratory testing &
   maintaining automated acceptance tests
Thursday, August 16, 12
remember

                 passing acceptance tests are necessary (but
                 insufficient) for “done”


                 encapsulate!



                 the acceptance tests are owned by - and the
                 responsibility of - the team
Thursday, August 16, 12
principle 2


        always interact with the system
        under test the same way a user
        would




Thursday, August 16, 12
browser based tests are unreliable

   "the test fails in CI, but when I run the app,
   everything seems fine"

  usually an indication that test mechanics and user
  interaction patterns differ

   ajax based tests?

   JS heavy applications, which need non-zero
   processing time to modify the UI
Thursday, August 16, 12
some solutions

   Test your application the way a user might use it.

   Understand when behavior is asynchronous and
   account for it explicitly

   Don’t use bare sleeps: poll

   If it’s hard to write the test, you need to have a
   conversation with the team

Thursday, August 16, 12
some solutions


    wait-utils (https://github.com/itspanzi/WaitUtils)



    for ajax tests, if your JS framework provides a pre-
    and post-call hook, intercept those to count the
    number of active calls before proceeding



Thursday, August 16, 12
var AjaxTracker = {                                                      https://gist.github.com/3315690

     PENDING_REQUESTS: $A([]),

     onCreate: function(request){
        if (!request.url.match(/gadgets/js//)) {
          this.PENDING_REQUESTS.push(request.url);
        }
     },

     onComplete: function(request){
        this.PENDING_REQUESTS = this.PENDING_REQUESTS.without(request.url);
     },

     onException: function(request, exception){
        try {
          this.onComplete(request);
        }catch(e){
          if (Prototype.isFireBugsEnabled) {
            console.log("Got Exception on request: " + request.url);
            console.log(e);
            throw(e);
          }
        }
     },

     allAjaxComplete: function(includeCardSummary){
       var requests = this.PENDING_REQUESTS.reject(function(url) {
           return url.match(/cards/card_summary/) || url.match(/also_viewing/);
         });
       return requests.size() == 0;
     }
};

Ajax.Responders.register(AjaxTracker);

Thursday, August 16, 12
remember

                 make time to go back and refactor your
                 tests


                 use layers and encapsulation: separate high
                 level intent and low level mechanics


                 use page object to interact with SUT; run
                 against service layer where possible
Thursday, August 16, 12
principle 3


        continuously curate the structure
        of your test suites




Thursday, August 16, 12
#1312
             #1301                                                                                              As a... I want... So that...
             As a... I want... So that...                                      #1310
                                                                               As a... I want... So that...     Given...       Given...
             Given...           Given...    #1306                                                               When...        When...
             When...            When...     As a... I want... So that...                                        Then...        Then...
                                                                               Given...      Given...
             Then...            Then...                                        When...     #1304
                                                                                             When...
                          #1315             Given...       Given...                        AsThen...
                                                                                              a... I want... So that...
                                                                               Then...
                                            When...        When...
                          As a... I want... So that...
                                            Then...        Then...                      Given...          Given...
                     Given...       Given...                      #1308                 When...           When...
                     When...        When...                       As a... I want... So that...
                                                                                        Then...           Then...
           #1307 Then...           Then...                                                                      #1313
           As a... I want... So that...                         Given...           Given...                    As a... I want... So that...
                                             #1317              When...            When...
                                                                                   Then...#1311
                                              As a... I want... Then...
                                                                So that...
           Given...           Given... #1303                                                                   Given...       Given...
           When...            When... As a... Given... So that...                         As a... I want... So When...
                                                                                                               that...        When...
                                               I want...     Given...
           Then...            Then...         When...        When...                                           Then...         Then...
                                                                                           Given...       Given...
                                       Given...
                                              Then... Given...
                                                             Then...                       When...        When...
                                       When...        When...
                                                                                           Then...        Then...
                                       Then...       Then...
                                                         #1305
                #1309                                    As a... I want... So that...                  #1302
                As a... I want... So that...                                                           As a... I want... So that...
                                                                             #1318
                                                         Given...          Given... I want... So that...
                Given...              #1316
                                   Given...              When...
                                                                            As a...
                                                                           When...                     Given... #1314Given...
                When...               As a... I want... So that...
                                   When...                                                             When... As a... I want... So that...
                                                                                                                     When...
                                                         Then...           Then...
                                                                            Given...       Given... Then...
                Then...            Then...                                                                          Then...
                                      Given...       Given...               When...        When...              Given...      Given...
                                      When...        When...                Then...       Then...               When...       When...
                                      Then...       Then...                                                     Then...       Then...



Thursday, August 16, 12
journey tests

                                     #1612
                                     As a customer
                                     I want a gift wrapping option
                                     So that I don’t have to wrap
    Buy Product                      them and post them myself       Buy Product

    Search product catalogue                                         Search product catalogue
    Add product to cart                                              Add product to cart
    Check out                                                        Check out
    Create new account                                               Create new account
    Provide address details                                          Provide address details
    Provide credit card details                                      Provide credit card details
    Complete order                                                   Select gift wrapping option
    Verify order created                                             Complete order
    Verify credit card debited                                       Verify order created
    Verify email sent                                                Verify gift wrapping option
                                                                     Verify credit card debited
                                                                     Verify email sent

Thursday, August 16, 12
some solutions

   identify user journeys

   ( journey: the path a persona takes through the
   application to achieve an end goal)

   most applications have very few distinct personas

   most stories in iterative development are
   enhancements to existing journeys
Thursday, August 16, 12
features
  Basic shopping cart functionality

  Searching for a product
  - searching for a word should bring up all products which have that word in their name
  - searching for a phrase should bring up all products which have any of the words in their
  name
  - searching for a quoted phrase should bring up all products which have all words in the
  the quoted phrase in their name

  Paginating search results
  - return 25 results per page by default
  - if there are fewer than 25 results, do not show pagination links
  - provide a "previous" link on every page other than the first page of results
  - provide a "next" link on every page other than the last page of results
  - if user supplies a page number which is less than 1 in the URL, stay on the first page
  - if the user supplies a page number greater than the last page of results, stay on the
  last page

  Gift-wrap option




Thursday, August 16, 12
story tests
    Story tests for search
    - test that searching for "friends" brings back 782 results
    -- results should include how to win friends and influence people

    - test that searching for dead friends brings back 8900 results
    -- results should include <how to win friends and influence people>
    -- results should include <The Zombie Survival Guide: Complete Protection from the
    Living Dead>

    - test that searching for "dead friends" brings back 57 results
    -- results should include <all my friends are dead>

    Story tests for pagination
    - with a catalog of 3 products, I should see no pagination links
    - with a catalog of 25 products, I should see no pagination links
    - with a catalog of 26 products, I should see 1 link to page two, along with a next link
    but no previous link
    - with a catalog of 26 products, on page 2, I should see one product, with a link to
    page one, a previous link but no next link

    Story tests for gift wrapping




Thursday, August 16, 12
journey tests
    Journey of user buying a book

    - Login as user "bob"
    - Search for <"my friends" dead>
    - Make sure that 3 pages of results show
    - Verify that "All My Friends Are Dead" by "Avery Monson" is on
    the first page
    - Add two copies of the book to the shopping cart
    - Gift wrap one of them
    - Proceed to checkout




Thursday, August 16, 12
more solutions

   extract journeys from your acceptance tests

   make them fast and run them first

   do test the most likely path that the team, business
   and UX folk agree upon

   do not test every possible path through the system

   extract negative tests and edge cases into a
   regression suite which runs after your journey tests
Thursday, August 16, 12
build quality in
                               “Cease dependence on
                               mass inspection to achieve
                               quality. Improve the
                               process and build quality
                               into the product in the first
                               place”

                                  W. Edwards Deming
Thursday, August 16, 12
why cross-functional teams?
   output of testing is not just bug reports


   feedback from testing to product design


   feedback from test framework to system architecture


   developers and testers share knowledge and skills

Thursday, August 16, 12
principle 4


        everyone owns acceptance tests




Thursday, August 16, 12
when acceptance tests break
                 Triage to find root cause
                     1. There was an environmental problem
                     2. There is a bug with the test
                     3. An assumption changed
                     4. The test actually caught a bug
                 Fix the problem
                 Add a guard to prevent it happening again
                 Optimise your test suite: detect failures fast
                 Optimise your process for time to fix tests
Thursday, August 16, 12
intermittent failures


                 flaky tests are worse than useless



                 quarantine flaky tests - but not forever


                 http://martinfowler.com/articles/
                 nonDeterminism.html
Thursday, August 16, 12
external systems

   not all tests should call the external system



   parameterize connections to external systems



   Run integration smoke tests before full
   acceptance suite

Thursday, August 16, 12
impersonator pattern

   create a proxy from SUT to external system

   cache results from integration smoke tests

   run integration smoke tests before acceptance suite

   periodically expire cache

   only run acceptance suite if integration smoke tests
   pass!
Thursday, August 16, 12
principle 5


        acceptance tests are responsible
        for managing their own test data




Thursday, August 16, 12
test data

   Test-specific data

   Test reference data

   Application reference data

   Ensure tests are independent

   Don’t use production data dumps (except for
   performance testing and staging)
Thursday, August 16, 12
recap
        1. treat acceptance tests like production code

        2. always interact with the SUT like a user would

        3. continuously curate your user journeys

        4. collective ownership of acceptance tests

        5. acceptance tests own their data




Thursday, August 16, 12
take-aways
        • quality is everybody’s responsibility

        • high quality test suites are continuously curated
          - by testers and developers working together

        • test code needs to receive the same care as
          production code

        • exhaustive story-level testing is not a good
          basis for maintainable acceptance suites



Thursday, August 16, 12
questions

        @badrij | badri@thoughtworks.com
        @jezhumble | jez@thoughtworks.com

        http://continuousdelivery.com/




                                                             © 2011 ThoughtWorks, Inc.


                          http://thoughtworks-studios.com/
Thursday, August 16, 12

More Related Content

What's hot

Stream Processing: Choosing the Right Tool for the Job
Stream Processing: Choosing the Right Tool for the JobStream Processing: Choosing the Right Tool for the Job
Stream Processing: Choosing the Right Tool for the Job
Databricks
 
A story of the passive aggressive sysadmin of AEM
A story of the passive aggressive sysadmin of AEMA story of the passive aggressive sysadmin of AEM
A story of the passive aggressive sysadmin of AEM
Frans Rosén
 
Prometheus - Intro, CNCF, TSDB,PromQL,Grafana
Prometheus - Intro, CNCF, TSDB,PromQL,GrafanaPrometheus - Intro, CNCF, TSDB,PromQL,Grafana
Prometheus - Intro, CNCF, TSDB,PromQL,Grafana
Sridhar Kumar N
 
Let's play with adf 3.0
Let's play with adf 3.0Let's play with adf 3.0
Let's play with adf 3.0
Eugenio Romano
 
Explain the explain_plan
Explain the explain_planExplain the explain_plan
Explain the explain_plan
Maria Colgan
 
Extending Flink SQL for stream processing use cases
Extending Flink SQL for stream processing use casesExtending Flink SQL for stream processing use cases
Extending Flink SQL for stream processing use cases
Flink Forward
 
Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...
Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...
Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...
Flink Forward
 
Troubleshooting Complex Oracle Performance Problems with Tanel Poder
Troubleshooting Complex Oracle Performance Problems with Tanel PoderTroubleshooting Complex Oracle Performance Problems with Tanel Poder
Troubleshooting Complex Oracle Performance Problems with Tanel Poder
Tanel Poder
 
Oracle flashback
Oracle flashbackOracle flashback
Oracle flashback
Cambodia
 
Oracle AWR Data mining
Oracle AWR Data miningOracle AWR Data mining
Oracle AWR Data mining
Yury Velikanov
 
Azure storage deep dive
Azure storage deep diveAzure storage deep dive
Azure storage deep dive
Sergio Navarro Pino
 
Best Practices for the Most Impactful Oracle Database 18c and 19c Features
Best Practices for the Most Impactful Oracle Database 18c and 19c FeaturesBest Practices for the Most Impactful Oracle Database 18c and 19c Features
Best Practices for the Most Impactful Oracle Database 18c and 19c Features
Markus Michalewicz
 
Oracle Database Performance Tuning Advanced Features and Best Practices for DBAs
Oracle Database Performance Tuning Advanced Features and Best Practices for DBAsOracle Database Performance Tuning Advanced Features and Best Practices for DBAs
Oracle Database Performance Tuning Advanced Features and Best Practices for DBAs
Zohar Elkayam
 
Build Your Own CMS with Apache Sling
Build Your Own CMS with Apache SlingBuild Your Own CMS with Apache Sling
Build Your Own CMS with Apache Sling
Bob Paulin
 
Performance Tuning With Oracle ASH and AWR. Part 1 How And What
Performance Tuning With Oracle ASH and AWR. Part 1 How And WhatPerformance Tuning With Oracle ASH and AWR. Part 1 How And What
Performance Tuning With Oracle ASH and AWR. Part 1 How And What
udaymoogala
 
How to Lock Down Apache Kafka and Keep Your Streams Safe
How to Lock Down Apache Kafka and Keep Your Streams SafeHow to Lock Down Apache Kafka and Keep Your Streams Safe
How to Lock Down Apache Kafka and Keep Your Streams Safe
confluent
 
Intro to Pentesting Jenkins
Intro to Pentesting JenkinsIntro to Pentesting Jenkins
Intro to Pentesting Jenkins
Brian Hysell
 
Getting Started Monitoring with Prometheus and Grafana
Getting Started Monitoring with Prometheus and GrafanaGetting Started Monitoring with Prometheus and Grafana
Getting Started Monitoring with Prometheus and Grafana
Syah Dwi Prihatmoko
 
Oracle E-Business Suite R12.2.6 on Database 12c: Install, Patch and Administer
Oracle E-Business Suite R12.2.6 on Database 12c: Install, Patch and AdministerOracle E-Business Suite R12.2.6 on Database 12c: Install, Patch and Administer
Oracle E-Business Suite R12.2.6 on Database 12c: Install, Patch and Administer
Andrejs Karpovs
 
Performance Management in Oracle 12c
Performance Management in Oracle 12cPerformance Management in Oracle 12c
Performance Management in Oracle 12c
Alfredo Krieg
 

What's hot (20)

Stream Processing: Choosing the Right Tool for the Job
Stream Processing: Choosing the Right Tool for the JobStream Processing: Choosing the Right Tool for the Job
Stream Processing: Choosing the Right Tool for the Job
 
A story of the passive aggressive sysadmin of AEM
A story of the passive aggressive sysadmin of AEMA story of the passive aggressive sysadmin of AEM
A story of the passive aggressive sysadmin of AEM
 
Prometheus - Intro, CNCF, TSDB,PromQL,Grafana
Prometheus - Intro, CNCF, TSDB,PromQL,GrafanaPrometheus - Intro, CNCF, TSDB,PromQL,Grafana
Prometheus - Intro, CNCF, TSDB,PromQL,Grafana
 
Let's play with adf 3.0
Let's play with adf 3.0Let's play with adf 3.0
Let's play with adf 3.0
 
Explain the explain_plan
Explain the explain_planExplain the explain_plan
Explain the explain_plan
 
Extending Flink SQL for stream processing use cases
Extending Flink SQL for stream processing use casesExtending Flink SQL for stream processing use cases
Extending Flink SQL for stream processing use cases
 
Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...
Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...
Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...
 
Troubleshooting Complex Oracle Performance Problems with Tanel Poder
Troubleshooting Complex Oracle Performance Problems with Tanel PoderTroubleshooting Complex Oracle Performance Problems with Tanel Poder
Troubleshooting Complex Oracle Performance Problems with Tanel Poder
 
Oracle flashback
Oracle flashbackOracle flashback
Oracle flashback
 
Oracle AWR Data mining
Oracle AWR Data miningOracle AWR Data mining
Oracle AWR Data mining
 
Azure storage deep dive
Azure storage deep diveAzure storage deep dive
Azure storage deep dive
 
Best Practices for the Most Impactful Oracle Database 18c and 19c Features
Best Practices for the Most Impactful Oracle Database 18c and 19c FeaturesBest Practices for the Most Impactful Oracle Database 18c and 19c Features
Best Practices for the Most Impactful Oracle Database 18c and 19c Features
 
Oracle Database Performance Tuning Advanced Features and Best Practices for DBAs
Oracle Database Performance Tuning Advanced Features and Best Practices for DBAsOracle Database Performance Tuning Advanced Features and Best Practices for DBAs
Oracle Database Performance Tuning Advanced Features and Best Practices for DBAs
 
Build Your Own CMS with Apache Sling
Build Your Own CMS with Apache SlingBuild Your Own CMS with Apache Sling
Build Your Own CMS with Apache Sling
 
Performance Tuning With Oracle ASH and AWR. Part 1 How And What
Performance Tuning With Oracle ASH and AWR. Part 1 How And WhatPerformance Tuning With Oracle ASH and AWR. Part 1 How And What
Performance Tuning With Oracle ASH and AWR. Part 1 How And What
 
How to Lock Down Apache Kafka and Keep Your Streams Safe
How to Lock Down Apache Kafka and Keep Your Streams SafeHow to Lock Down Apache Kafka and Keep Your Streams Safe
How to Lock Down Apache Kafka and Keep Your Streams Safe
 
Intro to Pentesting Jenkins
Intro to Pentesting JenkinsIntro to Pentesting Jenkins
Intro to Pentesting Jenkins
 
Getting Started Monitoring with Prometheus and Grafana
Getting Started Monitoring with Prometheus and GrafanaGetting Started Monitoring with Prometheus and Grafana
Getting Started Monitoring with Prometheus and Grafana
 
Oracle E-Business Suite R12.2.6 on Database 12c: Install, Patch and Administer
Oracle E-Business Suite R12.2.6 on Database 12c: Install, Patch and AdministerOracle E-Business Suite R12.2.6 on Database 12c: Install, Patch and Administer
Oracle E-Business Suite R12.2.6 on Database 12c: Install, Patch and Administer
 
Performance Management in Oracle 12c
Performance Management in Oracle 12cPerformance Management in Oracle 12c
Performance Management in Oracle 12c
 

Viewers also liked

How Spotify Does Test Automation - Kristian Karl
How Spotify Does Test Automation - Kristian KarlHow Spotify Does Test Automation - Kristian Karl
How Spotify Does Test Automation - Kristian Karl
SmartBear
 
Quality Built In @ Spotify
Quality Built In @ SpotifyQuality Built In @ Spotify
Quality Built In @ Spotify
Andrii Dzynia
 
Ross Snyder, Etsy, SXSW Lean Startup 2013
Ross Snyder, Etsy, SXSW Lean Startup 2013Ross Snyder, Etsy, SXSW Lean Startup 2013
Ross Snyder, Etsy, SXSW Lean Startup 2013
500 Startups
 
Tips for Designing, Testing & Delivering eLearning in a Multi-device World
Tips for Designing, Testing & Delivering eLearning in a Multi-device WorldTips for Designing, Testing & Delivering eLearning in a Multi-device World
Tips for Designing, Testing & Delivering eLearning in a Multi-device World
Upside Learning Solutions
 
Introduction to SoapUI day 1
Introduction to SoapUI day 1Introduction to SoapUI day 1
Introduction to SoapUI day 1
Qualitest
 
Introduction to Test Automation
Introduction to Test AutomationIntroduction to Test Automation
Introduction to Test Automation
Pekka Klärck
 
Continuous Delivery Sounds Great but it Won't Work Here
Continuous Delivery Sounds Great but it Won't Work HereContinuous Delivery Sounds Great but it Won't Work Here
Continuous Delivery Sounds Great but it Won't Work Here
Jez Humble
 
Applying the Lean Startup Model to the Enterprise
Applying the Lean Startup Model to the EnterpriseApplying the Lean Startup Model to the Enterprise
Applying the Lean Startup Model to the Enterprise
Jez Humble
 
Devops Scorecard
Devops ScorecardDevops Scorecard
Devops Scorecard
Jez Humble
 
Selenium Users Anonymous
Selenium Users AnonymousSelenium Users Anonymous
Selenium Users Anonymous
Dave Haeffner
 
Continuous Delivery
Continuous DeliveryContinuous Delivery
Continuous Delivery
Mike McGarr
 
Test coaching your agile team
Test coaching your agile teamTest coaching your agile team
Test coaching your agile team
Andrii Dzynia
 
What is the best way to measure progress on an Agile project?
What is the best way to measure progress on an Agile project?What is the best way to measure progress on an Agile project?
What is the best way to measure progress on an Agile project?
ThoughtWorks Studios
 
RTSJ
RTSJRTSJ
Building Real-Time Web Applications
Building Real-Time Web ApplicationsBuilding Real-Time Web Applications
Building Real-Time Web Applications
Tony Abou-Assaleh
 
HTML5 WebSockets in Python/Django
HTML5 WebSockets in Python/DjangoHTML5 WebSockets in Python/Django
HTML5 WebSockets in Python/Django
Tony Abou-Assaleh
 
QA процесс, часть 2
QA процесс, часть 2QA процесс, часть 2
QA процесс, часть 2
DressTester
 

Viewers also liked (18)

How Spotify Does Test Automation - Kristian Karl
How Spotify Does Test Automation - Kristian KarlHow Spotify Does Test Automation - Kristian Karl
How Spotify Does Test Automation - Kristian Karl
 
Quality Built In @ Spotify
Quality Built In @ SpotifyQuality Built In @ Spotify
Quality Built In @ Spotify
 
Ross Snyder, Etsy, SXSW Lean Startup 2013
Ross Snyder, Etsy, SXSW Lean Startup 2013Ross Snyder, Etsy, SXSW Lean Startup 2013
Ross Snyder, Etsy, SXSW Lean Startup 2013
 
Tips for Designing, Testing & Delivering eLearning in a Multi-device World
Tips for Designing, Testing & Delivering eLearning in a Multi-device WorldTips for Designing, Testing & Delivering eLearning in a Multi-device World
Tips for Designing, Testing & Delivering eLearning in a Multi-device World
 
Introduction to SoapUI day 1
Introduction to SoapUI day 1Introduction to SoapUI day 1
Introduction to SoapUI day 1
 
Introduction to Test Automation
Introduction to Test AutomationIntroduction to Test Automation
Introduction to Test Automation
 
Continuous Delivery Sounds Great but it Won't Work Here
Continuous Delivery Sounds Great but it Won't Work HereContinuous Delivery Sounds Great but it Won't Work Here
Continuous Delivery Sounds Great but it Won't Work Here
 
Applying the Lean Startup Model to the Enterprise
Applying the Lean Startup Model to the EnterpriseApplying the Lean Startup Model to the Enterprise
Applying the Lean Startup Model to the Enterprise
 
Devops Scorecard
Devops ScorecardDevops Scorecard
Devops Scorecard
 
Selenium Users Anonymous
Selenium Users AnonymousSelenium Users Anonymous
Selenium Users Anonymous
 
Continuous Delivery
Continuous DeliveryContinuous Delivery
Continuous Delivery
 
Test coaching your agile team
Test coaching your agile teamTest coaching your agile team
Test coaching your agile team
 
What is the best way to measure progress on an Agile project?
What is the best way to measure progress on an Agile project?What is the best way to measure progress on an Agile project?
What is the best way to measure progress on an Agile project?
 
When qa sucks
When qa sucksWhen qa sucks
When qa sucks
 
RTSJ
RTSJRTSJ
RTSJ
 
Building Real-Time Web Applications
Building Real-Time Web ApplicationsBuilding Real-Time Web Applications
Building Real-Time Web Applications
 
HTML5 WebSockets in Python/Django
HTML5 WebSockets in Python/DjangoHTML5 WebSockets in Python/Django
HTML5 WebSockets in Python/Django
 
QA процесс, часть 2
QA процесс, часть 2QA процесс, часть 2
QA процесс, часть 2
 

Similar to Creating Maintainable Automated Acceptance Tests

Java Course 6: Introduction to Agile
Java Course 6: Introduction to AgileJava Course 6: Introduction to Agile
Java Course 6: Introduction to Agile
Anton Keks
 
Quality assurance in distributed continuous delivery
Quality assurance in distributed continuous deliveryQuality assurance in distributed continuous delivery
Quality assurance in distributed continuous delivery
mingjin
 
Agile Testing 2020
Agile Testing 2020Agile Testing 2020
Agile Testing 2020
arzu TR
 
Agile Java Testing With Open Source Frameworks
Agile Java Testing With Open Source FrameworksAgile Java Testing With Open Source Frameworks
Agile Java Testing With Open Source Frameworks
Viraf Karai
 
Software Quality and Test Strategies for Ruby and Rails Applications
Software Quality and Test Strategies for Ruby and Rails ApplicationsSoftware Quality and Test Strategies for Ruby and Rails Applications
Software Quality and Test Strategies for Ruby and Rails Applications
Bhavin Javia
 
Test Automation Frameworks: Assumptions, Concepts & Tools
Test Automation Frameworks: Assumptions, Concepts & ToolsTest Automation Frameworks: Assumptions, Concepts & Tools
Test Automation Frameworks: Assumptions, Concepts & Tools
Amit Rawat
 
RahulAnand_Testing_5.9years_exp_CV
RahulAnand_Testing_5.9years_exp_CVRahulAnand_Testing_5.9years_exp_CV
RahulAnand_Testing_5.9years_exp_CV
Rahul Anand
 
Software Testing
Software TestingSoftware Testing
Software Testing
AdroitLogic
 
Test-Driven Development
Test-Driven DevelopmentTest-Driven Development
Test-Driven Development
John Blum
 
Continuous delivery agile_2012
Continuous delivery agile_2012Continuous delivery agile_2012
Continuous delivery agile_2012
drewz lin
 
Test automation - Building effective solutions
Test automation - Building effective solutionsTest automation - Building effective solutions
Test automation - Building effective solutions
Artem Nagornyi
 
Agile testing - Principles and best practices
Agile testing  - Principles and best practicesAgile testing  - Principles and best practices
Agile testing - Principles and best practices
Dr Ganesh Iyer
 
7 Deadly Sins of Agile Software Test Automation
7 Deadly Sins of Agile Software Test Automation7 Deadly Sins of Agile Software Test Automation
7 Deadly Sins of Agile Software Test Automation
Adrian Smith
 
Good practices for debugging Selenium and Appium tests
Good practices for debugging Selenium and Appium testsGood practices for debugging Selenium and Appium tests
Good practices for debugging Selenium and Appium tests
Abhijeet Vaikar
 
Continuous Security Testing
Continuous Security TestingContinuous Security Testing
Continuous Security Testing
Steven Mak
 
Test Driven Development Introduction
Test Driven Development IntroductionTest Driven Development Introduction
Test Driven Development Introduction
Nguyen Hai
 
Iasi code camp 20 april 2013 implement-quality-java-massol-codecamp
Iasi code camp 20 april 2013 implement-quality-java-massol-codecampIasi code camp 20 april 2013 implement-quality-java-massol-codecamp
Iasi code camp 20 april 2013 implement-quality-java-massol-codecamp
Codecamp Romania
 
Appium workshop technopark trivandrum
Appium workshop technopark trivandrumAppium workshop technopark trivandrum
Appium workshop technopark trivandrum
Syam Sasi
 
Webinar: Estrategias para optimizar los costos de testing
Webinar: Estrategias para optimizar los costos de testingWebinar: Estrategias para optimizar los costos de testing
Webinar: Estrategias para optimizar los costos de testing
Federico Toledo
 
Integration testing - A&BP CC
Integration testing - A&BP CCIntegration testing - A&BP CC
Integration testing - A&BP CC
JWORKS powered by Ordina
 

Similar to Creating Maintainable Automated Acceptance Tests (20)

Java Course 6: Introduction to Agile
Java Course 6: Introduction to AgileJava Course 6: Introduction to Agile
Java Course 6: Introduction to Agile
 
Quality assurance in distributed continuous delivery
Quality assurance in distributed continuous deliveryQuality assurance in distributed continuous delivery
Quality assurance in distributed continuous delivery
 
Agile Testing 2020
Agile Testing 2020Agile Testing 2020
Agile Testing 2020
 
Agile Java Testing With Open Source Frameworks
Agile Java Testing With Open Source FrameworksAgile Java Testing With Open Source Frameworks
Agile Java Testing With Open Source Frameworks
 
Software Quality and Test Strategies for Ruby and Rails Applications
Software Quality and Test Strategies for Ruby and Rails ApplicationsSoftware Quality and Test Strategies for Ruby and Rails Applications
Software Quality and Test Strategies for Ruby and Rails Applications
 
Test Automation Frameworks: Assumptions, Concepts & Tools
Test Automation Frameworks: Assumptions, Concepts & ToolsTest Automation Frameworks: Assumptions, Concepts & Tools
Test Automation Frameworks: Assumptions, Concepts & Tools
 
RahulAnand_Testing_5.9years_exp_CV
RahulAnand_Testing_5.9years_exp_CVRahulAnand_Testing_5.9years_exp_CV
RahulAnand_Testing_5.9years_exp_CV
 
Software Testing
Software TestingSoftware Testing
Software Testing
 
Test-Driven Development
Test-Driven DevelopmentTest-Driven Development
Test-Driven Development
 
Continuous delivery agile_2012
Continuous delivery agile_2012Continuous delivery agile_2012
Continuous delivery agile_2012
 
Test automation - Building effective solutions
Test automation - Building effective solutionsTest automation - Building effective solutions
Test automation - Building effective solutions
 
Agile testing - Principles and best practices
Agile testing  - Principles and best practicesAgile testing  - Principles and best practices
Agile testing - Principles and best practices
 
7 Deadly Sins of Agile Software Test Automation
7 Deadly Sins of Agile Software Test Automation7 Deadly Sins of Agile Software Test Automation
7 Deadly Sins of Agile Software Test Automation
 
Good practices for debugging Selenium and Appium tests
Good practices for debugging Selenium and Appium testsGood practices for debugging Selenium and Appium tests
Good practices for debugging Selenium and Appium tests
 
Continuous Security Testing
Continuous Security TestingContinuous Security Testing
Continuous Security Testing
 
Test Driven Development Introduction
Test Driven Development IntroductionTest Driven Development Introduction
Test Driven Development Introduction
 
Iasi code camp 20 april 2013 implement-quality-java-massol-codecamp
Iasi code camp 20 april 2013 implement-quality-java-massol-codecampIasi code camp 20 april 2013 implement-quality-java-massol-codecamp
Iasi code camp 20 april 2013 implement-quality-java-massol-codecamp
 
Appium workshop technopark trivandrum
Appium workshop technopark trivandrumAppium workshop technopark trivandrum
Appium workshop technopark trivandrum
 
Webinar: Estrategias para optimizar los costos de testing
Webinar: Estrategias para optimizar los costos de testingWebinar: Estrategias para optimizar los costos de testing
Webinar: Estrategias para optimizar los costos de testing
 
Integration testing - A&BP CC
Integration testing - A&BP CCIntegration testing - A&BP CC
Integration testing - A&BP CC
 

Recently uploaded

Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Precisely
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Pitangent Analytics & Technology Solutions Pvt. Ltd
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
Fwdays
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
saastr
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Alpen-Adria-Universität
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
DianaGray10
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
operationspcvita
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
Ivo Velitchkov
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
c5vrf27qcz
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
Neo4j
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
Fwdays
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Neo4j
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 

Recently uploaded (20)

Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Artificial Intelligence and Electronic Warfare
Artificial Intelligence and Electronic WarfareArtificial Intelligence and Electronic Warfare
Artificial Intelligence and Electronic Warfare
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 

Creating Maintainable Automated Acceptance Tests

  • 1. Maintainable Acceptance Tests Badrinath Janakiraman Jez Humble Agile 2012 Dallas @badrij | badri@thoughtworks.com @jezhumble | jez@thoughtworks.com http://thoughtworks-studios.com/ Thursday, August 16, 12
  • 2. what to expect • Creating high quality acceptance tests • How to structure a maintainable acceptance test suite • Patterns for effective teamwork • Managing test data Thursday, August 16, 12
  • 3. what to take away • Quality is everybody’s responsibility • High quality test suites are continuously curated - by testers and developers working together • Test code needs to receive the same care as production code • Exhaustive story-level testing is not a good basis for maintainable acceptance suites Thursday, August 16, 12
  • 4. different kinds of tests Business facing AUTOMATED MANUAL Showcases Support programming Functional acceptance Usability testing tests Critique project Exploratory testing Unit tests Non-functional Integration tests acceptance tests System tests (performance, scaling, ...) AUTOMATED MANUAL / AUTOMATED Technology facing Diagram invented by Brian Marick Thursday, August 16, 12
  • 5. UI Mike Cohn: Succeeding with Agile Service Unit Thursday, August 16, 12
  • 6. End to End Business Facing Localized Technology Facing Thursday, August 16, 12
  • 7. principle 0 Writing good acceptance tests is hard. (good: when the tests are green, we know the software works) Thursday, August 16, 12
  • 8. mingle 2006 2012 20 tests 3000 tests 500 LOC 50k LOC 2 minutes 12 hours • actual running time: 55 minutes • 7-10 times a day • for 6 years, across 4 offices now Thursday, August 16, 12
  • 9. why do test suites decay? for the same reasons code does we don’t pay enough attention to expressing intent only testers care about maintaining tests Thursday, August 16, 12
  • 11. principle 1 Tests are first-class citizens of your project Thursday, August 16, 12
  • 12. preventing decay in test code treat test code as production c0de refactor relentlessly don’t repeat yourself don’t repeat yourself use record-playback tools to build your suite Thursday, August 16, 12
  • 13. preventing decay of intention given-when-then is insufficient separate intention from mechanics express the test as steps of a user's journey Thursday, August 16, 12
  • 14. a solution use natural language to express intentions use a general purpose programming language to express test mechanics use a tool that allows you to operate in either domain transparently Thursday, August 16, 12
  • 16. page object https://gist.github.com/3345556 public class LoginPage { private final SeleniumSession browser; public LoginPage(Selenium browser){ this.browser = browser; } public HomePage loginAs(String user, String password){ browser.type('#login', login); browser.type('#password', password); browser.submit('#login-form'); return new HomePage(this.browser); } public HomePage loginExpectingError(String user, String password){ browser.type('#login', login); browser.type('#password', password); browser.submit('#login-form'); return new LoginPage(this.browser); } } Thursday, August 16, 12
  • 17. Acceptance Criteria Customer Tester Test implementation Developer Tester Thursday, August 16, 12
  • 18. tester / quality analyst ...is a role, not a person ...is not a failed developer ...advocates for the user and makes the quality of the system transparent ...should not be primarily working on manual regression testing ...should be focussed on exploratory testing & maintaining automated acceptance tests Thursday, August 16, 12
  • 19. remember passing acceptance tests are necessary (but insufficient) for “done” encapsulate! the acceptance tests are owned by - and the responsibility of - the team Thursday, August 16, 12
  • 20. principle 2 always interact with the system under test the same way a user would Thursday, August 16, 12
  • 21. browser based tests are unreliable "the test fails in CI, but when I run the app, everything seems fine" usually an indication that test mechanics and user interaction patterns differ ajax based tests? JS heavy applications, which need non-zero processing time to modify the UI Thursday, August 16, 12
  • 22. some solutions Test your application the way a user might use it. Understand when behavior is asynchronous and account for it explicitly Don’t use bare sleeps: poll If it’s hard to write the test, you need to have a conversation with the team Thursday, August 16, 12
  • 23. some solutions wait-utils (https://github.com/itspanzi/WaitUtils) for ajax tests, if your JS framework provides a pre- and post-call hook, intercept those to count the number of active calls before proceeding Thursday, August 16, 12
  • 24. var AjaxTracker = { https://gist.github.com/3315690 PENDING_REQUESTS: $A([]), onCreate: function(request){ if (!request.url.match(/gadgets/js//)) { this.PENDING_REQUESTS.push(request.url); } }, onComplete: function(request){ this.PENDING_REQUESTS = this.PENDING_REQUESTS.without(request.url); }, onException: function(request, exception){ try { this.onComplete(request); }catch(e){ if (Prototype.isFireBugsEnabled) { console.log("Got Exception on request: " + request.url); console.log(e); throw(e); } } }, allAjaxComplete: function(includeCardSummary){ var requests = this.PENDING_REQUESTS.reject(function(url) { return url.match(/cards/card_summary/) || url.match(/also_viewing/); }); return requests.size() == 0; } }; Ajax.Responders.register(AjaxTracker); Thursday, August 16, 12
  • 25. remember make time to go back and refactor your tests use layers and encapsulation: separate high level intent and low level mechanics use page object to interact with SUT; run against service layer where possible Thursday, August 16, 12
  • 26. principle 3 continuously curate the structure of your test suites Thursday, August 16, 12
  • 27. #1312 #1301 As a... I want... So that... As a... I want... So that... #1310 As a... I want... So that... Given... Given... Given... Given... #1306 When... When... When... When... As a... I want... So that... Then... Then... Given... Given... Then... Then... When... #1304 When... #1315 Given... Given... AsThen... a... I want... So that... Then... When... When... As a... I want... So that... Then... Then... Given... Given... Given... Given... #1308 When... When... When... When... As a... I want... So that... Then... Then... #1307 Then... Then... #1313 As a... I want... So that... Given... Given... As a... I want... So that... #1317 When... When... Then...#1311 As a... I want... Then... So that... Given... Given... #1303 Given... Given... When... When... As a... Given... So that... As a... I want... So When... that... When... I want... Given... Then... Then... When... When... Then... Then... Given... Given... Given... Then... Given... Then... When... When... When... When... Then... Then... Then... Then... #1305 #1309 As a... I want... So that... #1302 As a... I want... So that... As a... I want... So that... #1318 Given... Given... I want... So that... Given... #1316 Given... When... As a... When... Given... #1314Given... When... As a... I want... So that... When... When... As a... I want... So that... When... Then... Then... Given... Given... Then... Then... Then... Then... Given... Given... When... When... Given... Given... When... When... Then... Then... When... When... Then... Then... Then... Then... Thursday, August 16, 12
  • 28. journey tests #1612 As a customer I want a gift wrapping option So that I don’t have to wrap Buy Product them and post them myself Buy Product Search product catalogue Search product catalogue Add product to cart Add product to cart Check out Check out Create new account Create new account Provide address details Provide address details Provide credit card details Provide credit card details Complete order Select gift wrapping option Verify order created Complete order Verify credit card debited Verify order created Verify email sent Verify gift wrapping option Verify credit card debited Verify email sent Thursday, August 16, 12
  • 29. some solutions identify user journeys ( journey: the path a persona takes through the application to achieve an end goal) most applications have very few distinct personas most stories in iterative development are enhancements to existing journeys Thursday, August 16, 12
  • 30. features Basic shopping cart functionality Searching for a product - searching for a word should bring up all products which have that word in their name - searching for a phrase should bring up all products which have any of the words in their name - searching for a quoted phrase should bring up all products which have all words in the the quoted phrase in their name Paginating search results - return 25 results per page by default - if there are fewer than 25 results, do not show pagination links - provide a "previous" link on every page other than the first page of results - provide a "next" link on every page other than the last page of results - if user supplies a page number which is less than 1 in the URL, stay on the first page - if the user supplies a page number greater than the last page of results, stay on the last page Gift-wrap option Thursday, August 16, 12
  • 31. story tests Story tests for search - test that searching for "friends" brings back 782 results -- results should include how to win friends and influence people - test that searching for dead friends brings back 8900 results -- results should include <how to win friends and influence people> -- results should include <The Zombie Survival Guide: Complete Protection from the Living Dead> - test that searching for "dead friends" brings back 57 results -- results should include <all my friends are dead> Story tests for pagination - with a catalog of 3 products, I should see no pagination links - with a catalog of 25 products, I should see no pagination links - with a catalog of 26 products, I should see 1 link to page two, along with a next link but no previous link - with a catalog of 26 products, on page 2, I should see one product, with a link to page one, a previous link but no next link Story tests for gift wrapping Thursday, August 16, 12
  • 32. journey tests Journey of user buying a book - Login as user "bob" - Search for <"my friends" dead> - Make sure that 3 pages of results show - Verify that "All My Friends Are Dead" by "Avery Monson" is on the first page - Add two copies of the book to the shopping cart - Gift wrap one of them - Proceed to checkout Thursday, August 16, 12
  • 33. more solutions extract journeys from your acceptance tests make them fast and run them first do test the most likely path that the team, business and UX folk agree upon do not test every possible path through the system extract negative tests and edge cases into a regression suite which runs after your journey tests Thursday, August 16, 12
  • 34. build quality in “Cease dependence on mass inspection to achieve quality. Improve the process and build quality into the product in the first place” W. Edwards Deming Thursday, August 16, 12
  • 35. why cross-functional teams? output of testing is not just bug reports feedback from testing to product design feedback from test framework to system architecture developers and testers share knowledge and skills Thursday, August 16, 12
  • 36. principle 4 everyone owns acceptance tests Thursday, August 16, 12
  • 37. when acceptance tests break Triage to find root cause 1. There was an environmental problem 2. There is a bug with the test 3. An assumption changed 4. The test actually caught a bug Fix the problem Add a guard to prevent it happening again Optimise your test suite: detect failures fast Optimise your process for time to fix tests Thursday, August 16, 12
  • 38. intermittent failures flaky tests are worse than useless quarantine flaky tests - but not forever http://martinfowler.com/articles/ nonDeterminism.html Thursday, August 16, 12
  • 39. external systems not all tests should call the external system parameterize connections to external systems Run integration smoke tests before full acceptance suite Thursday, August 16, 12
  • 40. impersonator pattern create a proxy from SUT to external system cache results from integration smoke tests run integration smoke tests before acceptance suite periodically expire cache only run acceptance suite if integration smoke tests pass! Thursday, August 16, 12
  • 41. principle 5 acceptance tests are responsible for managing their own test data Thursday, August 16, 12
  • 42. test data Test-specific data Test reference data Application reference data Ensure tests are independent Don’t use production data dumps (except for performance testing and staging) Thursday, August 16, 12
  • 43. recap 1. treat acceptance tests like production code 2. always interact with the SUT like a user would 3. continuously curate your user journeys 4. collective ownership of acceptance tests 5. acceptance tests own their data Thursday, August 16, 12
  • 44. take-aways • quality is everybody’s responsibility • high quality test suites are continuously curated - by testers and developers working together • test code needs to receive the same care as production code • exhaustive story-level testing is not a good basis for maintainable acceptance suites Thursday, August 16, 12
  • 45. questions @badrij | badri@thoughtworks.com @jezhumble | jez@thoughtworks.com http://continuousdelivery.com/ © 2011 ThoughtWorks, Inc. http://thoughtworks-studios.com/ Thursday, August 16, 12