The Testable Web
       by Dave Haeffner
       Prepared for the NYC Web Design Meetup
       March 11, 2013




Saturday, March 16, 13
The Motley Fool




                         To Educate, Amuse, and Enrich

Saturday, March 16, 13
Arrgyle




                         Navigate The High Tech Seas

Saturday, March 16, 13
Saturday, March 16, 13
Saturday, March 16, 13
Saturday, March 16, 13
How many of you work on web things?
               (e.g. websites, web applications, mobile apps)




Saturday, March 16, 13
How many of you test what you build?
                    (or have someone on your team who does)




Saturday, March 16, 13
How many of you primarily test manually?




Saturday, March 16, 13
How many of you primarily with automation?




Saturday, March 16, 13
How many of you test with a mix of both?




Saturday, March 16, 13
How many of you don’t test?




Saturday, March 16, 13
Show of hands if any of these are true for you:

         Save testing for the end of the cycle
         Feel like there isn’t enough time to adequately test
         Feel unnerved when it’s time to release
         Find issues in production




Saturday, March 16, 13
The Dirty Little Secrets of Testing




Saturday, March 16, 13
#1: You don’t have to test everything




Saturday, March 16, 13
Pareto Analysis




Saturday, March 16, 13
Saturday, March 16, 13
#2: Everything You Need to know about testing,
                      you learned in Kindegarten.




Saturday, March 16, 13
Finding the Glitter Glue Path




Saturday, March 16, 13
Saturday, March 16, 13
Questions worth asking,
                         to help you find your glitter glue path




Saturday, March 16, 13
How does your business*
                           make make money/deliver value?




                         * the business you are building software for
Saturday, March 16, 13
How do the users actually use it?




Saturday, March 16, 13
What are the prominent/relevant traits of your users
        (e.g. browser types, geographic location, etc.)?




Saturday, March 16, 13
What things have bitten you before?




Saturday, March 16, 13
“An example would be handy right about now”
                            - Brian Marick




Saturday, March 16, 13
Saturday, March 16, 13
7 Automated Acceptance Tests*
                           Successful platform upgrade




Saturday, March 16, 13
Saturday, March 16, 13
Saturday, March 16, 13
The Motley Fool




                         To Educate, Amuse, and Enrich

Saturday, March 16, 13
These things can take time.

        You have my permission to try until you get it right.




Saturday, March 16, 13
ext




Saturday, March 16, 13
When To Automate?
                         Questions worth asking




Saturday, March 16, 13
How long will it take to test manually?




Saturday, March 16, 13
How long will it take to test manually?

    This will help you get a baseline before answering the
                      following questions.




Saturday, March 16, 13
How often will this functionality need to be checked?




Saturday, March 16, 13
How often will this functionality need to be checked?

If often, then it is likely a good candidate for automation.




Saturday, March 16, 13
Is it valuable (e.g. core functionality, used heavily, a legal
                  or political requirement)?




Saturday, March 16, 13
Is it valuable (e.g. core functionality, used heavily, a legal
                  or political requirement)?

    If yes, then it is very likely a candidate for automation.




Saturday, March 16, 13
How risky is the feature (e.g. fundamental changes,
        complex architecture, large user facing impact)?




Saturday, March 16, 13
How risky is the feature (e.g. fundamental changes,
        complex architecture, large user facing impact)?

                            High? Medium? Or Low?

                         If Medium to High, then maybe.




Saturday, March 16, 13
What is the level of effort to automate?




Saturday, March 16, 13
What is the level of effort to automate?

                             T-shirt sizing will help you here:
                                     Small: 0-2 hours
                                    Medium: 1-2 days
                                      Large: 2-4 days
                                    X-Large >= 1 week

                             If large or higher, probably not

                               But it really depends
                    on what other work you have committed to

Saturday, March 16, 13
Does it involve look and feel?




Saturday, March 16, 13
Does it involve look and feel?

 If so, then it is likely a ripe candidate for manual testing.




Saturday, March 16, 13
How To Automate (strategy)?




Saturday, March 16, 13
http://martinfowler.com/bliki/TestPyramid.html
Saturday, March 16, 13
Build a feedback loop

                         (Version Control are table stakes)




Saturday, March 16, 13
Version Control System (VCS)




Saturday, March 16, 13
Continuous Integration (CI)




Saturday, March 16, 13
How to Automate (tactics)?




Saturday, March 16, 13
Choose your weapon




Saturday, March 16, 13
Consider Your Current Tech Stack




Saturday, March 16, 13
Then Layer In Your Team’s Context
      (strengths, comfort, confidence, excitement/interest)




Saturday, March 16, 13
Programming Language?

    Doesn’t have to be the same as what you develop in.




Saturday, March 16, 13
Who will own this?




Saturday, March 16, 13
Open Source Testing Tools




Saturday, March 16, 13
Agile Alliance Functional Testing Tool group
                              http://bit.ly/AgileTestTools




Saturday, March 16, 13
But let’s assume UI is involved.

                         In that case, something with WebDriver.




Saturday, March 16, 13
Full Disclosure: I have some opinions on this

          Released an open-source web testing framework
                      for Selenium WebDriver

http://arrgyle.com/blog/automated-web-testing-is-hard/




Saturday, March 16, 13
World’s Quickest Overview on how Selenium works




Saturday, March 16, 13
Selenium...

       is a robot sent from the future to help us test web
       sites

       uses locators (id, name, link, css, xpath, etc.) to
       interact with the html elements of the page (a.k.a
       ‘find by strategy’)

                               ------

           The simplest and most sustainable way is to use
                          straight up CSS.
Saturday, March 16, 13
How to build UI tests so you won’t hate yourself




Saturday, March 16, 13
Santi says...




Saturday, March 16, 13
Record & Playback is merely a stepping stone
                             Export your tests
                           Treat them like code
                           Page Object Pattern




Saturday, March 16, 13
He also says:
                         Only use UI tests when you have to...




Saturday, March 16, 13
But won’t they be slow?




Saturday, March 16, 13
Saturday, March 16, 13
Saturday, March 16, 13
In Practice




Saturday, March 16, 13
Test Pack 1    Test Pack 2
                         VCS   CI
                                    Simple checks   Automated
                                       Builds,      Acceptance
                                      Unit Tests       Tests


   Local Dev




Saturday, March 16, 13
Garbage In, Garbage Out




Saturday, March 16, 13
A more testable UI




Saturday, March 16, 13
More semantic markup
                         (e.g. unique and descriptive locators)




Saturday, March 16, 13
Inventory and consolidate your design patterns




Saturday, March 16, 13
But really, it’s about usability




Saturday, March 16, 13
“The more people think about usability, the more
              things tend to work out. It's a virtuous path.”

                 - Ken Pier, Senior Director of Product Quality,
                                   Socialtext




Saturday, March 16, 13
“It’s only words and photos on the internet.”

              - John F. Croston III, Web Content Accessibility
               Manager for the United States Army (army.mil)




Saturday, March 16, 13
Outro




Saturday, March 16, 13
Recap

       • You Don’t Have To Test Everything


       • But do test, intelligently, asking yourself


             • What matters?


             • Should I automate this?


       • Choose a toolchain that’s right for you and your team


       • Build a feedback loop


       • Build a more descriptive, usable, and testable web


Saturday, March 16, 13
I’m writing a book about this stuff!

                              arrgyle.com/book



                         @TourDeDave                    dave@arrgyle.com
Saturday, March 16, 13
Appendix




Saturday, March 16, 13
• A practical guide to usability: http://jfciii.com/presentations/wasp/
         accessibility.html



       • How to address risk in Agile (view it like an investment portfolio): http://
         arrgyle.com/blog/agile-accounting/



       • Automated Web Testing Is Hard (Introducing ChemistryKit, an open source
         testing framework to make automated web testing easier): http://arrgyle.com/
         blog/automated-web-testing-is-hard/



       • CSS Selectors in Selenium: http://sauceio.com/index.php/2010/01/selenium-
         totw-css-selectors-in-selenium-demystified/



       • Locators in Selenium http://marakana.com/bookshelf/selenium_tutorial/
         locators.html



Saturday, March 16, 13
• Page Objects in Selenium: https://code.google.com/p/selenium/wiki/
         PageObjects



       • Page Objects Ruby Gem: https://github.com/cheezy/page-object




Saturday, March 16, 13

The Testable Web

  • 1.
    The Testable Web by Dave Haeffner Prepared for the NYC Web Design Meetup March 11, 2013 Saturday, March 16, 13
  • 2.
    The Motley Fool To Educate, Amuse, and Enrich Saturday, March 16, 13
  • 3.
    Arrgyle Navigate The High Tech Seas Saturday, March 16, 13
  • 4.
  • 5.
  • 6.
  • 7.
    How many ofyou work on web things? (e.g. websites, web applications, mobile apps) Saturday, March 16, 13
  • 8.
    How many ofyou test what you build? (or have someone on your team who does) Saturday, March 16, 13
  • 9.
    How many ofyou primarily test manually? Saturday, March 16, 13
  • 10.
    How many ofyou primarily with automation? Saturday, March 16, 13
  • 11.
    How many ofyou test with a mix of both? Saturday, March 16, 13
  • 12.
    How many ofyou don’t test? Saturday, March 16, 13
  • 13.
    Show of handsif any of these are true for you: Save testing for the end of the cycle Feel like there isn’t enough time to adequately test Feel unnerved when it’s time to release Find issues in production Saturday, March 16, 13
  • 14.
    The Dirty LittleSecrets of Testing Saturday, March 16, 13
  • 15.
    #1: You don’thave to test everything Saturday, March 16, 13
  • 16.
  • 17.
  • 18.
    #2: Everything YouNeed to know about testing, you learned in Kindegarten. Saturday, March 16, 13
  • 19.
    Finding the GlitterGlue Path Saturday, March 16, 13
  • 20.
  • 21.
    Questions worth asking, to help you find your glitter glue path Saturday, March 16, 13
  • 22.
    How does yourbusiness* make make money/deliver value? * the business you are building software for Saturday, March 16, 13
  • 23.
    How do theusers actually use it? Saturday, March 16, 13
  • 24.
    What are theprominent/relevant traits of your users (e.g. browser types, geographic location, etc.)? Saturday, March 16, 13
  • 25.
    What things havebitten you before? Saturday, March 16, 13
  • 26.
    “An example wouldbe handy right about now” - Brian Marick Saturday, March 16, 13
  • 27.
  • 28.
    7 Automated AcceptanceTests* Successful platform upgrade Saturday, March 16, 13
  • 29.
  • 30.
  • 31.
    The Motley Fool To Educate, Amuse, and Enrich Saturday, March 16, 13
  • 32.
    These things cantake time. You have my permission to try until you get it right. Saturday, March 16, 13
  • 33.
  • 34.
    When To Automate? Questions worth asking Saturday, March 16, 13
  • 35.
    How long willit take to test manually? Saturday, March 16, 13
  • 36.
    How long willit take to test manually? This will help you get a baseline before answering the following questions. Saturday, March 16, 13
  • 37.
    How often willthis functionality need to be checked? Saturday, March 16, 13
  • 38.
    How often willthis functionality need to be checked? If often, then it is likely a good candidate for automation. Saturday, March 16, 13
  • 39.
    Is it valuable(e.g. core functionality, used heavily, a legal or political requirement)? Saturday, March 16, 13
  • 40.
    Is it valuable(e.g. core functionality, used heavily, a legal or political requirement)? If yes, then it is very likely a candidate for automation. Saturday, March 16, 13
  • 41.
    How risky isthe feature (e.g. fundamental changes, complex architecture, large user facing impact)? Saturday, March 16, 13
  • 42.
    How risky isthe feature (e.g. fundamental changes, complex architecture, large user facing impact)? High? Medium? Or Low? If Medium to High, then maybe. Saturday, March 16, 13
  • 43.
    What is thelevel of effort to automate? Saturday, March 16, 13
  • 44.
    What is thelevel of effort to automate? T-shirt sizing will help you here: Small: 0-2 hours Medium: 1-2 days Large: 2-4 days X-Large >= 1 week If large or higher, probably not But it really depends on what other work you have committed to Saturday, March 16, 13
  • 45.
    Does it involvelook and feel? Saturday, March 16, 13
  • 46.
    Does it involvelook and feel? If so, then it is likely a ripe candidate for manual testing. Saturday, March 16, 13
  • 47.
    How To Automate(strategy)? Saturday, March 16, 13
  • 48.
  • 49.
    Build a feedbackloop (Version Control are table stakes) Saturday, March 16, 13
  • 50.
    Version Control System(VCS) Saturday, March 16, 13
  • 51.
  • 52.
    How to Automate(tactics)? Saturday, March 16, 13
  • 53.
  • 54.
    Consider Your CurrentTech Stack Saturday, March 16, 13
  • 55.
    Then Layer InYour Team’s Context (strengths, comfort, confidence, excitement/interest) Saturday, March 16, 13
  • 56.
    Programming Language? Doesn’t have to be the same as what you develop in. Saturday, March 16, 13
  • 57.
    Who will ownthis? Saturday, March 16, 13
  • 58.
    Open Source TestingTools Saturday, March 16, 13
  • 59.
    Agile Alliance FunctionalTesting Tool group http://bit.ly/AgileTestTools Saturday, March 16, 13
  • 60.
    But let’s assumeUI is involved. In that case, something with WebDriver. Saturday, March 16, 13
  • 61.
    Full Disclosure: Ihave some opinions on this Released an open-source web testing framework for Selenium WebDriver http://arrgyle.com/blog/automated-web-testing-is-hard/ Saturday, March 16, 13
  • 62.
    World’s Quickest Overviewon how Selenium works Saturday, March 16, 13
  • 63.
    Selenium... is a robot sent from the future to help us test web sites uses locators (id, name, link, css, xpath, etc.) to interact with the html elements of the page (a.k.a ‘find by strategy’) ------ The simplest and most sustainable way is to use straight up CSS. Saturday, March 16, 13
  • 64.
    How to buildUI tests so you won’t hate yourself Saturday, March 16, 13
  • 65.
  • 66.
    Record & Playbackis merely a stepping stone Export your tests Treat them like code Page Object Pattern Saturday, March 16, 13
  • 67.
    He also says: Only use UI tests when you have to... Saturday, March 16, 13
  • 68.
    But won’t theybe slow? Saturday, March 16, 13
  • 69.
  • 70.
  • 71.
  • 72.
    Test Pack 1 Test Pack 2 VCS CI Simple checks Automated Builds, Acceptance Unit Tests Tests Local Dev Saturday, March 16, 13
  • 73.
    Garbage In, GarbageOut Saturday, March 16, 13
  • 74.
    A more testableUI Saturday, March 16, 13
  • 75.
    More semantic markup (e.g. unique and descriptive locators) Saturday, March 16, 13
  • 76.
    Inventory and consolidateyour design patterns Saturday, March 16, 13
  • 77.
    But really, it’sabout usability Saturday, March 16, 13
  • 78.
    “The more peoplethink about usability, the more things tend to work out. It's a virtuous path.” - Ken Pier, Senior Director of Product Quality, Socialtext Saturday, March 16, 13
  • 79.
    “It’s only wordsand photos on the internet.” - John F. Croston III, Web Content Accessibility Manager for the United States Army (army.mil) Saturday, March 16, 13
  • 80.
  • 81.
    Recap • You Don’t Have To Test Everything • But do test, intelligently, asking yourself • What matters? • Should I automate this? • Choose a toolchain that’s right for you and your team • Build a feedback loop • Build a more descriptive, usable, and testable web Saturday, March 16, 13
  • 82.
    I’m writing abook about this stuff! arrgyle.com/book @TourDeDave dave@arrgyle.com Saturday, March 16, 13
  • 83.
  • 84.
    • A practicalguide to usability: http://jfciii.com/presentations/wasp/ accessibility.html • How to address risk in Agile (view it like an investment portfolio): http:// arrgyle.com/blog/agile-accounting/ • Automated Web Testing Is Hard (Introducing ChemistryKit, an open source testing framework to make automated web testing easier): http://arrgyle.com/ blog/automated-web-testing-is-hard/ • CSS Selectors in Selenium: http://sauceio.com/index.php/2010/01/selenium- totw-css-selectors-in-selenium-demystified/ • Locators in Selenium http://marakana.com/bookshelf/selenium_tutorial/ locators.html Saturday, March 16, 13
  • 85.
    • Page Objectsin Selenium: https://code.google.com/p/selenium/wiki/ PageObjects • Page Objects Ruby Gem: https://github.com/cheezy/page-object Saturday, March 16, 13