Specifications test automation pyramid public

10,014 views
9,870 views

Published on

For enterprise projects, maintaining automation tests suite is always a challenge.
* Detailed walkthru of automation pyramid containing different types of specification driven tests layer.
* Framework built using RSpec, Spinach, Cucumber, webkit and Webdriver with examples
* Process of evolving and building specification tests
* Tips to maintain test suite and minimize test run durations

Published in: Technology

Specifications test automation pyramid public

  1. 1. SPECIFICATIONS – TESTAUTOMATION PYRAMID By Sathyan Sethumadhavan satnair@thoughtworks.com
  2. 2. Pre-Requisite:http://www.slideshare.net/satnair/final -version-for-public
  3. 3. Agenda Why specifications Approach to specifications Specifications pyramid Learnings Q&A
  4. 4. Keywords Specification – More than test – Feature/Requirement Cucumber – Software behavior in plain text Capybara – Tests application by simulating a real user Spinach - Encapsulation and modularity of your step definitions Headless webkit - A capybara driver - WebKit via QtWebKit. R-Spec - Test-Driven Development
  5. 5. Let’s talk about Why specifications Approach to specifications Specifications pyramid Learnings Q&A
  6. 6. Trying to solve these challenges? Defects leakage - missing Requirements Tests and requirements Frequent changes - documents becomes stale breaks existing functionalities Long Regression Spends more time on defect Cycles management and triage Less confidence - Existing test coverage
  7. 7. Trying to solve these challenges? Defects leakage - missing Requirements Less confidence - Existing test coverage
  8. 8. Let’s talk about Why specifications Approach to specifications Specifications pyramid Learnings Q&A
  9. 9. Specifications Maintaining Specifications Executing Specifications Implementing Specifications Evolving Specifications
  10. 10. Evolving Implementing Executing Maintaining Story• Intial Cukes creation Development • Manual testing for• Inputs from Business, each Story Technical and Testing • For TDD, writing • Evolved as part of• Tests from Unit to RSpec exploratory testing End-to-End • Checklist for story completion • Referred during dev- Story box Testing Creation
  11. 11. Evolving Implementing Executing Maintaining Story• Intial Cukes creation Development • Manual testing for• Inputs from Business, each Story Technical and Testing • For TDD, writing • Evolved as part of• Tests from Unit to RSpec exploratory testing End-to-End • Checklist for story completion • Referred during dev- Story box Testing Creation
  12. 12. Evolving Implementing Executing Maintaining Story• Intial Cukes creation Development • Manual testing for• Inputs from each Story Business, Technical • For TDD, writing • Evolved as part of and Testing RSpec exploratory testing• Tests from Unit to • Checklist for story End-to-End completion • Referred during dev- Story box Testing Creation
  13. 13. Let’s talk about Why specifications Approach to specifications Specifications pyramid Learning’s Q&A
  14. 14. Evolving Implementing Executing Maintaining Cucumber + Capybara WebdriverSpecificationPyramid Spinach + Headless Webkit RSpec + API
  15. 15. Evolving Implementing Executing Maintaining Algorithms / business logic validation API methods validation Non-UI RSpec + API Created as part of story development
  16. 16. Evolving Implementing Executing Maintaining Algorithms / business logic validation API methods validation Non-UI RSpec + API Created as part of story development
  17. 17. Evolving Implementing Executing Maintaining Presentation layer validations Story level validations Spinach + Jobs related functionalities Headless Webkit Created after story development RSpec + API complete (QA’s / Dev’s)
  18. 18. Evolving Implementing Executing Maintaining Presentation layer validations Story level validations Spinach + Jobs related functionalities Headless Webkit Created after story development RSpec + API complete (QA’s / Dev’s)
  19. 19. Evolving Implementing Executing Maintaining End user entry and exit scenarios Cucumber + Capybara Webdriver Story level UI validations Spinach + Headless End-To-End business scenarios Webkit Created after story development RSpec + API complete (QA’s / Dev’s)
  20. 20. Evolving Implementing Executing Maintaining End user entry and exit scenarios Cucumber + Capybara Webdriver Story level UI validations Spinach + Headless End-To-End business scenarios Webkit Created after story development RSpec + API complete (QA’s / Dev’s)
  21. 21. Evolving Implementing Executing Maintaining Cucumber + Developer/Tester Laptop CI pipeline Capybara • Smoke Tests • Smoke and E2E tests Webdriver • Less mocks Spinach + Headless Webkit Developer/Tester CI pipeline Laptop • Respective modules • After check-in • Before each check-in • Before smoke and RSpec + API • Mocks and Cassettes end-to-end
  22. 22. Evolving Implementing Executing MaintainingChange requests Identify specification layerNew requirements Modify existing specifications/if existsRegression/User acceptancedefects Create/Update/Delete as requirement evolves
  23. 23. Let’s talk about Why specifications Approach to specifications Specifications pyramid Learnings Q&A
  24. 24. Key learnings Specifications - Before development Fail fast – 95% of functionalities gets validated before check-in Specifications verified during dev-box (including multiple browsers) Categorize specifications - Appropriate layer Parallelize test runs Keep test code equivalent to developer’s code Executable specifications Defects backlog not beyond a threshold (for example 20 defects) Defects root-cause - Feedback to the team
  25. 25. Let’s Recap Defects leakage - missing Requirements Tests and requirementsFrequent changes - documents becomes stalebreaks existing functionalities Long Regression Spends more time on defect Cycles management and triage Less confidence - Existing test coverage
  26. 26. Tools Available • Jbehave, Cucumber, Cuke4Duke with Java different drivers (Selenium,Sahi) • Twist (Thoughtworks Studios) • Specflow, Cucumber, Cuke4Nuke, .NET Coypu with drivers like Selenium, White, WatiN
  27. 27. Q&A
  28. 28. How can we help? ThoughtWorks is a global custom software solutions consultancy trusted by many of the world’s leading businesses with their most complex and critical systems. We deliver consulting grounded in delivery expertise, build custom applications and help organisations across all market sectors to drive IT efficiency – working to an exceptionally high standard. Contact us satnair@thoughtworks.comPrivate and Confidential. © 2012 ThoughtWorks. All rights reserved.

×