Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Test Automation Architecture That Works by Bhupesh Dahal


Published on

Bhupesh Dahal

Published in: Technology
  • Be the first to comment

Test Automation Architecture That Works by Bhupesh Dahal

  1. 1. Test Automation Architecture One that works! Presented by Bhupesh Dahal
  2. 2. Who am I? & Why am I here?
  3. 3. Agenda • Background - Why test automation, common view of test automation? • Define Problems - Issues most organizations face with test automation. • Fixing the above issues with architecture that has the most ROI. • Review
  4. 4. Why Test Automation? Dev Team’s Perspective • Increase effectiveness, efficiency, and coverage of software testing. • For regression testing. • Improve quality. Boss’s Perspective • Reduce operational cost. • Increase revenue.
  5. 5. What comes to your mind when you hear the term “We have Automated Tests”?
  6. 6. Replicating Manual testing - GUI Testing Functional Test
  7. 7. Take the weight off of the manual testers
  8. 8. Focused on GUI testing • Hire consultants onshore/offshore with mantra “Automate everything”. • Qtp, Watir, Selenium Webdriver etc. use tools specific to GUI tests. • Every single small scenario gets added to GUI automation.
  9. 9. Test starts failing randomly
  10. 10. When you need the test to run - It never works. Please work!
  11. 11. What’s the problem eh? • Test execution is slow. • Tests are fragile. • Tests are not reusable. • Tests needs more infrastructure. • Test never works on my box but fails on daily run.  More time spent on figuring why the tests are failing.  Resource tied up just reviewing failures. Maintenance takes way too much time.
  12. 12. Goal of Automation Ends up being
  13. 13. Why do we end up in this situation?
  14. 14. Ignoring the fundamentals.
  15. 15. Golden Rule Testing different layers.
  16. 16. GUI Layer test • Test the whole application end to end. • Sensitive to UI changes. • Does not require good code. • Works for Legacy system and with newer software architecture like SOA as it is independent of backend technology. e.g. Open a browser, fill in the form and submit the data.
  17. 17. API/Service Layer test • Test the application logic by making service calls. • Faster than UI testing. • Requires service to exist. • E.g. Instantiate the calculator service and get it to add two numbers.
  18. 18. Unit level test • Test individual classes. • Much faster than service level testing. • Requires good design.
  19. 19. Application UI Tightly Coupled Automated UI tests Service LayerAutomated One Level Below UI DAO Layer Fragile Lag with Current Development Robust & Stable With Current Dev cycle Fast Execution UI Independent
  20. 20. So how to build your pyramid ?
  21. 21. Start with Unit Test (Quality is not just QA’s responsibility) • Isolation. • Test small piece of code - preferably separate test for each method. • Biggest test should be one test for a class. • Use mock services to remove dependency. • Test only public endpoints. • This forces to use good design.
  22. 22. Invest more in Service Layer Test • Independent functional test. • Use BDD tools like Cucumber, SpecFlow, Jbehave. • Use mocks but also have specific tests to check third party calls. • Add data validation tests. • Tests for boundary condition. • Tests for error handling - wrong data type, missing data. • Tests around calculations, business rules etc. Gherkin/Plain test DSL(Domain Specific Language) Set up projects/helpers, mock Framework Nunit,Junit,Rspec, xunit GUI Service Layer
  23. 23. Limit the Number of GUI test • Ask if it can be tested in Service layer before starting. • Start out with use and throw style test. • Gradually build the framework. • Use GUI test as an assistance for manual testing. • Create fewer tests but with broader end to end coverage.
  24. 24. Quick Guide to Building GUI Test Framework • Do basic meta - programming and start automation. Don’t design the entire framework right away. • Re - factor often. • Review test automation architecture regularly. • Run daily. Gherkin/Plain test Framework Watir/Selenium Webdriver/QTP WorkFlows Pages Navigation UI Utilities
  25. 25. GUI Service/Api Layer Unit Test Unit Test Business Logic Data access layer Mocked Data access layer Mocked Database Unit tests that isolate dependencies with fakes, mocks, stubs etc Database Can use just unit test framework like Nunit, Xunit prefer BDD style like Cucumber, specflow , Jbehave etc End to end UI test with tools like Watir, Selenium, QTP etc Automated UI Test UI Unit Test Integration Test Integration Test Integration Test Business Logic Business Logic Data access layer Data access layer Data access layer Mocked Database Database Database Data access layer Business Logic
  26. 26. Once you get the pyramid built
  27. 27. Review • Prioritize Unit and API/Service test automation. • Limit the number of GUI tests, have a fixed number of stable end-to-end tests. • It’s ok to use multiple technologies. • It’s ok to use Record/Play tools like Selenium IDE for throwaway tests. • It is better to have 50 stable test cases than 500 fragile ones that breaks regularly.
  28. 28. Hope you all catch tons of Bugs!!