Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Continuous test automation


Published on

Continuous Test Automation is a practice that encourages and aligns with the overall Continuous Delivery process. It is NOT only “Test Automation” but Test Automation done “continuously”.

Published in: Software
  • Be the first to comment

Continuous test automation

  1. 1. Test Automation Strategy ATDD, Continuous Delivery, Continuous Test Automation -Pradeep Macharla (
  2. 2. North Pole
  3. 3. Architecture Overview North Pole
  4. 4. North Pole At a very high level, ATDD is an industry practice that promotes collaboration amongst teams to define an acceptance criteria and a definition of DONE
  5. 5. ATDD – Acceptance Test Driven Development North Pole
  6. 6. People Strategic: • In Sourcing vs. Out Sourcing • Budgets (Staffing and Training) • Organizational model and support from leadership team • Culture (Alignment between business and Engineering) • Centralization vs. Decentralization • Fungibility Tactical: • Adopters and enthusiasts • Training and Workshops • Self service model
  7. 7. Process Strategic: • ATDD (Automation to start with) • Internal Controls vs. Auditability vs. Regulatory Compliance etc. • Incentives vs. Disincentives (process compliance) • Customer focused Continuous Delivery Tactical: • Adopters and enthusiasts • Training and Workshops to help • Self service model
  8. 8. On a side note • Conduct workshops on Test Automation • Test Automation function in TCOE - Enterprise • ATDD/TDD/BDD and associated technical practices • Agile Manifesto • Open source contributions • Attend and present in conferences • Vendor support (staffing and skill augmentation) • Technical coaches support
  9. 9. Technology - Automation Frameworks • Languages vs. libraries vs. design patterns • Data Driven vs. Keyword Driven vs. page-object • Test Data • Service Virtualization • Infrastructure Needs • In the cloud….
  10. 10. Technology – Selenium • Selenium Web UI Automation – Leader in the market at this point • Cross Browser Test Automation • Parallel Testing • Scalable Solution • Maintainable scripts with good Automation framework patterns • Support for Java, Ruby, Python, JS and c #... (Works best and most matured in terms of open source community for Java/Ruby/Python/JS) • Java/Ruby/Python/JS world opens up a world of opportunities for other kinds of automation and integration with tools (CI Server for example)
  11. 11. Technology – Automation Framework patterns • Data Driven Framework (few pages and many sets of data, same set of actions for each data set) • Keyword Driven (Blocks of business functionality grouped into Keywords making it more modular) • Page-Object (A web app is a network of pages interconnected, define classes for each web page with html elements as properties and methods if needed that operates on elements) • Hybrid (Most of the time we end up mixing the above with page-object being the dominant pattern)
  12. 12. Technology – Frameworks available • Cucumber/Behave – BDD + Data Parsers + Web Services API + Database + thick clients • Test NG – Similar to Junit, but X10 times more powerful in terms of options available • Pluggable libraries with the above frameworks • Excellent Jenkins/Bamboo/Hudson et al. integration and aligning with Continuous Test Automation • Much much more….
  13. 13. Technology – Test Data • Ingestion Scripts (inserts if using SQL) • Be able to clearly define expected data in a data structure • Be able to reset data (at least the fields that matter to testing) • Continuously be able to execute the same script over and over again
  14. 14. Technology – Test Scripts • CRUD (Create, Read, Update and Delete) • R – Mostly Smoke Testing/ Regression (CI pipeline for promoting build from DEV -> QA) • CUD – Destructive (Test Automation done extensively during QA phase) • Slice and Dice Tests – Excellent Support both from Java and Ruby • Exploratory testing (since Test Automation is WIP, however the goal is to be able to automate as much as we can)
  15. 15. Build – Measure - Learn • What to Measure ? Success Criteria (Ex. Defect prevention over test cases) • Lunch and Learns • Pair programming (Developers help to testers to start with) • Attending conferences and codeathon, hackathon etc. • Less Meetings and more consecutive blocks of time for Engineers
  16. 16. Dev - Ops
  17. 17. What Next ? • Groom Epics/Stories and put them in Backlog • Accountability vs. Responsibility • Horizontal impact (stories that impact multiple LOB’s) • Vertical impact (stories specific to LOB and drill down to projects) • Alignment with Dev Ops and Application Development teams. • Less Meetings and more consecutive blocks of time for Engineers 
  18. 18. References • Strategic Decision Making • Test Automation Frameworks • Continuous Test Automation and Jenkins