Recommended
More Related Content
What's hot
What's hot (7)
Similar to Get Max From Automation
Similar to Get Max From Automation (20)
More from BabuDevanandam
More from BabuDevanandam (15)
Recently uploaded
Recently uploaded (20)
Get Max From Automation
- 1. 2/1/2016 1 GET MAX FROM YOUR AUTOMATED TESTS Nitin Mukhija - Director Of Engineering In Test PayU India Pvt. Ltd.
- 3. 2/1/2016 3 Hopefully this will not happen, so lets get started…
- 4. Why Automate? • Manual is slow • Faster feedback on Continuous Integration – 24*7 testing • Repetitive manual regression tests are boring and make testers lose focus • Manual testers focus on testing new features • Allows for faster upgrades and refactoring • Consistency and Documentation • Higher test coverage – multiple data combinations 2/1/2016 4
- 5. Let’s see an example of poorly written automation… 2/1/2016 5
- 6. Automation Design • Example of a bad code Hardcoded Browser Hardcoded Environment Hardcoded Timeout 2/1/2016 6
- 7. Automation Design • Example of a bad code Hardcoded Object Identifiers Hardcoded Test Data Hardcoded Selenium Constructs Repeated Business Logic 2/1/2016 7
- 8. Automation Design • Non-Actionable Report – High Failure Investigation Cost Manually run again to repro. Can’t log bugs from report No Screenshot / Page HTML Insufficient Logging 2/1/2016 8
- 10. Let’s see the things you can do to get MAX from your automation… 2/1/2016 10
- 11. Tip #1:- Choose Right Design • Example – Layered Architecture • Hotel Book Web App 2/1/2016 11
- 12. Layered Architecture • Automation Libs • Framework Core • Page Objects • Business Logic Helpers • Test Cases 2/1/2016 12
- 13. Page Object Action On Page Mandatory to Return next Page after click UI Elements of Page Page Name 2/1/2016 13
- 14. Layered Architecture Sample layered design:- • Automation Libs • Framework Core • Page Objects • Business Logic Helpers • Test Cases 2/1/2016 14
- 15. Business Logic Helpers Combine Page Actions to build business Logic Login Search Book 2/1/2016 15
- 16. Layered Architecture • Automation Libs • Framework Core • Page Objects • Business Logic Helpers • Test Cases 2/1/2016 16
- 17. Neat Test Case Extensively use Business Logic Helpers Easy to read & interpret manual test case TC Description 2/1/2016 17
- 18. Layered Architecture • Automation Libs • Framework Core • Page Objects • Business Logic Helpers • Test Cases 2/1/2016 18
- 19. Automation Libs • Selenium, QTP, etc. • All external third party jars 2/1/2016 19 Framework Core • Wrapper around various external libraries Class Description Log Log reporting Helper Various utility methods DataBase Methods to interact with database Popup Methods to handle popups, alerts on the page TestData Reader Methods to read from test data sheet Browser Methods to do operations on your Browser/Page Element Methods to do operations on Elements of the Page
- 20. Tip #1:- Right Design- Benefits • Reduce Maintenance cost • Business logic at one place • Reusable code • Localized changes – Data and Object Repo • Easy to understand test cases • Easy to change underlying automation lib 2/1/2016 20
- 21. Tip #2:- Configurable • Don’t hardcode – Load at runtime – Environment URL’s – Browser Name – Platform – ObjectWaitTime – Database credentials – Any other param which drives the execution 2/1/2016 21
- 22. Tip #3:- Data Driven • Run same test case with different data combinations Read login credentials from Excel Test Data Excel Row Number 2/1/2016 22
- 23. Tip #4:- Parallel Execution • Write non-interfering atomic test cases • Standard Win7 can run 12 parallel browsers – execution time reduced to 1/12th Faster Execution = Faster Feedback 2/1/2016 23
- 24. 2/1/2016 24
- 25. Tip #5:- Rich Reports Hyperlinks of Page Screenshot & HTML Uniform logging of TC steps in Green/Red/Black logs Failure Page URL Log bugs by just reading logs, even manual tester can do! TC Description 2/1/2016 25
- 27. Custom Listeners • Logging of unhandled exceptions • Cleanup tasks • Soft Asserts 2/1/2016 27
- 28. Tip #6:- Customized email results • Host Test Results on a web server - Reports history • Email Results - Customized Subject Pass Percentage Area Owner 2/1/2016 28
- 30. 2/1/2016 30 Tip #7:- Measure Automation Accuracy # of test cases failed due to Dev code issue Vs # of test cases failed due to automation code issue
- 31. Tip #8:- Measure Code Coverage Sample PHP Code Coverage Report 2/1/2016 31
- 32. Tip #9:- Automated Deployments • Using Docker - Build production like, testing environment on the fly • Jenkins job to:- – Deploy application code automatically – Run Automation • A Step towards Continuous Delivery 2/1/2016 32
- 33. TestLink - TCM Test Cases Test Plans Test Builds Reports Execution Engine Fetch Git Code & Compile TestLink Integration Module Get Test Cases To Run Launch & Drive Execution Retrieve Results/Exceptions Push Test Results & Email (ReportNG) Layered Automation Suite Test Cases (TestNG) Business Logic Page Objects Framework Core Selenium Webdriver Tip #10:- Continuous Integration 2/1/2016 33
- 34. 2/1/2016 34 That’s strange, not even a single Oxygen mask dropped!!
Editor's Notes
- Lets understand why we should automate at all …… Now we understand why we should automate. But we see a lot of automation projects failing. Instead of solving these problems, they actually become a burden on the team. Why this happens??
- If you this bad code, you wont be able to:- Run your automation on diff browser, On diff env On slower env
- This is a poorly written test case. What if dev makes a change in element of the page, you will have to fix all test cases. What if you want to run the same TC with different data. You can’t What if you want to remove selenium and use xyz What if there is change in business logic - Change in one logic, will break all tests
- Moreover a poorly written automation framework will give you a non-actionable report Imagine if you have 2000 cases running daily, out of which 500 fail daily. You will end up spending days investigating such kind of reports - High failure investigation cost due to poor reporting
- Over a period of time you will be in a complete mess. Your tests will be quite brittle, and you will have high Maintenance cost.
- You should design your automation framework in a layered fashion, starting with the test cases which use…and lastly the automation libs.
- For any new feature automation, this will usually be the starting point, where you will add classes for all the new pages introduced in the feature. Now if developer changes any page, there will be only one place where you need to fix. You don’t need to make any change in test cases.
- Your application will have multiple pages, once you have created page objects for all of them, start combining them into business logic. These are reusable functions, which others can also use when they want to use this feature. Also, these will make your test case body neat and clean. The logical unit, way PM or manual tester will describe
- Extensively use reusable Business Logic Helpers, so that test case body looks short and neat. Just by reading the names of the helper methods being called, the objective of manual test case should become clear. And if business logic of lets say doing a booking changes, I don’t need any change in my test case.
- Wrapper around various external libraries and utility methods, so that writing automation code is easy. Talk only about Browser & Element
- Eg. Here it is reading login credentials from excel You just add more rows in excel, and your TC will run for those. No need to write code for every data input.
- Very important aim of automation is faster feedback, so it should execute fast.
- You must reduce the failure investigation time, when you have large number of tests.
- This is a very nice reporting solution by Allure, you can integrate your automation reports with this. Let’s say you have 2000 TC, out of which 500 fail. This will group your failure reasons and show you the top defects.
- - Also you can do cleanup work like – closing browser, closing db connection, recording screenshot, html, firebug– test case writer does not needs to worry about it Also – you can use softassert, and then assertall in listener. Test case will not stop in between.
- Reduce failure investigation time
- Instrument dev code and run automation It will capture the lines of dev code that are executed, and by which test case
- It has 3 major parts, TestLink where we keep manual cases, Execution engine to read those cases, and layered automation suite where all test cases are automated. Jenkins is a Continuous Integration tool, which we use to build & schedule automation. We can create job for the same.