The Odds of End to End Testing Sunil Kalangi Arun Subramanian Tameem Ahmed firstname.lastname@example.org
Overview• The Problem• The Solution • Break it Out of Monotonous, Flaky, Time Consuming approach • TestNG Listeners • Master Suites and Sub suites • Workspace Area• How To’s• Demo• Odds & Ends• Advantages• Continuous Integration• Concept – Unique Rows
The ProblemEnd to End Automation Test Cases was always and is one bigblock touching various components/layer in one single block.What if Component n-1 fails ? Start from start ? Doesn’t make sense ….What if you want to test just component 3 ? Not possible in an environment where you have 20 components.
The Problem cont.…What if the team testing component n-1 wants to do somevaluable testing meaning generating the data required prior tohitting component n-1 ? Not Possible as it is additional learning, implementation and maintenance of scripts.What if some other component is introduced before componentn-1 is introduced ? Requires you to change the whole structure of the test cases modify inputs and outputs etc.,
The SolutionBreak it up…..Let them live independently on their own for component testing.Design the logic to tie them together for an end to end experience.Now you have the ability to connect any component output to anycomponent.Ability to test independently.
ConceptsTestNG listeners: Listeners are events which are triggered witha test context. These are the classes you write, whichimplement some interfaces that TestNG provides, so that whenTestNG Raises certain events it will look for all “classes” whichare basically looking for such events and call the respectiveevent handler’s in them. Link here on TestNG Listeners.We don’t have to write the listeners again and again, write oncefor one layer and you would have the same format just thatwhat information you record will differ.
ConceptsWorspace: Temporary store for a build job, has all of the results/source code / build files, etc., which is basically refreshed forevery build you kick off.
Odds & EndsThis approach will be fruitful for an environmentwhich has multiple layers/teams.
Advantages• Now the entire test case logic is component based.• Manageable• Readable• Robust• Independent• Reliable even in cases when some components are not working or being worked on.• You can do whole lot with the data produced, share with PD other teams and save time.
Continuous IntegrationData Capture is a challenge when it comes to continuous integration as the buildjobs clear the workspace every build you kick off. The whole theme for us is topreserve the data and move forward with layer below.Things to do in order to preserve data in continuous integration environmentsJenkins etc.,Archive the artifacts *.xls
Unique RowsUsing the layered approach described above, we can benefit in time at the lower level layers.How ?When capturing the data for a layer mark it as a unique entry in the excel file itself and when you use dataprovider to load the data pick it up in a hash map which will give you unique set of rows on which the lowerlevel layers can run tests.Example:Validations doesn’t really need to happen on all of the test cases but the validations have to be thorough.Lets say you do 100 different test runs on Component 2 with all possible comminations. You will mark like 10 –15 of them as unique rows in the results file and when Component 3 has to run the test it will load 15 of theunique rows and start running the tests on these 15 rows only. Real time benefit : 80% of time saved.