You have 10000+ possibilities of executing a test with different combinations of values from its Test Parameters
You want 500 e-mail IDs to test something
You want to generate a 6 MB file with Alphanumeric characters as your test input
TestersDesk.com provides Computer Aided Software Testing through an innovative Toolkit of System-independent tools and utilities that improve Test Productivity in small and big ways It targets "Test Design Automation" and is complementary to "Test Execution Automation" tools currently in use Imagine
Let us say there are 3 test inputs A, B and C. Let us further say that A has 2 critical classes of input values, B has 3, and C has 4 - How do you build/visualize the table of decisions that show all the combinations (2*3*4 = 24) in multiple rows? Manually prepare it in Excel?
Let us say your test can be configured with 5 test parameters that can be any of 10 different values each, and you have a total of 10 5 i.e., 1 million ways to run the test – while you use experience to identify the key tests, how do you get Test Coverage with reasonable acceptance? How do you reduce 1 million to a manageable number?
Rich Internet Application – uses AJAX and interoperable data formats such as CSV/XML
Completely online – Web based tools for easier access (keeping the test machines free from maintaining several tools)
Unicode support (input can be in any language, and so will the output be)
Requests can be made through screens as well as file-uploads
Output displayed inside the browser (which can be copied) and through downloadable files
Description on each tool/feature in the toolkits
TestersDesk.com Online Community Edition |_ Test Design Toolkit |____ Pairwise TestCase Generator
The Situation Many test cases are influenced by multiple test parameters, each with several significantly different inputs (or equivalence classes). Typical problem arises when they have to be executed covering all possibilities/combinations.
An Example Think that a registration page has to be tested which has four fields- Username, Password, E-mail, Receive_notifcations. Let us say that Username can have two types of valid classes on inputs and three classes of invalid inputs. Similarly let us say that Password can have four valid classes three invalid classes of inputs. E-mail, let us say, can have four valid classes and two invalid classes on inputs. Receive_notifications can be Yes or No. Let us say that all these fields have to be tested in three browsers IE6, IE7 and FF3. Here we are talking about each input having some significance in the system behavior. To summarize, following is what we are looking at as the Test Parameters that drive the registration tests. Username: Valid1 , Valid2 , Invalid1 , Invalid2 , Invalid3 Password: Valid1 , Valid2 , Valid3 , Valid4 , Invalid1 , Invalid2 , Invalid3 E-mail: Valid1 , Valid2 , Valid3 , Valid4 , Invalid1 , Invalid2 Browsers: IE6 , IE7 , FF3 Receive_notifications: Yes , No If we were to test all the combinations of values of the above Test Parameters, we are talking about 5 * 7 * 6 * 3 * 2 = 1260 ways to configure test inputs. There is a clear need to reduce this number without compromising Test Coverage.
Benefit from TestersDesk.com Welcome to Pairwise Testing (if you are not already enjoying its virtues). We generate a very reduced number of Test Cases that cover all the two-way interactions between all the values of all the test parameters. Each value of each test parameter talks to each value of all other parameters at least once. Now, that's intelligent test design. The list produced by our tool, and the additional tests identified by your judgment/experience yield the maximum number of defects in the shortest possible time. For the example presented in the previous slide, we provide as little as 42 Test Cases that guarantee Pairwise Coverage. Check it out!
Pairwise TestCase Generator test combinations can be reduced Value to Software Testers
TestersDesk.com Online Community Edition |_ Test Design Toolkit |____ T-way TestCase Generator
Extending the previous problem, in certain rare cases, one might want to not only cover 2-way interactions but also target 3-way interactions to gain more confidence, i.e., all combinations of three values of all the test parameters interacting at least once in the produced list of Test Cases. Research has shown that 50% to 90% interaction errors are found by Pairwise TestCase Generator itself, but in cases where there is an absolute need, the coverage strength can be increased to 3 (or 4 or 5) -way combinations of the Test Parameters.
In testing of a mission-critical Marine Control System that controls crew on the ship, communication devices etc, it may be desired to increase the Interaction Coverage from 2 to 3. This, of course, is in addition to judgment/experience based tests.
Much like the Pairwise TestCase Generator, the Test Parameters and their respective values are collected to generate a reduced set of Test Cases that guarantee Interaction Coverage of 3, 4 or 5 -way combinations.
T-way TestCase Generator test combinations can be reduced with increased coverage Value to Software Testers
TestersDesk.com Online Community Edition |_ Test Design Toolkit |____ N-way/Random TestCase Generator
Test Coverage is a serious problem with the decisions to be made in executing a test. This has been conceptually solved by the Decision Table matrix formation technique that covers all the possibilities and lets testers mark the appropriate action for each decision set. The problem with this approach is that it tends to grow horizontally as the decisions and their options grow, making it infeasible to use it as part of the Test Case Documentation. Further more, manually prepared decision tables are tough to be fed into Test Automation tools like QTP, for example, and call for re-work.
Let us there is a business function "Calculate Monthly Premium" in a Health Insurance Application where there are three decisions that influence the outcome, namely, Person Age Group, Gender, and Health Index. Let us also say that Person Age Group can be one of G1, G2, G3 and G4. Gender can be male, female. And Health Index can be 1 to 10 (and each of them has a difference influence on the calculated premium).
Now let us say tests have to be prepared covering all the possibilities of sets of options chosen for the respective decisions. We are looking at 4 values of Person Age Group * 2 values of Gender * 10 values of Health Index, a total of 80 possibilities. It may not be very desirable to have 80 columns in a decision table.
Let us say that the following is given as input to TestersDesk.com's N-way TestCase Generator.
Person Age Group: G1, G2, G3, G4 Gender: male, female Health Index: 1,2,3,4,5,6,7,8,9,10
A vertical matrix is created, each row representing a set of possibilities. So we have just 3 columns and 80 rows. This data can be pasted in a spreadsheet, and a 4th column can be added for the "Expected Behavior". This way you have the Test Case Documentation done in an accelerated way. If a particular row has no significance, you can delete it. Try it out! Further more, such a table can be easily integrated into test automation scripts developed using data-driven approaches.
N-way/Random TestCase Generator decision tables for test coverage can easily be generated Value to Software Testers
TestersDesk.com Online Community Edition |_ Test Design Toolkit |____ N-way/Random TestCase Generator (with random option)
Often times, while choosing multi-input test values, the problem of selecting a subset of test items from a larger list arises. There is a need to see all possible subsets of a given size from the list, and then select the needed subsets from that.
A 7-day week company's payroll application works differently based on which of the 5 days in the calendar week a given employee has attended the office. So, we need to select all possible ways of taking 5 days from 7 days.
TestersDesk.com generates all the subsets from the larger set by taking two simple inputs- the complete list of possible test items, and the size of the subset (i.e., "how many at a time"). For the example discussed here, there are 21 ways how 5 items can be picked out of the 7 test items.
Subset TestCase Generator all subsets of test inputs from a larger set can be generated Value to Software Testers
TestersDesk.com Online Community Edition |_ Test Design Toolkit |____ Boundary TestCase Generator (BVA++)
General theory of Boundary Value Analysis talks about testing the boundaries of a single test field, and when there are multiple range/list based fields, there is a need for a method to cover the interactions between their boundaries (instead of testing boundaries in isolation).
An Example There are three fields on a screen each of which take a numeric range. Each individual field has been tested with the seven tuples (values) viz. min-1, min, min+1, mid, max-1, max and max+1. The expected behavior was achieved, however there is no guarantee that the "interactions" between the seven tuples of all the fields together work as expected. What if there is an error that surfaces only when the min+1 value of field #1 and max - 1 of field #2 occur together in a single test - does our test suite has test cases to catch such defects?
By collecting range/list information about test fields, this tool applies pairwise coverage and a few other techniques to generate a list of test cases that cover the interactions between the boundaries (for example, the minimum value of field #1 interacts with maximum value of field #3 at least in one of the generated tests and so on).
Boundary TestCase Generator (BVA++) multi-field boundary value analysis tests can be generated Value to Software Testers
TestersDesk.com Online Community Edition |_ Test Design Toolkit |____ TestCase Permutations Generator
Change in order of actions can lead to change in behavior. It is a usually agreed fact in the way how software applications work. Testers have the need to change the order of key test actions to see if they cohesively work well-- some call them as Scenario Tests, Test Stories, Act-like-a-customer tests and so on... But in what ways can the order of 4 tests be changed – can we see a list of possibilities and select what we want?
An Example Let us say that a banking application works perfectly in showing the balance money when the test actions are done in the following sequence- Change Password-->Check Balance--->Transfer Funds Let us hypothetically say there is a conflict on the data updating process when balance is checked after changing the password, then the following sequence may not work- Transfer Funds--->Change Password-->Check Balance
You can provide the 3 or 4 critical test actions and get a list of all possible permutations (changes in order) of test execution. This does not necessarily formulate the actual Scenario Tests but at least helps you in visualizing what are the possible orders that the key test actions can take, and lets you pickup the interesting rows.
TestCase Permutations Generator permutations (changed orders) of critical tests can be generated Value to Software Testers
TestersDesk.com Online Community Edition |_ Test Data Generation Toolkit |____ Common Test Data Generators |____ Language String Generator
In the daily life of a typical Software Tester, different test fields are needed be tested for length requirements. Preparing these test strings with the specified lengths, determining whether they are accepted completely or got truncated at a wrong position, and other such related work is, by large done manually, and eats away a lot of during test execution (especially since there is no repeatable technique of preparing strings with specified lengths). Further more, the strings we use to test the length of a field are not self-descriptive and need to be again pasted back in a text editor to analyze where it got broke (if I type a 100 A’s and only 99 got pasted in my field, I cannot know that unless I again copy the value from the field into an editor).
A field should not take 30 characters. The tester types 5 times the letter 'a' in a text editor and pastes it six times. It is then pasted into the field, and now copied back into the editor to see all the characters are pasted or not. Then the same procedure is repeated for an invalid value like 31 characters and so on. Some testers use marker characters in end but the preparation process still remains big.
TestersDesk.com generates what it calls as Quantified Strings - Strings that can indicate the position of a character anywhere within. For the previous example, such a string will look like "$3$5$7$9$12$15$18$21$24$27$30$" - each number in the string specifies the position of the next $ character. Suppose let us say when this is pasted, we see only "$3$5$7$9$12$15$18$21$24$27$30", it is of immediate observation that it got truncated at 29, a typical boundary bug. 30 is saying that the $ that follows it is the 30th $, but we do not see it- concluding that it broke at 29. You can change the $ character to any other character you want.
Quantified String Generator quantified strings can be generated to simplify testing field lengths Value to Software Testers
TestersDesk.com Online Community Edition |_ Test Data Generation Toolkit |____ Template-based File Generator
The Situation The Application Under Test takes a file as test input. This file has several fields in a specified format. There is a need to prepare multiple files each having different test data. In the end you have, let us say, a large number of files with the same format and only data varies in each of them. In the next new build, let us say, the format has been changed or is added with place for a new field. This calls for individual opening and editing of each such file. Note: EDI, WebServices, XML, Configuration files... there are many cases where the test input is a file with a specified format.
By taking two input files- One a Template file that shows the format of your test input file that has placeholders where data needs to be embedded, and the other with several rows of test data, a single ZIP file is generated that has as many files as the number of rows in the CSV Test Data file provided. Each individual file represents each individual row of test data. The advantage is that many files can be generated in one-go, and editing and other maintenance work is done on just two files.
Template-based File Generator multiple test input files can be created with just one template and one data file Value to Software Testers
TestersDesk.com Online Community Edition |_ Test Data Generation Toolkit |____ Size-based File Generator
By providing three inputs- File size, Measurement unit (KB, MB, GB) and Type of data (Alphabets, Binary etc) a tiny batch file can be downloaded. This batch file creates the required file when executed on the test machine, in the same directory where it is executed from. After test execution if completed the generated file can be deleted to save space, and can be re-created by re-executing the batch file whenever needed. Currently this feature works on Windows.
Size-based File Generator files with the specified data and size can be generated Value to Software Testers
TestersDesk.com Online Community Edition |_ Test Data Generation Toolkit |____ Date/Time (Timestamp) Generator
It is a common test input requirement to provide a date/time value to an Application Under Test. Sometimes, several values may be needed instead of just one. It becomes tedious to prepare them manually.
By collecting some inclusion/exclusion options and length requirements, a set of valid and a set of values are generated that can be used as test inputs in testing the requirements of a password field. We are adding more options soon.
Password Test Generator test inputs to test password fields can be generated Value to Software Testers
TestersDesk.com Online Community Edition |_ Test Data Generation Toolkit |____ Luhn (Mod-10) Number Generator
A variety of identification numbers, such as credit card numbers and Canadian Social Insurance numbers need to comply with Luhn formula, also known as mod-10 check. Testers need test inputs covering both valid as well as invalid Luhn (mod-10) values to exercise the Application Under Test.
Let us say that a test system needs certain test inputs in a screen, and one of the field has to be globally unique, meaning it should have never been used before any time on any machine, you would need a tool that helps generate such values.
By taking a few options like whether you want only E-mail IDs ending with "dot com" or "all types", whether the generated random values should be new or from a previous seed, different download-ready E-mail IDs are generated.
E-Mail Address Generator test e-mail addresses can be generated Value to Software Testers
TestersDesk.com Online Community Edition |_ Miscellaneous Testing Toolkit |____ SQL Injection Test Generator
Malicious test inputs can be crafted to trick the SQL logic running in applications and compromise Application Security (especially when the user supplied inputs are consumed in building SQL). SQL Injection has stood out as the most popular vulnerability, generally in multi-user applications like Web Applications. It has become mandatory to test the applications for such vulnerabilities, at least at the entry-level pre-login screens. But to perform such tests you either need expensive tools or really tech-savvy testers on board.
A web application provides activation licenses to a game by taking an order ID. Traditional tests include testing it with valid order IDs, empty fields, lengthy order IDs, and so on... All such tests, let us say, have yielded expected results. There is a chance that it has one major test uncovered - that is whether or not SQL can be injected into the field to trick the server-side logic and compromise Application Security.
TestersDesk.com collects the basic information about a Web Form (its action URL, field names with some dummy values, and the target_field on which you want to perform the SQL injection) and then crafts automated forms with different types of SQL injections, and presents a single downloadable HTML file. Once downloaded, it can be executed on your application and the content within the page may reveal potential security holes in the Application Under Test.
SQL Injection Test Generator SQL injection security tests on web forms can be automated Value to Software Testers
TestersDesk.com Online Community Edition |_ Miscellaneous Testing Toolkit |____ OS Information Scripts
In several instances, it is required to gather OS level information before or after test execution on a particular test machine. This information could be related to the filesystem, software, processes, hardware and so on.
TestersDesk.com provides several simple VB Script files that can be downloaded and executed from the command line. All of them produce readable output that can be copied into outgoing test reports, bug reports etc. In future scripts for other Operating Systems will be available too but currently they are available for Windows.
OS Information Scripts system-dependent information scripts can be downloaded (for windows) Value to Software Testers