Software Testing

Software testing
➢ One of the most important phase in software life cycle
➢ Required to point out defects and errors that have made during the development
➢ Make sure customers satisfaction and reliability of the product
➢ Guarantees that the product/solution fulfills all the required functional and
nonfunctional requirements
➢ Decides the stability and quality of the product/solution
➢ Necessary while delivering the product to makes sure it requires lower
maintenance cost and it produces more accurate, consistent and reliable results.

Developers Belief on Software Testing
Developers believe that responsibility of the testing completely
holds the QA team
And,
➢ their responsibility is to give a functional bundle to the QA team which covers all
the happy day scenarios
➢ if there any issues assigned to them, fix those issues after the QA round and then
go for another QA round
➢ implementing a new functionality to the product is more important than
spending their time on testing the implemented features

Do you agree with that belief?
Absolutely not…. why?
➢ Developers and QAs have two different views about the product
➢ They are testing the functionalities of the product/solution based on their views
➢ Issues which can be found by developers, will not be found by QAs. Other way
around is also true
➢ Both teams should contribute to improve the test coverage of the
product/solution

Developers Responsibility for testing
➢ Should write unit test cases (and integration tests also if possible) to cover all the
important functionalities of the product
➢ Test cases should cover almost all the paths in the product
➢ As developers we do have a clear idea about the most of the error paths of the
system than the QA team because they will check error paths based on the user
experience and user inputs, but we can test the error paths based on the
implementation
➢ Test cases should be extensible if it needs to improve/extend at the QA cycle
➢ Developers should be able to be confident that the product is fully functional in
their point of view (Still QAs will be able to find issues which is ok, since
developers tried their best to make the product is reliable)

Test writing methods
There are two types of test writing approaches,
1. State based testing
2. Behavioural/interaction based testing

State Based Testing...
verifying that the system under the test(SUT) returns correct
results. In other words code under the test ended up at the
expected state with respect to the given input values
Ex : testSort () {
Sorter sorter = new Sorter(quicksort, bubbleSort);
//Verifying the input number list is sorted regardless of the sorting algorithm
//which used to sort the list
assertEquals(new ArrayList(1, 2, 3), sorter.sort(new ArrayList(3, 1, 2)));
}

Behavioural/Interaction Based Testing
verifying that the SUT goes through the expected flow, it calls
expected methods required number of times and with expected
arguments etc
Ex : public void testSortQuicksortIsUsed() {
// Passing the mocks to the class and call the method under test.
Sorter sorter = new Sorter(mockQuicksort, mockBubbleSort);
sorter.sort(new ArrayList(3, 1, 2));
// Verify that sorter.sort() used Quicksort. The test should
// fail if mockQuicksort.sort() is never called or if it's called with the
// wrong arguments (e.g. if mockBubbleSort is used to sort the numbers).
verify(mockQuicksort, times(1)).sort(new ArrayList(3, 1, 2));
}

There is another important
responsibility for developers

What is it?
Writing a Testable Code

Writing a Testable Code
Why is this important???
➢ Writing test cases should not be an overhead
➢ Test writer shouldn’t have to spend much time on setting up the environment for
testing
→ Focus only about the SUT(system under the test)
→ Setting up required collaborators should be easy and straightforward
→ Should be able to easily identify all the dependencies for the test, by looking
at the unit of code(API) that you are going to test
There are some Anti-patterns(flaws) that you should be aware of to write a testable
code

Flaw 1
Constructor does Real Work

Code in the constructor should be simple and straightforward. It
should not,
➢ Create/Initialize collaborators
➢ Communicate with other services
➢ Have the logic to set up its own state

How to identify this flaw…
➢ “new” keyword in a constructor or at field declaration
➢ “static” method calls in a constructor or at field declaration
➢ Anything more than field assignment in constructors
➢ Object not fully initialized after the constructor finishes (watch out for initialize
methods)
➢ Control flow (conditional or looping logic) in a constructor
➢ CL does complex object graph construction inside a constructor rather than using
a factory or builder
➢ Adding or using an initialization block

Why this is not a good practice?
➢ Difficult to test directly
➢ Forces collaborators on test writer
➢ Violates the Single Responsibility Principle

Work around??
➢ Subclassing and overriding
✓ Delegates collaborator creation logic to a method which is expected to
overriden in test sub classes
✓ This may work around the problem, but you will fail to test the method
which is overridden and it does lots of work
✓ This work around can keep the last option if you really couldn't get rid of
collaborator creation at the constructor of the SUT

Work around??
➢ Keep separate constructor for testing
✓ This is also same as the previous work around
✓ You are going to miss the first few steps in the path which is expected to test.
✓ Later changes in the actual constructor will not be captured from your test
cases

How to fix/avoid this flaw?
Do not create collaborators, pass them into the constructor
Don't look for things! Ask for things!
➢ Move the object graph construction responsibility to some other object like a
builder or factory object and pass these collaborators to your constructor
➢ Use DI to pass relevant implementation of collaborators for the construction of
SUT

Examples of the flaw and fixed version of them
Example 1:
SUT
class AccountView {
User user;
AccountView() {
user = PCClient.getInstance()
.getUser();
}
}
Test
class ACcountViewTest extends TestCase {
public void testUnfortunatelyWithRealRPC() {
AccountView view = new AccountView();
// Shucks! We just had to connect to a real
//RPCClient. This test is now slow.
}

Example 1:
SUT
class AccountView {
User user;
@Inject
AccountView(User user) {
this.user = user;
}
}
User getUser(RPCClient rpcClient) {
return rpcClient.getUser();
}
RPCClient getRPCClient() {
return new RPCClient();
}
Test
class AccountViewTest extends TestCase {
public void testLightweightAndFlexible() {
User user = new DummyUser();
AccountView view = new AccountView(user);
// Easy to test and fast with test-double user.
}
}

Example 2:
SUT
class Car {
Engine engine;
Car(File file) {
String model = readEngineModel(file);
engine = new EngineFactory()
.create(model);
}
}
Test
class CarTest extends TestCase {
public void testNoSeamForFakeEngine() {
// Aggh! I hate using files in unit tests
File file = new File("engine.config");
Car car = new Car(file);
// I want to test with a fake engine but I can't since the
//EngineFactory only knows how to make real engines.
}
}

Example 2:
SUT
class Car {
Engine engine;
Car(Engine engine) {
this.engine = engine;
}
}
// Have a provider in the Module to give you the
//Engine
Engine getEngine(EngineFactory engineFactory,
String model) {
return engineFactory.create(model);
}
Test
// Now we can see a flexible, injectible design
class CarTest extends TestCase {
public void testShowsWeHaveCleanDesign() {
Engine fakeEngine = new FakeEngine();
Car car = new Car(fakeEngine);
// Now testing is easy, with the car taking exactly what
//it needs.
}
}

Flaw 2
API lies about it's real
dependencies

API lies about it's real dependencies
➢ Your API lies about it’s real dependencies, if it
✓ uses context or holder objects in method signatures/APIs
✓ pass the the context objects as parameters instead of passing specific
property/object in the context object
✓ look for the object of interest by walking through the object graph
➢ If you are going through the object graph more than once,
then you are looking right at an example of this flaw

API lies about it's real dependencies
How to identify this flaw
➢ Objects are passed in but never used directly (only used to get access to other
objects)
➢ Method call chain walks an object graph with more than one dot (.)
➢ Suspicious names: context, environment, principal, container, or manager in
method signatures/APIs
➢ Have to create mocks that return mocks in tests
➢ Difficult to write tests, due to complex fixture setup

➢ Deceitful APIs
➢ Makes your code brittle and reduce the understandability
➢ Makes your code difficult to test

Instead of looking for things, simply ask for the objects that you need
Don't look for things! Ask for things!
➢ By asking required objects you will delegate the object creation responsibility to
the factory which is supposed to create objects and not to do any other things
➢ So you should,
✓ Only talk to your immediate friends.
✓ Inject (pass in) the more specific object that you really need.
✓ Leave the object location and configuration responsibility to the caller

Example 1:
SUT
class SalesTaxCalculator {
TaxTable taxTable;
SalesTaxCalculator(TaxTable taxTable) {
this.taxTable = taxTable;
}
float computeSalesTax(User user, Invoice invoice)
{
// note that "user" is never used directly
Address address = user.getAddress();
float amount = invoice.getSubTotal();
return amount*taxTable.getTaxRate(address);
}
}
Test
class SalesTaxCalculatorTest extends TestCase {
SalesTaxCalculator calc = new
SalesTaxCalculator(new TaxTable());
Address address = new Address("1600 Amphitheatre
Parkway...");
User user = new User(address);
Invoice invoice = new Invoice(1, new
ProductX(95.00));
// So much work wiring together all the objects
//needed
//….
assertEquals(0.09, calc.computeSalesTax(user,
invoice), 0.05);
}

Example 1:
SUT
class SalesTaxCalculator {
TaxTable taxTable;
SalesTaxCalculator(TaxTable taxTable) {
this.taxTable = taxTable;
}
// Note that we no longer use User, nor do we
// dig inside the address. (Note: We would
// use a Money, BigDecimal, etc. in reality).
float computeSalesTax(Address address, float
amount) {
return amount *
taxTable.getTaxRate(address);
}
}
Test
class SalesTaxCalculatorTest extends TestCase {
SalesTaxCalculator calc = new
SalesTaxCalculator(new TaxTable());
// Only wire together the objects that are needed
Address address = new Address("1600 Amphitheatre
Parkway...");
// …
assertEquals(0.09, calc.computeSalesTax(address,
95.00), 0.05);
}
}

Example 2:
SUT
class LoginPage {
RPCClient client;
HttpRequest request;
LoginPage(RPCClient client,
HttpServletRequest request) {
this.client = client;
this.request = request;
}
boolean login() {
String cookie = request.getCookie();
return client.getAuthenticator()
.authenticate(cookie);
}
}
Test
class LoginPageTest extends TestCase {
public void testTooComplicatedThanItNeedsToBe() {
Authenticator authenticator = new FakeAuthenticator();
IMocksControl control = EasyMock.createControl();
RPCClient client = control.createMock(RPCClient.class);
EasyMock.expect(client.getAuthenticator()).andReturn(authentica
or);
HttpServletRequest request =
control.createMock(HttpServletRequest.class);
Cookie[] cookies = new Cookie[]{new Cookie("g", "xyz123")};
EasyMock.expect(request.getCookies()) .andReturn(cookies);
control.replay();
LoginPage page = new LoginPage(client, request);
// …
assertTrue(page.login());

Example 2:
SUT
class LoginPage {
LoginPage(String cookie,
Authenticator authenticator) {
this.cookie = cookie;
this.authenticator = authenticator;
}
boolean login() {
return authenticator.authenticate(cookie);
}
}
Test
class LoginPageTest extends TestCase {
public void testMuchEasier() {
Cookie cookie = new Cookie("g", "xyz123");
Authenticator authenticator = new
FakeAuthenticator();
LoginPage page = new LoginPage(cookie,
authenticator);
// …
assertTrue(page.login());
}

Flaw 3
Brittle Global State &
Singletons

Brittle Global State & Singletons
➢ Global State and Singletons make APIs lie about their true
dependencies
➢ developers must read every line of code to understand the
real dependencies
➢ Spooky Action at a Distance
✓ when running test suites, global state mutated in one test
can cause a subsequent or parallel test to fail
unexpectedly

Brittle Global State & Singletons
How to identify this flaw…
➢ Adding or using singletons
➢ Adding or using static fields or static methods
➢ Adding or using static initialization blocks
➢ Tests fail if you change the order of execution
➢ Tests fail when run in a suite, but pass individually or vice versa

Global State Dirties your Design
➢ Global state is that it is globally accessible
➢ An object should be able to interact only with other objects which were directly
passed into it. But this practice violates it.
✓ if I instantiate two objects A and B, and I never pass a reference from A to B,
then neither A nor B can get hold of the other or modify the other’s state.
This is a very desirable property of code
✓ However, object A could(unknown to developers) get hold of singleton C
and modify it. If, when object B gets instantiated, it too grabs singleton C,
then A and B can affect each other through C.

Global State enables Spooky Action at a Distance
➢ When we run one thing that we believe is isolated (since we did not pass any
references in), there will be unexpected interactions and state changes happen in
distant locations of the system which we did not tell the object about
➢ This can only happen via global state
➢ Whenever you use static state, you’re creating secret communication channels and
not making them clear in the API
➢ Spooky Action at a Distance forces developers to read every line of code to
understand the potential interactions, lowers developer productivity, and confuses
new team members

➢ If you need a collaborator, use Dependency Injection
➢ If you need shared state, use DI framework which can manage Application Scope
singletons in a way that is still entirely testable
➢ If you’re stuck with a library class’ static methods, wrap it in an object that
implements an interface. Pass in the object where it is needed. You can stub the
interface for testing, and cut out the static dependency
Dependency Injection is your Friend

Example 1:
SUT
class LoginService {
private static LoginService instance;
private LoginService() {};
static LoginService getInstance() {
if (instance == null) {
instance = new RealLoginService();
}
return instance;}
static setForTest(LoginService testDouble) {
instance = testDouble;
}
static resetForTest() {
instance = null;}
SUT
// Elsewhere...
//A method uses the singleton
class AdminDashboard {
//…
boolean isAuthenticatedAdminUser(User user) {
LoginService loginService =
LoginService.getInstance();
return loginService.isAuthenticatedAdmin(user);
}
}

Example 1:
Test
// Trying to write a test is painful!
class AdminDashboardTest extends TestCase {
public void testForcedToUseRealLoginService() {
// …
assertTrue(adminDashboard.isAuthenticatedAdminUser(user));
// Arghh! Because of the Singleton, this is
// forced to use the RealLoginService()
}

Example 1:
SUT
//Dependency Injected into where it is needed,
//making tests very easy to create and run.
class LoginService {
// removed the static instance
// removed the private constructor
// removed the static getInstance()
// ... keep the rest
}
// Use dependency injection to inject required
//implementation of the login service
// eg: bind(LoginService.class).to(RealLoginService.class)
.in(Scopes.SINGLETON);
SUT
// Elsewhere...
// Where the single instance is needed
class AdminDashboard {
LoginService loginService;
// This is all we need to do, and the right
// LoginService is injected.
AdminDashboard(LoginService loginService) {
this.loginService = loginService;
}
boolean isAuthenticatedAdminUser(User user) {
return
loginService.isAuthenticatedAdmin(user);
}
}

Example 1:
Test
// With DI, the test is now easy to write.
class AdminDashboardTest extends TestCase {
public void testUsingMockLoginService() {
// Testing is now easy, we just pass in a test-
// double LoginService in the constructor.
AdminDashboard dashboard =
new AdminDashboard(new MockLoginService());
// ... now all tests will be small and fast
}
}

These are very simple facts. But if you don’t focus about the
testing when you write the code, definitely you will do at least
one of the above mistakes accidentally.
You may have your own list of guidelines to follow to write a
quality code. Make a note about these points and add them to
your list if you really need to write a testable code

Testing Frameworks and tools for Java...
✓ JUnit
✓ TestNG
✓ Mockito
✓ PowerMock
✓ Arquillian
✓ The Grinder
✓ JTest

Mockito and PowerMock...
In Real-Life Units are NOT Totally Self Contained
✓ Dependencies – In order for a function to run it often requires Files, DB, JNDI or
generally – other units
✓ Side effects – When a function runs we are often interested in data written to
files, saved to db or generally - passed to other units. So how can we test a single
unit, without being affected by other units
How can we test a single unit, without being affected by other units – their errors,
set-up complexities and performance issues?

You have two options….
Stubs
or
Mocked Objects

Testing Models
There are two type of testing models for unit testing,
1. Stub based testing model
➢ Stubs provide pre-programmed sample values to calls made to it during the test. They may record
the information about the calls such as number of search queries, number of success queries etc.
They are not responding anything outside what's programmed in for the test
2. Mock based testing model
➢ Mocks are preprogrammed objects which can be trained according to the requirement of the test.
The trained behaviour can be changed according to your requirement and same mock object can be
trained to behave in two different ways in two different test cases

Stubs Based Testing Model
Stubs provide pre-programmed sample values to calls made to it during the test. They
may record the information about the calls such as number of search queries, number
of success queries etc. They are not responding anything outside what's programmed
in for the test

Stubs Based Testing Model - eg:
public interface ConnectivityEngineFacade {
String search(String priceRequest);
}

Stubs Based Testing Model - eg: cntd...
public class ConnectivityEngineFacadeMock implements ConnectivityEngineFacade {
private String ceStubFile;
@Override
public String search(String priceRequest) {
return readFile();
}
private String readFile() {
//reading the content of the pre configured file(ceStubFile) and return the content
as string
}

Mocked Objects Based Testing Model
Mocks are preprogrammed objects which can be trained according to the requirement
of the test. The trained behaviour can be changed according to your requirement and
same mock object can be trained to behave in two different ways in two different test
cases

Mocked Objects Based Testing Model - eg:
@RunWith(PowerMockRunner.class)
@PowerMockIgnore("javax.management.*")
@PrepareForTest({ Util.class, XMLAdapterContext.class}) //classes to be used within
the testing process
public class ConnectivityEngineTest {
}

Mocked Objects Based Testing Model - eg:
@Mock
private ConnectivityEngineFacade connectivityEngineFacadeMock;
@Before
public void setUp() throws Exception {
MockitoAnnotations.initMocks(this);
....
when(connectivityEngineFacadeMock.search(Mockito.anyString())).thenReturn("This is the
sample value to be returned within the whole test class");
}
@Test
public void validateConnectivityEnginePath() throws Exception {
when(connectivityEngineFacadeMock.search(Mockito.anyString())).thenReturn("This is the
sample value to be returned within the validateConnectivityEnginePath test case");
xmlAdaptorRequestSequence.execute(msg, mediation); //execute the required sequence to test
…

Mockito and PowerMock are mocking framework that will help
you to create mock object to replace your collaborators and
make your testing process easier
✓ Help you to focus on the SUT
✓ Give ability verify behavioral/interaction based testing
✓ Testing result of SUT doesn’t depend on collaborator's
issues and performance
✓ Low setup costs

Be Careful while using mocking frameworks, if you are using mocked objects
unnecessarily then you will face some other problems such as slowing down your test
suites, extremely complicated set up code etc.
Read more :
http://googletesting.blogspot.com/2013/05/testing-on-toilet-dont-overuse-mocks.html
https://blog.8thlight.com/uncle-bob/2014/05/10/WhenToMock.html

JUit 4.+ and TestNG
Feature TestNG JUnit
Run test method @Test @Test
Run method before first method
in class
@BeforeClass @BeforeClass
Run method after last method in
class
@AfterClass @AfterClass
Run method before each method
in class
@BeforeMethod @Before
Run method after each method in
class
@AfterMethod @After
Ignore a test method @Ignore @Ignore

JUnit 4.+ and TestNG
Feature TestNG JUnit
Expected exception @Test(expected=Exception.class ) @Test(expected=Exception.class)
Timeout @Test(timeout=100) @Test(timeout=100)
in suite
@BeforeSuite You have to create a suite class
and add @BeforeClass to it
suite
@AfterSuite You have to create a suite class
and add @AfterClass to it
in group
@BeforeGroup JUnit has similar concept of
Category which can be included
or excluded in a suite
group
@AfterGroup JUnit has similar concept of
Category which can be included
or excluded in a suite

JUnit 4.+ vs TestNG
Ordering…
Ordering is needed when some test cases are supposed to be run
before/after other test cases
eg: running loginTest before updateNameTest or updateEmailTest
TestNG - test methods can be ordered via dependsOnMethods
@Test(dependsOnMethods={“login”})
JUnit - Doesn’t have straightforward ordering mechanism. Can be ordered
only by the test method names’ alphabetical order
@FixMethodOrder(MethodSorters.NAME_ASCENDING)

JUnit 4.+ vs TestNG
Test Suites…
You might want to,
✓ Create a suite of unit tests and another suite of integration tests and run these two
suites separately
➢ Both TestNG and JUnit have support for creating test suites
➢ TestNG - you can easily do that in a simple xml file
➢ JUnit - you have to create a class and add @RunWith, @SuiteClasses etc do
the same thing
➢ Suites also allow you to run specific types of test methods, TestNG calls them
as group and JUnit calls them as category.

JUnit 4.+ vs TestNG
Test Suites…
Test Suite example in with TestNG
<suite name=“unit tests” >
<test name=“manager tests”>
<groups>
<run><include name=“slow”/></run>
</groups>
<classes><class name=“com.manager”/><classes>
</test>
</suite>

JUnit 4.+ vs TestNG
Test Suites…
Test Suite example with JUnit4
@RunWith(Suite.class)@Suite.SuiteClasses({AdminManager.class,
UserManager.class})
@IncludeCategory(SlowTest.class)
public class UnitTestSuite {
// the class remains empty, used only as a holder for the above
// annotations
// add @BeforeClass or @AfterClass to run methods before/after first/last
// method in this suite
}

JUnit 4.+ vs TestNG
Data providers and parameterised tests…
JUnit4…
✓ provides @Parameters that can be added to a static method that contains data,
this method is called while instantiating the test class.
➢ return value is passed to the test class as an argument,
➢ test class constructor has to accept arguments.
➢ JUnit also provides @Parameter that can be added to any member field to set
the data directly to the field instead of constructor.

JUnit 4.+ - Parameterized Test Example
@RunWith(value = Parameterized.class)
public class JunitTest {
private int number;
public JunitTest(int number) {
this.number = number;
}
@Parameters
public static Collection<Object[]> data() {
Object[][] data = new Object[][] { { 1 }, { 2 }, { 3 }, { 4 } };
return Arrays.asList(data);
}
@Test
public void pushTest() {
System.out.println("Parameterized Number is : " + number);
}

JUnit 4.+ vs TestNG
Data providers and parameterised tests…
TestNG…
✓ provides better features,
➢ it allows you to add parameters directly at the test method
➢ if input data is not a complex type, you can configure input data with the xml
configuration file.
➢ allows you to assign a method as data provider which any test method can
use to get data dynamically.

TestNG - Parameterized Test Example(Primitive Types)
public class TestNGTest6_1_0 {
@Test
@Parameters(value="number")
public void parameterIntTest(int number) {
System.out.println("Parameterized Number is : " + number);
}
}
<suite name="My test suite">
<test name="testing">
<parameter name="number" value="2"/>
<classes><class name="com.fsecure.demo.testng.TestNGTest6_0" /></classes>
</test>
</suite>

TestNG - Parameterized Test Example(Compex Types)
@Test(dataProvider = "Data-Provider-Function")
public void parameterIntTest(TestNGTest6_3_0 clzz) {
System.out.println("Parameterized Number is : " + clzz.getMsg());
System.out.println("Parameterized Number is : " + clzz.getNumber());
}
@DataProvider(name = "Data-Provider-Function")
public Object[][] parameterIntTestProvider() {
TestNGTest6_3_0 obj = new TestNGTest6_3_0();
obj.setMsg("Hello");
obj.setNumber(123);
return new Object[][]{
{obj}
};
}

JUnit 4.+ vs TestNG - conclusion
✓ From the features point of view both frameworks are almost same
✓ Syntaxes are also same or at least with the same pattern
✓ But,
➢ JUnit introduces some unnecessary constraints while using its features while TestNG provides
more convenient approach of configuration
➢ Eg:
● For parameterized test, JUnit only allows to provide data to the complete test class not
for each test method
● JUnit expects most of the methods as static(@Parameters, @BeforeClass methods)
● JUnit expects to configure test suites at class level. while TestNG allows to configure
them via xml file
➢ TestNG allows to configure dependent test methods while JUnit doesn’t provide a flexible solution
for that

JUnit 4.+ vs TestNG - conclusion
If you aren’t already using any framework, you can go with
TestNG as it’s easy to configure and maintain. However, if you
are already using JUnit, I would suggest you to upgrade to latest
version of JUnit that has more features for grouping, parallelism,
assertions etc

Software Testing

More Related Content

What's hot

Similar to Software Testing

More from AdroitLogic

Recently uploaded

Software Testing