Netserv Software Testing

Software Testing Presentation About Us Our Offerings Our Methodology Metrics Case Study Way Forward

About Us

NetServ Applications – outsourced software services was designed to help businesses overcome limits – of time, of technology, of resources – to meet their business goals.

NETSERV Offshore Services was established in 2001 and has grown rapidly in .NET Application Development & Testing.

We provide compelling value to our clients by employing proven quality processes, excellence in project management, flexible project models and a blended global delivery model that leverages our strengths in US and Indian operations.

Offerings

Customized Test scripts Framework for Test automation Client Server Applications Functional Testing Web based application Mobile and Pocket PC solutions Performance Tuning Load Testing Usability Testing Non Functional Testing Test Automation Interoperability Testing Offerings

Functional Testing

Product Functionality

Understand product functionality

Write Test Scenarios

Design Test Cases

Check for Traceability of the test case against Requirements

Review Test Cases

Execute the test cases and log the results

Functional Testing….

Testing of web services

Identify the various inputs passed to the web service

Create scenarios to execute all the paths in the web service

Create Queries to check the service output based on the inputs passed to the service

Design the input data and output data to simulate the scenarios

Design test cases for the above scenarios

In case of calculations, design test cases to verify them

Execute the test cases from the various places where the web service is invoked

Tool Used

SOAPUI

Web Service Testing

SOAPUI automates the functional testing of Web services and measures the performance of Web services at runtime

We can perform Load ,Functional and Compliance Testing of Web Service.

SOAP UI

Soap UI is a free and open source desktop application for inspecting, invoking, developing, and functional/load/compliance testing of web services over HTTP.

To test any web service, we need to feed WSDL URL of the web service to SOAPUI tool.

It automatically detects all the web methods of the web service and adds the default request for each method.

We need to pass the parameters required to call the web service in SoapUI tool manually or from any file through groovy script.

Soap UI will invoke the web service and return with SOAP Response.

Functional Testing….

Testing of Reports

Identify the various inputs passed to the report

Create Queries to match the output of the report

Write individual queries to test the various filters, group bys etc. that can be applied to the report

Design the input data and output data to simulate the report

Design a checklist to test the report and use the queries appropriately

In case of calculations, design test cases to verify them

Create the required data using the application and then execute the test cases

Check for correct alignment of the report, exporting feature, performance etc.

Non Functional Testing – Performance Testing

Understand the problem faced by the customer and the need for performance testing and arrive at a problem statement

Define the performance goal clearly (Expected Response time for individual pages , reports, process etc.)

Create test scripts to simulate the scenario being analyzed like order entry process, activation process etc.

Identify the various performance counter that will be monitored (on the client machine, IIS server and db server)

Setup the test environment

Execute the tests for the various load conditions and monitor the performance counters in the test environment

Analyze the test result and fine tune the test script if required

Execute the tests in the production environment

Collate the data and present it to the development team

Non Functional Testing – Load Testing

Define the various operations that needs to be tested for load conditions

Identify the various load that will be applied

Create test scripts to simulate the scenario being tested for load

Identify the various performance counter that will be monitored (on the client machine, IIS server and db server)

Setup the test environment

Execute the tests for the various load conditions and monitor the counters in the test environment

Analyze the test result and fine tune the test script if required

Execute the tests in the production environment

Collate the data and analyze the behavior of the system under study for the various load conditions

We can also detect the peak load condition if required

May need to fine tune some parameters like # of threadpool etc. for achieving best performance when loaded

Non Functional Testing…

Compatibility & Interoperability Testing

Multi Browser Testing

We test the application on various browsers like IE5, IE6,

Mozilla Firefox, Netscape etc.

Database Independence

We have designed a db layer (Netdirect) which ensures database independence

For testing it, we setup the required test environment and test the software using different databases (SQL, Oracle etc.)

On different Operating Systems

We test the software for interoperability on different OS like

2000 server, XP etc.

Non Functional Testing

Usability Testing

Analysis Vision, Goals, Objectives, Challenges, User Categories List, Profiles, Personas / Characterization and more Evaluation Field Study, Contextual Inquiry, Design Walkthroughs, Heuristic Evaluation, Usability testing - paper, low fidelity - high fidelity; informal – formal Implementation Paper Prototype, Click Through Prototype, User Feedback and more Deployment Debrief developers on the design aspect, Handover reports and more

Test Automation

Test framework Components

Initialization functions

Script reads input values from an Excel file

Script performs the test

Reads the “expected output” from the excel file

Compare the results

Logs the result

We develop application specific customized scripts using

the test framework components

Tool Used

Test Complete and Selenium

Testing Tools Used

Test Automation

Test Complete and Selenium

Performance and Load

Open STA for web based Application

Ethereal to analyze the packets

ANTS Profiler on the web server

SQL Profiler

Web Services Testing

SOAPUI

Methodology

Testing Process - Methodology

Identify functional and non functional requirements Develop a test strategy Develop test scenarios document Submit test scenarios document for client review/ feedback Baseline the Test Scenario Document Design the test cases for all the test scenarios Develop “Tractability matrix” Test cases are reviewed by the development team and client Sanity Test the product before accepting the code for ST Execute the test cases and Log / Track Defects Release the product on meeting the “Release Criteria” Detailed Study of Requirement Spec Document Participate in the Requirement walk through -client Analyse all the Post Release Defects Do a Root cause Analysis and Implement Process Improvements System Testing Process Develop Test Strategy Develop Test Scenarios Implementation Phase Test Execution Phase Retrospective Understand the Requirements

Key Metrics

Total Number of Defects detected during System Testing Phase

Defects detected by Priority of the defect

Defects per module

Defect Removal Efficiency

Defect Arrival Curve

Effort Variance and Schedule Variance

Analyze defects based on Defect Type (Requirements defect, design defect etc.)

Key Metrics

Case Study

ERP Re-Engineering – A case study

Problem Statement

Legacy application, to be re-engineered to meet the latest features and functionalities, with complete integration with existing users.

Development should be in phases to ensure that we take up new functionalities in Phase 1 of development that are not available with existing customers. They wanted to sell these as add-on to their existing products thus, increasing their revenue flow, as well as keep their existing customers happy with additional features and facilities.

Application to work on multiple databases - SQL / Oracle / DB2 / Universe.

Architecture should enable uniformity across multiple geographic and multi location development activities

Ease of application maintenance

Solution provided

Service oriented architecture

System Framework has been developed for the entire architecture

Database independent application supports SQL 2005, Oracle, DB2 and Universe

Multilingual Support and Globalization and Localization implemented.

Multi user multi session license support

Web services are secured using MS WSE3.0 (Microsoft Web service Enhancement Version 3.0)

Concurrency handling

Report framework for all reporting purpose using crystal report. Log4Net - used for all logging activities

Net Direct Component – Provides database independence

Project Scope

Modules –

AP, AR, GL, Deals, Inventory, Pricing

Order Entry ,Transportation, purchase

Project Effort - 100,000+ Person hrs

Total Forms Estimated – 1500+

Total tables - 1009

Project Scope delivered to date

Modules –

AP, AR, GL, System Framework

Project Effort to date

25,000+ Person hrs

(System Test Effort – 7500+)

Forms delivered – 410

Framework Controls - 87

Complex Web services – 37

Current number of Tables - 239

System Testing for the ERP

Scope of Testing

Functionality Testing of

General Ledger Module

Accounts Payable Module

Accounts Receivable Module

System Admin

Current Phase

Order Entry

Pricing

Testing of all reports

Total Test Effort – 7500+ phrs

Total Test Cases Designed – 1000+ per module

Metrics Tracked

During Test Case design and Execution

Effort and Schedule Variance

% of test cases designed (planned versus actual)

% of test cases executed (planned versus actual)

Feature Coverage using traceability matrix

Status of defects (# Open, # Close, # Ready for Testing)

Defect Arrival Curve

Product Metrics

Total Number of defects per module

Defect Priority Distribution (# of High, Medium and Low defects detected)

Total Number of system test defects

Defects Detected by Priority

Defect Trend for a milestone Release

Case Study 2 – System testing of Web Application

Product

Online Art Shopping Portal

Product Details

“ Product Admin Module” is used to configure product details, Vendor details,User details

User can order

“ Art”, “Frames”

“ Customized portraits”

View the art so as to get a visual effect using a tool (Art tool)

Product uses the “Paypal Direct / Express” checkout method

Can view order status

Case Study - 2

Scope of the Project

Designed Test Strategy

Functional Testing of the Product

Volume Testing on the Database

Testing Payment using Paypal

Testing the “Order Return process”

Order History

Case Study 2 - Metrics

Product

Portal for Order Entry and Processing

Problem Definition

Test for performance of

Login process

Customer Search and Order Entry Process

Identify the bottleneck

Tune the code

Demonstrate improvement in performance

Case Study 3 – Performance Tuning

We simulated simultaneous users to perform the processes under study using Open STA

Setup the various parameters being studied on the IIS (web server and db server), as well as on the client machine (CPU processor time, MemoryPage faults per second, lock requests / sec, response time and code etc.)

Instrumented the code so as to log the time taken by critical queries that were being used on these pages

Monitored the Response code and response time (at the server and client end)

Plotted the graphs for the various load conditions and analyzed for bottlenecks

Analyzed the profiler results for dead locks, process locks etc. Also studied the time taken for each queries

Identified the problem areas.

Developer’s

reviewed the relevant code,

suggested the fixes,

Implemented them

Retested the application for improvement in numbers

Case Study 3 – Process followed

There were many instances where unnecessary usage of “Rowlocks” were identified.

Process blocks were identified when concurrent users were accessing the site.

We recommended

There were appropriate indexing missing - we suggested them to be mandatory.

Advised them to use "nolock" hint for the queries - which were on less critical business paths.

There were unnecessary indexes created on the columns that has larger datalength – which takes more database resource and will contribute to poor server’s response time.

There were database hits happening inside certain functions – as these functions are invoked inside the queries - this is a wrong approach. Advised them not to use functions for these scenarios, as it will reduce the query run performance.

Hardware was not sufficient both at the web server and database server. As a quick fix, advised them to upgrade the memory.

Case Study 3 – DBA Findings

Login Process

When we started the performance testing of the login process

It could not scale beyond 90 concurrent users

The loading of the post login page (default) was taking more than 28 seconds

Login Process

After identifying the bottle necks and tuning the code / queries, we found that the application could scale upto 500 concurrent users and was able to sustain the load

The CPU utilization was within 5%

Login Process - Response Time

The Response time for 100 concurrent users was well within the required 6 seconds and that of post login page was reduced to 15 seconds.

Customer Search

Initially (3 rd Feb) the Customer search operation would freeze when the concurrent users were increased beyond 50 users

During our second test (17 th Feb), Customer Search operations would not scale beyond 100 concurrent users but the search would work well for 100 concurrent users