Independent Testing


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Independent Testing

  1. 1. Independent Testing Introduction LUXline Independent Testing provides a complete software application testing solution for medium to large corporations, including Fortune 500 companies. The service is ideal for companies needing independent and/or additional testing on the software applications they develop for in-house use and external clients. LUXline Independent Testing personnel are highly trained, with access to the latest technology, to ensure thorough testing within most industry environments. LUXline Independent Testing provides: • Comprehensive contemporary testing types and practices • A variety of testing environments and testing technology expertise (know-how) • Modern testing methodologies such as “white”, “black” and “grey” box testing, rapid testing and incremental testing • Testing automation supported by a wide range of modern testing tools • Software Risk Reduction Framework (SRRF) ensuring a fast, structured and accurate way to target specific software or system operation risks • Full documentation support including process documentation and status reports in addition to documented test cases and results • Protection of intellectual property For test managers considering outsourcing solutions for some or all of the software application testing requirements, this paper describes how LUXline Independent Testing meets those needs. Challenges Facing In-House Testing Comprehensive Competent and thorough test plans are integral to the successful development of software testing reduces the applications. During the testing phase, there are many areas that, if not executed correctly, risk of system can compromise the function of the software, and there is usually far more work involved in malfunctions and is testing than is planned. A thoroughly tested application includes installation, feature, necessary to feature integration, usability, stress and performance testing. guarantee system reliability. General Service Description LUXline Independent Testing provides a number of quality control processes which assess whether the software application complies with prescribed requirements and functions. Testing determines which functions do not work correctly, as well as other discrepancies between the expected and actual system behavior. LUXline Independent Testing • • 1• • 12/23/05 A LUXOFT White Paper
  2. 2. Functional Capabilities LUXline Independent Testing can support the client’s testing requirements in any of the following ways: • As a comprehensive testing resource for clients who need top-quality, low-risk software applications. • Providing additional verification for mission-critical and other complex software to ensure the highest level of reliability. • As an alternative (independent) source of quality assessment, including software applications that have been developed by third parties. • Providing the automation test framework and test cases for clients who want to retain the responsibility of running the tests and evaluating the results. This is an ideal approach for clients who need to reduce costs, minimize the disruption of their current staff and budgets, and/or to accelerate the transition from manual to automated testing. • Packaged with LUXOFT application development services, as an entire software developing offering. Defect Management Testing is often coupled with defect management that encompasses the following procedures: • Accurate defect description • Defect correction decision making • Correction delegation • Correction implementation • Post-correction testing Testing normally lasts until some predetermined criteria have been met, usually the number Combined testing of defects that can be tolerated. The testing service yields general testing and defect statistics may statistics, including those defects that were discovered, corrected or kept unchanged. Thus, serve as a basis the combined testing statistics may serve as a basis for the software application quality for the software assessment. application quality assessment. While LUXOFT can provide any and all testing activities, including requirement analysis and test design, it is the client who determines the requirements and acceptance criteria. Methodologies LUXline Independent Testing is based on the LUXOFT Standard Software Development Process (SSDP) and can be efficiently integrated with the client’s established development processes. This testing methodology allows for all the necessary testing processes to be repeated until the software application can be accepted, as shown in the figure on the next page. LUXline Independent Testing • • 2• • 12/23/05 A LUXOFT White Paper
  3. 3. Initial Planning, Testing Approach Inspect Requirements • Estimates • Defects in Requirements • Schedule • Requirements Priorities • Risks • Acceptance Criteria • Strategy Define Test Cases • Test Plan Outline • Test Data Design • Traceability/Coverage Acceptance Automate and Execute Tests • Automation Scripts Develop Test Specs • Defects Documented • Test Plan, Detailed • Test Reports • Test Data Preparation • Defects Trends • Test Environment • Product Evaluation • UAT Entry Criteria The testing cycle is repeated until the software meets the acceptance criteria, as defined by the client. Testing Types Available LUXOFT testing services encompass a full range of state-of-the-art testing types and practices. The selection of appropriate types of testing is subject to specific system requirements, and can include any or all of the following: Functional testing Functional testing verifies that the functions of a system are present and correct as specified, and includes: requirements analysis, functional test design, functional test execution and reporting. Among other recognized approaches LUXOFT uses UniTesK*, a proprietary specification- based testing methodology, intended for mission-critical applications and components. UniTesK testing is automated and is complimentary to functional testing. Performance/Load/Stress Testing Performance testing determines how the software application behaves under anticipated levels of load. The resulting test data is used to determine the load at which the system's performance begins to degrade significantly, and the breaking point of the software application. LUXline Independent Testing • • 3• • 12/23/05 A LUXOFT White Paper
  4. 4. Compatibility/Configuration Testing Cross-platform testing ensures that the software application behaves as expected under various environment conditions. Usability Testing Usability testing ensures that all parts of the user interface are consistent and friendly. Security Testing Security testing ensures that the system meets security requirements. Data Migration/Conversion Testing Data Migration testing verifies whether programs and procedures used to convert data from existing systems for use in other applications is successful. Unit/Component Integration Testing (including API Testing) Integration testing ensures trouble-free functionality of the integrated software/system units and components. API Testing checks the methods of any given API for valid, invalid and inopportune inputs and ensures system integrity and uninterrupted operation. GUI and Non-GUI Testing GUI testing is different from other types of testing and requires the support of special tools. LUXOFT tools have features that are critical for effective automated GUI testing. Replay Scessory and KDExecutor employ effective testing processes to achieve high quality goals. The solutions for automatically testing non-GUI software involve creating sets of test cases and a set of expected results. As the automated tests run, the actual results are compared to the expected results, and deviations are flagged for examination by developers. These solutions are automated to save personnel costs while identifying issues that may be overlooked in the rush to release software. Application Program Interface (API) Testing LUXOFT Independent Testing uses a proprietary testing methodology, UniTesK, as a specification based automated testing structure. It is ideal for use when testing mission critical applications or components. UniTesK is applied at the unit/component level and is complementary to functional (black-box) testing. UniTesK enables mathematically exhaustive per-specification coverage and is most valuable when done at the unit testing tools. The rate of return for UniTesK automated testing increases with longer maintenance periods. Test Automation Test automation is crucial to implementing faster test cycles and increasing test reliability. LUXline Independent Testing architects and business analysts closely collaborate with the client’s test managers to develop the automation requirements based on current business needs, thus designing the best possible test plan for automating test cases. Test automation can be coupled with manual test execution, especially when performing regression testing. Tools LUXOFT Independent Testing personnel are trained on many different testing packages. The software applications used during the testing process can include, but are not limited too: • Test Specification, Requirements Traceability LUXline Independent Testing • • 4• • 12/23/05 A LUXOFT White Paper
  5. 5. o IBM Rational Test Manager o IBM Rational RequisitePro • Regression/Build Verification Testing o IBM Rational Robot/LogViewer o Mercury Interactive WinRunner (client provides license) • Performance/Load Testing o IBM Rational Robot o IBM Rational Test Manager o IBM Rational Quantify o Mercury Interactive LoadRunner (client provides license) • Unit Testing o IBM Rational PureCoverage o IBM Rational Purify o LUXOFT UniTesK (API-based testing) • Defect Management o IBM Rational Clear Quest/Clear Case • Version Control/Change Management o IBM Rational Clear Case o Microsoft Visual SourceSafe • GUI Testing o Replay Xcessory o KDExecutor Types of Business Models for the Testing Project LUXOFT Independent Testing employs three proprietary business models developed exclusively for testing services, each designed to address the different functions and deliverables of the testing process. Resource Augmentation: Highly trained and experienced testing specialists are provided to the client’s IT team on an as-needed basis. The typical deliverables of short-term consultants may include defect reports, test specifications, and coverage/traceability matrices. Turnkey Project: LUXOFT Independent Testing handles all the facets of the testing process, from the initial planning stages to software application acceptance, with milestone based deliverables. LUXwork Development Center: An offshore development center can be established as part of LUXwork, a unified LUXOFT development centers family when ongoing testing is required, such as frequently upgraded software applications, application life cycle management, and testing for clients who regularly release software applications. The LUXwork Development Center can provide all of the testing and related documentation required for software acceptance. The onsite presence is usually limited, but can be adjusted as necessary. LUXOFT Software Risk Reduction Framework (SRRF) LUXOFT employs Software Risk Reduction Framework, that generates the right testing engagement model and which is developed exclusively for testing services. This model LUXline Independent Testing • • 5• • 12/23/05 A LUXOFT White Paper
  6. 6. ensures that all client risks and concerns are effectively addressed. The following table shows the distribution of various risks across specific testing models: Client Concern LUXOFT Independent Solution details Business Testing Solution model There is no evidence available Test Plans are Test design Resource to show that testing has been measurable in terms of techniques Augmentation methodical thoroughness Gaps in the testing – some Test Plans are Traceability Resource features may be released measurable in terms of matrices Augmentation untested coverage Significant requirements and Requirements and design Technical Turnkey design defects are found defects are found early in reviews Project during system and acceptance development testing Decisions to release into Application readiness Turnkey Defect production are based on ‘”gut- qualification Project classification; feel”. No objective assessment defects of the quality and readiness of measurements the software. No regression and trends; test testing done on final version of entry and exit the software criteria; test- related priorities Testing is time-limited. Some Risk-based test design Prioritization Turnkey tests may never be run. and execution. policy Project, Release quality is Regression test strategy (requirements LUXOFT compromised because of time and tests) Development and budget constraints ROI based test Centers automation Test budgets are unreliable Test budget estimation Provider’s best Turnkey because there is no rational methods practices; Project, method of estimating test previous project LUXOFT activities metrics Development Centers Post-delivery defect density is Predictable amount of Process LUXOFT unknown. No data on how defects measurements; Development many defects may still reside Predictable residual Projects history Centers in the software after defect density acceptance. Quality must be high, such as Reduced defect density Root cause LUXOFT mission critical, or on-going analysis Development software update releases Centers Security and Protection of Intellectual Property Security and protection of intellectual property is always a concern for companies transferring their software to the third parties for independent testing. This software can be a unique product which the company plans to place on the market or it can support the company’s sensitive corporate information. LUXOFT has developed a comprehensive corporate security model that ensures the protection of IP at all levels. The model is fully compliant with the ISO / BS17799 international standard and includes: LUXline Independent Testing • • 6• • 12/23/05 A LUXOFT White Paper
  7. 7. 1) Physical Security – all buildings are secured with video surveillance, guarded entrances with card readers, and separate areas for each customer (if required). 2) Information Security - segregation of dedicated development center resources that range from a simple VLAN structure to a fully dedicated separate network. There is also a geographically dispersed corporate disaster recovery center. 3) Staff Security – there are thorough employee background checks; NDA (non- disclosure agreement), and AWA are required. 4) Market Security – if requested by the customer, LUXOFT will establish the necessary firewalls between teams and projects. LUXOFT information security provides the following elements: “There have been no recorded violations of • Dedicated physically secured areas [IP] rights in the offshore • Separate VLANs software development • Cisco PIX firewalls with IDS at all external entry points industry. As always, companies can ensure • Redundant and fail-proof configurations for critical points (network switches, their safety by working routers, firewalls, domain controllers) with reputable and • Scalability experienced • Virus protection on all feeds into and out of the customer cell organizations and contractors.” • Encryption of data feeds between Customer and LUXOFT using VPN or/and dedicated channels • Monitoring/logging of development center feeds into Vendor network American Chamber of • Monitoring/logging of development center feeds to external connections Commerce, 2002 “Update on Russia” • Monitoring/logging the removal of media from a development center Protection of Intellectual Property is a legal right that promotes economic and technology progress and is taken seriously by LUXOFT. Customers of any size require processes and systems to be in place to protect their property. LUXline Independent Testing is subject to the Russian and international legal acts and laws. Summary of Benefits LUXOFT Independent Testing Clients Enjoy the Following Benefits: • Software Risk Reduction Framework (SRRF) targeting the most critical client risks associated with system/software operation • Enhanced innovation (UniTesK*, SRRF Testing Solutions a.k.a. Testing Engagement Model). • Leverage of tools/reusable component libraries/best practices • Objective testing results based on a disciplined approach to each individual testing project • Access to a wide range of testing skills, methodologies, expertise and know-how • Well-defined and flexible delivery models • A range of cost-effective testing models with ratios of up to 1:30 CONTACT INFORMATION 1-866-LUXOFT1 or email us at LUXline Independent Testing • • 7• • 12/23/05 A LUXOFT White Paper