WHITE PAPERRightsourcing –The Art of OutsourcingTesting resources are usually expensive, often behindschedule, and generally scrambling to understand the currentand next technologies and business domains. The effectiveuse of outsourcing will result in the establishment of a testteam that can handle these challenges.Software testing is becoming an increasingly critical component of the development life cycleand, as such, demands focus and experience. As important as testing is, companies should befocused on testing in the most efficient, cost effective way. A blended choice of offshore,onshore, and onsite testing resources can help to minimize risks, while maintaining high levelsof effectiveness, efficiency, and cost savings.
WHITE PAPER: RIGHTSOURCING - THE ART OF OUTSOURCING 2Software Development Technologies(SDT) provides software test services Outsource Selectionto product development and IT Overviewpartners. SDT creates new test groupsor extends the software testing Outsourcing is hardly new. It has been used forcapability of our clients. SDT can decades for manufacturing, engineering,design, implement and execute software development and test, and otherFunction, System, and Performance services for many industries. Its popularity hastests to augment a partners test grown in the last decade, but there continue to becapability. Alternatively, SDT can build failures with outsourcing. The purpose of thisor transform a software testing paper is to discuss software test outsourcing, andorganization, by introducing an specifically to focus on software test outsourcingautomation framework, a wide range issues, strategies, and recommendations.of software test tools, and a completeset of test processes. Software testing is becoming an increasingly critical component of the development life cycle and, as such, demands focus and experience.Paul DiGrazia, the SDT engagement Because of its importance, companies shouldexecutive sponsor at WMS, execute software testing in the most efficient,summarized SDTs performance as cost effective way.follows: Without an integrated approach providing a"As a Product Development Executive, proven, defined test process, test tools, andmy Testing Partner decision is critical – training, outsourcing lacks the foundation forfor the last three years, I have trusted success. Software Development TechnologiesSDT to deliver the right consultants has worked with hundreds of firms to improveand test project teams. With Software their software quality. SDT’s software quality experts have worked with companies to assessDevelopment Technologies’ proven, and transform current software test processes,experienced consultants, we rapidly practices, technologies including test automation,developed a solution that is efficient organizational structure, and environments usedand repeatable. The SDT Outsourcing to develop and test a software application. A testapproach is to provide a skilled and transformation implements a consistentflexible resource offering, developed discipline across development, quality assurance,and properly staffed to meet our and operations.needs, and dedicated to our goals andobjectives. The resulting high quality When SDT works with a company to implementand strong teamwork provide the right outsourcing, outsourcing is not just added, it is integrated into the development, qualityreturn on investment with our critical assurance, and operations organizations and theirprojects being tested the right way”. use of processes, technologies, and environments. Effective outsourcing enables sustainable and efficient software testing under dynamic conditions, project after project. Testing is enabled by domain specific knowledge, by tools for test design, execution, and management, and by test methodologies and processes that verify and validate a range of work products of software application project teams. The testing processes are included within a Software Development Life Cycle (SDLC) and are constrained by costs, schedule, technology, and people resources. Testing resources are
WHITE PAPER: RIGHTSOURCING - THE ART OF OUTSOURCING 3Learn More expensive, often late, and often scrambling to understand the current and next technologies and business domains. Within this context,Software Testing in the Real World is which is often complex and chaotic, outsourcingan informational guide that is can be at best a partial solution to the challengesessentially a toolkit for continuous, of software testing.sustainable improvement of thesoftware testing process. Written byEdward Kit, CEO of Software Considerations forDevelopment Technologies, SoftwareTesting in the Real World addresses Outsourcingthe most frequently asked questionsabout methodologies, tools, Reasons to outsourcetechnology, and organizational issues SDT has helped many companies outsourcebeing posed in the testing community software testing as part of a program totoday. transform their testing. Some of the benefits of outsourcing that SDT’s clients have seen are:To request a discussion with SDT and • To provide additional resources to thegain insights into how your company test effortcan effectively outsource complete • To potentially shorten schedulessoftware testing, test departments, or • To improve the quality and value oftest projects, please call James Wright, testingSDT’s General Manager, Test Service • To reduce costs for testing by relocatingOperations, at 405-232-6000 x11. To selected testing activities to lessfind out more about Software expensive facilitiesDevelopment Technologies, visit • To improve efficiency and effectivenesswww.sdtcorp.com. • To provide around-the-clock test operations • To avoid employee overstaffing/understaffing or hiring/releasing cycles • To make use of technology and business domain expertise from third parties when resources are not available locally • To gain access to world-class capabilities • To make better use of in-house subject matter expertise Problems to avoid Increasingly outsourcing has been done at offshore facilities where costs are lower. Discussions with SDT clients and published articles indicate that there are serious issues with an outsourcing model based strictly on offshore testing when there is not sufficient support on- site.
WHITE PAPER: RIGHTSOURCING - THE ART OF OUTSOURCING 4Offshore models at first appear to provide a low • Outsourced resources must havecost advantage, but the problems can outweigh communication and collaboration skillsthe benefits: that match the on-site project team • Cultural, language, and time zone culture. differences • Excessive travel costs In addition, practices must be set up to achieve • Higher project management costs the following: • Lack of domain knowledge and • Outsourced resources must gain project expertise in the outsourcing firm application domain knowledge and • Lack of in-depth software test expertise experience. in many outsourcing firms • Outsourced resources must quickly • Ineffective test design and automation learn the project functional • Difficulty in knowledge transfer requirements, quickly absorb changes, • Difficulty setting up lab environments and quickly expose risks during each of • Vendor delivery failures (business the SDLC phases. interruptions, local outages, • Outsourced resources must deliver each communication technical issues) of their work products in support of the • Vendor employee turnover project schedule and compliant to all • Poor quality control requirements of the SDLC. • The need for data privacy must be • Intellectual Property risks maintained if testing is based on actual • Data privacy risks – such as when using data. Data may have to be scrambled. actual data as test data • The company must continue to spend • Serious issues managing offshore management effort to meet the needs of outsourced resources by the company the outsourced resources and to assure • Lack of expertise or experience with their success. Companies may need test project management in the company help initiating outsourcing and • Hidden costs / risks to the company sustaining effective management of (such as a reduction in morale on-site) outsourcing.Outsourcing as a pure cost reduction effort for With proper contingency planning it is possiblepoorly functioning test processes only degrades to bring the function back in-house ifvalue. outsourcing does not work.Best practices in After reviewing and experimenting with differentoutsourcing models, SDT recommends a “Blended Model” that uses a combination of offshore and onshoreOutsourcing of test resources, like any test centers as well as on-site representatives.engineering project, requires that the company This mix of resources and locations in theand the outsourcing vendors engage in planning, Blended Model is designed to improvehave clear requirements, plan detailed integration communication, knowledge transfer, and projectof people, technologies, and processes, build management while minimizing travel andcomprehensive metrics to show status and training costs, and the effects of cultural and timetrends, and manage all aspects of the effort. zone differences.Skills in these critical areas must be built toenable outsourcing success: • Outsourced resources must have the technical skills, and specifically the test skills, that complement the development and project management skills of the project team.
WHITE PAPER: RIGHTSOURCING - THE ART OF OUTSOURCING 5Based on research published in the SDT study on Implementation problemsoutsourcing, SDT recommends the following to avoidoutsource locations in a Blended Model for Some problems to avoid when implementingcompanies located in the US: outsourcing are: • Communication problemsOffshore: o To help minimize communication India problems, formalize processes for specifying and updating softwareOnshore: requirements. Anything ambiguous Oklahoma City, Oklahoma may be misinterpreted. Changes Kansas City, Missouri after work begins will increase Memphis, Tennessee costs and lengthen the project. Albuquerque, New Mexico o Also plan frequent meetings between the company team lead and the outsourcing team lead.This white paper is abstracted from a detailed Choose a meeting time that isSDT research study. For more information, relatively convenient for both teamplease contact SDT for “Optimized members.Outsourcing Practices for Software Testing – o Note that culturalRightsourcing.” misunderstandings can lead to communication problems.Implementing outsourcing • Problems when outsourcing tasks that require close communication withSome steps to execute when outsourcing are: cross-functional groups • Understand your strategic vision and • Starting with immature internal plan and how it relates to an processes outsourcing relationship • Lack of an internal process for • Understand your company goals and specifying work objectives • Lack of internal project management • Have senior executive support and skills involvement • Trying to use outsourcing as a means to • Select the right vendor attempt to fix broken processes • Charter an Outsource Test Management • Losing control of the resource Group to oversee the relationship, workflow, and metrics. • Losing critical internal expertise • Plan for the test lab infrastructure. • Not involving the right people – executive sponsors, users, and impacted • Ensure nondisclosure of confidential employees customer data including test databases. Ensure that data privacy is maintained. • Reengineer existing processes to fit the Types of testing projects outsourcing relationship best suited to outsourcing • Properly structure the contract and associated Service Level Agreements The following types of testing are suitable for (SLA’s), but provide flexibility outsourcing: • Measure and monitor work regularly • Function testing • Tie key metrics to Service Level • Compatibility testing Agreements • Internationalization and localization • Have open communication with testing affected individuals and groups • Performance testing, if the load can be • Pay careful attention to personnel issues generated at the company site but • Justify using additional funding in the controlled remotely near term
WHITE PAPER: RIGHTSOURCING - THE ART OF OUTSOURCING 6 • Security testing, if the outsourcing more crime, a less educated population, lower company is a specialist in this area quality talent, fewer desirable places to live, and • Usability testing, if the outsourcing few or no high quality service providers in company is a specialist in this area economically depressed areas.Projects and products that do not require deep Based on study factors, an onshore model in aknowledge of complex, undefined or changing popular “lifestyle” city that is also a low-costbusiness rules are better suited for outsourcing. area appears best suited to fulfill the criteria for an onshore center. “Lifestyle” cities are areasTypes of testing projects where people want to live, due to such factors as nearby educational institutions. One citynot well-suited to fulfilling requirements is Oklahoma City,outsourcing Oklahoma, with Kansas City, Missouri, Memphis, Tennessee, and Albuquerque, NewThe following types of testing are less Mexico also being potential locations.appropriate for outsourcing: • User acceptance testing University-Based Model • Testing normally done by the Definition: Outsourcing using a labor pool of development team, unless the university students in conjunction with an developers are also outsourced: unit organization to manage the physical and project testing and initial integration testing logistics. • System end-to-end testing, where some specialized domain knowledge may not A development organization should not depend be available to the outsourced team upon the University model as a major part of • Testing of products undergoing their outsource model at this time. Because the significant or late stage changes main focus of the student is on other priorities (getting an education) there could be issues withOutsourcing Models the quality and consistency of the work products. Near-shore ModelFor the purposes of this paper, SDT investigatedand compared the following types of working Definition: Outsourcing offsite in a countryrelationships: geographically close to the organization’s • On-site country. For US companies, consider Canada, • Onshore Mexico, etc. • University-based • Near-shore Offshore Model • Offshore Definition: Outsourcing at a country remote to • Blended the organization’s country. Countries to consider for US companies are India, China, VietOn-site Model Nam, etc.Definition: Outsourcing the work to workers on-site at the company’s location where Although becoming more expensive, Indiadevelopment takes place, sharing facilities with currently holds the best promise for offshoringemployees. activities from the US because of the relative experience, strong skills, and low costs of theOnshore Model workers. Knowledge of English is a strong benefit when outsourcing from US companies.Definition: Outsourcing offsite in the samecountry as the organization.An onshore model using depressed economicareas does not look promising because there is
WHITE PAPER: RIGHTSOURCING - THE ART OF OUTSOURCING 7Blended Models SDT Outsource Selection RecommendationsDefinition: Outsourcing that combines morethan one model into an overall outsourcingstrategy. This whitepaper describes one element of aA Blended Model gives the best combination of larger program to improve the quality andimpact and control. The mix of resources and effectiveness of testing at a developmentlocations in the Blended Model is designed to organization. Previous SDT presentations haveimprove communication, knowledge transfer, defined a need for an Enterprise Software Testand project management while minimizing travel Center of Excellence (EST CoE) andand training costs, and reducing the effect of recommended the formation of such an entity. Atime zone differences. development organization should use their employees for key Center of ExcellenceDepending on the location of the on-site and positions to maintain continuity and retain theiroffshore offices, in an On-site/Onshore/Offshore knowledge base.Blended Model it may be possible to find a timezone for the onshore team that supports The EST CoE is to provide guidance andcommunication with the offshore resources at coordination to foster world-class testingone time of the day and with the on-site team at throughout the organization. The mission of theanother time. EST CoE is to provide services and assistance in identifying, documenting, and planning softwareSub-models testing. The EST CoE leverages proven, state- of-the-practice technologies, procedures,The following business relationships can be methods, and tools such that the key mechanismsconsidered for the outsourcing company: of existing systems are better understood, that • Contracting Company Partner: A current business structures and operations are third-party vendor who provides testing improved, and that the business enterprise is able services. to foster innovation as a competitive strategy. • Wholly Owned Subsidiary: Rather than outsourcing to another Every outsourcing opportunity is different, and organization, a development SDT recommends a blended strategy that can organization might opt to start their own flex and solve the major problems of the subsidiary company. previously tried outsource models. An On- site/Onshore/Offshore Blended Model allows a • Build/Operate/Transfer Model: In development organization to shift resources from this approach, the partner provides offshore to onshore if any risk factor warrants a clients with a complete solution for change. This approach more easily deals with building a presence in a particular city ramping up testing capability. The approach or country. The partner provides a combines benefits and minimizes risks, comprehensive set of operational improving the overall chance for success. management services, from HR and staffing, to accounting, payroll, legal, SDT recommends that, when outsourcing with a facility management, and security. The Blended Model, a development organization partner should not be able to lock in the utilizes an outsourcing partner who starts with an client company. Instead, the client equal number of resources coming from the company should have the option to offshore team and resources coming from the bring the operation in-house at any onshore team to staff testing, and then adjusts the time. Typically, the outsourcing ratios based on project requirements. contract includes a clause that states the client company has the option to buy the entire operation after a fixed period.
WHITE PAPER: RIGHTSOURCING - THE ART OF OUTSOURCING 8 cost savings and effectiveness and mitigatesRecommendation 1: outsourcing risks.Establish Relationships Where Outsourced Resourceswith a Small Set of Testing are Located (initially) On-Partners At Offshore At Onshore site Where OutsourcedDevelopment organizations should establish Resourcessolid relationships with a small set of qualified Come Fromvendor partners that will provide the testing From Offshore 50% 45% 0% 5%resources. Ideally an organization would have From Onshore 50% 0% 25% 25%one or two partners, each with capability toprovide both onshore and offshore resources. Total 100% 45% 25% 30%Using a few partners reduces the risks associatedwith sole-source sub-contracting without over Note that a percentage of resources that comeextending the project and vendor management. from the offshore team and resources that come from the onshore team are located on-site to actRecommendation 2: as communication conduits and liaisons. The recommendation is to minimize from the startEstablish an Outsource Test the percentage of resources that come from theManagement Group offshore team that need to be located on-site. In addition, the recommendation is to begin withSDT also recommends that the company the percentage of outsourced resources that comeestablish an Outsource Test Management Group from the onshore team that are located on-site asto dedicate internal resources to qualify vendors, shown (25% may be optimal) and reduce thisto qualify projects, and to coordinate and oversee number over time. This is an area wherethe outsourcing process. This approach outsourcing efforts in many companies fail. Ifconcentrates the outsourcing experience and the number of outsource resources that areexpertise for the company and allows for better located on-site is too small from the beginning,coordination of the various test activities. transition efforts can be overlooked, and domain knowledge transfer can be at risk. Once domainRecommendation 3: Use knowledge has been gained by the outsourcethe Blended Outsourcing resources, and the process understood and implemented, the resources from the onshoreModel team who are located on-site can successfully beThe Blended Model that SDT proposes combines redeployed to their onshore location.on-site, offshore, and onshore resources. SDT’ssoftware testing experts can determine the right Analysis supporting themix of onsite, offshore, and onshore resources.One approach that is often optimal is for the Blended Modeldevelopment company to initially plan for about Recommendation50% of the outsourced resources to come fromthe pool of onshore resources and 50% to come Primary factorsfrom the pool of offshore resources as shown inthe table below. Some percentage of the SDT considered five primary factors to compareresources coming from offshore would actually outsourcing alternatives. The details of eachbe located on-site. Similarly, some percentage of follow this list.the resources coming from the onshore team of • Location Riskthe outsourcing company would also be located • Direct Cost Savingson-site. The actual percentage distribution will • Flex Capacityvary depending on project and work • Efficiency and Effectivenessrequirements and will evolve over time. • NationalismDividing resources in this manner maximizes
WHITE PAPER: RIGHTSOURCING - THE ART OF OUTSOURCING 9 • Costs for technical support (lab hardwareLocation Risk and software administration, network management, telephone charges, etc.)This is the risk of a failure due to factors other • Costs for training the outsource resources inthan vendor technical performance. These would the company’s applications and processesinclude security risks, language and cultural • Increased project management costs – thedifferences, communication over time zones, cost of additional project oversight that ispolitical interference, the risk of terrorism, and required within the company’s developmentother non-technological issues based upon or test organization to manage the outsourcegeographic location. Factors to consider are: testing• Language and cultural differences• Security risks (risks associated with building Flex Capacity access, improper protection of Intellectual Property, unauthorized application access, Flex Capacity measures how quickly the number unauthorized access to corporate data, of resources can be changed. Factors to consider unauthorized access to information backups, are: etc.) • Flexibility of staffing levels• Geopolitical risks • Capacity – size of the labor pool available• Natural geographical risks (due to • Ability to timely ramp resources up or down earthquakes or weather) • Start-up and shut-down logistics• Risks doing business in the region Effectiveness and Efficiency• Risks due to laws of the region• Time zone differences This reflects the quality and consistency of the• Distance from the company site work products and the effort required. Factors to• Visa requirements for travel or on-site work consider are:• Quality of life issues in the outsource • Efficiency of offsite resources compared to location on-site resources• Lack of transportation availability and • Efficiency of outsourced resources choices compared to regular employees• Currency volatility • Quality of work • Ability to meet schedulesDirect Cost Savings • Accuracy of resource estimates • RepeatabilityThese are the financial elements that contribute • Reliability and bandwidth of networkto higher or lower overall direct costs. This connectionsincludes labor costs, travel costs, and project • Inefficiencies due to communicationmanagement costs. technical issues. Consider telephone• Labor cost (averaged direct labor rates due reliability, availability of teleconferencing to the mix of resources based on the work facilities, desktop sharing and web location and experience) conferencing capabilities, etc.• Travel costs • Complexity of contracts and Service Level• Costs to ramp-up resources (cost to hire and Agreements prepare resources for performing testing tasks) Nationalism• Maintenance costs (minimum overhead costs to enable resource availability) This measures the favorable or unfavorable• Costs to reduce staff, if any, when fewer impact of using nationals from a different resources are needed country.• Technical infrastructure costs (costs for • The desire to use the nation’s labor pool equipment, floor space, network • Employee, customer, and community connectivity, etc.) backlash over a perceived loss of jobs by shifting work out of the country
WHITE PAPER: RIGHTSOURCING - THE ART OF OUTSOURCING 10 Each of the five factors was evaluated and Figure 1 provides a summary of computed values on the assigned a value of 0 to 10 for each alternative. row labeled “One Hundred Scale”, using a scale of 0 to Based on feedback from SDT clients, a 100 to compare the options. Zero would represent the subjective weighting factor on level of worst possible case and 100 the best situation. The importance was also applied. Extended and computed values fall between 53 and 80, with the normalized scores were then computed and Blended Model ranking highest. examined. Figure 1: Summary of Findings Near-shore University Offshore/ Onshore/ Offshore Onshore Blended On-site On-site On-site WeightRisk Factors (describedabove)Location Risk 5 10 8 8 8 7 6 4 7Direct Cost Savings 10 0 8 4 6 7 8 9 8Flex Capacity 10 2 5 7 7 7 9 9 9Efficiency andEffectiveness 10 9 5 8 7 7 6 5 8Nationalism 5 10 10 10 10 6 3 0 7Total 400 210 270 280 290 275 275 250 320One Hundred Scale 53 68 70 73 69 69 63 80Notes:- On-site refers to company employees at the development location- Blended refers to the On-site/Onshore/Offshore Blended Model
WHITE PAPER: RIGHTSOURCING - THE ART OF OUTSOURCING 11Review - Outsourcing for voice calls. Alternatively, or in addition, reliable telecommunications infrastructure can beModel Critical Success used for voice calls. 8. Personnel issues must be managed. Disgruntled,Factors discouraged, in-the-dark employees can undermine relationships, withhold critical information, andThe effectiveness of any outsourcing model’s cause the relationship to fail.capability relies on the following success factors: 9. Performance incentives and penalties based on1. A well-defined and well-documented objective criteria (measurable, quantifiable, and Software Development Life Cycle is comparable against pre-established criteria) should mandatory, including standard be used. The relationship managers need to be documentation formats. Training to provide trained on the criteria, incentives, and penalties. the necessary common terminology and 10. Clearly communicated goals and metrics enable understanding enables efficiency and teams to manage to success. effective delivery of work products. 11. “Best Practice” companies prefer global sourcing2. Project documents, such as requirements over the Offshore Model. Companies with and specifications, must be updated significant outsourcing experience have moved or throughout the development process. are moving toward a strategy of multiple locations, Change happens, and when it does, its multiple vendors, and a judicious blend of both impact must timely propagate through the onsite and outsourced models to minimize risks sequence of development documents. and maximize flexibility.23. Participation at technical reviews from software test personnel, especially during the analysis and design phases of the Software Development Life Cycle, is required in order to understand the functionality and scope of the application to be tested.4. Program Management (for managing the relationships) and Change Management (for managing the work products) are keys to success. Most of the companies that responded to a survey (85%) believe that the establishment of a Program Management Office (PMO) provides the foundation for outsourcing/offshoring success, with two- thirds (67%) stressing the importance of proactive communications programs.15. Regular teleconferences and site visits with the outsource test team are critical to good communication and collaboration.6. An on-site manager who functions as the contact point for the outsource test team and who works closely with the outsource vendor manages timely delivery of work products and resolution of issues.7. Sufficient, reliable network bandwidth is required to allow data communication between the offsite and on-site groups. Skype or a similar technology can be used1 Outsourcing Institute, “Top Ten Outsourcing 2Survey” Ibid.