Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
• Cognizant 20-20 InsightsOpen Source Testing Tools:The Paradigm Shift   Executive Summary                                ...
Open Source Advantage                                                                               •         Open source ...
Case Study>>Open Source Software Yields Accelerated Testingfor Entertainment Guide Site         Client Details            ...
Approaches to Adopt Open Source Testing Tools                                              End-to-End Integration of Open ...
Serving Commercial Operations              XPlanner           Redmine             Jira                  Bugzilla          ...
About CognizantCognizant (NASDAQ: CTSH) is a leading provider of information technology, consulting, and business process ...
Upcoming SlideShare
Loading in …5

Open Source Testing Tools: The Paradigm Shift


Published on

With costs increasing for software testing tools, organizations can reduce spending by adopting open source testing tools, which offer equivalent functionality to their commercial counterparts.

Published in: Technology
  • Be the first to comment

Open Source Testing Tools: The Paradigm Shift

  1. 1. • Cognizant 20-20 InsightsOpen Source Testing Tools:The Paradigm Shift Executive Summary Testing Tools: Commercial Businesses today demand faster time-to-market vs. Open Source for their software products without significant Traditionally, QA teams prefer using commercial expenditures in testing tools, and at the same products over open source software. Their time they want to ensure that software bugs don’t rationale includes the following factors. eat up their investments. They need cost-effective testing solutions that do not compromise quality • Ease of use: to achieve accelerated time-to-value. > Easily usable with little training. In less than a decade, the software testing market > Most have clear documentation has evolved from a small segment of the appli- and tutorials. cation development lifecycle to a market unto • Availability of dedicated support teams: itself. Plenty of industry-standard testing tools are now available to address different facets of > Support user-base through multiple chan- nels for queries, patches and enhance- the QA process, ranging from simple unit testing ments. to end-to-end life-cycle management. • ROI: Market growth has led to rising end-to-end costs for testing tools. This is motivating IT organiza- > Teams spread cost of tools over the life of its usage. tions to seek open source testing tools, which are often less expensive from a total cost of ownership > Tool requires long-term commitment, perspective and offer equivalent functionality. thereby reducing probability of migrating Open source tools are now an integral part of the to another tool. IT spectrum, and have penetrated nearly every aspect of the software testing domain. This paper • Easier to recruit testing staff: explores the trends influencing the adoption of > More popular commercial tools would have open source tools in the software testing arena. It larger number of people within the knowl- highlights approaches that can help an organiza- edge base. tion reduce spending on testing tools. cognizant 20-20 insights | october 2011
  2. 2. Open Source Advantage • Open source had risen to the top of the executive agenda: “faster, cheaper, better.”• Lower total cost of ownership.• Business strategy: • Open source is strategically used to gain competitive advantage. > Reduced dependencies on specific products/toolsets. • Open source provides increased innovation opportunities for IT professionals. > Promote faster time-to-market.• IT strategy: Trends Watch: Open Source in Agile > Flexibility. Traditionally, the testing lifecycle has been viewed as a “V” model or a “modified waterfall.” However, > Ease of procurement (off-the-shelf product). many organizations have achieved significant > Ease of usage. success by leveraging open source tools using > Tool customization. the Agile development methodology as well. The reason has to do with the role these tools play in• Open source community: enabling early code-driven tests and continuous > Strategy and affinity to contribute to the integration mechanisms, which are key to the community, thereby promoting open and success of Agile projects. extensible architectural design of software. The Way ForwardThe Paradigm Shift QA organizations keen to gain value from openThe following developments have raised the ante source tools have common questions regardingfor open source tools: the best approach for adopting open source• Recent recession has tightened the organiza- testing tools. Figure 2 describes the approach tional budget, making open source tools more towards open source tools adoption. attractive. The migration from commercial to open source• Low ownership cost, flexibility, increased tools should be approached as a phased activity. innovation, shorter development and faster procurement as reasons for picking open source.Open Source Tools Across Testing Lifecycle ct ional Automatio Fun n g Watir Ac tin nium Sah c ele i s ep S Te C tan gement & e ana nc r uc ce ete um ma M JM Test b Tr Perfor er F ct ac Open ing Proje Load UI itNesse king Source Testware ation Test son Test M tegr Hud Jir Lin a, X i ne Planner, Redm k In a st is na Te e Cr ntr us op ol ia ge Co o en ti nu Red m la t mine Bugzil on C Defe t ct ManagemenFigure 1 cognizant 20-20 insights 2
  3. 3. Case Study>>Open Source Software Yields Accelerated Testingfor Entertainment Guide Site Client Details Engagement Description End-result This client operates leading The client has a local entertainment • 72% cumulative time savings interactive news and informa- guide where one can find details ranging through automation testing tion websites in major markets from what-to-do and how-to-go, to vis-à-vis manual testing. across the U.S., including 18 of discovering the “happening” restaurants • 80% testing coverage achieved the top 30 markets. The Web and bars in town and the latest music, through automation. sites attract more than 9 million movies and entertainment informa- unique visitors per month, and tion within most cities in the U.S. We • 60% reusability of automation rank among the top 20 interac- provided an automation solution for the scripts drastically reduced tive news/information networks local entertainment guide application: timelines. in the country. • Over 40% savings on time • New set of automation scripts/functions executed as a result of This client is a leader in distrib- created to completely automate existing automation scripts executed on uted contact centers, enabling Manual Test Cases using Selenium RC multiple browsers. businesses to deliver a high- 0.9 with Ruby script. • Dynamic collection and organiza- performance call center and • Star Quality Framework implemented tion of test cases. high quality customer contact in to suit application under test. the most cost-effective manner • Easy tracking of results and • Scripts executed on IE 6,7 and FF 3.5 possible. metrics associated with test • We performed an analysis on open execution. The company provides best- source test management tools for this • Capturing and reporting details in-class on-demand call center project, as per the client’s request. The by conducting a more thorough technology, as well as virtual tools taken in for analysis included: testing process. call center services through > Testopia. a network of over 20,000 • Customized TestLink to fit project > TestLink. requirements and processes. independent home agents to hundreds of companies in both We analyzed details for the requested direct response and enterprise tools and recommended TestLink for test markets. management, as it enabled: • Tree structured test projects, plans, suites and cases. • MySQL back-end. • XML RPC API provides easy integration points with external systems. • Easy export of test cases to migrate to another system, if needed. cognizant 20-20 insights 3
  4. 4. Approaches to Adopt Open Source Testing Tools End-to-End Integration of Open Source Tools In the scenario of trimmed budgets and diligent IT spending, cost associated with purchasing and maintaining commercial testing tools is under constant scrutiny in every organization. One premium alternative to commercial tools is to leverage the growing end-to-end open source testing tools. Considerations on Moving from Commercial to Open Source Tools Enterprises involved in making a shift from commercial to open source tools face the challenge of migrating existing data, as there are no seamless individual connectors. However, the hybrid approach is recommended for optimized utility of tools. Hybrid Option: Commercial + Open Source Tools Phase1: • Identify the reliable open source alternative for the commercial tool. • Pilot and scale the usage of open source tools in place of commercial tools. Phase 2: • Use the commercial tool in areas where there are no reliable OS tools. • Develop integration accelerators between commercial and OS tools for smooth operation.Figure 2Phase 1: Analysis End-to-end deployment of open source tools willAlternative open sources tools must be identified involve analyzing different tools across phasesas substitutes for commercial off-the-shelf tools. that will integrate with each other. With the rightIn doing so, certain analyses — such as proof mix of tool sets, IT organization can leverageof concepts, feasibility studies, etc. — must be end-to-end testing functionalities.performed for technical compliance. For example, Challenges/Considerationsreplacing commercial automation tools with afunctional tool such as Selenium or changing the QA organizations need to consider the followingdefect management tool to Bugzilla will reduce key aspects of the open source testing toolthe number of test management tool licenses. deployment, configuration and usage process:Phase 2: Pilot • Most tools are built to serve a specific purpose; however, availability of source code meansOnce the open source tools are identified, the next organizations can customize the tools asstep is to pilot and scale the usage of these tools required, with some the place of commercial tools. While doing so, itis important to understand and test the migration • Bugs encountered can cause major issues when a project is under way; when ample support isof data from the existing test. provided by vendors for respective commercialPhase 3: Recommend and Implement tools, tool support is limited to general forums and the tools community.The identified open source tools are now readyto be implemented. A thorough recommendation • Administration and maintenance coststhat involves the process to migrate, the tools increase with increase in project complexityto adopt and also the timelines for a complete and size; most organizations tend to neglectmigration will be provided in this stage. The rec- this component while budgeting and end upommendation should involve ROI calculations, with major deviations.capturing risks involved in migration to open • Tools have non-intuitive user interfaces, whichsource, along with the necessity for training QA makes it difficult to provide dedicated training.staff. However, training material is usually available in forums, communities and blogs. cognizant 20-20 insights 4
  5. 5. Serving Commercial Operations XPlanner Redmine Jira Bugzilla Testopia Project Management and Tracking Defect Management Concordion Cucumber Fitnesse Acceptance Testing Cruise Control Hudson Testing Process Continuous Integration Application Under Test WatiR Sahi Selenium Functional TestingFigure 3• Different modes of open source licensing • They help fill in the gaps in the life cycle, par- mechanisms like GPL, LGPL, etc. need to be ticularly if IT has process maturity relative to understood and appropriately used. its overall approach to application life-cycle management solutions.It can be difficult to directly integrate open sourcesoftware with other tools; this is one of the limi- • Open source tools play a significant role intations of open source. Limited reporting options product development, especially in Agileand limited derivation of metrics are other development processes where time-to-marketproblems for QA groups and IT management. is paramount.Recommendations and Conclusion • Many QA organizations look forward to blending their processes with open source and• Open source tools can drive significant quality commercial testing tools. A strong mixture of and predictability in the testing life cycle, both would be the best to ensure cost optimi- thereby reducing overall application develop- zation and testing reliability. ment costs by keeping tool investments lower.About the AuthorsRamakrishnan Venkatasubramanian heads Cognizant’s Test Automation R&D and Offshore MobileTesting groups. He has more than 12 years of experience in the information technology industry, spanningsoftware development, product engineering, testing, test automation and technology consulting. He canbe reached at Mahajan works within Cognizant’s Test Automation Center of Excellence and heads the AutomationSolutioning team. He has more than seven years of experience in the IT industry. He can be reached cognizant 20-20 insights 5
  6. 6. About CognizantCognizant (NASDAQ: CTSH) is a leading provider of information technology, consulting, and business process out-sourcing services, dedicated to helping the world’s leading companies build stronger businesses. Headquartered inTeaneck, New Jersey (U.S.), Cognizant combines a passion for client satisfaction, technology innovation, deep industryand business process expertise, and a global, collaborative workforce that embodies the future of work. With over 50delivery centers worldwide and approximately 118,000 employees as of June 30, 2011, Cognizant is a member of theNASDAQ-100, the S&P 500, the Forbes Global 2000, and the Fortune 500 and is ranked among the top performing andfastest growing companies in the world. Visit us online at or follow us on Twitter: Cognizant. World Headquarters European Headquarters India Operations Headquarters 500 Frank W. Burr Blvd. 1 Kingdom Street #5/535, Old Mahabalipuram Road Teaneck, NJ 07666 USA Paddington Central Okkiyam Pettai, Thoraipakkam Phone: +1 201 801 0233 London W2 6BD Chennai, 600 096 India Fax: +1 201 801 0243 Phone: +44 (0) 20 7297 7600 Phone: +91 (0) 44 4209 6000 Toll Free: +1 888 937 3277 Fax: +44 (0) 20 7121 0102 Fax: +91 (0) 44 4209 6060 Email: Email: Email:© Copyright 2011, Cognizant. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by anymeans, electronic, mechanical, photocopying, recording, or otherwise, without the express written permission from Cognizant. The information contained herein issubject to change without notice. All other trademarks mentioned herein are the property of their respective owners.