'Mixing Open And Commercial Tools' by Mauro Garofalo


Published on

Mixing open source and commercial software is a challenge we face today. The right combined solution offers advantages in flexibility, functionality, performance, and management that aren't available when either open source or commercial technologies are used alone. But one of the major issues is that they don’t always play well together. Some of them can’t be loaded together or they fail to integrate properly.

In this presentation we will provide a case study of successful blending open source and commercial software for testing Java applications. The testing environment consisted of using Subversion as versioned repository for tests, JIRA for issue management and in mixing IBM Rational Functional Tester with an open framework for automated functional, regression and GUI testing. These tools provide a rich set of high-level Java API useful for integrating with each other, by minimizing the integration costs. During the presentation, we will explore the costs (open tools used have no fees, integration and training costs …), the advantage (innovation, products that are best-of-breed …), the risks (dedicated support, community feedbacks, ability to add extensions …) and the product’s quality achieved by our solution compared to a full open source or commercial approach. By our experience, we will provide some hints and tips to guide testers and managers to choose a good mix between open and commercial tools based on: budget, technology, know-how and requested quality.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

'Mixing Open And Commercial Tools' by Mauro Garofalo

  1. 1. Mixing Open and CommercialToolsMauro Garofalo, MaveryxW22 – Automation
  2. 2. Summary Costs, Advantages and Risks of mixing opensource and commercial software A case study of blending open source andcommercial tools for testing Java applications Some practical rules to combine open andcommercial test tools2All trademarks referenced herein are the properties of their respective owners.
  3. 3. So, what is Open Source? First, no official definition exists (http://www.opensource.org/osd.html) Open source doesnt just mean access to the sourcecode Open source rules are, simply enough: Allow free redistribution Allow source code access Permit derived works Protect the integrity of the author‟s source code ….3
  4. 4. Key Predictions By 2012, at least 80% of all commercial software solutions willinclude elements of open source technology (Gartner Highlights KeyPredictions for IT Organizations and Users in 2008 and Beyond) By 2010, Global 2000 IT organizations will use open-source productsin 80% of infrastructure-focused software investments and 25% ofbusiness software investments (Gartners Positions on the Five Hottest ITTopics and Trends in 2005) …4
  5. 5. Survey: Do you use open source software? Do you use… Firefox to browse the web? OpenOffice to write documents? Linux or Ubuntu as server or end user OS? Subversion or CVS for the source code? MySQL as database? Eclipse to write code? FileZilla for file transfer? … http://downloadpedia.org/Open_Source_Alternative_to_Commercial_Software5
  6. 6. Companies use open source because… COST is the main reason why companies are turning to open source[Jeffrey Hammond, Forrester Research] But cost is not the only reason for open source‟s growing popularity:many firms now know that it offers more FLEXIBILITY thanproprietary programs [Matthew Aslett, 451 Group] Improving quality of products a/o processes Achieving faster time-to-market Higher level of security …6
  7. 7. OSS vs. Commercial Commonly it is framed as Linux vs. Microsoft Security vs. Innovation7 Rewards of OSSavailability of source codefreedom to customize, enhance orrepairlower price, sometimes freedoes not depend on vendorsecurity… Riskslack of support, training and docsfrequent updates & forksbackwards compatibility… Rewards of Commercial softwarevendor professional serviceseasier to adopt in organizationscomply with industrial standardsmore user friendlymature products offer morefeatures… Riskscostsdependent upon vendordo not allow users to alter orcustomize the product.…
  8. 8. A ‘mixed source’ strategy The greatest chance for success with open source software involves astrategy of mixing open with commercial software A mixed-source strategy means identifying and integrating both openand commercial products that are best-of-breed technologies fits the requirements minimizes integration „headaches‟rather than using “good enough” products included in legacy solutions or available (for free…) on SourceForge!8
  9. 9. Integration Mixing open & commercial tools requires integration and interoperability Integrating any two programs is often challenging… Integrating commercial software & OSS can be difficult because: variety of technologies (commercial) tools use proprietary „approaches‟ data integration is not friendly (OpenOffice vs. Microsoft Office file formats) no design for interoperability different products have varying requirements for integration many products are intended for use with specific combinations of tools or designated platforms … The result is that integration of OSS with commercial offerings is often partialor missing completely, or must be provided by a third-party vendor…9
  10. 10. Test Tools Application Test Tools for: Source testing Functional & GUI testing Load & Performance testing Embedded testing Database testing … Supporting Tools for: Test Management Bug Tracking Requirements Management Configuration / version control …10
  11. 11. Can they cooperate?11 Open Source  Commercial
  12. 12. Case Study Testing a Java™ stand-alone, GUI-based application for theTelecom market The application-under-test has „legacy‟ functionalities (taken from past project / application) new features continuously changing (specially at early stage) Reusing of many automated functional tests built with IBM RationalFunctional Tester for legacy functionality testing12
  13. 13. The Test Environment13
  14. 14. Subversion Subversion is a free & open source version controlsystem Industry standard Easy to use Local baseline Atomic commits One repository to maintain (single url for all projects) … There are several plug-ins implementing SVN support intoEclipse (→ Subversive)14
  15. 15. JIRA JIRA is a commercial issue, bug and project tracking tool Easy to use Highly configurable (e.g. to manage a complex workflow) Flexible for different kind of issues (e.g. for test reporting) Accessible from tests scripts through Java API Extensible and customizable (source code is available to customers) Eclipse integration Subversion integration Fully supported Quite cheap …15
  16. 16. Eclipse Eclipse is a free & open source development environmentcomprising an IDE and an extensible plug-in system Industry standard Fully supported Extensible and configurable more than 1200 plug-in available at theMarketplace (most free!) Simple and powerful Workbench → Views, Editors and Perspectives …16
  17. 17. IBM Rational Functional Tester Rational Functional Tester is a commercial toolfor automated functional and regression testing Built-in support for Java applications Java scripting language Eclipse rich client application ScriptAssure technology to accommodate UI changes Data-driven & Keyword-driven testing …17
  18. 18. Maveryx Maveryx is a free & open source tool for automatedfunctional, regression, GUI and data-driven testing Built-in support for Java applications No „GUI Map‟ needed to create and run tests Intelligent GUI objects recognition (including fuzzy matching) Available as Eclipse plug-in Extensible and configurable through plug-ins Integrated with all Java IDEs and testing frameworks (Eclipse,NetBeans, JUnit, IBM Rational Functional Tester, …) …18
  19. 19. Test Strategy Legacy functionalities tested by reusing test scriptsdeveloped with IBM Rational Functional Tester Needed to update / re-capture the GUI Maps for thecurrent application New functionalities tested by using Maveryx Maximizing the Rational Test Manager usage Refactoring of some tests by mixing, wherepossible, Rational code with Maveryx code19
  20. 20. Mixing open & commercial tools You must decide which components be open vs. closed source A possible strategy: closed source for “commodity” components open source for “differentiating” components Following such a strategy: classify features as commodity or differentiating determine for each feature whether using open or closed SW based on: budget technology know-how requested quality20
  21. 21. Evaluation Process21Identify candidatesRead existing reviewsCompare attributes to your needsAnalyze top candidatesWrap-up
  22. 22. Identify Candidates First step : find out your options Search for open and commercial tools: ask friends and co-workers search using specialized sites use search engines thumb specialized magazines …22
  23. 23. Read existing reviews After youve identified your options, read existing evaluations about thealternatives: use search engines search for specialized web sites read blogs, forum thumb specialized & independent magazines look at social networks …23
  24. 24. Evaluation Criteria24TOOL #1 TOOL #2 TOOL #3FunctionalityCostSupportMarket ShareMaintenance/ LongevityReliabilityPerformanceScalabilityUsabilitySecurityFlexibility /CustomizabilityInteroperability…
  25. 25. Analyze top candidates After the initial evaluation, pick the top candidates, and perform a morein-depth analysis Try any contender in non-critical situations first, by evaluating thefunctionality that you‟re interested in using (use a checklist) Examine also the program‟s documentation, source code (for OSS),and other related materials If the tool had some but all the functions you need, examine what itwould take to add those functions25
  26. 26. Evaluating Software In general, a mature platform will meet the followingcriteria: Project extensions are available The project has reached a one-year maturity mark Security patches, bug fixes and new features/enhancements are deliveredseparately The software has reasonable automated unit and functional tests with codecoverage in the 30 percent - 80 percent range The software easily integrates with external tools The component‟s bug database is kept up-to-date with revision numbers for eachproduct enhancement The solution has been ported across multiple platforms (Linux, Windows, Solarisand Mac) Large-scale adoption, including both public and well-known large-scaleorganizational deployments exists There is documentation purpose separation: User guide (or Getting StartedGuide), Installation guide, Admin guide and Development guide26
  27. 27. Important factors influencing testing tools Factors influencing the choice of Software Testing Tools Programming Language Support Long feedback Evaluation Period Selling is driven by Marketing Vendor locking Specialization & Generalization …27
  28. 28. Wrap-up Review and present to your manager the results of your evaluation Identify the recommended solution Identify the main alternatives, first of all the recommended alternative Once a decision has been made, get the software & go!28
  29. 29. Conclusion A “mixed-source” approach combines open source & proprietary software,by taking the best of both worlds The right mixed solution offers advantages in flexibility, functionality,performance, and management that arent available when either opensource or commercial technologies are used alone But, integrating commercial software and OSS can be difficult… For testing purposes, evaluate open source and commercial tools by makingdecisions based on cost, technical and integration factors29
  30. 30. 30info@maveryx.com