Fitnesse, Watir and Ruby Based Test Automation System

HSTW-3a01 Fitnesse, Watir and Ruby Based Test Automation System A Harbinger Systems White Paper

Harbinger Systems Your partner in technology innovation About Harbinger Systems: Harbinger Systems (www.harbinger-systems.com) is a leading provider of software engineering services to some of the world's best product companies. This white paper is part of Harbinger’s Software Product Development practice. Harbinger provides software product development and testing services to service the needs of enterprise businesses as well as consumer products. Visit our Software Product Development Practice page to learn more. Visit our website to download or request our white papers on leading edge technologies and trends. Contents Executive Summary.............................................................................. 3 Current Test Automation Technologies ................................................ 3 Open Source Technologies ..................................................................................................... 3 Proposed Test Automation System ...................................................... 4 System Architecture ............................................................................................................... 5 Test Execution ........................................................................................................................ 6 Compatibility .......................................................................................................................... 7 Benefits ............................................................................................... 8 Copyright © Harbinger Systems. All Rights Reserved Page 2 of 8 www.harbinger-systems.com

Executive Summary Test Automation is a strategic necessity, as the delivery cycles are shrinking and becoming repetitive. In this white paper, we propose and illustrate a complete test automation solution based on open source technologies, Fitnesse, Ruby and Watir. This system is web based, and enables a diverse set of project stakeholders to carry out automated testing from anywhere. After a brief introduction to the available options in open source test automation technologies, we will look at the building blocks of the solution, and finally the details of the proposed test automation system and its application to the testing of Web based applications. Current Test Automation Technologies The current automation solutions are available as both proprietary and open source tools. While it is generally thought that an end-to-end test automation can be achieved using proprietary tools like QTP, Rational Robot etc. very easily, as the application undergoes revisions over time, many of these systems begin to fall short on their promise. For one thing, maintenance of the test scripts becomes expensive. Added to that, there are issues with them like licensing restrictions, support for scripting language, pricing and dependencies for add-ons. Sooner or later, due to these restrictions, the expensive automation ware rests on the shelf. Open Source Technologies Test Automation has been the buzzword in the open source forums. The Agile Development methodologies insist on automated testing as a crucial component in the success of projects. This has led to the development of several test frameworks for different stages of software development life cycle. For the sake of this discussion, these frameworks can be broadly categorized into two categories: Unit Test Frameworks and Systems Testing Frameworks. Unit Test Frameworks The most popular unit testing frameworks are JUnit or NUnit based on the application technology in question. These frameworks are robust and they provide an engineering view of software testing. The primary drawback with these frameworks is that these are too technical to be used by all project stakeholders. Moreover, these frameworks perform white box testing over the classes in codebase, and hence, do not provide sufficient information for the system level tests and application behavior. Copyright © Harbinger Systems. All Rights Reserved Page 3 of 8 www.harbinger-systems.com

System Testing Frameworks Selenium has traditionally been a well known open source framework for system testing. Selenium executes tests within the browser. The selenium IDE also provides easy access to test cases. In Selenium, however, JavaScript’s same origin policy is seen as the show stopper. To deal with this problem, the framework was enhanced with a proxy server (Selenium RC) where the automation test suite is to be deployed. The browser, where actual tests are being run, will then point to the proxy server which will redirect the request to application server. While this configuration works, there are two main issues: The configuration is not exactly the same as the actual test environment in production. The overall installation and deployment of this configuration is very heavy and time consuming. Proposed Test Automation System Harbinger Systems’ proposed test automation system is built on the following building blocks. Fitnesse An open source framework called Fit is used for integration testing. Fitnesse is a Wiki wrapper over the Fit server, and thus provides a web interface to the test suite. The front end of Fitnesse is intuitive and simple, and it enables customers, testers and programmers to compare what their software actually does against what it should do. Very importantly, Fitnesse is thus seen as a software development collaboration tool since all the stakeholders can collaboratively define the acceptance tests, run those and see the results. Test cases in Fitnesse are written in Excel-like tables on wiki pages. Fitnesse provides different types of fixture libraries to perform data-driven or action-driven testing. It must be noted, however, that Fitnesse was written to test the application code and not application UI. Hence, it requires interfacing code, which can call the functions of the application testing class. Ruby Ruby is an open source object oriented programming language with a focus on simplicity and productivity. It doesn’t run very fast, but it saves a lot of programming time. Further, Ruby code is very concise and readable and is much simpler to maintain. These attributes make it a good programming language to write the automation scripts in. Copyright © Harbinger Systems. All Rights Reserved Page 4 of 8 www.harbinger-systems.com

Watir Watir, pronounced water, stands for Web Application Testing in Ruby, and is an open source library of functions used to automate browser based tests. Watir provides programming API for Ruby to drive browser actions. While Watir traditionally supported Internet Explorer, it now also supports Firefox and Safari, and enjoys a strong community support. System Architecture Figure 1: Schematic diagram of Harbinger's Proposed Test Automation System The system is architected such that all tests pertaining to a module are packaged and saved in separate folders. All testing related decisions are made by a class called Test Engine, which is the heart of the Copyright © Harbinger Systems. All Rights Reserved Page 5 of 8 www.harbinger-systems.com

system, and is responsible for maintaining the context of testing. It is also a completely open source solution, and hence, does not have licensing restrictions from its subcomponents. In the proposed system, all the common navigational functionalities are abstracted in a separate class. This is then sub-classed into module specific exploratory test functions. All the test wares are developed in Ruby. A schematic diagram of this system is shown in Figure 1. All artifacts of the automation system are deployed on the automation server machine. The system is carefully architected after a long research on abstracting and organizing the test procedures into various components. These components and their responsibilities are as follows: Fixtures: o Parses test data from Test cases in Wiki pages and to fetch them into test engine o Compares the outcome of test execution with the expected behavior and to mark the test case as pass or fail Test Engine: Test Engine is the intelligent class of the system, which makes decisions during testing, and keeps the test context of the system persistent over the next test case / scenario Helper classes: Helper classes help the Test Engine in exploring the application in conformance to the test execution steps described in the test case Test Execution The test automation server can be accessed from a browser, and the user can initiate testing just by clicking the test button in the test scenario page of the wiki. Fitnesse would start executing tests as per the procedure described in test cases and will report the outcome in the same test tables as shown in Figure 2. Copyright © Harbinger Systems. All Rights Reserved Page 6 of 8 www.harbinger-systems.com

Figure 2: Outcome of test execution Table cells filled with green indicate success. When the test fails, the fixture class finds that the actual state of the application is not the same as expected. The corresponding cell in the test case is then marked in Red. The actual state of application is also shown in the same cell. Compatibility The testing system has been successfully tested on Web 2.0 applications with the following advanced attributes: Web elements in nested framesets Ajax controls Element identifiers created at runtime by Ext-JS Copyright © Harbinger Systems. All Rights Reserved Page 7 of 8 www.harbinger-systems.com

Benefits The proposed Test Automation System yields the following advantages: Project stakeholders can access the test automation server via web Test outcome clearly shows what the application is expected to do and what it actually does. Thus, the system is an ideal collaboration tool for teams located geographically apart As testing becomes very simple and can be executed by any team member, this system creates ground for agile development cycles Open source based automation solution keeps the system free from licensing restrictions Modular design of test wares that is in conformance with modules of system testing designed by manual test teams Decision making business function classes have been isolated from the navigational details in separate classes The Test Engine component keeps context of test execution, and hence, can take appropriate decisions in the next test scenario execution cycle Learn More: Visit our website, http://www.harbinger-systems.com to download more white papers on leading edge technologies and trends. Sources 1 The Fitnesse Acceptance Testing Framework: http://fitnesse.org/ 2 Using Fitnesse pages effectively: http://gojko.net/2008/10/21/using-fitnesse-pages-as-templates/ 3 Architectures of Test Automation Systems - Cem Kaner: http://www.kaner.com/testarch.html 4 Getting Automated Testing Under Control – Hans Buwalda: http://www.logigear.com/logi_media_dir/Documents/whitepapers/Getting_Automated_Testing_Under_Control.pdf 5 Watir: http://watir.com/ 6 Using Watir for Test Automation: http://svn.openqa.org/fisheye/browse/~raw,r=267/watir/trunk/watir/doc/watir_user_guide.html 7 Firewatir – The Support of Watir for Firefox: http://angrez.blogspot.com/search/label/Firewatir 8 On Ruby Programming Language: http://www.ruby-lang.org/en/ Copyright © Harbinger Systems. All Rights Reserved Page 8 of 8 www.harbinger-systems.com

Fitnesse, Watir and Ruby Based Test Automation System

by Harbinger Systems

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 1

Statistics

Fitnesse, Watir and Ruby Based Test Automation System Document Transcript