2009 Second International Conference on Computer and Electrical Engineering


                   Classification of Softwar...
mine the load under which it fails and                           In this paper, we will classify the soft-
               ...
Fig. 1: Distribution of the used testing tools over the software types.

                 Hereinafter, we will focus on th...
For the second type of software prod-                                While, the smallest number of testing
              u...
4. Conclusion                                                       overcome limitations and restrictions of
             ...
Upcoming SlideShare
Loading in...5
×

Classification of Software Testing Tools Based on the ...

1,440

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,440
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
19
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Classification of Software Testing Tools Based on the ...

  1. 1. 2009 Second International Conference on Computer and Electrical Engineering Classification of Software Testing Tools Based on the Software Testing Methods Khaled M. Mustafa, Rafa E. Al-Qutaish, Mohammad I. Muhairat Al-Zaytoonah University of Jordan, Airport Street, Amman, Jordan Abstract procedures on approved test cases [1]. Furthermore, the software testing is de- During the software development life cy- fined in [2] as "the dynamic verification cle, testing is highly needed to assure the of the behavior of a program on a finite quality of the software process and prod- set of test cases, suitably selected from uct. For many years, researchers and the usually infinite executions domain, practitioners came-up with a variety of against the expected behavior". However, software tools to automate the testing the software testing process can be as- process. sisted with software tools to make it au- In this paper, we will classify and dis- tomated. tribute a set of testing tools over the types In this paper, we will classify a set of of testing (testing methods) for three testing tools and distribute them over the types of software products (web applica- types of software products. Furthermore, tion, application software, and network we will distribute a set of testing tools protocol). However, the outcomes of this over the types of testing for three types of classification are dedicated to the re- software products. However, the out- searchers and practitioners in software comes of this classification are dedicated testing to know which types of testing to the researchers and practitioners in have no or limited automated tools. software testing to know which types of testing have no automated tools. Keywords: Software Testing Tools, The rest of this paper is organized as Classification, Types of Software, CASE. follows: section 2 presents an overview of the types of software testing which is used in this paper. Section 3 discusses the 1. Introduction classification and distribution of the soft- Through the life cycle of the software de- ware testing tools over the types of soft- velopment, testing is highly needed to ware testing. Finally, section 4 concludes assure the quality of software process and the paper. product. Many years ago, the researchers come-up with a variety of software tools 2. Types of Software Testing to automate the testing process. Software testing is a formal process Although there are many types of soft- carried out by a specialized testing team ware testing, this paper will only include in which a software unit, several inte- the following types: grated units or an entire software package • Stress Testing: it is the testing which are examined by running the programs on conducted to evaluate a system or a computer. All the associated tests are component at or beyond the limits of performed according to approved test its specified requirements to deter- 978-0-7695-3925-6/09 $26.00 © 2009 IEEE 229 DOI 10.1109/ICCEE.2009.9 Authorized licensed use limited to: Naval Surface Warfare Center (Dahlgren). Downloaded on March 10,2010 at 14:42:46 EST from IEEE Xplore. Restrictions apply.
  2. 2. mine the load under which it fails and In this paper, we will classify the soft- how. Often this is performance testing ware products based on their intended us- using a very high level of simulated age, that is: load. Performance testing is often done in conjunction with stress testing 1. Application Software. [3]. 2. Database. 3. Network Protocol (TCP). • Load Testing: it is the testing which 4. Open Source Software. conducted to evaluate the compliance 5. System Software. of a system or component with speci- 6. Embedded Software. fied performance requirements. [3]. 7. Web Application. • Regressions Testing: it is the testing 8. Java Software. which is to be done to software that was previously working correctly and We have collected 135 software testing stops working as intended due to tools from the internet [5, 6, 7, 8, 9, 10]. changes [4]. However, after studying and analyzing • Functional Testing: it is the testing these software testing tools, we have clas- which is conducted on a complete and sified them into different types based on integrated system to evaluate its com- which software product could be applied. pliance with its specified requirements Figure 1shows this classification. [4]. From figure 1 we can note that the web • Unit Testing: it is the verification and application software products have the validation technique where the pro- largest number of software testing tools grammer gains confidence that indi- (63 software testing tools), this large vidual units of source code are fit for number of testing tools may be due to the use [4]. wide usage of web applications on the • Performance Testing: it is the testing web sites, and are common to be devel- which refers to the assessment of the oped to provide a variety of services to performance of a human examinee. the intended users of such web sites. [4]. Whereas, the system software and em- • Acceptance Testing: it is the testing bedded software products have the small- which involves running a suite of tests est number of software testing tools (2 on the completed system [4]. software testing tools for each of them). • Security Testing: it is the testing Furthermore, the following is the order which determines that an Information of software products types based on how System protects data and maintains many software testing tools are intended functionality as intended [4]. to be used by such software product • Open Source Testing: it is a func- types: tional and/or unit testing framework 1. Web Application (63 tools). for open source software products [4]. 2. Network Protocol (TCP) (27 tools). 3. Application Software (18 tools). 3. The Classification of the Software 4. Java Software (16 tools). Testing Tools 5. Open Source Software (10 tools). 6. Database (7 tools). Software products can be classified to 7. System Software (2 tools). different types based on different criteria 8. Embedded Software (2 tools). (intended usage, complexity, develop- ment technology, etc.). 230 Authorized licensed use limited to: Naval Surface Warfare Center (Dahlgren). Downloaded on March 10,2010 at 14:42:46 EST from IEEE Xplore. Restrictions apply.
  3. 3. Fig. 1: Distribution of the used testing tools over the software types. Hereinafter, we will focus on three be used with the web applications, we types of software products, that is, web found that most testing tools are designed application, application software, and to be used for the functional testing. Fig- network protocols (TCP). Also, we se- ure 2 presents the distribution of the test- lected the common used types of software ing tools over the types of testing for the testing, that is, stress, load, regression, web applications. In addition, the smallest functional, unit, performance, acceptance, number of testing tools is for the unit, ac- security, and open source testing meth- ceptance, and open source testing meth- ods. ods. We should note that there are many However, after we classified the 63 testing tools which can be used for differ- software testing tools which are built to ent types of testing. Fig. 2: Distribution of the testing tools for web application over the types of testing. 231 Authorized licensed use limited to: Naval Surface Warfare Center (Dahlgren). Downloaded on March 10,2010 at 14:42:46 EST from IEEE Xplore. Restrictions apply.
  4. 4. For the second type of software prod- While, the smallest number of testing uct (application software), we found that tools are built for the system and security there are 18 software testing tools. After testing methods. we studied and analyzed these testing Figure 4 presents the distribution of the tools, we found that the large number of testing tools for application software over testing tools is intended to be used for the the nine types of testing. functional testing. Fig. 3: Distribution of the testing tools for application software over the types of testing. For the networks (TCP protocol), there Furthermore, there is no testing tool for are 27 testing tools. However these test- the system, acceptance, unit, functional, ing tools are mostly used for the perform- and regression testing methods. This is ance, security, load, and stress testing due to that network protocols are special methods, see figure 4. purpose software products. Fig. 4: Distribution of the testing tools for networks (TCP) over the types of testing. 232 Authorized licensed use limited to: Naval Surface Warfare Center (Dahlgren). Downloaded on March 10,2010 at 14:42:46 EST from IEEE Xplore. Restrictions apply.
  5. 5. 4. Conclusion overcome limitations and restrictions of the current software testing tools. In the software development life cycle, testing is highly needed to assure the quality of the software process and prod- 5. References uct. For many years, researchers and [1] D. Galin, Software Quality Assur- practitioners come-up with a variety of ance: From Theory to Implementa- software tools to automate the testing tion, Addison Wesley, New York, process. NY, USA, 2003. In this paper, we have classified and [2] ISO, ISO/IEC TR 19759: Guide to distributed 135 testing tools over eight the Software Engineering Body of types of software products. Furthermore, Knowledge (SWEBOK), Interna- we have distributed the testing tools over tional Organization for Standardiza- the types of testing for three types of tion, Geneva, Switzerland, 2005. software products (web application, ap- [3] S. Chat, Performance Management of plication software, network protocol) Software Architecture, online: However, the outcomes of this classifi- http://www.findwhitepapers.com/whit cation are dedicated to the researchers epaper2373/, visited on July 4, 2009. and practitioners in software testing to [4] G. J. Myers, T. Badgett, T. M. Tho- know which testing methods have no au- mas, and C Sandler, the Art of Soft- tomation tools. In addition, this classifica- ware Testing, Wiley, USA, 2004. tion gives a direction on which types of [5] ApTest, Web QA Test Tool Links, tests have limited automated tools. online: http://www.aptest.com/webres Based on our study and analysis of the ources.html, visited on April 15, testing tool, we have concludes the fol- 2009. lowing comments and suggestions: [6] INSECURE, Top 100 Network Secu- 1. The testing tools for the embedded rity Tools online: http://sectools.org/, and system applications are very lim- visited on April 20, 2009. ited. [7] Java-Source, Open Source Testing 2. For the web application software Tools in Java, online: http://java- products, the testing tools for the unit, source.net/open-source/testing-tools, acceptance, and open source testing visited on April 20, 2009. methods are restricted. [8] Ranorex, Web Testing, online: 3. For the application software products, http://www.ranorex.com/support/user the testing tools for the security and -guide-20/web-testing.html, visited system testing methods are restricted. on April 22, 2009. 4. The testing tools for functional testing [9] Bright-Hub, Sniffing Data with Etter- in both the web applications and ap- cap for Linux and Windows, online: plication software product are very http://www.brighthub.com/computing common. /smb-security/articles/35545.aspx, vi- 5. For the network software products sited on April 22, 2009. (TCP protocol), there is no any testing [10] QFS, Facts & Features, online: tool for the system, acceptance, unit, http://www.qfs.de/en/qftest/, visited functional, and regression testing me- on April 23, 2009. thods. However, the above comments could be taken into account by the researchers and practitioners to build set of tools to 233 Authorized licensed use limited to: Naval Surface Warfare Center (Dahlgren). Downloaded on March 10,2010 at 14:42:46 EST from IEEE Xplore. Restrictions apply.

×