QUICK LOOK
the browser preferences dialog box.       operating environment. More com-             server configuration issue. However...
more of them. For example, with OLE-                                                            ty issues that are similar...
Dynamic Environments (i.e., resource
                                                                 Physical Server     ...
I am not suggesting that we read the code                                     ment cannot be replicated (due to
s Error detection and handling for potential
       duplication be performed on the client-          ...
       side using...
activities. The dynamic operating en-          fect your ability to replicate errors.                are investigating, bu...
Upcoming SlideShare
Loading in …5

Testing Web Based Applications[1]


Published on

Published in: Technology
1 Comment
  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Testing Web Based Applications[1]

  1. 1. Testing QUICK LOOK s Why errors in Web-based Testing applications are hard to reproduce s Analyzing these errors in a way that saves testing time Web-based Applications Analyzing and reproducing errors in a Web environment by Hung Q. Nguyen he testing of Web-based done … especially in Web environ- T 1 applications has much in ments. Web environments are dense common with the testing with error-prone technology vari- of desktop systems: You ables. Here are five fundamental con- need to test the usual func- siderations of Web-application test- What are we really tionality, configuration, ing: and compatibility, as well as perform- seeing–an error ing all the standard test types. But 1. When we see an error on the Web application testing is more diffi- client side, we are seeing the or a symptom? cult because complexities are multi- symptom of an error—not the er- plied by all the distributed system ror itself. Without diagnosing the environment, components that interact with the ap- we can’t be certain what causes a plication. When we see an error in a 2. Errors may be environment-de- symptom to appear. If one of the en- Web environment, it’s often difficult pendent and may not appear in vironment-specific variables from ei- to pinpoint where the error occurs, different environments. ther the client side or the server side and, because the behavior we see or is removed or altered, we might not the error message we receive may be be able to reproduce the problem. 3. Errors may be in the code or in the result of errors happening on dif- Here is an example. I’m testing a the configuration. ferent parts of the Web system, the er- Web-based defect tracking applica- ror may be difficult to reproduce. So tion, and going through the process how do we analyze errors within a 4. Errors may reside in any of sever- of creating a new bug report. When I Web-based system, and what consid- al layers. select the NEW button, I receive an erations should be made for reproduc- error message: ing such errors? 5. Examining the two classes of op- When we have an understanding erating environments — static Microsoft OLE DB Provider for ODBC of the underlying technology, we are versus dynamic—demands dif- Drivers error '80040e14' better able to maximize testing effi- ferent approaches. ciencies—writing more reproducible After spending some time investigat- bug reports and finding more errors Now let’s take a look at each of these ing my browser environment, I dis- in less time. This is easier said than five considerations in more detail: cover that JavaScript is disabled in 23 May/June 2000 Software Testing & Quality Engineering www.stqemagazine.com
  2. 2. the browser preferences dialog box. operating environment. More com- server configuration issue. However, Enabling JavaScript eliminates the monly, we refer to environment-in- if the installation program failed to error. (Whether or not this is a bug is dependent errors as functionality- programmatically configure the Web not part of this discussion.) The idea specific errors. server according to specification, here is that if I add additional infor- then this is a software error. If a mation regarding the JavaScript set- system administrator fails to proper- ting to the bug report, I can save our ly configure the Web server accord- team some time in analyzing this problem. Furthermore, “disabling JavaScript” is added to my test suite 3 ing to specification, this then be- comes a user error. from this point on; it will be applied to all areas of the application so that Is it a coding error Application directory has not been con- figured properly to execute scripts A all potentially related errors can be uncovered. or a configuration typical application-server directory contains scripts to be executed when problem? they are called by a Web server on the behalf of a client. For security Errors (or the symptoms of sup- reasons, a Web server can be config- 2 To reproduce an environment-dependent error Is the error environment- we have to perfectly replicate both the exact sequence dependent? of activities and the environment conditions. To reproduce an environment-de- pendent error we have to perfectly posed errors) may be resolved with ured to allow or disallow scripts to replicate both the exact sequence of code fixes (assuming the errors are be executed within certain directo- activities and the environment con- in fact real) or system reconfigura- ries. If your application-server direc- ditions (operating system, browser tion (client, ser ver, or network). tory is designed to contain scripts version, add-on components, data- Don’t jump too quickly to the conclu- that will be executed—but the Web base server, Web server, third-party sion that it’s a bug! server is configured to disable script components, ser ver/client re- execution in that directory—the ap- sources, network bandwidth and Microsoft OLE DB Provider for ODBC plication will not work. Is this a soft- traffic, etc.) in which the application Drivers error '80004005' ware error or a configuration operates. For example, when you try problem? to log into your Web application Here is an example illustrating the while using a 28.8 kbps dial-up con- challenge of identifying possible Default Web page has not been set up nection, you experience login fail- configuration problems as opposed properly The issue is similar to the ures due to timeout in the authenti- to actual software errors. It shows problem above. cation process—but the same login an error message caused by a steps will authenticate successfully “failed login” that has been generat- SQL Server is not running The appli- if you are on a T-1 connection at ed by a Web application. By simply cation server needs to connect to the 1.54 mbps. In this case, you have looking at this error message, it is backend database living on the SQL an environment-dependent error impossible to determine whether server in order to execute queries, where the dependency is in the this error is the result of a software store procedures, and access data. If bandwidth. bug, a server-side configuration is- the SQL server process itself is not Environment-independent er- sue, a compatibility issue, a browser running, then obviously the applica- rors, on the other hand, are relatively configuration issue, or all of the tion will not work. easier to reproduce—it’s not neces- above. sary to replicate the operating envi- After further analyzing the fail- DLL/COM objects are missing or were ronment. With environment-inde- ure, I discover several possible con- unsuccessfully registered Perhaps the pendent errors, all that need be ditions that might generate this error installation program failed to copy replicated is the steps that reveal the message: all the DLLs used by the application error. For example, if the company server during setup. If any DLL need- name is misspelled on all of the prod- IIS (Web server) virtual directory has not ed by the application server is miss- uct’s online pages as WebTessting. been set up properly When the virtual ing, the application will not work. Con, you will always see this error— directory is not properly configured, Perhaps the installation program independent of the hardware, soft- the requested files, scripts, or data correctly copied all the needed mod- ware, and resource variables in your will not be found. Typically, this is a ules, but failed to register one or 24 www.stqemagazine.com Software Testing & Quality Engineering May/June 2000
  3. 3. more of them. For example, with OLE- ty issues that are similar to PC envi- based objects such as COM or DCOM, their class ID (CLSID) must be regis- 4 ronments, where all components are in one box. Issues multiply within tered in the Registry Database before client/server systems, however, be- they can be used. If an application Which layer cause there may be many clients and tries to access a COM object that was ser vers connected on a network. not registered successfully, the appli- really causes Typical client/server configuration cation will not work. and compatibility issues involve the This problem is often caused by the problem? hardware and operating system mix errors in the installation procedures. (UNIX-based boxes versus Windows- If, on the other hand, the compo- Errors in Web systems are often dif- based boxes, for example) and the nents must be manually registered ficult to consistently reproduce be- software mix on the server side (Web then this becomes a configuration cause of the many variables intro- server packages, database server issue. duced by the distributed nature of packages, firewalls, COM objects, client/server architecture (i.e., serv- CORBA objects, etc.). Issues may also Browser-side JavaScript setting has er, client, and networking compo- involve the software mix on the been disabled This is a browser-side nents). There are at least three usu- client side (TCP/IP stacks, dialer configuration problem since the ap- al suspects in a Web environment: software, helper components, brows- plication requires the browser to The client, the server, and the net- er brands, and browser versions). have JavaScript enabled. Is this a work. Additionally, browser settings, such software error, a configuration prob- Both the client and the server as general settings, connection set- EDIT PASS 05/21/99 lem, or a technical support issue? carry configuration and compatibili- tings, security settings (including Ac- Making Your Web Application Test Report More Reproducible s Check if the client operating system, versions, and patches s Check for proper registration of components (COMs, meet system requirements Java, etc.) s Check if the correct version of the browser is installed on s Check to ensure that DNS is properly configured the client machine s Check if firewall configuration is causing packets to drop s Check if the browser is properly installed on the machine or blocking access (for example, the JVM is also successfully installed) s Check if a slow connection is causing the application to s Check the browser settings time-out s Try the same set of steps with different browsers (e.g., s Check for potential race or time-related conditions Netscape Navigator versus Internet Explorer) s Check for potential network inaccessibility issues on the s Try the same set of steps with different supported versions client machines of the same browsers (e.g., 3.1, 3.2, 4.2, 4.3, etc.) s Check for potential network inaccessibility issues on the s Check to ensure that all servers are running server machines s Check to ensure that all service-based components have s Check if the server operating system version and patches been started meet system requirements s Check to ensure that application access privileges are s Check if the proper versions of the server software such CREDIT CREDIT CREDIT properly set up as Web server, SQL database, and other middle-ware packages are installed s Check for missing components on the server (DLLs, scripts, etc.) s Check server configurations for proper settings 25 May/June 2000 Software Testing & Quality Engineering www.stqemagazine.com
  4. 4. Dynamic Environments (i.e., resource Physical Server and time-related errors) in which Client Client Database otherwise compatible components Server may exhibit errors due to memory- Web related errors and latency condi- Server tions. (We’ll discuss dynamic envi- Application ronments in more detail later in this Server section.) Static Operating Ethernet Environment: Configuration and Compatibility Variables Configuration and compatibility issues FIGURE 1 Web server, application server, and database server in one box may occur at any point within a Web system: client, server, or network. Physical Server 1 Configuration issues involve various Client Client server software and hardware set-ups, browser settings, network connec- tions, and TCP/IP stack set-ups. The Database Server browser setting/JavaScript example discussed earlier illustrated one type of configuration issue. A different type of configuration issue is shown in Fig- ures 1 and 2, with two possible physi- Ethernet cal server configurations: one-box and two-box configurations. Our sample application under test has some charting capabilities Web Server that enable a user to generate met- rics reports, such as bar charts and line charts. When a user requests a Application metrics report, the application server Server pseudo code runs as follows: Physical Server 2 1. Connect to the database server and run the query. FIGURE 2 Web server and application server in one box; database server in another box 2. Write the query result to a file named c:tempchart.val tiveX controls, plug-ins, Java, script- ing, downloads, user authentication, 5 3. Execute the Chart JavaApplet. Read from c:tempchart.val and etc.), content settings, program set- use the data to draw a graph. tings, and other advanced settings (including browsing options, multi- Static and 4. Send the JavaApplet to the media options, Java VM options, printing options, and HTTP options) dynamic operating browser. introduce a multitude of variables that should be tested and included in environments During testing for this application, I discovered that the charting feature analyses. The network offers another set are different. worked on one of the above configu- rations, but not the other. After I in- of variables. The network affects the In general, there are two classes of vestigated further, I learned that the Web application in several ways, in- operating environments—each with problem only occurred in the two- cluding timing-related issues (race its own unique testing implications: box configuration. After examining conditions, performance, time-outs, the code, I realized that the problem etc.) due to bandwidth and latency, Static Environments (i.e., configura- is in steps 2 and 3. In step 2, the potential configuration and compati- tion and compatibility errors) in quer y result is written to bility issues due to hardware devices which incompatibility issues may ex- c:tempchart.val of the database serv- ANNIE BISSETT such as gateways and routers, and ist regardless of variable conditions er local drive. In step 3, the Chart side effects related to security imple- such as processing speed and avail- JavaApplet is running on the applica- mentations. able memory. tion server that is not in the same 26 www.stqemagazine.com Software Testing & Quality Engineering May/June 2000
  5. 5. I am not suggesting that we read the code ment cannot be replicated (due to its dynamic nature), the error be- comes irreproducible or hard-to-re- every time we come across an error...I merely produce. By the way, this is the reason want to point out that it is essential to that memory-related errors are often hard to reproduce. When a memory- identify which server configurations are problematic, overwrite error exists in the code, for example, it will always cause a memory-overwritten problem. How- and include such information in bug reports. ever, from a black-box testing per- spective, we will never have a chance I would also run a cursory suite of test cases to see the symptom of this error until the specific overwritten byte(s) of code or data is executed or read. In on all distributed configurations that are supported this example, the set of steps repre- sents the exact set of black-box ac- by the application server under test. tivities. The memory-overwrite error represents the actual error in the code. The condition in which the box with the database server. When each time a test procedure is execut- overwritten byte is executed or read it attempts to open the file ed, it causes the operating environ- represents the dynamic operating en- c:tempchart.val on the application ment to become dynamic. The vironment or condition needed to re- ser ver local drive, the file is not attribute can be anything from re- veal (reproduce) the error. there. source-specific (available RAM, disk Here is a Web application exam- In this case, I am not suggesting space, etc.) to timing-specific (net- ple of a dynamic environment-related that we read the code every time we work latency, the order of user trans- error in which we will examine a come across an error; I leave the de- actions being submitted, etc.). time-related error. The specification bugging work for the developers. I When a test case depends on requires that: merely want to point out that it is es- the exact replication of both the set sential to identify which server con- of steps and the operating envi- s Project names within the system must be figurations are problematic, and in- ronment but the operating environ- unique clude such information in bug reports. I would also run a cursory suite of test cases on all distributed The home directory path for the Web server on the host myserver is mapped to: configurations that are supported by C:INETPUBWWWROOT the application server under test. Compatibility issues are also When a page is requested from http://myserver/ data will be pulled from: important in static operating envi- C:INETPUBWWWROOT ronments. As an example, in Figure 3 we see a compatibility difference be- A filename (mychart.jar) is stored at C:INETPUBWWWROOTMYAPPBIN. tween Netscape Navigator and Inter- net Explorer. The application session path (relative path) is pointing to This is not to say that Internet C:INETPUBWWWROOTMYAPPBIN, and a file is requested from .LIB. Explorer is better than Netscape If I use Internet Explorer version 3.x, the Web server looks for the file in Navigator; it simply means that there C:INETPUBWWWROOTMYAPPBINLIB because the browser relies on the relative are incompatibility issues between paths. This is the intended behavior and the file will be found; this tells me that my browsers—and that the code should application will work as expected using Internet Explorer 3.x. not assume that relative paths work for all browsers. More importantly, it If instead I use Netscape Navigator version 3.x (a browser that doesn’t like .), the Web suggests that when you experience server defaults to C:INETPUBWWWROOTLIB and tries to look for mychart.jar from an error in one environment, the there instead. This is a problem for this particular application because the file same error may not appear in a dif- (mychart.jar) will not be found there—so I know this feature will not work using Netscape ferent environment if it’s an environ- 3.x. ment-dependent error. When I brought up the Java Console, I saw the following, which confirmed my finding: Dynamic Operating #Unable to load archive Environment: Things http://myserver/lib/mychart.jar:java.io.IOException:<null> Don’t Stay the Same When the value of a specific environ- ment attribute does not stay constant FIGURE 3 Compatibility issue between browsers 27 May/June 2000 Software Testing & Quality Engineering www.stqemagazine.com
  6. 6. s Error detection and handling for potential duplication be performed on the client- ... side using JavaScript <td width=quot;80quot; bgcolor=#00CCCC>&nbsp;&nbsp;</td> <td width=quot;80quot; bgcolor=#00CCCC align=quot;leftquot; height=quot;9quot;> s Users will be able to add or delete project <font size=1 face=quot;Arialquot; color=quot;#400040quot;> names by requesting the Setting Up Proj- ects page Project:<br></font> <select name=quot;namelistquot; size=quot;9quot; OnChange=quot;ListSelected()quot;> s When a user creates a new project name, <option value=quot;Anotherquot;>Another</option> a browser-side JavaScript checks the in- <option value=quot;NewProjquot;>NewProj</option> put name against the select list embed- </select></td> ded in the HTML page (as illustrated in Figure 4) <td width=quot;100quot; bgcolor=#00CCCC>&nbsp;&nbsp;</td> ... Take a look at the time-related error illustrated in Figure 5. These before and after screenshots of the Setting FIGURE 4 Browser-side JavaScript checks the input name against the values in Up Projects page illustrate that the the namelist application failed to detect the dupli- cate name “Doomed.” Figure 4 walks BEFORE you through the explanation of this time-related error that involves two users adding new project names to the same database. As illustrated in Table 1, User A and User B create new projects si- multaneously, but without knowl- edge of each other’s actions. In step 3, User A adds a project named An- other. Since that project name already exists, his browser’s JavaScript displays a message prompting him for a different proj- ect name. User B adds a project named Doomed. Her browser’s JavaScript does not detect Doomed as a preexist- ing project name and so adds it to both the database and the returned AFTER list. The updated project name list is sent back to User B. User A subsequently adds the same name, Doomed, to the project list. His browser’s JavaScript does not detect the name on the HTML list, so it adds the name Doomed to the database again—as well as to the re- turned list. The updated project name list is sent back to User A with two Doomed entries included. This result fails to meet the product’s specification. Unless this situation happens to be a well-de- signed test case, accidentally discov- ering this error and attempting to re- produce it is not a simple task. In this example, the actual error is in the failure of the application to check for ser ver-side duplicate names (in addition to client-side FIGURE 5 TOP : Before the project name “Doomed” has been entered by the user; checking). The steps include User A’s BOTTOM : After the application failed to detect the duplicate project named “Doomed” 28 www.stqemagazine.com Software Testing & Quality Engineering May/June 2000
  7. 7. activities. The dynamic operating en- fect your ability to replicate errors. are investigating, but also the undis- vironment is created by User B’s ac- With the application of some of the covered errors that are related to tivities—which are hidden or un- skills covered in this article, I hope them. STQE known to User A. that your Web testing experience will be less frustrating and more enjoy- Hung Q. Nguyen (hungn@logigear. able. com) is the president and CEO of In Conclusion Remember that nothing will re- place your testing skills—your ability LogiGear Corporation, a full-ser- vice consulting firm offering out- To be effective in analyzing and re- to come up with good test cases, ask sourced testing, QA training, and producing errors in a Web environ- relevant what-if questions, keep TRACKGEAR™ (a Web-based de- ment, you need to have a command careful notes, and methodically in- fect tracking solution). He is co-au- over the operating environment. You vestigate hard-to-reproduce errors. It thor of Testing Computer Software also need to understand how envi- is these skills that will assist you in and author of the soon-to-be-pub- ronment-specific variables may af- finding not only the errors that you lished Testing Web Applications. STEP REQUEST HTML LIST BEFORE HTML LIST AFTER D ATA B A S E B E F O R E D ATA B A S E A F T E R 1 USER A gets the Another Another Another Setting Up Project NewProj NewProj NewProj Page 2 USER B gets the Another Another Another Setting Up Project NewProj NewProj NewProj Page 3 USER A adds a Another Another Another Another new project named NewProj NewProj NewProj NewProj quot;Anotherquot; 4 USER B adds a Another Another Another Another new project named NewProj NewProj NewProj NewProj quot;Doomedquot; Doomed Doomed 5 USER A adds a Another Another Another Another new project named NewProj NewProj NewProj NewProj quot;Doomedquot; Doomed Doomed Doomed Doomed Doomed TABLE 1 User A and User B activities 29 May/June 2000 Software Testing & Quality Engineering www.stqemagazine.com