web based testing

Why is Web Based Testing Different? w End Users w Browsers w Multiple Technologies w Less time-to-market w No idea about – how many users? w Several Layers w Certain things we can’t control Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Web-Based Architecture with Real Users Application Server Web Cluster Server Users MQ Series WebSphere Server EJB Server Application Server • Load Balancing • Firewalls • Secure Gateways Database Server Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Special Techniques for Web Testing Page Flow Testing w Security Testing w Cookies/Sessions Testing w Links Testing w Performance Testing w Connectivity Testing w Browser Testing w Usability Testing w Navigation Testing w Content Testing w Fail over Testing w Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Page Flow Testing w Page Flow Testing § Page Flow testing deals with ensuring that the application does not get confused by jumping to random pages. § Each page should typically check to ensure that it can only be viewed via specific previous pages, and if the referring page was not one of that set, then an error page should be displayed. § A page flow diagram is a very useful aid for the tester to use when checking for correct page flow within the application. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Page Flow Testing w Manual Execution w Use Bookmarks w Try Navigate in an unnatural path w Establish a session – jump into any page in any order w Try to use faked session information Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Security Testing w Unauthenticated access to the application w Unauthorized access to the application w Unencrypted data passing w Protection of the data w Log files should be checked to ensure they do not contain sensitive information Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Security Testing w Faked sessions. Session information must be valid and secure. (e.g.,., a URL containing a session identifier cannot be copied from one system to another and then the application be continued from the different system without being detected) w Multiple login testing by a single user from several clients w Automatic logout after N minutes of inactivity with positive feedback to the user w Attempt to break into the application by running username/password checks using a password-cracking program. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Connectivity Testing w Connectivity testing involves determining if the servers and clients behave appropriately under varying circumstances w Difficult to accomplish from a server perspective since it is expected that the servers will be operating with standby power supplies as well as being in a highly available configuration w Two aspects of connectivity testing § Voluntary, where a user actively interacts with the system in an unexpected way § Involuntary, where the system acts in an unpredictable manner Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Connectivity Testing - Voluntary w Quit from session without the user saving state w Quit from session with the user saving state w Server-forced quit from session due to inactivity w Server-forced quit from session due to server problem w Client forced quit from session due to visiting another site in the middle of a session for a brief period of time w Client-forced quit from session due to visiting another site/application for an extended period of time w Client-forced quit from session due to client PC crashing w Client-forced quit due to browser crashing Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Connectivity Testing - Involuntary w Forcing the browser to prematurely terminate during a page load using a task manager to kill the browser, w Hitting the ESC key and reloading or revisiting the same page via a bookmark w Simulation of Hub Failure between PC and the Web Server. § While browsing remove the network cable from the PC, attempt to visit a page, abort the visit, then reconnect the cable. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Connectivity Testing - Involuntary w Web Server On/Off Test. Shutdown the web server, then restart the server. § The user should be able to connect back to the application without being redirected to the login page this will prove the statelessness of individual pages. § Note the shutdown is only for the web server. not for an application server w Database Server On/Off Test. Shutdown the database server and restart it. § The user should be able to connect back to the application without being redirected to the login page. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Connectivity Testing - Involuntary w Application Server On/Off test. § Shutdown the application server and restart it. T § Two possible outcomes for this depending on how session management is implemented •• The first outcome is that the application redirects to an error page indicating loss The first outcome is that the application redirects to an error page indicating loss of connectivity, and the user is requested to login and retry. of connectivity, and the user is requested to login and retry. •• The second outcome is the application continues normally since no session The second outcome is the application continues normally since no session information was lost because it was held in a persistent state that transcends information was lost because it was held in a persistent state that transcends application server restarts. application server restarts. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Links Testing w Can the page be downloaded and displayed? w Do all the objects on a page load? w Do all the objects on a page load in an acceptable time? w If the user turns images off, uses a non-graphical or no- frames browser, does it still work? w Do all the text and graphical links work? Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Link Checkers w Linked pages (other pages to be navigated to by clicking on hyperlinks). w Frame pages (where a page is partitioned into frames and each frame has its own HTML page to create the image displayed in the browser window. w Images used for graphical appearance or as buttons to navigate (e.g. GIFs and JPEGs) w Form handlers, where these are CGI scripts, Active Server Pages etc. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Link Checkers w ActiveX, Java applets and other objects that are downloaded and executed within the Browser w Other content files, such as video (AVI, MPEG), and audio (WAV, AU, MIDI, MPEG) files w Other Internet protocols such as email links, FTP, Newsgroups links Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

HTML Validators w The best way to validate is by running your documents through one or more HTML validators. 1.http://validator.w3.org 2.http://valet.webthing.com/page/ 3.http://www.htmlhelp.com/tools/validator/ 4.And many more….. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Link Checkers w http://validator.w3.org/checklink w http://www.dead-links.com/ w Offline crawling link checker § Download from: http://home.snafu.de/tilman/XENU.ZIP § For more help: http://home.snafu.de/tilman/xenulink.html § http://www.netmechanic.com/toolbox/power_user.htm Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Cross-Browser Testing w General info § Cross browser testing and debugging can be the most frustrating exercise § Debugging is a headache, but good practices can help you deal with the issue § The first practice is to discuss the issue of cross browser compatibility with client as early as possible. § Get an agreement on which browsers do you guarantee that the site will match the approved layouts and make it clear that the rest of the browsers may not look perfect. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Cross-Browser Testing w Browsers § Internet Explorer Internet Explorer (IE) by Microsoft is the most common Internet browser today. IE was introduced in 1995 and passed Netscape in popularity in 1998. § Netscape Netscape was the first commercial Internet browser. It was introduced in 1994. Netscape gradually lost its popularity to Internet Explorer. § Mozilla The Mozilla Project has grown from the ashes of Netscape. Browsers based on Mozilla code is the second largest browser family on the Internet today, representing about 30% of the Internet community. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Cross-Browser Testing w Browsers § Firefox Firefox is a new browser from Mozilla. It was released in 2004 and has grown to be the second most popular browser on the Internet. § Opera Opera is another Internet browser. It is known to be fast and small, standards-compliant, and available for many operating systems. Opera is the preferred browser for a number of small devices like mobile phones and hand-held computers. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Cross-Browser Testing w Browser Statistics – Most common Browsers? § Study the browser statistics to make a conscious decision as to what are your priorities. § From the statistics below, you can see that Internet Explorer is the most common browser. However, FireFox have become quite popular as well. Source: http://www.w3schools.com/browsers/browsers_stats.asp Source: http://www.w3schools.com/browsers/browsers_stats.asp Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Cross-Browser Testing w Browser Display Statistics - Most common display resolutions? § From statistics, most users are using a display with 1024x768 pixels or more Source: http://www.w3schools.com/browsers/browsers_display.asp Source: http://www.w3schools.com/browsers/browsers_display.asp § Be aware: Many users still have only 800x600 display screens. § This fact indicates that the figures above might not be 100% realistic. The average user might have display screens with a lower resolution. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Cross-Browser Testing w Guideline § Make your site as simple as possible. § The more complicated your HTML and CSS the more difficult debugging will be. § Use HTML Validator and CSS Validator to check HTML and CSS errors § Use Browser Compatibility Testing Tools § http://www.browsershots.org § http://www.browsercam.com w Browser Compatibility § http://www.netmechanic.com/products/Browser-Tutorial.shtml Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w Java-compatible Web browser § is a web browser that can display and execute any Java applets and Java Script § has a Java interpreter § Kinds of Java-compatible Web browser : • HotJava browser • Microsoft Internet Explorer • Netscape Navigator Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w Testing to see if Java works. § Here is a simple test to see if your browser is Java- compatible. § Open http://library.thinkquest.org/12255/java/tester.html in your browser § Then see which of the following messages appeared on your screen. § If you saw a button that says, \"Java Enabled (Click to Go Back),\" Java works on your computer. § If you saw a message, \"Sorry, your browser is not Java- compatible\" then Java doesn't works on your computer. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w Incompatible Browser § Think of your web site that makes extensive use of the scripting language, Java Script or has a Java applet § But your browser either has this feature turned off, or is too old to support it. § New browsers, support W3C standards which enables the Java Compatibility. § Old browsers, don’t support W3C standards. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w Getting Java to work on your computer § If Java does not work on your computer, there are a few solution to the problem. § Below are listed some operating systems and possible solutions for too old machines. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w Getting Java to work on your computer § Windows95 • Your Browser has Java disabled. Go to an options menu and enable Java. • Your Browser is not 32-bit. You can download the 32-bit versions of your browsers by clicking on the logo near the top right corner of your browser. This will bring you to the browser creator's web page where the latest versions will be posted. • If all else fails, close all applications and try rebooting your system after you save all important data. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w Getting Java to work on your computer § Macintosh • Make sure your computer can handle 32-bit operations. • Your Browser is not 32-bit. You can download the 32-bit versions of your browsers by clicking on the logo near the top right corner of your browser. This will bring you to the browser creator's web page where the latest versions will be posted. • Your Browser has Java disabled. Go to an options menu and enable Java. § Unix/Linux • Your Browser has Java disabled. Go to an options menu and enable Java Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w I have an updated browser; how do I turn on Java Compatibility? § Mozilla 1.X 1. Open Mozilla. 2. Select Preferences from the Edit menu. 3. Click the arrow next to Advanced. 4. Click Scripts & Plugins. 5. Check Navigator beneath \"Enable JavaScript for\". 6. Click OK. 7. Click Reload. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w I have an updated browser; how do I turn on Java Compatibility? § Opera 5.X and Opera 7.X 1. Open Opera. 2. Select Quick Preferences from the File menu. 3. Make sure Enable JavaScript is checked. 4. Click Reload. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w I have an updated browser; how do I turn on Java Compatibility? § Safari for MAC OSX 1. Open Safari. 2. Select Preferences from the Safari menu. 3. Click Security. 4. Check both Enable Java and Enable JavaScript. 5. Close the window. 6. Click Reload. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w I have an updated browser; how do I turn on Java Compatibility? § Internet Explorer 3.X 1. Select Options from the View menu. 2. Click Security. 3. Check Enable Java Programs. 4. Click OK. 5. Click Reload. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w I have an updated browser; how do I turn on Java Compatibility? § Internet Explorer 4.X 1. Select Internet Options from the View menu. 2. Click the Security tab. 3. Click Custom. 4. Click Settings. 5. Scroll down to locate Scripting. 6. Click Enable for Active Scripting. 7. Click OK. 8. Click Reload. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w I have an updated browser; how do I turn on Java Compatibility? § Internet Explorer 5.X or better 1. Open Internet Explorer. 2. Select Internet Options from the Tools menu. 3. In Internet Options dialog box select the Security tab. 4. Click Custom level button at bottom. The Security Settings dialog box will pop up. 5. Under Scripting category enable Active Scripting, Allow paste options via script and Scripting of Java applets 6. Click OK twice to close out. 7. Hit Refresh. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w I have an updated browser; how do I turn on Java Compatibility? § Internet Explorer 5.X for MAC OSX or MAC OS9 1. Open Internet Explorer. 2. Select Preferences from the Explorer menu. 3. Click the arrow next to Web Browser. 4. Click Web Content. 5. Under Active Content check Enable Scripting. 6. Click OK. 7. Click Refresh. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w I have an updated browser; how do I turn on Java Compatibility? § Netscape 3.X 1. Select Options from the Edit menu. 2. Click Network Preferences. 3. Click Languages. 4. Check both Enable Java and Enable JavaScript. 5. Click OK. 6. Click Reload. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w I have an updated browser; how do I turn on Java Compatibility? § Netscape 4.X (Windows or MACOS9) 1. Open Netscape. 2. Select Preferences from the Edit menu. 3. Click Advanced. 4. Check both Enable Java and Enable JavaScript 5. Click OK. 6. Click Reload. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w I have an updated browser; how do I turn on Java Compatibility? § Netscape 7.X 1. Open Netscape. 2. Select Preferences from the Edit menu. 3. Click the arrow next to Advanced. 4. Click Scripts & Plugins. 5. Check Navigator beneath \"Enable JavaScript for\". 6. Click OK. 7. Click Reload. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w I have an updated browser; how do I turn on Java Compatibility? § AOL 3.0 1. Click Prefs. 2. Click Security. 3. Check Enable Java Programs, if the box is blank, single click on it. 4. Click OK . 5. Click the browser's back button, once back, click reload. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Java Compatibility w I have an updated browser; how do I turn on Java Compatibility? § AOL 4.0 and 5.0 1. Click My AOL. 2. Click Preferences. 3. Click WWW. 4. Click the Security tab. 5. Click Custom. 6. Click Settings. 7. Scroll down to locate Scripting. 8. Click Enable for Active Scripting. 9. Click OK, then click the Reload button. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

What is a cookie w Cookie is small information stored in text file on user’s hard drive by web server. w This information is later used by web browser to retrieve information from that machine. w Generally cookie contains personalized user data or information that is used to communicate between different web pages Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Why cookies w Cookies are nothing but the user’s identity and used to track where the user navigated throughout the web site pages. w The communication between web browser and web server is stateless w Cookies serve the purpose of maintaining the user interactions with web server. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Types of cookies w Session cookies § This cookie is active till the browser that invoked the cookie is open. § When we close the browser this session cookie gets deleted § Some time session of say 20 minutes can be set to expire the cookie w Persistent cookies § The cookies that are written permanently on user machine and lasts for months or years Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Where cookies are stored w When any web page application writes cookie it get saved in a text file on user hard disk drive. w The path where the cookies get stored depends on the browser w Different browsers store cookie in different paths. w E.g. Internet explorer store cookies on path “C:\\Documents and Settings\\Default User\\Cookies” Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Drawbacks of cookies w Too many cookies w Cookies are disabled on the browser w Security issues w Sensitive information Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Test Cases for Cookies Testing w Privacy policy § Make sure no sensitive data or personal data is stored in the cookie § If sensitive data is used inside the cookie, to be tested for encryption No overuse of cookies w Disable the cookies in the browser settings w Accept / Reject some cookies w Delete a cookie w Corrupt the cookie w Deletion of cookies from your web application page w Cookies testing on multiple browsers w Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Session Testing w Application Session should get expired after a predefined a period of inactivity w Basically for the Security w Back – Forward Buttons w Multiple Logins – from the same M/c § Using Multiple Browser Sessions •• Same Browser Same Browser •• Different Browser (IE & NN) Different Browser (IE & NN) Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Content Testing w Two types of Contents – Static & Dynamic w Static Content § Verify for correctness § Verify for accuracy § Verify organization of content w Dynamic Content § Test by feeding new content § Try all possible combinations •• Wrong data Wrong data •• Huge amount of data Huge amount of data •• Not matching the expected type of content Not matching the expected type of content •• With and without graphics With and without graphics Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Usability Testing w Objectives § Informational Content § UI Functionality § User Performance § Load Imposed on the End-User § Satisfaction of End-User § Cost-Benefit Analysis Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Some Usability Tests w Content § Citation § Credibility § Coverage § Currency § Continuity § Language § Objectivity Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Some Usability Tests w Creative Advanced Components § Colors & Backgrounds § Alignment & Layout § Consistency § Error Prevention & Control § Feedback § Form Design § Graphic Design § HTML Quality § Labeling § Page/Site Design § Search § Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Navigation w Global, Local, Contextual Navigation w Indication of “Where am I” w Grouped Properly w Placement w Site Structure (Site Map etc..) w Clearly Marked Exits w Bookmark – Name is easily understandable w Is there a Consistency? Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Configuration Testing w Definition § is the system testing of different variations of an integrated, black box application against its configurability requirements w Goals § Cause the application to fail to meet its configurability requirements so that the underlying defects can be identified, analyzed, fixed, and prevented in the future. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Configuration Testing w Objectives § Partially validate the application (i.e., to determine if it fulfills its configurability requirements). § Cause failures concerning the configurability requirements that help identify defects that are not efficiently found during unit and integration testing: • Functional Variants. • Internationalization (e.g., multiple languages, currencies, taxes and tariffs, time zones, etc.). • Personalization Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Configuration Testing w Objectives § Report these failures to the development teams so that the associated defects can be fixed. § Determine the effect of adding or modifying hardware resources such as: • Memory • Disk and tape resources • Processors • Load balancers § Determine an optimal system configuration. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Configuration Testing w Typical examples for configuration testing include an application that must: § Have multiple functional variants. § Support internationalization. § Support personalization. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Internationalization w Definition § Internationalization (i18n) is the process of designing software so that it can be adapted (localized) to various languages and regions easily, cost-effectively, and in particular without engineering changes to the software. § The term internationalization is abbreviated as i18n, because there are 18 letters between the first \"i\" and the last \"n.\" Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Internationalization w Characteristics: § With the addition of localization data, the same executable can run worldwide. § Textual elements, such as status messages and the GUI component labels, are not hard coded in the program. Instead they are stored outside the source code and retrieved dynamically. § Support for new languages does not require recompilation. § Culturally-dependent data, such as dates and currencies, appear in formats that conform to the end user's region and language. § It can be localized quickly. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Localization w What is Localization? § Localization refers to the adaptation of a product, application or document content to meet the language, cultural and other requirements of a specific target market (a \"locale\"). § Localization is sometimes written as \"l10n\", where 10 is the number of letters between 'l' and 'n'. § A synonym for translation of the user interface and documentation § substantially more complex issue Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Localization w Localization can entail customization related to: 1. Numeric, date and time formats 2. Use of currency 3. Keyboard usage 4. Collation and sorting 5. Symbols, icons and colors 6. Text and graphics containing references to objects, actions or ideas which, in a given culture, may be subject to misinterpretation or viewed as insensitive. 7. Varying legal requirements 8. and many more things. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Internationalization Testing w Test suites – Configurations § Internationalisation Taxonomy Matrix § Locales - system, client, program § Combinations - single locale set-up, varied locale set-up § Representative sampling - Western Europe, Eastern Europe, Asia, Middle East § Time zones - different from server to client § Char sets - different data passed through system Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Internationalization Testing w Test suites – Data § Text - characters, punctuation, symbols, word wrap, sorting, search § Numeric - date/time, currency, telephone numbers, measure, search § Graphics - icons, banners, colours, buttons § Layout - screens, pop-up windows, frames § Language range - Western European, Eastern European (non Latin), Far Eastern, Right-to-left § Edge testing - min-max values, min-max length, incorrect chars & formats, mismatched dates/times Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Internationalization Testing w What to look for -Text § Truncation - characters cut off, or partial multi-byte character § Rendering - proper characters, display configuration, cut-and- paste § Line wrap - logical place for language, line lengths (client vs. server) § Search - exact vs. fuzzy matches, false matching, per language/multilingual § Sort - logical order for the language, multilingual § Indexing - layout makes sense for language, order correct Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Internationalization Testing w What to look for - Numeric Number groupings – amount per group, separator, good for locale § Decimal point – correct character and number of digits after for locale § Date/Time – order of day, month, year and separator, short and long § form, abbr. Of day and month names Currency – symbol, 3-letter id, positioning, no. of digits after decimal, § expansion room Measure – proper units, expansion room § Telephone numbers – no restriction format, allowance of certain non- § numeric characters Address – general address lines, states/province optional, postal code, § country field Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Internationalization Testing w What to look for – Graphics and Layout § Images - internationally appropriate, no imbedded text, sizing § Colours - no implied meaning by color, consistency in use § Windows - expansion room, word order dependency § Screen - positioning, resolution, window layering § Sound - non-culture specific, switch off capability Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Reference sites for Internationalization w http://www.w3.org/International/getting-started/ w http://www.w3.org/International/ w http://java.sun.com/j2se/1.4.2/docs/guide/intl/ w http://www.w3.org/International/quicktips/ Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Load Testing w Why do we need to load test? § Large volumes of network traffic caused by `intelligent clients'. § Increased number of `architectural layers' of which a system is composed. § Delays between distributed processes communicating across networks. § The increased number of suppliers of architectural components who must be dealt with Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

A 3 Tier Application Display Driver Middleware w w GUI Business Logic w w Application Node DB Connectivity w w Toolkit Layer Server OS w w Middleware Network Transport DB w w Connectivity Client OS w w SQL Services LAN / WAN Network w Support w DBMS DB Server OS Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Load Testing w Objectives § Assessing the system's capacity for growth – •• The load and response data gained from the tests can be used to validate the The load and response data gained from the tests can be used to validate the capacity planning model and assist decision making. capacity planning model and assist decision making. § Identifying weak points in the architecture – •• The controlled load can be increased to extreme levels to stress the architecture The controlled load can be increased to extreme levels to stress the architecture and break it -- bottlenecks and weak components can be fixed or replaced. and break it bottlenecks and weak components can be fixed or replaced. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Load Testing w Objectives § Detect obscure bugs in software – •• tests executed for extended periods can cause failures caused by memory leaks tests executed for extended periods can cause failures caused by memory leaks and reveal obscure contention problems or conflicts. and reveal obscure contention problems or conflicts. § Tuning the system – •• repeat runs of tests can be performed to verify that tuning activities are having the repeat runs of tests can be performed to verify that tuning activities are having the desired effect -- improving performance. desired effect improving performance. § Verifying resilience and reliability – •• executing tests at production loads for extended periods is the only way to assess executing tests at production loads for extended periods is the only way to assess the system's resilience and reliability to ensure required service levels are likely to the system's resilience and reliability to ensure required service levels are likely to be met. be met. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Pre Requisites for Load Testing - Requirements w Quantitative § Expressed in quantifiable terms such that when response times are measured, a sensible comparison can be made § For example, response time requirements should be expressed as a number of seconds, minutes or hours w Relevant § Response time must be relevant to a business process § For example, a response time might be defined within the context of a telesales operator capturing customer enquiry details and so should be suitably quick, § A report generated as part of a monthly management reporting process and which might have an acceptable delay of ten minutes Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Pre Requisites for Load Testing - Requirements w Measurable § Response time should be defined such that it can be measured using a tool or stopwatch and at reasonable cost. § It will not be practical to measure the response time of every transaction in the system in the finest detail w Achievable § Response times should take some account of the cost of achieving them. § There is little point in agreeing to response times which are clearly unachievable for a reasonable cost (i.e. within the budget for the system) Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Pre Requisites for Load Testing - Requirements w Realistic § Response time requirements should be justifiable when compared with the durations of the activities within the business process the system supports § Clearly, it is not reasonable to demand sub-second response times for every system function, where some functions relate to monthly or occasional business processes which might actually take many minutes or hours to prepare or complete Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Pre Requisites for Load Testing – Stable System w System Should be § System Tested & Bug Fixed § Stable § Reliable Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Pre Req. - Load Testing – Realistic Environment w Ideally similar to Production Environment § Hardware Configurations § Software § Database Servers § O/s versions § LAN/WAN simulations w Should be controlled Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Tools w Test Database Creation/Maintenance § To create the large volumes of data on the database which will be required for the test. § Usually SQL or `Procedural SQL' database tools. w Load generation § Tools can be of two basic types, either a test running tool which drives the client application, or a test driver which simulates clients workstations. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Tools w Application Running Tool § Test running tool which drives the application under test and records response time measurements. (May be the same tool used for load generation) w Resource Monitoring § Utilities which can monitor and log both client and server system resources, network traffic, database activity w Results Analysis and Reporting § Test running and resource monitoring tools can capture large volumes of results data Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Performance Requirements w Response time requirements. w Transaction volumes detailed in `Load Profiles'. w Database volumes w Any other requirements Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Load Profiles Transa TXN ID ction User Type Users Location rate 23 Log Telesales 100 BHM 20/hr Custome Telesales 80 BTL 15/hr r Fault Telesales 140 WEM 25/hr 24 Allocate Fault Controller 5 BHM 10/hr Fault Fault Controller 7 WEM 14/hr 25 Escalate Section leader 10 BHM 5/hr Fault Section leader 10 WEM 10/hr 26 Clear Fault Controller 5 BHM 10/hr Fault Fault Controller 7 WEM 14/hr Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Sample Business Scenarios w Average load, busy hour, busy 5 minutes - useful where the mix of activities is relatively constant, but the volume of tasks undertaken varies. w Normal, end of month, end of year - where an organization's activities change over time with peaks occurring at specific periods. w Quiescent, local fault, widespread emergency - where a support organization might have quiet periods interspersed with occasional peaks and must cater for 1 in 200 year disasters. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Database Considerations w # of records available in the table(s) w # of tables queried per transaction w # of tables updated per transaction w Replication services Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Summary w Pre-requisites § Well defined requirements •• Response Times Response Times •• Load Profiles Load Profiles •• Database Volumes Database Volumes § Stable system § Realistic & Controlled Environment § Tools Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Planning w Documentation of performance requirements including: •• database volumes database volumes •• load profiles having relevance to the business load profiles having relevance to the business •• response time requirements. response time requirements. w Preparation of a schedule of load profile tests to be performed (e.g. normal, busy hour, busy 5 minutes or some other scheme). w Inventory of system transactions comprising the loads to be tested. w Inventory of system transactions to be executed and response times measured. w H/w & S/w attributes whose performance needs to be monitored w Description of analyses and reports to be produced. Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Preparation Preparation of a test database with appropriate data volumes w Scripting of system transactions to comprise the load w Scripting of system transactions whose response is to be w measured (possibly the same as the load transactions) § User Names & Passwords (matching the load#) § Account Numbers § Mail Ids w Development of Workload Definitions (i.e. the implementations of Load Profiles) w Preparation of test data to parameterize automated scripts Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Execution w Trial Runs w Execution of load tests. § Incremental – users as well as load w Ensure that environment is in tact w Repeat test runs, as required w Repeat Scenarios w Performance Monitoring (all the while) Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Sample Steps w Preparation of database w Prepare test environment as required and verify its state. w Start monitoring processes (network, clients and servers, database). w Start the load simulation and observe system monitor(s). w When the load is stable, start the application test running tool and response time measurement Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Sample Steps w Monitor the test closely for the duration of the test. w If the test running tools do not stop automatically, terminate the test when the test period ends. w Stop monitoring tools and save results. w Archive all captured results, and ensure all results data is backed up securely Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Analysis w Collection and organization of test results. § The count of measurements. § Minimum response time. § Maximum response time. § Mean response time w Preparation of tabular and graphical analysis w Preparation of reports including interpretation and recommendations Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

Optimization/Tuning w Small changes to application software, middleware, database organization. w Changes to server system parameters. w Upgrades to client or server hardware, network capacity or routing. w DB Optimization w DB Sizing Copyright © 2006 Bigtec, all rights reserved Copyright © 2006 Bigtec, all rights reserved

web based testing

14 comments

Notes on slide 1

17 Favorites