Mobile Application Testing


Published on

The presentation basics about the mobile application strategies and concepts. Good source for the beginners. Get your app tested by us:

Published in: Technology
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Mobile Application Testing

  1. 1. By Waqar Ahmad Swaam Tech (Pvt) Ltd Contact us: for Ur App TestingNote: The slide was presented in the Software Testing Course at NUCES-FAST Lhr
  2. 2. Classification of Mobile Applications Web Applications  Could be Simple Markup (WML , HTML, XHTML, XHTML MP) or AJAX Applications  A Web Application may be Designed for  A Single Type of Device or  Multiple Different Devices with Customization for each Device (or each Family of Devices)  Multiple Different Devices—A Transcoder/Proxy Residing Between Web Application Server and Device Transforms Content based on Device User-Agent Capabilities
  3. 3. Classification of Mobile Applications On-Device Applications May be  Developed for Better Portability  Java ME  FlashLite  Native Platform Applications  IPhone: Objective C  Symbian: C/C++  Windows Mobile: .Net CF  Android : Java for Application and C/C++ for Low Level
  4. 4. Mobile App Testing—Challenges  Device Fragmentation is a Big Challenge  Devices Vary in Screen Size, Memory, Processing Power, Hardware Features etc.  Apple iPhone is Least Fragmented among All Mobile Platforms  Testing on All Target Handset/Devices  Almost Impractical if Number of Target Handsets is Large  Testing on All Target Operator Networks  Almost Impractical if Number of Target Operators is Large  Network Operator may Impose Certain Constraints
  5. 5. Mobile App Testing—Challenges Device Vendor (Not Necessarily Manufacturer) may Customize a Device  Changes may be Done in Both User Interface and Functionality  Many a Times Changes are Not Well-Documented  Device is Not the Same as Released by Manufacturer Testing on All Versions of Operating System  In Many Situations, it May Not be Possible to Upgrade the Device to New OS Version Same Device Models may have Multiple OS Configurations based on Target Market Requirements  Localization  Hundreds of Devices with Many Variants of Each Device
  6. 6. Mobile App Testing—Challenges Testing in All Web Browsers  Browsers Significantly Vary Across Devices  May Support HTML, xHTML, WML, AJAX Web Content Transcoders  Web Content may be Transformed to Meet User-Agent Constraints  Transcoder may Not Respect User Experience Factors in some Situations
  7. 7. Mobile App Testing—Challenges Inconsistencies in Specification Implementations  Each Vendor is Responsible for Java ME Specifications Implementations  Implementations may Not be Consistent across Devices and Vendors Performance Measurement  Extremely Difficult to Benchmark Performance due to Highly Fragmented Market Detection of Display Issues  Human Involvement is Must Generally it is Not Possible to Purchase All Target Handsets due to Financial Constraints
  8. 8. Mobile App Testing—Challenges Application Usability Testing  End Users may Not even be Skilled at Using a Mobile Device  End User may Use Application in Multiple Contexts  Driving in Rush Hour  Walking in Crowded Market  Multiple Input / Output Modalities  Diverse User Interface Styles In Reality, A Testing Lab Cannot Simulate all Possible End User Contexts
  9. 9. Mobile App Testing—Challenges Development and Deployment Platforms are Different  Development Platforms  Windows, Linux  Deployment Platforms  iPhone, Symbian, Android, Blackberry, Windows Mobile Emulators Do Not Capture All the Characteristics of a Device Multiple Design Specifications may be Required for Single Application due to Device Fragmentation— Requires Test Cases Accordingly
  10. 10. Testing Techniques: General Gather Useful Data Decide Target Handsets as Early as Possible in Development Lifecycle (Better in Requirements Analysis Phase) Document Incompatibilities in Devices e.g. Differences in  Installation Processes  Audio/Video Capabilities  Memory, Processing Power, Screen Size etc. Document Device Software (Firmware) Limitations  ‘Bugs in Firmware’ are Not Unusual Document Device Web Browser Constraints  Supported Version of HTML  AJAX Supported or Not
  11. 11. Testing Techniques: General Gather Useful Data Better Acquire Meta Data of All Target Handsets from an External Source  Extensive Databases Containing Device Meta Data  WURFL—An Open Source XML Database of Device Characteristics  DeviceAtlas—A Proprietary Database of Device Characteristics  Some Websites can Capture and Return HTTP Headers  For instance, Access the Website using a Phone, Website will Return Headers in HTTP Requests  Useful when Data is Not Available in Databases and also to Verify Already Available Data in Meta Data Source  A Development House may also Maintain an Internal Database Containing Data Not Available Otherwise
  12. 12. Testing Techniques: General Classify Devices Classify Devices  Category—1: All Devices that Support XHTML  Category—2: All Devices that Support AJAX  Category—3: All Devices that Support Java ME Pick Representative Device(s) from each Category for Testing Purposes Prepare Test Cases based on Data Gathered about Target Handsets
  13. 13. Testing Techniques: General Perform Testing on Real Devices Start Testing on Real Devices as Early as Possible There is No Good Alternative to a Real Device for Testing
  14. 14. Testing Techniques: General Emulator is Useful but is Not a Real Device Testing on Emulators  An Emulator is a Software Application that Imitates a Smartphlone  A Device Emulator may be Used to Test Application on Desktop  Some Emulators Emulate General Characteristics of a Platform (like Java ME Emulator)  Some Emulator Emulate Relatively Faithful Representation of Specific Phones (BlackBerry Dev Kits)  Emulator may or may Not Require Compilation for Target Platform  Single Compiled Code for both Emulator and Target Devices  Java ME  Windows Mobile  Different Compilers for Compilation for Symbian Emulator and Target ARM Device  Useful for Quick Testing in Development Environment  Emulators are Not Real Devices  Testing on Emulator does Not Guarantee Application Success on Real Device
  15. 15. Testing Techniques: General Application UI Should Preferably be Consistent with Device UI Style Test Application User Interface Consistency with Device Interface Style  Is the Application User Interface Consistent with User Interface of Underlying Device Applications?  Does the Application Offer Interaction Mechanism Similar to Interaction Mechanism Generally Implemented in Native Device Applications?  Softkey Mapping  Look&feel
  16. 16. Testing Techniques: General Different Test Cases are Required to Cater Diverse Input / Output Modalities Perform Testing for Various Input Modalities  Touch Screen  Testing with Single and Multiple Touch Input  Testing with Virtual Keyboard  Trackball/Track Wheel  Numeric Keypad  QWERTY Keypad  4- or 5-Way Navigator Perform Testing for Various Output Modalities  Different Screen Sizes
  17. 17. Testing Techniques: General Test Applications in Real User-Contexts Test the Application in Relevant User Contexts  User Context Involves User, Application and Surroundings  Design Various User Interaction Scenarios in Applicable Contexts (user driving in rush hour, walking, in classroom etc.)  Test Applications in Real Contexts
  18. 18. Testing Techniques: General Mobile Ecosystem Requirements must be Addressed Application are Generally Launched through Various Application Stores (Mobile Ecosystem)  Apple App Store, Nokia Ovi, Blackberry AppWorld, Android Market, GetJar, Handango etc. Test Applications to Meet Requirements of various Ecosystem Stakeholders  Manufacturers/Vendors  UI Guidelines from Apple for Iphone Applications  Symbian Signed  Operators  Operators may also have some Requirements  Third Parties  Java Verified Process
  19. 19. Testing Techniques: Web Applications Rules-Based Testing Rules-Based Testing with User-Agent Impersonation  Create Testing Rules based on  Device Capabilities  Known Bugs in the Device  Device Data in Data Sources (like WURFL, Internal Database)  Application Requirements  Test Cases are Written that Checks HTTP Response for Targeted Content  Impersonate a Device and Communicate with Web Server  Change HTTP Headers like  Accept: image/gif, image/jpeg  User-Agent: ‘NokiaN95…….+Profile/MIDP-2.1 + …..’  Check the Server Response Against Rules and Identify Discrepancies, if any
  20. 20. Testing Techniques: Web Applications Rules-Based Testing  Rules-Based Tests  Testing is Automated for Multiple Types of User-Agents  Testing can be Done Using  An HTTP Library for Sending Requests and Receiving Response  Web Browser (FireFox, Safari, Internet Explorer)  Useful to View the Rendered Screen with some Limitations  Rules Based Testing is Useful for Automated Testing of  Web Applications that Deliver Content on Phone  Regression Testing  Web Content Accessibility
  21. 21. Testing Techniques: Web Applications Transcoder Behavior must be Thoroughly Tested Transcoder Behavior  Web Content may be Transformed by a Transcoder to Meet User-Agent Constraints  A Transcoder Resides Between User-Agent and Web Server  Cookies should Not be Shared  Use Automated Tests for Communicating with Server in Parallel  User Experience Should Not be Affected  Manual Testing—See and Record the Behavior
  22. 22. Testing Techniques: Web Applications W3C mobileOK Standard  W3C has Defined mobileOK Tests  Tests for Checking Mobile Readiness (W3C Best Practices) of a Website   W3C mobileOK Checker  Performs mobileOK Tests on a Web Page to Determine its level of Mobile-Friendliness 
  23. 23. Testing Techniques: On-Device Applications Unit Testing and On-Device Debugging  Unit Testing Frameworks Exist for almost All Platforms  GUI Layer, Event Handling and System Testing is Not Possible  On-Device Debugging is Available on Most of the Platforms
  24. 24. Testing Techniques: On-Device Applications Custom Test Applications may Reveal Major Issues  Custom Test Applications  Can be Used to Test Critical Aspects  Whether an Application has Access to Device File System  Performance of Device Database System  Write Applications to Diagnose Key Issues e.g., Develop an Application that Helps in Testing When  Lot of Free Memory is Available on Device  Very Limited Memory is Available on Device
  25. 25. Testing Techniques: On-Device Applications Network Availability and Bandwidth  Perform Application Testing in Different Network Signal Strengths  No Network  Intermittent Connectivity  Strong Network Signals Presence  Application Testing May be Required in Various Network Types  2G, 3G
  26. 26. Testing Techniques: On-Device Applications External Events may Affect Application Behavior  Test Application against Events that may Occur during Application Installation and Execution  Phone Call Received  SMS Received  Battery Removed  Device Switched Off  Bluetooth Device Requested to Contact
  27. 27. Testing Techniques: On-Device Applications Combine ‘Automated Input’ with ‘Manual Verification’  Generate Automatic Events  Key Press  Pointer Press  Touch  Capture Screenshots during Event Generation Process and Compare them
  28. 28. Testing Techniques: On-Device Applications Track Behavior when App is being Used by Consumers  Track Application Behavior during Various Lifecycle Stages when in Production  Installation  Execution  Removal  Application may Send Important Data to Server that can be Used to Identify and Fix some Bugs  Java ME Includes some Parameters for this Purpose  ‘GetJar ’ Offers Such a Service  Application can Call Server and Report some Information 
  29. 29. Testing Techniques: CommercialGet Online Services for Devices Not Available In-house Testing on Real Devices that Developers Don’t Own  A Service that Enables Application Testing on Real Devices  Devices are Located on Remote Servers  Devices are Connected to Various Operator Networks Worldwide  Desktop Client Software/ Web Application Enables All Testing Features Remotely  Device Acquisition, Cleanup  Application Installation, Execution, Removal  Device Shutdown, Battery Removal and Installation  Service Providers  DeviceAnywhere (  Over 2000 Devices Connected to over 30 Worldwide Carrier Networks across USA, Canada, UK, France, Germany, and Spain  Perfecto Mobile (
  30. 30. Testing Techniques: Commercial Crowdsourcing is also an Option CrowdSourcing (Crowd + Outsourcing)  Testing by Human Testers (Mobsters) Located Worldwide  If Done Effectively, Real Usability Testing Results can also be Obtained  Testing in Real User Contexts is Possible  How it Works?  Developers and Testers are Registered with a Third Party Service Provider  Third Party Service Provider is Responsible for Coordination of All Parties Involved in the Process (Application submission, monitoring tools, results reporting etc.)  Service Providers  Mob4Hire (  GetJar (  uTest (
  31. 31. Testing Techniques: Commercial User Feedback is Important App Stores may Provide Useful Feedback from End Users  User Reported Bugs  User Reviews on Usability  User Ratings Data is Helpful for  Quickly Fixing some Bugs  Future Application Releases
  32. 32. Testing Tools: Web Applications HttpUnit  Unit Testing for Web Applications HtmlUnit  Unit Testing with JavaScript Support Urllib  Provides APIs for Web Access Check a Web Application for Mobile Readiness  mobiReady:  W3C mobileOk Checker:
  33. 33. Testing Tools: Web Applications User-Agent Capture Tools  Captures Characteristics of User-Agent  Content Type, HTML version Supported on Device etc.  Pycopia: FireFox and Others Browsers—Use User-Agent Switcher Add-On. However, there are Some Issues  Differences in how Desktop and Mobile Browsers Layout Content may Render Testing Results Unreliable  Performance Differences in Desktop and Mobile Browsers  Feature Differences in Desktop and Mobile Browsers
  34. 34. Testing Tools: Web Applications Desktop Browser Automation Tools  WebDriver (  Java APIs that can be Integrated with Eclipse  Selenium (  Firefox add-on that records clicks, typing, and other actions to make a test, which can be played back in the browser.  Selenium RC can be Used in a Desktop Browser with HTTP Header Emulation IPhoneDriver  A WebDriver based Tool for Testing on UIWebView
  35. 35. Testing Tools: On-Device Applications  iPhone  Embedded Browser Component (UIWebView)  iPhoneUnitTesting  Memory Sweep (For memory snapshot of iPhone)  Java ME  J2MEUnit (For Unit Testing)  JInjector (For System Testing)  ASM (To Instrument the Code)
  36. 36. Testing Tools: On-Device Applications  Android  Dalvik Debug Monitor Server (DDMS)  Thread and Heap Information on Device  Screen Capture on Device  Incoming Call, SMS, Location Data Spoofing  Monkey Tool (for Both Simulator and Device)  Generate Random Streams of User Events Click and Touch  Windows Mobile  Hopper  Simulates Input Stress by Rapidly Sending Keystrokes and Screen Taps in a Random Fashion
  37. 37. Testing Tools: On-Device Applications  Symbian  Bullseye  Bullseye Coverage is a code coverage analyzer for C++ and C that tells how much of the application source code was tested. Code coverage analysis is useful during unit testing, integration testing, and final release.  EUnitPro  EUnit Pro enables creation and automation of unit and module tests for Symbian C++ Software.  Symbian AnalyseHeap Utility  This tool analyses the contents of Symbian OS heaps to produce meaningful representations of their structure and content.  TRK (Target Resident Kernel)  Enables On-Device Debugging  Yucca Tools  File browsing , Crash Monitor, Supported Fonts Preview, Memory Status etc.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.