Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Mobile Application Testing

21,714 views

Published on

The presentation basics about the mobile application strategies and concepts. Good source for the beginners. Get your app tested by us: info@swaam.com

Published in: Technology
  • Best Programming Languages for Mobile Application Development. https://goo.gl/qZnyJL
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Best Appium Training Institutes in Chennai http://thecreatingexperts.com/appium-training-in-chennai/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Mobile Application Testing

  1. 1. By Waqar Ahmad Swaam Tech (Pvt) Ltd Contact us: info@swaam.com for Ur App TestingNote: The slide was presented in the Software Testing Course at NUCES-FAST Lhr
  2. 2. Classification of Mobile Applications Web Applications  Could be Simple Markup (WML , HTML, XHTML, XHTML MP) or AJAX Applications  A Web Application may be Designed for  A Single Type of Device or  Multiple Different Devices with Customization for each Device (or each Family of Devices)  Multiple Different Devices—A Transcoder/Proxy Residing Between Web Application Server and Device Transforms Content based on Device User-Agent Capabilities
  3. 3. Classification of Mobile Applications On-Device Applications May be  Developed for Better Portability  Java ME  FlashLite  Native Platform Applications  IPhone: Objective C  Symbian: C/C++  Windows Mobile: .Net CF  Android : Java for Application and C/C++ for Low Level
  4. 4. Mobile App Testing—Challenges  Device Fragmentation is a Big Challenge  Devices Vary in Screen Size, Memory, Processing Power, Hardware Features etc.  Apple iPhone is Least Fragmented among All Mobile Platforms  Testing on All Target Handset/Devices  Almost Impractical if Number of Target Handsets is Large  Testing on All Target Operator Networks  Almost Impractical if Number of Target Operators is Large  Network Operator may Impose Certain Constraints
  5. 5. Mobile App Testing—Challenges Device Vendor (Not Necessarily Manufacturer) may Customize a Device  Changes may be Done in Both User Interface and Functionality  Many a Times Changes are Not Well-Documented  Device is Not the Same as Released by Manufacturer Testing on All Versions of Operating System  In Many Situations, it May Not be Possible to Upgrade the Device to New OS Version Same Device Models may have Multiple OS Configurations based on Target Market Requirements  Localization  Hundreds of Devices with Many Variants of Each Device
  6. 6. Mobile App Testing—Challenges Testing in All Web Browsers  Browsers Significantly Vary Across Devices  May Support HTML, xHTML, WML, AJAX Web Content Transcoders  Web Content may be Transformed to Meet User-Agent Constraints  Transcoder may Not Respect User Experience Factors in some Situations
  7. 7. Mobile App Testing—Challenges Inconsistencies in Specification Implementations  Each Vendor is Responsible for Java ME Specifications Implementations  Implementations may Not be Consistent across Devices and Vendors Performance Measurement  Extremely Difficult to Benchmark Performance due to Highly Fragmented Market Detection of Display Issues  Human Involvement is Must Generally it is Not Possible to Purchase All Target Handsets due to Financial Constraints
  8. 8. Mobile App Testing—Challenges Application Usability Testing  End Users may Not even be Skilled at Using a Mobile Device  End User may Use Application in Multiple Contexts  Driving in Rush Hour  Walking in Crowded Market  Multiple Input / Output Modalities  Diverse User Interface Styles In Reality, A Testing Lab Cannot Simulate all Possible End User Contexts
  9. 9. Mobile App Testing—Challenges Development and Deployment Platforms are Different  Development Platforms  Windows, Linux  Deployment Platforms  iPhone, Symbian, Android, Blackberry, Windows Mobile Emulators Do Not Capture All the Characteristics of a Device Multiple Design Specifications may be Required for Single Application due to Device Fragmentation— Requires Test Cases Accordingly
  10. 10. Testing Techniques: General Gather Useful Data Decide Target Handsets as Early as Possible in Development Lifecycle (Better in Requirements Analysis Phase) Document Incompatibilities in Devices e.g. Differences in  Installation Processes  Audio/Video Capabilities  Memory, Processing Power, Screen Size etc. Document Device Software (Firmware) Limitations  ‘Bugs in Firmware’ are Not Unusual Document Device Web Browser Constraints  Supported Version of HTML  AJAX Supported or Not
  11. 11. Testing Techniques: General Gather Useful Data Better Acquire Meta Data of All Target Handsets from an External Source  Extensive Databases Containing Device Meta Data  WURFL—An Open Source XML Database of Device Characteristics  DeviceAtlas—A Proprietary Database of Device Characteristics  Some Websites can Capture and Return HTTP Headers  For instance, Access the Website using a Phone, Website will Return Headers in HTTP Requests  Useful when Data is Not Available in Databases and also to Verify Already Available Data in Meta Data Source  A Development House may also Maintain an Internal Database Containing Data Not Available Otherwise
  12. 12. Testing Techniques: General Classify Devices Classify Devices  Category—1: All Devices that Support XHTML  Category—2: All Devices that Support AJAX  Category—3: All Devices that Support Java ME Pick Representative Device(s) from each Category for Testing Purposes Prepare Test Cases based on Data Gathered about Target Handsets
  13. 13. Testing Techniques: General Perform Testing on Real Devices Start Testing on Real Devices as Early as Possible There is No Good Alternative to a Real Device for Testing
  14. 14. Testing Techniques: General Emulator is Useful but is Not a Real Device Testing on Emulators  An Emulator is a Software Application that Imitates a Smartphlone  A Device Emulator may be Used to Test Application on Desktop  Some Emulators Emulate General Characteristics of a Platform (like Java ME Emulator)  Some Emulator Emulate Relatively Faithful Representation of Specific Phones (BlackBerry Dev Kits)  Emulator may or may Not Require Compilation for Target Platform  Single Compiled Code for both Emulator and Target Devices  Java ME  Windows Mobile  Different Compilers for Compilation for Symbian Emulator and Target ARM Device  Useful for Quick Testing in Development Environment  Emulators are Not Real Devices  Testing on Emulator does Not Guarantee Application Success on Real Device
  15. 15. Testing Techniques: General Application UI Should Preferably be Consistent with Device UI Style Test Application User Interface Consistency with Device Interface Style  Is the Application User Interface Consistent with User Interface of Underlying Device Applications?  Does the Application Offer Interaction Mechanism Similar to Interaction Mechanism Generally Implemented in Native Device Applications?  Softkey Mapping  Look&feel
  16. 16. Testing Techniques: General Different Test Cases are Required to Cater Diverse Input / Output Modalities Perform Testing for Various Input Modalities  Touch Screen  Testing with Single and Multiple Touch Input  Testing with Virtual Keyboard  Trackball/Track Wheel  Numeric Keypad  QWERTY Keypad  4- or 5-Way Navigator Perform Testing for Various Output Modalities  Different Screen Sizes
  17. 17. Testing Techniques: General Test Applications in Real User-Contexts Test the Application in Relevant User Contexts  User Context Involves User, Application and Surroundings  Design Various User Interaction Scenarios in Applicable Contexts (user driving in rush hour, walking, in classroom etc.)  Test Applications in Real Contexts
  18. 18. Testing Techniques: General Mobile Ecosystem Requirements must be Addressed Application are Generally Launched through Various Application Stores (Mobile Ecosystem)  Apple App Store, Nokia Ovi, Blackberry AppWorld, Android Market, GetJar, Handango etc. Test Applications to Meet Requirements of various Ecosystem Stakeholders  Manufacturers/Vendors  UI Guidelines from Apple for Iphone Applications  Symbian Signed  Operators  Operators may also have some Requirements  Third Parties  Java Verified Process
  19. 19. Testing Techniques: Web Applications Rules-Based Testing Rules-Based Testing with User-Agent Impersonation  Create Testing Rules based on  Device Capabilities  Known Bugs in the Device  Device Data in Data Sources (like WURFL, Internal Database)  Application Requirements  Test Cases are Written that Checks HTTP Response for Targeted Content  Impersonate a Device and Communicate with Web Server  Change HTTP Headers like  Accept: image/gif, image/jpeg  User-Agent: ‘NokiaN95…….+Profile/MIDP-2.1 + …..’  Check the Server Response Against Rules and Identify Discrepancies, if any
  20. 20. Testing Techniques: Web Applications Rules-Based Testing  Rules-Based Tests  Testing is Automated for Multiple Types of User-Agents  Testing can be Done Using  An HTTP Library for Sending Requests and Receiving Response  Web Browser (FireFox, Safari, Internet Explorer)  Useful to View the Rendered Screen with some Limitations  Rules Based Testing is Useful for Automated Testing of  Web Applications that Deliver Content on Phone  Regression Testing  Web Content Accessibility
  21. 21. Testing Techniques: Web Applications Transcoder Behavior must be Thoroughly Tested Transcoder Behavior  Web Content may be Transformed by a Transcoder to Meet User-Agent Constraints  A Transcoder Resides Between User-Agent and Web Server  Cookies should Not be Shared  Use Automated Tests for Communicating with Server in Parallel  User Experience Should Not be Affected  Manual Testing—See and Record the Behavior
  22. 22. Testing Techniques: Web Applications W3C mobileOK Standard  W3C has Defined mobileOK Tests  Tests for Checking Mobile Readiness (W3C Best Practices) of a Website  http://www.w3.org/TR/mobileOK-basic10-tests/  W3C mobileOK Checker  Performs mobileOK Tests on a Web Page to Determine its level of Mobile-Friendliness  http://validator.w3.org/mobile/
  23. 23. Testing Techniques: On-Device Applications Unit Testing and On-Device Debugging  Unit Testing Frameworks Exist for almost All Platforms  GUI Layer, Event Handling and System Testing is Not Possible  On-Device Debugging is Available on Most of the Platforms
  24. 24. Testing Techniques: On-Device Applications Custom Test Applications may Reveal Major Issues  Custom Test Applications  Can be Used to Test Critical Aspects  Whether an Application has Access to Device File System  Performance of Device Database System  Write Applications to Diagnose Key Issues e.g., Develop an Application that Helps in Testing When  Lot of Free Memory is Available on Device  Very Limited Memory is Available on Device
  25. 25. Testing Techniques: On-Device Applications Network Availability and Bandwidth  Perform Application Testing in Different Network Signal Strengths  No Network  Intermittent Connectivity  Strong Network Signals Presence  Application Testing May be Required in Various Network Types  2G, 3G
  26. 26. Testing Techniques: On-Device Applications External Events may Affect Application Behavior  Test Application against Events that may Occur during Application Installation and Execution  Phone Call Received  SMS Received  Battery Removed  Device Switched Off  Bluetooth Device Requested to Contact
  27. 27. Testing Techniques: On-Device Applications Combine ‘Automated Input’ with ‘Manual Verification’  Generate Automatic Events  Key Press  Pointer Press  Touch  Capture Screenshots during Event Generation Process and Compare them
  28. 28. Testing Techniques: On-Device Applications Track Behavior when App is being Used by Consumers  Track Application Behavior during Various Lifecycle Stages when in Production  Installation  Execution  Removal  Application may Send Important Data to Server that can be Used to Identify and Fix some Bugs  Java ME Includes some Parameters for this Purpose  ‘GetJar ’ Offers Such a Service  Application can Call Server and Report some Information  http://www.getjar.com/
  29. 29. Testing Techniques: CommercialGet Online Services for Devices Not Available In-house Testing on Real Devices that Developers Don’t Own  A Service that Enables Application Testing on Real Devices  Devices are Located on Remote Servers  Devices are Connected to Various Operator Networks Worldwide  Desktop Client Software/ Web Application Enables All Testing Features Remotely  Device Acquisition, Cleanup  Application Installation, Execution, Removal  Device Shutdown, Battery Removal and Installation  Service Providers  DeviceAnywhere (http://www.deviceanywhere.com/)  Over 2000 Devices Connected to over 30 Worldwide Carrier Networks across USA, Canada, UK, France, Germany, and Spain  Perfecto Mobile (http://www.perfectomobile.com/)
  30. 30. Testing Techniques: Commercial Crowdsourcing is also an Option CrowdSourcing (Crowd + Outsourcing)  Testing by Human Testers (Mobsters) Located Worldwide  If Done Effectively, Real Usability Testing Results can also be Obtained  Testing in Real User Contexts is Possible  How it Works?  Developers and Testers are Registered with a Third Party Service Provider  Third Party Service Provider is Responsible for Coordination of All Parties Involved in the Process (Application submission, monitoring tools, results reporting etc.)  Service Providers  Mob4Hire (http://www.mob4hire.com/)  GetJar (http://www.getjar.com/)  uTest (http://www.utest.com/)
  31. 31. Testing Techniques: Commercial User Feedback is Important App Stores may Provide Useful Feedback from End Users  User Reported Bugs  User Reviews on Usability  User Ratings Data is Helpful for  Quickly Fixing some Bugs  Future Application Releases
  32. 32. Testing Tools: Web Applications HttpUnit  Unit Testing for Web Applications HtmlUnit  Unit Testing with JavaScript Support Urllib  Provides APIs for Web Access Check a Web Application for Mobile Readiness  mobiReady: http://ready.mobi/launch.jsp?locale=en_EN  W3C mobileOk Checker: http://validator.w3.org/mobile/
  33. 33. Testing Tools: Web Applications User-Agent Capture Tools  Captures Characteristics of User-Agent  Content Type, HTML version Supported on Device etc.  Pycopia: http://www.pycopia.net/webtools/headers FireFox and Others Browsers—Use User-Agent Switcher Add-On. However, there are Some Issues  Differences in how Desktop and Mobile Browsers Layout Content may Render Testing Results Unreliable  Performance Differences in Desktop and Mobile Browsers  Feature Differences in Desktop and Mobile Browsers
  34. 34. Testing Tools: Web Applications Desktop Browser Automation Tools  WebDriver (http://code.google.com/p/webdriver/)  Java APIs that can be Integrated with Eclipse  Selenium (http://seleniumhq.org/)  Firefox add-on that records clicks, typing, and other actions to make a test, which can be played back in the browser.  Selenium RC can be Used in a Desktop Browser with HTTP Header Emulation IPhoneDriver  A WebDriver based Tool for Testing on UIWebView
  35. 35. Testing Tools: On-Device Applications  iPhone  Embedded Browser Component (UIWebView)  iPhoneUnitTesting  Memory Sweep (For memory snapshot of iPhone)  Java ME  J2MEUnit (For Unit Testing)  JInjector (For System Testing)  ASM (To Instrument the Code)
  36. 36. Testing Tools: On-Device Applications  Android  Dalvik Debug Monitor Server (DDMS)  Thread and Heap Information on Device  Screen Capture on Device  Incoming Call, SMS, Location Data Spoofing  Monkey Tool (for Both Simulator and Device)  Generate Random Streams of User Events Click and Touch  Windows Mobile  Hopper  Simulates Input Stress by Rapidly Sending Keystrokes and Screen Taps in a Random Fashion
  37. 37. Testing Tools: On-Device Applications  Symbian  Bullseye  Bullseye Coverage is a code coverage analyzer for C++ and C that tells how much of the application source code was tested. Code coverage analysis is useful during unit testing, integration testing, and final release.  EUnitPro  EUnit Pro enables creation and automation of unit and module tests for Symbian C++ Software.  Symbian AnalyseHeap Utility  This tool analyses the contents of Symbian OS heaps to produce meaningful representations of their structure and content.  TRK (Target Resident Kernel)  Enables On-Device Debugging  Yucca Tools  File browsing , Crash Monitor, Supported Fonts Preview, Memory Status etc.

×