Need:The Mobile Boom : In 2010, mobile app stores will recorded an estimated $6.2 billion in overallsales, including 4.5 billion app downloads. By 2013, some analysts expect mobile app revenuesto exceed $21 billion. With charts like this, we wouldn‘t bet against them.Challenges: What platform is it running (Windows Mobile, Symbian, J2ME, RIM, Android, etc)? Do you know its limitations (RAM, Storage Memory, Maximum application size, etc)? Do you know what type of connection can you use (Edge, 3G, CDMA, Wifi, etc)? Device fragmentation, which means that devices come in many forms. The main factors for the diversity are hardware, software, user preferences and localization and environment. Device and its Versions: Samsung, iPhone, LG, Nokia 5800, BlackBerry Bold, or Sony Ericsson W910
Native Apps or Mobile Web? A native app as one that is specifically designed to run on a device’s operating system and machine firmware, and typically needs to be adapted for different devices. A Web app, or browser application, is one in which all or some parts of the software are downloaded from the Web each time it is run. It can usually be accessed from all Web- capable mobile devices.
Other Considerations: DEVICE MANUFACTURERS Operating Systems Carriers ANDROID Devices Available in the Market:
Data base TestingDatabase Testing is very important for all applications. We check for data integrity and errorswhile editing, deleting and modifying the forms and all other DB related functionality. Thistesting is done manually, without the use of any testing tools.Compatibility TestingCompatibility Testing assures the application works as intended with the selected device,operating system, screen size, display and internal hardware. Following are a list of tools thatsimulate different devices, operating systems, screens, etc.: o iPhoney is a free iPhone simulator powered by Safari (used on a MAC OS platform only).
o iPad Peek allows you to see how your websites look when rendered on the iPad. This simulator is also free. o Adobe Device Central CS5 allows you to plan, preview, and test and delivers mobile applications. It is available with the Adobe Creative Suite® editions: Photoshop, Illustrator, Flash Professional, Dreamweaver After Effects and Fireworks. o DeviceAnywhere™ allows you to compose automated tests that run across multiple devices and multiple platforms/OS’s. DeviceAnywhere™ is a paid solution providing monthly and/or hourly options.Functionality TestingFunctionality Testing includes the testing of controls, storage media handling options, and otheroperational aspects. Functionality testing for the mobile application is black-box testing andassures that the application functions per the business specifications. This testing is donemanually.Power Consumption Testing uncovers defects related to battery drainage caused by theapplication. Device settings can drain the battery life and this makes it hard to determine if themobile application or the settings are the cause. Following are list of devices and the differenttesting methods for testing power consumption: o iPhone, iPod & iPad settings are adjusted; Screen Brightness, Minimize use of location services, Turn off push notifications, Turn off other Downloaded Applications, Fetch new data less frequently and Turn off push mail. Then run the mobile application to determine the rate it took for the battery life to decrease. This testing is done manually without any testing tools. o Nokia Energy profiler is a stand-alone test and measurement application which lets you monitor the battery consumption on target device.Usability TestingUsability specifies the ease, elegance and clarity with which a user can perform a task toachieve a goal. Focusing on usability and user experience is a key element in creating successfulhigh-quality applications. Usability ensures that the user is able to interact with the applicationproperly and a positive user experience makes the use emotionally desirable and pleasing.
Interoperability testing is the activity of proving that end-to-end functionality between (atleast) two communicating systems is as required by those base systems standards.Performance Testing Memory CPU, Power, Battery (Comparing 2 phones together) Network UtilizationOther Testing Installation/Un-installation and product activations UI Testing File formats and conversions Device and application backup Device-PC sync Stability Connectivity and data traffic – Use sniffers to monitor traffic and find bottlenecks or data scrambling (mainly done by specific carriers which plant their data in incoming/outgoing XMLs) Benchmark TestingUpgrade Testing App and OS UpgradesEase of Installation Notifications of Upgrades Use App Store to distribute Is install experience clear and concise? Installation for Android o Through Market::
Copy the APK file to your Android’s memory card and insert the card into your phone. Download and install the Apps Installer application from the Android Market o Through Android SDK Configure your Android device: Settings > Applications > Check Unknown sources Settings > Applications > Development > Check USB Debugging Settings > Applications > Development > Check Stay Awake Connect your Android Phone to your PC iwth a USB cable Open your project in Titanium Studio: Run Menu in App Explorer Bar > Android Device o adb -d install command Installation for iOS o Through PC using iTunes. Connect to the Internet, open iTunes, and click on iTunes Store in the left- hand column of the window. When the iTunes Store appears click the link for the App Store. Browse the App Store Confirm the purchase/cost for apps. Click Buy or Cancel, as appropriate. o Through AppStore Launch Appstore Browse Apps available Confirm cost and read details. Click Buy buttonHow to trace LogsIn iOS through Xcode - Logs and screenshots
- File System BrowsingIn Android through DDMS and LogcatEMULATORS Not good for Performance/resource/sensor/memory testing Good for Functionality testing Emulators Available: Android emulator, Blackberry emulator, Dot Mobi emulator, Firefox Mobile emulator, iPhone / iPad / iPod Touch emulator, Klondike WML emulator, LG emulator, Microsoft Devices emulator, Motorola emulator, Mozilla Fennec emulator, NetFront emulator, Nokia emulator, Opera Mini emulator, Opera Mobile emulator, Palm emulator, Palm Pre / iPhone emulator, Samsung Java emulator, Samsung Platform emulator, Windows Mobile emulatorApps for Performance Testing First Party: Xcode(iOS) Third Party: Available through Market Place (Android: System Profiler, Battery Mix, Task Manager)Tools to aid in Mobile Testing:>>Deviceanywhere, Perfectomobile, foneMonkey , Robotium, Plank, Selenium ,<< FlurryFor Unit Testing / RSpec :OCUnit/SenTestingKit, CATCH, GHUnit, Kiwi, Cedar ,JasmineFor Acceptance Testing:UI Automation, Frank, Fone Monkey, KIFSeeTest from Experitest - has a recorder and plugs into all existing testing frameworks such asQTP, TestComplete, MSTest, RFT and open source frameowrks such as JUnit, Perl and Python.Free trial downloads www.experitest.comFor UI testing for iOS: Zucchini Framework Commonly Used Tools for iOS & Android are:For iOS Testing : XCode, Instruments, PlankAndroid Testing:
DDMS: DDMS stands for “Dalvik Debug Monitor Server” and is a tool that is included in theAndroid SDK, in the tools/ directory. Type ddms.bat command in command prompt Click on device under test and view logs available. There are seven tabs on the right hand side of the DDMS tool such as: o Allocation Tracker tab which helps in: Memory Usage Tracking of memory allocated to objects Tracking of threads & classes that are allocating objects o The Threads tab in DDMS shows you the currently running threads for a selected process. Select the process that you want to examine the threads for. Click the Update Threads button. In the Threads tab, you can view the thread information for the selected process. “main” (see name column) thread is your applications process Click on refresh button to view class and method information. If Native = false, method/class is from application, else method/class is from Android OS. File System DDMS provides a File Explorer tab that allows you to view, copy, and delete files on the device. You can view this tab by going to the ‘Device’ menu and clicking on the ‘File Explorer’ tab. using this feature the user can examine files that are created by your application or if you want to transfer files to and from the emulator/device to the local machine. To work with an emulator or devices file system: In the File explorer tab, select the file that you want ot view. To copy a file from the device/emulator, locate the file in the File Explorer and click the Pull file button. To copy a file to the device/emulator, click the Push file button on the File Explorer tab.Vertual Device Testing Subscription Based Multiple Vender o Device Anywhere o Perfecto Mobile
Remote Access to hundreds of Real devices(All Platform) New devices added regulary Save money buying your own deviceAutomation Still in its infancy,not large selection of options Built in security of mobile platforms limits capabilities & cross app testing Virtual Device Companies offer Automation o Requires dedicated device o Costly Third Party Automation options o Platform specific o Free and paid o Most are screen scrappersCOMMUNITY TESTING/CROWD Testing:Crowd testing is when a large group of perfect strangers try your product then give youphenomenally helpful feedback on usability, bugs and features.Important Tips to Mobile Testing: - For in-house apps , start testing when development stage is in progress - Know your target platform & devices, versions - Don’t assume that if app works on one device, it will work on all - Don’t underestimate the importance of Performance Testing. - Always have at least a few physical devices available for testing (especially the old ones like iphone 3gs and early android etc) - When possible have group of Beta Testers to help with real world testing - Don’t under/over-estimate the importance of anecdotes test results - Know your target audience - Be aware of your app and OS upgrades, either of them can break your app. - Carriers – Test your application via different carriers (e.g. Vodafone, O2, Orange, AT&T, Cingular, etc). You may find some differences when running the SAME application on different carriers. - Resolutions – Get familiar with different resolutions, e.g. 240×260, 320×240, 128×128 (square devices) and identify differences and UI bugs. For more advance testers, plan a
test matrix which covers all possible/available resolutions for the tested devices/platforms. - Connections/airplane Mode – Test your application with different connection types (Edge, 3G, CDMA,Wifi) and also in airplane mode on supported devices. Also test GPS location connectivity. - Device Settings - Change the device settings: Date and Time, Languages (Advanced regional settings), Connections, Shortcuts – anything you have control over – and learn how each and combined settings impact the device. - Interrupts – Test how the device and applications behave on interrupts, e.g. Incoming/outgoing Call, SMS, MMS, Video-Call, Low battery, Low/No connectivity. - Local Device Storage Vs. Memory Card – Install/Uninstall and test your application on both local device storage and memory card. You’ll be amazed by applications behavior when installed on each storage option. - Device input entry modes – Physical keyboard, Virtual keyboard/Touch screen. -References Used: http://maasters.maas360.com/forum/expertise/ http://www.mobileappstesting.com/category/mobile-application-testing/mobile-application- testing-and-automation-tools/ Microsoft Windows Mobile Emulator:http://www.youtube.com/watch?v=NHEVGZBHON8 Best Practices for Testing Mobile Applications http://www.youtube.com/watch?v=B0E0BSc55Eg http://mobithinking.com/native-or-web-app http://www.zucchiniframework.org/ >>SeeTest http://www.youtube.com/watch?v=LRoK2QZFq1o&feature=related (http://experitest.com/ Web Tools: http://sixrevisions.com/tools/10-excellent-tools-for-testing-your-site-on-mobile- devices/ Deviceanywhere:http://www.keynotedeviceanywhere.com/ Perfectomobile: http://www.perfectomobile.com/ http://www.youtube.com/watch?v=or26ldUu6YQ MobileCloud for QTP http://www.perfectomobile.com/
For Unit Testing/ Acceptance Testing Refer:http://stackoverflow.com/questions/4114083/ios-tests-specs-tdd-bdd-and-integration-acceptance-testinghttp://www.developer.nokia.com/Community/Wiki/Usability_Testing:_Key_for_developing_high_quality_mobile_applicationsFor ddms usage: http://mobapptesting.blogspot.in/2012/01/monitor-performance-using-ddms-android.html