Mobile App Testing Checklist


Published on

Published in: Technology, Business
  • Really a nice content...Thanks a lot
    Are you sure you want to  Yes  No
    Your message goes here
  • This is a really good thorough checklist. I've been doing the same thing by keeping track of things I have forgotten to test in the past with iOS development. My list, which isn't very big yet, can be found on my iOS Blog: Thanks for sharing this checklist, this gives me some good ideas.
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Mobile App Testing Checklist

  1. 1. Author Mobility & App testing CommunityVersion 1.0Date June 20, 2012Location VianenCharacteristics Final versionTMAP NEXTCHECKLIST MOBILE APP TESTEN
  2. 2. Checklist Mobile App testenTable of contentsINTRODUCTION...................................................................................2DEVICE SPECIFIC CHECKS........................................................................3NETWORK SPECIFIC CHECKS....................................................................5APP SPECIFIC CHECKS............................................................................6APP USER INTERFACE CHECKS..................................................................7STORE SPECIFIC CHECKS.........................................................................9TERMINOLOGY...................................................................................13DOCUMENTATION...............................................................................14Sogeti Nederland B.V. 1.01June 20, 2012
  3. 3. Checklist Mobile App testenINTRODUCTIONThis checklist is specifically designed to test the characteristics of a mobile app.Obviously, it tests only generic app characteristics and not the functionality of the app.For this a separate test approach and test script must be created. The same goes ofcourse for performance testing, usability testing, security testing and other testingactivities necessary for your specific app.The checklist is split into five different fields:• Device specific characteristics. These are characteristics that are related to thedevice on which the app is installed.• Network specific checks• App checks. These are things to check that have to do with functionality that isfrequently used in an app.• App User interface checks.• Store specific checks.The checks don’t have to be executed in the order they are given.If you have any questions or suggestions, please contact us at tmap@sogeti.nlSogeti Nederland B.V. 1.02June 20, 2012
  4. 4. Checklist Mobile App testenDEVICE SPECIFIC CHECKS# Description Pass/FailRemarks1.1 Can the app be installed on the device? N/A1.2 Does the app behave as designed/desired ifthere is an incoming call?N/A1.3 Does the app behave as designed/desired ifthere is an incoming SMS?N/A1.4 Does the app behave as designed/desired ifthe charger is connected?N/A1.5 Does the app behave as designed/desired ifthe charger is disconnected?N/A1.6 Does the app behave as designed/desired ifthe device goes to sleeping modeN/A1.7 Does the app behave as designed/desired ifthe device resumes from sleeping modeN/A1.8 Does the app behave as designed/desired ifthe device resumes from lock screen?N/A1.9 Does the app behave as designed/desired ifthe device is tilted?N/A1.10 Does the app behave as designed/desired ifthe device is shaken?N/A1.11 Does the app behave as designed/desired ifa local message is coming from anotherapp (think of: calendar reminders, to-dotask etc.).N/A1.12 Does the app behave as designed/desired ifa push message is coming from anotherapp (think of: twitter mentions, whatsappmessage, wordfeud invitation, etc).N/A1.13 Does the app interact with the GPS sensorcorrectly (switch on/off, retrieve GPS data)?N/A1.14 Is the functionality of all the buttons or keyson the device defined for this app?N/A1.15 Verify that buttons or keys which have nodefined function have no unexpectedbehaviour on the app when activating.N/A1.16 In case there’s a true “back” buttonavailable on the device does the “back”button take the user to the previous screen?N/A1.17 In case there’s a true “menu” buttonavailable on the device, does the menubutton show the app’s menu?N/ASogeti Nederland B.V. 1.03June 20, 2012
  5. 5. Checklist Mobile App testen1.18 In case there’s a true “home” buttonavailable on the device, does the homebutton get the user back to the home screenof the device?N/A1.19 In case there’s a true “search” buttonavailable on the device, does this get theuser to some form of search within the app?N/A1.20 Does the app behave as designed/desired ifthe “Battery low” message is pushedN/A1.21 Does the app behave as designed/desired ifthe sound on the device is turned off?N/A1.22 Does the app behave as designed/desired ifthe device is in airplane mode?N/A1.23 Can the app be de-installed from the device? N/A1.24 Does the application function as expectedafter re-installation?N/A1.25 Can the app be found in the app store?(Check after go-live)N/A1.26 Can the app switch to different apps on thedevice through multitasking asdesigned/desired?N/A1.27 Are all touch screen positions (buttons)working when a screen protector is used.N/ASogeti Nederland B.V. 1.04June 20, 2012
  6. 6. Checklist Mobile App testenNETWORK SPECIFIC CHECKS# Description Pass/FailRemarks2.1#Does the app behave according tospecification if connected to the internetthrough Wi-Fi?N/A2.2 Does the app behave according tospecification if connected to the internetthrough 3G?N/A2.3 Does the app behave according tospecification if connected to the internetthrough 2G?N/A2.4 Does the app behave according tospecification of the app is out of networkreach?N/A2.5 Does the app resume working when it getsback into network reach from outsidereach of the network?N/A2.6 Update transactions are processed correctlyafter re-establishing connection.N/A2.7 Does the app still work correctly whentethering or otherwise connected toanother deviceN/A2.8 What happens if the app switches betweennetworks (Wi-Fi, 3G, 2G)N/A2.9 Does the app use standard network ports(Mail: 25, 143, 465, 993 or 995 HTTP: 80or 443 SFTP: 22) to connect to remoteservices, as some providers block certainports.N/ASogeti Nederland B.V. 1.05June 20, 2012
  7. 7. Checklist Mobile App testenAPP SPECIFIC CHECKS# Description Pass/FailRemarks3.1 Has the app been tested on different type ofdevices and different versions of OS?N/A3.2 Stability check: if the app has a list (forinstance of pictures) in it, try scrollingthrough it at high speed.N/A3.3 Stability check: if the app has a list (forinstance of pictures) in it, try scrolling tobefore the first picture or behind the lastpicture.N/A3.4 Is downloading of the app prevented in caseit’s bigger than the OS allows downloadingwhen connected to cellular networks.N/A3.5 Integration: does the app connect correctlyto the different social networks (LinkedIn,twitter, facebook, etc).N/A3.6 The app does not interfere with other appswhen in background/multitasking mode(using GPS, playing music, etc.).N/A3.7 Can the user print from the app (ifapplicable)N/A3.8 The search option in the app displaysrelevant resultsN/A3.9 Verify most common gestures used tocontrol the app.N/A3.10 What happens if you select different optionsat the same time (undesired multitouch,for example – select two contacts fromthe phone book at the same time).N/A3.11 App name should be self explanatory N/A3.12 Does the app limit or clean the amount ofcached data.N/A3.13 Reloading of data from remote service hasbeen properly designed to preventperformance issues at server-side.(manual reloading of data can reduce theamount of server calls)N/A3.14 Does the app go to sleep mode whenrunning in the background (preventbattery drain)N/ASogeti Nederland B.V. 1.06June 20, 2012
  8. 8. Checklist Mobile App testenAPP USER INTERFACE CHECKSThis checklist is based on the recommendations of Apple and some other experts. Thischecklist is not a substitute for a usability test, to get a good feeling of the userexperience of an app a user experience test is always the most reliable method. Userinterface checks toelichten.# Description Pass/FailRemarks4.1 To keep controls as unobtrusive as possiblefor instance by fading them out if they arenot used for a while.N/A4.1 Make it possible for users to go back to aprevious screen for instance by adding aback or cancel buttonN/A4.2 The main function of the app should beapparent immediately. It should speak foritself.N/A4.3 Use at most one action on the screen that ishighlighted as the most likely for theuser. (Example: in iOS a blue buttonrepresents the default or most likelyaction).N/A4.4 Minimize user actions by using a picker or atable view where users can select acertain choice over a data entry fieldwhere users have to type a choiceN/A4.5 In an app, the user should not be able tostore files locally, outside the appsandbox.N/A4.6 In an app, the user should not be exposedto the permissions of a specific fileN/A4.7 If there is a long list of data to scroll trough,provide a search option above the list.N/A4.8 If performance is slow, indicate a progressstatus icon (“Loading…”), preferably withspecific message.N/A4.9 In case of ‘live’ filtering of data while theuser enters his search query, verify theperformance.N/A4.10 The appearance of buttons that performstandard actions are not altered in theapp (for instance: refresh, organize,trash, Reply, back, etc.)N/ASogeti Nederland B.V. 1.07June 20, 2012
  9. 9. Checklist Mobile App testen4.11 Do not use standard buttons for otherfunctions then that they are normallyused forN/A4.12 The app should respond to all changes indevice orientation, as per the designN/A4.13 Tapable elements should be about 7x7 mmin size, using the pixel density of thetarget device you can calculate theamount of pixels (chapter documentationcontains a link to different devicescompared).N/A4.14 Do not redefine gestures in your app thathave a standard meaning (example:swiping from top to bottom enables thenotification center)N/A4.15 Requirement to login is delayed in the appas long as possibleN/A4.16 If the app is stopped at an unexpected time,user data should be saved locally andavailable at start-up.N/A4.17 Users should be warned of the consequencesof deleting a documentN/A4.18 Keyboard adjusts to expected input (forinstance numbers/letters whenexpected).N/A4.19 Are inactive buttons clearly distinguishedfrom active buttons?N/ASogeti Nederland B.V. 1.08June 20, 2012
  10. 10. Checklist Mobile App testenSTORE SPECIFIC CHECKSThese checks are important to be confident that an Apple app will pass the submissionprocess of the app store.These checks are largely based on the apple App store guidelines. The other leading appstore, Google Play, has much less strict demands. Other app stores like the Amazon appstore have stronger restrictions, we’ve added them in a special paragraph in case they arerelevant.Keep in mind that the Apple app store submission process is not crystal clear. On manyoccasions, it is unclear why apps are rejected or accepted. Keep the following line in mindfrom the apple app review guidelines:We will reject Apps for any content or behaviour that we believe is over the line. Whatline, you ask?Well, as a Supreme Court Justice once said, "Ill know it when I see it". And we think thatyou will also know it when you cross it.Note: the guidelines that are covered here are the ones that are special to the app store.For instance: Guideline 2.1 states that “Apps that crash will be rejected”. Making surethat an app doesn’t crash is not remedied by a simple check – it requires a functionaltest.Second Note: Part of the iOS guidelines is the following guideline 10.1 Apps must complywith all the terms and conditions explained in the Apple iOS Human Interface guidelines.We’ve incorporated these in the Chapter “App user interface checks”.So if you’re not planning to submit the app to an app store, these checks are notnecessary? In principle, yes. However, many of these have relevance even if your app isnot distributed through a store but for instance to a limited set of your employees. Sothese checks might still be useful.# Description Pass/FailRemarks5.1 The app can’t use any “non-public API’s”.This means that you can’t use somefunctions that the distributing platform usesfor its own apps. (This can generally bechecked best by some sort of automatedtool, like The app can’t reprogram controls of thedevice that are not intended for that use.(For instance: using the volume button as ashutter for the camera).N/A5.3 The app should not access information on N/ASogeti Nederland B.V. 1.09June 20, 2012
  11. 11. Checklist Mobile App testenthe device outside the app without theuser’s permission (for instance, copying theaddress book or getting information fromother apps).5.4 The app should not access or write filesoutside the “Bundle” and “Documents”directory. (because the app can’t read orwrite data outside the designated containerarea).N/A5.5 The app cannot download code to beinstalled without the users consent.N/A5.6 The app can only get new functionality byway of an upgrade through the app store.N/A5.7 After download, an app should remainworking. An app cannot turn off after a fewdays.N/A5.8 An app can’t be a “trail”, “beta”, “demo” or“test” version.N/A5.9 Apple product names should be spelledcorrectly in the app. (For instance: IPhonezis wrong).N/A5.10 If the app uses the web, it is not done usingthird party (i.e. non-Apple) browsers.N/A5.11 You cannot mention other app platforms inyour app (for instance: “Also available onandroid!”)N/A5.12 An app cannot use old interfaces, like forinstance the iPod click wheel.N/A5.13 Multitasking functionality of the app can onlybe used for its intended purposes, i.e. VoIP,Audio playback, location, task completion,local notifications, etc. This means thatgenerally an app can’t run in the backgroundbut has to be closed off if it’s not used anymore.N/A5.14 The app must have some functionality. Forinstance, it can’t be just a title page leadingto some text. It can’t be just a song, movieor book as there are different platforms forthat.N/A5.15 Functionality should be in sync withfunctionality described in store.N/A5.16 In general, the app has to be decent. So no N/ASogeti Nederland B.V. 1.010June 20, 2012
  12. 12. Checklist Mobile App testenexplicit material in the sense of sex,violence, drugs, alcohol or tobacco. Itcannot address a specific ethnic or religiousgroup in a derogatory way.5.17 The app has to be honest. This means thatthe description of the app has to be correct,and all functionality has to work asdescribed. If an app gives diagnosticinformation, it has to be reliable. This alsomeans that the genre and category in thedescription must be appropriate. The appicons should be consistent and appropriate.N/A5.18 An app can’t restrict the users of the app forinstance by location or carrier.N/A5.19 An app cannot send spam or introduceviruses, or use other apple platforms likeGame Center and Push Notifications to doso.N/A5.20 The app should aim at backing up aminimum of information on iCloud. Theinformation in iCloud should be just the usergenerated information. Information that canbe recreated or downloaded should not bebacked up.N/A5.21 An app cannot use location services of thedevice without asking permission.N/A5.22 All url’s in the app code should be fullyfunctionalN/A5.23 The app can’t use the user’s location withoutpermission.N/A5.24 The location services cannot be used toautonomously control of vehicles or foremergency services.N/A5.25 An app cannot use push notifications withoutuser consent.N/A5.26 Push notifications have to be send using theApple Push Notification (APN) API. This hasto be done using an APN ID.N/A5.27 Push notification can’t send personalinformation.N/A5.28 The App may not distribute any privateinformation of users (like Player ID) throughthe game center.N/ASogeti Nederland B.V. 1.011June 20, 2012
  13. 13. Checklist Mobile App testen5.29 Ad banners must be hidden when there areno ads available.N/A5.30 The app must respect copyright of apple andother parties.N/A5.31 The in app purchase mechanism cannot beused to purchase goods and services usedoutside the app.N/A5.32 The in app purchase mechanism cannot beused to collect money for charities. This hasto be done through SMS.N/A5.33 The in app purchase mechanism cannot beused to buy a raffle or lottery ticket directlyfrom the app.N/A5.34 Apps that encourage the users to use thedevice in a way that may damage the devicewill be rejectedN/A5.35 An app cannot require user’s personalinformation (for instance email address) inorder for it to function.N/ASogeti Nederland B.V. 1.012June 20, 2012
  14. 14. Checklist Mobile App testenTERMINOLOGYTrue button = Hardware button, opposite of a touch screen button.Sogeti Nederland B.V. 1.013June 20, 2012
  15. 15. Checklist Mobile App testenDOCUMENTATIONThe following documentation has been consulted compiling this checklist: App store review guidelines: Data storage guidelines Human interface guidelines: details:Pixel density compared: Nederland B.V. 1.014June 20, 2012