Performance Testing for Mobile Apps - Introduction

1. Performance Testing Mobile
Apps
Session 1
By: Jyothirmayee Pola

2. App on Mobile Device vs UI
• Different devices in market which differ in screen size, memory,
enhancement mem availability, OS, touchscreen or not etc. Bottom line:
Same script cannot be re-used if any of the above factors change.
• Different devices connect differently to the server as compared to browsers
from desktop. Tech stack differs with each OS.
• More number of mobile users as compared to desktop users as on today
• Network quality
• Network latency affect is significantly higher in devices as compared to
desktop
• More packet loss with devices comparatively
• Different apps behave differently on the same device though they are
comparatively same apps. Example: Myntra which is a dedicated mobile
site as compared to Jabong which is a responsive web design app

3. App on Mobile Device vs UI
• Mobile layouts differ using the same device for the same app. Example:
landscape or portrait
• Browser cache limitation on device as compared to browser on desktop
• More frequent OS changes on devices and upgrades on mobile apps. Harder to
control upgrades.
• Keypad locks on device affect performance of the same app on the same device.
• Offline apps perform differently as compared to online apps on devices. There is
no such concept on desktop applications
• Native drivers on device
• Content rendering time on device: The time it takes to display the response
delivered to the client on the device is measured here. The platform type,
configuration of device and other hardware configurations affects this time.
• Last but not the least: limited number of tools to support device testing

4. Types of Mobile Apps – Native App
• Live on the device and launch by tapping icons on home screen. Executed
directly by OS
• Installed through an application store Ex: Google Play or Apple’s App Store
• A binary executable is directly downloaded from app store and stored on
the device file system
• Developed specifically for one platform and firmware to take full advantage
of all the device features Ex: Camera, GPS, Contacts, device specific
gestures etc
• Apps explicitly use the device’s/ Operating System’s APIs
• Allows function without internet connection
• Examples of native apps are Twitter, Skype, Tagging, Ingress, Instagram

5. Types of Mobile Apps – Native App
• Performance is better with Native Apps as they will have access to all native APIs
• Requires users to download and install updates. Or, at least approve them if set in
auto mode.

6. Native App – Continued

7. Types of Mobile Apps – Web App
• Websites which look feel like native apps Ex: Uber, Shutterfly, Google maps,
Linkedin
• Run by browser and typically written in html5, Javascript, CSS or other
languages.
• Once opened from browser, they navigate to a special url which in turn
gives the facility to download like an app. Ex: Financial times, Times of India
etc
• With browser caching its possible to access web apps offline
• Native APIs of the device are not exposed to web apps
• Performance depends on the browser speed, cache, internet connection’s
speed
• Same web app can be used on multiple devices of different make and
models

8. Web App – Continued …
• Latency over the air transmission is the biggest culprit for
performance degradation
• Performance of a web app is different with Wi-Fi, 3G, 2G, public Wi-Fi
etc
• User experience is lower as compared to native apps

9. Web App – Performance caveats
• Poorly written code: The same apps are written commonly for multiple
devices of different make model. Can never scale up for all devices equally.
• Start up or page load time depends on various factors like browser version,
available RAM on the device, app framework compatibility with the device
under test, device power, frame rate of the device, firmware version etc
• Parallel browsers or applications opened at the same time on the device
may lower the app performance under test
• How network calls and computations are handled. If they are not handled
in a background thread then, server response will increase.
• Performance due to not incorporating lazy loading
• All data needs to be downloaded for an fully operational app. Once closed,
the app (generally) will download all data again.

10. Web App – Performance caveats
• CPU intensive calls and complex video/ page rendering are to be checked
• Network calls during the use case flow need to be monitored and
optimized for better performance. Consider queuing multiple network calls
and sending them together in a single request.
• Usage of JSONs over XMLs will improve performance. Gzipping payload is
another option.
• Restricting the ad usage on the app
• Optimize algorithms: return to your computer science roots and analyze
the order (big Oh) of your algorithms and try running your application
inside a code profiler that will identify your bottlenecks

11. Web App – Performance caveats
• Choose resources with lesser resolution (easier to paint) for devices
with screens of lesser resolution. For example, the latest Samsung
Galaxy S5 has a 1920x1080 resolution, but the two-year-old Galaxy S3
only has a 1280x720 resolution. There is no benefit to sending a
device images with greater resolutions than the device itself
• Gracefully degrade complex functionality: remove some of the nice-
to-have features such as anti-aliasing, complex texture mapping, and
so forth from your painting algorithms if you identify performance
issues. Users with powerful devices will see all of the niceties, but
users with less powerful devices will still have a good user experience.

12. Types of Mobile Apps – Hybrid App
• They install on your device by downloading them from app stores
• Built with a combination of web technologies like HTML, CSS, and
JavaScript
• Hybrid apps are hosted inside a native application that utilizes a mobile
platform’s WebView
• Access device capabilities such as the accelerometer, camera, contacts, and
more
• Most hybrid mobile applications leverage Apache Cordova
(earlier PhoneGap), a platform that provides a consistent set of JavaScript
APIs to access device capabilities through plug-ins, which are built with
native code
• Android 5.0 introduced updatable WebViews via the Android System
WebView

13. Hybrid App – Continued…

14. Comparison

15. Comparison

16. Comparison

17. Comparison

18. Comparison

19. Comparison

20. Interesting Stats
The top 5 operating systems according to IDC’s analysis and their percentage of market
share:
• Android: 78.6%
• iOS: 15.2%
• Windows Phone: 3.3%
• BlackBerry: 1.9%
• Other: 1.0%
An article in Apple Insider says iOS operating system versions are distributed as follows:
• iOS 7: 74%
• iOS 6: 22%
• Older: 4%

21. Interesting Stats
According to Google, Android operating system versions are distributed as
follows:
• 4.4 (KitKat): 8.5%
• 4.3: 8.5%
• 4.2.x: 18.8%
• 4.1.x (Jelly Bean): 33.5%
• 4.0.3 4.0.4 (Ice Cream Sandwich): 13.4%
• 3.2 (Honeycomb): 0.1%
• 2.3.3 2.3.7 (Gingerbread): 16.2%
• 2.2 (Froyo): 1%

22. Interesting Stats