The webinar discusses performance testing for mobile applications, emphasizing the importance of optimal testing strategies, particularly in varying network conditions. It highlights that user experience is critical, with abandonment rates increasing if apps respond slowly, and suggests solutions such as network virtualization for comprehensive testing. A case study on a financial app reveals the challenges of testing under different network configurations while aiming for enhanced user experience and faster delivery.

1. April 2015
Hexaware & Experitest Co-webinar:
Performance Testing- Is Your Mobile Application Ready for Scale?

2. Speakers
Guy Arieli
CTO
Experitest
Vinayak Gaur
Head of Mobile
Application Testing
Hexaware

3. Agenda
Optimal testing strategy
1
2
3
5
6
Zoom into Mobile Network Virtualization
Why performance testing is important?
Experitest Solution
7
Case Study
Demo
Importance
Basic terms
Challenges
Q & A

4. Mobile (App) Experience
60%
of mobile users will abandon your app if it
doesn’t respond within three seconds or
provide the desired user experience in any
circumstance (Google)
>65%
Over 65% Of the apps on App Store and
Play Store have poor ratings (2 or less stars)
Brand
A poor mobile experience can have a
negative impact on the transaction; and in
the long term, it could be even more
detrimental to a brand’s reputation.

5. Composite Testing Strategy = Optimal Testing Strategy
Is this the new Trend?
Compatibility
Regression Test
Automation
Network
Virtualization
Functional Security
Device level
Performance
Usability /
Platform
Server Level
Performance
Compliance
> Segregation of Horizontal and vertical Test types
> Combination of 2 or more test types
> Implementation Strategy defines the parameters for combining Test types
> Network Virtualization provides critical coverage across test types

6. Network Virtualization Testing
Why is it Important for your testing Strategy?
30% of bugs found by end users were caused by running the application in "non optimal" network conditions
Functionality
Fragmentation
Performance
Network Virtualization

7. Drilling Down Into Network Virtualization

8. Network Virtualization- Basic Terms
If your network is
a water pipe your
data is the water
Throughput
The width of the pipe, or how
match water it can move in a
second.
Latency
The length of the pipe , or how much time it will take for a drop of water to path the pipe.

9. What have Changed When We Moved to Mobile ?
Built in disconnection
Different Connectivity
Technologies
Different physical geography
conditions
Built in latency Operator/Provider
In the mobile age the connection is dynamic.
Wi Fi
GSMLTE

10. Lab Network Conditions
Latency= <2 ms
Throughput=~30,000 Kb/sec
• Your lab device is connected to a high speed wi-fi connection
• The device connects through the Wi fi to the application server
Wi fi

11. Real Life Network Conditions
So the difference is huge!
• Your customers are using a slow dynamic GSM network.
• The client device is on a different network in India
• The servers are in the US.
Throughput=~50 Kb
Latency= ~250 ms
GSM

12. Why does my application
care about the network
condition?
 Your application assumes the data requested from the server
will be delivered in a reasonable amount of time.
 A part of the quality of your application is related to tuning the
timeouts and verifying the error message are appropriate.
 Other aspects of your application quality related
to the user experience.
 If your application is not responding after a few
seconds your user is likely to delete it.

13. SeeTest Network Virtualization Architecture
N
a b c e
d

14. End-to-End Product Suite - Manual, Automation, Monitoring, Load, Network Virtualization, Manager
• Manual testing tool
• Mini-recorder for automating short,
repetitive, manual scenarios
• Report with screenshots of each step
• Manual to Automation Script Convertor
• Device Lab located onsite (in the customer VPN)
• Devices can be reserved and accessed by any remote user
• Used in conjunction with SeeTestAutomation/Manual for testing of remote devices
SeeTestManual
SeeTestCloud
• Automation tool
• Same test runs on different devices
• Plugs into all testing & CI environments:
SeeTestAutomation
SeeTestNetworkVirtualization
• Device performance under varying network conditions
• Simulating mobile network bandwidth latency and reception while on the move
SeeTestLoad
• Real device generated mobile load
• Full integration to LoadRunner, Jmeter, etc.
Mobile Add-On for UFT
• Full & complete integration into HP UFT:
• Record a test, edit it, run it, and view results
• Integration of object repository and more…
SeeTestManager
Free
Tool!
N
L
G
U

15. Demo

16. Call for action
Providing this information to the developers will enable them to use few technics to solve the issues:
Reduce the data size
Use CDM storage where part of the data (images) will be served dependent on your user device location.
Cache the data
Reduce the number of round-trips

17. Analyzing the Completeness of a Mobile
Testing Strategy
0%
10%
20%
30%
40%
50%
Iteration
1
Iteration
2
Iteration
3
Iteration
4
Functionality
Compatibility
Usability
Compliance
Performance-Server Level
Performance-Device Level
Security
Change in Distribution of Test Types across Sprints
 Performance – Server Level
 Compliance
 Security
> Functionality
> Compatibility
> Usability
> Performance – Device Level
Percentageoftotal
Testing

18. Challenges – Including Network Virtualization in
your Testing Strategy
Test Strategy Components Impacted 
Test
Infrastructure
Test Planning &
Implementation
Efficiency in Test
Execution
Test Coverage
1 Distribution of Testing teams X X
2
Support for multiple Mobile Platforms –
versions, devices, configurations X X X X
3 Organizational Info-security constraints X
4 Need for network coverage across regions X X
5
Application availability restricted to client
devices X
6
Cost of infrastructure setup and
sustenance; Tools & Licenses X X X
7
Lack of adequate support from Mobile
Test tools X X

19. Case Study

20. Case Study:
End User Performance Testing of a Mobile Financial App
CUSTOMER
Leading global financial institution (140+ countries)
 End to End Testing of Banker
Application on Mobile Platform
(aimed at providing the end
user with information to
conduct banker business
activities)
• Business need: Improve Time to
Market & Exceptional User
Experience
Speedy Delivery of the
Application to the
business Users in limited
time frame
User Experience
surpassing the business
needs for users in
different countries
 SDLC – Custom Agile SCRUM
BACKGROUND TESTING IMPERATIVES
iPad Banker
ApplicationUsability
Testing
Compatibility
Testing
Sync Testing
User Security
Testing
Platform
Testing
Functional
Testing
Performance
Testing
Device Build
Management
Test Data
Management
Adherence to human
interface principles
for iPad platform
Supported iPad
& network
configurations
Web <_>Mobile cliet
synchronization
Validation of
user level access
controls
Support of iPad
features in conjunction
with the App
Server & Device level
performance testing
Built distribution
strategy on mobile
devices
Setup & creation of business
specific test data for different
levels of testing

21. Case Study:
End User Performance Testing of a Mobile Financial App…
SAMPLE
SCENARIOS
 App performance in different network configurations – WiFi, 3G, 4G
 Mobile workloads during various business hours
 Performance characteristics based on geographical domain specific latencies
CHALLENGES Validating app user experience in varying network bandwidth & conditions
Performance benchmarking for real life usage scenarios (driving, low signal, etc.)
WORKAROUNDS /
TACTICAL SOLUTION
Usage of Performance Testing – NV integrated solution for for Server Performance testing
Distribution of testing between Nearshore, Onsite & Offshore at different locations
Manually vary parameters to enable repetitive testing in varying network conditions – support
from test automation

22. Thank you 