The document discusses the challenges of mobile performance testing. It notes that mobile introduces diverse channels and new risks compared to traditional websites. Performance impacts key metrics like conversion rates and can significantly affect business profits if not addressed. Testing mobile apps is more complex as performance can vary significantly based on network conditions, device type and location. Effective testing requires simulating real-world networks and remote users to identify performance issues before full deployment.

1. Why Makes Mobile
Different?
Session 2
Business Challenges
Technical Challenges
Performance Management Challenges
1

2. Mobile is a Very Diverse Channel
Automotive
Banking
Insurance
Professional Services
www.shunra.com
2
Healthcare
Engineering Services
Network Virtualization for Software Testing

3. Mobile Performance Introduces New
Risks
www.shunra.com
3
Network Virtualization for Software Testing

4. Mobile Performance Introduces New
Risks – Tablet Banking Example
www.shunra.com
4
Network Virtualization for Software Testing

5. Bounce
Rate
Conversion
Rate
Cart
Size
Page
Views
+0.2s
̶
̶
̶
̶
+0.5s
+4.7%
-1.9%
̶
-5.7%
+1.0s
Profit Impact
Delay
Mobile Performance Impact on the
Business
+8.3%
-3.5%
-2.1%
-9.4%
Brand Impact
www.shunra.com
5
Network Virtualization for Software Testing

6. Business Impact - Good performance vs. Bad
Poor performance impacts Customer adoption
vs
ING NL is a high performing mobile app that keeps increasing in
monthly installs, while BNP “Mes Comptes” which is a low
performing mobile app, is decreasing in popularity .
www.shunra.com
6
Network Virtualization for Software Testing

7. There isn’t a Single Performance Number
38.1
40
33.3
35
30
28.6
27.5
25
19.9
22.7
21.2
20
15
20.5
20.2
WiFi
17.5
17.2
3G
13.9
3G Busy
8.8
10
6.2
5.3
5.3
5
4.6
0
Launch App
Auto
Auto
Accident
Dental
Home
Get Life
Insurance
Quote
Transaction Response Times of an Insurance Mobile App [Sec]
www.shunra.com
7
Network Virtualization for Software Testing

8. What Happens under Mobile Load?
www.shunra.com
8
Network Virtualization for Software Testing

9. MyAT&T Mobile App – Prod Vs. Test
Mobile Customers
3G
LTE
Edge
WiFi
AT&T Cloud Partner Services
ATT Data Center
HP - Virtual Users
Hi Speed Lab Network
myAT&T – Dev/Test
www.shunra.com

10. Impact of Remote Users
Local Load Test Results
VUsers
Multi Site Load Test
Results
VUsers
Avg. TRT
Avg. TRT
Why?
www.shunra.com
Network Virtualization for Software Testing
10

11. What Happens under Mobile Load?
WiFi
3G
Each User Slows Down
Server Resources get Exhausted Quicker
www.shunra.com
11
Network Virtualization for Software Testing

12. The story of a mobile app
Testing - Singapore
Development - Pune
Deployment - Spain
www.shunra.com
Network Virtualization for Software Testing

13. Different Geographies have Different
Network Conditions
www.shunra.com
Network Virtualization for Software Testing

14. When it is time to Deploy
How will it Perform?
What will the App reviews
be?
We better run a field pilot for
a month!!!
www.shunra.com
Network Virtualization for Software Testing

15. When it is time to Deploy
What Do you Mean Pilot?
Sprint = 4 weeks
Week 1
Sprint
Planning
Meeting
Week 2
Week 3
Sprint n
Sprint
Review
Sprint n – Development / QA
Sprint n + 1 - Preparation
www.shunra.com
Week 4
Sprint n
Deployment
to Production
on following
Wednesday
Sprint n –
Integr. Test
Sprint
n+1
Network Virtualization for Software Testing