The Mobile Testing Checklist2. 1© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Lee Barnes
Founder and CTO
Utopia Solutions
If you have questions during the webinar, submit in chat box during event
Mike Ostenberg
Director Solutions Engineering
SOASTA
Moderator: Brad Johnson
VP of Product Marketing
SOASTA
3. 2© 2014 SOASTA CONFIDENTIAL - All rights reserved.
1. Introduction
2. The Landscape of Mobile Testing
1. Why is Mobile Testing Different
2. Checklist for Mobile Testing
3. Mobile Testing Framework
3. SOASTA Solutions
1. TouchTest Key Capabilities
2. CloudTest for Mobile Load Testing
3. mPulse RUM for Mobile Monitoring
4. Resources and Conclusion
4. 3© 2014 SOASTA CONFIDENTIAL - All rights reserved.Source: “Mobile In the Enterprise”, Zendesk 2014
5. 4© 2014 SOASTA CONFIDENTIAL - All rights reserved.
“Do you currently have a mobile initiative”
Spring 2013 Fall 2013 Spring 2014
25 attendees 28 attendees 21 attendees
56%
44%
6. 5© 2014 SOASTA CONFIDENTIAL - All rights reserved.
CHALLENGES
o Most traditional dev practices don’t work with mobile
o Apps require constant refinement to tune the user experience
o Rapid change requires rapid development & deployment
Source: Gartner “Traditional Development Practices Will Fail for Mobile Apps” April 15, 2014
7. 6© 2014 SOASTA CONFIDENTIAL - All rights reserved.
CHALLENGES
o Most traditional dev practices don’t work with mobile
o Apps require constant refinement to tune the user experience
o Rapid change requires rapid development & deployment
RECOMMENDATIONS
o Collaborate with business, users, devs…and observe work practices
o Use all test types: functional, performance & load, and user experience
o Analyze how apps are used in production by real users
o Adopt agile development and deployment
Source: Gartner “Traditional Development Practices Will Fail for Mobile Apps” April 15, 2014
9. 8© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Lee Barnes
Founder and CTO
11. 10© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Source: OpenSignal Android Fragmentation Report August 2014
iOS 8
iOS 7
Earlier
4.4
4.3
4.2
4.1
4.03 - 4.04
2.3.3 - 2.3.7
2.2 & earlier
14. 13© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Mobile
Testing
Iceberg
App
Functionality
15. 14© 2014 SOASTA CONFIDENTIAL - All rights reserved.
App
Functionality
Mobile Specific
Test Conditions
16. 15© 2014 SOASTA CONFIDENTIAL - All rights reserved.
App
Functionality
Mobile Specific
Test Conditions
17. 16© 2014 SOASTA CONFIDENTIAL - All rights reserved.
App
Functionality
Mobile Specific
Test Conditions
18. 17© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Installation
Interrupt
Network
Device
Integration
Performance
19. 18© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Incoming / Outgoing Calls
Popup Alerts (Calendar, Alarm, SMS, etc.)
Power Cycle
Extended Background
20. 19© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Install
Uninstall
Reinstall
Update
21. 20© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Network Type (Mobile variations and
WiFi)
Network Quality
Network Loss
Network Transition
22. 21© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Speed / Responsiveness
CPU and Memory Usage
Battery Drain
On-Device / Memory Card Storage
23. 22© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Location / GPS
Video / Still Camera
Screen Size, Resolution, Orientation
Accelerometer
24. 23© 2014 SOASTA CONFIDENTIAL - All rights reserved.
App Type
App Usage
App/Device
Integration
25. 24© 2014 SOASTA CONFIDENTIAL - All rights reserved.
App
Functionality
& Technology
User Profile
Business
Objectives
• Deployment platforms &
versions
• App interaction with device
• Integrated systems
• Prevalent Devices
• Preferred Browser
• Concurrent usage /
load profile
• Application value to the
business
• Risk of application failure
29. 28© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Execute test
cases across a
set of diverse
devices
30. 29© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Mike Ostenberg
Director Solutions Engineering
32. 31© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Expertise
Run local and remote
private device clouds.
Compress testing from
weeks to overnight.
33. 32© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Expertise
Performance test the
mobile backend to any
scale or global
distribution.
Continuously.
34. 33© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Expertise
Measure real mobile
users in production to
determine scale and
usage patterns.
Test better.
35. 34© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Lee Barnes
Founder and CTO
Utopia Solutions
Email: lee.barnes@utopiasolutions.com
Twitter: twitter.com/USI_LeeBarnes
LinkedIn: linkedin.com/in/leebarnes
Blog: www.RaiseYourMQ.com
Mike Ostenberg
Director Solutions Engineering
SOASTA
Email: mostenberg@soasta.com