The document discusses how a mobile test strategy should be part of an organization's overall test strategy, not a separate strategy. It notes the challenges of testing across different devices, operating systems, and platforms. However, the key mission of testing is still to validate products and report risk. A tester's role is to provide input into an organization's mobile strategy and understand where new technologies might be accessed. A mobile test strategy should consider requirements, environments, test phases, and risk analysis for native apps, hybrid apps, responsive web designs, and other platforms like TVs and cars. The strategy must evolve to holistically address usability, performance, security and more across all potential user experiences.

1. Why a
Mobile Test Strategy
is just Test Strategy
27 | 06 | 2015
HCMC Software Testing Club 2015

2. 2

3. 3

4. 4

5. Are we scared of Mobile?
Challenges:
• Creating separate teams
• Managing multiple devices
• Webservers don’t run low on battery get taken
home
• Browser, OS, Device, Platform combinations
are more than we have ever had to manage
But it still boils down our mission – Validate
products, report risk
5

6. Our role is greater than a mobile test strategy
Building a mobile approach is a key point in your overall
strategy….
…But we must do more!
Our mission, to report risk, must include a focus on where
risk is introduced
We must have a seat at the table:
- What is the mobile strategy of the organization?
- What will the next technology be?
- Where will our content and products be accessed?
6

7. What must we understand
7

8. What must we understand
8

9. What must we understand
• For public and consumer applications,
mobile does not just mean an “app”
• For internal, private applications, our
users or still customers and usability
can’t be taken for granted
9

10. There are two primary drivers to Mobile Quality
10

11. There are two primary drivers to Mobile Quality
11

12. But our apps aren’t really meant for mobile
12

13. What are we really testing?
• Native apps – Yes
• Hybrid apps – Yes
• Mobile web – Yes
• Responsive design – Yes
• What about:
- TV’s?
- Cars?
- Refrigerators!!!
13

14. What are we really testing?
14

15. What are we really testing?
15

16. What are we really testing?
16

17. What are we really testing?
17

18. But, I just want to make sure it renders
We have a lot more to test than functionality and
usability
18

19. But, we don’t have the devices
19

20. What does this mean for the tester?
• Defect escape risk is exponentially higher with
changes in platform and technology
• Our challenges are increasing and so are our
options
• Working with product teams on strategy can manage the
risk
• Knowing that you’re mobile even when you’re not
• Communicate cost of defects and customer loss
• Building for future needs will strengthen non-mobile
aspects of your strategy
20

21. Mobile is not only about mobile
• Building for future mobile needs will strengthen
non-mobile aspects of your strategy:
- Usability is not a uniquely mobile trait
- Customer adoption and community acceptance
isn’t either
- Size, performance, battery consumption, and
connectivity options all affect functionality
- Biometrics
- Augmented reality
21

22. How my strategy should change for mobile
• Clearly, new techniques are needed
• The functional tester is no longer only able to
focus on functionality!
• But, we can still break this down into similar
buckets:
1. Estimation
2. Requirements
3. Environment
4. Test Phases
5. Risk Analysis
22

23. Strategy – Estimating for Mobile
The complexity of combinations is a significant drain on
test estimation
We must focus on production statistics (if Available) or
expected usage to drive initial, risk based estimations
Permutations of device, OS, Browser will create
distraction – These are important, but must be based on
target usage
AUT loading, charging, OS selection, and setup are also
factors
23

24. Strategy – Managing Mobile Requirements
This is a change in the role of the tester:
• Functional requirements will not address the
infrastructure and technical needs of a mobile
application or web experience
• We must create best practices for product owners
and architects to follow that drive quality solutions
• Testing must become the voice of the mobile strategy
24

25. Strategy – Managing Mobile Requirements
Functional Requirements:
• Usability, accessibility, notifications and interactions with OS
Infrastructure Requirements:
• Wi-Fi vs. Network, lost transmission, security, and on device
storage
Consumption requirements:
• Battery, data, bandwidth
Input requirements:
• Camera, biometrics, keyboard types
Error Handling:
• Call interruption, multi-tasking, background usage
25

26. Strategy – Managing Mobile Environments
Traditionally, the test environment is managed by the
infrastructure or technical team
But with mobile, who do we “request” an environment
from?
And what constitutes a mobile environment?
It is now the tester’s job to load OS, AUT, prepare the
environment, and ready for testing
26

27. Strategy – Managing Mobile Test Phases
Traditional webdesktop testing has included a standard
approach that usually consists of: Functional, System,
Integration, NFR, Regression and UAT (along with many
other phases)
Often, with mobile, we are forced to only consider
functional (does it render) when we must also include:
• Usability
• Performance
• Security
27

28. Strategy – Managing Mobile QualityRisk Analysis
Between nativehybrid apps, “m.”, responsive design,
and mobile web, our risk reporting must be focused on
financial impact, customer adoption, and usability
Risk reporting now includes intended
platformOSDevicebrowser combinations
Our roles in this process will be continuing to change
but we must define risk criteria as part of our strategy
from the onset
28

29. Summary - Quick Hits
• Mobile Strategy is as much a part of test strategy
as regression, especially for customerpublic
portals
• Responsive design must be implemented in any
external website and can be tested outside of the
device
• Test strategy must evolve to include influence to
design and development based on intended use
• Risk mitigation includes the identification of core
aspects of product usage
• Mobile is not just for phones and tablets!
29

30. Brian Bernknopf
bbernknopf@kms-technology.com
30