Mobile testing is a complex task. In this talk I shared insights on how to gather more data about your customers and how you can make use of this data for your testing activities. Furthermore, I share insights on how to scale mobile releases across multiple development teams.

1. Mobile Test Management
Done Right
By Daniel Knott
@dnlkntt
📺 youtube.com/c/DanielKnott
www.adventuresinqa.com

2. Hi, I’m
Daniel Knott
Head of Software Testing @MaibornWolff (Germany)
Mobile Tester by ❤
Author of the books:
• Hands-On Mobile App Testing (EN)
• Mobile App Testing (DE)
• Smarwatch App Testing (EN)
📺 youtube.com/c/DanielKnott
@dnlkntt
www.adventuresinqa.com

3. Key
takeaways
✅
Release mobile
Products frequently
🚢
Mobile testing
challenges
🤳
Mobile testing
scenarios
📲
Involve customers
in the development
👩💻
Getting to know
your customers
👪
High user
expectations
⭐
Agenda
Current state of
the mobile market
🚦
@dnlkntt

4. Current State of the Mobile Market
By end of 2023
4.3 billion will use a smartphone
By end of 2023
Mobile app revenue +$935 billion
Apple and Google
+4 million apps
21% of Millennials
Open apps +50 times/ day
69% of all US digital media time
Comes from mobile apps
Avg. smartphone user
Uses 10 apps per day and
30 each month
In 2020
+200 million app downloads
In 2020
+1 billion devices sold
@dnlkntt

5. Mobile Technology
Changed Our Life
@dnlkntt

6. https://twitter.com/_adrianhinojosa/status/757015786057641985
@dnlkntt

7. Test Management Process
Test Plan
Test
Design
Test
Execution
Exit
Criteria
Test
Reporting
User
ExpectaZons
Mobile
Devices
Mobile Test
Scenarios
Release
Cycles
…
@dnlkntt

8. High User
Expecta=ons
for Mobile
Products
@dnlkntt

9. High User Expecta?ons for Mobile Products
Avg. adult checks every 12 minutes
their smartphone
Top 4 reasons for deleting an app
1. Bad Design
2. Bad Usability
3. Loading time
4. App Crashes
88% of overall smartphone time
spent on apps
Nearly 60% of users will delete an
app that requires a registration
80% of users delete an app after the
first usage
Users expect that an app loads
under 2 seconds
@dnlkntt

10. Ge@ng to
Know your
Customers
@dnlkntt

11. Getting to Know your Customers
Get information about
user needs and
preferences
Collect data from
market research,
studies, tracking or
user interviews
Outline most important
scenarios and make
them transparent to
company and team
Focus on user
values
Mobile Testers can
plan their testing
activities around this
data
@dnlkntt

12. Getting to Know your Customers
Get information about
user needs and
preferences
Collect data from
market research,
studies, tracking or
user interviews
Outline most important
scenarios and make
them transparent to
company and team
Focus on user
values
Mobile Testers can
plan their tes?ng
ac?vi?es around this
data
@dnlkntt

13. GeDng to Know your Customers
Get information about
user needs and
preferences
Collect data from
market research,
studies, tracking or
user interviews
Outline most important
scenarios and make
them transparent to
company and team
Focus on user
values
Mobile Testers can
plan their testing
activities around this
data
@dnlkntt

14. Getting to Know your Customers
Get informa?on about
user needs and
preferences
Collect data from
market research,
studies, tracking or
user interviews
Outline most important
scenarios and make
them transparent to
company and team
Focus on user
values
Mobile Testers can
plan their testing
activities around this
data
@dnlkntt

15. GeDng to Know your Customers
Get information about
user needs and
preferences
Collect data from
market research,
studies, tracking or
user interviews
Outline most important
scenarios and make
them transparent to
company and team
Focus on user
values
Mobile Testers can
plan their testing
activities around this
data
@dnlkntt

16. Involve
Customers in
Development
@dnlkntt

17. Involve Customers in the Development
Conduct user
interviews in
the own
office
Establish a
beta testing
community
Sync on a
regular basis
with PM, UI &
UX
Send regular
pre-builds to
colleagues
and
customers
Integrate
feedback
surveys to
your apps
Check app
stores for
reviews &
comments
Check error
reports and
log files from
users
@dnlkntt

18. Mobile Testing
Scenarios
@dnlkntt

19. • Depending on app use cases this differs from app to app
• The app tech stack also influences the test scenarios for mobile testers and developers
Mobile Testing Scenarios
@dnlkntt

20. Mobile Testing Scenarios
• Interrupt Testing
• Permissions
• Standby testing
• Sensors
• Accessibility Testing
• Security
• 3rd Party Apps
• Gestures
• Multitasking
• App Data
• In App Purchase
• Performance
• Usability
• Mobile Networks
• Battery
• Local Storage
@dnlkntt

21. Mobile Testing Scenarios
https://adventuresinqa.com/wp-content/uploads/2022/01/Mobile-Testing-Cheat-Sheet.pdf @dnlkntt

22. MOBILE APP TESTING
https://www.ministryoftesting.com/dojo/lessons/mobile-app-testing-mnemonic
Mnemonic might help you as well
@dnlkntt

23. Mobile Testing
Challenges
@dnlkntt

24. Mobile Testing Challenges
@dnlkntt

25. Mobile Testing Challenges - Fragmentation
Group 1, Prio A
• High End Devices
• Quad Core CPU
• RAM >4048MB
• Retina, Full HD
Display >= 6“
Group 2, Prio B
• Mid-range devices
• Dual Core CPU
• RAM <=2048MB
• Screen size < 6“
• Software not older
1 year
Group 3, Prio C
• Small devices
• Single Core
• RAM < 1024MB
• Low screen size
and resolution
• Older OS, older
browser
Based on your target group
@dnlkntt

26. Mobile Testing Challenges - Automation
There is no silver bullet
Invest time to find the right test
automation tool that fits your needs
and requirements
Start simple with automation
Invest time in your CI pipeline and
testing infrastructure
Automation should support your
testing activities. It will NEVER
replace testing
Get support from developers
@dnlkntt

27. Release Mobile
Products
Frequently
@dnlkntt

28. Release Mobile Products Frequently
Users expect a bug “free” app
When working with multiple teams,
a release train can help
Planned code freezes and app
releases help to organize & to give
guidance for teams
Every team need to catch the train
in case they want to release
It’s helpful to have a regular release
cycle e.g. every 2 weeks
Freeze the code automatically to
avoid late passengers
@dnlkntt

29. Release Mobile Products Frequently
Code Freeze/
Upload to Google/ Apple
100%
Users
Coding
10 days
Beta
Testing
7 days
20% Users
2 days
50% Users
5 days
Code Freeze/
Upload to Google/ Apple
100%
Users
Coding
10 days
Beta
Testing
7 days
20% Users
2 days
50% Users
5 days
@dnlkntt

30. Release Mobile Products Frequently
Monitor the app release
React fast in case of a critical bug
Check crash logs
Check social media and feedback in
app stores
@dnlkntt

31. Mobile Test
Management
Takeaways
@dnlkntt

32. Mobile Test Management Takeaways
Ø Mobile technology changed our life’s
Ø Mobile market is still growing
Ø Mobile users have a high expectation to your product and services
Ø Get to know your customers and interact with them
Ø Define test scenarios based on your app and tech stack
ØIdentify your challenges and start to tackle them
Ø Start simple and reconsider your approach from time to time
@dnlkntt

33. Thank You!
@dnlkntt – www.adventuresinqa.com
📺 youtube.com/c/DanielKnott

34. Resources
• https://www.statista.com/statistics/263437/global-smartphone-sales-to-end-users-since-2007/
• https://www.statista.com/statistics/330695/number-of-smartphone-users-worldwide/
• https://www.statista.com/statistics/269025/worldwide-mobile-app-revenue-forecast/
• https://www.statista.com/statistics/276623/number-of-apps-available-in-leading-app-stores/
• https://mindsea.com/app-stats/
• https://www.marketingcharts.com/digital/mobile-phone-111093
• https://www.statista.com/statistics/271644/worldwide-free-and-paid-mobile-app-store-downloads/
• https://adventuresinqa.com/wp-content/uploads/2015/12/Mobile-Testing-Cheat-Sheet-Adventures-in-QA.pdf
• https://www.ministryoftesting.com/dojo/lessons/mobile-app-testing-mnemonic
• https://www.ben-evans.com/benedictevans/2015/8/19/how-many-pictures
• https://twitter.com/_adrianhinojosa/status/757015786057641985
• https://adventuresinqa.com/2014/08/11/kifsu-and-the-mobile-user-expectations/
• https://buildfire.com/app-statistics/
• https://nypost.com/2017/11/08/americans-check-their-phones-80-times-a-day-study/
• https://www.emarketer.com/content/the-majority-of-americans-mobile-time-spent-takes-place-in-apps