The document presents insights on mobile app testing, led by Jason Arbon, outlining challenges and best practices in achieving app quality within a fragmented ecosystem. Key points include user expectations, correlation of app ratings with release timeline, and the importance of agile development. The document also highlights Arbon's background and contributions to mobile app testing methodologies.

1. T4
Mobile Testing
5/8/2014 9:45:00 AM
Mobile App Testing Secrets
Presented by:
Jason Arbon
Applause
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Jason Arbon
Applause
Jason Arbon currently leads the labs engineering team at uTest.com and applause.com. In the
past, Jason worked on Google Chrome, ChromeOS, Google+, several Microsoft teams
including Bing and most importantly WinFS. He coauthored the book How Google Tests
Software. Jason has started several open source projects, more recently the opentipmap.com
project to codify mobile best practices. Jason is the cofounder and mobile app developer for a
personalized discovery, search, and sharing app at herecandy.com. In his free time he’s
working on a mobile app quality book, OnAppQuality.com. Free time? Wait, Jason doesn’t
actually have free time.

3. 4/26/2014
1
On App Quality
Jason Arbon
Dir. Product, applause.com/utest.com
window on humanity

4. 4/26/2014
2
shaped my thinking
coding eng director
Analytics
1,000,000+ apps
100,000,000+ reviews
Crowd Testing
1,000+ apps
100,000+ Testers

5. 4/26/2014
3
author
most aim for 4 stars
...and then stop shipping.

6. 4/26/2014
4
monsters
“agile”
“continuous
integration”
app store monster

7. 4/26/2014
5
app quality: difficult
user expectations
reviews/stars
location/network
few best practices
fragmented tools, platforms, devices
awesome future
rich appstore data
cloud devices
cloud humans
best practices
moving to agile and CI (finally!)
finally quantify quality

8. 4/26/2014
6
quantify/compare

9. 4/26/2014
7

10. 4/26/2014
8

11. 4/26/2014
9
window on humanity
competitive info
where winning
what to focus on

12. 4/26/2014
10
devices in the cloud
humans in the cloud

13. 4/26/2014
11
agile doesn’t mean stars
Correlation between number
of days since release and star
rating
x-axis: star rating
y-axis: number of days since
release
Pearson correlation: -0.31
(threshold for weak/medium
correlation is 0.3)
top 1k apps (>10k reviews)
agile doesn’t mean stars
Correlation between number
of versions and star rating
x-axis: star rating
y-axis: number of versions
Pearson correlation: -0.28
(threshold for weak/medium
correlation is 0.3)
top 1k apps (>10k reviews)

14. 4/26/2014
12
open tip map
encoding best practices
programmatic access
data triggered tips
open

15. 4/26/2014
13
window on humanity
contact
jasona@utest.com
@jarbon
jarbon@gmail.com
best practices for book
tips

16. 4/26/2014
14
far future: no backlog

17. 4/26/2014
15
app-differencing
monkey-bots compare every build with the last
screenshots of differences sent to developers
and testers in real time
Analytics brought to you by...
Matthew, Heidi and Jason Stredwick and Joe