The exploding apps economy has increased the businesses’ need to have a strong mobile app presence. This has spurred a dramatic upward shift in mobile app development. Traditionally, testing has been done in the lab, replicating user environments and usage scenarios. However, that approach can be insufficient. Complementing in-the-lab manual and automated testing with testing in real user environments is a critical new component of mobile app development. Today, leveraging the right holistic mix of software testing tools, in-house testing and automation, mobile SDKs, analytics tied to ongoing user sentiment, and outside-the-lab testers can be the difference between mobile app success and failure. Roy Solomon provides information, analysis, and real world examples showing the benefits of a 360 degree, user-focused approach to app quality. Learn the science behind this approach, and what you can do to overcome the ever-present challenges of speed, cost, and risk to launch apps that users love.

1. |November 2014 | Roy Solomon | VP & CO-FOUNDER |
rsolomon@applause.com
INCORPORATING
360 °APP QUALITY IN
MOBILE DEVELOPMENT

2. | 2
APPS ARE GATEWAY TO YOUR BRAND & BUSINESS

3. | 3
THEY MUST DO MORE THAN MERELY WORK…
3

4. | 4
THE NEW NECESSITY IS TO DELIGHT USERS

5. | 5
U S E R S A R E M O R E V O C A L T H A N E V E R

6. | 6
U S E R P E R C E P T I O N I S R E A L I T Y
Source: Compuware Mobile App Usage survey

7. | 7
5 REASONS THIS MATTERS MORE THAN EVER

8. | 8
1. THE APPS ECONOMY IS ULTRA -COMPETITIVE

9. | 9
2. SWITCHING COSTS ARE AT OR NEAR ZERO

10. | 10
W H A T I S T H I S N U M B E R ?
16%
% O F U S E R S T H A T W O U L D R E T R Y A N A P P A F T E R 2 F A I L U R E S

11. | 11OpenSignal Android Fragmentation Visualized (July 2013)
3. FRAGMENTATION (OF ALL KINDS) SPIKING

12. |
Personal Computers
(Desktop And Notebook)
Smartphones
Tablets
Smart TVs
Wearables
0
1,000
2,000
3,000
4,000
5,000
6,000
7,000
8,000
9,000
10,000
#OfDevices(InMillions)
Global Internet Device Installed Base Forecast
12Business Insider (Business Intelligence Estimates) 2013
4. IT’S NOT JUST MOBILE

13. | 13
5. THE SPEED OF DEVELOPMENT & TESTING

14. | 14
ARE YOU PREPARED TO SURVIVE – AND THRIVE?

15. |
USERS DON’T COMPARTMENTALIZE QUALITY
15
A H A ! M O M E N T

16. | 16
WINNING APPS & COMPANIES DON’T EITHER

17. |
360°APP QUALITY
QUALITY, DEFINED BY YOUR USERS
17
T H E N E W M E A S U R I N G S T I C K

18. |
The Challenge
•••
WHAT THIS MEANS
FOR YOU

19. |
Your Test Labs Are Here Your Users Are Here
Modern apps need real-world testing, where users live, work & play
19
T H E P R O B L E M W I T H T H E S T A T U S Q U O

20. |
C L O S E D L O O P O F T E S T & F E E D B A C K
20
Tester End UsersDeveloper
Build distribution1 In-app bug reporting2
Crash log reporting
4
In-app user feedback3
Product/Marketer
Sentiment Analysis5
Builds
User Feedback
Crash logs
BugsBugs & Crashes
Quality Dashboard
with Sentiment Analysis

21. |
O U T S I D E - I N A P P Q U A L I T Y S I G N A L S
• Inside-out POV valuable, but incomplete
– Regardless of dev & test methodologies
• 360
ₒ
app quality requires an outside-in worldview
– Comes in many forms
- Usability feedback
- User analytics
- L10N issues
- Bug reports
- Crash reports
- Security vulnerabilities
– And at many times
- Research
- Pre-launch
- Post-launch
User Perception Is Reality 21

22. |
WATERFALL
AGILE FORRESTER
LEAN
22

23. |
PRE-LAUNCH POST-LAUNCH
MEASURE USER
APPLAUSE
EARN USER
APPLAUSE
23

24. |
PRE-LAUNCH POST-LAUNCH
DESIGN &
DEVELOP
1
TEST &
DEPLOY
2
LISTEN &
LEARN
3
ANALYZE
& PLAN
4
360º
APPQUALITY
5
EG: IBM
EG: JIRA EG: FLURRY
EG: BAZAARVOICE
24

25. |
The Challenge
•••
DIVING DEEPER

26. | 26
3 6 0 ° A P P Q U A L I T Y D A S H B O A R D

27. | 27
I N - T H E - W I L D T E S T I N G S E R V I C E S
• Testing beyond the QA lab (in-
house or offshore)
• Test coverage where your
users work, live & play
• Across devices, OSes &
carriers
• Real testers, real devices, real-
world conditions
• Across the SDLC

28. |
A C T I O N A B L E D E F E C T R E P O R T S
FYI, THIS IS WHAT ACTIONABLE LOOKS LIKE

29. |
S D K - E N A B L E D C R A S H R E P O R T S

30. | 30
S D K - E N A B L E D S E S S I O N R E P O R T S

31. | 31
V A L I D A T E D L O C A L I Z A T I O N I S S U E S

32. | 32
M O B I L E B E T A P R O G R A M S

33. | 33
U S E R S E N T I M E N T A N A L Y T I C S

34. |
H E A R I N G W H A T U S E R S A R E S A Y I N G
ENORMOUS BLIND SPOT FOR COMPANIES

35. |
I F Y O U C A N C U T T H R O U G H T H E N O I S E
AND FIND THE SIGNALS 35

36. | 36
U S E R R E V I E W S E N T I M E N T
TURN DATA INTO ACTIONABLE INSIGHTS

37. | 37
C O M P E T I T I V E A N A L Y S I S
DETERMINE WHERE TO INVEST IN QUALITY

38. |
L I S T E N I N G T O U S E R S D R I V E S Q U A L I T Y
• Analytics = listen to your users
• End opinion-based debates
• Continual improvement
• Measure what matters
• Compare to competitors
• Measure before & after launch
IT’S POWERFUL IN CREATING WINNING APPS 38

39. |
• Modern apps have different recipe for success
– Infinitely more variables (devices, OSes, carriers, locations, use cases)
– Apps must go beyond merely “good enough” to delight users
– Traditional methods & playbooks won’t suffice
• Companies must play catch-up
– App quality no longer just a CIO & CTO problem; matters to brand & biz
– Move beyond lab to where users work, live & play
– Closed loop between user POV and your definition of quality
• 360°app quality isn’t just about dev ops or testing
– Requires modern tools and analytics that focus on users-centric quality
– Necessity due to outsized user expectations + voice
• A few mobile app war stories & examples (time permitting)
K E Y T A K E A W A Y S
39

40. |
The Challenge
•••
QUESTIONS?
ANSWERS.
Matt Johnston | CMO & CSO | mjohnston@applause.com