Build Your Mobile Testing Expertise1. MO
PM Tutorial
9/30/2013 1:00:00 PM
"Build Your Mobile Testing
Expertise"
Presented by:
Karen Johnson
Software Test Management, Inc.
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
2. Karen N. Johnson
Software Test Management, Inc.
Karen N. Johnson is a software test consultant, working on-site and remotely to help her clients
build testing teams and strengthen their testing practice. Karen is considered an expert in
software testing and has been part of the context-driven software testing community for a
number of years. She is a frequent speaker at testing conferences; a contributing author to the
book, Beautiful Testing; and co-founder of WREST, the Workshop on Regulated Software
Testing. Karen has published numerous articles and blogs about her experiences with software
testing.
3. 8/20/2013
Build Your Mobile Knowledge
section 1
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
1
agenda
•
•
•
•
device selection
UX
device settings
competitive analysis
Build Your Mobile Testing
Expertise v7
© Karen N. Johnson, 2013
2
1
4. 8/20/2013
One thumb design
Fat Thumb: A One-Handed
Alternative To Pinch-ToZoom
http://www.fastcodesign.c
om/1671741/fat-thumb-aone-handed-alternative-topinch-to-zoom#1
Testing One Thumb, One
Eyeball Mobile Use
http://www.lukew.com/ff/
entry.asp?1664
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
3
© Karen N. Johnson, 2013
4
Touch Gesture Icons
http://www.mobiletuxedo.com/t
ouch-gesture-icons/
Input Guidelines
http://uxdesign.smashingmagazin
e.com/2011/10/06/not-yourparents-mobile-phone-ux-designguidelines-smartphones/
Touch Gesture Reference Cards
http://www.lukew.com/ff/entry.a
sp?1370
Build Your Mobile Testing Expertise v7
2
5. 8/20/2013
Twitter
Content
Karen McGrane @karenmcgrane
Deane Barker gadgetopia
@
cleveg @cleveg
SaraWachterBoettcher @sara_ann_marie
UX: Theresa Neil @theresaneil (watch her website
and slideshare)
RWD
Ethan Marcotte @beep
Responsive Design @RWD
Jordan Moore @jordanmoore
Mobile
Brad Frost @brad_frost
Brian Fling @fling
Jason Grigsby, ☁4 @grigs
Jonathan Kohl @jonathan_kohl
steven hoober @shoobe01
Josh Clark @globalmoxie
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
5
Content Everywhere
by Sara Wachter-Boettcher
Books
Jump Start Responsive Web Design
by Craig Sharkie & Andrew Fisher
Mobile First
by Luke Wrobleski
Mobile Developer's Guide to the Galaxy
see the Enough Software website
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
6
3
6. 8/20/2013
device selection
part 1
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
1
Device Selection:
How will you decide which devices?
Possible Factors:
budget
procurement
time
usage statistics
manufacturers & models
operating systems
network carriers
m-commerce
mobile browsers
Build Your Mobile Testing
Expertise v7
cookies
installation
responsive web design
input methods
location awareness & map
providers
proprietary network
barcodes and scanners
flash
© Karen N. Johnson, 2013
2
1
7. 8/20/2013
Device Matrix
source: Mobile Design and Development, by Brian Fling
*** This is a model for making a decision. Does this model help you?
Class
Markup
CSS
Javascript
Class A
Class B
Class C
Class D
Class F
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
3
Budget, Procurement & Time:
Budget:
Can you afford the devices you want to test?
Procurement:
Can you find the devices?
Time:
If you resolve budget and purchase, would your team
have the time to test all of the devices?
•
•
Teams often solve the budget issue with using their own devices.
Teams often solve finding the devices through eBay or Craigslist.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
4
2
8. 8/20/2013
Mobile Statistics:
Do market statistics influence which devices to test?
•
•
•
Country: Usage statistics vary
greatly by country.
Source: Which source do you
trust?
Analytics: Are you capturing
device usage for your own
application?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
5
Mobile Manufacturers: & Models:
Does manufacturers or specific models influence which
devices to test?
•
•
•
Manufacturers vary by country.
Devices are updated frequently.
How often do you update your
device coverage?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
6
3
9. 8/20/2013
Operating Systems:
How do you decide which OS's to cover ?
•
How do you address forced OS
updates?
•
How do you maintain older versions?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
7
Network Carriers and Settings:
• How do you acquire
network simulation for
'other' countries?
• What about different
keyboards and languages?
• What about SMS text
testing?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
8
4
10. 8/20/2013
Does testing from the m-commerce perspective
impact device selection?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
9
Mobile Browsers:
If you're testing on mobile browsers, how does that
influence device selection?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
10
5
11. 8/20/2013
Cookies:
Do cookies influence device selection?
• Does your app or site offer
cookies?
What about:
• Auto-login
• Third party login access (i.e.
Use Facebook account to
login)
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
11
Installation
Application install & updates
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
12
6
12. 8/20/2013
Location Aware:
Does location aware influence device selection?
• How does map provider
selection influence device
selection?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
13
Input Selector & Navigation:
Does the input method impact device selection?
•
•
•
•
•
•
•
Rotate
Touch
Click wheel
Rollerball
Pinch & zoom
Typing/keyboard inputs
Gestures
Build Your Mobile Testing Knowledge v7
© Karen N. Johnson, 2013
14
7
13. 8/20/2013
Mobile Proprietary Network Access:
Are you required to test particular devices?
•
•
Does your mobile testing need to
pass an audit or regulations?
Are there VPN considerations?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
15
Responsive Web Design:
Do you need to check content and layout on different
devices?
•
People often use both at the same
time.
For more on RWD, see:
• http://blog.creativemagma.com/feat
ured/2209/
Build Your Mobile Testing Knowledge v7
© Karen N. Johnson, 2013
16
8
14. 8/20/2013
exercise: build a device matrix (or mind map)
•
•
•
With your team, determine what factors impact your decision.
Update the checklist of factors to reflect your decisions.
Build your own device matrix or mindmap to identify what devices to test.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
17
Device Selection:
How will you decide which devices?
Possible Factors:
budget
procurement
time
usage statistics
manufacturers & models
operating systems
network carriers
m-commerce
mobile browsers
Build Your Mobile Testing Expertise v7
cookies
installation
responsive web design
input methods
location awareness & map
providers
proprietary network
barcodes and scanners
flash
© Karen N. Johnson, 2013
18
9
15. 8/20/2013
Device Matrix
source: Mobile Design and Development, by Brian Fling
*** This is a model for making a decision. Does this model help you?
Class
Markup
CSS
Javascript
Class A
Class B
Class C
Class D
Class F
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
19
10
16. 8/20/2013
device selection
part 1
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
1
Device Selection:
How will you decide which devices?
Possible Factors:
budget
procurement
time
usage statistics
manufacturers & models
operating systems
network carriers
m-commerce
mobile browsers
Build Your Mobile Testing
Expertise v7
cookies
installation
responsive web design
input methods
location awareness & map
providers
proprietary network
barcodes and scanners
flash
© Karen N. Johnson, 2013
2
1
17. 8/20/2013
Device Matrix
source: Mobile Design and Development, by Brian Fling
*** This is a model for making a decision. Does this model help you?
Class
Markup
CSS
Javascript
Class A
Class B
Class C
Class D
Class F
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
3
Budget, Procurement & Time:
Budget:
Can you afford the devices you want to test?
Procurement:
Can you find the devices?
Time:
If you resolve budget and purchase, would your team
have the time to test all of the devices?
•
•
Teams often solve the budget issue with using their own devices.
Teams often solve finding the devices through eBay or Craigslist.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
4
2
18. 8/20/2013
Mobile Statistics:
Do market statistics influence which devices to test?
•
•
•
Country: Usage statistics vary
greatly by country.
Source: Which source do you
trust?
Analytics: Are you capturing
device usage for your own
application?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
5
Mobile Manufacturers: & Models:
Does manufacturers or specific models influence which
devices to test?
•
•
•
Manufacturers vary by country.
Devices are updated frequently.
How often do you update your
device coverage?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
6
3
19. 8/20/2013
Operating Systems:
How do you decide which OS's to cover ?
•
How do you address forced OS
updates?
•
How do you maintain older versions?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
7
Network Carriers and Settings:
• How do you acquire
network simulation for
'other' countries?
• What about different
keyboards and languages?
• What about SMS text
testing?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
8
4
20. 8/20/2013
Does testing from the m-commerce perspective
impact device selection?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
9
Mobile Browsers:
If you're testing on mobile browsers, how does that
influence device selection?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
10
5
21. 8/20/2013
Cookies:
Do cookies influence device selection?
• Does your app or site offer
cookies?
What about:
• Auto-login
• Third party login access (i.e.
Use Facebook account to
login)
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
11
Installation
Application install & updates
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
12
6
22. 8/20/2013
Location Aware:
Does location aware influence device selection?
• How does map provider
selection influence device
selection?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
13
Input Selector & Navigation:
Does the input method impact device selection?
•
•
•
•
•
•
•
Rotate
Touch
Click wheel
Rollerball
Pinch & zoom
Typing/keyboard inputs
Gestures
Build Your Mobile Testing Knowledge v7
© Karen N. Johnson, 2013
14
7
23. 8/20/2013
Mobile Proprietary Network Access:
Are you required to test particular devices?
•
•
Does your mobile testing need to
pass an audit or regulations?
Are there VPN considerations?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
15
Responsive Web Design:
Do you need to check content and layout on different
devices?
•
People often use both at the same
time.
For more on RWD, see:
• http://blog.creativemagma.com/feat
ured/2209/
Build Your Mobile Testing Knowledge v7
© Karen N. Johnson, 2013
16
8
24. 8/20/2013
exercise: build a device matrix (or mind map)
•
•
•
With your team, determine what factors impact your decision.
Update the checklist of factors to reflect your decisions.
Build your own device matrix or mindmap to identify what devices to test.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
17
Device Selection:
How will you decide which devices?
Possible Factors:
budget
procurement
time
usage statistics
manufacturers & models
operating systems
network carriers
m-commerce
mobile browsers
Build Your Mobile Testing Expertise v7
cookies
installation
responsive web design
input methods
location awareness & map
providers
proprietary network
barcodes and scanners
flash
© Karen N. Johnson, 2013
18
9
25. 8/20/2013
Device Matrix
source: Mobile Design and Development, by Brian Fling
*** This is a model for making a decision. Does this model help you?
Class
Markup
CSS
Javascript
Class A
Class B
Class C
Class D
Class F
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
19
10
26. 8/20/2013
Mobile User Interface, Design &
User Experience
section 3
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
1
UX Testing:
How will you decide what to test?
Possible Factors:
menu styles
navigation
input selection
UI controls
custom controls
links
lists
colors & symbols
currency
Build Your Mobile Testing
Expertise v7
images
video
completing forms
login
search, filter & sort
app version
sitemap
RWD
accessibility
© Karen N. Johnson, 2013
2
1
28. 8/20/2013
Menu Styles
Carousel
Build Your Mobile Testing Expertise v7
Metaphor
© Karen N. Johnson, 2013
5
Navigating
Giving the user a unique way to
move forward & back.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
6
3
29. 8/20/2013
Navigating
• Paging
• Forward & backward
• Cookie crumb trail
(websites)
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
7
Navigating
• Rotate
• Pinch/zoom
• Accelerometer
• Vertical/horizontal
• Paging forward & backward
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
8
4
31. 8/20/2013
Custom Controls
• The slider bar on AA’s
mobile app is an unusual
custom UI control.
• Does your app have any
custom controls?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
11
Links
• Do links work?
• Does your site have links to
other sites?
• What about the sitemap
links?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
12
6
32. 8/20/2013
Lists
• Alignment
• Consistency
• Disruptions from ads or 3rd
party feeds
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
13
Color-alternating lists
•
Grouped
rows
•
Zebra
stripping lists
•
Text
Alignment
•
User
Accessibility
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
14
7
34. 8/20/2013
Images
• Where’s the image?
• You might use your desktop
to see if the issue occurs
outside of the mobile
environment.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
17
Images
•
•
•
•
•
•
•
•
•
Size of files
Cache
Refresh rate
Full screen
Wi-Fi drops connection
Rotate
Pinch/zoom
Accelerometer
Vertical or horizontal
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
18
9
35. 8/20/2013
Video & Streaming
• Flash?
• Videos
• Connection speed
• Availability
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
19
© Karen N. Johnson, 2013
20
Completing Forms
At some point, a user
needs to enter
information. Minimizing
input – such as offering
smart fill fields on location
awareness helps to cut
down on the amount of
information a user has to
enter.
Build Your Mobile Testing Expertise v7
10
36. 8/20/2013
Completing Forms
• Sign-in and search are
two of the most
common forms on
mobile devices.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
21
Login
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
22
11
37. 8/20/2013
Passwords
• Case sensitive
• Changing the case
• Seeing the text entry
• Staying logged in (cookies)
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
23
Search
• Wildcards
• Text entry
• Case sensitivity
• Alpha and numeric entry
• Dropdowns
• Location based searching
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
24
12
38. 8/20/2013
Search & Filter
• Location-based
filtering
• UX considerations
on filters, applying
& clearing
• UX considerations
– scrolling through
many search
results
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
25
Search & Sort
When results are pages
long on a mobile device,
offer the user a means to
sort or refine the result
set.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
26
13
39. 8/20/2013
Search
• Are there fields that are prefilled based on date, location or
time?
• OpenTable is an example of time and location.
• Are there fields that are not offered on mobile but are offered
on the desktop version?
• Is there a search button? Is there another method to start or
run the search?
• Is there a way to refine the results?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
27
Search: filtering results
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
28
14
40. 8/20/2013
A short lesson on
responsive design
http://blog.creativemagma.com/featur
ed/2209/
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
29
© Karen N. Johnson, 2013
30
Phone vs. tablet
People often use both at the
same time.
Testing by planning user
scenarios is a good way to
uncover and identify
situations to test.
Build Your Mobile Testing Expertise v7
15
41. 8/20/2013
App Version
• A version page is an
opportunity to once
again show the app icon
and/or company logo.
• What about other
maintenance pages?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
31
Sitemaps
“They visually represent the relationship of
content to other content and provide a map for
how the user will travel through the
informational space.” Brian Fling, author Mobile Design and Development
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
32
16
42. 8/20/2013
exercise: build a list of ideas for testing based on
UX factors
•
•
•
With your team, determine what UX factor influence testing
Update the checklist of factors to reflect your decisions.
Build your own checklist or mindmap to identify what devices to test.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
33
UX Testing:
How will you decide what to test?
Possible Factors:
menu styles
navigation
input selection
UI controls
custom controls
links
lists
colors & symbols
currency
Build Your Mobile Testing
Expertise v7
images
video
completing forms
login
search, filter & sort
app version
sitemap
RWD
accessibility
© Karen N. Johnson, 2013
34
17
43. 8/20/2013
Mobile App & Device Settings
section 4
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
1
app, device & browser settings
•
•
•
•
•
•
•
•
•
•
•
•
•
Overall app permissions
Specific app permissions
App permission-dependencies
Device settings
Browser settings
Cookies
Wireless & network settings
Data Roaming
SD card settings
Swype
Display settings
Battery Settings/Status
Task Killers
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
2
1
44. 8/20/2013
Overall App Permissions
Know the app permissions
your mobile app requires.
Consider what the user may or
may not have enabled on the
device for all applications.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
3
Specific App permissions
What permissions does
your application require?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
4
2
45. 8/20/2013
App permission-dependencies
If your app interfaces with 3rd
party data such as location
finding, what happens when
that access is off, down or
unavailable?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
5
Device settings
• Consider the environment of
the device: what has been
enabled or not.
• Consider the ability or
restrictions to downloading
and installing applications.
• Consider upgrade scenarios
and whether permissions have
been changed or reset.
• Install testing has returned.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
6
3
46. 8/20/2013
Browser settings
© Karen N. Johnson, 2013
Build Your Mobile Testing Expertise v7
7
Language settings
• Finding what languages are
available on the device
• Choosing a different
language
• Accessing and using a site or
an app that’s multilingual
• Using different keyboards
on devices
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
8
4
47. 8/20/2013
Cookies
• Does your site use cookies?
• Check the login process
with cookies on/off.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
9
Wireless & Network Settings
Have you ever tested your
website when connectivity is
down or disrupted? This
scenario is more likely with
mobile devices.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
10
5
48. 8/20/2013
Network Mode
CDMA is used in the US (also in China
and India) but everywhere else uses
GSM or other modes of connection.
It can be challenging to test like a
user when you live in a different
country and some options are not
available on devices where you are
located.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
11
Network search
Users can switch from
and to an application or
browser session while
other activities are
taking place (including
incoming phone calls)
does your app handle
that disruption?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
12
6
49. 8/20/2013
Connecting to a network
A user could switch to
your app or website when
they’re not connected.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
13
Connection errors
If your app uses the SD
card for storage – what if
the card is already full or
another app writes to the
card?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
14
7
50. 8/20/2013
Data roaming
Would you enable your
testing team to leave the
office and experience
mobile as they travel
around the office?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
15
SD card settings
• Does your app store
data on a card?
• What if the card is
removed?
• What if a different card
is inserted?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
16
8
51. 8/20/2013
SIM cards
• A SIM card or Subscriber
Identity Module is a chip.
• SIM holds personal identity
information, cell number,
phone book, text messages
and other data.
• Again, what happens if the
card is full? Changed?
Removed?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
17
Swype settings
• Does your app work
with swyping or voice
input?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
18
9
52. 8/20/2013
Display settings
Consider whether a user
has made device settings
that can impact how your
application appears.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
19
Battery conditions
• Have you tested your
app under low battery
conditions?
• Is your application
responsible for battery
drainage?
-
See Julian Harty’s work
See Android Forum
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
20
10
53. 8/20/2013
Battery conditions: power save
Is this a variable you need to
consider in testing?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
21
Task Killers
•
•
•
What background tasks are
running?
What data is being synced?
Does a Task Killer help or not?
Android
http://www.pcworld.com/article/24
4067/taskkiller_apps_will_they_help
_or_hurt_your_battery_life.html
IOS
http://www.youtube.com/watch?v=
nEJA1yUgR80
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
22
11
54. 8/20/2013
Exercise:
Device Settings
• What device settings make an
impact on the app you’re testing?
1. Identify what settings
need to be factored into
testing or configuring a
device.
• What app permissions does your app
require?
2. Build a mind map or
outline structure to group
these settings together in
a logical way.
• What settings make an impact on
mobile browser testing?
3. Should the settings be
grouped according to
device manufacturers?
Such as grouping Nokia
settings vs. Samsung
settings?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
23
12
55. 8/20/2013
The Competitive Mobile
Marketplace
section 5
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
1
What do mobile apps compete on?
According to uTest’s white paper from the product
Applause, there are intuitive metrics such as:
• content
• elegance
• pricing
• usability
• and satisfaction
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
2
1
56. 8/20/2013
How steep is the competition?
•
•
•
•
Apple Store: 700,00+ apps
Google Play: 700,000+ apps
Blackberry App World: 100,000+ apps
By 2015, the International Data Corporation (IDC) predicts
182.7 billion mobile apps will be downloaded
• In 2010, 10.7 billion apps were downloaded
- source: uTest Applause white paper
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
3
MARKET RANKING
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
4
2
57. 8/20/2013
Market Rankings
Applyzer is a product that watches
market rankings. Notice how they
segregate data by country and mobile
app category.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
5
© Karen N. Johnson, 2013
6
Market Rankings
Distimo is another market watching
application.
Build Your Mobile Testing Expertise v7
3
58. 8/20/2013
Market Rankings
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
7
© Karen N. Johnson, 2013
8
Market Rankings
RankedNow is another product
tracking the competitive market.
Build Your Mobile Testing Expertise v7
4
59. 8/20/2013
MOBILE COMPETITORS
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
9
Competitive Analysis – Vertical Competition
mobile app to app
AccuWeather
Build Your Mobile Testing Expertise v7
The Weather Channel
© Karen N. Johnson, 2013
10
5
60. 8/20/2013
Competitive Analysis – Vertical Competition
tablet app to app
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
11
Competitive Analysis – Mobile Competition
the app Shazam vs. other ways to spend time
Notice how competitors might not
be offering the same product but
compete equally well for a user’s
time, purchase and use.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
12
6
61. 8/20/2013
Mobile competition:
Shazam’s “competitors”
See how this “competitor” isn’t offering
the same product but is vying for the
user’s time.
Build Your Mobile Testing Expertise v7
This competitor is a more typical
“competitor.”
© Karen N. Johnson, 2013
13
Mobile competition:
a sample of differences by store
Competitors vary from store to store.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
14
7
62. 8/20/2013
MARKET COMMENTS
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
15
Charmin – Sit or Squat
This Charmin app was liked until the app was tied to Facebook and requires a FB
Account in order to login or comment. Users have scathing comments about the change.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
16
8
63. 8/20/2013
Tide Stain Brain
The Tide app made a change when the company removed user comments on how to
Remove stains and in turn infuriated the user community.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
17
Chlorex myStain
The app gains respect and ratings by now being solely focused on pushing their product.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
18
9
64. 8/20/2013
Out of Milk
Note the number of users for the free version. And also note the overall ratings by users.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
19
Spotify
Note how users comment on updates that cause issues or comment on missing
functionality.
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
20
10
65. 8/20/2013
Competitive Analysis
Exercise:
1.
2.
3.
4.
5.
Select two mobile applications that
would seem to be competitors such as
AccuWeather and the Weather
Channel.
Review each app for the “intuitive
metrics” listed as factors.
Make notations on each of the factors
for both apps.
For satisfaction, you can use either your
satisfaction with the app, app store
comments or a combination of both.
If you were to apply a scoring to each
app, which app would you choose over
the other app?
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
21
Build Your Mobile Testing Expertise v7
© Karen N. Johnson, 2013
22
11