Mobile is an integral part of our daily lives, and if it’s not already part of your business model, it soon will be. When that happens, will you be ready to tackle the demands of testing web and native mobile apps? From the perspective of a test lead, Eric Montgomery describes the challenges Progressive Insurance, a company with a strong web presence, recently faced—learning new technologies, transforming the approach of testers from PC-based to mobile-based, and working with testing tools in a market that has yet to see a definitive leader emerge. Learn from Eric's experiences and return to your job with ideas on training web testers to be mobile testers. Take back proven techniques for testing mobile devices, ways of choosing devices for test, methods of sharing information, developing a sense of community among testers, choosing tools from the available market, and keeping up with rapid technology changes.

1. T16
Mobile Testing
5/2/2013 1:30:00 PM
Introducing Mobile Testing to Your
Organization
Presented by:
Eric Montgomery
Progressive Insurance
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Eric Montgomery
A system test engineer for Progressive Insurance, Eric Montgomery joined Progressive as a C/C++/C#
developer, programming their quoting, rating, and renewal logic. On a project to replace Progressive's
mainframe servicing system with a .NET version, Eric transitioned to testing, first developing headless test
harnesses for business logic functions and then becoming test lead of one of the largest test events ever
run at Progressive. Since 2011, Eric has been the test lead for Progressive’s mobile transformation
program, which created several native apps, including the award-winning Progressive flagship app and
the “mobilization†of many of Progressive's web sites.

3. 4/19/2013
Introducing Mobile Testing
to Your Organization
Eric W Montgomery
System Test Engineer | Progressive Insurance
Who is Progressive?
»
»
»
»
Mayfield Village, OH
Offering insurance since 1937
Web awards (Webby)
Mobile awards (Keynote Brand Innovation)
1

4. 4/19/2013
Who Am I?
»
»
»
»
Cleveland native
Miami University, Oxford, OH
Progressive since 2003
Mobile Test Lead, 2011-2013
How it all Began
»
»
»
»
2 developers (1 iOS/iPhone, 1 Android)
2 devices (1 iPhone, 1 Android)
1 tester
Mobile Program
2

5. 4/19/2013
Agenda
»
»
»
»
People
Devices, tools
Testing techniques
Stories, experiences
Overcoming the Fear of Mobile
» Common issue
» Practice testing mobile with a device
» Discuss terminology
–
–
–
–
App types: native, web, hybrid
Device features
Operating systems: Android, iOS
so many more
3

6. 4/19/2013
People
» Key component
» Be willing to adjust quickly
» Constant
learning
Mobile Testers’ Forum
» Meets bi-weekly
» Variety of topics
– Project work
– Technology trends
– Testing techniques
» Network
» Answer questions without
asking
» Presentations
» Learn/teach technologies
4

7. 4/19/2013
Devices
» 2 devices (1 iPhone, 1 Android), supplement with
emulators
» How we learned what phone our CEO had
» Emulators are insufficient for System Testing
– Not exactly the same
– Not handling a device
– Different hardware: camera,
location services
– Android: manufacturers add code
» Still good for unit, smoke testing
Device Selection
X515, 1.37%
SCH-I510,
SGH-I717, 1.41% 1.35%
XT907, 1.26% SGH-I997, PH39100,ADR6425LVW,
SCH-I405,
GT-I9000, 0.90%
SGH-I777,
VILLE, 0.87%
1.24%
0.92%
0.94%
XT894, 1.27%
0.94%
1.01%
SGH-T769, 0.83%
XT862, 0.87%
SGHSCH-I535, 1.41%
Z710, 0.85%
SPH-D700,
T679,
ADR6350, 1.44%
0.80%
VS840, 0.80%
0.74%
SCH- PJ83100, 1.47%
MB855, 0.79%
I500, MDX2, 1.47%
MS770, 0.72%MB886, 0.74%
1.48%
N860,
MB865, 0.57%
P999, 0.55% 0.70%
X325C, 1.49%
MD2, 0.54%0.55%
SPH-930,
MDX2, 1.51%
VS920, GNEX,
ADR6400L, 1.57%
0.50% 0.51%
HDH, 1.82%
ADR6300,
0.48%
SGH-I727, 1.89%
Other, 8.70%
PC36100, 2.13%
C771,
0.47%
SGN, 2.18%
XT875, 2.27%
MOTXT912B,
8.98%
SGH-989, 3.26%
SGHGTI747, I9300,
4.39%
4.99%
SPH-D710, 4.06%
SGH-T999, 3.80%
SPHL710,
5.50%
GLAC,
0.84%
P9070, 0.47%
AMZ, 0.47%
ELC, 0.46%
SCH-I200, 0.45%
MB860, 0.41%
VS910, 0.41%
SCH-R760, 0.39%
E739BK, 0.38%
MYTSLD, 0.37%
SGH-I897, 0.37%
PH44100, 0.36%
C800, 0.35%
ADR6330, 0.34%
G2TCH, 0.33%
MS910, 0.32%
SCH-R720, 0.30%
ADR6410LVW, 0.29%
SCH-R930, 0.28%
P925, 0.27%
SPH-D600, 0.26%
M886, 0.25%
LS840, 0.25%
SGH-T959, 0.24%
SCH-R920, 0.23%
SCH-T839, 0.23%
SGH-I577, 0.23%
5

8. 4/19/2013
Device Selection
»
»
»
»
»
Consider project goals
Ask questions
Eliminate options!
Equivalence classes
Crowdsourcing
Other
8%
D
10%
A
47%
C
10%
B
25%
Device Selection
6

9. 4/19/2013
Device Selection: Using Data
» Data
» Market trends
» Making decisions jointly with customer
Devices: Owning
» Warehousing/library
» Maintaining
– Cleaning
– Operating systems
» Cords
7

10. 4/19/2013
Testing Tools, Tips & Techniques
Old Techniques Made New Again
» Error handling
– Edits
– Nonsense messages
– Error logging
» Dates
– Valid ranges
– Formats
Testing Tools, Tips & Techniques
Automation
» How it works
» Native
– Tree structure
– Unit/Integration level tests
» Web
8

11. 4/19/2013
Testing Tools, Tips & Techniques
New Techniques
» Brain
– Perspective
– Pair testing
» Devices
» Chrome
– Developer panel
– Remote debugging
» View source bookmarklet for iOS
» Screen shots
– Recording apps
– Camera, video
» Bug reports/exploratory sessions
Testing Tools, Tips & Techniques
New Techniques
» Settings
– User settings/airplane mode
– Moving between networks
– Cookies, JavaScript
» Navigation
– Service calls
– Links
» Keyboards
– Numeric
– Email
– iOS vs. Android
» Data Usage
9

12. 4/19/2013
Testing Tools, Tips & Techniques
New Techniques
» Mobile
–
–
–
–
Rotations
Location services, camera
Moving around
Interruptions
» Look and Feel/UX
–
–
–
–
Consistency
Font, colors, headings, images
Spelling, grammar
Alignment, margins
» Change browsers, devices
» Deployment
Summary
10

13. 4/19/2013
Eric W Montgomery
emontgo1@progressive.com
11