Software quality assurance days
20 Международная конференция
по вопросам качества ПО
sqadays.com
Минск. 24–26 ноября 2016
Владислав Романенко
Intellectsoft. Минск, Беларусь
Эвристики, мнемоники и другие греческие
слова в исследовательском тестировании
мобильных приложений
В тестировании ~ 3 лет,
ранее - инженер географических
информационных систем (ГИС)
Тестирую мобильные и веб
приложения
ISTQB Certified Test Analyst
QA-инженер в Intellectsoft
laechoppe@gmail.com
2
@laechoppe
Reasons for
Experience-based testing:
- lack of time
- lack of documentation
- issues could hardly be located using more
systematic approaches
3
Experience-based techniques
Error guessing
Checklist-based
testing
Exploratory testing
4
Error guessing
5
Checklist-based testingChecklist-based testing
6
Exploratory testing
- tests are designed and executed concurrently
- usually guided by a charter
- lightweight documentation (if any)
7
+ more interesting
+ more apparent bugs first
+ good if time is limited
+ works if docs are absent
+ more checks due to variations of tests
+ ….
Exploratory testing
- less experienced testers won't get as much as
more experienced
- could be hard to reproduce the issues
- not suitable if requirements are mandatory
- hard to collect metrics
- …
8
Heuristic
- a set of research methods to facilitate the discovery of a previously
unknown
Mnemonic
- a learning technique, allowing to remember the sequence and
structure of something.
Richard Of York Gave Battle In Vain
e
d
r
a
n
g
e
e
l
l
o
w
r
e
e
n
l
u
e
n
d
i
g
o
i
o
l
e
t
9
Mnemonic
Not Every Cat Eats Sardines
(Some Are Really Yummy)
?
10
Mnemonic
Not
Every
Cat
Eats
Sardines
(Some
Are
Really
Yummy)
11
Goldilocks Principle
- in any dataset there’s always a U-shaped distribution
12
Goldilocks Principle
too short text too long text just right text
13
SFDPOT (San Francisco Depot)
S Structure
F Function
D Data
P Platform
O Operations
T Time
14
S Structure
F Function
D Data
P Platform
O Operations
T Time
What files does it have?
Can I download the app?
Can I download an update?
Does the app write to an SD card? What
if the card is filled, removed or changed?
SFDPOT
15
S Structure
F Function
D Data
P Platform
O Operations
T Time
SFDPOT
16
S Structure
F Function
D Data
P Platform
O Operations
T Time
Does the app perform the tasks or
features it was designed to?
Does the app or site prevent, block or not
include features not intended on a mobile
device?
Does the app direct me to turn on
necessary related services?
SFDPOT
17
S Structure
F Function
D Data
P Platform
O Operations
T Time
SFDPOT
18
S Structure
F Function
D Data
P Platform
O Operations
T Time
What kinds of input does it process?
What does its output look like?
Does it come packaged with preset data
Does the app integrate with any
hardware such as bar code scanners for
payment or other purposes?
SFDPOT
19
S Structure
F Function
D Data
P Platform
O Operations
T Time
SFDPOT
20
S Structure
F Function
D Data
P Platform
O Operations
T Time
What operating systems does it run on?
Does the environment have to be
configured in any special way?
Does it depend on third-party
components?
Does the app depend on any device
settings?
SFDPOT
21
S Structure
F Function
D Data
P Platform
O Operations
T Time
SFDPOT
22
S Structure
F Function
D Data
P Platform
O Operations
T Time
Where and how the app will be used?
Does the app function when I am moving
and traveling around (switch wi-fi <-> 3G)?
What happens when a mobile interruption
occurs such as an incoming call or text
message?
SFDPOT
23
S Structure
F Function
D Data
P Platform
O Operations
T Time
SFDPOT
24
S Structure
F Function
D Data
P Platform
O Operations
T Time
What happens if the time zone is
switched?
What happens when my location is
switched?
Does the device auto detect a change in
time zones and the day and date?
Does the app find time-related data
based on my device time?
SFDPOT
25
S Structure
F Function
D Data
P Platform
O Operations
T Time
SFDPOT
26
SFDPOTI
S Structure
F Function
D Data
P Platform
O Operations
T Time
I Interface
Does the app work well on phones and
tablets?
Does the app work well when used with
“one hand?”
Is the mobile version consist or
complementary to the desktop version?
Is the app consistent with UI current
constructs in the marketplace?
27
SFDPOTI
S Structure
F Function
D Data
P Platform
O Operations
T Time
I Interface
28
SFDPOTIE
S Structure
F Function
D Data
P Platform
O Operations
T Time
I Interface
E Errors
Does the app continue gracefully after an
error?
Is the message helpful, instructive?
If a user provokes multiple errors in
succession, does the app continue?
Can I get to an error state that forces the
app to shut down?
What happens when I restart the crashed
app – is previous data or state saved?
Is there a method to send an error
report?
29
SFDPOTIE
S Structure
F Function
D Data
P Platform
O Operations
T Time
I Interface
E Errors
30
I SLICED UP FUN
I Inputs
S Store
L Location
I Interactions
C Communication
E Ergonomics
D Data
U Usability
P Platform
F Function
U User Scenarios
N Network
31
I Inputs
S Store
L Location
I Interactions
C Communication
E Ergonomics
D Data
U Usability
P Platform
F Function
U User Scenarios
N Network
I SLICED UP FUN
32
I Inputs
S Store
L Location
I Interactions
C Communication
E Ergonomics
D Data
U Usability
P Platform
F Function
U User Scenarios
N Network
I SLICED UP FUN
33
I Inputs
S Store
L Location
I Interactions
C Communication
E Ergonomics
D Data
U Usability
P Platform
F Function
U User Scenarios
N Network
I SLICED UP FUN
34
I Inputs
S Store
L Location
I Interactions
C Communication
E Ergonomics
D Data
U Usability
P Platform
F Function
U User Scenarios
N Network
I SLICED UP FUN
35
I Inputs
S Store
L Location
I Interactions
C Communication
E Ergonomics
D Data
U Usability
P Platform
F Function
U User Scenarios
N Network
I SLICED UP FUN
36
I Inputs
S Store
L Location
I Interactions
C Communication
E Ergonomics
D Data
U Usability
P Platform
F Function
U User Scenarios
N Network
I SLICED UP FUN
37
I Inputs
S Store
L Location
I Interactions
C Communication
E Ergonomics
D Data
U Usability
P Platform
F Function
U User Scenarios
N Network
I SLICED UP FUN
38
I Inputs
S Store
L Location
I Interactions
C Communication
E Ergonomics
D Data
U Usability
P Platform
F Function
U User Scenarios
N Network
I SLICED UP FUN
39
I Inputs
S Store
L Location
I Interactions
C Communication
E Ergonomics
D Data
U Usability
P Platform
F Function
U User Scenarios
N Network
I SLICED UP FUN
40
I Inputs
S Store
L Location
I Interactions
C Communication
E Ergonomics
D Data
U Usability
P Platform
F Function
U User Scenarios
N Network
I SLICED UP FUN
41
I Inputs
S Store
L Location
I Interactions
C Communication
E Ergonomics
D Data
U Usability
P Platform
F Function
U User Scenarios
N Network
I SLICED UP FUN
42
I Inputs
S Store
L Location
I Interactions
C Communication
E Ergonomics
D Data
U Usability
P Platform
F Function
U User Scenarios
N Network
I SLICED UP FUN
43
I Inputs
S Store
L Location
I Interactions
C Communication
E Ergonomics
D Data
U Usability
P Platform
F Function
U User Scenarios
N Network
Start from
I SLICED UP FUN
44
Heuristics help to:
find out what to start testing from
remember the context of app usage
avoid repeating the mistakes
…
TO SUM UP:
45
эвристика
мнемоника
техника
теория
диапазон
системный
параллельно
метод
символ
идея
эргономика
телефон
библиотека
график
микрофон
параметр
риск
…
Some of the Greek words:
46
http://www.rockynook.com/press-release-software-test-engineers-handbook-2nd-edition/
http://www.computing.dcu.ie/~ray/teaching/CA358/dorothy_graham.pdf
http://karennicolejohnson.com/2012/05/applying-the-sfdpot-heuristic-to-mobile-testing/
http://karennicolejohnson.com/2012/12/applying-the-sfdpot-heuristic-to-mobile-testing-adding-i/
http://karennicolejohnson.com/wp-content/uploads/2012/11/KNJohnson-2012-heuristics-mnemonics.pdf
https://www.utest.com/articles/rotation-tour-in-mobile-testing
http://www.kohl.ca/articles/ISLICEDUPFUN.pdf
http://www.satisfice.com/articles/sfdpo.shtml
http://michaeldkelly.com/blog/2005/9/20/touring-heuristic.html
https://goritskov.com/blog/heuristics-in-testing.html
Pictures from: mycutegraphics.com
References
47
laechoppe@gmail.com
48
@laechoppe

Эвристики, мнемоники и другие греческие слова в исследовательском тестировании мобильных приложений