The document discusses mobile application testing and provides examples of issues that can arise. It describes current problems like management not allowing enough testing time or prioritizing speed over quality. This can lead to bugs being missed and poor user experiences. The document also introduces taxonomies to help categorize different types of bugs, like those related to timing, to aid in more effective testing. Overall, it advocates for improved mobile app testing practices to avoid common pitfalls and ensure high quality user experiences.

1. 4/23/15
1
Mobile
App
Testing:
The
Good,
The
Bad,
and
The
Ugly
Jon
D.
Hagar,
Consultant,
Grand
Software
Testing
embedded@ecentral.com
Author:
Software
Test
Attacks
to
Break
Mobile
and
Embedded
Devices
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
1
* Gaming
Testing
Story
* It
only
takes
a
few
minutes
using
an
App
before
users
like
or
hate
it
* Worse
than
that.
.
.
* Many
users
will
post
a
social
media
review
of
the
app
* You
don’t
want
to
be
a
BAD
Copyright
2015,
Jon
D.
Hagar
Mobile-­‐Embedded
Taxonomies
from
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
2
The
Mobile
Opportunity

2. 4/23/15
2
* Depth
* Passion
* Speed
What
Does
it
Take
to
be
a
Great
Mobile
App
Tester?
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
Software Test Attacks to Break Mobile and Embedded Devices
3
* As
the
names
imply,
these
are
devices—small,
held
in
the
hand,
connected
to
communication
networks,
including
* Cell
and
smart
phones
–
apps
* Tablets
* Medical
devices
* Typically
have:
* Many
of
the
problems
of
classic
embedded
systems
* The
power
of
PCs/IT
* More
user
interface
(UI)
than
classic
embedded
systems
* Fast
and
frequent
updates
* However,
mobile
devices
are
“evolving”
with
more
power,
resources,
apps,
etc.
* Mobile
is
the
“hot”
area
of
computers/software
* Testing
rules
and
concepts
are
still
evolving
* Now
starting
to
include
IoT
You
know
what
they
are
right?
Mobile
and
Handheld?
Copyright
2015,
Jon
D.
Hagar
Mobile-­‐Embedded
Taxonomies
from
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”

3. 4/23/15
3
* Requirements
verification
checking
* Necessary
but
not
sufficient
* Risk–based
testing
* Tried
and
true
in
many
contexts
including
mobile,
but
we
need
more
Here
comes
the
Good,
Bad
and
Ugly
We
Need
Better
App
Testing
Copyright
2015,
Jon
D.
Hagar
Mobile-­‐Embedded
Taxonomies
from
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
The
Bad
You
are
between
a
Management
Rock
and
a
Hard
App
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
6

4. 4/23/15
4
* Management
directed
“No
testing”
* Dev-­‐ops
without
enough
“thinking”
of
context
and
risk
to
find
the
big
BUGS
* Stupid
requirements
verification
checking
without
GOOD
test
activities
* Testing
without
thinking
of
* cost
* schedule
* users
Con:
Current
Badness
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
7
* Are
you
part
of
the
problem?
* Do
you
help
management
“SEE”
the
info
they
need?
* Are
you
Agile?
* Are
you
using
your
testing
skills
daily?
* Bug
are
out
there
(and
always
will
be)…………..
Pro:
In
the
Bad
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
Software Test Attacks to Break Mobile and Embedded Devices
8

5. 4/23/15
5
* From
Wikipedia:
Taxonomy
is
the
practice
and
science
of
classification.
The
word
finds
its
roots
in
the
Greek
τάξις,
taxis
(meaning
'order',
'arrangement')
and
νόμος,
nomos
('law'
or
'science').
Taxonomy
uses
taxonomic
units,
known
as
taxa
(singular
taxon).
In
addition,
the
word
is
also
used
as
a
count
noun:
a
taxonomy,
or
taxonomic
scheme,
is
a
particular
classification
("the
taxonomy
of
..."),
arranged
in
a
hierarchical
structure.
* Helping
to
“understand
and
know”
Copyright
2015,
Jon
D.
Hagar
Mobile-­‐Embedded
Taxonomies
from
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
9
A
Bad
Situation
-­‐
Lets
look
for
bugs,
but
where?
Copyright
2015,
Jon
D.
Hagar
Mobile-­‐Embedded
Taxonomies
from
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
10
Pro:
Taxonomy
(researched)
Super
Category
Aero-­‐Space
Med
sys
Mobile
General
Time
3
2
3
Interrupted
-­‐
Satura>on
(over
>me)
5.5
Time
Boundary
–
failure
resul>ng
from
incompa>ble
system
>me
formats
or
values
0.5
1
Time
-­‐
Race
Condi>ons
3
1
Time
-­‐
Long
run
usages
4
1
20
Interrupt
-­‐
>ming
or
priority
inversions
0.7
3
Date(s)
wrong/cause
problem
0.5
1
Clocks
4
2
Computa>on
-­‐
Flow
6
23
19
Computa>on
-­‐
on
data
4
1
3
1

6. 4/23/15
6
Copyright
2015,
Jon
D.
Hagar
Mobile-­‐Embedded
Taxonomies
from
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
11
Taxonomy
part
2
Super
Category
Aero-­‐Space
Med
sys
Mobile
General
Data
(wrong
data
loaded
or
used)
4
5.00
2
Ini>aliza>on
6
2.00
3
5
Pointers
8
2.00
18
10
Logic
and/or
control
law
ordering
8
43
3
30
Loop
control
–Recursion
1
Decision
point
(if
test
structure)
0.5
1
1
Logically
Impossible
&
dead
code
0.7
Opera>ng
system
–
(Lack
of
Fault
tolerance
,
interface
to
OS,
other)
1.5
2
6
Software - Hardware interfaces
16
13
So9ware
-­‐
Software Interface
5
2.00
3
So9ware
-­‐
Bad command- problem
on server
3
5
UI
-­‐
User/
operator
interface
4
5.00
20
10
UI
-­‐
Bad
Alarm
0.5
3
UI
-­‐
Training
–
system
fault
resul>ng
from
improper
training
3
Other
10.6
9.00
5
5
Note:
one
report
on
C/C++
indicated
70%
of
errors
found
involved
pointers
* How
many
of
you
have
a
Mobile
App
taxonomy
that
you
use?
Question
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
12

7. 4/23/15
7
The
Ugly
We need Wisdom, Tooling, and Security
13
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
* Some
of
you
lack
mobile
tester
skills
* Many
of
you
suffer
from
group
think
and
lack
wisdom
* We
listen
to
the
loudest
voices
* Testers
do
not
use
available
ideas
to
aide
their
skill
base
* Attacks,
techniques,
tools,
concepts,
standards,
etc.
Con:
Mobile
can
have
an
Ugly
Face
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
14

8. 4/23/15
8
* Danger
of
group
think
in
Agile
Mobile
Teams
* Amplification
* Snowballing
effect
* Polarization
* Ignoring
critical
minority
opinions
Pro:
You
Need
Test
Wisdom
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
15
* Stop
talking
and
LISTEN
to
all
sides,
particularly
the
ones
you
may
not
agree
with
* Question
beliefs
* Be
passionate
and
follow
your
bliss
about
testing
* Try
to
remain
open
minded
* Do
not
submit
to
the
negatives
of
group
think
* Consider
the
context
of
the
testing
and
believe
that
context
matters
* Seek
the
council
of
people
you
believe
to
be
wise
* Reward
your
test
team
for
being
open
and
providing
other
views
without
fear
* Try
to
take
a
role
of
“devil’s
advocate”
in
your
test
team
* Fight
the
“me
too”
syndrome
and
everyone
falling
in
line
to
the
loudest
voice
* Work
to
be
a
knowledgeable
and
skilled
tester
(they
are
different)
* Be
the
voice
of
loyal
opposition
in
the
team
and
think
outside
of
the
group
“box”
* Don’t
paint
a
viewpoint
as
totally
invalid,
when
a
few
ideas
of
the
viewpoint
conflict
with
local
ideals
Seeking
Test
Wisdom
(Pro:
try
these
tricks)
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
16

9. 4/23/15
9
Categories
of
Automation
Tooling
(Open
Source
and
Commercial)
*
Capture
Playback
-­‐
Actual
devices
(cabinet
vs
a
pile)
vs
Emulator
-­‐
API
vs
GUI/UI
*
Planning
and
lifecycle
support
*
Modeling
-­‐
Risks
-­‐
Mind-­‐mapping
-­‐
Formal
models
(UTP)
-­‐
Test
Techniques
Pro/Con?
-­‐
Mobile/Handheld
Test
Tools
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
17
* To
Automate
or
Not?
* When
testing
configurations
of
hw/sw
(good
idea)
* When
testing
combinations
(combinatorial
test
tools)
* When
dealing
with
testing
qualities
* Security
(very
good
idea)
* Reliability
(necessary)
* Configuration
management
(can
not
be
done
without)
* Usability
(important
but
a
hard
one
and
questionable
tools)
* When
supporting
Development
* Structural
testing
(measures
coverage)
* Static
code
analysis
(finds
hard
to
test
bugs)
* Dev-­‐Ops,
Continuous
Integration
and
Agile
(really
good)
More
on
Test
Tools
–
Now
in
Mobile
Support
has
Improved
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
Software Test Attacks to Break Mobile and Embedded Devices
18

10. 4/23/15
10
* Your
app
gets
on
the
nightly
news
* Your
team
sees
security
as
someone
else’s
problem
Real
Ugly:
Security
and
Privacy
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
19
* Mobile–
IoT
systems
are
highly
integrated
hardware–
software–system
solutions
which:
* Must
be
highly
trustworthy
since
they
handle
sensitive
data
* Often
perform
critical
tasks
* Security
holes
and
problems
abound
* Coverity
Scan
2010
Open
Source
Integrity
Report
-­‐
Android
* Static
analysis
test
attack
found
0.47
defects
per
1,000
SLOC
* 359
defects
in
total,
88
of
which
were
considered
“high
risk”
in
the
security
domain
* OS
hole
Android
with
Angry
Birds
* Researchers
Jon
Oberheide
and
Zach
Lanier
* Robots
and
Drones
rumored
to
be
attacked
* Cars
and
medical
devices
being
hacked
The
Current
Security
Situation
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”

11. 4/23/15
11
* Fraud
–
Identity
* Worms,
virus,
etc.
* Fault
injection
* Processing
on
the
run
* Hacks
impact
* Power
* Memory
* CPU
usage
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
Con:
Mobile
Security
Bugs
(taxonomy)
• Eavesdropping
–
“yes
everyone
can
hear
you”
• Hijacking
• Click-­‐jacking
• Voice/Screen
• Physical
Hacks
• File
snooping
• Lost
phone
* A
pattern
(of
testing)
based
on
a
common
mode
of
failure
seen
over
and
over
* Part
of
Exploratory
Testing
* May
be
seen
as
a
negative,
when
it
really
is
a
positive
* Goes
after
the
“bugs”
that
may
be
in
the
software
* May
include
or
use
classic
test
techniques
and
test
concepts
* Lee
Copeland’s
book
on
test
design
* Many
other
good
books
* A
Pattern
(more
than
a
process)
which
must
be
modified
for
the
context
at
hand
to
do
the
testing
* Testers
learn
mental
attack
patterns
working
over
the
years
in
a
specific
domain
Pro:
Apply
Attack-­‐based
Testing
What
is
an
attack?
Copyright
2015,
Jon
D.
Hagar
Mobile-­‐Embedded
Taxonomies
from
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”

12. 4/23/15
12
* Apply
when
the
device
is
mobile
and
has
* Account
numbers
* User-­‐ids
and
passwords
* Location
tags
* Restricted
data
*
Current
authentication
approaches
in
use
on
mobile
devices
* Server-­‐based
* Registry
(user/password)
* Location
or
device-­‐based
* Profile-­‐based
Security
Attacks
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
* Attack
28
Penetration
Attack
Test
* Attack
28.1
Penetration
Sub–Attacks:
Authentication
—
Password
* Attack
28.2
Sub–Attack
Fuzz
Test
* Attack
29:
Information
Theft—Stealing
Device
Data
* Attack
29.1
Sub
Attack
–Identity
Social
Engineering
* Attack
30:
Spoofing
Attacks
* Attack
30.1
Location
and/or
User
Profile
Spoof
Sub–Attack
* Attack
30.2
GPS
Spoof
Sub–Attack
Security
Attacks
(Con:
only
a
starting
point,
a
checklist
of
things
to
start
with)
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”

13. 4/23/15
13
* What
kind
of
App
software
do
you
work
on?
* Security
concerns?
* Privacy
concerns?
What
is
missing?
Exercise
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
25
§ Security
attacks
must
be
done
with
the
knowledge
and
approval
of
owners
of
the
system
and
software
§ Severe
legal
implications
exist
in
this
area
§ Many
of
these
attacks
must
be
done
in
a
lab
(sandbox)
§ In
these
attacks,
I
tell
you
conceptually
how
to
“drive
a
car
very
fast
(150
miles
an
hour)
but
there
are
places
to
do
this
with
a
car
legally
(a
race
track)
and
places
where
you
will
get
a
ticket
(most
public
streets)”
§ Be
forewarned
-­‐
Do
not
attack
you
favorite
app
on
your
phone
or
any
connected
server
without
the
right
permissions
due
to
legal
implications
Warnings
When
Conducting
Security
Attacks
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”

14. 4/23/15
14
Finally,
The
Good
–
Functional
and
Non-­‐functional
Experiments
and
Attacks
(Exploratory
testing)
Skills
App
testers
should
have
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–”So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
27
Attacks
(from
Software
Test
Attacks
to
Break
Mobile
and
Embedded
Devices)
* Attack
1:
Static
Code
Analysis
* Attack
2:
Finding
White–Box
Data
Computation
Bugs
* Attack
3:
White–Box
Structural
Logic
Flow
Coverage
* Attack
4:
Finding
Hardware–System
Unhandled
Uses
in
Software
* Attack
5:
Hw-­‐Sw
and
Sw-­‐Hw
signal
Interface
Bugs
* Attack
6:
Long
Duration
Control
Attack
Runs
* Attack
7:
Breaking
Software
Logic
and/or
Control
Laws
* Attack
8:
Forcing
the
Unusual
Bug
Cases
* Attack
9
Breaking
Software
with
Hardware
and
System
Operations
* 9.1
Sub–Attack:
Breaking
Battery
Power
* Attack
10:
Finding
Bugs
in
Hardware–Software
Communications
* Attack
11:
Breaking
Software
Error
Recovery
* Attack
12:
Interface
and
Integration
Testing
* 12.1
Sub–Attack:
Configuration
Integration
Evaluation
* Attack
13:
Finding
Problems
in
Software–System
Fault
Tolerance
* Attack
14:
Breaking
Digital
Software
Communications
* Attack
15:
Finding
Bugs
in
the
Data
* Attack
16:
Bugs
in
System–Software
Computation
* Attack
17:
Using
Simulation
and
Stimulation
to
Drive
Software
Attacks
* Attack
18:
Bugs
in
Timing
Interrupts
and
Priority
Inversion
* Attack
19:
Finding
Time
Related
Bugs
* Attack
20:
Time
Related
Scenarios,
Stories
and
Tours
* Attack
21:
Performance
Testing
Introduction
* Attack
22:
Finding
Supporting
(User)
Documentation
Problems
* Sub–Attack
22.1:
Confirming
Install–ability
* Attack
23:
Finding
Missing
or
Wrong
Alarms
* Attack
24:
Finding
Bugs
in
Help
Files
* Attack
25:
Finding
Bugs
in
Apps
* Attack
26:
Testing
Mobile
and
Embedded
Games
* Attack
27:
Attacking
App–Cloud
Dependencies
* Attack
28
Penetration
Attack
Test
* Attack
28.1
Penetration
Sub–Attacks:
Authentication
—
Password
Attack
* Attack
28.2
Sub–Attack
Fuzz
Test
* Attack
29:
Information
Theft—Stealing
Device
Data
* Attack
29.1
Sub
Attack
–Identity
Social
Engineering
* Attack
30:
Spoofing
Attacks
* Attack
30.1
Location
and/or
User
Profile
Spoof
Sub–Attack
* Attack
30.2
GPS
Spoof
Sub–Attack
* Attack
31:
Attacking
Viruses
on
the
Run
in
Factories
or
PLCs
* Attack
32:
Using
Combinatorial
Tests
* Attack
33:
Attacking
Functional
Bugs
Copyright
2015,
Jon
D.
Hagar
Mobile-­‐Embedded
Taxonomies
from
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”

15. 4/23/15
15
Attack
1:
Static
Code
Analysis
(testing)
* When
to
apply
this
attack?
*
After/during
coding
* What
faults
make
this
attack
successful?
* Many
* Example:
Issues
with
pointers
* Who
conducts
this
attack?
*
Developer,
tester,
independent
party
* Where
is
this
attack
conducted?
* Tool/test
lab
* How
to
determine
if
the
attack
exposes
failures?
* Review
warning
messages
and
find
true
bugs
* How
to
conduct
this
attack
* Obtain
and
run
tool
* Find
and
eliminate
false
positive
* Identify
and
address
real
bugs
* Repeat
as
code
evolves
* Single
unit/object
* Class/Group
* Component
* Full
system
29
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–”So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
Attack
2:
Finding
White–Box
Data
Computation
Bugs
* When
to
apply
this
attack?
*
After/during
coding
* What
faults
make
this
attack
successful?
* Mistakes
associated
with
data
* Example:
Wrong
value
of
Pi
* Who
conducts
this
attack?
*
Developer,
tester,
independent
party
* Where
is
this
attack
conducted?
* Development
Tool/test
lab
* How
to
determine
if
the
attack
exposes
failures?
* Structural-­‐data
test
success
criteria
not
met
* How
to
conduct
this
attack
* Obtain
tool
* Determine
criteria
and
coverage
* Create
test
automation
with
specific
values
(really
a
programing
problem)
* NOT
NICE
NUMBERS
* Run
automated
test
cases
* Resolve
failures
* Peer
check
test
cases
* Repeat
as
code
evolves
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–”So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”

16. 4/23/15
16
* When
to
apply
this
attack?
…when
your
app/device
has
a
user
* What
faults
make
this
attack
successful?
…devices
are
increasingly
complex
* Who
conducts
this
attack?
…see
chart
on
Roles
* Where
is
this
attack
conducted?
…throughout
lifecycle
and
in
user’s
environments
* How
to
determine
if
the
attack
exposes
failures?
* Unhappy
“users”
* Bugs
found
* See
sample
checklist
Attack
:
Testing
Usability
Mobile IoT Usability Tends to be “Poor”
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
* Refine
checklist
to
context
scope
* Define
a
role
* Watch
what
is
happening
with
this
role
* Define
a
usage
(many
different
user
roles)
* Guided
explorations
or
ad
hoc
* Stress,
unusual
cases,
explore
options
* Capture
understanding,
risk,
observations,
etc.
* Checklist
(watch
for
confusion
of
the
tester)
* Run
Exploratory
Attack
(s)
* Learn
* Re-­‐plan-­‐design
* Watch
for
Bias
* Switch
testers
* Repeat
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
Usability
Attack
Pattern

17. 4/23/15
17
The
Good,
Bad,
and
Ugly
of
Mobile
App
Testing
Lots
of
room
for
Growth
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
33
How
to
be
Better
after
This
Section
Pick
One
or
Two
to
work
On
Cons:
Bad
and
Ugly
* Taxonomy
help
only
if
you
use
them
* Skill
improvement
* Knowledge
and
Skill
* Security
Testing
* Attack,
Attack,
Attack
Pro:
The
Good
* Better
and
Faster
* Functional
testing
* Test
strategy
and
planning
* Test
Attacks
* Tools
and
technique
maturing
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices
“
34 After Mobile
comes IoT

18. 4/23/15
18
* There
will
always
be
Good,
Bad,
and
Ugly
* Work
with
the
Good
* Work
to
over
come
the
Bad
* Change
the
Ugly
into
good
* Understanding
your
local
context
and
error
patterns
is
important
(one
size
does
NOT
fit
all)
* Attacks
are
patterns…you
must
still
THINK
and
tailor
Wrap
Up
of
this
Session
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
* James
Whittaker
(attacks)
* Elisabeth
Hendrickson
(simulations)
* Lee
Copeland
(techniques)
* Brian
Merrick
(testing)
* James
Bach
(exploratory
and
tours)
* Cem
Kaner
(test
thinking)
* Jean
Ann
Harrison
(her
thinking
and
help)
* Many
teachers
* Generations
past
and
future
* Books,
references,
and
so
on
Notes:
Thank
You
(ideas
used
from)
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”

19. 4/23/15
19
* “Software
Test
Attacks
to
Break
Mobile
and
Embedded
Devices”
–
Jon
Hagar
* “How
to
Break
Software”
James
Whittaker,
2003
* And
his
other
“How
To
Break…”
books
* “A
Practitioner’s
Guide
to
Software
Test
Design”
Copeland,
2004
* “A
Practitioner’s
Handbook
for
Real-­‐Time
Analysis”
Klein
et.
al.,
1993
* “Computer
Related
Risks”,
Neumann,
1995
* “Safeware:
System
Safety
and
Computers”,
Leveson,
1995
* Honorable
mentions:
* “Systems
Testing
with
an
Attitude”
Petschenik
2005
* “Software
System
Testing
and
Quality
Assurance”
Beizer,
1987
* “Testing
Computer
Software”
Kaner
et.
al.,
1988
* “Systematic
Software
Testing”
Craig
&
Jaskiel,
2001
* “Managing
the
Testing
Process”
Black,
2002
Book/Notes
List
(my
favorites)
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
• www.stickyminds.com
–
Collection
of
test
info
• www.embedded.com
–
info
on
attacks
* www.sqaforums.com
-­‐
Mobile
Devices,
Mobile
Apps
-­‐
Embedded
Systems
Testing
forum
• Association
of
Software
Testing
– BBST
Classes
http://www.testingeducation.org/BBST/
• Your
favorite
search
engine
More
Resources
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices