Barely any modern software development project will be initiated without the discussion on test automation. With the demand to deliver software products in higher quality faster, test automation is oftentimes perceived by project leads as the silver bullet, allowing to ‘test everything’ without high overhead in employing skilled workforce. The tools are being discussed more often, than development and maintenance costs of automation frameworks that can support projects from inception into delivery and post-production. Incorrect approaches that result in choosing invalid test automation strategies are often the cause of much frustration later in the project, when the reality of test automation activities become visible and need to be accounted for. In this webinar, Anna will discuss her experiences in selecting short term and long term test automation strategies applied within various contexts, various skill sets that are required for running a successful test automation project, and suggest alternatives to full-blown test automation with the insufficient project resources. Learn: * when the test automation is an enabler, and when it’s an impediment to project success * when to start planning for automation, and how to select the tools and methods most suitable for specific project needs * how to anticipate investment costs and support long term automation effort across multiple projects * skills and roles in test automation

1. XBOSo&,
Inc.
All
Rights
Reserved.
1
Winning
Strategies In Test
Automation

2. XBOSo&
Founded in 2006
Dedicated to software quality
Software QA Consulting
Software Testing
Offices in San Francisco, Beijing and Amsterdam
XBOSo&,
Inc.
All
Rights
Reserved.
2

3. House Rules
Everyone
except
the
speakers
are
muted
QuesDons
via
the
gotowebinar
control
on
the
right
side
of
your
screen
or
through
TwiIer
@XBOSo&
QuesDons
can
be
asked
throughout
the
webinar
-­‐
we’ll
try
to
answer
them
at
the
end.
You
will
receive
info
on
recording
a&er
the
webinar
XBOSo&,
Inc.
All
Rights
Reserved.
3

4. Meet Our Speakers
• CEO and Founder of XBOSoft
• Relevant specialties and passions
– Software quality process,
evaluation, measurement and
improvement
– Software quality in use / UX design
– Mobile User Experience and
usability
– Cycling and travel
XBOSo&,
Inc.
All
Rights
Reserved.
4
Philip Lew
• Guest Speaker
• 15+ years in Testing and QA management
• International conferences speaker
• STP Community Advisory Board Member
• BBST Instructor
• Passions:
• Cross-functional team collaboration
• Testers’ skills development
• Quality advocacy
Anna Royzman
Sabrina Gasson
• Marketing Manager of XBOSoft
• Emails you all regularly to join our
industry hot topic webinars
• And invites you all to download our
latest trends in software testing
whitepapers.

5. MY
EXPERTISE
• Manager
of
so&ware
testers
and
automaDon
engineers
over
variety
of
projects,
in
both
agile
and
waterfall
• Leading
AT
SIG
(special
interest
group)
in
Technology
department
of
my
organizaDon
• 15+
years
experience
with
home-­‐grown
test
automaDon
frameworks,
commercial
and
open
source
tools
XBOSo&,
Inc.
All
Rights
Reserved.
5

6. POLL
QUESTION
WHAT’S
YOUR
PRIMARY
RESPONSIBILITY:
• A.
MANAGER
RESPONSIBLE
FOR
TESTING
• B.
AUTOMATION
ENGINEER
• C.
TESTER
• D.
NONE
OF
THE
ABOVE
XBOSo&,
Inc.
All
Rights
Reserved.
6

7. DEFINITION
OF
AUTOMATION
UTILIZATION
OF
TOOLS,
FRAMEWORKS
AND
SCRIPTS
THAT
ASSIST
WITH
TESTING
OR
CHECKING
TASKS
XBOSo&,
Inc.
All
Rights
Reserved.
7

8. EXAMPLES
• Unit
tesDng
frameworks
• Test
controllers
• Recording
and
playback
• Performance
tesDng
tools
• Security
tesDng
tools
• AnalyDcs
collectors
• Screen
capturing
tools
• Code
coverage
tools
• Protocol
tesDng
tools
• Monitoring
tools
• Database
tesDng
tools
• Simulators,
log
helpers,
debuggers
• Data
populaDon
scripts:
setups,
etc.
• Keystroke
recording
tools
XBOSo&,
Inc.
All
Rights
Reserved.
8

9. AUTOMATION
IS
A
HELPER
– CreaDng
a
‘safety
net’
of
business-­‐criDcal
workflows
and
logic
that
must
work
every
Dme
– Allowing
to
check
code
on
various
levels:
from
unit
components
to
distributed
systems
– AssisDng
humans
with
humongous
or
repeatable
tasks
(stress
or
load
tesDng,
system
integraDon
tesDng,
data
tesDng,
3rd
party
cerDficaDon
etc.)
– Debugging,
analyzing
or
replicaDng
failures
– Monitoring
running
programs
and
capturing
results
for
later
analysis
XBOSo&,
Inc.
All
Rights
Reserved.
9

10. AUTOMATION
IS
A
CODE
• It’s
created
by
humans:
IT
HAS
BUGS
• It
needs
development
and
debugging
Dme
• New/changed
funcDonality
affects
test
suite,
it
requires
maintenance
to
keep
up
to
date
• It
needs
skills
• It
has
upfront
costs
• There
is
always
version
2
XBOSo&,
Inc.
All
Rights
Reserved.
10

11. SKILLS
AND
RESPONSIBILITIES
• Toolsmith
(A
tester?
possibly!)
• AutomaDon
Engineer
(A
tester?
Not
necessarily!)
• Scripter
(A
tester?
Not
necessary,
but
it’s
good
to
be
one)
• Customer
of
automaDon
(Must
be
a
tesDng
expert
and
has
skills
in
tesDng
strategies)
XBOSo&,
Inc.
All
Rights
Reserved.
11

12. TEST
AUTOMATION
STRATEGY
• CriDcal
coverage
• Low
maintenance:
has
to
be
ready
for
next
iteraDon/
release
with
low
input
• Easy
to
run
on
demand
(one
buIon
click,
ways
to
select
tests
for
specific
scenarios,
pre-­‐selected
suites
for
different
coverage)
• Fast
(mulDple
environments,
distributed,
overnight,
etc.)
• TargeDng
for
‘0
tolerance’
–
failures
are
acceptable
for
bugs
in
the
product
only
XBOSo&,
Inc.
All
Rights
Reserved.
12

13. QUICK
POLL
WHAT
AUTOMATION
TOOLS/FRAMEWORKS
DO
YOU
USE:
A. LICENCED
TOOLS
B. OPEN
SOURCE
C. HOME-­‐GROWN
D. MIX
OF
THE
ABOVE
E. NONE
XBOSo&,
Inc.
All
Rights
Reserved.
13

14. SELECTING
TOOLS
AND
METHODS
• What’s
your
long
term
and
short
term
goals?
• Build,
buy
or
open
source?
• What
to
look
for
in
a
tool
• Who
will
be
supporDng
it:
– Do
you
have
specialists?
– Is
that
their
sole
responsibility?
– What
will
happen
when
your
company
has
RIF
or
specialist
leaves?
XBOSo&,
Inc.
All
Rights
Reserved.
14

15. COMMON
MISTAKES
TO
AVOID
• AutomaDng
new
versions
through
GUI
(it
can
work
well
with
legacy
version
for
tesDng
upgrades)
• Hardcoding
values
• Trying
to
test
all
aIributes
all
the
Dme.
The
test
failing
on
insignificant
aIribute
is
noise.
All
tests
failing
due
to
one
aIribute
is
more
noise
and
makes
AT
unusable.
• Product
code
replicas
• Get
carried
away
with
cool
features
of
less
value
• StarDng
automaDon
too
early
or
too
late
• Using
only
one
tool
for
everything
• Assuming
automaDon
replaces
human
interacDons
XBOSo&,
Inc.
All
Rights
Reserved.
15

16. MY ADVICE BASED ON MANY
YEARS EXPERIENCE:
XBOSo&,
Inc.
All
Rights
Reserved.
16
• AT
is
good
when
is
up
to
date;
it
should
be
available
when
team
needs
it
• Cross-­‐team
integraDon:
• build
simulators
for
majority
of
tests
• limit
integraDon
tests
to
the
most
criDcal
paths.
• Building
a
framework
for
your
own
product:
• Try
to
test
each
layer
separately
instead
of
tesDng
everything
from
one
layer
• Try
to
avoid
heavy
UI
automaDon
as
much
as
possible.
• Take
responsibility!
As
a
tester,
demand
that
the
features
important
to
you
are
prioriDzed
by
automaDon
engineers

17. QUICK
POLL
ARE
YOU
WINNING
WITH
YOUR
TEST
AUTOMATION:
A. YES
B. NO
C. WE
DON’T
HAVE
IT
XBOSo&,
Inc.
All
Rights
Reserved.
17

18. RISK
MITIGATION
HINTS
• If
you
don’t
have
a
toolsmith
in
your
team,
consider
lending
from
other
teams
or
asking
programmers
• Don’t
replace
exploratory
tesDng
with
AT
(it
needs
human
brains)
• Don’t
replace
usability
tesDng
with
AT
(it
needs
human
emoDons)
• Always
have
“plan
B”
in
an
emergency
such
as:
• test
server
failure
• DB
permission
errors
• blocking
issue
takes
too
long
to
fix
right
before
release
• no
electricity
XBOSo&,
Inc.
All
Rights
Reserved.
18

19. Post your questions on Twitter and we'll answer them @XBOSoft
Join us to keep updated on all our webinars, reports and whitepapers:
facebook.com/xbosoft
+xbosoft
linkedin.com/company/xbosoft
We post regularly on our blog – check us out! http://xbosoft.com/software-quality-blog/
Why not download our free Whitepapers, available here: http://xbosoft.com/knowledge-center/
You will receive an email with information on slides and recording. Any further queries regarding our services or ideas for
future webinars please email us! Services@xbosoft.com
Q+A
www.xbosoft.com