Mike Cohn’s Test Pyramid describes a test automation strategy consisting of a wide base of unit tests, service-oriented acceptance tests for business logic, and a thin layer of tests exercising the user interface. Tests that provide the quickest feedback and fault precision serve as the testing foundation. So, how does this work in practice? How does a team achieve this level of test automation and maintain it over time? How can the team avoid redundancy in the various test layers? Jim Weaver demonstrates the different types of tests for a real feature—enforcing business rules for prescribing a specific class of medications—of a product in the healthcare domain. He explains how testing decisions were made as the feature was developed, and how the team baked their testing strategy into the overall process for product development. Jim describes decisions made by the team regarding testing, and shows the unit tests, FitNesse tests, and WebDriver tests built by the team to support the feature.

1.
W2
Test
Techniques
10/4/17
11:30
The
Software
Testing
Pyramid:
A
Concrete
Example
Presented
by:
Jim
Weaver
Vanderbilt
University
Medical
Center
Brought
to
you
by:
350
Corporate
Way,
Suite
400,
Orange
Park,
FL
32073
888-­‐-­‐-­‐268-­‐-­‐-­‐8770
·∙·∙
904-­‐-­‐-­‐278-­‐-­‐-­‐0524
-­‐
info@techwell.com
-­‐
http://www.starwest.techwell.com/

2.
Jim
Weaver
Vanderbilt
University
Medical
Center
Jim
Weaver
is
a
principal
developer
and
mentor
at
Vanderbilt
University
Medical
Center
(VUMC).
Jim
has
decades
of
experience
building
software
products
in
a
variety
of
industries,
with
a
focus
on
agile
practices
and
mentoring.
He
is
a
co-­‐
inventor
of
MyHealthAtVanderbilt,
a
VUMC
patient
web
portal.
Previously
at
ThoughtWorks,
Jim
coauthored
JWebUnit,
an
open
source
web
testing
framework.
At
GTE,
Jim
supported
and
trained
elements
of
the
US
Army
Signal
Corps
on
network
control
software
he
helped
build.
He
regularly
works
with
developers
and
testers
on
testing
strategies
for
products
built
in
a
variety
of
languages,
and
has
helped
train
teams
in
Extreme
Programming.
Find
Jim
on
Twitter
at
@tenoxtweets
and
on
LinkedIn.

3. The So'ware Tes,ng Pyramid:
A Concrete Example
Jim
Weaver
Vanderbilt
University
Medical
Center
(weaver.je@gmail.com)
Source
Code
and
Tests
hBps://github.com/weaverj/testpyramidexample
Selenium
tests:
Cindy
Leffler,
VUMC

4. Mike Cohn’s Test Pyramid
• FoundaKonal
base
of
unit
tests
• Minimum
necessary
tests
through
user
interface
• Service
tests
in
the
middle
• Service
refers
to
logical
services
–
business
logic,
required
outputs
to
inputs.

5. Mo,va,ons
Many
teams
tested
primarily
through
the
UI
TesKng
through
UI
slower,
more
briBle,
expensive,
less
precise
Unit
tests
are
easy
for
developers
to
write,
usually
in
same
language
as
applicaKon,
locate
bugs
more
precisely
Service
level
tests
can
test
logic
independent
of
UI

6. Challenges
• IntenKonal
Planning
and
E
• Shared
Understanding
• Frequent
CommunicaKon
CollaboraKon

7. Sample Applica,on : RxDemo
Rx
Demo:
Web-­‐based
Prescribing
ApplicaKon,
extracted
from
real
clinical
app
in
producKon
for
over
a
decade.

8. RxDemo Automated Test Strategy
Purpose
Authors
Tools
UI
Test
UI
funcKonality,
provide
some
end-­‐end
confirmaKon
QA
automaKon
Junit
and
WebDriver
to
Selenium
Server
Service
Test
business
logic,
publically
verifiable
Customers
(including
QA)
and
Developers
Fitnesse
Unit
Test
implementaKon,
aid
with
design
and
internal
code
quality
Developers
Junit,
Jasmine

9. Feature: Limit Opiate Rx Dura,on
When
clinician
is
prescribing
Opiates
or
Benzodiazepines,
limit
the
duraKon
of
the
prescripKon
to
30
days
or
under.

10. Feature Test Plan
Feature
Part
Unit
(Junit/Jasmine)
Service
(Fitnesse)
UI
(Selenium
+
Junit)
denKfy
if
drug
being
prescribed
is
Opiate
Calculate
duraKon
Apply
rule
to
prescripKon
and
eturn
response
nform
user
in
UI
if
ule
violated

11. Iden,fy Opiates / Benzos
Feature
Part
Unit
(Junit/Jasmine)
Service
(Fitnesse)
UI
(Selenium
+
Junit)
denKfy
if
drug
being
prescribed
is
Opiate
Test
source
Test
source

12. Calculate Dura,on in Days
Feature
Part
Unit
(Junit/Jasmine)
Service
(Fitnesse)
UI
(Selenium
+
Junit)
Calculate
duraKon
DuraKonParserShould
EDuraKonUnitShould
DuraKonToDays

13. Dura,on Valida,on Rule Applied
Feature
Part
Unit
(Junit/Jasmine)
Service
(Fitnesse)
UI
Apply
rule
to
prescripKon
and
eturn
response
RxValidatorShould
OpiateDuraKonValidaKonRuleShould
OpiateDuraKonRule
ValidatePrescripKonCommandShould

14. Inform User in UI if Rule Violated
Feature
Part
Unit
(Junit/Jasmine)
Service
(Fitnesse)
UI
(Selenium
/
Junit)
nform
user
in
UI
if
ule
violated
OpiateUnder30Allowed
OpiateOver30NotAllowed
RxValidatorShould
App.spec.js
RxValidaKonResponseHandler.spec.js

15. Team Process and Habits
Discuss
applicaKon-­‐wide
tesKng
strategy
–
layers
of
tests,
their
purpose,
tools,
team-­‐member
responsibiliKes.
Keep
all
team
members
involved
in
all
types
of
tests.
For
each
feature,
explicitly
discuss
a
test
plan
–
what
automated
tests
will
there
be
at
what
levels?
Watch
for
unwanted
duplicaKon
or
gaps
in
test
coverage.
When
a
bug
hits
producKon,
don’t
just
fix
it
-­‐
find
the
test
gaps
and
close
them.

16. References
Succeeding
with
Agile:
Sobware
Development
Using
Scrum
by
Mike
Cohn,
Addison-­‐Wesley,
2009
The
ForgoBen
Layer
of
the
Test
AutomaKon
Pyramid:
Mike
Cohn,
2009.
TestPyramid:
MarKn
Fowler,
2012.
Small,
Medium,
Large:
Mike
Bland,
2011.
Methodology
Per
Project:
Alistair
Cockburn,
1999.

17. Ques,ons?