You name the testing topic, and Alan Page has an opinion on it, hands-on practical experience with it—or both. Spend the morning with Alan as he discusses a variety of topics, trends, and tales of software engineering and software testing. In an interactive format loosely based on discovering new testing ideas—and bringing new life to some of the old ideas—Alan shares experiences and stories from his twenty year career as a software tester. Topics may include philosophical rants about code coverage and test pass rates; thoughts on the developer/tester relationship and quality ownership; and insights on test leadership and the real future of test. Join Alan for a unique opportunity to participate in intriguing discussions about testing that will expand your testing knowledge, give you the insight you need to grow your own career, and help your organization succeed.

1. TF
AM Tutorial
10/1/2013 8:30:00 AM
"Alan Page: On Testing"
Presented by:
Alan Page
Microsoft Corporation
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Alan Page
Microsoft
Alan Page is a principal SDET—a fancy name for tester—on the Xbox console team at
Microsoft. Edging up on twenty years in software testing, Alan has previously worked on a
variety of Microsoft products including Windows, Windows CE, Internet Explorer, and Office
Lync. He spent some time as Microsoft’s director of test excellence where he developed and ran
technical training programs for testers throughout the company.

3. 8/27/2013
Alan Page: On
Testing
Test Ideas are…
Ideas
1

4. 8/27/2013
2

5. 8/27/2013
The Adjacent Possible
Liquid Networks
The Slow Hunch
Serendipity
Error
Exaptation
Platforms
On Test
Automation
On Exploratory
Testing
On Test Design
On Test
Careers
On the Value
of Test
On Test
Management
On working
with others
On The Death
of Test
On the Future
of Test
On Coding
Testers
On Test
Communities
On Agile and
Testing
On Writing
On Learning
On Leading
On Innovation
3

6. 8/27/2013
Test Automation
4

7. Exercise 1
8/27/2013
5

8. 8/27/2013
Exploratory
Testing
6

9. 8/27/2013
See Test Design
Test Design
7

10. 8/27/2013
Test This
Testing Design?
What is TestIdeas
8

11. 8/27/2013
Testing Quadrants: Marick, Crispin, Gregory, Hendrickson, others…
Critique Product
Suppor t Programming
Business (Custom er ) Facing
Te c h n o l o g y F a c i n g
Testing Quadrants: Marick, Crispin, Gregory, Hendrickson, others…
Runtime Analysis Tools
Functional Testing
Acceptance Tests
Scenarios
Usability Testing
World Readiness
Exploratory Testing
Take home / Beta Testing
Unit Testing
Code Coverage
Static Analysis
Code Churn Analysis
Performance Testing
Security / Privacy Testing
Stress Testing
Critique Product
Suppor t Programming
Business (Custom er ) Facing
Te c h n o l o g y F a c i n g
9

12. 8/27/2013
Functionality First?
Why are we looking for functional bugs? They're
easy. Stop looking for functional bugs. @docjamesw
“…testing, for a long time and in many places, has
been myopically focused on functional correctness,
rather than on value to people.”
-Michael Bolton
Random Internet Quotes
“I don't care about features or plug-ins or add-ons or
whatever. Just load the page quickly and let me get
on with my work”
“ <some company> makes wonderfully full featured
and functionally correct software, but my wife can’t
figure out how to use it”
10

13. 8/27/2013
Customer Focused Test
Design
Customer Focused Test
Design
11

14. 12
Upside Down Testing
8/27/2013

15. 8/27/2013
The Cost of Bugs
Is The Cost Curve Always True?
13

16. 8/27/2013
Hypothetical Cost Curve –
Functional Bugs
Hypothetical Cost Curve –
Design Bugs
14

17. 8/27/2013
Security
Privacy
Reliability
Performance
Usability
Globalization
Reliability
Send Web Query
10,000 times
Send malformed web
query 10,000 time
15

18. 8/27/2013
Performance
Test timing / latency of an action or scenario
Test large number of simultaneous connections
Scenario Testing
Customer Experience Validation
16

19. 8/27/2013
Customer Experience
Validation
Assess the
experience
Collaborate with
user research
Balance with
other
approaches
Evaluating a Scenario
Ease of
use
Responsi
veness
Usefulln
ess
Visual
Appeal
17

20. 8/27/2013
Example:
Showcase (marketing speak)
Scenario (project manager speak)
End-to-End Test (something
actionable)
(Optional) Variations
Xbox Knows You
Better Identity
Step in front of the console, get recognized and see
my curated content
Light/Dark settings
Child/Adult/Gender/Height/Size/Apparel
Curated content
Ease of Use
• Am I able to complete the scenario?
• Is it complicated? Do I need to perform
extraordinary steps to get what I need done?
• Are there glitches in the system that make it
difficult?
• Was it hard to find how to execute this
scenario? Are the features hidden?
• Is the experience consistent?
18

21. 8/27/2013
Responsiveness
• Did I feel like the scenario was fast and
fluid?
• At any point did I feel like I had to wait a
frustrating amount of time?
• Was I effectively distracted while waiting
for an action to take place? (ex. Movie
or animation while I wait)
Usefulness
• Would you use this scenario yourself
to accomplish this specific outcome?
• Does this scenario meet a need for our
consumer?
• Does this let me do something that I
want to do?
19

22. 8/27/2013
Visual Appeal
• Is the experience exciting to see
and hear?
• Is the UI polished?
• Does the UX make the experience
enjoyable?
Evaluation Scale
5
Love It! – You love it so much you’d shout it out at the top of
your lungs through a bullhorn from every rooftop you encounter
4
Like It – You like it a lot and might mention it during a lull in
conversation at a dinner party
3
Meh – You can live with it and neither like it or hate it. It’s
nothing special. There are some improvements that can be
made
2
Don’t Like It – The experience leaves a bad taste in your
mouth. You’d use it if you really had to otherwise, you’d stay
away from it.
1
Hate it! – You hate it so much that you would only use it if you
were under a Hogwarts compulsion spell
0
Not Implemented
20

23. 8/27/2013
Scenario
Love It Like It
Meh Don't Like Hate
Do this
43
10
1
1
1
Do that
3
2
9
14
20
Do the other thing
40
14
5
0
1
Do something else
20
3
5
7
15
Test Management
21

24. 8/27/2013
What is the role of a manager?
What are the activities of a manager?
learn your A-C-B’s
22

25. 8/27/2013
find the steepest learning curve
ride the gravy train
23

26. 8/27/2013
who do you know?
follow the leader - lead the follower
24

27. 8/27/2013
find a mentor
never let yourself get blocked
25

28. 8/27/2013
try a new way
don‘t flip the bozo bit
26

29. 8/27/2013
…and don’t burn bridges either
the three p’s
27

30. 8/27/2013
a career is a journey, not a sprint
have fun
28