“It works on my machine”
Closing the loop between
Development and Testing
Joe Shirey/Steven Lange
joe.shirey@microsoft.com/stevenl@microsoft.com

Agenda
Overview
•
Whose Bug is it anyway?
•
Yours, Mine and Ours
•
The bug stops here
•
Bug to the Future
•
Summary
•
Q&A
•

Famous Bugs
1945 – Harvard Mark II
•
1985-1987 – Therac 25
•
1990 – AT&T Switches
•
1996 - Ariane 5 Flight 501
•
2007 – Excel calculation
•

Why Test? Why Not?
• Why are we testing?
• What are we testing?
• Who is responsible for
testing?
• Do you want it on time
or do you want it to be
perfect?
• How do you define a
good test?
• Bugs are bound to
happen, why can’t we
just live with it?

Whose bug is it anyway?
• Picking teams
– Beyond Development vs. Test
• Finding bugs
– Developer, tester or customer?
• Communicating the status
– We know about it, now what?
• It’s fixed, what next?
– Likely to resurface?

Picking Teams
• producer
• Development
generates • non-mgmt
• business
• technical
• customer
• test
• Management
consumes • Business
• Technical

Finding Bugs
• Developers, testers or customers?
• Is it important?
• Understanding values

V-Model

Communicating Status

What’s Next?
• Where do we go from
here?
• We’ve identified :
• our teams,
• our bug spotters
• how they will be fixed
• Will we have to repeat
this during the next
iteration?
• Yes, if we don’t
document and learn
from this cycle.
• No, if we do apply these
lessons to our process.

DEMO
Work Items for All

Yours, Mine and Ours
Lines of code written Test Results gathered
• •
Number of bugs closed Number of bugs opened
• •
Shipping on time Shipping quality
• •
On to the next project Clearing the backlog
• •
So, what’s the common
ground?

DEMO
The Bug’s Life

The bug stops here
Communicating Results
•
Are your statistics realistic?
•
Reinventing the wheel
•
Recognition
•

DEMO
We’re in this together, making sense of results

Remaining Work

Quality Indicators

Quality Indicators

Scope Creep

Reactivations

Bug Rates

Requirement Details

2010…
Bug to the Future
• Hierarchal work items
• Eliminating “no-
repro” bugs
• Test impact analysis
• Test prioritization
• Viewing the quality of
requirements and the
value of testing
• Reduce the manual
effort for automation-
capable tests

In summary
• The most important tool is communication
between dev and test.
• Quality belongs to everyone, as does lack
of.
• Automation is a good friend to have.
• Respect for your work, your colleagues,
your customers.

Q&A
Joe Shirey/Steven Lange
joe.shirey@microsoft.com/stevenl@microsoft.com