Great! We found a bug!
Pascal Van Cauwenberghe

Hello Consults Programs Manages projects
Creates Games Organises conferences NAYIMA We make play work www.nayima.be blog.nayima.be

SAFETY WARNING

Safety Warning • This presentation
contains code – No guarantee that it actually compiles • This presentation may contain bugs – Avoid if suffering from entomophobia

Based on true stories •
In different countries • In different projects • With different teams • Using different (programming) languages

Great! Another bug! Or how
to spend all day with the whole team to fix a tiny bug

Mummy.... Where do bugs come
from?

Well.... Once upon a time
there was a project

Context • Global company •
Large public web application • Mission critical, 24/7 • Major revenue source for the company • First agile project for this team • I coach the team • We need to modify and extend a small section of the application

The project
The application
US

Scene 1: a developer spots
a bug

How do you react when
someone says “I’ve found a bug” ?

MERCI!
Thank You!

11 steps to perfect code
1. SPOT A PROBLEM 2. THANK THE REPORTER 3. ...

MERCI!
Thank You!

Scene 2: What do we
do now?

THIS IS NOT A BUG

THIS IS AN OPPORTUNITY

TIPS • Never BLAME •
No need to find the “guilty” • Watch your language: – “Exercise”, “Experiment”, “Learn”, “Improve” – “We”, “Our code”, “Our problem” • “Solution Focus” approach

Scene 3: The team performs
a Root Cause Analysis 5 developers + architect

Business rule: “The customer can
ask for a refund until delivery”

Who can spot the problem?
boolean refundAllowed(Product product) { Datetime now = Datetime.now; Datetime final = Datetime.parse("yyyy-MM-dd“, product.deliveryAt()); return now <= final ; }

MERCI!
Thank You!

Let’s test the application •
Delivery on 27/10/2012 at 16:00 • It is now 26/10/2012 10:20 • 26/10/2012 10:20 <= 27/10/2012 00:00 ?

Let’s test the application •
Delivery on 27/10/2012 at 16:00 • It is now 26/10/2012 10:20 • 26/10/2012 10:20 <= 27/10/2012 00:00 ? Refund? YES

Let’s test the application •
Delivery on 26/10/2012 at 16:00 • It is now 26/10/2012 10:20 • 26/10/2012 10:20 <= 26/10/2012 00:00 ?

Let’s test the application •
Delivery on 26/10/2012 at 16:00 • It is now 26/10/2012 10:20 • 26/10/2012 10:20 <= 26/10/2012 00:00 ? Refund? NO

11 steps to perfect code
1. SPOT A PROBLEM 2. THANK THE REPORTER 3. REPRODUCE THE PROBLEM 4. ...

Can we fix the bug?
The fix is really simple!

11 steps to perfect code
1. SPOT A PROBLEM 2. THANK THE REPORTER 3. REPRODUCE THE PROBLEM 4. ADD (AT LEAST) ONE FAILING TEST 5. ...

A new test void testRefundGivenWhenDeliveryLaterToday()
{ Datetime now = DateTime.now ; String endofDay = now.strftime(“yyyy-MM-dd") + " 23:59" ; Product product = ... product.deliveryAt(endOfDay) ; BusinessRule businessRule = ... bool refund = businessRule.refundAllowed(product); assertTrue(refund,“Refund allowed until delivery, even on the day of delivery") ; }

A new test void testRefundGivenWhenDeliveryLaterToday()
{ Datetime now = DateTime.now ; String endofDay = now.strftime(“yyyy-MM-dd") + " 23:59" ; Product product = ... product.deliveryAt(endOfDay) ; BusinessRule businessRule = ... bool refund = businessRule.refundAllowed(product); assertTrue(refund,“Refund allowed until delivery, even on the day of delivery") ; }

Can we fix the bug?
The fix is really simple!

We finally fix the bug
boolean refundAllowed(Product product) { Datetime now = Datetime.now; Datetime final = Datetime.parse("yyyy-MM-dd HH:MM”, product.deliveryAt()); return now <= final ; }

Rerun the test void testRefundGivenWhenDeliveryLaterToday()
{ Datetime now = DateTime.now ; String endofDay = now.strftime(“yyyy-MM-dd") + " 23:59" ; Product product = ... product.deliveryAt(endOfDay) ; BusinessRule businessRule = ... bool refund = businessRule.refundAllowed(product); assertTrue(refund,“Refund allowed until delivery, even on the day of delivery") ; }

11 steps to perfect code
1. SPOT A PROBLEM 2. THANK THE REPORTER 3. REPRODUCE THE PROBLEM 4. ADD (AT LEAST) ONE FAILING TEST 5. CORRECT THE PROBLEM 6. AND RERUN ALL TESTS UNTIL THEY PASS 7. ...

“This agile thing is going
to be very slow if we do this for every bug!” “Yes, but at least now we have some confidence that the bug is fixed correctly and that we haven’t broken anything else.”

“We’ve really applied our team
value of ‘quality without compromise’. We took the time to do it right, even if no customer has complained yet.”

“We should contact the team
that’s responsible for this code and inform them of our correction.” “There may have been customer complaints already. We should contact the customer support people to see if there are complaints and inform them of the existing problem”

TIPS • Define and make
the team values visible at all times • Keep a list of things to do after the Root Cause Analysis (RCA) • Language: – “We should...” => “We will...”

WE WILL... • Contact the
development team responsible for this code • Inform the customer support team

11 steps to perfect code
1. SPOT A PROBLEM 2. THANK THE REPORTER 3. REPRODUCE THE PROBLEM 4. ADD (AT LEAST) ONE FAILING TEST 5. CORRECT THE PROBLEM 6. AND RERUN ALL TESTS UNTIL THEY PASS 7. PERFORM THE ACTIONS OF THE RCA 8. ...

“GOOD JOB!” “Now, back to
work!”

THE END And they lived
happily ever after...

It’s not over yet!

Scene 4: After the fix
The tester joins the developers and architect

11 steps to perfect code
1. SPOT A PROBLEM 2. THANK THE REPORTER 3. REPRODUCE THE PROBLEM 4. ADD (AT LEAST) ONE FAILING TEST 5. CORRECT THE PROBLEM 6. AND RERUN ALL TESTS UNTIL THEY PASS 7. PERFORM THE ACTIONS OF THE RCA 8. IMPROVE YOUR TESTS 9. ...

How can we improve our
tests? We should have found this bug sooner

How can we improve our
tests? We should have will fouind this type of bug sooner

How can we improve our
tests? We will find this type of bug sooner

Context (2) • Almost 80%
code coverage with automated unit tests in the whole application • This module has 100% test coverage • But still contains (at least) one bug • Why? • Let’s look at the tests...

Who can spot the problem?
void testRefundGiven() { Product product = ... product.deliveryAt(“2050-12-31 19:00”) ; BusinessRule businessRule = ... bool allowed = businessRule.refundAllowed(product) }

MERCI!
Thank You!

How can we improve our
tests? • There’s no ASSERT ! – It’s easy to get 100% code coverage that way • Why 2050? – What happens on January 1st 2051? • We need at least 4 tests – Delivery before today – Delivery today before now – Delivery today after now – Delivery after today

Why? • Developers don’t know
how to test date and time (and timezone!)-related code – Lots of tests use dates in 2050 • Few developers with unit testing experience • Agile coaching in the past did not include technical practices

We will... • Contact the
development team responsible for this code • Inform the customer support team • Add more tests about dates that can serve as examples • Show our example tests to other teams

Developers: “we have a lot
to learn about unit testing” Architect: “I’ve always had doubts about our automated tests. Now I know why.” Tester: “If you want, I can help you to define and improve tests”

Let’s dig deeper... Why tests
without ASSERT? • Goal: “increase code coverage of automated tests” instead of “increase quality” • Pressure to “do more with less (money and time)” • Not much experience with unit testing • TEST LAST development instead of Test First • No training or coaching on technical practices • Not all Agile coaches in the company have experience with technical practices

We will... • Contact the
development team responsible for this code • Inform the customer support team • Add more tests about dates that can serve as examples • Show our example tests to other teams • Apply TDD by pairing with the coach and tester • Raise « lack of technical practice coaching » risk at coach meeting

Systems Thinking

11 steps to perfect code
1. SPOT A PROBLEM 2. THANK THE REPORTER 3. REPRODUCE THE PROBLEM 4. ADD (AT LEAST) ONE FAILING TEST 5. CORRECT THE PROBLEM 6. AND RERUN ALL TESTS UNTIL THEY PASS 7. PERFORM THE ACTIONS OF THE RCA 8. IMPROVE YOUR TESTS 9. IMPROVE THE WAY YOU WRITE TESTS 10. ...

The End They lived happily
ever after... “We have a lot to learn”

It’s not over yet!

Scene 5: After the tests
A bug never travels alone...

Is this bug unique?

Let’s search the code... •
We find 10 cases where this datetime is parsed – 5 times with “yyyy-MM-dd HH:MM” – 5 times with “yyyy-MM-dd” • Each developer analyses one case • Result: two more bugs

What do you say?

MERCI!
Thank You!

And again...

MERCI!
Thank You!

11 steps to perfect code
1. SPOT A PROBLEM 2. THANK THE REPORTER 3. REPRODUCE THE PROBLEM 4. ADD (AT LEAST) ONE FAILING TEST 5. CORRECT THE PROBLEM 6. AND RERUN ALL TESTS UNTIL THEY PASS 7. PERFORM THE ACTIONS OF THE RCA 8. IMPROVE YOUR TESTS 9. IMPROVE THE WAY YOU WRITE TESTS 10. LOOK FOR SIMILAR PROBLEMS. GOTO 2 11. ...

“Quality without compromise”. Easy to
say, hard to do “Writing tests and correcting problems is starting to become routine. It goes faster each time we do it.”

Finally, The End? Can we
live happily ever after now?

It’s not DONE !

Did you spot another problem?

MERCI!
Thank You!

Digging deeper... Why did we
introduce the bug? • We aren’t aware that there’s a date+time • We have 10x the same parsing code • => 10 opportunities to make a mistake • Let’s remove the duplication that leads to mistakes

Improving Product class Product {
... // Deprecated: Remove when obsolete String deliveryAt() ; // New: gradually refactor all clients DateTime deliveryAtDateTime() ; ... } From “stringly typed” to “strongly typed”

11 steps to perfect code
1. SPOT A PROBLEM 2. THANK THE REPORTER 3. REPRODUCE THE PROBLEM 4. ADD (AT LEAST) ONE FAILING TEST 5. CORRECT THE PROBLEM 6. AND RERUN ALL TESTS UNTIL THEY PASS 7. PERFORM THE ACTIONS OF THE RCA 8. IMPROVE YOUR TESTS 9. IMPROVE THE WAY YOU WRITE TESTS 10. LOOK FOR SIMILAR PROBLEMS. GOTO 2 11. MAKE THIS TYPE OF PROBLEM IMPOSSIBLE

TIP • If you think
you have to comment your code, think again

With comments class A {
void methodA() ; // Call methodA before calling these void methodB() ; void methodC() ; } ... a.methodB() ; // ERROR a.methodA() ; // ERROR!!

Without comment class A {
B methodA() ; } class B { void methodB() ; void methodC() ; } a.methodA().methodB() ;

Mummy.... Where do strings come
from?

The application ABCDEFGHIJKLMNO ABCDEFGHIJKLMNO System
A ABCDEFGHIJKLMNO ABCDEFGHIJKLMNO Application ABCDEFGHIJKLMNO ABCDEFGHIJKLMNO ABCDEFGHIJKLMNO ABCDEFGHIJKLMNO System B

The application(vision) ABCDEFGHIJKLMNO System A
L’application ABCDEFGHIJKLMNO System B

We will... • Contact the
development team responsible for this code • Inform the customer support team • Add more tests about dates that can serve as examples • Show our example tests to other teams • Apply TDD by pairing with the coach and tester • Raise « lack of technical practice coaching » risk at coach meeting • Encapsulate data coming from outside

LARGE REFACTORING

A3 proposal

A3 Proposal Problem description BEFORE
AFTER STEPS: Visualisation 1. .jdlkjds 2. Kmlkdmlkd 3. Dkqjdlkjds 4. Sqkjlkdlksqmk 5. BEER !

TIPS • Keep the A3
visible during the whole refactoring • Put the A3 where it can’t be missed • Limit the number of A3s that can be published

Scene 6:
Final act

Looking back 1. 7 team
members + one coach have spent all afternoon to correct a 6-character bug? Is this reasonable? 2. In the end, there were a lot less bugs found in test than on “normal” projects. Is there a link? 3. This project was delivered in 5 months instead of the 8 that were estimated. How can you finish faster by going slower?

Theory of Constraints in a
nutshell

How much do they deliver?
Test & Analysis: Dev: 20 10 Deploy: 15 ???

How much do they deliver?
Test & Analysis: Dev: 20 10 Deploy: 15 <= 10

How much do they deliver?
Test & Analysis: Dev: 20 10 Deploy: 15 ??? Analysis: Dev: 10 12

How much do they deliver?
Test & Analysis: Dev: 20 10 Deploy: 15 <= 15 Analysis: Dev: 10 12

What if development speeds up?
Test & Analysis: Dev: 20 20 Deploy: 15 ??? Analysis: Dev: 10 12

Nothing happens? Test & Analysis:
Dev: 20 20 Deploy: 15 <= 15 Analysis: Dev: 10 12

Buffers start to fill up!
Test & Analysis: Dev: 20 20 Deploy: 15 < 15 Analysis: Dev: 10 12

How much do they deliver?
Test & Analysis: Dev: 20 10 Deploy: 15 <= 15 Analysis: Dev: 10 12

What if we go slower?
Test & Analysis: Dev: 20 8 Deploy: 17 <= 17 Analysis: Dev: 10 12

Why are we analysing so
fast? Test & Analysis: Dev: 20 8 Deploy: 17 <= 17 Analysis: Dev: 10 12

We do less Test &
Dev: <= 17 Analysis: 10 Deploy: 8 17 Analysis: Dev: 10 12

Play The Bottleneck Game

Reading tips

How much does quality cost?

TIPS • Once you run
out of reported bugs... Apply IDD™ Irritation Driven Development Contact me if you want to become a Certifiable Irritating Person

TIPS • Pair with the
bottleneck – ToC calls this “subordinating” • Walk a mile in their shoes – Carefully observe and note irritations – Quietly remove the irritations • Example: pair-installing dev-ops

Summary

11 steps to a life
with less stress 1. SPOT A PROBLEM 2. THANK THE REPORTER 3. REPRODUCE THE PROBLEM 4. ADD (AT LEAST) ONE FAILING TEST 5. CORRECT THE PROBLEM 6. AND RERUN ALL TESTS UNTIL THEY PASS 7. PERFORM THE ACTIONS OF THE RCA 8. IMPROVE YOUR TESTS 9. IMPROVE THE WAY YOU WRITE TESTS 10. LOOK FOR SIMILAR PROBLEMS. GOTO 2 11. MAKE THIS TYPE OF PROBLEM IMPOSSIBLE

But, most importantly...

MERCI!
Thank You!

The End And they lived
happily ever after

SESSION FEEDBACK

MERCI!
Thank You!

Thank you, Dank u, Merci,
Danke, Tak, Kiitos, Gracias, Grazie, Tack, Obrigado Consults Programs Manages projects Creates Games Organises conferences NAYIMA We make play work http://www.nayima.be http://www.agilecoach.net