Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Why$Your$Test$Suite$Sucks
(and%what%you%can%do%about%it)
by#Ciaran&McNulty#at#PHPUK&2015
It’s%not%working%for%me
===
TDD#must#suck1
1
"BIT.LY/1LEF0GI"&"BIT.LY/1SLFGTQ"
The$problem$is$not$TDD
The$problem$is$your%suite
Value!=!Benefits!)!Costs
Suites'suck'when'you'aren’t'ge)ng'
all'the'benefits'of'TDD
Suites'suck'when'the'tests'are$hard$
to$maintain
Reason'1:
You$don't$have$a$test$suite
The$Evolu*on$of$a$TDD$Team
Maturity(model
• Manual"tes&ng
• Automated"tes&ng
• Test.first"development
• Test.driven"development
Manual&tes*ng
• Design"an"implementa+on
• Implement"that"design
• Manually"test
Growing(from(Manual(to(Automated
• “When'we'make%changes,'it'breaks'things!”
• “We'spend'a%lot%of%/me'on'manual'tes:ng”
• ...
Automated)tes+ng
• Design"an"implementa+on
• Implement"that"design
• Write.tests"to"verify"it
Growing(from(Automated(to(Test3first
• “It’s'a'pain'having'to'write'tests'once'I’m'done”
• “My'test'suite'is'bri'le”
• “Mai...
Test%first(development
• Design"an"implementa+on
• Write*tests"to"verify"that"implementa+on
• Implement"that"design
Growing(from(Test/first(to(Test/driven
• “Test'suite'is's"ll'bri,le”
• “Maintaining'the'suite'is's"ll'hard”'
• ”TDD'Doesn’t...
Test%driven+development
• Write&tests"that"describe"behaviour
• Implement"a"system"that"has"that"behaviour
Problems:
*+“I+...
Reason'2:
You$are$tes*ng$implementa*on
Problem:)Mocking)Queries
• A#query#is#a#method#that#returns'a'value'without'side'effects#
• We#shouldn’t'care#how#many#4mes...
function testItGreetsPeopleByName()
{
$user = $this->getMock('User');
$user->expects($this->once())
->method('getName')
->...
function testItGreetsPeopleByName()
{
$user = $this->getMock('User');
$user->method('getName')
->willReturn('Ciaran');
$gr...
Don’t&mock&queries,"just"use"stubs
Do#you#really#care#how#many#/mes#it's#called?
Problem:)Tes,ng)the)sequence)of)calls
• Most&of&the&*me&we&don’t&care&what%order%calls%are%made%in
• Tes*ng&the&sequence&m...
public function testBasketTotals()
{
$priceList = $this->getMock('PriceList');
$priceList->expect($this->at(0))
->method('...
public function testBasketTotals()
{
$priceList = $this->getMock('PriceList');
$priceList->method('getPrices')
->will($thi...
Test%behaviour%you%care%about,"
don’t"test"implementa/on
PROTIP:!The!best!way!to!do!this!is!to!not!
have!an!implementa3on!...
Reason'3:
Because'your'design'sucks
Problem:)Too)many)Doubles
• It$is$painful$to$have$to$double,lots,of,objects$in$your$test
• It$is$a$signal$your$object,has,...
public function setUp()
{
$chargerules = $this->getMock('ChargeRules');
$chargetypes = $this->getMock('ChargeTypes');
$not...
class InvoiceProcessor
{
public function __construct(
ChargeRules $chargeRules,
ChargeTypes $chargeTypes,
Notifier $notifi...
Too#many#doubles!in!your!test!mean!
your!class!has!too#many#
dependencies
Problem:)Stubs)returning)stubs
• A#related#painful#issue#is#stubs%returning%a%stubs
• It#is#a#signal#our#object#does%not%h...
public function it_notifies_the_user(
User $user,
Contact $contact,
Email $email,
Emailer $emailer
)
{
$this->beConstructe...
class Notifier
{
public function __construct(Emailer $emailer)
{
// …
}
public function notify(User $user)
{
$email = $use...
class Notifier
{
public function __construct(Emailer $emailer)
{
// …
}
public function notify(User $user)
{
$this->emaile...
class Notifier
{
public function __construct(Emailer $emailer)
{
// …
}
public function notify(Email $email)
{
$this->emai...
public function it_notifies_the_user(
Email $email,
Emailer $emailer
)
{
$this->beConstructedWith($emailer);
$this->notify...
Stubs&returning&stubs&indicate&
dependencies(are(not(correctly(
defined
Problem:)Par+ally)doubling)the)object)under)
test
• A#common%ques*on#is#“how#can#I#mock#methods#on#the#SUT?”#
• Easy#to#do...
class Form
{
public function handle($data)
{
if ($this->validate($data)) {
return 'Valid';
}
return 'Invalid';
}
protected...
function testItOutputsMessageOnValidData()
{
$form = $this->getMock('Form', ['validate']);
$form->method('validate')->will...
class Form
{
protected function __construct(Validator $validator)
{
// …
}
public function handle($data)
{
if ($this->vali...
function testItOutputsMessageOnValidData()
{
$validator = $this->getMock(‘Validator’);
$validator->method('validate')->wil...
If#you#want#to#mock#part#of#the#SUT#
it#means#you$should$really$have$
more$than$one$object
How$to$be$a$be)er$so,ware$designer:
• Stage&1:#Think#about#your#code#before#you#write#it#
• Stage&2:#Write#down#those#thou...
How$long$will$it$take$without$TDD?
• 1"hour"thinking"about"how"it"should"work"
• 30"mins"Implemen6ng"it
• 30"mins"Checking...
How$long$will$it$take$with$TDD?
• 1"hour"Thinking"in"code"about"how"it"should"work
• 30"mins"Implemen8ng"it
• 1"min"Checki...
Use$your$test$suite$as$a$way$to$
reason'about'the'code
Customers)should)not)dictate)your)
design'process
Reason'4:
You$are$unit$tes,ng$across$domain$
boundaries
Problem:)Tes,ng)3rd)party)code
• When&we&extend&a&library&or&framework&we&use$other$people’s$
code
• To&test&our&code&we&e...
class UserRepository extends FrameworkRepository
{
public function findByEmail($email)
{
$this->find(['email' => $email]);...
public function it_finds_the_user_by_email(
FrameworkDatabaseConnection $db,
FrameworkSchemaCreator $schemaCreator,
Framew...
class UserRepository
{
private $repository;
public function __construct(FrameworkRepository $repository)
{
$this->reposito...
public function it_finds_the_user_by_email(
Repository $repository,
User $user
)
{
$this->beConstructedWith($repository);
...
When%you%extend%third%party%code,%
you%take%responsibility%for%its%
design%decisions
Favour'composi'on'over'
inheritance
Problem:)Doubling)3rd)party)code
• Test&Doubles&(Mocks,&Stubs,&Fakes)&are&used&to&test&
communica(on&between&objects&
• Te...
class FileHandlerSpec extends ObjectBehaviour
{
public function it_uploads_data_to_the_cloud_when_valid(
CloudApi $client,...
Coupled(architecture
Coupled(architecture
Layered'architecture
Tes$ng'layered'architecture
class FileHandlerSpec extends ObjectBehaviour
{
public function it_uploads_data_to_the_cloud_when_valid(
FileStore $filest...
Tes$ng'layered'architecture
class CloudFilestoreTest extends PHPUnit_Framework_TestCase
{
function testItStoresFiles()
{
$testCredentials = …
$file = ...
Tes$ng'layered'architecture
Don’t&test&other&people’s&code
You$don't$know$how$it's$supposed$to$behave
Don’t&test&how&you&talk&to&other&
people’s&code
You$don't$know$how$it's$supposed$to$respond
Pu#ng&it&all&together
• Use%TDD%to%drive&development
• Use%TDD%to%replace&exis1ng&prac1ces,%not%as%a%new%extra%task%
Pu#ng&it&all&together
Don’t&ignore&tests&that&suck,&improve&them:&
• Is$the$test$too$!ed$to$implementa!on?
• Does$the$desi...
Image&credits
• Human&Evolu+on"by"Tkgd2007
h-p://bit.ly/1DdLvy1"(CC"BY=SA"3.0)
• Oops"by"Steve"and"Sara"Emry"
h-ps://flic.k...
Thank&you!
h"ps://joind.in/13393
h"ps://github.com/ciaranmcnulty
h"p://www.slideshare.net/CiaranMcNulty
@ciaranmcnulty
Upcoming SlideShare
Loading in …5
×

Why Your Test Suite Sucks

2,909 views

Published on

Many teams adopt TDD attracted by the promise of a more productive workflow, fewer regressions and higher code quality. Sometimes this goes wrong and these benefits do not materialise, despite a healthy-seeming test suite. In this talk we will look at what the common pitfalls of testing are, why teams fall into these traps, and they can dig themselves out.

Published in: Software

Why Your Test Suite Sucks

  1. 1. Why$Your$Test$Suite$Sucks (and%what%you%can%do%about%it) by#Ciaran&McNulty#at#PHPUK&2015
  2. 2. It’s%not%working%for%me === TDD#must#suck1 1 "BIT.LY/1LEF0GI"&"BIT.LY/1SLFGTQ"
  3. 3. The$problem$is$not$TDD
  4. 4. The$problem$is$your%suite
  5. 5. Value!=!Benefits!)!Costs
  6. 6. Suites'suck'when'you'aren’t'ge)ng' all'the'benefits'of'TDD
  7. 7. Suites'suck'when'the'tests'are$hard$ to$maintain
  8. 8. Reason'1: You$don't$have$a$test$suite
  9. 9. The$Evolu*on$of$a$TDD$Team
  10. 10. Maturity(model • Manual"tes&ng • Automated"tes&ng • Test.first"development • Test.driven"development
  11. 11. Manual&tes*ng • Design"an"implementa+on • Implement"that"design • Manually"test
  12. 12. Growing(from(Manual(to(Automated • “When'we'make%changes,'it'breaks'things!” • “We'spend'a%lot%of%/me'on'manual'tes:ng” • “Bugs'keep'making'it'into'produc:on” Enabled(by: +(Tutorials,(training +(Team(policies
  13. 13. Automated)tes+ng • Design"an"implementa+on • Implement"that"design • Write.tests"to"verify"it
  14. 14. Growing(from(Automated(to(Test3first • “It’s'a'pain'having'to'write'tests'once'I’m'done” • “My'test'suite'is'bri'le” • “Maintaining'the'suite'is'hard”' • ”TDD/Doesn’t/Work!” Supported)by: -)Coaching,)pairing -)Leadership
  15. 15. Test%first(development • Design"an"implementa+on • Write*tests"to"verify"that"implementa+on • Implement"that"design
  16. 16. Growing(from(Test/first(to(Test/driven • “Test'suite'is's"ll'bri,le” • “Maintaining'the'suite'is's"ll'hard”' • ”TDD'Doesn’t'Work!” Supported)by: -)Prac1ce
  17. 17. Test%driven+development • Write&tests"that"describe"behaviour • Implement"a"system"that"has"that"behaviour Problems: *+“I+don’t+know+what+to+do+with+all+this+spare&'me!” *+“Where+should+we+keep+this+extra&money?”
  18. 18. Reason'2: You$are$tes*ng$implementa*on
  19. 19. Problem:)Mocking)Queries • A#query#is#a#method#that#returns'a'value'without'side'effects# • We#shouldn’t'care#how#many#4mes#(if#any)#a#query#is#called • Specifying#that#detail#makes'change'harder
  20. 20. function testItGreetsPeopleByName() { $user = $this->getMock('User'); $user->expects($this->once()) ->method('getName') ->willReturn('Ciaran'); $greeting = $this->greeter->greet($user); $this->assertEquals('Hello, Ciaran!', $greeting); }
  21. 21. function testItGreetsPeopleByName() { $user = $this->getMock('User'); $user->method('getName') ->willReturn('Ciaran'); $greeting = $this->greeter->greet($user); $this->assertEquals('Hello, Ciaran!', $greeting); }
  22. 22. Don’t&mock&queries,"just"use"stubs Do#you#really#care#how#many#/mes#it's#called?
  23. 23. Problem:)Tes,ng)the)sequence)of)calls • Most&of&the&*me&we&don’t&care&what%order%calls%are%made%in • Tes*ng&the&sequence&makes&it&hard%to%change
  24. 24. public function testBasketTotals() { $priceList = $this->getMock('PriceList'); $priceList->expect($this->at(0)) ->method('getPrices') ->willReturn(120); $priceList->expect($this->at(1)) ->method('getPrices') ->willReturn(200); $this->basket->add('Milk', 'Bread'); $total = $this->basket->calculateTotal(); $this->assertEqual(320, $total); }
  25. 25. public function testBasketTotals() { $priceList = $this->getMock('PriceList'); $priceList->method('getPrices') ->will($this->returnValueMap([ ['Milk', 120], ['Bread', 200] ])); $this->basket->add('Milk', 'Bread'); $total = $this->basket->calculateTotal(); $this->assertEqual(320, $total); }
  26. 26. Test%behaviour%you%care%about," don’t"test"implementa/on PROTIP:!The!best!way!to!do!this!is!to!not! have!an!implementa3on!yet
  27. 27. Reason'3: Because'your'design'sucks
  28. 28. Problem:)Too)many)Doubles • It$is$painful$to$have$to$double,lots,of,objects$in$your$test • It$is$a$signal$your$object,has,too,many,dependencies$
  29. 29. public function setUp() { $chargerules = $this->getMock('ChargeRules'); $chargetypes = $this->getMock('ChargeTypes'); $notifier = $this->getMockBuilder('Notifier') ->disableOriginalConstructor()->getMock(); $this->processor = new InvoiceProcessor( $chargerules, $chargetypes, $notifier ); }
  30. 30. class InvoiceProcessor { public function __construct( ChargeRules $chargeRules, ChargeTypes $chargeTypes, Notifier $notifier ) { $this->chargeRules = $chargeRules; $this->chargeTypes = $chargeTypes; $this->notifier = $notifier; } // …
  31. 31. Too#many#doubles!in!your!test!mean! your!class!has!too#many# dependencies
  32. 32. Problem:)Stubs)returning)stubs • A#related#painful#issue#is#stubs%returning%a%stubs • It#is#a#signal#our#object#does%not%have%the%right%dependencies
  33. 33. public function it_notifies_the_user( User $user, Contact $contact, Email $email, Emailer $emailer ) { $this->beConstructedWith($emailer); $user->getContact()->willReturn($contact); $contact->getEmail()->willReturn($email); $this->notify($user); $emailer->sendTo($email)->shouldHaveBeenCalled(); }
  34. 34. class Notifier { public function __construct(Emailer $emailer) { // … } public function notify(User $user) { $email = $user->getContact()->getEmail() $this->emailer->sendTo($email); } }
  35. 35. class Notifier { public function __construct(Emailer $emailer) { // … } public function notify(User $user) { $this->emailer->sendTo($user); } }
  36. 36. class Notifier { public function __construct(Emailer $emailer) { // … } public function notify(Email $email) { $this->emailer->sendTo($email); } }
  37. 37. public function it_notifies_the_user( Email $email, Emailer $emailer ) { $this->beConstructedWith($emailer); $this->notify($email); $emailer->sendTo($email)->shouldHaveBeenCalled(); }
  38. 38. Stubs&returning&stubs&indicate& dependencies(are(not(correctly( defined
  39. 39. Problem:)Par+ally)doubling)the)object)under) test • A#common%ques*on#is#“how#can#I#mock#methods#on#the#SUT?”# • Easy#to#do#in#some#tools,#impossible%in%others • It#is#a#signal#our%object%has%too%many%responsibili*es
  40. 40. class Form { public function handle($data) { if ($this->validate($data)) { return 'Valid'; } return 'Invalid'; } protected function validate($data) { // do something complex } }
  41. 41. function testItOutputsMessageOnValidData() { $form = $this->getMock('Form', ['validate']); $form->method('validate')->willReturn(true); $result = $form->handle([]); $this->assertSame('Valid', $result); }
  42. 42. class Form { protected function __construct(Validator $validator) { // … } public function handle($data) { if ($this->validator->validate($data)) { return 'Valid'; } return 'Invalid'; } }
  43. 43. function testItOutputsMessageOnValidData() { $validator = $this->getMock(‘Validator’); $validator->method('validate')->willReturn(true); $form = new Form($validator); $result = $form->handle([]); $this->assertSame('Valid', $result); }
  44. 44. If#you#want#to#mock#part#of#the#SUT# it#means#you$should$really$have$ more$than$one$object
  45. 45. How$to$be$a$be)er$so,ware$designer: • Stage&1:#Think#about#your#code#before#you#write#it# • Stage&2:#Write#down#those#thoughts • Stage&3:#Write#down#those#thoughts#as#code#
  46. 46. How$long$will$it$take$without$TDD? • 1"hour"thinking"about"how"it"should"work" • 30"mins"Implemen6ng"it • 30"mins"Checking"it"works"
  47. 47. How$long$will$it$take$with$TDD? • 1"hour"Thinking"in"code"about"how"it"should"work • 30"mins"Implemen8ng"it • 1"min"Checking"it"works"
  48. 48. Use$your$test$suite$as$a$way$to$ reason'about'the'code
  49. 49. Customers)should)not)dictate)your) design'process
  50. 50. Reason'4: You$are$unit$tes,ng$across$domain$ boundaries
  51. 51. Problem:)Tes,ng)3rd)party)code • When&we&extend&a&library&or&framework&we&use$other$people’s$ code • To&test&our&code&we&end&up&tes.ng$their$code&too • This&:me&is&wasted&–&we&should&trust&their&code&works&
  52. 52. class UserRepository extends FrameworkRepository { public function findByEmail($email) { $this->find(['email' => $email]); } }
  53. 53. public function it_finds_the_user_by_email( FrameworkDatabaseConnection $db, FrameworkSchemaCreator $schemaCreator, FrameworkSchema $schema, User $user ) { $this->beConstructedWith($db, $schemaCreator); $schemaCreator->getSchema()->willReturn($schema); $schema->getMappings()->willReturn('email'=>'email'); $user = $this->findByEmail('bob@example.com'); $user->shouldHaveType('User'); }
  54. 54. class UserRepository { private $repository; public function __construct(FrameworkRepository $repository) { $this->repository = $repository; } public function findByEmail($email) { return $this->repository->find(['email' => $email]); } }
  55. 55. public function it_finds_the_user_by_email( Repository $repository, User $user ) { $this->beConstructedWith($repository); $repository->find(['email' => 'bob@example.com') ->willReturn($user); $this->findByEmail('bob@example.com'); ->shouldEqual($user); }
  56. 56. When%you%extend%third%party%code,% you%take%responsibility%for%its% design%decisions
  57. 57. Favour'composi'on'over' inheritance
  58. 58. Problem:)Doubling)3rd)party)code • Test&Doubles&(Mocks,&Stubs,&Fakes)&are&used&to&test& communica(on&between&objects& • Tes:ng&against&a&Double&of&a&3rd&party&object&does,not,give,us, confidence&that&we&have&described&the&communica:on&correctly
  59. 59. class FileHandlerSpec extends ObjectBehaviour { public function it_uploads_data_to_the_cloud_when_valid( CloudApi $client, FileValidator $validator, File $file ) { $this->beConstructedWith($client, $validator); $validator->validate($file)->willReturn(true); $client->startUpload()->shouldBeCalled(); $client->uploadData(Argument::any())->shouldBeCalled(); $client->uploadSuccessful()->willReturn(true); $this->process($file)->shouldReturn(true); } }
  60. 60. Coupled(architecture
  61. 61. Coupled(architecture
  62. 62. Layered'architecture
  63. 63. Tes$ng'layered'architecture
  64. 64. class FileHandlerSpec extends ObjectBehaviour { public function it_uploads_data_to_the_cloud_when_valid( FileStore $filestore, FileValidator $validator, File $file ) { $this->beConstructedWith($filestore, $validator); $validator->validate($file)->willReturn(true); $this->process($file); $filestore->store($file)->shouldHaveBeenCalled(); } }
  65. 65. Tes$ng'layered'architecture
  66. 66. class CloudFilestoreTest extends PHPUnit_Framework_TestCase { function testItStoresFiles() { $testCredentials = … $file = new File(…); $apiClient = new CloudApi($testCredentials); $filestore = new CloudFileStore($apiClient); $filestore->store($file); $this->assertTrue($apiClient->fileExists(…)); } }
  67. 67. Tes$ng'layered'architecture
  68. 68. Don’t&test&other&people’s&code You$don't$know$how$it's$supposed$to$behave
  69. 69. Don’t&test&how&you&talk&to&other& people’s&code You$don't$know$how$it's$supposed$to$respond
  70. 70. Pu#ng&it&all&together • Use%TDD%to%drive&development • Use%TDD%to%replace&exis1ng&prac1ces,%not%as%a%new%extra%task%
  71. 71. Pu#ng&it&all&together Don’t&ignore&tests&that&suck,&improve&them:& • Is$the$test$too$!ed$to$implementa!on? • Does$the$design$of$the$code$need$to$improve?$ • Are$you$tes!ng$across$domain$boundaries?$
  72. 72. Image&credits • Human&Evolu+on"by"Tkgd2007 h-p://bit.ly/1DdLvy1"(CC"BY=SA"3.0) • Oops"by"Steve"and"Sara"Emry" h-ps://flic.kr/p/9Z1EEd"(CC"BY=NC=SA"2.0) • Blueprints&for&Willborough&Home"by"Brian"Tobin" h-ps://flic.kr/p/7MP9RU"(CC"BY=NC=ND"2.0) • Border"by"Giulia"van"Pelt" h-ps://flic.kr/p/9YT1c5"(CC"BY=NC=ND"2.0)
  73. 73. Thank&you! h"ps://joind.in/13393 h"ps://github.com/ciaranmcnulty h"p://www.slideshare.net/CiaranMcNulty @ciaranmcnulty

×