Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Why Your Test Suite Sucks - PHPCon PL 2015

957 views

Published on

Many teams adopt TDD attracted by the promise of a more productive workflow, fewer regressions and higher code quality. Sometimes this goes wrong and these benefits do not materialise, despite a healthy-seeming test suite. In this talk we will look at what the common pitfalls of testing are, why teams fall into these traps, and they can dig themselves out.

Published in: Software

Why Your Test Suite Sucks - PHPCon PL 2015

  1. 1. Why Your Test Suite Sucks (and what you can do about it) by Ciaran McNulty at PHPCon Poland 2015
  2. 2. It’s not working for me === TDD must suck1 1 BIT.LY/1LEF0GI & BIT.LY/1SLFGTQ
  3. 3. The problem is not TDD
  4. 4. The problem is your suite
  5. 5. Value = Benefits - Costs
  6. 6. Suites suck when you aren’t ge)ng all the benefits of TDD
  7. 7. Suites suck when the tests are hard to maintain
  8. 8. Reason 1: You don't have a test suite
  9. 9. The Evolu*on of a TDD Team
  10. 10. Maturity model • Manual tes&ng • Automated tes&ng • Test-first development • Test-driven development
  11. 11. Manual tes*ng • Design an implementa+on • Implement that design • Manually test
  12. 12. Growing from Manual to Automated • “When we make changes, it breaks things!” • “We spend a lot of /me on manual tes:ng” • “Bugs keep making it into produc:on” Enabled by: - Tutorials, training - Team policies
  13. 13. Automated tes+ng • Design an implementa+on • Implement that design • Write tests to verify it
  14. 14. Growing from Automated to Test-first • “It’s a pain having to write tests once I’m done” • “My test suite is bri'le” • “Maintaining the suite is hard” • ”TDD Doesn’t Work!” Supported by: - Coaching, pairing - Leadership
  15. 15. Test-first development • Design an implementa+on • Write tests to verify that implementa+on • Implement that design
  16. 16. Growing from Test-first to Test-driven • “Test suite is s"ll bri,le” • “Maintaining the suite is s"ll hard” • ”TDD Doesn’t Work!” Supported by: - Prac1ce
  17. 17. Test-driven development • Write tests that describe behaviour • Implement a system that has that behaviour Problems: - “I don’t know what to do with all this spare 'me!” - “Where should we keep this extra money?”
  18. 18. Reason 2: You are tes*ng implementa*on
  19. 19. Problem: Mocking Queries • A query is a method that returns a value without side effects • We shouldn’t care how many 4mes (if any) a query is called • Specifying that detail makes change harder
  20. 20. function testItGreetsPeopleByName() { $user = $this->getMock('User'); $user->expects($this->once()) ->method('getName') ->willReturn('Ciaran'); $greeting = $this->greeter->greet($user); $this->assertEquals('Hello, Ciaran!', $greeting); }
  21. 21. function testItGreetsPeopleByName() { $user = $this->getMock('User'); $user->method('getName') ->willReturn('Ciaran'); $greeting = $this->greeter->greet($user); $this->assertEquals('Hello, Ciaran!', $greeting); }
  22. 22. Don’t mock queries, just use stubs Do you really care how many /mes it's called?
  23. 23. Problem: Tes,ng the sequence of calls • Most of the *me we don’t care what order calls are made in • Tes*ng the sequence makes it hard to change
  24. 24. public function testBasketTotals() { $priceList = $this->getMock('PriceList'); $priceList->expect($this->at(0)) ->method('getPrices') ->willReturn(120); $priceList->expect($this->at(1)) ->method('getPrices') ->willReturn(200); $this->basket->add('Milk', 'Bread'); $total = $this->basket->calculateTotal(); $this->assertEqual(320, $total); }
  25. 25. public function testBasketTotals() { $priceList = $this->getMock('PriceList'); $priceList->method('getPrices') ->will($this->returnValueMap([ ['Milk', 120], ['Bread', 200] ])); $this->basket->add('Milk', 'Bread'); $total = $this->basket->calculateTotal(); $this->assertEqual(320, $total); }
  26. 26. Test behaviour you care about, don’t test implementa/on PROTIP: The best way to do this is to not have an implementa3on yet
  27. 27. Reason 3: Because your design sucks
  28. 28. Problem: Too many Doubles • It is painful to have to double lots of objects in your test • It is a signal your object has too many dependencies
  29. 29. public function setUp() { $chargerules = $this->getMock('ChargeRules'); $chargetypes = $this->getMock('ChargeTypes'); $notifier = $this->getMockBuilder('Notifier') ->disableOriginalConstructor()->getMock(); $this->processor = new InvoiceProcessor( $chargerules, $chargetypes, $notifier ); }
  30. 30. class InvoiceProcessor { public function __construct( ChargeRules $chargeRules, ChargeTypes $chargeTypes, Notifier $notifier ) { $this->chargeRules = $chargeRules; $this->chargeTypes = $chargeTypes; $this->notifier = $notifier; } // …
  31. 31. Too many doubles in your test mean your class has too many dependencies
  32. 32. Problem: Stubs returning stubs • A related painful issue is stubs returning a stubs • It is a signal our object does not have the right dependencies
  33. 33. public function testItNotifiesTheUser() { $user = $this->getMock(User::class); $contact = $this->getMock(Contact::class); $email = $this->getMock(Email::class); $emailer = $this->getMock(Emailer::class); $user->method('getContact')->willReturn($contact); $contact->method('getEmail')->willReturn($email); $emailer->expects($this->once) ->method('sendTo') ->with($this->equalTo($email)); (new Notifier($emailer))->notify($user); }
  34. 34. class Notifier { public function __construct(Emailer $emailer) { // … } public function notify(User $user) { $email = $user->getContact()->getEmail() $this->emailer->sendTo($email); } }
  35. 35. class Notifier { public function __construct(Emailer $emailer) { // … } public function notify(User $user) { $this->emailer->sendTo($user); } }
  36. 36. class Notifier { public function __construct(Emailer $emailer) { // … } public function notify(Email $email) { $this->emailer->sendTo($email); } }
  37. 37. public function testItNotifiesTheUser() { $email = $this->getMock(Email::class); $emailer = $this->getMock(Emailer::class); $emailer->expects($this->once) ->method('sendTo') ->with($this->equalTo($email)); (new Notifier($emailer))->notify($user); }
  38. 38. Stubs returning stubs indicate dependencies are not correctly defined
  39. 39. Problem: Par+ally doubling the object under test • A common ques*on is “how can I mock methods on the SUT?” • Easy to do in some tools, impossible in others • It is a signal our object has too many responsibili*es
  40. 40. class Form { public function handle($data) { if ($this->validate($data)) { return 'Valid'; } return 'Invalid'; } protected function validate($data) { // do something complex } }
  41. 41. function testItOutputsMessageOnValidData() { $form = $this->getMock('Form', ['validate']); $form->method('validate')->willReturn(true); $result = $form->handle([]); $this->assertSame('Valid', $result); }
  42. 42. class Form { protected function __construct(Validator $validator) { // … } public function handle($data) { if ($this->validator->validate($data)) { return 'Valid'; } return 'Invalid'; } }
  43. 43. function testItOutputsMessageOnValidData() { $validator = $this->getMock(‘Validator’); $validator->method('validate')->willReturn(true); $form = new Form($validator); $result = $form->handle([]); $this->assertSame('Valid', $result); }
  44. 44. If you want to mock part of the SUT it means you should really have more than one object
  45. 45. How to be a be)er so,ware designer: • Stage 1: Think about your code before you write it • Stage 2: Write down those thoughts • Stage 3: Write down those thoughts as code
  46. 46. How long will it take without TDD? • 1 hour thinking about how it should work • 30 mins Implemen6ng it • 30 mins Checking it works
  47. 47. How long will it take with TDD? • 1 hour Thinking in code about how it should work • 30 mins Implemen8ng it • 1 min Checking it works
  48. 48. Use your test suite as a way to reason about the code
  49. 49. Customers should not dictate your design process
  50. 50. Reason 4: You are unit tes,ng across domain boundaries
  51. 51. Problem: Tes,ng 3rd party code • When we extend a library or framework we use other people’s code • To test our code we end up tes.ng their code too • This :me is wasted – we should trust their code works
  52. 52. class UserRepository extends FrameworkRepository { public function findByEmail($email) { $this->find(['email' => $email]); } }
  53. 53. public function testItFindsTheUserByEmail() { $db = $this->getMock(FrameworkDatabaseConnection::class); $schemaCreator = $this->getMock(FrameworkSchemaCreator::class); $schema = $this->getMock(FrameworkSchema::class); $schemaCreator->method('getSchema')->willReturn($schema); $schema->method('getMappings')->willReturn(['email'=>'email']); $db->method('query')->willReturn(['email'=>'bob@example.com'); $repo = new UserRepository($db, $schemaCreator); $user = $repo->findByEmail('bob@example.com'); $this->assertType(User::class, $user); }
  54. 54. class UserRepository { private $repository; public function __construct(FrameworkRepository $repository) { $this->repository = $repository; } public function findByEmail($email) { return $this->repository->find(['email' => $email]); } }
  55. 55. public function testItFindsTheUserByEmail() { $fwRepo = $this->getMock(FrameworkRepository::class); $user = $this->getMock(User::class); $repository = new UserRepository($fwRepo); $fwRepo->method('find') ->with(['email' => 'bob@example.com') ->willReturn($user); $actualUser = $repository->findByEmail('bob@example.com'); $this->assertEqual($user, $actualUser); }
  56. 56. When you extend third party code, you take responsibility for its design decisions
  57. 57. Favour composi'on over inheritance
  58. 58. Problem: Doubling 3rd party code • Test Doubles (Mocks, Stubs, Fakes) are used to test communica(on between objects • Tes:ng against a Double of a 3rd party object does not give us confidence that we have described the communica:on correctly
  59. 59. public function testItUploadsValidDataToTheCloud() { $validator = $this->getMock(FileValidator::class); $client = $this->getMock(CloudApi::class); $file = $this->getMock(File::class); $validator->method('validate') ->with($this->equalTo($file)) ->willReturn(true); $validator->expects($this->once())->method('startUpload'); $client->expects($this->once())->method('uploadData'); $client->expects($this->once()) ->method('uploadSuccessful') ->willReturn(true); (new FileHandler($client, $validator))->process($file); }
  60. 60. Coupled architecture
  61. 61. Coupled architecture
  62. 62. Layered architecture
  63. 63. Tes$ng layered architecture
  64. 64. public function testItUploadsValidDataToTheCloud() { $fileStore = $this->getMock(FileStore::class); $file = $this->getMock(File::class); $validator->method('validate') ->with($this->equalTo($file)) ->willReturn(true); (new FileHandler($fileStore, $validator))->process($file); }
  65. 65. Tes$ng layered architecture
  66. 66. class CloudFilestoreTest extends PHPUnit_Framework_TestCase { /** @group integration */ function testItStoresFiles() { $testCredentials = … $file = new File(…); $apiClient = new CloudApi($testCredentials); $filestore = new CloudFileStore($apiClient); $filestore->store($file); $this->assertTrue($apiClient->fileExists(…)); } }
  67. 67. Tes$ng layered architecture
  68. 68. Don’t test other people’s code You don't know how it's supposed to behave
  69. 69. Don’t test how you talk to other people’s code You don't know how it's supposed to respond
  70. 70. Pu#ng it all together • Use TDD to drive development • Use TDD to replace exis1ng prac1ces, not as a new extra task
  71. 71. Pu#ng it all together Don’t ignore tests that suck, improve them: • Is the test too !ed to implementa!on? • Does the design of the code need to improve? • Are you tes!ng across domain boundaries?
  72. 72. Image credits • Human Evolu+on by Tkgd2007 h-p://bit.ly/1DdLvy1 (CC BY-SA 3.0) • Oops by Steve and Sara Emry h-ps://flic.kr/p/9Z1EEd (CC BY-NC-SA 2.0) • Blueprints for Willborough Home by Brian Tobin h-ps://flic.kr/p/7MP9RU (CC BY-NC-ND 2.0) • Border by Giulia van Pelt h-ps://flic.kr/p/9YT1c5 (CC BY-NC-ND 2.0)
  73. 73. Thank you! h"ps://joind.in/16248 @ciaranmcnulty

×