Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Building a Test
Pyramid: Symfony
testing strategies
with Ciaran McNulty
Before we start:
You must test your
applications!
What kind of tests to use?
» Manual testing
» Acceptance testing
» Unit testing
» Integration testing
» End-to-end testing...
What tools to use?
» PHPUnit
» PhpSpec
» Behat
» Codeception
» BrowserKit / Webcrawler
Question:
Why can't someone
tell me which one to
use?
Answer:
Because there is no
best answer that fits
all cases
You have to find the
Testing different layers
Introducing the pyramid
» Defined by Mike Cohn in Succeeding with Agile
» For understanding diffe...
UI layer tests
» Test the whole application end-to-end
» Sensitive to UI changes
» Aligned with acceptance criteria
» Does...
Service layer tests
» Test the application logic by making service calls
» Faster than UI testing
» Aligned with acceptanc...
Unit level tests
» Test individual classes
» Much faster than service level testing
» Very fine level of detail
» Requires...
Why a pyramid?
» Each layer builds on the one below it
» Lower layers are faster to run
» Higher levels are slower and mor...
Why do you want
tests?
The answer will affect the type of
tests you write
If you want existing
features from
breaking
... write Regression Tests
Regression tests
» Check that behaviour hasn't changed
» Easiest to apply at the UI level
» ALL tests become regression te...
'Legacy' code
class BasketController extends Controller
{
public function addAction(Request $request)
{
$productId = $requ...
Regression testing with PHPUnit
+ BrowserKit
class PostControllerTest extends WebTestCase
{
public function testShowPost()...
Regression testing with
Codeception
$I = new AcceptanceTester($scenario);
$I->amOnPage('/products/1234');
$I->click('Add t...
Regression testing with Behat +
MinkExtension
Scenario: Adding a product to the basket
Given I am on "/product/1234"
When ...
Regression testing with Ghost
Inspector
When regression testing
» Use a tool that gets you coverage quickly and easily
» Plan to phase out regression tests later
...
If you want to match
customer
requirements better
... write Acceptance Tests
Acceptance Tests
» Check the system does what the customer wants
» Are aligned with customer language and intention
» Writ...
Start with an
example-led
conversation
... before you start working on it
... but not too long before
» "What should the system do when X happens?"
» "Does Y always happen when X?"
» "What assumptions Z are causing Y to be t...
Write the examples
out in business-
readable tests
Try and make the code look like
the natural conversation you had
Easiest to test through
the User Interface
UI Acceptance testing with
PHPUnit + BrowserKit
class PostControllerTest extends WebTestCase
{
public function testAddingP...
UI Acceptance testing with
Codeception
$I = new AcceptanceTester($scenario);
$I->amGoingTo('Add a product to the basket');...
UI Acceptance testing with Behat
+ MinkExtension
Scenario: Adding a product to the basket
When I add product 1234 to the b...
UI Acceptance testing with Behat
+ MinkExtension
/**
* @When I add product :productId to the basket
*/
public function iAd...
Acceptance testing
through the UI is slow
and brittle
To test at the service layer, we
need to introduce services
'Legacy' code
class BasketController extends Controller
{
public function addAction(Request $request)
{
$productId = $requ...
'Service-oriented' code
class BasketController extends Controller
{
public function addAction(Request $request)
{
$basket ...
A very small change
but now the business logic is out of
the controller
Service layer Acceptance testing
with PHPUnit
class PostControllerTest extends PHPUnit_Framework_TestCase
{
public functio...
Service layer acceptance testing
with Behat + MinkExtension
Scenario: Adding a product to the basket
When I add product 12...
Service layer acceptance testing
with Behat
/**
* @When I add product :productId to the basket
*/
public function iAddProd...
When all of the
acceptance tests are
running against the
Service layer
... how many also need to be run
through the UI?
Symfony is a controller for your
app
If you test everything
through services
... you only need
enough UI tests to be
sure the UI is
Multiple Behat suites
Scenario: Adding a product to the basket
When I add product 1234 to the basket
Then I should see pro...
Multiple Behat suites
default:
suites:
ui:
contexts: [ UiContext ]
filters: { tags: @ui }
service:
contexts: [ ServiceConte...
If you want the design
of your code to be
better
... write Unit Tests
Unit Tests
» Check that a class does what you expect
» Use a tool that makes it easy to test classes in isolation
» Move t...
Unit tests are too granular
Customer: "The engine needs to produce 500bhp"
Engineer: "What should the diameter of the main...
Unit testing in PHPUnit
class BasketTest extends PHPUnit_Framework_Testcase
{
public function testGetsProductsFromStorage(...
Unit testing in PhpSpec
class BasketSpec extends ObjectBehavior
{
function it_gets_products_from_storage(BasketStorage $st...
Unit test
... code that is responsible for
business logic
... not code that interacts with
infrastructure including Symfony
You can unit test
interactions with
Symfony (e.g.
controllers)
You shouldn't need to if you have
acceptance tests
Coupled architecture
Unit testing third party
dependencies
class FileHandlerSpec extends ObjectBehaviour
{
public function it_uploads_data_to_t...
Coupled architecture
Layered architecture
Testing layered architecture
class FileHandlerSpec extends ObjectBehaviour
{
public function it_uploads_data_to_the_cloud_when_valid(
FileStore $filesto...
Testing layered architecture
class CloudFilestoreTest extends PHPUnit_Framework_TestCase
{
function testItStoresFiles()
{
$testCredentials = …
$file = n...
Testing layered architecture
To build your
pyramid...
Have isolated unit-
tested objects
representing your core
business logic
10,000s of tests running in <10ms
each
Have acceptance tests
at the service level
1,000s of tests running in <100ms
each
Have the bare
minimum of
acceptance tests at the
UI level
10s of tests running in <10s each
Thank You!
Any questions?
https://joind.in/talk/view/14972
@ciaranmcnulty
ciaran@sessiondigital.co.uk
Building a Pyramid: Symfony Testing Strategies
Upcoming SlideShare
Loading in …5
×

Building a Pyramid: Symfony Testing Strategies

1,897 views

Published on

The last few years have seen a huge adoption of testing practices, and an explosion of different testing tools, in the PHP space. The difficulties come when we have to choose which tools to use, in what combinations, and how to apply them to existing codebases.

In this talk we will look at what tools are available, what their strengths are, how to decide which set of tools to use for new or legacy projects, and when to prioritise decoupling and testability over the convenience we get from our frameworks.

Published in: Technology

Building a Pyramid: Symfony Testing Strategies

  1. 1. Building a Test Pyramid: Symfony testing strategies with Ciaran McNulty
  2. 2. Before we start: You must test your applications!
  3. 3. What kind of tests to use? » Manual testing » Acceptance testing » Unit testing » Integration testing » End-to-end testing » Black box / white box
  4. 4. What tools to use? » PHPUnit » PhpSpec » Behat » Codeception » BrowserKit / Webcrawler
  5. 5. Question: Why can't someone tell me which one to use?
  6. 6. Answer: Because there is no best answer that fits all cases You have to find the
  7. 7. Testing different layers Introducing the pyramid » Defined by Mike Cohn in Succeeding with Agile » For understanding different layers of testing
  8. 8. UI layer tests » Test the whole application end-to-end » Sensitive to UI changes » Aligned with acceptance criteria » Does not require good code » Probably slow e.g. Open a browser, fill in the form and submit it
  9. 9. Service layer tests » Test the application logic by making service calls » Faster than UI testing » Aligned with acceptance criteria » Mostly written in the target language » Requires high-level services to exist e.g. Instantiate the Calculator service and get it to add two numbers
  10. 10. Unit level tests » Test individual classes » Much faster than service level testing » Very fine level of detail » Requires good design
  11. 11. Why a pyramid? » Each layer builds on the one below it » Lower layers are faster to run » Higher levels are slower and more brittle » Have more tests at the bottom than at the top
  12. 12. Why do you want tests? The answer will affect the type of tests you write
  13. 13. If you want existing features from breaking ... write Regression Tests
  14. 14. Regression tests » Check that behaviour hasn't changed » Easiest to apply at the UI level » ALL tests become regression tests eventually
  15. 15. 'Legacy' code class BasketController extends Controller { public function addAction(Request $request) { $productId = $request->attributes->get('product_id'); $basket = $request->getSession()->get('basket_context')->getCurrent(); $products = $basket->getProducts(); $products[] = $productId; $basket->setProducts($products); return $this->render('::basket.html.twig', ['basket' => $basket]); } }
  16. 16. Regression testing with PHPUnit + BrowserKit class PostControllerTest extends WebTestCase { public function testShowPost() { $client = static::createClient(); $crawler = $client->request('GET', '/products/1234'); $form = $crawler->selectButton('Add to basket')->form(); $client->submit($form, ['id'=>1234]); $product = $crawler->filter('html:contains("Product: 1234")'); $this->assertCount(1, $product); } }
  17. 17. Regression testing with Codeception $I = new AcceptanceTester($scenario); $I->amOnPage('/products/1234'); $I->click('Add to basket'); $I->see('Product: 1234');
  18. 18. Regression testing with Behat + MinkExtension Scenario: Adding a product to the basket Given I am on "/product/1234" When I click "Add to Basket" Then I should see "Product: 1234"
  19. 19. Regression testing with Ghost Inspector
  20. 20. When regression testing » Use a tool that gets you coverage quickly and easily » Plan to phase out regression tests later » Lean towards testing end-to-end » Recognise they will be hard to maintain
  21. 21. If you want to match customer requirements better ... write Acceptance Tests
  22. 22. Acceptance Tests » Check the system does what the customer wants » Are aligned with customer language and intention » Write them in English (or another language) first » Can be tested at the UI or Service level
  23. 23. Start with an example-led conversation ... before you start working on it ... but not too long before
  24. 24. » "What should the system do when X happens?" » "Does Y always happen when X?" » "What assumptions Z are causing Y to be the outcome?" » "Given Z when X then Y" » "What other things aside from Y might happen?" » "What if...?"
  25. 25. Write the examples out in business- readable tests Try and make the code look like the natural conversation you had
  26. 26. Easiest to test through the User Interface
  27. 27. UI Acceptance testing with PHPUnit + BrowserKit class PostControllerTest extends WebTestCase { public function testAddingProductToTheBasket() { $this->addProductToBasket(1234); $this->productShouldBeShownInBasket(1234); } private function addProductToBasket($productId) { //... browser automation code } private function productShouldBeShownInBasket($productId) { //... browser automation code } }
  28. 28. UI Acceptance testing with Codeception $I = new AcceptanceTester($scenario); $I->amGoingTo('Add a product to the basket'); $I->amOnPage('/products/1234'); $I->click('Add to basket'); $I->expectTo('see the product in the basket'); $I->see('Product: 1234');
  29. 29. UI Acceptance testing with Behat + MinkExtension Scenario: Adding a product to the basket When I add product 1234 to the basket Then I should see product 1234 in the basket
  30. 30. UI Acceptance testing with Behat + MinkExtension /** * @When I add product :productId to the basket */ public function iAddProduct($productId) { $this->visitUrl('/product/' . $productId); $this->getSession()->clickButton('Add to Basket'); } /** * @Then I should see product :productId in the basket */ public function iShouldSeeProduct($productId) { $this->assertSession()->elementContains('css', '#basket', 'Product: ' . $productId); }
  31. 31. Acceptance testing through the UI is slow and brittle To test at the service layer, we need to introduce services
  32. 32. 'Legacy' code class BasketController extends Controller { public function addAction(Request $request) { $productId = $request->attributes->get('product_id'); $basket = $request->getSession()->get('basket_context')->getCurrent(); $products = $basket->getProducts(); $products[] = $productId; $basket->setProducts($products); return $this->render('::basket.html.twig', ['basket' => $basket]); } }
  33. 33. 'Service-oriented' code class BasketController extends Controller { public function addAction(Request $request) { $basket = $this->get('basket_context')->getCurrent(); $productId = $request->attributes->get('product_id'); $basket->addProduct($productId); return $this->render('::basket.html.twig', ['basket' => $basket]); } }
  34. 34. A very small change but now the business logic is out of the controller
  35. 35. Service layer Acceptance testing with PHPUnit class PostControllerTest extends PHPUnit_Framework_TestCase { public function testAddingProductToTheBasket() { $basket = new Basket(new BasketArrayStorage()); $basket->addProduct(1234); $this->assertContains(1234, $basket->getProducts()); } }
  36. 36. Service layer acceptance testing with Behat + MinkExtension Scenario: Adding a product to the basket When I add product 1234 to the basket Then I should see product 1234 in the basket
  37. 37. Service layer acceptance testing with Behat /** * @When I add product :productId to the basket */ public function iAddProduct($productId) { $this->basket = new Basket(new BasketArrayStorage()); $this->basket->addProduct($productId); } /** * @Then I should see product :productId in the basket */ public function iShouldSeeProduct($productId) { assert(in_array($productId, $this->basket->getProducts()); }
  38. 38. When all of the acceptance tests are running against the Service layer ... how many also need to be run through the UI?
  39. 39. Symfony is a controller for your app
  40. 40. If you test everything through services ... you only need enough UI tests to be sure the UI is
  41. 41. Multiple Behat suites Scenario: Adding a product to the basket When I add product 1234 to the basket Then I should see product 1234 in the basket Scenario: Adding a product that is already there Given I have already added product 1234 to the basket When I add product 1234 to the basket Then I should see 2 instances of product 1234 in the basket @ui Scenario: Adding two products to my basket Given I have already added product 4567 to the basket When I add product 1234 to the basket Then I should see product 4567 in the basket And I should also see product 1234 in the basket
  42. 42. Multiple Behat suites default: suites: ui: contexts: [ UiContext ] filters: { tags: @ui } service: contexts: [ ServiceContext ]
  43. 43. If you want the design of your code to be better ... write Unit Tests
  44. 44. Unit Tests » Check that a class does what you expect » Use a tool that makes it easy to test classes in isolation » Move towards writing them first » Unit tests force you to have good design » Probably too small to reflect acceptance criteria
  45. 45. Unit tests are too granular Customer: "The engine needs to produce 500bhp" Engineer: "What should the diameter of the main drive shaft be?"
  46. 46. Unit testing in PHPUnit class BasketTest extends PHPUnit_Framework_Testcase { public function testGetsProductsFromStorage() { $storage = $this->getMock('BasketStorage'); $storage->expect($this->once()) ->method('persistProducts') ->with([1234]); $basket = new Basket($storage); $basket->addProduct(1234); } }
  47. 47. Unit testing in PhpSpec class BasketSpec extends ObjectBehavior { function it_gets_products_from_storage(BasketStorage $storage) { $this->beConstructedWith($storage); $this->addProduct(1234); $storage->persistProducts([1234])->shouldHaveBeenCalled([1234]); } }
  48. 48. Unit test ... code that is responsible for business logic ... not code that interacts with infrastructure including Symfony
  49. 49. You can unit test interactions with Symfony (e.g. controllers) You shouldn't need to if you have acceptance tests
  50. 50. Coupled architecture
  51. 51. Unit testing third party dependencies class FileHandlerSpec extends ObjectBehaviour { public function it_uploads_data_to_the_cloud_when_valid( CloudApi $client, FileValidator $validator, File $file ) { $this->beConstructedWith($client, $validator); $validator->validate($file)->willReturn(true); $client->startUpload()->shouldBeCalled(); $client->uploadData(Argument::any())->shouldBeCalled(); $client->uploadSuccessful()->willReturn(true); $this->process($file)->shouldReturn(true); } }
  52. 52. Coupled architecture
  53. 53. Layered architecture
  54. 54. Testing layered architecture
  55. 55. class FileHandlerSpec extends ObjectBehaviour { public function it_uploads_data_to_the_cloud_when_valid( FileStore $filestore, FileValidator $validator, File $file ) { $this->beConstructedWith($filestore, $validator); $validator->validate($file)->willReturn(true); $this->process($file); $filestore->store($file)->shouldHaveBeenCalled(); } }
  56. 56. Testing layered architecture
  57. 57. class CloudFilestoreTest extends PHPUnit_Framework_TestCase { function testItStoresFiles() { $testCredentials = … $file = new File(…); $apiClient = new CloudApi($testCredentials); $filestore = new CloudFileStore($apiClient); $filestore->store($file); $this->assertTrue($apiClient->fileExists(…)); } }
  58. 58. Testing layered architecture
  59. 59. To build your pyramid...
  60. 60. Have isolated unit- tested objects representing your core business logic 10,000s of tests running in <10ms each
  61. 61. Have acceptance tests at the service level 1,000s of tests running in <100ms each
  62. 62. Have the bare minimum of acceptance tests at the UI level 10s of tests running in <10s each
  63. 63. Thank You! Any questions? https://joind.in/talk/view/14972 @ciaranmcnulty ciaran@sessiondigital.co.uk

×