Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Property based testing - MageTestFest 2019

470 views

Published on

This talk is an introduction to property based testing in PHP in the context of Magento.
It was given at MageTestFest 2019 in Florence.

Published in: Software
  • Be the first to comment

Property based testing - MageTestFest 2019

  1. 1. Property BasedTesting ❤(c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  2. 2. ExampleScenario: MultiSourceInventory (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  3. 3. Iwantit! Isitready? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  4. 4. ! (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  5. 5. Let'stest! (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  6. 6. Whatdowetest? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  7. 7. Scenario 1 website 1 source 1 stock 1 product qty 1 1 order qty 1 assertTrue($isItemSoldOut) (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  8. 8. Itworks! (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  9. 9. ...butdoesit? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  10. 10. Scenario2 2 website 2 sources 2 stocks 1 product, qty 1 in each source 1 order from source 1, qty 1 assertTrue($isItemSoldOutForStock1) (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  11. 11. Howconfidentareyou? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  12. 12. Okay,afewmorecases... (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  13. 13. Eachofthepriortestsbutwith: 2 products (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  14. 14. Oh,andalsowith: 2 orders (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  15. 15. Whataboutbackorders? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  16. 16. Butwhataboutqtythresholds? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  17. 17. Butwhataboutpositive& negativeqtythresholds? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  18. 18. Anddecimalquantities? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  19. 19. Andshouldn'twetestthat ifaproductisnotsoldout itstillissalable? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  20. 20. Aretherenewedgecasesthatare introducedby interactingfeatures? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  21. 21. ! (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  22. 22. ...It'simpossible! (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  23. 23. Let'sjustforgetabout automatedtesting. (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  24. 24. kthxbye Questions? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  25. 25. Realhero:JohnHughes (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  26. 26. Don't write tests, generate them! -- John Hughes (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  27. 27. Instead of examples... (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  28. 28. We describe the full range of valid inputs (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  29. 29. 1+ websites 1+ sources 1+ products assigned to sources at random Each with a qty in of -max_int to max_int Each with or without backorders enabled Each with a threshold of -max_int to max_int Each with or without decimal quantities 1+ orders per product with a total qty from .0001 to the salable amount (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  30. 30. Let the test framework generate a random sample for each input (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  31. 31. And we do that hundreds of times. (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  32. 32. And we do that hundreds N times. (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  33. 33. Unrealistic! It takes too much time. (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  34. 34. 1-5 websites 1-5 sources 1-100 products assigned to sources at random Each with a qty of -100 to 100 Each with or without backorders enabled Each with a threshold of -10 to 10 Each with or without decimal quantities n orders per product with a total qty from .0001 to the salable amount (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  35. 35. S.S.A.F. (stillslowashell) (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  36. 36. 2 fix websites 2 fix sources 1 product Stock qty of -100 - 100 With a threshold of -10 to 10 With backorders enabled if threshold < 0 With decimal quantities 1 - 10 orders per product (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  37. 37. Thatmightwork (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  38. 38. Howexactly? Details please (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  39. 39. (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  40. 40. $check = $this->forAll(...$generators); (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  41. 41. And ...$generators is what? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  42. 42. $simpleProducts = GeneratorsimpleProduct(); $inStockQtys = GeneratorchooseFloat(-100, 100); $minQtyThresholds = GeneratorchooseFloat(-100, 100); $nOrdersForProduct = Generatorchoose(1, 10); $check = $this->forAll( $simpleProducts, $inStockQtys, $minQtyThresholds, $nOrdersForProduct ); (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  43. 43. Generatorsdescribethe rangeofvalidinputs (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  44. 44. Generators are Composeable (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  45. 45. Generatorint(); // => 29384 Generatorneg(); // => -6 Generatornat(); // => 0 Generatorpos(); // => 15 Generatorfloat(); // => 21.92836 (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  46. 46. Generatorstring(); // => Jx@ Generatornames(); // Alice (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  47. 47. Generatorchoose(1, 1000) // => 55 (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  48. 48. Generatorvec(4, Generatornames()); // => ["Zénobin", "Dee", "Allen", "Theophania" ], (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  49. 49. public static function skuGenerator(): Generator { return GeneratorsuchThat( function (string $s) { return $s !== '' && strlen($s) <= 64; }, Generatorstring() ); } (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  50. 50. public static function simpleProductGenerator(): Generator { return Generatorbind( Generatortuple( self::skuGenerator(), self::productNameGenerator() ), function (array $skuAndName) { [$sku, $name] = $skuAndName; $product = self::createSimpleProduct($sku, $name); return Generatorconstant($product); } ); } (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  51. 51. public static function simpleProductsGenerator(): Generator { return Generatorseq( self::simpleProductGenerator() ); // => array of zero or more products } (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  52. 52. Generating interdependent values is a bit more complex... (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  53. 53. Generatorbind( GeneratorsuchThat( function (array $tuple) { [$inStockQty, $minInStockQty] = $tuple; return $minInStockQty <= 0 || $minInStockQty > $inStockQty; }, Generatortuple( $this->chooseFloat(0, 100), // inStockQty $this->chooseFloat(-100, 100) // minInStockQty ) ), function (array $tuple): Generator { [$inStockQty, $minInStockQty] = $tuple; $salableQty = $inStockQty - $minInStockQty; return Generatorassociative([ 'inStockQty' => Generatorconstant($inStockQty), 'minQtyThreshold' => Generatorconstant($minInStockQty), 'orderQty' => Generatorfrequency( [1, $salableQty], // place orders for full purchasable quantity [2, $this->chooseFloat(0, $salableQty - 0.0001)] // purchase below salable quantity )] ); } ); (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  54. 54. Generatorsdescribethefullscope ofvalidinputs $check = $this->forAll(...$generators); (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  55. 55. andthen? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  56. 56. Declare System Properties (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  57. 57. Property: Something that is always true (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  58. 58. $check = $this->forAll(Generatorint(), Generatorint()); $check(function (int $a, int $b) { $this->assertSame( $a + $b, $b + $a ); }); (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  59. 59. How about something Magento related? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  60. 60. QuoteMerging: (customer quote item qty) + (guest quote item qty) == merged quote item qty (customer quote item count) + (guest quote item count) >= merged quote item count (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  61. 61. $check = $this->forAll( $this->simpleProducts(), $this->simpleProducts() ); (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  62. 62. $check(function (array $inCustomerQuote, array $inGuestQuote) { $customerQuote = $this->createQuote(); $guestQuote = $this->createQuote(); $this->addProductsToQuote($customerQuote, $inCustomerQuote); $this->addProductsToQuote($guestQuote, $inGuestQuote); $customerQuoteItemQtyBefore = $customerQuote->getItemsQty(); $guestQuoteItemQtyBefore = $guestQuote->getItemsQty(); (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  63. 63. $customerQuote->merge($guestQuote); (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  64. 64. $this->assertSame( $customerQuoteItemQtyBefore + $guestQuoteItemQtyBefore $customerQuote->getItemsQty() ); $this->assertLessThanOrEqual( count($inCustomerQuote) + count($inGuestQuote), count($customerQuote->getAllVisibleItems() ); }); (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  65. 65. FindingProperties (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  66. 66. Reversible algorithms $this->assertSame( $input, json_decode(json_encode($input), true) ); (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  67. 67. Algorithms with algebraic properties $this->assertEquals( $quoteC->merge($quoteB->merge($quoteA)); $quoteA->merge($quoteB->merge($quoteC)); ); // $quoteC . ($quoteB . $quoteA) // ($quoteC . $quoteB) . $quoteA (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  68. 68. Generative smoke tests $check = $this->forAll(Generatorstring()); $check(function(string $requestPayload) use ($api) { $response = $api->post($requestPayload); $this->assertContains( $response->status(), [200, 201, 202, 203, 204, 205, 206] ); $this->assertNotNull( json_decode($response->body(), true) ); }); (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  69. 69. Alternative implementation $this->assertSame( $system->process($a, $b), $otherImplementation->process($a, $b) ); (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  70. 70. Testing stateful systems $check = $this->forAll( Generatorseq(GeneratordomainAction()) ); $check(function(array $actions)) use ($system, $model) { foreach ($actions as $takeAction) { $takeAction($system); $takeAction($model); } $this->assertSame($system->getState(), $model->getState()); }); (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  71. 71. Inanutshell: (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  72. 72. 1. Define valid inputs with generators 2. Write tests for properties (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  73. 73. Howfastisit? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  74. 74. As fast as the system under test (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  75. 75. Howhardisit? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  76. 76. Ittakeseffort (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  77. 77. Isitworththetime? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  78. 78. Doesitfindbugs? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  79. 79. Itdoesindeed. (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  80. 80. While preparing this talk... (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  81. 81. Decimal qtys Backorders enabled Negative min qty threshold O!en impossible to purchase the complete salable quantity when multiple decimal orders are used. ApiGetProductSalableQtyInterface reports that only -8.3266726846887E-17 are in stock. (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  82. 82. Zero product stock qty Backorders enabled Negative min qty threshold Wrong salable quantity ApiGetProductSalableQtyInterface reports salable quantity as zero instead of the min qty threshold (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  83. 83. Propertybasedtesting isbetterthanhumans atfindingedgecases (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  84. 84. So...isitworthit? (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  85. 85. Thanks to John Hughes QuickCheck Sebastian Bergman PHPUnit Giorgio Sironi Eris (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019
  86. 86. Pleasecomeandchat! IRL RIGHT HERE twitter://@VinaiKopp slack://vinai@magentocommeng.slack.com slack self signup: http://tinyurl.com/engcom-slack (c) Vinai Kopp - http://vinaikopp.com - twitter://@VinaiKopp - #MageTesFest 2019

×