Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Assemble Your Code in Stages: Leveling Up With Pipelines

56 views

Published on

Applications grow, specs change, bugs happen, and our code can quickly get out of hand. Duplicated code, ifs, elses, switches, and statements like “I used this there, but it needs to be slightly different here”, help turn our work of art into a garbled mess. But what if we could fix that?

That’s where Pipelines come in. We can break out our code into smaller chunks, called stages, that we can group or combine into configurations called pipelines. Separating our code into stages allows for easier and isolated testing. Reassembling stages sequentially into a pipeline allows us to have consistent results.

In this talk, we’ll define what stages and pipelines are. We'll examine when pipelines can help us and when they are not the right solution. We will look at example pipelines ranging from simple to multi-stage reusable pipelines. We'll implement what we've learned by walking through a refactor and discover how testing becomes easier with stages. You will walk away with an understanding of the what the Pipeline pattern is and when it can benefit your application.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Assemble Your Code in Stages: Leveling Up With Pipelines

  1. 1. Assemble Your Code in Stages: Leveling Up With Pipelines
  2. 2. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 About me Steven Wade • Husband, father • Founder/Organizer of UpstatePHP Twitter: @stevenwadejr Email: stevenwadejr@gmail.com
  3. 3. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Problem
  4. 4. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Problem class OrderProcessController { public function processOrder(Request $request) { $order = new Order; $order->billing = $request->get('billing'); $order->shipping = $request->get('shipping'); $order->products = $request->get('products'); $order->save(); return response(null); } }
  5. 5. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Problem class OrderProcessController { public function processOrder(Request $request) { $order = new Order; $order->billing = $request->get('billing'); $order->shipping = $request->get('shipping'); $order->products = $request->get('products'); // Calculate sub-total $productsTotal = 0; foreach ($request->get('products', []) as $product) { $productsTotal += ($product['price'] * $product['quantity']); } $order->order_total += round($productsTotal, 2); $order->save(); return response(null); } }
  6. 6. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Problem class OrderProcessController { public function processOrder(Request $request) { $order = new Order; $order->billing = $request->get('billing'); $order->shipping = $request->get('shipping'); $order->products = $request->get('products'); // Calculate sub-total $productsTotal = 0; foreach ($request->get('products', []) as $product) { $productsTotal += ($product['price'] * $product['quantity']); } $order->order_total += round($productsTotal, 2); // Process payment $receipt = $this->paymentGateway->process($order); $order->confirmation = $receipt->transaction_id; event(new OrderProcessed($order)); $order->save(); return response(null); } }
  7. 7. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Problem class OrderProcessController { public function processOrder(Request $request) { $order = new Order; $order->billing = $request->get('billing'); $order->shipping = $request->get('shipping'); $order->products = $request->get('products'); // Calculate sub-total $productsTotal = 0; foreach ($request->get('products', []) as $product) { $productsTotal += ($product['price'] * $product['quantity']); } $order->order_total += round($productsTotal, 2); // Add tax $taxRate = TaxFactory::getRate($order->billing['state']); $tax = $order->order_total * $taxRate; $tax = round($tax, 2); $order->order_total += $tax; $order->tax = $tax; // Process payment $receipt = $this->paymentGateway->process($order); $order->confirmation = $receipt->transaction_id; event(new OrderProcessed($order)); $order->save(); return response(null); } }
  8. 8. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Problem class OrderProcessController { public function processOrder(Request $request) { $order = new Order; $order->billing = $request->get('billing'); $order->shipping = $request->get('shipping'); $order->products = $request->get('products'); // Calculate sub-total $productsTotal = 0; foreach ($request->get('products', []) as $product) { $productsTotal += ($product['price'] * $product['quantity']); } $order->order_total += round($productsTotal, 2); // Add tax $taxRate = TaxFactory::getRate($order->billing['state']); $tax = $order->order_total * $taxRate; $tax = round($tax, 2); $order->order_total += $tax; $order->tax = $tax; // Calculate shipping // Free shipping on orders $100+ if ($order->order_total < 100) { $calculator = new ShippingCalculator; $shipping = $calculator->calculate($order->shipping); $order->order_total += $shipping; } // Process payment $receipt = $this->paymentGateway->process($order); $order->confirmation = $receipt->transaction_id; event(new OrderProcessed($order)); $order->save(); return response(null); } }
  9. 9. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Problem class OrderProcessController { public function processOrder(Request $request) { $order = new Order; $order->billing = $request->get('billing'); $order->shipping = $request->get('shipping'); $order->products = $request->get('products'); // Calculate sub-total $productsTotal = 0; foreach ($request->get('products', []) as $product) { $productsTotal += ($product['price'] * $product['quantity']); } $order->order_total += round($productsTotal, 2); // Add coupons $couponCode = $request->get('coupon'); if ($couponCode) { $couponRepo = app(CouponRepository::class); $coupon = $couponRepo->findByCode($couponCode); $discount = $coupon->isPercentage() ? round($coupon->amount / $order->order_total * 100, 2) : $coupon->amount; $order->order_total -= $discount; } // Add tax $taxRate = TaxFactory::getRate($order->billing['state']); $tax = $order->order_total * $taxRate; $tax = round($tax, 2); $order->order_total += $tax; $order->tax = $tax; // Calculate shipping // Free shipping on orders $100+ if ($order->order_total < 100) { $calculator = new ShippingCalculator; $shipping = $calculator->calculate($order->shipping); $order->order_total += $shipping; } // Process payment $receipt = $this->paymentGateway->process($order); $order->confirmation = $receipt->transaction_id; event(new OrderProcessed($order)); $order->save(); return response(null); } }
  10. 10. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853
  11. 11. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853
  12. 12. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Pipelines!*(possibly)
  13. 13. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Goals • Understand what a pipeline is, and what stages are • Learn to recognize a pipeline in our code • Refactor code to stages • See how stages make testing easier • Understand when a pipeline is not the appropriate option
  14. 14. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 What is a pipeline?
  15. 15. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 |
  16. 16. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 cat logs.txt | grep "ERROR" | wc -l
  17. 17. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 So, what is a pipeline?
  18. 18. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Pipeline A series of processes chained together to where the output of each is the input of the next
  19. 19. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Stages cat logs.txt | grep "ERROR" | wc -l
  20. 20. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Real World Example
  21. 21. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Bedtime <?php $bedtime = (new Pipeline) ->pipe(new Bath) ->pipe(new Diaper) ->pipe(new Pajamas) ->pipe(new BrushTeeth) ->pipe(new Book('Catalina Magdalena...')) ->pipe(new Song('Radio Gaga')); $bedtime->process($penny);
  22. 22. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853
  23. 23. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853
  24. 24. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853
  25. 25. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853
  26. 26. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853
  27. 27. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Gulp gulp.task('css', function(){ return gulp.src('client/templates/*.less') .pipe(less()) .pipe(minifyCSS()) .pipe(gulp.dest('build/css')) });
  28. 28. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853
  29. 29. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 PHP Land
  30. 30. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Nested Function Calls function timesTwo($payload) { return $payload * 2; } function addOne($payload) { return $payload + 1; } // outputs 21 echo addOne( timesTwo(10) );
  31. 31. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Nested Mess $slug = strtolower( preg_replace( '~-+~', '-', trim( preg_replace( '~[^-w]+~', '', preg_replace( '~[^pLd]+~u', '-', 'My Awesome Blog Post!' ) ), '-' ) ) ); echo $slug;
  32. 32. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Nested Function Calls function timesTwo($payload) { return $payload * 2; } function addOne($payload) { return $payload + 1; } // outputs 21 echo addOne( timesTwo(10) );
  33. 33. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Looping Through Stages $stages = ['timesTwo', 'addOne']; $payload = 10; foreach ($stages as $stage) { $payload = call_user_func($stage, $payload); } // outputs 21 echo $payload;
  34. 34. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853
  35. 35. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 –Martin Fowler “Any fool can write code that a computer can understand. Good programmers write code that humans can understand.”
  36. 36. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 LeaguePipeline
  37. 37. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 LeaguePipeline Frank de Jonge
 @frankdejonge Woody Gilk
 @shadowhand
  38. 38. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 LeaguePipeline - Functional $pipeline = (new Pipeline) ->pipe('timesTwo') ->pipe('addOne'); // Returns 21 $pipeline->process(10);
  39. 39. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 LeaguePipeline - Class Based $pipeline = (new Pipeline) ->pipe(new TimeTwoStage) ->pipe(new AddOneStage); // Returns 21 $pipeline->process(10);
  40. 40. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Putting it into practice
  41. 41. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 eCommerce • Create the order • Calculate the total • Process the payment • Subtract coupons from total • Add appropriate taxes • Calculate and add shipping costs
  42. 42. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 eCommerce • Create the order • Calculate the sub-total • Subtract coupons from total • Add appropriate taxes • Calculate and add shipping costs • Process the payment
  43. 43. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 eCommerce class OrderProcessController { public function processOrder(Request $request) { $order = new Order; $order->billing = $request->get('billing'); $order->shipping = $request->get('shipping'); $order->products = $request->get('products'); // Calculate sub-total $productsTotal = 0; foreach ($request->get('products', []) as $product) { $productsTotal += ($product['price'] * $product['quantity']); } $order->order_total += round($productsTotal, 2); // Add coupons $couponCode = $request->get('coupon'); if ($couponCode) { $couponRepo = app(CouponRepository::class); $coupon = $couponRepo->findByCode($couponCode); $discount = $coupon->isPercentage() ? round($coupon->amount / $order->order_total * 100, 2) : $coupon->amount; $order->order_total -= $discount; } } }
  44. 44. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 eCommerce class OrderProcessController { public function processOrder(Request $request) { $order = new Order; $order->billing = $request->get('billing'); $order->shipping = $request->get('shipping'); $order->products = $request->get('products'); // Calculate sub-total $productsTotal = 0; foreach ($request->get('products', []) as $product) { $productsTotal += ($product['price'] * $product['quantity']); } $order->order_total += round($productsTotal, 2); // Add coupons $couponCode = $request->get('coupon'); if ($couponCode) { $couponRepo = app(CouponRepository::class); $coupon = $couponRepo->findByCode($couponCode); $discount = $coupon->isPercentage() ? round($coupon->amount / $order->order_total * 100, 2) : $coupon->amount; $order->order_total -= $discount; } // Add tax $taxRate = TaxFactory::getRate($order->billing['state']); $tax = $order->order_total * $taxRate; $tax = round($tax, 2); $order->order_total += $tax; $order->tax = $tax; // Calculate shipping // Free shipping on orders $100+ if ($order->order_total < 100) { $calculator = new ShippingCalculator; $shipping = $calculator->calculate($order->shipping); $order->order_total += $shipping; } // Process payment $receipt = $this->paymentGateway->process($order); $order->confirmation = $receipt->transaction_id; event(new OrderProcessed($order)); $order->save(); return response(null); } }
  45. 45. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 eCommerce - testing class OrderProcessController { public function processOrder(Request $request) { $order = new Order; $order->billing = $request->get('billing'); $order->shipping = $request->get('shipping'); $order->products = $request->get('products'); // Calculate sub-total $productsTotal = 0; foreach ($request->get('products', []) as $product) { $productsTotal += ($product['price'] * $product['quantity']); } $order->order_total += round($productsTotal, 2); // Add tax $taxRate = TaxFactory::getRate($order->billing['state']); $tax = $order->order_total * $taxRate; $tax = round($tax, 2); $order->order_total += $tax; $order->tax = $tax; } }
  46. 46. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 eCommerce - testing class OrderProcessController { public function processOrder(Request $request) { $order = new Order; $order->billing = $request->get('billing'); $order->shipping = $request->get('shipping'); $order->products = $request->get('products'); // Calculate sub-total $productsTotal = 0; foreach ($request->get('products', []) as $product) { $productsTotal += ($product['price'] * $product['quantity']); } $order->order_total += round($productsTotal, 2); // Add tax $taxRate = TaxFactory::getRate($order->billing['state']); $tax = $order->order_total * $taxRate; $tax = round($tax, 2); $order->order_total += $tax; $order->tax = $tax; } }
  47. 47. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 eCommerce class OrderProcessController { public function processOrder(Request $request) { $order = new Order; $order->billing = $request->get('billing'); $order->shipping = $request->get('shipping'); $order->products = $request->get('products'); // Calculate sub-total $productsTotal = 0; foreach ($request->get('products', []) as $product) { $productsTotal += ($product['price'] * $product['quantity']); } $order->order_total += round($productsTotal, 2); // Add coupons $couponCode = $request->get('coupon'); if ($couponCode) { $couponRepo = app(CouponRepository::class); $coupon = $couponRepo->findByCode($couponCode); $discount = $coupon->isPercentage() ? round($coupon->amount / $order->order_total * 100, 2) : $coupon->amount; $order->order_total -= $discount; } // Add tax $taxRate = TaxFactory::getRate($order->billing['state']); $tax = $order->order_total * $taxRate; $tax = round($tax, 2); $order->order_total += $tax; $order->tax = $tax; // Calculate shipping // Free shipping on orders $100+ if ($order->order_total < 100) { $calculator = new ShippingCalculator; $shipping = $calculator->calculate($order->shipping); $order->order_total += $shipping; } // Process payment $receipt = $this->paymentGateway->process($order); $order->confirmation = $receipt->transaction_id; event(new OrderProcessed($order)); $order->save(); return response(null); } }
  48. 48. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 eCommerce class OrderProcessController { public function processOrder(Request $request) { $order = new Order; $order->billing = $request->get('billing'); $order->shipping = $request->get('shipping'); $order->products = $request->get('products'); // Calculate sub-total $productsTotal = 0; foreach ($request->get('products', []) as $product) { $productsTotal += ($product['price'] * $product['quantity']); } $order->order_total += round($productsTotal, 2); // Add coupons $couponCode = $request->get('coupon'); if ($couponCode) { $couponRepo = app(CouponRepository::class); $coupon = $couponRepo->findByCode($couponCode); $discount = $coupon->isPercentage() ? round($coupon->amount / $order->order_total * 100, 2) : $coupon->amount; $order->order_total -= $discount; } } } Create order Sub-total Subtract coupons
  49. 49. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 eCommerce public function processOrder(Request $request) { $order = $this->createOrder($request); $this->calculateSubTotal($order); $this->applyCoupon($request, $order); $this->applyTaxes($order); $this->calculateShipping($order); $this->processPayment($order); $order->save(); return response(null); }
  50. 50. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 eCommerce - Testing protected function applyCoupon(Request $request, Order $order): void { $couponCode = $request->get('coupon'); if ($couponCode) { $couponRepo = new CouponRepository; $coupon = $couponRepo->findByCode($couponCode); $discount = $coupon->isPercentage() ? round($coupon->amount / $order->order_total * 100, 2) : $coupon->amount; $order->order_total -= $discount; } }
  51. 51. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 eCommerce - Testing protected function applyCoupon(Request $request, Order $order): void { $couponCode = $request->get('coupon'); if ($couponCode) { $couponRepo = app(CouponRepository::class); $coupon = $couponRepo->findByCode($couponCode); $discount = $coupon->isPercentage() ? round($coupon->amount / $order->order_total * 100, 2) : $coupon->amount; $order->order_total -= $discount; } }
  52. 52. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 eCommerce - Testing public function __construct(CouponRepository $couponRepository) { $this->couponRepository = $couponRepository; }
  53. 53. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 eCommerce public function processOrder(Request $request) { $order = $this->createOrder($request); $this->calculateSubTotal($order); $this->applyCoupon($request, $order); $this->applyTaxes($order); $this->calculateShipping($order); $this->processPayment($order); $order->save(); return response(null); }
  54. 54. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 eCommerce - Testing public function __construct( CouponRepository $couponRepository, ShippingCalculator $shippingCalculator, PaymentGateway $paymentGateway ) { $this->couponRepository = $couponRepository; $this->shippingCalculator = $shippingCalculator; $this->paymentGateway = $paymentGateway; }
  55. 55. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 eCommerce public function processOrder(Request $request) { $order = $this->createOrder($request); $this->calculateSubTotal($order); $this->applyCoupon($request, $order); $this->applyTaxes($order); $this->calculateShipping($order); $this->processPayment($order); $order->save(); return response(null); }
  56. 56. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Stages!
  57. 57. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 eCommerce - Refactored class OrderProcessController { protected $stages = [ CalculateSubTotal::class, ApplyCoupon::class, ApplyTaxes::class, CalculateShipping::class, ProcessPayment::class, ]; public function processOrder(Request $request) { $order = OrderFactory::fromRequest($request); $pipeline = new LeaguePipelinePipeline; foreach ($this->stages as $stage) { $stage = app($stage, ['request' => $request]); $pipeline->pipe($stage); } $pipeline->process($order); $order->save(); return response(null); } }
  58. 58. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Testing Stages class OrderProcessTest { public function test_sales_tax() { $subTotal = 100.00; $taxRate = 0.06; $expected = 106.00; $order = new Order; $order->order_total = $subTotal; $order->billing['state'] = 'SC'; $stage = new ApplyTaxes; $stage($order); $this->assertEquals($expected, $order->order_total); } }
  59. 59. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Recap • Pipeline: a series of processes (stages) chained together to where the output of each is the input of the next. • Stages create readability, reusability, and testability
  60. 60. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853
  61. 61. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853
  62. 62. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Don't
  63. 63. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Choose wisely
  64. 64. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Encore!
  65. 65. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Fun Stuff - Variable Pipeline class RelatedData { protected $stages = [ LatestActivityPipeline::class, ListMembershipsStage::class, WorkflowMembershipsStage::class, DealsStage::class, ]; public function process() { $pipeline = new Pipeline; foreach ($this->stages as $stage) { if ($this->stageIsEnabled()) { $pipeline->pipe(new $stage); } } return $pipeline->process([]); } }
  66. 66. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Fun Stuff - Async Stages class DealsStage { public function __invoke(array $payload) { if ($cache = $this->cache->get('deals')) { $promise = new FulfilledPromise($cache); } else { $promise = $this->api->getDeals(); $promise = $promise->then(function ($deals) { $this->cache->set('deals', $deals); return $deals; }); } $payload['deals'] = $promise; return $payload; } }
  67. 67. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Fun Stuff - Async Stages class RelatedData { public function process() { $promises = $this->pipeline->process([]); return GuzzleHttpPromiseunwrap($promises); } }
  68. 68. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 LeaguePipeline - Added Benefits Processors • FingersCrossedProcessor • InterruptibleProcessor • ProcessorInterface
  69. 69. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 LeaguePipeline - InterruptibleProcessor $processor = new InterruptibleProcessor(function($payload) { return $payload < 10; }); $pipeline = new Pipeline( $processor, function ($payload) { return $payload + 2; }, // 7 function ($payload) { return $payload * 10; }, // 70 function ($payload) { return $payload * 10; } // 700 ); // outputs 70 echo $pipeline->process(5);
  70. 70. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Laravel's Pipeline $stages = [ StageOne::class, StageTwo::class ]; $result = app(Pipeline::class) ->send($payload) ->through($stages) ->then(function($payload) { return $payload; });
  71. 71. Thank you! Questions / Feedback? Email: stevenwadejr@gmail.com Twitter: @stevenwadejr
  72. 72. Steven Wade - @stevenwadejrhttps://joind.in/talk/4c853 Suggested Questions • Is it better to have dynamic stages (e.g. - conditions are run in advance to determine the steps) or pass all the steps and let the stage contain it's own condition? • When should one stage be broken into 2?

×