How to optimize background processes - when Sylius meets Blackfire

How to optimize background processes
when Sylius meets Black
fi
re
Photo by Laura Ockel on Unsplash

Introduction
01
Photo by Mikhail Vasilyev on Unsplash

Photo by Ryan Quintal on Unsplash
Feature overview

sylius.com
4
Product variants

Always have Y Channel Pricings,

Where Y equals amount of Channels in shop
Product

May have from 1 to N variants
Channel Pricing

Contains end pricing for customer
Sylius pricing model

sylius.com
Discounting rules set
5
Main con
fi
guration

Dates

Channels

Enabled

Priority

Exclusiveness
Processing scope

Variants

Products

Taxonomy
Resulting actions

Fixed discount

Percentage discount

sylius.com
6
1 - 10
To process updates of

Products through UI interface
Minimal processable scope

As a business rule
10k - 100k
Default behaviour should

Be consider for these values
Typical Sylius project

Based on the clients we are working the most
Over 1kk
Feature should be possible

To be used with this scope
Maximal processable scope

From the projects we know
Dataset size

Photo by Brendan Church on Unsplash
Initial dilema

sylius.com
8
Take all eligible catalog promotions

This step can be outsourced to DB
Fetch variants for given promotion

To ensure that only correct ones applies
Apply promotion for given variants

Go to the next promotion

Execute second step
Catalog Promotion
Processing order
Take all product variants

Take all eligible catalog promotions

Apply promotion for given variant

Product Variant

Performance test setup
Photo by Loubna Aggoun on Unsplash

sylius.com
?
Dataset

7000 Product Variants

3 Catalog Promotions covering all of products at least once

2 Channels
Execution on single catalog promotion update

Triggering CatalogPromotionUpdated event
Environment

Production environment

No debug option

Removal of cache and cache warmup before each execution
<
E
10

sylius.com
11
Command Batching
Doctrine

Fetching only codes to avoid hydration

Flushing & transaction out-of-the-box

Clearance of entity manager
Symfony Messenger

No need for custom command

Flushing & transaction out-of-the-box

Sync & async switch is piece of cake

Photo by Shane Aldendorff on Unsplash
Let’s improve!
02

Photo by Bastien Plu on Unsplash
First round - bug

sylius.com
23
public function assignLocale(
TranslatableInterface $translatableEntity
): void {
$fallbackLocale = $this
->translationLocaleProvider
->getDefaultLocaleCode()
;
try {
$currentLocale = $this->localeContext->getLocaleCode();
} catch (LocaleNotFoundException) {
$currentLocale = $fallbackLocale;
}
$translatableEntity->setCurrentLocale($currentLocale);
$translatableEntity->setFallbackLocale($fallbackLocale);
}

sylius.com
24
): void {
;
try {
}
}

sylius.com
25
): void {
;
try {
}
}

sylius.com
26
): void {
;
try {
}
}

sylius.com
27
): void {
;
try {
}
}

sylius.com
28
if ($this->commandBasedChecker->isExecutedFromCLI()) {
$translatableEntity->setCurrentLocale($fallbackLocale);
return;
}
try {
}

Photo by Pietro Mattia on Unsplash
Second round - N+1

sylius.com
41
public function findByCodes(array $codes): array
{
return $this->createQueryBuilder('o')
->andWhere('o.code IN (:codes)')
->setParameter('codes', $codes)
->getQuery()
->getResult()
;
}

sylius.com
42
{
->addSelect('product')
->addSelect('channelPricings')
->addSelect('appliedPromotions')
->addSelect('productTaxon')
->addSelect('taxon')
->leftJoin('o.channelPricings', 'channelPricings')
->leftJoin('channelPricings.appliedPromotions', 'appliedPromotions')
->leftJoin('o.product', 'product')
->leftJoin('product.productTaxons', 'productTaxon')
->leftJoin('productTaxon.taxon', 'taxon')
->getQuery()
->getResult()
;
}

sylius.com
43
{
->addSelect('product')
->addSelect('channelPricings')
->addSelect('appliedPromotions')
->addSelect('productTaxon')
->addSelect('taxon')
->leftJoin('o.channelPricings', 'channelPricings')
->leftJoin('channelPricings.appliedPromotions', 'appliedPromotions')
->leftJoin('o.product', 'product')
->leftJoin('product.productTaxons', 'productTaxon')
->leftJoin('productTaxon.taxon', 'taxon')
->getQuery()
->getResult()
;
}

Photo by Ralfs Blumbergs on Unsplash
Third round - EM Clear

sylius.com
52
Up to this moment

All operations were executed synchronously

sylius.com
54
After implementation of EntityManagerClearer and
switching to async processing

Photo by Bastien Plu on Unsplash
Summary

Photo by Glen Rushton on Unsplash
Final round - native query

sylius.com
64
private function clearChannelPricing(
ChannelPricingInterface $channelPricing
): void {
if ($channelPricing->getAppliedPromotions()->isEmpty()) {
return;
}
if ($channelPricing->getOriginalPrice() !== null) {
$channelPricing->setPrice($channelPricing->getOriginalPrice());
}
$channelPricing->clearAppliedPromotions();
}

sylius.com
65
): void {
return;
}
}
}

sylius.com
66
): void {
return;
}
}
}

sylius.com
67
): void {
return;
}
}
}

sylius.com
68
public function __invoke(
ApplyCatalogPromotionsOnVariants $updateVariants
): void {
$catalogPromotions = $this->catalogPromotionsProvider->provide();
$variants = $this->productVariantRepository->findByCodes(
$updateVariants->variantsCodes
);
foreach ($variants as $variant) {
$this->clearer->clearVariant($variant);
foreach ($catalogPromotions as $promotion) {
$this->catalogPromotionApplicator->applyOnVariant(
$variant,
$promotion
);
}
}
}

sylius.com
69
): void {
);
$variant,
$promotion
);
}
}
}

sylius.com
70
): void {
);
$variant,
$promotion
);
}
}
}

sylius.com
71
): void {
);
$variant,
$promotion
);
}
}
}

sylius.com
72
): void {
);
$variant,
$promotion
);
}
}
}

sylius.com
73
): void {
);
$variant,
$promotion
);
}
}
}

sylius.com
74
): void {
);
$variant,
$promotion
);
}
}
}

sylius.com
75
}
$codes = array_column($updateVariants->variantsCodes, 'code');
$connection = $this->entityManager->getConnection();
$connection->executeQuery('
DELETE scpcp FROM sylius_channel_pricing_catalog_promotions scpcp
JOIN sylius_channel_pricing scp ON scpcp.channel_pricing_id = scp.id
JOIN sylius_product_variant spv ON scp.product_variant_id = spv.id
WHERE spv.code IN (?)’,
[$codes],
[Connection::PARAM_STR_ARRAY]
);
// Apply Catalog promotion
}

sylius.com
76
}
[$codes],
);
}

sylius.com
77
}
[$codes],
);
}

sylius.com
78
}
[$codes],
);
}

sylius.com
79
}
[$codes],
);
}

sylius.com
80
}
[$codes],
);
}

sylius.com
81
}
[$codes],
);
}

sylius.com
82
}
[$codes],
);
}

sylius.com
83
}
[$codes],
);
}

sylius.com
88
It didn’t work according to acceptance tests!

sylius.com
92
ClearCatalogPromotionsOnVariants $clearCatalogPromotionsOnVariants
): void {
$codes = array_column($clearCatalogPromotionsOnVariants->variantsCodes, ‘code');
UPDATE sylius_channel_pricing scp
INNER JOIN sylius_channel_pricing_catalog_promotions scpcp ON
scpcp.channel_pricing_id = scp.id
SET scp.price=scp.original_price
WHERE spv.code IN (?);', [$codes], [Connection::PARAM_STR_ARRAY]);
WHERE spv.code IN (?)', [$codes], [Connection::PARAM_STR_ARRAY]);
}

sylius.com
93
): void {
}

sylius.com
94
): void {
}

sylius.com
95
): void {
}

sylius.com
96
): void {
}

sylius.com
?
101
What if we change

the catalog promotion

con
fi
guration?

Photo by Mikhail Vasilyev on Unsplash
03
Takeaways

sylius.com
109
Check the highest class in your callgraph

And look for bugs
Remember to clear Entity Manager when batch processing

Unless you are executing asynchronously messages with Symfony Messenger
Load all data required to perform operation

But not more
Pure DQL may bring bene
fi
ts

But is hard to do it properly for complicated objects or mix it with ORM

sylius.com
110
SyliusCon
This autumn - at 27th of October

In a Sylius hometown: Łódź

sylius.com
111
Thank you!
Photo by Maksym Harbar on Unsplash

How to optimize background processes - when Sylius meets Blackfire

More Related Content

What's hot

Similar to How to optimize background processes - when Sylius meets Blackfire

More from Łukasz Chruściel

Recently uploaded

How to optimize background processes - when Sylius meets Blackfire