With the recent Sylius release v1.11, we've introduced a brand new feature - catalog promotions. As this feature may require processing a significant amount of data(the whole product catalog), we had to decide on async processing by default. But it does not mean that we can waste resources and not optimize the processing algorithm! We had to improve it and to do that we've been using Blackfire.

1. How to
How to
optimize
optimize
background
background
processes
processes
when Sylius meets Blackfire
when Sylius meets Blackfire

2. Introduction
Introduction

6. QUIZ
QUIZ

7. Feature overview
Feature overview

8. Sylius pricing
Sylius pricing
model
model

9. Sylius pricing model
Sylius pricing model
Product
Product
May have from 1 to N variants

10. Sylius pricing model
Sylius pricing model
Product
Product
May have from 1 to N variants
Product variants
Product variants
Always have Y Channel Pricings,
Where Y equals amount of Channels in shop

11. Sylius pricing model
Sylius pricing model
Product
Product
May have from 1 to N variants
Product variants
Product variants
Always have Y Channel Pricings,
Where Y equals amount of Channels in shop
Channel Pricing
Channel Pricing
Contains end pricing for customer

12. Discounting rules set
Discounting rules set

13. Main
Main
configuration
configuration
Processing
Processing
scope
scope
Channel
Channel
Pricing
Pricing

14. Main
Main
configuration
configuration
Dates
Channels
Enabled
Priority
Exclusiveness
Processing
Processing
scope
scope
Channel
Channel
Pricing
Pricing

15. Main
Main
configuration
configuration
Dates
Channels
Enabled
Priority
Exclusiveness
Processing
Processing
scope
scope
Variants
Products
Taxonomy
Channel
Channel
Pricing
Pricing

16. Main
Main
configuration
configuration
Dates
Channels
Enabled
Priority
Exclusiveness
Processing
Processing
scope
scope
Variants
Products
Taxonomy
Channel
Channel
Pricing
Pricing
Fixed discount
Percentage
discount

17. Tons of
Tons of
configuration
configuration

18. Dataset size
Dataset size
1 - 10
1 - 10
Minimal processable scope
10k - 100k
10k - 100k
Typical Sylius project
Typical Sylius project
Over 1kk
Over 1kk
Maximal processable scope

19. Performance test
Performance test
setup
setup

20. Dataset
Dataset
7000 Product Variants

21. Dataset
Dataset
7000 Product Variants
3 Catalog Promotions covering all of
products at least once

22. Dataset
Dataset
7000 Product Variants
3 Catalog Promotions covering all of
products at least once
2 Channels

23. Execution on single
Execution on single
catalog promotion update
catalog promotion update

24. Environment
Environment
Production environment

25. Environment
Environment
Production environment
No debug option

26. Environment
Environment
Production environment
No debug option
Removal of cache and cache warmup
before each execution

27. Let's improve
Let's improve

34. First round - bug
First round - bug

39. public function assignLocale(
TranslatableInterface $translatableEntity
): void {
$fallbackLocale = $this
->translationLocaleProvider
->getDefaultLocaleCode()
;
try {
$currentLocale = $this->localeContext->getLocaleCode();
} catch (LocaleNotFoundException) {
$currentLocale = $fallbackLocale;
}
$translatableEntity->setCurrentLocale($currentLocale);
$translatableEntity->setFallbackLocale($fallbackLocale);
}

40. public function assignLocale(
TranslatableInterface $translatableEntity
): void {
$fallbackLocale = $this
->translationLocaleProvider
->getDefaultLocaleCode()
;
try {
$currentLocale = $this->localeContext->getLocaleCode();
} catch (LocaleNotFoundException) {
$currentLocale = $fallbackLocale;
}
$translatableEntity->setCurrentLocale($currentLocale);
$translatableEntity->setFallbackLocale($fallbackLocale);
}

41. public function assignLocale(
TranslatableInterface $translatableEntity
): void {
$fallbackLocale = $this
->translationLocaleProvider
->getDefaultLocaleCode()
;
try {
$currentLocale = $this->localeContext->getLocaleCode();
} catch (LocaleNotFoundException) {
$currentLocale = $fallbackLocale;
}
$translatableEntity->setCurrentLocale($currentLocale);
$translatableEntity->setFallbackLocale($fallbackLocale);
}

42. public function assignLocale(
TranslatableInterface $translatableEntity
): void {
$fallbackLocale = $this
->translationLocaleProvider
->getDefaultLocaleCode()
;
try {
$currentLocale = $this->localeContext->getLocaleCode();
} catch (LocaleNotFoundException) {
$currentLocale = $fallbackLocale;
}
$translatableEntity->setCurrentLocale($currentLocale);
$translatableEntity->setFallbackLocale($fallbackLocale);
}

43. public function assignLocale(
TranslatableInterface $translatableEntity
): void {
$fallbackLocale = $this
->translationLocaleProvider
->getDefaultLocaleCode()
;
try {
$currentLocale = $this->localeContext->getLocaleCode();
} catch (LocaleNotFoundException) {
$currentLocale = $fallbackLocale;
}
$translatableEntity->setCurrentLocale($currentLocale);
$translatableEntity->setFallbackLocale($fallbackLocale);
}

44. public function assignLocale(
TranslatableInterface $translatableEntity
): void {
$fallbackLocale = $this
->translationLocaleProvider
->getDefaultLocaleCode()
;
if ($this->commandBasedChecker->isExecutedFromCLI()) {
$translatableEntity->setCurrentLocale($fallbackLocale);
$translatableEntity->setFallbackLocale($fallbackLocale);
return;
}
try {
$currentLocale = $this->localeContext->getLocaleCode();
} catch (LocaleNotFoundException) {
$currentLocale = $fallbackLocale;
}
$translatableEntity->setCurrentLocale($currentLocale);
$translatableEntity->setFallbackLocale($fallbackLocale);
}

45. QUIZ
QUIZ

51. Conclusion
Conclusion
Correctly placed "if"
Correctly placed "if" may
may
boost your app
boost your app
significantly
significantly

52. Second round -
Second round -
N+1
N+1

53. Lazy loading
Lazy loading

59. public function findByCodes(
array $codes
): array {
return $this->createQueryBuilder('o')
->andWhere('o.code IN (:codes)')
->setParameter('codes', $codes)
->getQuery()
->getResult()
;
}

60. public function findByCodes(array $codes): array
{
return $this->createQueryBuilder('o')
->addSelect('product')
->addSelect('channelPricings')
->addSelect('appliedPromotions')
->addSelect('productTaxon')
->addSelect('taxon')
->leftJoin('o.channelPricings', 'channelPricings')
->leftJoin('channelPricings.appliedPromotions',
'appliedPromotions')
->leftJoin('o.product', 'product')
->leftJoin('product.productTaxons', 'productTaxon')
->leftJoin('productTaxon.taxon', 'taxon')
->andWhere('o.code IN (:codes)')
->setParameter('codes', $codes)
->getQuery()
->getResult()
;
}

61. public function findByCodes(array $codes): array
{
return $this->createQueryBuilder('o')
->addSelect('product')
->addSelect('channelPricings')
->addSelect('appliedPromotions')
->addSelect('productTaxon')
->addSelect('taxon')
->leftJoin('o.channelPricings', 'channelPricings')
->leftJoin('channelPricings.appliedPromotions',
'appliedPromotions')
->leftJoin('o.product', 'product')
->leftJoin('product.productTaxons', 'productTaxon')
->leftJoin('productTaxon.taxon', 'taxon')
->andWhere('o.code IN (:codes)')
->setParameter('codes', $codes)
->getQuery()
->getResult()
;
}

62. Quiz
Quiz

74. Conclusion
Conclusion
Lazy loading
Lazy loading boosts your
boosts your
app
app
but
but
can be your enemy
can be your enemy

75. Third round - EM Clear
Third round - EM Clear

77. Quiz
Quiz

79. Conclusion
Conclusion
Clearing entity manager
Clearing entity manager
may
may boost your batch
boost your batch
processing
processing

80. Summary
Summary

81. Quiz
Quiz

84. Final round - native query
Final round - native query

87. private function clearChannelPricing(
ChannelPricingInterface $channelPricing
): void {
if ($channelPricing->getAppliedPromotions()->isEmpty()) {
return;
}
if ($channelPricing->getOriginalPrice() !== null) {
$channelPricing->setPrice($channelPricing->getOriginalPrice());
}
$channelPricing->clearAppliedPromotions();
}

88. private function clearChannelPricing(
ChannelPricingInterface $channelPricing
): void {
if ($channelPricing->getAppliedPromotions()->isEmpty()) {
return;
}
if ($channelPricing->getOriginalPrice() !== null) {
$channelPricing->setPrice($channelPricing->getOriginalPrice());
}
$channelPricing->clearAppliedPromotions();
}

89. private function clearChannelPricing(
ChannelPricingInterface $channelPricing
): void {
if ($channelPricing->getAppliedPromotions()->isEmpty()) {
return;
}
if ($channelPricing->getOriginalPrice() !== null) {
$channelPricing->setPrice($channelPricing->getOriginalPrice());
}
$channelPricing->clearAppliedPromotions();
}

90. private function clearChannelPricing(
ChannelPricingInterface $channelPricing
): void {
if ($channelPricing->getAppliedPromotions()->isEmpty()) {
return;
}
if ($channelPricing->getOriginalPrice() !== null) {
$channelPricing->setPrice($channelPricing->getOriginalPrice());
}
$channelPricing->clearAppliedPromotions();
}

91. $codes = array_column($updateVariants->variantsCodes, 'code');
$connection = $this->entityManager->getConnection();
$connection->executeQuery(
'DELETE scpcp FROM sylius_channel_pricing_catalog_promotions scpcp
JOIN sylius_channel_pricing scp ON scpcp.channel_pricing_id = scp.id
JOIN sylius_product_variant spv ON scp.product_variant_id = spv.id
WHERE spv.code IN (?)',
[$codes],
[Connection::PARAM_STR_ARRAY]
);
foreach ($variants as $variant) {
// Apply Catalog promotion
}

92. $codes = array_column($updateVariants->variantsCodes, 'code');
$connection = $this->entityManager->getConnection();
$connection->executeQuery(
'DELETE scpcp FROM sylius_channel_pricing_catalog_promotions scpcp
JOIN sylius_channel_pricing scp ON scpcp.channel_pricing_id = scp.id
JOIN sylius_product_variant spv ON scp.product_variant_id = spv.id
WHERE spv.code IN (?)',
[$codes],
[Connection::PARAM_STR_ARRAY]
);
foreach ($variants as $variant) {
// Apply Catalog promotion
}

93. $codes = array_column($updateVariants->variantsCodes, 'code');
$connection = $this->entityManager->getConnection();
$connection->executeQuery(
'DELETE scpcp FROM sylius_channel_pricing_catalog_promotions scpcp
JOIN sylius_channel_pricing scp ON scpcp.channel_pricing_id = scp.id
JOIN sylius_product_variant spv ON scp.product_variant_id = spv.id
WHERE spv.code IN (?)',
[$codes],
[Connection::PARAM_STR_ARRAY]
);
foreach ($variants as $variant) {
// Apply Catalog promotion
}

94. $codes = array_column($updateVariants->variantsCodes, 'code');
$connection = $this->entityManager->getConnection();
$connection->executeQuery(
'DELETE scpcp FROM sylius_channel_pricing_catalog_promotions scpcp
JOIN sylius_channel_pricing scp ON scpcp.channel_pricing_id = scp.id
JOIN sylius_product_variant spv ON scp.product_variant_id = spv.id
WHERE spv.code IN (?)',
[$codes],
[Connection::PARAM_STR_ARRAY]
);
foreach ($variants as $variant) {
// Apply Catalog promotion
}

95. $codes = array_column($updateVariants->variantsCodes, 'code');
$connection = $this->entityManager->getConnection();
$connection->executeQuery(
'DELETE scpcp FROM sylius_channel_pricing_catalog_promotions scpcp
JOIN sylius_channel_pricing scp ON scpcp.channel_pricing_id = scp.id
JOIN sylius_product_variant spv ON scp.product_variant_id = spv.id
WHERE spv.code IN (?)',
[$codes],
[Connection::PARAM_STR_ARRAY]
);
foreach ($variants as $variant) {
// Apply Catalog promotion
}

96. $codes = array_column($updateVariants->variantsCodes, 'code');
$connection = $this->entityManager->getConnection();
$connection->executeQuery(
'DELETE scpcp FROM sylius_channel_pricing_catalog_promotions scpcp
JOIN sylius_channel_pricing scp ON scpcp.channel_pricing_id = scp.id
JOIN sylius_product_variant spv ON scp.product_variant_id = spv.id
WHERE spv.code IN (?)',
[$codes],
[Connection::PARAM_STR_ARRAY]
);
foreach ($variants as $variant) {
// Apply Catalog promotion
}

97. $codes = array_column($updateVariants->variantsCodes, 'code');
$connection = $this->entityManager->getConnection();
$connection->executeQuery(
'DELETE scpcp FROM sylius_channel_pricing_catalog_promotions scpcp
JOIN sylius_channel_pricing scp ON scpcp.channel_pricing_id = scp.id
JOIN sylius_product_variant spv ON scp.product_variant_id = spv.id
WHERE spv.code IN (?)',
[$codes],
[Connection::PARAM_STR_ARRAY]
);
foreach ($variants as $variant) {
// Apply Catalog promotion
}

98. $codes = array_column($updateVariants->variantsCodes, 'code');
$connection = $this->entityManager->getConnection();
$connection->executeQuery(
'DELETE scpcp FROM sylius_channel_pricing_catalog_promotions scpcp
JOIN sylius_channel_pricing scp ON scpcp.channel_pricing_id = scp.id
JOIN sylius_product_variant spv ON scp.product_variant_id = spv.id
WHERE spv.code IN (?)',
[$codes],
[Connection::PARAM_STR_ARRAY]
);
foreach ($variants as $variant) {
// Apply Catalog promotion
}

99. Quiz
Quiz

116. Conclusion
Conclusion
Always
Always have tests
have tests
It is hard to beat libraries
It is hard to beat libraries
properly
properly

117. Takeaways
Takeaways

118. Check the highest class in your call-graph
Check the highest class in your call-graph
And look for bugs
Load all data required to perform operation
Load all data required to perform operation
But not more
Remember to clear Entity Manager when batch processing
Remember to clear Entity Manager when batch processing
Unless you are executing asynchronously messages with Symfony
Messenger
Pure DQL may bring benefits
Pure DQL may bring benefits
But is hard to do it properly for complicated objects or mix it with ORM
Amount of queries matter
Amount of queries matter
Especially in managed databases

119. Thank you!
Thank you!