WebGriffe is a digital agency with 5+ years of experience and 5 certified Magento developers. They have 350+ customers and have created 20+ Magento extensions. Their presentation discusses queueing asynchronous tasks in Magento to improve performance. They demonstrate how to build a queueing system with a queue manager and workers using a queue extension they developed called Lilmuckers_Queue.

2. Webgriffe
Tailored Digital Works
|webgriffe.com @webgriffe
5+ Years of Experience with Magento
5 Certified Developers (Zend & Magento)
350+ Customers
20+ Magento Extensions
450+ Extensions Sold

3. Simple Scenario

4. Complex Scenario

5. Can we go over this limit?
Yes!
Queueing several syncrounous tasks allows us to go further.

6. Separation of components with
Queue Manager & Workers
Queue Manager: collects & assigns tasks to workers
Workers: execute tasks (with repeat-until logic)

7. Response time improvments
Save 30% of the time by queuing index update on product save
Magento CE v. 1.8.1.0 with sample data. Index update on save enabled. Data collected with Xhprof profiler.

8. Online Hackathon Worldwide Experience
www.mage-hackathon.de/passed/online-hackathon-worldwide-31st-jan-1st-feb.html
4 continents
10+ countries
100+ people
80+ open source Magento projects
github.com/magento-hackathon

9. Online Hackathon Worldwide Experience
Lilmuckers_Queue
( )
Beanstalk
( )
Your custom logic
github.com/lilmuckers/magento-lilmuckers_queue
kr.github.io/beanstalkd

10. Which queue manager to use?
The ones we can easly integrate with Magento are:
Beanstalk ( )
Amazon SQS ( )
Gearman ( )
kr.github.io/beanstalkd
aws.amazon.com/sqs
gearman.org

11. Lilmuckers_Queue Extension
Defining a Queue Backend (for example Beanstalk)
<?xmlversion="1.0"?>
<!--app/etc/local.xml-->
<config>
<global>
...
<queue>
<backend>beanstalkd</backend>
<beanstalkd>
<servers>
<server>
<host>127.0.0.1</host>
</server>
</servers>
</beanstalkd>
</queue>
...
</global>
</config>

12. Lilmuckers_Queue Extension
Defining a Queue
<?xmlversion="1.0"?>
<!--app/code/local/MyVendor/MyModule/etc/config.xml-->
<config>
...
<queues>
<queueName>
<label>TheQueueName</label>
<class>module/queueHandler</class>
<workers>
<taskName>
<class>module/worker</class>
<method>methodName</method>
</taskName>
</workers>
</queueName>
</queues>
...
</config>

13. Lilmuckers_Queue Extension
Adding tasks to Queue
$_queue=Mage::helper('lilqueue')->getQueue('queueName');
$_task=Mage::helper('lilqueue')->createTask(
'taskName',
array('data'=>'toprovide','to'=>'theworker'),
$storeToRunAs
);
//Optionally
$_task->setPriority(100)
->setDelay(60)
->setTtr(60);
$_queue->addTask($_task);

14. Lilmuckers_Queue Extension
Worker
classMy_Module_Model_WorkerextendsLilmuckers_Queue_Model_Worker_Abstract
{
publicfunctionmethodName(Lilmuckers_Queue_Model_Queue_Task$task)
{
//getthestoreassignedwiththetask
$store=$task->getStore();
//getthequeuehandlerforthisqueue
$queue=$task->getQueue();
//getthedataassignedwiththetask
$data=$task->getData();
//Thistaskendedproperly
$task->success();
//thistaskneedstoberepeated
$task->retry();
//thistaskerroredandweshoulddropitfromthequeue
$task->hold();
//thisworkeristakingalongtime,weshouldextendit
$task->touch();
}
}
$php/path/to/magento/shell/queue.php--watch<queues>

15. Queueing Magento Index Update
Open source code on GitHub
buildbuild passingpassing
github.com/webgriffe/index-queue-extension

16. Webgriffe_IndexQueue
Extension
Architecture

17. The same can be done with
other components or tasks
Queues are for…
Entity index update
Cache cleaning
Stock update
Rendering email templates
…
Any task which the user doesn't need to wait for

18. Any Question?
Webgriffe
Tailored Digital Works
|webgriffe.com @webgriffe
5+ Years of Experience with Magento
5 Certified Developers (Zend & Magento)
350+ Customers
20+ Magento Extensions
450+ Extensions Sold

19. Thank you!