Asynchronous processing with PHP and Symfony2. Do it simple

12,766 views

Published on

The presentation for Symfony Camp UA 2013 about asynchronous processing.

We will talk about a near real-time processing of a business logic which does not require synchronous actions.

How to do it fast.
How to do it simple.
And how to do not ask PHP to struggle through alone.

We will analyze our experience, bad decisions and will try to get Zen of simplicity.

Published in: Technology, Economy & Finance
0 Comments
11 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
12,766
On SlideShare
0
From Embeds
0
Number of Embeds
107
Actions
Shares
0
Downloads
44
Comments
0
Likes
11
Embeds 0
No embeds

No notes for slide
  • {}
  • Asynchronous processing with PHP and Symfony2. Do it simple

    1. 1. Asynchronous processing with PHP and Symfony2 Do it simple Kirill chEbba Chebunin
    2. 2. Overview. The Problem REQUEST RESPONSE Expected Response Complete Logic Kirill chEbba Chebunin
    3. 3. Overview. Examples • Data transfer and manipulation ! • File upload! • Video conversion! • Massive business logic ! • Billing! • Social! • 3rd-party communication! • API calls ! • Notifications Kirill chEbba Chebunin
    4. 4. Overview. The Solution REQUEST RESPONSE Required Logic Remaining Logic Kirill chEbba Chebunin
    5. 5. Experience. The Real Project http://hopeliesat24framespersecond.files.wordpress.com/2011/12/love_actually_movie_image_bill_nighy_01.jpg Kirill chEbba Chebunin
    6. 6. Experience. Billing Partners Billing Aggregator Service Providers Kirill chEbba Chebunin
    7. 7. Experience. Billing. 2012 Worker Worker Worker Worker Master Worker Worker Kirill chEbba Chebunin
    8. 8. Experience. Billing. Revolution Made b y Vasily coyl Ku lakov http://darkroom.baltimoresun.com/wp-content/uploads/2012/05/REU-CUBA-MAYDAY-1.jpg Kirill chEbba Chebunin
    9. 9. Theory http://marketmybook.in/wp-content/uploads/2013/05/Library-Books.jpg Kirill chEbba Chebunin
    10. 10. Theory. Master-Worker Worker Worker Master Worker Kirill chEbba Chebunin
    11. 11. Theory. Master-Worker. Conclusions • Process control! • Resources in workers! • Long living workers! • Memory control! • Signal support Kirill chEbba Chebunin
    12. 12. Theory. RabbitMQ http://www.rabbitmq.com/img/tutorials/intro/hello-world-example-routing.png Kirill chEbba Chebunin
    13. 13. RabbitMQ • 1 exchange => multiple queries = topic! • Query = Worker type! • Durable exchanges/queries! • Persistent messages! • autoAck=false Kirill chEbba Chebunin
    14. 14. Experience. Billing. Review http://bi.gazeta.pl/im/84/96/d8/z14194308Q,Najglupsze-pomysly-mistrzow-prowizorki.jpg Kirill chEbba Chebunin
    15. 15. New world. Master • Master process => supervisor! • Start workers! • Worker count! • Signaled shutdown! • Auto-restart on failure! • Memory control! • …………. Kirill chEbba Chebunin
    16. 16. New world. Supervisor Example config ! [program:billing_statistics] process_name=%(program_name)s_%(process_num)d numprocs=4 stdout_logfile=/var/log/supervisor/%(program_name)s.out.log stderr_logfile=/var/log/supervisor/%(program_name)s.error.log command=billing rabbitmq:consumer statistics autorestart=true autostart=true Kirill chEbba Chebunin
    17. 17. New world. Publish/Subscribe • oldsound/rabbitmq-bundle! • Exchange/Queue setup! • Producers! • Consumers! • Command = worker! • Events! • Special event name! • Producer Listener Kirill chEbba Chebunin
    18. 18. Experience. Billing. 2013 Worker Worker Worker Worker Worker Worker Supervisor Kirill chEbba Chebunin
    19. 19. Profit? Kirill chEbba Chebunin
    20. 20. Future. Publish/Subscribe. Improve! • Send any event to async processing! • Use listeners as workers! • Different message bus Kirill chEbba Chebunin
    21. 21. Future Kirill chEbba Chebunin
    22. 22. EventBand EventBand http://baadu.ru/ph/8/polosy_na_stene_1280x1024.jpg Kirill chEbba Chebunin
    23. 23. EventBand. Overview Dispatcher Band#1 Listener Listener Band#2 Listener Band#3 Event Listener Listener Listener Listener Listener Listener Listener Band#4 Listener Listener Band#5 Listener Listener Band#6 Listener Current Listener Default Band Kirill chEbba Chebunin
    24. 24. EventBand. Architecture PublishListener Listener Listener Listener Listener T R A N S P O R T Listener Band#1 Listener Listener Band#2 Listener Listener Band#3 Listener Kirill chEbba Chebunin
    25. 25. EventBand. Architecture • Dispatcher! • Band support! • Abstract Subscribers! • Adapters (Symfony, ZF, …)! • Transport! • Setup! • Publisher! • Storage! • Routing! • Consumer! • Adapters (RabbitMQ, ActiveMQ, Gearman, …) Kirill chEbba Chebunin
    26. 26. EventBand. Utilities • Event Serialization! • Native! • JMS! • Event Routing! • By name! • By properties (PropertyAccess) Kirill chEbba Chebunin
    27. 27. EventBand. Example. Code Dispatch: ! $dispatcher->dispatch('foo.bar', new FooEvent($foo, $bar)); ! ………………………………………………………… ! Subscribe: ! public function logFooBar(FooEvent $event) { $this->service->saveLog($event->getFoo(), $event->getBar()); } ! ! <service id="log_subscriber" class="FooBarLogSubscriber"> <tag name="kernel.event_subscriber" band=“foo.bar.log" /> </service> Kirill chEbba Chebunin
    28. 28. EventBand. Example. Config transports: amqp: driver: amqplib connections: default: ~ exchanges: foo: ~ queues: foo.bar.log: bind: foo: "foo.bar" ! publishers: foo: events: "foo.#" ! consumers: foo.bar.log: ~ Kirill chEbba Chebunin
    29. 29. EventBand. Wanna Play? https://packagist.org/packages/event-band/ Kirill chEbba Chebunin
    30. 30. Experience. Patterns • Identified Event! • id, time, sequence, process! • logging (sqlite)! • Delayed Event! • original event, delay! • delay time => queue! • ordered queue! • sleep until delay Kirill chEbba Chebunin
    31. 31. Experience. Troubleshooting • Doctrine! • SqlLogger! • Unit of Work! • flush, detach! • Array properties Kirill chEbba Chebunin
    32. 32. Questions? Cool Image with Cats and Boobs Kirill chEbba Chebunin iam@chebba.org https://github.com/chEbba https://twitter.com/iamchEbba Kirill chEbba Chebunin

    ×