Theres a rabbit on my symfony

Theres a Rabbit on my Symfony RabbitMQ and Symfony2 Integration Álvaro Videla Symfony Live 2011Thursday, March 3, 2011

Who?Thursday, March 3, 2011

About Me • Development Manager at TheNetCircle.com • Blog: http://videlalvaro.github.com/ • Twitter: @old_soundThursday, March 3, 2011

About Me • Developer at Liip • Blog: http://videlalvaro.github.com/ • Twitter: @old_soundThursday, March 3, 2011

About Me Co-authoring RabbitMQ in Action http://bit.ly/rabbitmqThursday, March 3, 2011

Why Do I need RabbitMQ?Thursday, March 3, 2011

OrThursday, March 3, 2011

Why Do I need Messaging?Thursday, March 3, 2011

An Upload Picture Form as seen by:Thursday, March 3, 2011

The UserThursday, March 3, 2011

I don’t want to wait till your app resizes my image!Thursday, March 3, 2011

The Product OwnerThursday, March 3, 2011

Can we also notify the user friends when she uploads a new image?Thursday, March 3, 2011

Can we also notify the user friends when she uploads a new image? I forgot to mention we need it for tomorrow…Thursday, March 3, 2011

The SysadminThursday, March 3, 2011

Dumb! You’re delivering full size images! The bandwidth bill has tripled!Thursday, March 3, 2011

Dumb! You’re delivering full size images! The bandwidth bill has tripled! We need this ﬁxed for yesterday!Thursday, March 3, 2011

The Developer in the other teamThursday, March 3, 2011

I need to call your PHP stuff but from PythonThursday, March 3, 2011

I need to call your PHP stuff but from Python And also Java starting next weekThursday, March 3, 2011

YouThursday, March 3, 2011

FML!Thursday, March 3, 2011

Is there a solution?Thursday, March 3, 2011

RabbitMQ & AMQPThursday, March 3, 2011

AMQPThursday, March 3, 2011

AMQP • Advanced Message Queuing Protocol • Suits Interoperability • Completely Open Protocol • Binary Protocol • AMQP Model • AMQP Wire FormatThursday, March 3, 2011

AMQP Model • Exchanges • Message Queues • Bindings • Rules for binding themThursday, March 3, 2011

AMQP Wire Protocol • Functional Layer • Transport LayerThursday, March 3, 2011

Message Flow http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/chap-Messaging_Tutorial-Initial_Concepts.htmlThursday, March 3, 2011

Exchange Types • Fanout • Direct • TopicThursday, March 3, 2011

http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts- Fanout_Exchange.htmlThursday, March 3, 2011

http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts- Direct_Exchange.htmlThursday, March 3, 2011

http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts- Topic_Exchange.htmlThursday, March 3, 2011

Usage ScenariosThursday, March 3, 2011

Usage Scenarios • Batch ProcessingThursday, March 3, 2011

Usage Scenarios • Batch Processing • Image UploadingThursday, March 3, 2011

Usage Scenarios • Batch Processing • Image Uploading • Distributed LoggingThursday, March 3, 2011

Scenario Batch ProcessingThursday, March 3, 2011

RequirementsThursday, March 3, 2011

Requirements • Generate XMLThursday, March 3, 2011

Requirements • Generate XML • Distribution Over a ClusterThursday, March 3, 2011

Requirements • Generate XML • Distribution Over a Cluster • Elasticity - Add/Remove new workersThursday, March 3, 2011

Requirements • Generate XML • Distribution Over a Cluster • Elasticity - Add/Remove new workers • No Code ChangesThursday, March 3, 2011

DesignThursday, March 3, 2011

Publisher Code $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST); $channel = $conn->channel(); $channel->exchange_declare(video-desc-ex, direct, false, true, false); $msg = new AMQPMessage($video_info, array(content_type => text/plain, delivery_mode => 2)); $channel->basic_publish($msg, video-desc-ex); $channel->close(); $conn->close();Thursday, March 3, 2011

Consumer Code $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST); $channel = $conn->channel(); $channel->exchange_declare(video-desc-ex, direct, false, true, false); $channel->queue_declare(video-desc-queue, false, true, false, false); $channel->queue_bind(video-desc-queue, video-desc-ex); $channel->basic_consume(video-desc-queue, $consumer_tag, false, false, false, false, $consumer); while(count($channel->callbacks)) { $channel->wait(); }Thursday, March 3, 2011

Scenario Upload PicturesThursday, March 3, 2011

Requirements • Upload PictureThursday, March 3, 2011

Requirements • Upload Picture • Reward UserThursday, March 3, 2011

Requirements • Upload Picture • Reward User • Notify User FriendsThursday, March 3, 2011

Requirements • Upload Picture • Reward User • Notify User Friends • Resize PictureThursday, March 3, 2011

Requirements • Upload Picture • Reward User • Notify User Friends • Resize Picture • No Code ChangesThursday, March 3, 2011

Publisher Code $channel->exchange_declare(upload-pictures, fanout, false, true, false); $metadata = json_encode(array( image_id => $image_id, user_id => $user_id, ‘image_path => $image_path)); $msg = new AMQPMessage($metadata, array(content_type => application/json, delivery_mode => 2)); $channel->basic_publish($msg, upload-pictures);Thursday, March 3, 2011

Consumer Code $channel->exchange_declare(upload-pictures, fanout, false, true, false); $channel->queue_declare(resize-picture, false, true, false, false); $channel->queue_bind(resize-picture, upload-pictures); $channel->basic_consume(resize-picture, $consumer_tag, false, false, false, false, $consumer); while(count($channel->callbacks)) { $channel->wait(); }Thursday, March 3, 2011

Consumer Code $consumer = function($msg){ $meta = json_decode($msg->body, true); resize_picture($meta[image_id], $meta[image_path]); $msg->delivery_info[channel]-> basic_ack($msg->delivery_info[delivery_tag]); };Thursday, March 3, 2011

Scenario Distributed LoggingThursday, March 3, 2011

Requirements • Several Web ServersThursday, March 3, 2011

Requirements • Several Web Servers • Logic Separated by Module/ActionThursday, March 3, 2011

Requirements • Several Web Servers • Logic Separated by Module/Action • Several Log Levels:Thursday, March 3, 2011

Requirements • Several Web Servers • Logic Separated by Module/Action • Several Log Levels: • Info, Warning, ErrorThursday, March 3, 2011

Requirements • Several Web Servers • Logic Separated by Module/Action • Several Log Levels: • Info, Warning, Error • Add/Remove log listeners at willThursday, March 3, 2011

Publisher Code $channel->exchange_declare(logs, topic, false, true, false); $msg = new AMQPMessage(some log message, array(content_type => text/plain)); $channel->basic_publish($msg, logs, server1.user.profile.info);Thursday, March 3, 2011

Consumer Code Get messages sent by host: server1Thursday, March 3, 2011

Consumer Code $channel->exchange_declare(logs, topic, false, true, false); $channel->queue_declare(server1-logs, false, true, false, false); $channel->queue_bind(server1-logs, logs, server1.#);Thursday, March 3, 2011

Consumer Code Get all error messagesThursday, March 3, 2011

Consumer Code $channel->exchange_declare(logs, topic, false, true, false); $channel->queue_declare(error-logs, false, true, false, false); $channel->queue_bind(error-logs, logs, #.error);Thursday, March 3, 2011

Symfony IntegrationThursday, March 3, 2011

RabbitMQ BundleThursday, March 3, 2011

RabbitMQ Bundle Goal: To be easy to useThursday, March 3, 2011

RabbitMQ Bundle Publish a Message: $msg = array(user_id => 1235, image_path => /path/to/new/ pic.png); $this->get(rabbitmq.upload_picture_producer)-> publish(serialize($msg));Thursday, March 3, 2011

RabbitMQ Bundle Consume a Message: $ ./app/console_dev rabbitmq:consumer -m 1 upload_pictureThursday, March 3, 2011

RabbitMQ Bundle Consume 50 Messages: $ ./app/console_dev rabbitmq:consumer -m 50 upload_pictureThursday, March 3, 2011

RabbitMQ Bundle Consume inﬁnite* Messages: $ ./app/console_dev rabbitmq:consumer -m -1 upload_picture *messages may not be inﬁniteThursday, March 3, 2011

RabbitMQ Bundle Conﬁguration: # app/config/config.yml rabbitmq.config: connections: default: host: localhost port: 5672 user: guest password: guest vhost: /Thursday, March 3, 2011

RabbitMQ Bundle Conﬁguration: # app/config/config.yml rabbitmq.config: connections: ... producers: upload_picture: connection: default exchange_options: {name: upload-picture, type: direct}Thursday, March 3, 2011

RabbitMQ Bundle Conﬁguration: # app/config/config.yml rabbitmq.config: connections: ... consumers: upload_picture: connection: default exchange_options: {name: upload-picture, type: direct} queue_options: {name: upload-picture} callback: upload_picture_serviceThursday, March 3, 2011

RabbitMQ Bundle Using a Producer: public function indexAction($name) { ... $msg = array(user_id => 1235, image_path => /path/to/new/pic.png); $this->get(rabbitmq.upload_picture_producer)-> publish(serialize($msg)); ... }Thursday, March 3, 2011

RabbitMQ Bundle Consumer Class: class UploadPictureConsumer extends ContainerAware implements ConsumerInterface { public function execute($msg) { // Process picture upload. // $msg will be what was published // from the Controller. } }Thursday, March 3, 2011

RabbitMQ Bundle Recap:Thursday, March 3, 2011

RabbitMQ Bundle Recap: • Add an entry for the consumer/producer in the conﬁguration.Thursday, March 3, 2011

RabbitMQ Bundle Recap: • Add an entry for the consumer/producer in the conﬁguration. • Implement your Callback.Thursday, March 3, 2011

RabbitMQ Bundle Recap: • Add an entry for the consumer/producer in the conﬁguration. • Implement your Callback. • Start the consumer from the CLI.Thursday, March 3, 2011

RabbitMQ Bundle Recap: • Add an entry for the consumer/producer in the conﬁguration. • Implement your Callback. • Start the consumer from the CLI. • Add code to publish messages.Thursday, March 3, 2011

RabbitMQ Bundle There’s more:Thursday, March 3, 2011

RabbitMQ Bundle There’s more: • RPC clients and serversThursday, March 3, 2011

RabbitMQ Bundle There’s more: • RPC clients and servers • Anonymous consumersThursday, March 3, 2011

RabbitMQ Bundle There’s more: • RPC clients and servers • Anonymous consumers • Parallel RPCThursday, March 3, 2011

RabbitMQ Bundle Fork it at http://github.com/videlalvaro/RabbitMqBundleThursday, March 3, 2011

Why RabbitMQ?Thursday, March 3, 2011

RabbitMQ • Enterprise Messaging System • Open Source MPL • Written in Erlang/OTP • Commercial SupportThursday, March 3, 2011

Features • Reliable and High Scalable • Easy To install • Easy To Cluster • Runs on: Windows, Solaris, Linux, OSX • AMQP 0.8 - 0.9.1Thursday, March 3, 2011

Client Libraries • Java • .NET/C# • Erlang • Ruby, Python, PHP, Perl, AS3, Lisp, Scala, Clojure, HaskellThursday, March 3, 2011

Docs/Support • http://www.rabbitmq.com/documentation.html • http://dev.rabbitmq.com/wiki/ • #rabbitmq at irc.freenode.net • http://www.rabbitmq.com/email-archive.htmlThursday, March 3, 2011

One Setup for HAThursday, March 3, 2011

ConclusionThursday, March 3, 2011

Conclusion • FlexibilityThursday, March 3, 2011

Conclusion • Flexibility • ScalabilityThursday, March 3, 2011

Conclusion • Flexibility • Scalability • InteroperabilityThursday, March 3, 2011

Conclusion • Flexibility • Scalability • Interoperability • Reduce OpsThursday, March 3, 2011

Questions?Thursday, March 3, 2011

Thanks! Álvaro Videla http://twitter.com/old_sound http://github.com/videlalvaro http://github.com/tnc http://www.slideshare.net/old_soundThursday, March 3, 2011

http://www.lafermeduweb.net	2191
http://www.willdurand.fr	699
http://www.symfony.es	162
http://www.symfonylab.com	149
http://blogdwich.fr	140
http://pocky.github.com	101
http://willdurand.fr	19
http://feeds.feedburner.com	14
http://www.emmanuelpereira.com	11
http://www.sfexception.com	6
http://webcache.googleusercontent.com	6
http://pocky.github.io	4
http://translate.googleusercontent.com	3
http://m.lafermeduweb.net	1
http://coderwall.com	1
http://feedproxy.google.com	1

Theres a rabbit on my symfony

by Alvaro Videla on Mar 03, 2011

Accessibility

Categories

Upload Details

Usage Rights

16 Embeds 3,508

Statistics

Theres a rabbit on my symfony Presentation Transcript