Interoperability
                            with RabbitMQ

                                     by Alvaro Videla
                            at Shanda Innovation - March 2010




Thursday, March 18, 2010
About Me

                    • Lead Developer at TheNetCircle.com
                    • Twitter: @old_sound
                    • Blog: http://obvioushints.blogspot.com/
                    •

Thursday, March 18, 2010
The
                           Interoperability
                               Problem


Thursday, March 18, 2010
The Problem



          Movie
         Publisher




Thursday, March 18, 2010
The Problem



          Movie            New
                           Movie
         Publisher




Thursday, March 18, 2010
The Problem
                                   Java   WebService
                                            XML




          Movie            New
                           Movie
         Publisher




Thursday, March 18, 2010
The Problem
                                   Java   WebService
                                            XML



                                   C++      Image
                                          Thumbnail
                                          Generator

          Movie            New
                           Movie
         Publisher




Thursday, March 18, 2010
The Problem
                                   Java   WebService
                                            XML



                                   C++      Image
                                          Thumbnail
                                          Generator

          Movie            New
                           Movie
         Publisher                 PHP    PHP Cache
                                          Generator




Thursday, March 18, 2010
The Problem
                                   Java   WebService
                                            XML



                                   C++      Image
                                          Thumbnail
                                          Generator

          Movie            New
                           Movie
         Publisher                 PHP    PHP Cache
                                          Generator



                                   PHP      Static
                                            HTML
                                           Publisher




Thursday, March 18, 2010
The Problem
                                   Java   WebService
                                            XML



                                   C++      Image
                                          Thumbnail
                                          Generator

          Movie            New
                           Movie
         Publisher                 PHP    PHP Cache
                                          Generator



                                   PHP      Static
                                            HTML
                                           Publisher



                                   PHP     Trailers
                                          Generator




Thursday, March 18, 2010
The Problem   SYSTEMS

                              WebService
                                XML



                                Image
                              Thumbnail
                              Generator




                              PHP Cache
                              Generator



                                Static
                                HTML
                               Publisher



                               Trailers
                              Generator




Thursday, March 18, 2010
The Problem           SYSTEMS

                                      WebService
                                        XML
     Written In Different Languages
                                        Image
                                      Thumbnail
                                      Generator




                                      PHP Cache
                                      Generator



                                        Static
                                        HTML
                                       Publisher



                                       Trailers
                                      Generator




Thursday, March 18, 2010
The Problem           SYSTEMS

                                      WebService
                                        XML
     Written In Different Languages
                                        Image
     Different Operating Systems      Thumbnail
                                      Generator




                                      PHP Cache
                                      Generator



                                        Static
                                        HTML
                                       Publisher



                                       Trailers
                                      Generator




Thursday, March 18, 2010
The Problem           SYSTEMS

                                      WebService
                                        XML
     Written In Different Languages
                                        Image
     Different Operating Systems      Thumbnail
                                      Generator



     Written By Different People      PHP Cache
                                      Generator



                                        Static
                                        HTML
                                       Publisher



                                       Trailers
                                      Generator




Thursday, March 18, 2010
The Problem           SYSTEMS

                                      WebService
                                        XML
     Written In Different Languages
                                        Image
     Different Operating Systems      Thumbnail
                                      Generator



     Written By Different People      PHP Cache
                                      Generator
     They Need To Share Information
                                        Static
                                        HTML
                                       Publisher



                                       Trailers
                                      Generator




Thursday, March 18, 2010
The Problem           SYSTEMS

                                      WebService
                                        XML
     Written In Different Languages
                                        Image
     Different Operating Systems      Thumbnail
                                      Generator



     Written By Different People      PHP Cache
                                      Generator
     They Need To Share Information
                                        Static
     They Need Real Time Data           HTML
                                       Publisher



                                       Trailers
                                      Generator




Thursday, March 18, 2010
The Problem           SYSTEMS

                                      WebService
                                        XML
     Written In Different Languages
                                        Image
     Different Operating Systems      Thumbnail
                                      Generator



     Written By Different People      PHP Cache
                                      Generator
     They Need To Share Information
                                        Static
     They Need Real Time Data           HTML
                                       Publisher


     They Need To Scale                Trailers
                                      Generator




Thursday, March 18, 2010
Is There a Solution?



Thursday, March 18, 2010
AMQP
                              and
                           RabbitMQ


Thursday, March 18, 2010
AMQP
                    • Advanced Message Queuing Protocol
                    • Suits Interoperability
                    • Completely Open Protocol
                    • Binary Protocol
                    • AMQP Model
                    • AMQP Wire Format
Thursday, March 18, 2010
AMQP Model

                    • Exchanges
                    • Message Queues
                    • Bindings
                    • Rules for binding them

Thursday, March 18, 2010
AMQP Wire Protocol


                    • Functional Layer
                    • Transport Layer


Thursday, March 18, 2010
Message Flow




              http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/chap-Messaging_Tutorial-Initial_Concepts.html




Thursday, March 18, 2010
Exchange Types

                    • Fanout
                    • Direct
                    • Topic


Thursday, March 18, 2010
Default Exchanges

                    • amqp.fanout
                    • amqp.direct
                    • amqp.topic


Thursday, March 18, 2010
http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts-
                                                                      Fanout_Exchange.html




Thursday, March 18, 2010
http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts-
                                                                       Direct_Exchange.html




Thursday, March 18, 2010
http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts-
                                                                       Topic_Exchange.html




Thursday, March 18, 2010
Sample usage




Thursday, March 18, 2010
Sample Usage II




Thursday, March 18, 2010
Other Advantages
                    • Fire and Forget
                    • Stop/Start consumers safely (no need for
                           last processed id)
                    • No need for Cronjobs locks
                    • Load Distribution
                    • Transactions
Thursday, March 18, 2010
RabbitMQ



Thursday, March 18, 2010
RabbitMQ

                    • Enterprise Messaging System
                    • Open Source MPL
                    • Written in Erlang/OTP
                    • Commercial Support

Thursday, March 18, 2010
Features

                    • Reliable and High Scalable
                    • Easy To install
                    • Easy To Cluster
                    • Runs on: Windows, Solaris, Linux, OSX
                    • AMQP 0.8

Thursday, March 18, 2010
Client Libraries

                    • Java
                    • .NET/C#
                    • Erlang
                    • Ruby, Python, PHP, Perl, AS3, Lisp, Scala

Thursday, March 18, 2010
Docs/Support

                    •      http://www.rabbitmq.com/documentation.html

                    •      http://dev.rabbitmq.com/wiki/

                    •      #rabbitmq at irc.freenode.net

                    •      http://www.rabbitmq.com/email-archive.html




Thursday, March 18, 2010
Our Setup:
                           Publishers - 28 PHP Frontend Machines




                                      2 PHP Machines
                                 running 8 consumers each

Thursday, March 18, 2010
Our Setup

                    • Deployed in mid 2009
                    • Reduced site SPOF
                    • 400.000+ messages routed a day (and
                           increasing)
                    • Very easy to install/deploy

Thursday, March 18, 2010
Easy To Install?



Thursday, March 18, 2010
Easy To Install?




                            http://gist.github.com/334219
Thursday, March 18, 2010
Show me the code!



Thursday, March 18, 2010
Resources:

                    •      http://www.rabbitmq.com/

                    •      https://www.amqp.org/confluence/display/AMQP/About+AMQP

                    •      http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/
                           Messaging_Tutorial/index.html

                    •      http://wso2.org/library/articles/using-axis2-c-amqp-transport-part-i

                    •      http://www.skaag.net/2010/03/12/rabbitmq-for-beginners/

                    •      http://github.com/careo/rabbitmq-erlang-client-examples




Thursday, March 18, 2010
Questions?



Thursday, March 18, 2010
Thanks!
                                      Alvaro Videla
                              http://twitter.com/old_sound
                           http://www.slideshare.net/old_sound




Thursday, March 18, 2010

Interoperability With RabbitMq

  • 1.
    Interoperability with RabbitMQ by Alvaro Videla at Shanda Innovation - March 2010 Thursday, March 18, 2010
  • 2.
    About Me • Lead Developer at TheNetCircle.com • Twitter: @old_sound • Blog: http://obvioushints.blogspot.com/ • Thursday, March 18, 2010
  • 3.
    The Interoperability Problem Thursday, March 18, 2010
  • 4.
    The Problem Movie Publisher Thursday, March 18, 2010
  • 5.
    The Problem Movie New Movie Publisher Thursday, March 18, 2010
  • 6.
    The Problem Java WebService XML Movie New Movie Publisher Thursday, March 18, 2010
  • 7.
    The Problem Java WebService XML C++ Image Thumbnail Generator Movie New Movie Publisher Thursday, March 18, 2010
  • 8.
    The Problem Java WebService XML C++ Image Thumbnail Generator Movie New Movie Publisher PHP PHP Cache Generator Thursday, March 18, 2010
  • 9.
    The Problem Java WebService XML C++ Image Thumbnail Generator Movie New Movie Publisher PHP PHP Cache Generator PHP Static HTML Publisher Thursday, March 18, 2010
  • 10.
    The Problem Java WebService XML C++ Image Thumbnail Generator Movie New Movie Publisher PHP PHP Cache Generator PHP Static HTML Publisher PHP Trailers Generator Thursday, March 18, 2010
  • 11.
    The Problem SYSTEMS WebService XML Image Thumbnail Generator PHP Cache Generator Static HTML Publisher Trailers Generator Thursday, March 18, 2010
  • 12.
    The Problem SYSTEMS WebService XML Written In Different Languages Image Thumbnail Generator PHP Cache Generator Static HTML Publisher Trailers Generator Thursday, March 18, 2010
  • 13.
    The Problem SYSTEMS WebService XML Written In Different Languages Image Different Operating Systems Thumbnail Generator PHP Cache Generator Static HTML Publisher Trailers Generator Thursday, March 18, 2010
  • 14.
    The Problem SYSTEMS WebService XML Written In Different Languages Image Different Operating Systems Thumbnail Generator Written By Different People PHP Cache Generator Static HTML Publisher Trailers Generator Thursday, March 18, 2010
  • 15.
    The Problem SYSTEMS WebService XML Written In Different Languages Image Different Operating Systems Thumbnail Generator Written By Different People PHP Cache Generator They Need To Share Information Static HTML Publisher Trailers Generator Thursday, March 18, 2010
  • 16.
    The Problem SYSTEMS WebService XML Written In Different Languages Image Different Operating Systems Thumbnail Generator Written By Different People PHP Cache Generator They Need To Share Information Static They Need Real Time Data HTML Publisher Trailers Generator Thursday, March 18, 2010
  • 17.
    The Problem SYSTEMS WebService XML Written In Different Languages Image Different Operating Systems Thumbnail Generator Written By Different People PHP Cache Generator They Need To Share Information Static They Need Real Time Data HTML Publisher They Need To Scale Trailers Generator Thursday, March 18, 2010
  • 18.
    Is There aSolution? Thursday, March 18, 2010
  • 19.
    AMQP and RabbitMQ Thursday, March 18, 2010
  • 20.
    AMQP • Advanced Message Queuing Protocol • Suits Interoperability • Completely Open Protocol • Binary Protocol • AMQP Model • AMQP Wire Format Thursday, March 18, 2010
  • 21.
    AMQP Model • Exchanges • Message Queues • Bindings • Rules for binding them Thursday, March 18, 2010
  • 22.
    AMQP Wire Protocol • Functional Layer • Transport Layer Thursday, March 18, 2010
  • 23.
    Message Flow http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/chap-Messaging_Tutorial-Initial_Concepts.html Thursday, March 18, 2010
  • 24.
    Exchange Types • Fanout • Direct • Topic Thursday, March 18, 2010
  • 25.
    Default Exchanges • amqp.fanout • amqp.direct • amqp.topic Thursday, March 18, 2010
  • 26.
  • 27.
  • 28.
  • 29.
  • 30.
  • 31.
    Other Advantages • Fire and Forget • Stop/Start consumers safely (no need for last processed id) • No need for Cronjobs locks • Load Distribution • Transactions Thursday, March 18, 2010
  • 32.
  • 33.
    RabbitMQ • Enterprise Messaging System • Open Source MPL • Written in Erlang/OTP • Commercial Support Thursday, March 18, 2010
  • 34.
    Features • Reliable and High Scalable • Easy To install • Easy To Cluster • Runs on: Windows, Solaris, Linux, OSX • AMQP 0.8 Thursday, March 18, 2010
  • 35.
    Client Libraries • Java • .NET/C# • Erlang • Ruby, Python, PHP, Perl, AS3, Lisp, Scala Thursday, March 18, 2010
  • 36.
    Docs/Support • http://www.rabbitmq.com/documentation.html • http://dev.rabbitmq.com/wiki/ • #rabbitmq at irc.freenode.net • http://www.rabbitmq.com/email-archive.html Thursday, March 18, 2010
  • 37.
    Our Setup: Publishers - 28 PHP Frontend Machines 2 PHP Machines running 8 consumers each Thursday, March 18, 2010
  • 38.
    Our Setup • Deployed in mid 2009 • Reduced site SPOF • 400.000+ messages routed a day (and increasing) • Very easy to install/deploy Thursday, March 18, 2010
  • 39.
  • 40.
    Easy To Install? http://gist.github.com/334219 Thursday, March 18, 2010
  • 41.
    Show me thecode! Thursday, March 18, 2010
  • 42.
    Resources: • http://www.rabbitmq.com/ • https://www.amqp.org/confluence/display/AMQP/About+AMQP • http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/ Messaging_Tutorial/index.html • http://wso2.org/library/articles/using-axis2-c-amqp-transport-part-i • http://www.skaag.net/2010/03/12/rabbitmq-for-beginners/ • http://github.com/careo/rabbitmq-erlang-client-examples Thursday, March 18, 2010
  • 43.
  • 44.
    Thanks! Alvaro Videla http://twitter.com/old_sound http://www.slideshare.net/old_sound Thursday, March 18, 2010