Your SlideShare is downloading. ×
0
RabbitMQ is the new king
Jan Machacek | Alvaro Videla

© 2013 SpringOne 2GX. All rights reserved. Do not distribute withou...
Jan Machacek
•
•
•

Works at Cake Solutions

•
•

Editor of the Open Source Journal

Author of Pro Spring, Pro Spring 2.5 ...
Alvaro Videla
•
•
•
•
•

Developer Advocate at Pivotal / RabbitMQ
Co-Author of RabbitMQ in Action
Creator of the RabbitMQ ...
Why do we
need messaging?
Classic Web Apps
Implement a
Photo Gallery
Two Parts:
Pretty Simple
‘Till new
requirements arrive
The Product Owner
Can we also notify the user
friends when she uploads a new
image?
Can we also notify the user
friends when she uploads a new
image?
I forgot to mention we need it for tomorrow…
The Social Media Guru
We need to give badges to
users for each picture upload
We need to give badges to
users for each picture upload
and post uploads to Twitter
The Sysadmin
Dumb! You’re delivering full size
images!
The bandwidth bill has tripled!
Dumb! You’re delivering full size
images!
The bandwidth bill has tripled!
We need this fixed for yesterday!
The Developer in the other
team
I need to call your Java stuff but
from Python
I need to call your PHP stuff but
from Python
And also PHP starting next week
The User
I don’t want to wait
till your app resizes
my image!
You
(╯°□°)╯︵ ┻━┻
Let’s see the
code evolution
Pseudo Code
Comments

%% image_controller
handle('PUT', "/user/image", ReqData) ->
image_handler:do_upload(ReqData:get_fil...
Pseudo Code
Function Name

%% image_controller
handle('PUT', "/user/image", ReqData) ->
image_handler:do_upload(ReqData:ge...
Pseudo Code
Arguments

%% image_controller
handle('PUT', "/user/image", ReqData) ->
image_handler:do_upload(ReqData:get_fi...
Pseudo Code
Function Body

%% image_controller
handle('PUT', "/user/image", ReqData) ->
image_handler:do_upload(ReqData:ge...
Pseudo Code
Return Value

%% image_controller
handle('PUT', "/user/image", ReqData) ->
image_handler:do_upload(ReqData:get...
First Implementation:
%% image_controller
handle('PUT', "/user/image", ReqData) ->
image_handler:do_upload(ReqData:get_fil...
Second Implementation:
%% image_controller
handle('PUT', "/user/image", ReqData) ->
{ok, Image} = image_handler:do_upload(...
Third Implementation:
%% image_controller
handle('PUT', "/user/image", ReqData) ->
{ok, Image} = image_handler:do_upload(R...
Fourth Implementation:
%% image_controller
handle('PUT', "/user/image", ReqData) ->
{ok, Image} = image_handler:do_upload(...
Final Implementation:
%% image_controller
handle('PUT', "/user/image", ReqData) ->
{ok, Image} = image_handler:do_upload(R...
Can our code scale to new
requirements?
What if
What if
• We need to speed up image conversion
What if
• We need to speed up image conversion
• User notifications sent by email
What if
• We need to speed up image conversion
• User notifications sent by email
• Stop tweeting about new images
What if
• We need to speed up image conversion
• User notifications sent by email
• Stop tweeting about new images
• Resize...
What if
• We need to speed up image conversion
• User notifications sent by email
• Stop tweeting about new images
• Resize...
Can we do better?
Sure.
Using messaging
Design
Publish / Subscribe Pattern
First Implementation:
%% image_controller
handle('PUT', "/user/image", ReqData) ->
{ok, Image} = image_handler:do_upload(R...
First Implementation:
%% image_controller
handle('PUT', "/user/image", ReqData) ->
{ok, Image} = image_handler:do_upload(R...
First Implementation:
%% image_controller
handle('PUT', "/user/image", ReqData) ->
{ok, Image} = image_handler:do_upload(R...
First Implementation:
%% image_controller
handle('PUT', "/user/image", ReqData) ->
{ok, Image} = image_handler:do_upload(R...
Second Implementation:
Second Implementation:

THIS PAGE INTENTIONALLY LEFT BLANK
What is RabbitMQ
RabbitMQ
RabbitMQ
• Multi Protocol Messaging Server
RabbitMQ
• Multi Protocol Messaging Server
• Open Source (MPL)
RabbitMQ
• Multi Protocol Messaging Server
• Open Source (MPL)
• Polyglot
RabbitMQ
• Multi Protocol Messaging Server
• Open Source (MPL)
• Polyglot
• Written in Erlang/OTP
Multi Protocol

http://bit.ly/rmq-protocols
Polyglot
Polyglot
• Java
Polyglot
• Java
• node.js
Polyglot
• Java
• node.js
• Erlang
Polyglot
• Java
• node.js
• Erlang
• PHP
Polyglot
• Java
• node.js
• Erlang
• PHP
• Ruby
Polyglot
• Java
• node.js
• Erlang
• PHP
• Ruby
• .Net
Polyglot
• Java
• node.js
• Erlang
• PHP
• Ruby
• .Net
• Haskell
Polyglot

Even COBOL!!!11
Some users of RabbitMQ
Some users of RabbitMQ
•

Instagram
Some users of RabbitMQ
•
•

Instagram
Indeed.com
Some users of RabbitMQ
•
•
•

Instagram
Indeed.com
MailboxApp
Some users of RabbitMQ
•
•
•
•

Instagram
Indeed.com
MailboxApp
Mercado Libre
Messaging with RabbitMQ
A demo with the RabbitMQ Simulator

https://github.com/RabbitMQSimulator/RabbitMQSimulator
What are we building?
What are we building?
Server

AMQP

CV
What are we building?
Server

AMQP

CV
RecogService
mjpegChunk(CorrelationId)
(ChunkData): Unit
imageChunk(CorrelationId)
(ChunkData): Unit

RecogServiceActivato...
Demo
RecogService
recogChannel: MessageChannel
mjpegChunk(ChunkData): Unit
imageChunk(ChunkData): Unit

DirectChannel

int:chai...
RecogController
recogService: RecogService
mjpegChunk(SessionId,
ChunkData): Unit
imageChunk(SessionId,
ChunkData): Unit

...
RecogSessions

WebSocket
send

Messaging
WebSocketHandler

send

SubProtocol
WebSocketHandler

send

MessageSendingOperati...
Questions

https://github.com/eigengo/springone2gx2013
RabbitMQ is the new King
RabbitMQ is the new King
Upcoming SlideShare
Loading in...5
×

RabbitMQ is the new King

2,196

Published on

Speakers: Jan Machacek and Alvaro Videla
Jan will show how to use RabbitMQ to connect components on different platforms. After a few introductory slides explaining the main concepts, the rest of the talk will be live code. The coding will begin by showing Spring Integration code to connect to RabbitMQ to send and receive messages--think byte[] values back and forth to start with. Jan will then show the real power of RabbitMQ by replacing the RPC server component by native code in C++: we will write image processing component. To make matters even better--faster--we will then use CUDA to perform the image processing. Come to Jan's talk and demo if you want to find out how to use RabbitMQ in the core of your application, especially if your application needs to tie together code on completely different platforms and if it performs heavy number crunching or image processing. You do not need to have deep knowledge of Spring Integration, AMQP or OpenCV, …; all that you will need is your inner geek.

Published in: Technology, Business
0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,196
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
63
Comments
0
Likes
9
Embeds 0
No embeds

No notes for slide

Transcript of "RabbitMQ is the new King"

  1. 1. RabbitMQ is the new king Jan Machacek | Alvaro Videla © 2013 SpringOne 2GX. All rights reserved. Do not distribute without permission.
  2. 2. Jan Machacek • • • Works at Cake Solutions • • Editor of the Open Source Journal Author of Pro Spring, Pro Spring 2.5 and other books & articles Author of Akka Patterns, Specs2 Spring, Scalad; Spring Extensions, Spock Spring Integration @honzam399 | janm@cakesolutions.net github.com/janm399
  3. 3. Alvaro Videla • • • • • Developer Advocate at Pivotal / RabbitMQ Co-Author of RabbitMQ in Action Creator of the RabbitMQ Simulator Blogs about RabbitMQ Internals: http://videlalvaro.github.io/internals.html @old_sound | avidela@gopivotal.com github.com/videlalvaro
  4. 4. Why do we need messaging?
  5. 5. Classic Web Apps
  6. 6. Implement a Photo Gallery
  7. 7. Two Parts:
  8. 8. Pretty Simple
  9. 9. ‘Till new requirements arrive
  10. 10. The Product Owner
  11. 11. Can we also notify the user friends when she uploads a new image?
  12. 12. Can we also notify the user friends when she uploads a new image? I forgot to mention we need it for tomorrow…
  13. 13. The Social Media Guru
  14. 14. We need to give badges to users for each picture upload
  15. 15. We need to give badges to users for each picture upload and post uploads to Twitter
  16. 16. The Sysadmin
  17. 17. Dumb! You’re delivering full size images! The bandwidth bill has tripled!
  18. 18. Dumb! You’re delivering full size images! The bandwidth bill has tripled! We need this fixed for yesterday!
  19. 19. The Developer in the other team
  20. 20. I need to call your Java stuff but from Python
  21. 21. I need to call your PHP stuff but from Python And also PHP starting next week
  22. 22. The User
  23. 23. I don’t want to wait till your app resizes my image!
  24. 24. You
  25. 25. (╯°□°)╯︵ ┻━┻
  26. 26. Let’s see the code evolution
  27. 27. Pseudo Code Comments %% image_controller handle('PUT', "/user/image", ReqData) -> image_handler:do_upload(ReqData:get_file()), ok.
  28. 28. Pseudo Code Function Name %% image_controller handle('PUT', "/user/image", ReqData) -> image_handler:do_upload(ReqData:get_file()), ok.
  29. 29. Pseudo Code Arguments %% image_controller handle('PUT', "/user/image", ReqData) -> image_handler:do_upload(ReqData:get_file()), ok.
  30. 30. Pseudo Code Function Body %% image_controller handle('PUT', "/user/image", ReqData) -> image_handler:do_upload(ReqData:get_file()), ok.
  31. 31. Pseudo Code Return Value %% image_controller handle('PUT', "/user/image", ReqData) -> image_handler:do_upload(ReqData:get_file()), ok.
  32. 32. First Implementation: %% image_controller handle('PUT', "/user/image", ReqData) -> image_handler:do_upload(ReqData:get_file()), ok.
  33. 33. Second Implementation: %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), resize_image(Image), ok.
  34. 34. Third Implementation: %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), resize_image(Image), notify_friends(ReqData:get_user()), ok.
  35. 35. Fourth Implementation: %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), resize_image(Image), notify_friends(ReqData:get_user()), add_points_to_user(ReqData:get_user()), ok.
  36. 36. Final Implementation: %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), resize_image(Image), notify_friends(ReqData:get_user()), add_points_to_user(ReqData:get_user()), tweet_new_image(User, Image), ok.
  37. 37. Can our code scale to new requirements?
  38. 38. What if
  39. 39. What if • We need to speed up image conversion
  40. 40. What if • We need to speed up image conversion • User notifications sent by email
  41. 41. What if • We need to speed up image conversion • User notifications sent by email • Stop tweeting about new images
  42. 42. What if • We need to speed up image conversion • User notifications sent by email • Stop tweeting about new images • Resize in different formats
  43. 43. What if • We need to speed up image conversion • User notifications sent by email • Stop tweeting about new images • Resize in different formats • Swap Language / Technology (No Down Time)
  44. 44. Can we do better?
  45. 45. Sure. Using messaging
  46. 46. Design Publish / Subscribe Pattern
  47. 47. First Implementation: %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), Msg = #msg{user = ReqData:get_user(), image = Image}, publish_message('new_image', Msg).
  48. 48. First Implementation: %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), Msg = #msg{user = ReqData:get_user(), image = Image}, publish_message('new_image', Msg). %% friends notifier on('new_image', Msg) -> notify_friends(Msg.user, Msg.image).
  49. 49. First Implementation: %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), Msg = #msg{user = ReqData:get_user(), image = Image}, publish_message('new_image', Msg). %% friends notifier on('new_image', Msg) -> notify_friends(Msg.user, Msg.image). %% points manager on('new_image', Msg) -> add_points(Msg.user, 'new_image').
  50. 50. First Implementation: %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), Msg = #msg{user = ReqData:get_user(), image = Image}, publish_message('new_image', Msg). %% friends notifier on('new_image', Msg) -> notify_friends(Msg.user, Msg.image). %% points manager on('new_image', Msg) -> add_points(Msg.user, 'new_image'). %% resizer on('new_image', Msg) -> resize_image(Msg.image).
  51. 51. Second Implementation:
  52. 52. Second Implementation: THIS PAGE INTENTIONALLY LEFT BLANK
  53. 53. What is RabbitMQ
  54. 54. RabbitMQ
  55. 55. RabbitMQ • Multi Protocol Messaging Server
  56. 56. RabbitMQ • Multi Protocol Messaging Server • Open Source (MPL)
  57. 57. RabbitMQ • Multi Protocol Messaging Server • Open Source (MPL) • Polyglot
  58. 58. RabbitMQ • Multi Protocol Messaging Server • Open Source (MPL) • Polyglot • Written in Erlang/OTP
  59. 59. Multi Protocol http://bit.ly/rmq-protocols
  60. 60. Polyglot
  61. 61. Polyglot • Java
  62. 62. Polyglot • Java • node.js
  63. 63. Polyglot • Java • node.js • Erlang
  64. 64. Polyglot • Java • node.js • Erlang • PHP
  65. 65. Polyglot • Java • node.js • Erlang • PHP • Ruby
  66. 66. Polyglot • Java • node.js • Erlang • PHP • Ruby • .Net
  67. 67. Polyglot • Java • node.js • Erlang • PHP • Ruby • .Net • Haskell
  68. 68. Polyglot Even COBOL!!!11
  69. 69. Some users of RabbitMQ
  70. 70. Some users of RabbitMQ • Instagram
  71. 71. Some users of RabbitMQ • • Instagram Indeed.com
  72. 72. Some users of RabbitMQ • • • Instagram Indeed.com MailboxApp
  73. 73. Some users of RabbitMQ • • • • Instagram Indeed.com MailboxApp Mercado Libre
  74. 74. Messaging with RabbitMQ A demo with the RabbitMQ Simulator https://github.com/RabbitMQSimulator/RabbitMQSimulator
  75. 75. What are we building?
  76. 76. What are we building? Server AMQP CV
  77. 77. What are we building? Server AMQP CV
  78. 78. RecogService mjpegChunk(CorrelationId) (ChunkData): Unit imageChunk(CorrelationId) (ChunkData): Unit RecogServiceActivator onCoinResponse(CorrelationId, CoinResponse): Unit
  79. 79. Demo
  80. 80. RecogService recogChannel: MessageChannel mjpegChunk(ChunkData): Unit imageChunk(ChunkData): Unit DirectChannel int:chain ChunkData Collection[FrameData] FrameData FrameData FrameData FrameData Array[Byte] String int:chain ServiceActivating Handler ChunkDecoder decodeFrame(ChunkData): Collection[FrameData] DefaultMessage Splitter AmqpOutbound Endpoint ObjectToString Transformer ServiceActivating Handler RecogServiceActivator onCoinResponse(CorrelationId, CoinResponse): Unit
  81. 81. RecogController recogService: RecogService mjpegChunk(SessionId, ChunkData): Unit imageChunk(SessionId, ChunkData): Unit RecogService mjpegChunk(CorrelationId) (ChunkData): Unit imageChunk(CorrelationId) (ChunkData): Unit RecogServiceActivator recogSessions: RecogSessions onCoinResponse(CorrelationId, CoinResponse): Unit RecogSessions sessions: Map[SessionId, CoinResponse] messageSender: MessageSendingOperations[...] sessionEnded(SessionId): Unit onCoinResponse(CorrelationId, CoinResponse): Unit
  82. 82. RecogSessions WebSocket send Messaging WebSocketHandler send SubProtocol WebSocketHandler send MessageSendingOperations[] send SubscribableChannel webSocketHandlerChannel subscribe send subscribe RecogService SimpleBroker MessageHandler Annotation MessageHandler DispatcherServlet SubscribableChannel dispatchChannel subscribe send RecogController HTTP
  83. 83. Questions https://github.com/eigengo/springone2gx2013
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×