AMQP
Advanced Message Queueing Protocol
Caveats
Event Logging
Legacy Architecture & Problem
Event Logging
Client

Client

Server

MongoDB

Client
Connection Issues
80
Client

110
Client

Server

MongoDB

65

Client
Connection Issues
80
Client

110
Client

255
Server

MongoDB

65

Client
Server MIA
0

Client

0

0

Server

Client

MongoDB

0

Client
Network Segmentation
0

Client

0

Server

Client

MongoDB

0

Client
Total data loss
DATA LOSS?!
Why AMQP?
• Stable and mature	

• Designed for SX markets	

• Widely supported (good interop)	

• Modular, Fast & Flexible
AMQP Overview
AMQP

Service Model
Wire

Advanced Message Queueing Protocol
AMQP Overview
AMQP

AMQ Model

Wire
AMQP Overview
AMQP

AMQ Model

Server-side service model	

defined components	

rules for wiring

Wire
AMQP Overview
AMQP

AMQ Model

Wire

Network wire-level protocol	

command architecture	

connections, channels, etc
AMQ Model

AMQ Model
Exchange
Queues
Binding
AMQ Model
AMQ Model

Exchange

Queues

Binding
AMQ Model
Virtual Host

AMQ Model

Exchange

Queues

Binding
AMQ Model
Virtual Host

AMQ Model

Publisher

Exchange

Queues

Binding

Consumer
AMQ Model
Virtual Host

Exchange

Publisher

Queues

Binding

Consumer
AMQ Model
Virtual Host
Router
Message

Publisher

Exchange

Queues

Binding

Consumer
AMQ Model
Virtual Host
Router
Message

Publisher

Exchange

Queues

Binding

Consumer
AMQ Model
Virtual Host
Router
Message

Publisher

Exchange

?

Binding

Queues

Consumer
AMQ Model
Virtual Host
Router
Message

Publisher

Exchange

?

Binding

Queues

Consumer
AMQ Model
Virtual Host
Router
Message

Publisher

Exchange

Queues

Binding
Queue:
critical_error_queue	

Exchange:
errors...
AMQ Model
Virtual Host
Router
Message

Publisher

Exchange

Queues

Binding
Queue:
critical_error_queue	

Exchange:
errors...
Important notes
Publisher

‣ can create

exchanges and
queues

Exchange

‣ routes messages based on

criteria	

‣ doesn’t ...
The Exchange
Exchange
The Exchange
Exchange

Types	

Headers
System
Fanout
Topic
(routing algo)
The Exchange
Exchange

Types	

(routing algo)

Direct

Fanout

Topic

Headers

System
The Exchange
Exchange

Types	

(routing algo)

Direct

Fanout

Topic

Headers

System
Direct Type Exchange
Exchange	

‘events’
Direct Type Exchange
Message	


event.user.click

Message	


event.user.view

Message	


event.user.share

Exchange	

‘eve...
Direct Type Exchange
Message	


event.user.click
user_click_queue

Message	


event.user.view

Message	


event.user.share...
Direct Type Exchange
Message	


event.user.click
user_click_queue

Message	


event.user.view

Exchange	

‘events’

Messag...
Direct Type Exchange
Message	


event.user.click
user_click_queue

Message	


event.user.view

Exchange	

‘events’

Messag...
Direct Type Exchange
Message	


event.user.click
user_click_queue

Message	


event.user.view

Exchange	

‘events’

Messag...
Direct Type Exchange
Message	


PHP Script

event.user.click
user_click_queue

Message	


event.user.view

Exchange	

‘eve...
Direct Type Exchange
Message	


event.user.click
user_click_queue

Message	


event.user.view

Message	


Exchange	

‘even...
Yeah, so?
What if…
user_interaction_queue

Message	


event.user.click

Message	


event.user.view

Message	


event.user.share

Exc...
What if…
user_interaction_queue

Message	


event.user.click

Message	


event.user.view

Message	


event.user.share

Exc...
What if…
user_interaction_queue

Message	


event.user.click

Message	


event.user.view

Message	


event.user.share

Exc...
Topic Exchange
Message	


event.user.click

Message	


event.user.view

user_interaction_queue

Exchange	

‘events’

Messa...
Topic Exchange
Message	


PHP Script

event.user.click

Message	


event.user.view

user_interaction_queue

Exchange	

‘ev...
Topic Exchange
user_interaction_queue

Message	


event.user.click

Message	


event.user.view

Message	


event.user.shar...
Can be done at runtime
Topic Exchange
user_interaction_queue

Message	


event.user.click

Message	


event.user.view

Message	


Message
Message...
Topic Exchange
user_interaction_queue

Message	


event.user.click

Message	


event.user.view

Message	


Message
Message...
Topic Exchange
user_interaction_queue

Message	


event.user.click

Message	


event.user.view

Message	


Message
Message...
I’m still pissed
New Event Logging
Explorations & Implementation
Revisiting Our Logging
Client

Client

Client

MongoDB
Event Logging Revised
Client
MQ

Client

MongoDB
MQ

Client
MQ
Event Logging Revised
Client
MQ

Client

MongoDB
MQ

Client
MQ
Event Logging Revised
Client
MQ

Client

MongoDB
MQ

Client
MQ
No data loss!
SRSLY?!
Event Logging Revised
80

Client
MQ

110

MQ

Client
MQ

65

Client
MQ

MongoDB
Event Logging Revised
80

Client
MQ

110

MQ

Client
MQ

65

Client
MQ

MongoDB
Event Logging Revised
80

Client
MQ

110

MQ

Client
MQ

65

Client

MQ
MQ

MongoDB
Event Logging Revised
80

Client
MQ

MQ
110

Client

MongoDB
MQ

MQ
65

Client
MQ
Remember Topic Exchange?
user_interaction_queue

Message	


event.user.click

Message	


event.user.view

Message	


Messa...
Provides flexibility
80

MongoDB

Client
MQ

110

MQ

Client

Reporting	

SQL

MQ

65

Client

Notification
MQ
Not Covered
•

durable	


•

auto-delete	


•

passive	


•

qos	


•

no-wait	


•

ack	


•

exclusive	


•

message rec...
A quick demo of AMQP
with RabbitMQ
Thank you
weekeat@peazie.com
AMQP for phpMelb
Upcoming SlideShare
Loading in...5
×

AMQP for phpMelb

1,103

Published on

Shorter version of AMQP presentation for Melbourne PHP Users Group.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,103
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

AMQP for phpMelb

  1. 1. AMQP Advanced Message Queueing Protocol
  2. 2. Caveats
  3. 3. Event Logging Legacy Architecture & Problem
  4. 4. Event Logging Client Client Server MongoDB Client
  5. 5. Connection Issues 80 Client 110 Client Server MongoDB 65 Client
  6. 6. Connection Issues 80 Client 110 Client 255 Server MongoDB 65 Client
  7. 7. Server MIA 0 Client 0 0 Server Client MongoDB 0 Client
  8. 8. Network Segmentation 0 Client 0 Server Client MongoDB 0 Client
  9. 9. Total data loss
  10. 10. DATA LOSS?!
  11. 11. Why AMQP? • Stable and mature • Designed for SX markets • Widely supported (good interop) • Modular, Fast & Flexible
  12. 12. AMQP Overview AMQP Service Model Wire Advanced Message Queueing Protocol
  13. 13. AMQP Overview AMQP AMQ Model Wire
  14. 14. AMQP Overview AMQP AMQ Model Server-side service model defined components rules for wiring Wire
  15. 15. AMQP Overview AMQP AMQ Model Wire Network wire-level protocol command architecture connections, channels, etc
  16. 16. AMQ Model AMQ Model Exchange Queues Binding
  17. 17. AMQ Model AMQ Model Exchange Queues Binding
  18. 18. AMQ Model Virtual Host AMQ Model Exchange Queues Binding
  19. 19. AMQ Model Virtual Host AMQ Model Publisher Exchange Queues Binding Consumer
  20. 20. AMQ Model Virtual Host Exchange Publisher Queues Binding Consumer
  21. 21. AMQ Model Virtual Host Router Message Publisher Exchange Queues Binding Consumer
  22. 22. AMQ Model Virtual Host Router Message Publisher Exchange Queues Binding Consumer
  23. 23. AMQ Model Virtual Host Router Message Publisher Exchange ? Binding Queues Consumer
  24. 24. AMQ Model Virtual Host Router Message Publisher Exchange ? Binding Queues Consumer
  25. 25. AMQ Model Virtual Host Router Message Publisher Exchange Queues Binding Queue: critical_error_queue Exchange: errors_exchange Filter: *.error.critical Consumer
  26. 26. AMQ Model Virtual Host Router Message Publisher Exchange Queues Binding Queue: critical_error_queue Exchange: errors_exchange Filter: *.error.critical Consumer
  27. 27. Important notes Publisher ‣ can create exchanges and queues Exchange ‣ routes messages based on criteria ‣ doesn’t store messages ‣ can inspect message content ‣ can be created at runtime consumers Queues store messages named bound-able to exchange criteria can be created at runtime by consumers ‣ ‣ ‣ ‣ ‣ Binding ‣ creates a relationship between Consumer ‣ can create queues and exchanges exchanges and ‣ contains criteria and properties queues ‣ can be created at runtime by consumers
  28. 28. The Exchange Exchange
  29. 29. The Exchange Exchange Types Headers System Fanout Topic (routing algo)
  30. 30. The Exchange Exchange Types (routing algo) Direct Fanout Topic Headers System
  31. 31. The Exchange Exchange Types (routing algo) Direct Fanout Topic Headers System
  32. 32. Direct Type Exchange Exchange ‘events’
  33. 33. Direct Type Exchange Message event.user.click Message event.user.view Message event.user.share Exchange ‘events’
  34. 34. Direct Type Exchange Message event.user.click user_click_queue Message event.user.view Message event.user.share Exchange ‘events’
  35. 35. Direct Type Exchange Message event.user.click user_click_queue Message event.user.view Exchange ‘events’ Message event.user.share Binding ➡Queue: ‣ user_click_queue
  36. 36. Direct Type Exchange Message event.user.click user_click_queue Message event.user.view Exchange ‘events’ Message event.user.share Binding ➡Queue: ‣ user_click_queue ➡Exchange: ‣ events
  37. 37. Direct Type Exchange Message event.user.click user_click_queue Message event.user.view Exchange ‘events’ Message event.user.share Binding ➡Queue: ‣ user_click_queue ➡Exchange: ‣ events ➡Routing Key: ‣ event.user.click
  38. 38. Direct Type Exchange Message PHP Script event.user.click user_click_queue Message event.user.view Exchange ‘events’ PHP Script Message PHP Script event.user.share Binding ➡Queue: ‣ user_click_queue ➡Exchange: ‣ events ➡Routing Key: ‣ event.user.click
  39. 39. Direct Type Exchange Message event.user.click user_click_queue Message event.user.view Message Exchange ‘events’ user_view_queue event.user.share user_share_queue
  40. 40. Yeah, so?
  41. 41. What if… user_interaction_queue Message event.user.click Message event.user.view Message event.user.share Exchange ‘events’ user_shares_queue
  42. 42. What if… user_interaction_queue Message event.user.click Message event.user.view Message event.user.share Exchange ‘events’ user_shares_queue
  43. 43. What if… user_interaction_queue Message event.user.click Message event.user.view Message event.user.share Exchange ‘events’ user_shares_queue
  44. 44. Topic Exchange Message event.user.click Message event.user.view user_interaction_queue Exchange ‘events’ Message event.user.share Binding
  45. 45. Topic Exchange Message PHP Script event.user.click Message event.user.view user_interaction_queue Exchange ‘events’ PHP Script Message PHP Script event.user.share Binding ➡Queue: ‣ user_interaction_queue ➡Exchange: ‣ events ➡Routing Key: ‣ event.user.*
  46. 46. Topic Exchange user_interaction_queue Message event.user.click Message event.user.view Message event.user.share Message Message event.user.* Exchange ‘events’ event.user.share user_shares_queue Message
  47. 47. Can be done at runtime
  48. 48. Topic Exchange user_interaction_queue Message event.user.click Message event.user.view Message Message Message event.user.* Exchange ‘events’ event.user.share user_shares_queue event.user.share Click vs Share Conversion? Message
  49. 49. Topic Exchange user_interaction_queue Message event.user.click Message event.user.view Message Message Message event.user.* Exchange ‘events’ event.user.share user_shares_queue Message event.user.share event.user.share event.user.click click_share_queue Click vs Share Conversion? Message Message
  50. 50. Topic Exchange user_interaction_queue Message event.user.click Message event.user.view Message Message Message event.user.* Exchange ‘events’ event.user.share user_shares_queue Message event.user.share Message *.*.view event.staff.view live_view_queue Staff vs user views? Message Message
  51. 51. I’m still pissed
  52. 52. New Event Logging Explorations & Implementation
  53. 53. Revisiting Our Logging Client Client Client MongoDB
  54. 54. Event Logging Revised Client MQ Client MongoDB MQ Client MQ
  55. 55. Event Logging Revised Client MQ Client MongoDB MQ Client MQ
  56. 56. Event Logging Revised Client MQ Client MongoDB MQ Client MQ
  57. 57. No data loss!
  58. 58. SRSLY?!
  59. 59. Event Logging Revised 80 Client MQ 110 MQ Client MQ 65 Client MQ MongoDB
  60. 60. Event Logging Revised 80 Client MQ 110 MQ Client MQ 65 Client MQ MongoDB
  61. 61. Event Logging Revised 80 Client MQ 110 MQ Client MQ 65 Client MQ MQ MongoDB
  62. 62. Event Logging Revised 80 Client MQ MQ 110 Client MongoDB MQ MQ 65 Client MQ
  63. 63. Remember Topic Exchange? user_interaction_queue Message event.user.click Message event.user.view Message Message Message event.user.* Exchange ‘events’ event.user.share user_shares_queue Message event.user.share Message event.staff.view *.*.view live_view_queue Message Message
  64. 64. Provides flexibility 80 MongoDB Client MQ 110 MQ Client Reporting SQL MQ 65 Client Notification MQ
  65. 65. Not Covered • durable • auto-delete • passive • qos • no-wait • ack • exclusive • message recovery
  66. 66. A quick demo of AMQP with RabbitMQ
  67. 67. Thank you weekeat@peazie.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×