Your SlideShare is downloading. ×
0
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Leave this to me.  I'm British.  I know how to queue.
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Leave this to me. I'm British. I know how to queue.

1,332

Published on

Slides from J and Beyond 2014 presentation on asynchronous message queues in Joomla. …

Slides from J and Beyond 2014 presentation on asynchronous message queues in Joomla.

Asynchronous message queues are an important component of many distributed computing environments. This presentation will show how queues can be used to help deliver scalable and robust web applications.

A lightweight library for message queuing in Joomla will be presented and some possible applications in areas such as search indexing and image processing will be explored.

Chris is a Joomla developer with a fondness for the British national sport of queuing. He's also a member of the Joomla Production Leadership Team.

Published in: Internet, Technology, Sports
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,332
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
5
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. “Leave it to me, I'm British, I know how to queue” Chris Davenport Joomla Production Leadership Team J and Beyond 2014
  • 2. Leave it to me, I'm British, I know how to queue.
  • 3. Leave it to me, I'm British, I know how to queue.
  • 4. Leave it to me, I'm British, I know how to queue. “to form a line while waiting for something”
  • 5. Leave it to me, I'm British, I know how to queue. Queues are bad?
  • 6. Leave it to me, I'm British, I know how to queue. We hate queues
  • 7. Leave it to me, I'm British, I know how to queue. Especially when it's raining
  • 8. Leave it to me, I'm British, I know how to queue. Why make me wait!?
  • 9. Leave it to me, I'm British, I know how to queue. The aim of this presentation
  • 10. Leave it to me, I'm British, I know how to queue. The aim of this presentation Some queues are a “good thing”
  • 11. Leave it to me, I'm British, I know how to queue. Sometimes it's good to wait
  • 12. Leave it to me, I'm British, I know how to queue. Synchronisation is good
  • 13. Leave it to me, I'm British, I know how to queue. Good queues Bad queues
  • 14. Leave it to me, I'm British, I know how to queue. Queue messages, not people
  • 15. Leave it to me, I'm British, I know how to queue. Why queue at all?
  • 16. Leave it to me, I'm British, I know how to queue. 1: Image processing
  • 17. Leave it to me, I'm British, I know how to queue. 2: Sending documents
  • 18. Leave it to me, I'm British, I know how to queue. Queue messages, not people Asynchronous message queues
  • 19. Leave it to me, I'm British, I know how to queue. What is a message queue?
  • 20. Leave it to me, I'm British, I know how to queue. What is a message queue? Client (Producer) Queue
  • 21. Leave it to me, I'm British, I know how to queue. What is a message queue? Server (Consumer) Queue
  • 22. Leave it to me, I'm British, I know how to queue. 1: Image processing
  • 23. Leave it to me, I'm British, I know how to queue. 1: Image processing Takes time. Queue: But not the user's time. Requires a lot of memory. Queue: But not the user's memory. Demand spikes can threaten the server. Queue: Naturally throttled.
  • 24. Leave it to me, I'm British, I know how to queue. 2: Sending documents
  • 25. Leave it to me, I'm British, I know how to queue. It's about decoupling ● Decoupling in space ● Decoupling in time ● Decoupling synchronisation
  • 26. Leave it to me, I'm British, I know how to queue. Space decoupling ● Different processes ● Different machines ● Different locations
  • 27. Leave it to me, I'm British, I know how to queue. Time decoupling ● Heavy tasks may be deferred ● Task streams may be throttled ● Can cope with unreliable systems ● Can cope with unreliable networks
  • 28. Leave it to me, I'm British, I know how to queue. Synchronisation decoupling ● Clients should never block. ● Redundancy ● Scalability
  • 29. Leave it to me, I'm British, I know how to queue. Queue patterns
  • 30. Leave it to me, I'm British, I know how to queue. Request - Reply Producer Q 12345 Consumer
  • 31. Leave it to me, I'm British, I know how to queue. Request - Reply Producer ConsumerQ 1 2345
  • 32. Leave it to me, I'm British, I know how to queue. Request - Reply Producer Q Q R1 Consumer 2345
  • 33. Leave it to me, I'm British, I know how to queue. Request - Reply Producer Q Q R1 Consumer2 345
  • 34. Leave it to me, I'm British, I know how to queue. Request - Reply Producer Q Q? R1 Consumer2 345
  • 35. Leave it to me, I'm British, I know how to queue. Request - Reply Producer Q Q? Consumer Consumer Consumer 12345 Consumer
  • 36. Leave it to me, I'm British, I know how to queue. Request - Reply Producer Q Q? 2345 Consumer Consumer Consumer Consumer1
  • 37. Leave it to me, I'm British, I know how to queue. Request - Reply Producer Q Q? 345 Consumer Consumer Consumer Consumer1 2
  • 38. Leave it to me, I'm British, I know how to queue. Request - Reply Producer Q Q? 45 Consumer Consumer Consumer Consumer1 2 3
  • 39. Leave it to me, I'm British, I know how to queue. Request - Reply Producer Q Q? 5 Consumer Consumer Consumer Consumer1 2 3 4
  • 40. Leave it to me, I'm British, I know how to queue. Request - Reply Producer Q Q? R1 Consumer Consumer Consumer Consumer 2 3 4 5
  • 41. Leave it to me, I'm British, I know how to queue. Request - Reply Producer Q Q? R1 Consumer Consumer Consumer Consumer5 2 3 4
  • 42. Leave it to me, I'm British, I know how to queue. Pipeline Producer Q Consumer 12345 ConsumerQ ConsumerQ
  • 43. Leave it to me, I'm British, I know how to queue. Pipeline Producer Q Consumer1 2345 ConsumerQ ConsumerQ
  • 44. Leave it to me, I'm British, I know how to queue. Pipeline Producer Q Consumer Consumer Q 1' 2 345 ConsumerQ
  • 45. Leave it to me, I'm British, I know how to queue. Pipeline Producer Q Consumer Consumer Q 1' 2' 3 45 ConsumerQ
  • 46. Leave it to me, I'm British, I know how to queue. Pipeline Producer Q Consumer Consumer Q 2' 3' 4 5 ConsumerQ 1''
  • 47. Leave it to me, I'm British, I know how to queue. Pipeline Producer Q Consumer Consumer Q 3' 4' 5 ConsumerQ 2'' 1''
  • 48. Leave it to me, I'm British, I know how to queue. Publish - Subscribe Producer Q Consumer Consumer Consumer Consumer 12345
  • 49. Leave it to me, I'm British, I know how to queue. Publish - Subscribe Producer Q 2345 Consumer Consumer Consumer Consumer1 1 1 1
  • 50. Leave it to me, I'm British, I know how to queue. Queue packages
  • 51. Leave it to me, I'm British, I know how to queue. Queue packages AMQP – Apache Qpid, OpenAMQ, RabbitMQ ZeroMQ IBM WebsphereMQ Amazon SQS MSMQ MQTT
  • 52. Leave it to me, I'm British, I know how to queue. Queue characteristics Locality Persistent or non-persistent Messages have an “in-flight” period or not Preserve message order or not Eliminate duplicates or not Queue full behaviour Message delivery guarantees
  • 53. Leave it to me, I'm British, I know how to queue. Using queues in Joomla
  • 54. Leave it to me, I'm British, I know how to queue. Using queues in Joomla Most require special server software Not available on most shared hosts
  • 55. Leave it to me, I'm British, I know how to queue. Using queues in Joomla Need a native PHP implementation Optional adapters for mainstream message queues
  • 56. Leave it to me, I'm British, I know how to queue. Queuing in Joomla GitHub: chrisdavenport/joomla-mq
  • 57. Leave it to me, I'm British, I know how to queue. Queuing in Joomla Provide a very basic standard interface to a choice of message queues. Currently supports: Array, filesystem, database, Unix System V, Amazon SQS. Documented and unit tested. Pull requests welcome.
  • 58. Leave it to me, I'm British, I know how to queue. JMq sending a message $config = new JRegistry($configData); $q = new JMqQueue($config); $q->send('myqueue', 'Hello World');
  • 59. Leave it to me, I'm British, I know how to queue. JMq receiving a message $config = new JRegistry($configData); $q = new JMqQueue($config); $msg = $q->receive('myqueue'); // Process the message here. $q->delete('myqueue', $msg->messageid);
  • 60. Leave it to me, I'm British, I know how to queue. Problem... Generally needs some kind of cron. Low-cost hosts will need to implement some kind of “pseudo-cron” system. Might be possible to use a plugin event to transparently process a queue on hosts that do not allow cron.
  • 61. Leave it to me, I'm British, I know how to queue. Joomla use-cases Image resizing and watermarking. Smart Search indexing. Hit counters. Ecommerce. Throttling. Monitoring, logging, clickstream analysis. Statistics and report generation.
  • 62. Leave it to me, I'm British, I know how to queue. Convinced?
  • 63. Leave it to me, I'm British, I know how to queue. Queue Tips
  • 64. Leave it to me, I'm British, I know how to queue. Queues are cheap So feel free to use lots of them
  • 65. Leave it to me, I'm British, I know how to queue. Avoid fat messages Message systems can rarely handle large messages efficiently
  • 66. Leave it to me, I'm British, I know how to queue. Guaranteed delivery There's no such thing Murphy's Law will always win in the end
  • 67. Leave it to me, I'm British, I know how to queue. Duplicate messages Assume that some messages may be duplicated
  • 68. Leave it to me, I'm British, I know how to queue. Message ordering Don't assume messages will arrive in the same order they were sent
  • 69. Leave it to me, I'm British, I know how to queue. Avoid transactions Atomic transactions across message queues are fraught with difficulty, so avoid them If you absolutely must have transactions then lookup Raft and Paxos algorithms
  • 70. Leave it to me, I'm British, I know how to queue. Error handling Invalid or undeliverable messages should be forwarded to a dead-letter queue Process failures should be reported to an error queue
  • 71. Leave it to me, I'm British, I know how to queue. Questions?
  • 72. Leave it to me, I'm British, I know how to queue. Image credits 1 http://commons.wikimedia.org/wiki/File:Queuing_z01.jpg?uselang=en-gb http://commons.wikimedia.org/wiki/File:Form_an_orderly_queue_to_the_left_%286180460759%29.jpg http://commons.wikimedia.org/wiki/File:Angry_woman.jpg http://commons.wikimedia.org/wiki/File:20110529_London_38.JPG http://commons.wikimedia.org/wiki/File:Take_off_queue,_Heathrow,_10_Sept._2010_-_Flickr_-_PhillipC.jpg http://commons.wikimedia.org/wiki/File:DigitalPicture.jpg http://commons.wikimedia.org/wiki/File:Contrast_improvement.jpg http://commons.wikimedia.org/wiki/File:Base-isolation.gif
  • 73. Leave it to me, I'm British, I know how to queue. Image credits 2 http://commons.wikimedia.org/wiki/File:Traffic_queues_likely.JPG http://commons.wikimedia.org/wiki/File:Motorway_queue_near_Lisburn_%282%29_-_geograph.org.uk_- _1380275.jpg http://commons.wikimedia.org/wiki/File:Queue_for_the_Banksy_Exhibition_-_geograph.org.uk_- _1429992.jpg http://commons.wikimedia.org/wiki/File:1925_ES%26ARobinson_Invoice_with_illustrations.jpg? uselang=en-gb http://commons.wikimedia.org/wiki/File:Cirencester_..._snazzy_trolley..jpg http://commons.wikimedia.org/wiki/File:Copertina_facebook.jpg http://commons.wikimedia.org/wiki/File:Q_Tips_plain_BG.jpg?uselang=en-gb http://commons.wikimedia.org/wiki/File:Bus_queue_in_Redhill_Road,_Cobham_-_geograph.org.uk_- _1800438.jpg
  • 74. Leave it to me, I'm British, I know how to queue. Image credits 3 http://commons.wikimedia.org/wiki/File:Queue_at_Pashupatinath01.JPG http://commons.wikimedia.org/wiki/File:Cracow_queue_1986.jpg (short queue) http://commons.wikimedia.org/wiki/File:Newport_Staplers_Road_traffic_queue_4.JPG http://commons.wikimedia.org/wiki/File:Queue_outside_a_soup_kitchenat_Klostertorv_in_ %C3%85lborg_1943.jpg http://commons.wikimedia.org/wiki/File:Bathroom_Scale.jpg?uselang=en-gb http://commons.wikimedia.org/wiki/File:Super_Cheap_Auto.jpg http://commons.wikimedia.org/wiki/File:Clones_id.jpg http://commons.wikimedia.org/wiki/File:Blue_Angels_all_in_a_line_%282501423714%29.jpg
  • 75. Leave it to me, I'm British, I know how to queue. Image credits 4 http://commons.wikimedia.org/wiki/File:Three_in_a_line_-_geograph.org.uk_-_652726.jpg http://commons.wikimedia.org/wiki/File:Viruses.jpg

×