What’s Messaging?
An Introduction
History of Message Queuing
Manual Telegraphy Machine Assisted Telegraphy
1920s1911 - 192019th Century 1900s
Telegrams sent...
History of Message Queuing
Electronic Telegraphy
1950s1940s1950s1940s
Electronic Telegram
Machines, eg Plan 55-A
1948
IBM
...
History of Message Queuing
Telcos UseElectronic Telegraphy
1950s1940s 1960s
IBM System/360 with
BTAM & QTAM
Message Switch...
History of Message Queuing
l
Financial Trading UsersBanking Users FMCG & Utilities
1980s 1990s
Growth of SMTP
Origins of T...
History of Message Queuing
l
FMCG & Utilities
1990s
MQSeries
phereMQ)
92
Corporates Large Websites YOU
Noughties Today
Sun...
The Integration Tag Soup
WS –File Transfer
!"#$%&'(#)#*#+%+
Message Queuing
SOAP
EMail
SMTP
FTP
RSYNC
NTFS
!','-
!).$%&'/$...
The Integration Tag Soup
WS –File Transfer
!"#$%&'(#)#*#+%+
Message Queuing
SOAP
EMail
SMTP
FTP
RSYNC
NTFS
!','-
!).$%&'/$...
Under Examination, though
File Transfer
!"#$%&'(#)#*#+%+
Remote Procedure Call
Don’tWork in the Cloud
Under Examination, though
Deployment
Intimate System Knowledge
Configuration
Admin
File Transfer
!"#$%&'(#)#*#+%+
Remote Pr...
Ideal: Message Queuing
Systems are “loosely-coupled”
Message
Queuing
Ideal: Message Queuing
Systems are “loosely-coupled”
Message
Queuing
Ideal: Message Queuing
Systems are “loosely-coupled”
Message
Queuing
Systems don’t
know each other
Why Use it: Loose Coupling
Billing
Catalogue
Shipping
S-a-a-S Inventory
How do we connect them, without one outage or syst...
Why Use it: Loose Coupling
Billing
Catalogue
Shipping
S-a-a-S Inventory
How do we connect them, without one outage or syst...
It should be ideal for the cloud
It should be ideal for the cloud
!
!
!
Until today, messaging is …
(ZeroMQ)
(SQS uses HTTP)
(MSMQ uses COM)
(memcached)
(Java JMS
uses
Source)
(MQSeries / ...
!
!
!
Until today, messaging is …
(ZeroMQ)
(SQS uses HTTP)
(MSMQ uses COM)
(memcached)
(Java JMS
uses
Source)
(MQSeries / ...
Why is it Hell?
You need more staff And moneyYou need bridge technology
Why is it Hell?
You need more staff And moneyYou need bridge technology
However, there’s a solution
A-MQPAdvanced Message Queue Protocol
However, there’s a solution
However, AMQP fixes this
A common wire-level binary
format and protocol
An explicit definition of a
server (aka broker)’s
se...
That is good …
“AMQP will be to Messaging what HTTP was to theWeb”
MRG
Clients run on any Platform Vendors are Interoperab...
That is good …
“AMQP will be to Messaging what HTTP was to theWeb”
MRG
Clients run on any Platform Vendors are Interoperab...
From Anywhere to Anywhere
From Anywhere to Anywhere
From Anywhere to Anywhere
From Anywhere to Anywhere
From Anywhere to Anywhere
Quick Recap
For Beer
The fifth male member
of the A-Team, Frankie,
was played by
Question
Message Queuing connects systems ...
EddieVelez
Quick Recap
✓
The fifth male member
of the A-Team, Frankie,
was played by
Question
Message Queuing connects syst...
Common Terms and Jargon
Jargon
Messaging: Which Jargon?
EMail
(SMTP, POP3, IMAP)
VoIP
(VoiceMail, XMPP)
Texting
(SMS)
Instant Messaging
(ICQ, MSN, Jabber...
Essential Terms
Message Queue
Behaves like a Queue
Is First-In First-Put (FIFO)
Elements are Messages
Essential Terms
Message Queue What’s a Message?
Envelope
Payload
Header
Essential Terms
*Strictly speaking
a receiver polls for messages
a consumer has messages pushed
≣ ≊Send Publish Enqueue
≣ ...
So how can you use it?
Concepts
Useful Book:
Enterprise
Integration Patterns
Hohpe &Woolf
Store and Forward
Billing
“Could you send me a duplicate of my bill please”
REST
Store and Forward
Billing
“Could you send me a duplicate of my bill please”
REST
!
Store and Forward
Billing
“Could you send me a duplicate of my bill please”
REST
StormMQ
Messaging
Billing
!
Store and Forward
Billing
“Could you send me a duplicate of my bill please”
REST
StormMQ
Messaging
Billing
!
!
Store and Forward
Billing
“Could you send me a duplicate of my bill please”
REST
StormMQ
Messaging
Billing
!
Fire and Forget
Shipping
“Too many orders.The website is running like a dog.”
SQL Push SQL Pull
Fire and Forget
Shipping
“Too many orders.The website is running like a dog.”
SQL Push SQL Pull
StormMQ
Messaging
Shipping
One-To-Many
“There’s a new widget in the Catalogue: Tell all the systems.”
Catalogue
Shipping
S-a-a-S Inventory
File Trans...
One-To-Many
“There’s a new widget in the Catalogue: Tell all the systems.”
Catalogue
Shipping
S-a-a-S Inventory
StormMQ
Me...
Publish-Subscribe (“Topics”)
“Shipments Sent, Delivered and Returned”
Shipping
S-a-a-S Inventory
Billing
AnySent
Sent or
R...
StormMQ
Messaging
Publish-Subscribe (“Topics”)
“Shipments Sent, Delivered and Returned”
Shipping
S-a-a-S Inventory
Billing...
Round-Robin
Billing
“How do we easily scale a massive batch job like Billing?”
EMail BillGenerate
StormMQ
Messaging
Round-Robin
Billing
“How do we easily scale a massive batch job like Billing?”
EMail BillGenerate
Getting Technical
Delving into AMQP
AMQP Client AMQP ServerTCP / IP Network
Connection Virtual Host
Connections and Channels
Connection
AMQP Client AMQP ServerTCP / IP Network
Connection
Virtual Host
Connections and Channels
TLS
“Shielding”
Channe...
Basic AMQP: Connections
Basic AMQP: Connections
Open a Connection to aVirtual Host
Open a Channel
Send a Message
Receive a...
// The API is styled similarly to Functionally Orientated Programming (FOP)
// It makes extensive use of Constructor Injec...
// Sending a Message
public void use(final @NotNull ConvenientChannel convenientChannel)
{
// ConvenientChannel has static ...
// Receiving a Message using Polling
public void use(final @NotNull ConvenientChannel convenientChannel)
{
final boolean noA...
// Consuming Messages using Push
public void use(final @NotNull ConvenientChannel convenientChannel)
{
final boolean noAck =...
!
✓
Channels
Complex Applications
Consume on several threads
Send on several threads
Why Use More than One Channel?
Recomm...
Exchanges Route Messages
Exchanges route Messages to Message Queues
Exchange
MQ
A
MQ
B
Exchanges Route Messages
Exchanges route Messages to Message Queues
You send messages to an exchange, not a message queue
...
How do Exchanges Route?
Exchange
MQ
A
The Exchange finds a Binding
matching the Routing Key
#
A Binding connects a Routing ...
Types of Exchange
More rarely used exchange types include amq.headers and extensions
Message Queue Name
direct
Like a ‘Map...
// Declaring Exchanges
public void use(final @NotNull ConvenientChannel convenientChannel)
{
final boolean passivelyDeclare ...
// Declaring and Binding Message Queues
public void use(final @NotNull ConvenientChannel convenientChannel)
{
final boolean ...
AMQP: Introducing Users
but
StormMQ enforces that the same
Users and Permissions are in
all of a System’s Environments
Pas...
// Example of create-system.json used with API call stormmq-create-system
{
"companyName" : "usuallyYourUserName",
"system...
Configuring AMQP
“*Our REST API has Java, Ruby, PHP and .NET bindings”
AMQP StormMQ
✓ ActiveMessage Queues
Exchanges
Bindin...
AMQP: Getting Statistics
“*Password revelation is restricted to specific individuals”
AMQP StormMQ
Message Queues
Exchanges...
// Getting Statistics from StormMQ using the REST API in Java
// If you need to use proxies or experience the pain of Java...
Virtual Hosts
Virtual Hosts: Think Apache!
Virtual Hosts isolate Message Queues
Virtual Hosts isolate User Groups*
Virtual...
Message Properties
Server Understood Good Practice
“This is just the tip of messaging best practice with AMQP”
Persistent ...
// Message Properties: Examples of Good Practice
// Being explicit is good practice
final BasicProperties messageHeaders = ...
And there is more!
Transactions
More Message
Properties
Custom Message
Properties
Immediate Delivery
Additional Exchange
T...
Useful Book:
Enterprise
Integration Patterns
Hohpe &Woolf
An Introduction to AMQP with Code Samples
Upcoming SlideShare
Loading in...5
×

An Introduction to AMQP with Code Samples

3,612

Published on

An introduction to AMQP with code samples by StormMQ at SkillsMatter, London on the 26th October.

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,612
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
86
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Transcript of "An Introduction to AMQP with Code Samples"

  1. 1. What’s Messaging? An Introduction
  2. 2. History of Message Queuing Manual Telegraphy Machine Assisted Telegraphy 1920s1911 - 192019th Century 1900s Telegrams sent using “Store and Forward” 1900 1930s
  3. 3. History of Message Queuing Electronic Telegraphy 1950s1940s1950s1940s Electronic Telegram Machines, eg Plan 55-A 1948 IBM M ne Assisted Telegraphy 1920s1920 1930s
  4. 4. History of Message Queuing Telcos UseElectronic Telegraphy 1950s1940s 1960s IBM System/360 with BTAM & QTAM Message Switching 1964 First Electronic Mail Solutions 1965 Banking Users 1970s IBM TCAM which is the first true solution Retired 1990! 1971
  5. 5. History of Message Queuing l Financial Trading UsersBanking Users FMCG & Utilities 1980s 1990s Growth of SMTP Origins of Tibco in Stock Price Messaging 1980s IBM Launch MQSeries (now WebsphereMQ) 1992 1970s IBM TCAM which is the first true solution Retired 1990! 1971
  6. 6. History of Message Queuing l FMCG & Utilities 1990s MQSeries phereMQ) 92 Corporates Large Websites YOU Noughties Today Sun Release Java JMS, Reinvigorating Enterprise Messaging 2001 AMQP Working Group Formed by Investment Banks 2006 Cloud Enables and Drives StormMQ Adoption 2009
  7. 7. The Integration Tag Soup WS –File Transfer !"#$%&'(#)#*#+%+ Message Queuing SOAP EMail SMTP FTP RSYNC NTFS !','- !).$%&'/$.0%&1$%+ 234%5)6!%$7%$ SOA Message Orientated Middleware JMS MQ MQSeries AMQP Amazon SQS Tibco StormMQ RabbitMQ HTTP TCP/IP DCE / RPC CORBA DCOM REST RMI XML-RPC .NET Remoting Remote Procedure Call
  8. 8. The Integration Tag Soup WS –File Transfer !"#$%&'(#)#*#+%+ Message Queuing SOAP EMail SMTP FTP RSYNC NTFS !','- !).$%&'/$.0%&1$%+ 234%5)6!%$7%$ SOA Message Orientated Middleware JMS MQ MQSeries AMQP Amazon SQS Tibco StormMQ RabbitMQ HTTP TCP/IP DCE / RPC CORBA DCOM REST RMI XML-RPC .NET Remoting Remote Procedure Call
  9. 9. Under Examination, though File Transfer !"#$%&'(#)#*#+%+ Remote Procedure Call Don’tWork in the Cloud
  10. 10. Under Examination, though Deployment Intimate System Knowledge Configuration Admin File Transfer !"#$%&'(#)#*#+%+ Remote Procedure Call Don’tWork in the Cloud Scaling
  11. 11. Ideal: Message Queuing Systems are “loosely-coupled” Message Queuing
  12. 12. Ideal: Message Queuing Systems are “loosely-coupled” Message Queuing
  13. 13. Ideal: Message Queuing Systems are “loosely-coupled” Message Queuing Systems don’t know each other
  14. 14. Why Use it: Loose Coupling Billing Catalogue Shipping S-a-a-S Inventory How do we connect them, without one outage or system change taking everything down like a pack of cards?
  15. 15. Why Use it: Loose Coupling Billing Catalogue Shipping S-a-a-S Inventory How do we connect them, without one outage or system change taking everything down like a pack of cards? Message Queuing lets Systems and Components exchange data, events, commands and actions with one another with no explicit knowledge or need for them to be online
  16. 16. It should be ideal for the cloud
  17. 17. It should be ideal for the cloud
  18. 18. ! ! ! Until today, messaging is … (ZeroMQ) (SQS uses HTTP) (MSMQ uses COM) (memcached) (Java JMS uses Source) (MQSeries / WebsphereMQ) (Talarian, Rendezvous, etc) Platform Restricted BespokeProprietary
  19. 19. ! ! ! Until today, messaging is … (ZeroMQ) (SQS uses HTTP) (MSMQ uses COM) (memcached) (Java JMS uses Source) (MQSeries / WebsphereMQ) (Talarian, Rendezvous, etc)
  20. 20. Why is it Hell? You need more staff And moneyYou need bridge technology
  21. 21. Why is it Hell? You need more staff And moneyYou need bridge technology
  22. 22. However, there’s a solution
  23. 23. A-MQPAdvanced Message Queue Protocol However, there’s a solution
  24. 24. However, AMQP fixes this A common wire-level binary format and protocol An explicit definition of a server (aka broker)’s semantics Open Means Interoperable
  25. 25. That is good … “AMQP will be to Messaging what HTTP was to theWeb” MRG Clients run on any Platform Vendors are Interoperable
  26. 26. That is good … “AMQP will be to Messaging what HTTP was to theWeb” MRG Clients run on any Platform Vendors are Interoperable 64K
  27. 27. From Anywhere to Anywhere
  28. 28. From Anywhere to Anywhere
  29. 29. From Anywhere to Anywhere
  30. 30. From Anywhere to Anywhere
  31. 31. From Anywhere to Anywhere
  32. 32. Quick Recap For Beer The fifth male member of the A-Team, Frankie, was played by Question Message Queuing connects systems and components. Is it ideal for the cloud? Does Loose-Coupling make individual systems more likely to suffer outages? Yes No AMQP is Open. This makes it suitable for programming in C and Javascript?
  33. 33. EddieVelez Quick Recap ✓ The fifth male member of the A-Team, Frankie, was played by Question Message Queuing connects systems and components. Is it ideal for the cloud? Does Loose-Coupling make individual systems more likely to suffer outages? Yes No AMQP is Open. This makes it suitable for programming in C and Javascript? ! ✓
  34. 34. Common Terms and Jargon Jargon
  35. 35. Messaging: Which Jargon? EMail (SMTP, POP3, IMAP) VoIP (VoiceMail, XMPP) Texting (SMS) Instant Messaging (ICQ, MSN, Jabber) Twitter Enterprise Service Bus (ESB) Dynamic OO Languages (eg Ruby) Message Queuing (MQ) Message Queuing (MQ) What do we mean by Messaging?
  36. 36. Essential Terms Message Queue Behaves like a Queue Is First-In First-Put (FIFO) Elements are Messages
  37. 37. Essential Terms Message Queue What’s a Message? Envelope Payload Header
  38. 38. Essential Terms *Strictly speaking a receiver polls for messages a consumer has messages pushed ≣ ≊Send Publish Enqueue ≣ ≊ DequeueConsume*Receive
  39. 39. So how can you use it? Concepts Useful Book: Enterprise Integration Patterns Hohpe &Woolf
  40. 40. Store and Forward Billing “Could you send me a duplicate of my bill please” REST
  41. 41. Store and Forward Billing “Could you send me a duplicate of my bill please” REST !
  42. 42. Store and Forward Billing “Could you send me a duplicate of my bill please” REST StormMQ Messaging Billing !
  43. 43. Store and Forward Billing “Could you send me a duplicate of my bill please” REST StormMQ Messaging Billing ! !
  44. 44. Store and Forward Billing “Could you send me a duplicate of my bill please” REST StormMQ Messaging Billing !
  45. 45. Fire and Forget Shipping “Too many orders.The website is running like a dog.” SQL Push SQL Pull
  46. 46. Fire and Forget Shipping “Too many orders.The website is running like a dog.” SQL Push SQL Pull StormMQ Messaging Shipping
  47. 47. One-To-Many “There’s a new widget in the Catalogue: Tell all the systems.” Catalogue Shipping S-a-a-S Inventory File Transfer ETL
  48. 48. One-To-Many “There’s a new widget in the Catalogue: Tell all the systems.” Catalogue Shipping S-a-a-S Inventory StormMQ Messaging
  49. 49. Publish-Subscribe (“Topics”) “Shipments Sent, Delivered and Returned” Shipping S-a-a-S Inventory Billing AnySent Sent or Returned
  50. 50. StormMQ Messaging Publish-Subscribe (“Topics”) “Shipments Sent, Delivered and Returned” Shipping S-a-a-S Inventory Billing AnySent Sent or Returned
  51. 51. Round-Robin Billing “How do we easily scale a massive batch job like Billing?” EMail BillGenerate
  52. 52. StormMQ Messaging Round-Robin Billing “How do we easily scale a massive batch job like Billing?” EMail BillGenerate
  53. 53. Getting Technical Delving into AMQP
  54. 54. AMQP Client AMQP ServerTCP / IP Network Connection Virtual Host Connections and Channels
  55. 55. Connection AMQP Client AMQP ServerTCP / IP Network Connection Virtual Host Connections and Channels TLS “Shielding” Channels Each Channel is Independent: Effectively, aVirtual Connection
  56. 56. Basic AMQP: Connections Basic AMQP: Connections Open a Connection to aVirtual Host Open a Channel Send a Message Receive a Message Close Channel Close Connection You only need one channel! " # $ % & ' How does this look in Code?
  57. 57. // The API is styled similarly to Functionally Orientated Programming (FOP) // It makes extensive use of Constructor Injection and anonymous classes modelling 'blocks' // Closes are handled implicitly by design with exceptions thrown, ie try-finally is internal to the API public void demonstrateConnection() { final AmqpConnectionFactory amqpConnectionFactory = amqpConnectionFactoryWhichVerifiesStormMQ(“mycompany”, “mysystem”, “development”, “user”, “password”); // Creates and closes a connection amqpConnectionFactory . useConnection ( // Creates and closes a channel on a connection new ChannelCreatingConnectionUser ( // Adapts a channel to make it convenient to use new ConvenientChannelCreatingConnectionUser ( new ConvenientChannelUser() ( public void use(final @NotNull ConvenientChannel convenientChannel) { // Send a message, or, // Receive a message (or both) } ); ) ) ) }
  58. 58. // Sending a Message public void use(final @NotNull ConvenientChannel convenientChannel) { // ConvenientChannel has static convenience methods to create messageHeaders (meta-data) // Use them to signify a message's content’s MIME type or an unique id // Use meta data sparingly; think of it like email headers. More later final BasicProperties messageHeaders = emptyBasicProperties(); // All messages are byte arrays final byte[] messageBody = "HelloWorld" . getBytes( forName("UTF-8") ); // Send the message final String routingKey = "usually the destination message queue"; final boolean mandatory = false; final boolean immediate = false; convenientChannel . basicPublish( "", routingKey, mandatory, immediate, messageHeaders, messageBody ); } /* Notes * Mandatory (mandatory = true) messages are returned unless routed to a queue * Immediate (mandatory = true) messages are returned unless there is another channel consuming */
  59. 59. // Receiving a Message using Polling public void use(final @NotNull ConvenientChannel convenientChannel) { final boolean noAck = false; convenientChannel . basicGet( "queue", noAck, DoNothingWhenMessageNotReceived, new MessageReceived() { public void messageReceived(final @NotNull GetResponse message) { final byte[] messageBody = message . getBody(); final BasicProperties messageHeaders = message . getProps(); // Acknowledge successfully received messages if noAck = false // If not acknowledged, they are eventually placed back on the queue for re-delivery convenientChannel . basicAck( message ); } }); } /* Notes * DoNothingWhenMessageNotReceived is a singleton of MesageNotReceived * Implement DoNothing to do something else, eg for debugging * Polling is thread-safe but inefficient; using consumers (“push”) for more efficiency… */
  60. 60. // Consuming Messages using Push public void use(final @NotNull ConvenientChannel convenientChannel) { final boolean noAck = true; convenientChannel . basicConsume( "my queue", noAck, new ConvenientConsumer() { // Consumers run on a different thread public void handleDelivery( final String consumerTag, final Envelope envelope, final BasicProperties messageHeaders, final byte[] messageBody ) { final byte[] messageBody = message . getBody(); final BasicProperties messageHeaders = message . getProps(); // No need to acknowledge message as noAck = true } }); } /* Notes * Consumers only exist as long as a channel is open * Add a threading primitive (eg volatile boolean) and reference it at * * It is possible to use multiple channels with several consumers… */ *
  61. 61. ! ✓ Channels Complex Applications Consume on several threads Send on several threads Why Use More than One Channel? Recommendation: Keep it Simple! Simple Applications Send on a queue Receive on another Transactional Applications NeedTransactions on a queue Don’t NeedTransactions on another ✓
  62. 62. Exchanges Route Messages Exchanges route Messages to Message Queues Exchange MQ A MQ B
  63. 63. Exchanges Route Messages Exchanges route Messages to Message Queues You send messages to an exchange, not a message queue Exchange MQ A MQ B
  64. 64. How do Exchanges Route? Exchange MQ A The Exchange finds a Binding matching the Routing Key # A Binding connects a Routing Key to one or more Message Queues $ Every sent message has a Routing Key " The Exchange delivers the Message to the Message Queue % A message queue can be bound more than once to one or more exchanges ( “string”
  65. 65. Types of Exchange More rarely used exchange types include amq.headers and extensions Message Queue Name direct Like a ‘Map’:All MQs bound with the routing key receive copies of the message “” (blank) amq.direct fanout Empty String All MQs bound to the Exchange receive copies of the message amq.fanout topic Dotted Bindings use globbing expressions (wildcards) to route messages to MQs amq.topic A message queue can be bound more than once to an exchange; A message queue can be bound to more than one exchange But a message queue will only receive a message sent once Routing Key Routing Behaviour Default Definitions Point – to – Point One – to – Many Publish – SubscribeTypical Use
  66. 66. // Declaring Exchanges public void use(final @NotNull ConvenientChannel convenientChannel) { final boolean passivelyDeclare = false; final boolean durable = true; final boolean autoDelete = false; // Just change the ExchangeType to one of direct, fanout or topic convenientChannel . exchangeDeclare( "my exchange A", direct, passivelyDeclare, durable, autoDelete ); convenientChannel . exchangeDeclare( "my exchange B", fanout, passivelyDeclare, durable, autoDelete ); convenientChannel . exchangeDeclare( "my exchange C", topic, passivelyDeclare, durable, autoDelete ); // A passive declaration asserts that an exchange exists, causing a channel exception if it does not convenientChannel . exchangeDeclare( "my exchange A", direct, true, durable, autoDelete ); } /* Notes * Always use durable = true with StormMQ as we are a hosted service * Nearly always use autoDelete = false as exchanges should be long-lived * Declaring an exchange twice is not an error; if parameters differ, new values are used if possible * Passive declaration is interesting but typical deployment practices mitigate its usefulness */
  67. 67. // Declaring and Binding Message Queues public void use(final @NotNull ConvenientChannel convenientChannel) { final boolean passivelyDeclare = false; final boolean durable = true; final boolean autoDelete = false; final boolean exclusive = false; // A declared queue is bound to the default ("") exchange with the routing key the queue name ("my queue") convenientChannel . queueDeclare( "my queue", passivelyDeclare, durable, exclusive, autoDelete ); // A message queue can be bound to another exchange with whatever key you choose convenientChannel . queueBind( "my queue", "another exchange", "some routing key" ); // When binding to a fanout exchange the routing key is the empty string ("") convenientChannel . queueBind( "my queue", "a fanout exchange, eg amq.fanout", "" ); // When binding to a topic exchange, set the routing key as a wildcard // Publish messages with specific routing keys eg "BONDS.USD.FTSE", "BONDS.USD.NYSE", "BONDS.GBP.FTSE" convenientChannel . queueBind( "my queue", "a fanout exchange, eg amq.topic", "BONDS.USD.*" ); } /* Notes * Always use durable = true with StormMQ as we are a hosted service * Nearly always use autoDelete = false as message queues should be long-lived * Declaring a message queue twice is not an error; if parameters differ, new values are used if possible * Exclusive message queues are for advanced usages and often aren’t needed */
  68. 68. AMQP: Introducing Users but StormMQ enforces that the same Users and Permissions are in all of a System’s Environments Passwords are different Passwords are securely generated by StormMQ and are 512 bit keys Configuration Managers can prevent revelation of passwords to different system users, eg developers vs sysadmins Aim: Separation of Environments A Connection can use several authentication mechanisms which are perVirtual Host but Commonest is User - Password andand Aim: Separation of Responsibility Users are intended to be system accounts (robots), not sysadmins Users have read, write and create permissions using a wildcard syntax: important to name queues, etc well
  69. 69. // Example of create-system.json used with API call stormmq-create-system { "companyName" : "usuallyYourUserName", "systemName" : "anythingAlphabetic", "environments" : [{ "environmentName" : "development", // Different StormMQ clusters have different characteristics (and costs) "clusterName" : "free-1", // These are the StormMQ user accounts to which passwords for AMQP users (below) can be revealed for this environment (development) "permittedStormMQUserNames" : [ "developerPeter", "developerJames", "developerJohn" ] }], // User permissions operate on Message Queues and Exchanges "amqpUserPermissions" : { "queueReader" : { "create" : "^$", "read" : ".*", "write" : "^$" }, "queueWriter" : { "create" : "^$", "read" : "^$", "write" : ".*" }, "queueCreator" : { "create" : ".*", "read" : "^$", "write" : "^$" }, "onlyProcessingObjects" : { "create" : "^processing.*", "read" : "^processing.*", "write" : "^processing.*" } } }
  70. 70. Configuring AMQP “*Our REST API has Java, Ruby, PHP and .NET bindings” AMQP StormMQ ✓ ActiveMessage Queues Exchanges Bindings Users User Permissions Entire Systems ! Extra ! Extra ! Extra ✓ Active ✓ Active ✓ REST ✓ REST ✓ REST* Programmatic Config ✓ Same ✓ Same ✓ Same
  71. 71. AMQP: Getting Statistics “*Password revelation is restricted to specific individuals” AMQP StormMQ Message Queues Exchanges Bindings Generated Passwords Usage Patterns Entire Systems ! Extra ! Extra ! Extra ✓ REST ✓ REST ✓ REST Programmatic Info ! Extra ✓ REST ! Extra ✓ REST* ! Extra ✓ REST
  72. 72. // Getting Statistics from StormMQ using the REST API in Java // If you need to use proxies or experience the pain of Java SSL choose a different method in ApiConfiguration final ApiConfiguration apiConfiguration = directConnectionIfHavingProblemsWithSslVerification(); // If your Magic Secret Key is in an unusual place choose a different method in Api final Api api = apiByGuessingUsingOsConfigurationFiles( apiConfiguration, stormMqUserName ); final List<AmqpQueue> amqpQueues = api . listQueues( companyName, systemName, environmentName ); final List<AmqpExchange> amqpExchanges = api . listExchanges( companyName, systemName, environmentName ); final List<AmqpBinding> amqpBindings = api . listBindings( companyName, systemName, environmentName ); final Map<AmqpUserName, AmqpUserPassword> passwords = api . listAmqpUserNamesAndPasswords( companyName, systemName, environmentName ); final CompanyDetails companyDetails = api . describeCompany( companyName ); // Finding “what we’ve got” final List<CompanyName> companyNames = api . listCompanies(); final List<ClusterName> clusterNames = api . listClusters( companyName ); final List<SystemName> systemNames = api . listSystems( companyName ); final SystemDetails systemDetails = api . describeSystem( companyName, systemName ); final Set<Environment> environments = systemDetails . getEnvironments(); /* Notes * Also consider the command-line tools stormmq-list-queues, stormmq-list-exchanges and stormmq-list-bindings * Further examples are in com.stormmq.api.Example (part of the url signer jar) */
  73. 73. Virtual Hosts Virtual Hosts: Think Apache! Virtual Hosts isolate Message Queues Virtual Hosts isolate User Groups* Virtual Hosts isolate Exchanges* Virtual Hosts isolate Bindings* Virtual Hosts isolate Virtual Hosts: StormMQ Virtual Hosts isolateYour Company Virtual Hosts isolateYour Systems Virtual Hosts isolateYour Environments Enable Configuration Management Virtual Hosts stop Data Accidents Widgets Ltd / Invoicing System / / Testing Development Production
  74. 74. Message Properties Server Understood Good Practice “This is just the tip of messaging best practice with AMQP” Persistent = 1 Priority = 0 Timestamp = 4568965345 MessageId = 567A-GH-20100709 Priority Queues are per-Message Control life of a Message Persistence is per-Message Timestamps expire Messages Completely Optional MIME Content-Type Unique MessageId Schema &Version
  75. 75. // Message Properties: Examples of Good Practice // Being explicit is good practice final BasicProperties messageHeaders = new BasicProperties(); {{ DeliveryMode . NonPersistent . setOn( this ); Priority . Zero . setOn( this ); setMessageId( "My unique ID, eg a GUID or ascending value" ); // Explicit MIME type including charset; use application/octect-stream for trully binary data setContentType("application/json; charset=utf-8"); // Use compression for larger textual data; use the same values as HTTP headers; use "identity" for none setContentEncoding( "gzip" ); // Timestamp each message setTimestamp( new Date( System . currentTimeMillis() ) ); // Schema and version, perhaps using a XML namespace definition setType( "syslog-message_1.0.4" ); // Optional: Use an expiration to discard old messages that are stale (milliseconds, example is 5 minutes) setExpiration( Integer . toString( 5 * 60 * 1000) ); // Optional: Identify the sender for later debugging setAppId( "sending application name" ); setUserId( "role of sender, or, user of machine" ); // Optional: Identify domain or environment (consumer could error if working in different one) setClusterId( "production" ); }};
  76. 76. And there is more! Transactions More Message Properties Custom Message Properties Immediate Delivery Additional Exchange Types Auto Deletion Fine-Grained User Permissions Queue Purging QoS
  77. 77. Useful Book: Enterprise Integration Patterns Hohpe &Woolf
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×