Achieving Scale with
Messaging & the Cloud
Tony Garnock-Jones <tonyg@lshift.net>
Thursday, 9 July 2009 1

Outline
• Messaging – what, why
• Cloud Messaging – some examples
• Achieving Scale – what, why, how
Thursday, 9 July 2009 2

Messaging
Thursday, 9 July 2009 3

Messaging Elements
...relaying, filtering, multicasting, forwarding, buffering,
distribution/scheduling, subscription, topics, streaming, ...
Element
Element
...
Element
Element
Element
Element
...
Thursday, 9 July 2009 4

What is it good for?
Decoupling communication from behaviour, enabling:
• debuggability • cross-language
integration
• manageability
• fault-tolerance
• monitorability
• store-and-forward
• rolling upgrades
• ...
• load-balancing
Thursday, 9 July 2009 5

Messaging Elements
...relaying, filtering, multicasting, forwarding, buffering,
distribution/scheduling, subscription, topics, streaming, ...
messages
message
Element
subscribe unsubscribe
Thursday, 9 July 2009 6

Messaging Elements
• Delivery:
• deliver(target, message);
• Subscription:
• bind(source, topic, target);
• unbind(source, topic, target);
Thursday, 9 July 2009 7

AMQP for Messaging
C
P X
C
P X
C
• Exchanges perform • Queues perform
relaying, copying, and buffering and round-
filtering robin delivery
Thursday, 9 July 2009 8

It’s Easy
byte[] body = ...;
ch.basicPublish(exchange, routingKey, null, body);
ch.queueBind(queueName, exchange, routingKeyPattern);
QueueingConsumer consumer = new QueueingConsumer(ch);
ch.basicConsume(queueName, consumer);
while (true) {
QueueingConsumer.Delivery delivery =
consumer.nextDelivery();
// ... use delivery.getBody(), etc ...
ch.basicAck(delivery.getEnvelope().getDeliveryTag(),
false);
}
Thursday, 9 July 2009 9

RabbitMQ Universe
...plus new
developments like
RabbitHub, BBC
FeedsHub, Trixx, ...
Thursday, 9 July 2009 10

Cloud Messaging
Thursday, 9 July 2009 11

Examples
• Soocial are using RabbitMQ + EC2 to
coordinate their contacts-database
synchronisation application
• The Ocean Observatories Initiative is using
RabbitMQ + relays + gateways for a global
sensing & data distribution network
Thursday, 9 July 2009 12

Soocial.com
Thursday, 9 July 2009 13

OOI – Ocean
Observatories Initiative
Thursday, 9 July 2009 14

OOI – Ocean
Observatories Initiative
Service.DomainName
Region Virtual IP
Cloud IPs
Thursday, 9 July 2009 15

Nanite
• Developed by Ezra Zygmuntowicz at
Engineyard
• Presence (by pinging; we’re still working out the best way
of putting presence into RabbitMQ itself)
• Self-assembling compute fabric
• Load-balancing & recovers from failure
• An easy way of deploying to the cloud
Thursday, 9 July 2009 16

VPN3 (VPN-cubed)
• A secure Virtual Private Network in the
cloud
• Uses RabbitMQ as a backend for IP
multicast (!) which otherwise doesn’t work in EC2
• Uses BGP to arrange routing between
nodes
• Can meld your datacentre(s) with EC2
Thursday, 9 July 2009 17

Achieving Scale
Thursday, 9 July 2009 18

Paul Baran, Introduction to Distributed Communications Networks, 1964
Paul Baran’s Networks
Thursday, 9 July 2009 19

Achieving Scale
• Capacity – throughput, latency,
responsiveness
• Synchronisation & Sharing
• Availability – uptime, data integrity
• Network Management – lots of devices,
lots of addresses, authentication &
authorisation
Thursday, 9 July 2009 20

Capacity
• Service capacity: bandwidth, throughput
• Service responsiveness: latency
• Data capacity: storage
Thursday, 9 July 2009 21

Simple load-balancing
Request Queue
(Private)
Reply
Queue
• Shared queue mediates • Load-balancing, live
access to service upgrades, fault-tolerance
instances
Thursday, 9 July 2009 22

Clustering
Thursday, 9 July 2009 23

Clustering
Thursday, 9 July 2009 24

Clustering
Physical
Machine
Erlang Node
Listening
Socket
Message Erlang’s inter-node
Store message routing
Virtual Host
Thursday, 9 July 2009 25

Divide and Conquer
• By record – parallelisable data lets separate
broker clusters handle separate shards of
your data
• By component – each independent part of
your application can use separate brokers;
“federation of distributed systems”
Thursday, 9 July 2009 26

WAN Cluster
The WAN cluster has local
queues but global exchanges
Apps decide
which broker-
cluster to
publish to
Thursday, 9 July 2009 27

Overfeeding the server
• Message backlogs can cause catastrophe
• Goal: O(0) RAM cost per message
• Spool out to disk when memory pressure
gets high
• Testing and tuning: should land for
RabbitMQ 1.7
Thursday, 9 July 2009 28

Synchronisation
When clustering might
not be right:
• huge networks
• intermittent
connectivity
• ruling bandwidth
with an iron fist
• different
administrative
domains
Thursday, 9 July 2009 29

Synchronisation
Thursday, 9 July 2009 30

Ring
Pass it on to your
neighbour if your
neighbour’s name
isn’t in the list yet
Thursday, 9 July 2009 31

Complete
Graph
Pass it on to your
neighbour if it
hasn’t been
labelled at all yet
Thursday, 9 July 2009 32

Multicast
Pass it on to your
neighbour if it
hasn’t been
labelled at all yet
Thursday, 9 July 2009 33

Availability
• Broker Availability
• Application/Service Availability
• Data Availability
Thursday, 9 July 2009 34

Responsibility transfer
Consumer Broker
basic.deliver
basic.deliver
basic.deliver
...
basic.ack
Thursday, 9 July 2009 35

Responsibility transfer
Producer Broker
basic.publish
basic.publish
basic.publish
...
tx.commit
tx.commit-ok
Thursday, 9 July 2009 36

Responsibility transfer
Producer Broker Consumer
...
...
Thursday, 9 July 2009 37

Responsibility transfer
Producer Broker Consumer
Thursday, 9 July 2009 38

Redundancy for HA
Pfailure = (Ploss) n
(assuming independence)
Thursday, 9 July 2009 39

Redundancy for HA
Pfailure = (Ploss) n
Uptime Ploss n Pfailure Overall Uptime
99% 0.01 2 0.0001 99.99%
99% 0.01 3 0.000001 99.9999%
99.9% 0.001 2 0.000001 99.9999%
99.9% 0.001 3 0.000000001 99.9999999%
(assuming independence)
Thursday, 9 July 2009 39

Redundancy for HA
P P
One X X X X Two
broker, brokers,
two nodes one node
each each
C C
Thursday, 9 July 2009 40

Deduplication
(“Idempotency Barrier”)
id=123 id=123
id=123
id=123
many messages one message
enter leaves
Thursday, 9 July 2009 41

Network Management
• Small systems can be managed by hand
• Large systems need automation:
• Naming & address management
• Service directories
• Authentication & authorization
• Logging, monitoring & alerting
Thursday, 9 July 2009 42

The Future
Thursday, 9 July 2009 43

Questions?
http://www.rabbitmq.com/how
Thursday, 9 July 2009 44