A S Y N C H R O N O U S D ATA P R O C E S S I N G
Andrea Giuliano
@bit_shark
B u l g a r i a P H P
C O N F E R E N C ESeptember 27th, 2015
Andrea Giuliano
@bit_shark
$WHOAMI
COMMUNICATION
SYNCHRONOUS
COMMUNICATION
time
system A
system B
Request
Response
blocked
Pros Cons
• easy to reason about
• immediate feedback
• client can get stuck
• impractical in low latency env
SYNCHRONOUS
COMMUNICATION
ASYNCHRONOUS
COMMUNICATION
time
system A
system B
Request
• useful for long-running jobs
• systems decoupled
• scales well (event-based)
• additional system
requirements
ASYNCHRONOUS
COMMUNICATION
Pros Cons
INTEROPERABILITY
ADVANCED MESSAGE
QUEUING PROTOCOL
• message orientation
• queuing
• routing
• reliability
• security
P R O D U C E R C O N S U M E R
Produce Consumes
B R O K E R
ENTITIES INVOLVED
P R O D U C E R C O N S U M E R
Produce
Exchange
Consumes
B R O K E R
THE BROKER
P R O D U C E R C O N S U M E R
Produce
Exchange
Routes
Consumes
B R O K E R
THE BROKER
P R O D U C E R C O N S U M E R
Produce
Exchange Queue
Routes
Consumes
B R O K E R
THE BROKER
MESSAGING PATTERNS
{
"require": {
"videlalvaro/php-amqplib": "@stable",
...
}
}
$ composer.phar install
PRODUCER/CONSUMER
P R O D U C E R C O N S U M E R
Q U E U E
PRODUCER/CONSUMER
PRODUCING MESSAGES
PRODUCING MESSAGES
PRODUCING MESSAGES
PRODUCING MESSAGES
CONSUMING MESSAGES
CONSUMING MESSAGES
CONSUMING MESSAGES
SCALING UP
P R O D U C E R
C O N S U M E R
Q U E U E
C O N S U M E R
…
CONSUMERS CAN DIE
MESSAGE
ACKNOWLEDGEMENT
MESSAGE ACKS
switch on acks
send ack
QOS POLICY
C O N S U M E R
C O N S U M E R
ROUND ROBIN DISPATCHING
C O N S U M E R
C O N S U M E R
ROUND ROBIN DISPATCHING
C O N S U M E R
C O N S U M E R
ROUND ROBIN DISPATCHING
C O N S U M E R
C O N S U M E R
ROUND ROBIN DISPATCHING
!
FAIR DISPATCHING
C O N S U M E R
C O N S U M E R
FAIR DISPATCHING
C O N S U M E R
C O N S U M E R
FAIR DISPATCHING
C O N S U M E R
C O N S U M E R
FAIR DISPATCHING
C O N S U M E R
C O N S U M E R
ack
FAIR DISPATCHING
C O N S U M E R
C O N S U M E R
FAIR DISPATCHING
BROKER CAN DIE
DURABILITY
DURABILITY
declare the queue
persistent
PUBLISH/SUBSCRIBE
PUBLISH/SUBSCRIBE
P U B L I S H E R
S U B S C R I B E R
Q U E U E
S U B S C R I B E R S U B S C R I B E R
PUBLISH/SUBSCRIBE
M A I L
S E R V I C E
A U T H
S E R V I C E
C O U P O N
S E R V I C E
B R O K E R
tell me when a user is created
tell me when a user is created
tell me when a user is created
PUBLISH/SUBSCRIBE
M A I L
S E R V I C E
A U T H
S E R V I C E
C O U P O N
S E R V I C E
U S E R
S E R V I C E
B R O K E R
username: agiuliano
password: ******
mail: ***@gmail.com
UserCreated
PUBLISH/SUBSCRIBE
M A I L
S E R V I C E
A U T H
S E R V I C E
C O U P O N
S E R V I C E
B R O K E R
UserCreated
UserCreated
UserCreated
PUBLISH/SUBSCRIBE
U S E R
S E R V I C E
B R O K E R
UserCreated
PUBLISH/SUBSCRIBE
M A I L
S E R V I C E
B R O K E R
tellmewhenauseriscreated
PUBLISH/SUBSCRIBEUserCreated
M A I L
S E R V I C E
B R O K E R
ROUTING
ROUTING
M A I L
S E R V I C E
A U T H
S E R V I C E
C O U P O N
S E R V I C E
B R O K E R
[UserCreated, UserUpdated]
UserCreated
UserCreated
ROUTING
U S E R
S E R V I C E
B R O K E R
UserUpdated
routing key: updated
ROUTING
M A I L
S E R V I C E
B R O K E R
routing keys:
[created, updated]
ROUTING
M A I L
S E R V I C E
B R O K E R
routing keys:
[created, updated]
TOPIC
TOPIC
M A I L
S E R V I C E
A U T H
S E R V I C E
C O U P O N
S E R V I C E
B R O K E R
[user. *]
[user. created]
[user. created, *.purchased]
TOPIC
TOPIC
M A I L
S E R V I C E
A U T H
S E R V I C E
C O U P O N
S E R V I C E
B R O K E R
[user. *]
[user. created]
[user. created, *.purchased]
U S E R
S E R V I C E
<user. created>
TOPIC
M A I L
S E R V I C E
A U T H
S E R V I C E
C O U P O N
S E R V I C E
B R O K E R
[user. *]
[user. created]
[user. created, *.purchased]
U S E R
S E R V I C E
<user. created>
TOPIC
M A I L
S E R V I C E
A U T H
S E R V I C E
C O U P O N
S E R V I C E
B R O K E R
[user. *]
[user. created]
[user. created, *.purchased]
U S E R
S E R V I C E
<user. updated>
TOPIC
M A I L
S E R V I C E
A U T H
S E R V I C E
C O U P O N
S E R V I C E
B R O K E R
[user. *]
[user. created]
[user. created, *.purchased]
U S E R
S E R V I C E
<user. updated>
TOPIC
M A I L S E R V I C E
A U T H
S E R V I C E
C O U P O N
S E R V I C E
B R O K E R
[user. *]
[user. created]
[user. created, *.purchased]
S T O R E
S E R V I C E
<item.purchased>
TOPIC
M A I L S E R V I C E
A U T H
S E R V I C E
C O U P O N
S E R V I C E
B R O K E R
[user. *]
[user. created]
[user. created, *.purchased]
S T O R E
S E R V I C E
<item.purchased>
TOPIC
M A I L S E R V I C E
A U T H
S E R V I C E
C O U P O N
S E R V I C E
B R O K E R
[user. *]
[user. created]
[user. created, *.purchased]
S T O R E
S E R V I C E
<item.created>
TOPIC
M A I L S E R V I C E
A U T H
S E R V I C E
C O U P O N
S E R V I C E
B R O K E R
[user. *]
[user. created]
[user. created, *.purchased]
S T O R E
S E R V I C E
<item.created>
SUMMARY
PERFORMANCE
DECOUPLED SYSTEM
INTEROPERABILITY
SCALABILITY
joind.in/14878
Please rate the talk
joind.in/14878
CREDITS
• www.flickr.com/photos/rayofsun/9401226342
• www.flickr.com/photos/kalexanderson/6231391820
• www.flickr.com/photos/dhammza/2891991931
• www.flickr.com/photos/kevlewis/15372562472
• www.flickr.com/photos/hohumhobo/6349729535
• www.flickr.com/photos/107346206
• www.flickr.com/photos/berlin_steve/8414515260
• www.flickr.com/photos/s3a/4816966416
• www.flickr.com/photos/thatguyfromcchs08/2300190277
• www.flickr.com/photos/toniblay/52445415
• www.flickr.com/photos/bobsfever/8226802321
• www.flickr.com/photos/activars/6590201161
• www.flickr.com/photos/michaeljzealot/6485009571

Asynchronous data processing