SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.
SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.
Successfully reported this slideshow.
Activate your 14 day free trial to unlock unlimited reading.
2.
WHAT !?
RestMQ - Message Queue based on Redis
Data manipulation:
HTTP GET/POST/DELETE, COMET and WebSockets
No signaling or specific protocol (optional JSON protocol)
http://www.restmq.com/
Gleicon Moraes
http://github.com/gleicon
http://zenmachine.wordpress.com
http://www.7co.cc
3.
Key/Value storage
Think memcached, GET and SET operations
[key] = value
>> SET key meh
>> GET key
meh
>> SET fucounter 0
>> INCR fucounter
1
>> GET fucounter
1
4.
Redis
Key-Value database
Atomic operations
Semi-persistent or persistent storage
Publish/Subscription channels
Different Data types: Sets, Sorted Sets, Lists, Hashes
http://code.google.com/p/redis/
http://github.com/antirez/redis
http://rediscookbook.org/
5.
Redis
Data types
Strings
Lists
Sets and Ordered Sets
Hashes
Publish/Subscribe channels
6.
Redis
Things you can do:
Cache
Inverted indexes
Fast counters
Throttle control
Cooler stuff at Redis Cookbook
Load Balancing
Things you can't do:
Search inside all values for a given string
7.
Redis
Things you can't do:
- Search inside all values for a given key
- Search inside all values for a given key
- Search inside all values for a given key
8.
Message Queues
Does everything really needs to be tightly coupled ?
Order: first come, first served
Transactions: there is no transaction, just tracking
Job Scheduling
Publish/Subscribe
Map/Reduce
9.
Message Queues
SMTP: Oldest Message Queue around
Avoid hitting your DB real time with MQ
Real time: Ratings, Voting and Comments
Twitter
Do I need NoSQL or I really need to clean my mess ?
12.
RestMQ - Messages
/c/<queue> : Comet Endpoint
$ curl http://localhost:8888/c/testq
(hangs until there is a message to be received)
/ws/<queue> : WebSocket Endpoint
Needs the proper javascript code
13.
RestMQ - Messages
JSON Protocol inspired by Amazon SQS
First prototype at http://jsonqueue.appspot.com
add get
{ {
"cmd" : "add", "cmd" : "get",
"queue" : "testq", "queue" : "testq",
"value" : "abacab" }
}
del take
{ {
"cmd" : "del", "cmd" : "take",
"queue" : "testq", "queue" : "testq",
"key" : "testq:31" }
}
15.
RestMQ - Controls
/p/<queue> : Distribution Policy for COMET and WS
Round Robin a message for each consumer
Broadcast a message to all consumers
/control/<queue> : Queue start/stop
Pause all COMET consumers
(useful for job schedulers)
17.
RestMQ - Algorithm
Algorithm for pushing messages into a queue named 'testq'
SADD testq into a SET called queueset
INCR a per-queue UUID generator called testq:uuid
SET testq:<<id>>, message
LPUSH testq:<<id>> in to a LIST named testq:queue
Works for new and existing queues. On-the-fly queue creation.
18.
RestMQ - Algorithm
Algorithm for reading messages from a queue named 'testq'
RPOP a key from a LIST named testq:queue
GET the value associated to this key
For non-destructive GET operations:
LINDEX -1 to get a key in testq:queue LIST
INCR key:refcount +1
GET the value stored for key
19.
RestMQ - Implementation
(the core functionality can be implemented on any language)
Main branch 'Enterprise Edition'
Python
Twisted
Cyclone
Redis async client
Embedded version
Sinatra + Ruby
Reduced endpoints (GET/POST)
20.
RestMQ - Sinatra Implementation
http://gist.github.com/524240
GET /q - List all queues
21.
RestMQ - Sinatra Implementation
http://gist.github.com/524240
GET /q/<queue> - Get a message from <queue>
22.
RestMQ - Sinatra Implementation
http://gist.github.com/524240
POST /q/<queue> - Insert a message in <queue>