1. Scaling PHP Applications with Redis
Josh Butts
Zendcon 2011
http://joind.in/3783

2. About Me
• Director of Development at Vertive, LLC
• Zend Certified in PHP 5 and ZF
• Organizer of AustinPHP
• Find me online:
• Twitter: @jimbojsb
• Github: jimbojsb

3. Agenda
• Redis Overview
• Types of keys
• PHP Strategies

4. What is Redis?
• Redis is an “Advanced” key-value store database
• Backed by VMWare
• Redis works like memcached, but better:
• Atomic commands & transactions
• Server-side data structures
• In-memory + disk persistence

5. Redis in an in-memory database
• You need as much RAM as you have data
• There are no plans to improve support for “virtual” memory
• Disk persistence is exactly that
• Customize disk writes interval to suit your pain threshold
• Something interesting happens when you run out of memory

6. How to get Redis
• http://redis.io
• Doesn’t run (well) on Windows
• There are a few ports out there
• Has virtually no compile dependencies
• Most distros have a package
• Make sure you’re running at least 2.2
• 2.4.1 became stable 10/17/11

7. How to explore
• There aren’t any good GUI tools out there
• redis-cli is your friend

8. A bit about organization
• Redis can have multiple databases
• Analogous to a MySQL database within a server instance
• Theoretically, over 1000 databases per server
• One data file on disk per instance
• Within a database, namespace your keys
• Ex: myapp:mykey
• Keep them concise but useful. Keys take memory too!

9. Types of Keys

10. String Keys http://redis.io/commands#string
• Simple key-value
• Memcache equivalent
• Common Commands
• SET
• GET
• INCR
• STRLEN

11. Hash Keys http://redis.io/commands#hash
• Key + multiple fields / values • Common commands
• Like an associative array in PHP • HSET
• mykey => [field1 => value1, field2 => value2] • HGET
• No nesting (unlike PHP) • HGETALL
• HDEL
• HVALS
• HKEYS

12. Hash Keys

13. Set Keys http://redis.io/commands#set
• key + unordered list of strings
• myset => [item2, item5, item1,]
• Common Commands
• SADD
• SMEMBERS
• SISMEMBER
• SREM

14. Set Keys

15. List Keys http://redis.io/commands#list
• Like sets, except insertion order matters
• Build queues or stacks
• Optional blocking
• Common commands
• [B]LPUSH
• [B]LPOP
• LLEN

16. List Keys

17. Sorted Set Keys
• Like sets, but sorted by a user-provided score value
• Extremely fast access by score or range of scores, because it’s sorted in
storage
• Common commands
• ZADD
• ZRANGE
• ZREVRANGE

18. Sorted Set Keys

19. Other commands that work on all keys
• DEL - delete a key, regardless of type
• KEYS - search for keys (usually with a wildcard)
• EXPIRE / PERSIST - change expiration values for keys

20. Connecting from PHP

21. Connecting from PHP
• Several libraries out there
• Rediska (http://rediska.geometria-lab.net/)
• PHP 5.2 / ZF 1.x friendly
• Predis (https://github.com/nrk/predis)
• The best in my experience
• Requires PHP 5.3

22. Predis
• All the examples here assume you’re using Predis
• Connect to a localhost redis: $p = new PredisClient();
• Redis commands implemented as magic methods on Client();
• $p->set($key, $val);
• $val = $p->get($key);

23. Attribute Display Logic
items_attributes
items
id INT(11)
id INT(11) item_id INT(11)
attr_name VARCHAR(32)
name VARCHAR(32)
attr_value VARCHAR(32)
10k rows 100k rows
• An item, a Dell Latitude Laptop, has:
• Free Shipping, Financing Available, Expires Soon, etc

24. Attribute Display Logic
• Display “Free Shipping” graphic on the item if it has a free
shipping attribute row
50 items per page

25. Attribute Display - Traditional
class Item
{
public function hasAttribute($name)
{
$sql = "SELECT 1
FROM items_attributes
WHERE item_id = $this->id
AND attr_name='$name'
LIMIT 1";
$result = $this->pdo->execute($sql);
return $result != false;
}
}

26. Denormalize data from MySQL to Redis
• Smart caching
• Define a consistent way to represent a relational object in Redis
• I prefer [object class]:[object id]:[attribute]
• ex: product:13445:num_comments
• This prevents data collisions, and makes it easy to work with data on
the command line

27. Attribute Display - Redis
class Item
{
public function hasAttribute($name)
{
return $this->redis->sismember(“item:$this->id:attributes”, $name);
}
public function addAttribute($name, $value)
{
//traditional mysql stuff here still
$this->redis->sadd('item:45:attributes', $name);
}
}

28. Advantages
• The more items you have, the less MySQL will scale this solution for you on
it’s own
• Frequently updating your items kills the MySQL query cache
• Checking existence of a set member is O(1) time
• On a laptop, I can check roughly 10,000 attributes per second

29. Session Clustering
WEB - 1 WEB - 2 WEB - 3
PHP Sessions PHP Sessions PHP Sessions
Inconsistent state

30. Session Clustering
WEB - 1 WEB - 2 WEB - 3
DB - 1
PHP Sessions
Slow with many users (never cached), replication lag

31. Session Clustering
WEB - 1 WEB - 2 WEB - 3
REDIS - 1 DB - 1
PHP Sessions
Constant time lookups, in-memory sessions

32. Job Queues
• Redis lists make great job queues
• Offload your intensive workloads to some other process
• Blocking I/O allows you to easily build long-running daemons

33. Job Queues
class Queue
{
protected $name;
protected $predis;
public function push(Array $job)
{
$this->predis->lpush($this->name, json_encode($job));
}
public function pop($block = false)
{
$job = null;
if ($block) {
$data = $this->predis->brpop($this->name, 0);
$job = $data[1];
} else {
$job = $this->predis->rpop($this->name);
}
if ($job) {
return json_decode($job);
}
}
}

34. Queuing Jobs
$q = new Queue('test_queue');
$form = new My_Zend_Form();
if ($form->isValid($_POST)) {
$q->push($form->getValues());
$message = “Thanks for your submission”;
} else {
$message = “Error - something went wrong”;
}
echo “<h1>$message</h1>”;

35. Processing Jobs - Crontab style
function processJob(Array $job)
{
//...something really cool here
// throw an exception on error
}
// process all pending jobs
$q = new Queue(‘test_queue’);
while ($job = $q->pop()) {
try {
processJob($job);
} catch (Exception $e) {
Echo “error processing job”;
$q = new Queue(‘errors’);
$q->push($job);
}
}

36. Processing Jobs - Worker style
function processJob(Array $job)
{
//...something really cool here
// throw an exception on error
}
// keep processing jobs as they become available
$q = new Queue(‘test_queue’);
while ($job = $q->pop(true)) {
try {
processJob($job);
} catch (Exception $e) {
Echo “error processing job”;
$q = new Queue(‘errors’);
$q->push($job);
}
}

37. MVC Routing
• Example:
• Offers.com has about 3900 stores
• Store pages live at /[store]/

38. Old Store Routing
class Store_Route
{
public function route($url)
{
$sql = “SELECT id, type
FROM stores
WHERE url=’$url’”;
$store = $this->pdo->execute($sql);
//... do logic to determine what action to use based on type ...//
return array(“module” => “core”,
“controller” => “store”,
“action” => $action);
}
}
• And then there’s offers.com/[holiday]/....

39. New Routing
class Redis_Route
{
public function route($url)
{
$p = $this->predis;
if ($p->exists($url)) {
list($module, $controller, $action) = $this->redis->hVals($url);
return array(“module” => $module,
“controller” => $controller,
“action” => $action);
}
return false;
}
}

40. Filling in the Redis keys
class Store
{
public function create(Array $data)
{
// ... traditional SQL stuff to put store in the database ... //
$route = array(“module” => “core”,
“controller” => “store”,
“action” => $data[“type”]);
$this->predis->hmset($data[“url”], $route);
}
}

41. Advantages
• I can now create offers.com/[anything]/ and route it to the right place, in
O(1) time
• I’m only adding a few lines of code to my existing models

42. Questions?

43. http://joind.in/3783

44. Vertive is Hiring
• We help people save money
• We’re looking for engineers in Austin, TX to work on: