Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Scaling php applications with redis

21,620 views

Published on

Redis is a NoSQL technology that rides a fine line between database and in-memory cache. Redis also offers "remote data structures", which gives it a significant advantage over other in-memory databases. This session will cover several PHP clients for Redis, and how to use them for caching, data modeling and generally improving application throughput.

Published in: Technology

Scaling php applications with redis

  1. 1. Scaling PHP Applications with RedisJosh ButtsZendcon 2011http://joind.in/3783
  2. 2. About Me• Director of Development at Vertive, LLC• Zend Certified in PHP 5 and ZF• Organizer of AustinPHP• Find me online: • Twitter: @jimbojsb • Github: jimbojsb
  3. 3. Agenda• Redis Overview• Types of keys• PHP Strategies
  4. 4. What is Redis?• Redis is an “Advanced” key-value store database• Backed by VMWare• Redis works like memcached, but better: • Atomic commands & transactions • Server-side data structures • In-memory + disk persistence
  5. 5. Redis in an in-memory database• You need as much RAM as you have data• There are no plans to improve support for “virtual” memory• Disk persistence is exactly that• Customize disk writes interval to suit your pain threshold• Something interesting happens when you run out of memory
  6. 6. How to get Redis• http://redis.io• Doesn’t run (well) on Windows • There are a few ports out there• Has virtually no compile dependencies• Most distros have a package• Make sure you’re running at least 2.2 • 2.4.1 became stable 10/17/11
  7. 7. How to explore• There aren’t any good GUI tools out there• redis-cli is your friend
  8. 8. A bit about organization• Redis can have multiple databases • Analogous to a MySQL database within a server instance • Theoretically, over 1000 databases per server • One data file on disk per instance• Within a database, namespace your keys • Ex: myapp:mykey • Keep them concise but useful. Keys take memory too!
  9. 9. Types of Keys
  10. 10. String Keys http://redis.io/commands#string• Simple key-value• Memcache equivalent• Common Commands • SET • GET • INCR • STRLEN
  11. 11. Hash Keys http://redis.io/commands#hash• Key + multiple fields / values • Common commands• Like an associative array in PHP • HSET• mykey => [field1 => value1, field2 => value2] • HGET• No nesting (unlike PHP) • HGETALL • HDEL • HVALS • HKEYS
  12. 12. Hash Keys
  13. 13. Set Keys http://redis.io/commands#set• key + unordered list of strings• myset => [item2, item5, item1,]• Common Commands • SADD • SMEMBERS • SISMEMBER • SREM
  14. 14. Set Keys
  15. 15. List Keys http://redis.io/commands#list• Like sets, except insertion order matters• Build queues or stacks• Optional blocking• Common commands • [B]LPUSH • [B]LPOP • LLEN
  16. 16. List Keys
  17. 17. Sorted Set Keys• Like sets, but sorted by a user-provided score value• Extremely fast access by score or range of scores, because it’s sorted in storage• Common commands • ZADD • ZRANGE • ZREVRANGE
  18. 18. Sorted Set Keys
  19. 19. Other commands that work on all keys• DEL - delete a key, regardless of type• KEYS - search for keys (usually with a wildcard)• EXPIRE / PERSIST - change expiration values for keys
  20. 20. Connecting from PHP
  21. 21. Connecting from PHP• Several libraries out there • Rediska (http://rediska.geometria-lab.net/) • PHP 5.2 / ZF 1.x friendly • Predis (https://github.com/nrk/predis) • The best in my experience • Requires PHP 5.3
  22. 22. Predis• All the examples here assume you’re using Predis• Connect to a localhost redis: $p = new PredisClient();• Redis commands implemented as magic methods on Client();• $p->set($key, $val);• $val = $p->get($key);
  23. 23. Attribute Display Logic items_attributes items id INT(11) id INT(11) item_id INT(11) attr_name VARCHAR(32) name VARCHAR(32) attr_value VARCHAR(32) 10k rows 100k rows• An item, a Dell Latitude Laptop, has: • Free Shipping, Financing Available, Expires Soon, etc
  24. 24. Attribute Display Logic • Display “Free Shipping” graphic on the item if it has a free shipping attribute row 50 items per page
  25. 25. Attribute Display - Traditional class Item { public function hasAttribute($name) { $sql = "SELECT 1 FROM items_attributes WHERE item_id = $this->id AND attr_name=$name LIMIT 1"; $result = $this->pdo->execute($sql); return $result != false; } }
  26. 26. Denormalize data from MySQL to Redis• Smart caching• Define a consistent way to represent a relational object in Redis • I prefer [object class]:[object id]:[attribute] • ex: product:13445:num_comments • This prevents data collisions, and makes it easy to work with data on the command line
  27. 27. Attribute Display - Redisclass Item{ public function hasAttribute($name) { return $this->redis->sismember(“item:$this->id:attributes”, $name); } public function addAttribute($name, $value) { //traditional mysql stuff here still $this->redis->sadd(item:45:attributes, $name); }}
  28. 28. Advantages• The more items you have, the less MySQL will scale this solution for you on it’s own• Frequently updating your items kills the MySQL query cache• Checking existence of a set member is O(1) time• On a laptop, I can check roughly 10,000 attributes per second
  29. 29. Session Clustering WEB - 1 WEB - 2 WEB - 3 PHP Sessions PHP Sessions PHP Sessions Inconsistent state
  30. 30. Session Clustering WEB - 1 WEB - 2 WEB - 3 DB - 1 PHP SessionsSlow with many users (never cached), replication lag
  31. 31. Session Clustering WEB - 1 WEB - 2 WEB - 3 REDIS - 1 DB - 1 PHP Sessions Constant time lookups, in-memory sessions
  32. 32. Job Queues• Redis lists make great job queues• Offload your intensive workloads to some other process• Blocking I/O allows you to easily build long-running daemons
  33. 33. Job Queuesclass Queue{ protected $name; protected $predis; public function push(Array $job) { $this->predis->lpush($this->name, json_encode($job)); } public function pop($block = false) { $job = null; if ($block) { $data = $this->predis->brpop($this->name, 0); $job = $data[1]; } else { $job = $this->predis->rpop($this->name); } if ($job) { return json_decode($job); } }}
  34. 34. Queuing Jobs$q = new Queue(test_queue);$form = new My_Zend_Form();if ($form->isValid($_POST)) { $q->push($form->getValues()); $message = “Thanks for your submission”;} else { $message = “Error - something went wrong”;}echo “<h1>$message</h1>”;
  35. 35. Processing Jobs - Crontab style function processJob(Array $job) { //...something really cool here // throw an exception on error } // process all pending jobs $q = new Queue(‘test_queue’); while ($job = $q->pop()) { try { processJob($job); } catch (Exception $e) { Echo “error processing job”; $q = new Queue(‘errors’); $q->push($job); } }
  36. 36. Processing Jobs - Worker stylefunction processJob(Array $job){ //...something really cool here // throw an exception on error}// keep processing jobs as they become available$q = new Queue(‘test_queue’);while ($job = $q->pop(true)) { try { processJob($job); } catch (Exception $e) { Echo “error processing job”; $q = new Queue(‘errors’); $q->push($job); }}
  37. 37. MVC Routing• Example: • Offers.com has about 3900 stores • Store pages live at /[store]/
  38. 38. Old Store Routing class Store_Route { public function route($url) { $sql = “SELECT id, type FROM stores WHERE url=’$url’”; $store = $this->pdo->execute($sql); //... do logic to determine what action to use based on type ...// return array(“module” => “core”, “controller” => “store”, “action” => $action); } }• And then there’s offers.com/[holiday]/....
  39. 39. New Routingclass Redis_Route{ public function route($url) { $p = $this->predis; if ($p->exists($url)) { list($module, $controller, $action) = $this->redis->hVals($url); return array(“module” => $module, “controller” => $controller, “action” => $action); } return false; }}
  40. 40. Filling in the Redis keys class Store { public function create(Array $data) { // ... traditional SQL stuff to put store in the database ... // $route = array(“module” => “core”, “controller” => “store”, “action” => $data[“type”]); $this->predis->hmset($data[“url”], $route); } }
  41. 41. Advantages• I can now create offers.com/[anything]/ and route it to the right place, in O(1) time• I’m only adding a few lines of code to my existing models
  42. 42. Questions?
  43. 43. http://joind.in/3783
  44. 44. Vertive is Hiring• We help people save money• We’re looking for engineers in Austin, TX to work on:

×