Scaling php applications with redis

20,673 views

Published on

Redis is a NoSQL technology that rides a fine line between database and in-memory cache. Redis also offers "remote data structures", which gives it a significant advantage over other in-memory databases. This session will cover several PHP clients for Redis, and how to use them for caching, data modeling and generally improving application throughput.

Published in: Technology
2 Comments
41 Likes
Statistics
Notes
No Downloads
Views
Total views
20,673
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
311
Comments
2
Likes
41
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Can do increment, etc on individual fields\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Scaling php applications with redis

    1. 1. Scaling PHP Applications with RedisJosh ButtsZendcon 2011http://joind.in/3783
    2. 2. About Me• Director of Development at Vertive, LLC• Zend Certified in PHP 5 and ZF• Organizer of AustinPHP• Find me online: • Twitter: @jimbojsb • Github: jimbojsb
    3. 3. Agenda• Redis Overview• Types of keys• PHP Strategies
    4. 4. What is Redis?• Redis is an “Advanced” key-value store database• Backed by VMWare• Redis works like memcached, but better: • Atomic commands & transactions • Server-side data structures • In-memory + disk persistence
    5. 5. Redis in an in-memory database• You need as much RAM as you have data• There are no plans to improve support for “virtual” memory• Disk persistence is exactly that• Customize disk writes interval to suit your pain threshold• Something interesting happens when you run out of memory
    6. 6. How to get Redis• http://redis.io• Doesn’t run (well) on Windows • There are a few ports out there• Has virtually no compile dependencies• Most distros have a package• Make sure you’re running at least 2.2 • 2.4.1 became stable 10/17/11
    7. 7. How to explore• There aren’t any good GUI tools out there• redis-cli is your friend
    8. 8. A bit about organization• Redis can have multiple databases • Analogous to a MySQL database within a server instance • Theoretically, over 1000 databases per server • One data file on disk per instance• Within a database, namespace your keys • Ex: myapp:mykey • Keep them concise but useful. Keys take memory too!
    9. 9. Types of Keys
    10. 10. String Keys http://redis.io/commands#string• Simple key-value• Memcache equivalent• Common Commands • SET • GET • INCR • STRLEN
    11. 11. Hash Keys http://redis.io/commands#hash• Key + multiple fields / values • Common commands• Like an associative array in PHP • HSET• mykey => [field1 => value1, field2 => value2] • HGET• No nesting (unlike PHP) • HGETALL • HDEL • HVALS • HKEYS
    12. 12. Hash Keys
    13. 13. Set Keys http://redis.io/commands#set• key + unordered list of strings• myset => [item2, item5, item1,]• Common Commands • SADD • SMEMBERS • SISMEMBER • SREM
    14. 14. Set Keys
    15. 15. List Keys http://redis.io/commands#list• Like sets, except insertion order matters• Build queues or stacks• Optional blocking• Common commands • [B]LPUSH • [B]LPOP • LLEN
    16. 16. List Keys
    17. 17. Sorted Set Keys• Like sets, but sorted by a user-provided score value• Extremely fast access by score or range of scores, because it’s sorted in storage• Common commands • ZADD • ZRANGE • ZREVRANGE
    18. 18. Sorted Set Keys
    19. 19. Other commands that work on all keys• DEL - delete a key, regardless of type• KEYS - search for keys (usually with a wildcard)• EXPIRE / PERSIST - change expiration values for keys
    20. 20. Connecting from PHP
    21. 21. Connecting from PHP• Several libraries out there • Rediska (http://rediska.geometria-lab.net/) • PHP 5.2 / ZF 1.x friendly • Predis (https://github.com/nrk/predis) • The best in my experience • Requires PHP 5.3
    22. 22. Predis• All the examples here assume you’re using Predis• Connect to a localhost redis: $p = new PredisClient();• Redis commands implemented as magic methods on Client();• $p->set($key, $val);• $val = $p->get($key);
    23. 23. Attribute Display Logic items_attributes items id INT(11) id INT(11) item_id INT(11) attr_name VARCHAR(32) name VARCHAR(32) attr_value VARCHAR(32) 10k rows 100k rows• An item, a Dell Latitude Laptop, has: • Free Shipping, Financing Available, Expires Soon, etc
    24. 24. Attribute Display Logic • Display “Free Shipping” graphic on the item if it has a free shipping attribute row 50 items per page
    25. 25. Attribute Display - Traditional class Item { public function hasAttribute($name) { $sql = "SELECT 1 FROM items_attributes WHERE item_id = $this->id AND attr_name=$name LIMIT 1"; $result = $this->pdo->execute($sql); return $result != false; } }
    26. 26. Denormalize data from MySQL to Redis• Smart caching• Define a consistent way to represent a relational object in Redis • I prefer [object class]:[object id]:[attribute] • ex: product:13445:num_comments • This prevents data collisions, and makes it easy to work with data on the command line
    27. 27. Attribute Display - Redisclass Item{ public function hasAttribute($name) { return $this->redis->sismember(“item:$this->id:attributes”, $name); } public function addAttribute($name, $value) { //traditional mysql stuff here still $this->redis->sadd(item:45:attributes, $name); }}
    28. 28. Advantages• The more items you have, the less MySQL will scale this solution for you on it’s own• Frequently updating your items kills the MySQL query cache• Checking existence of a set member is O(1) time• On a laptop, I can check roughly 10,000 attributes per second
    29. 29. Session Clustering WEB - 1 WEB - 2 WEB - 3 PHP Sessions PHP Sessions PHP Sessions Inconsistent state
    30. 30. Session Clustering WEB - 1 WEB - 2 WEB - 3 DB - 1 PHP SessionsSlow with many users (never cached), replication lag
    31. 31. Session Clustering WEB - 1 WEB - 2 WEB - 3 REDIS - 1 DB - 1 PHP Sessions Constant time lookups, in-memory sessions
    32. 32. Job Queues• Redis lists make great job queues• Offload your intensive workloads to some other process• Blocking I/O allows you to easily build long-running daemons
    33. 33. Job Queuesclass Queue{ protected $name; protected $predis; public function push(Array $job) { $this->predis->lpush($this->name, json_encode($job)); } public function pop($block = false) { $job = null; if ($block) { $data = $this->predis->brpop($this->name, 0); $job = $data[1]; } else { $job = $this->predis->rpop($this->name); } if ($job) { return json_decode($job); } }}
    34. 34. Queuing Jobs$q = new Queue(test_queue);$form = new My_Zend_Form();if ($form->isValid($_POST)) { $q->push($form->getValues()); $message = “Thanks for your submission”;} else { $message = “Error - something went wrong”;}echo “<h1>$message</h1>”;
    35. 35. Processing Jobs - Crontab style function processJob(Array $job) { //...something really cool here // throw an exception on error } // process all pending jobs $q = new Queue(‘test_queue’); while ($job = $q->pop()) { try { processJob($job); } catch (Exception $e) { Echo “error processing job”; $q = new Queue(‘errors’); $q->push($job); } }
    36. 36. Processing Jobs - Worker stylefunction processJob(Array $job){ //...something really cool here // throw an exception on error}// keep processing jobs as they become available$q = new Queue(‘test_queue’);while ($job = $q->pop(true)) { try { processJob($job); } catch (Exception $e) { Echo “error processing job”; $q = new Queue(‘errors’); $q->push($job); }}
    37. 37. MVC Routing• Example: • Offers.com has about 3900 stores • Store pages live at /[store]/
    38. 38. Old Store Routing class Store_Route { public function route($url) { $sql = “SELECT id, type FROM stores WHERE url=’$url’”; $store = $this->pdo->execute($sql); //... do logic to determine what action to use based on type ...// return array(“module” => “core”, “controller” => “store”, “action” => $action); } }• And then there’s offers.com/[holiday]/....
    39. 39. New Routingclass Redis_Route{ public function route($url) { $p = $this->predis; if ($p->exists($url)) { list($module, $controller, $action) = $this->redis->hVals($url); return array(“module” => $module, “controller” => $controller, “action” => $action); } return false; }}
    40. 40. Filling in the Redis keys class Store { public function create(Array $data) { // ... traditional SQL stuff to put store in the database ... // $route = array(“module” => “core”, “controller” => “store”, “action” => $data[“type”]); $this->predis->hmset($data[“url”], $route); } }
    41. 41. Advantages• I can now create offers.com/[anything]/ and route it to the right place, in O(1) time• I’m only adding a few lines of code to my existing models
    42. 42. Questions?
    43. 43. http://joind.in/3783
    44. 44. Vertive is Hiring• We help people save money• We’re looking for engineers in Austin, TX to work on:

    ×