Scaling php applications with redis

  • 15,824 views
Uploaded on

Redis is a NoSQL technology that rides a fine line between database and in-memory cache. Redis also offers "remote data structures", which gives it a significant advantage over other in-memory …

Redis is a NoSQL technology that rides a fine line between database and in-memory cache. Redis also offers "remote data structures", which gives it a significant advantage over other in-memory databases. This session will cover several PHP clients for Redis, and how to use them for caching, data modeling and generally improving application throughput.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • 'There aren't any good GUI tools out there' that was true in 2011 but now you should definitely give a try to Redsmin: https://redsmin.com :)
    Are you sure you want to
    Your message goes here
  • very good, brief, coincise but cover topic in depth
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
15,824
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
234
Comments
2
Likes
28

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Can do increment, etc on individual fields\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Transcript

  • 1. Scaling PHP Applications with RedisJosh ButtsZendcon 2011http://joind.in/3783
  • 2. About Me• Director of Development at Vertive, LLC• Zend Certified in PHP 5 and ZF• Organizer of AustinPHP• Find me online: • Twitter: @jimbojsb • Github: jimbojsb
  • 3. Agenda• Redis Overview• Types of keys• PHP Strategies
  • 4. What is Redis?• Redis is an “Advanced” key-value store database• Backed by VMWare• Redis works like memcached, but better: • Atomic commands & transactions • Server-side data structures • In-memory + disk persistence
  • 5. Redis in an in-memory database• You need as much RAM as you have data• There are no plans to improve support for “virtual” memory• Disk persistence is exactly that• Customize disk writes interval to suit your pain threshold• Something interesting happens when you run out of memory
  • 6. How to get Redis• http://redis.io• Doesn’t run (well) on Windows • There are a few ports out there• Has virtually no compile dependencies• Most distros have a package• Make sure you’re running at least 2.2 • 2.4.1 became stable 10/17/11
  • 7. How to explore• There aren’t any good GUI tools out there• redis-cli is your friend
  • 8. A bit about organization• Redis can have multiple databases • Analogous to a MySQL database within a server instance • Theoretically, over 1000 databases per server • One data file on disk per instance• Within a database, namespace your keys • Ex: myapp:mykey • Keep them concise but useful. Keys take memory too!
  • 9. Types of Keys
  • 10. String Keys http://redis.io/commands#string• Simple key-value• Memcache equivalent• Common Commands • SET • GET • INCR • STRLEN
  • 11. Hash Keys http://redis.io/commands#hash• Key + multiple fields / values • Common commands• Like an associative array in PHP • HSET• mykey => [field1 => value1, field2 => value2] • HGET• No nesting (unlike PHP) • HGETALL • HDEL • HVALS • HKEYS
  • 12. Hash Keys
  • 13. Set Keys http://redis.io/commands#set• key + unordered list of strings• myset => [item2, item5, item1,]• Common Commands • SADD • SMEMBERS • SISMEMBER • SREM
  • 14. Set Keys
  • 15. List Keys http://redis.io/commands#list• Like sets, except insertion order matters• Build queues or stacks• Optional blocking• Common commands • [B]LPUSH • [B]LPOP • LLEN
  • 16. List Keys
  • 17. Sorted Set Keys• Like sets, but sorted by a user-provided score value• Extremely fast access by score or range of scores, because it’s sorted in storage• Common commands • ZADD • ZRANGE • ZREVRANGE
  • 18. Sorted Set Keys
  • 19. Other commands that work on all keys• DEL - delete a key, regardless of type• KEYS - search for keys (usually with a wildcard)• EXPIRE / PERSIST - change expiration values for keys
  • 20. Connecting from PHP
  • 21. Connecting from PHP• Several libraries out there • Rediska (http://rediska.geometria-lab.net/) • PHP 5.2 / ZF 1.x friendly • Predis (https://github.com/nrk/predis) • The best in my experience • Requires PHP 5.3
  • 22. Predis• All the examples here assume you’re using Predis• Connect to a localhost redis: $p = new PredisClient();• Redis commands implemented as magic methods on Client();• $p->set($key, $val);• $val = $p->get($key);
  • 23. Attribute Display Logic items_attributes items id INT(11) id INT(11) item_id INT(11) attr_name VARCHAR(32) name VARCHAR(32) attr_value VARCHAR(32) 10k rows 100k rows• An item, a Dell Latitude Laptop, has: • Free Shipping, Financing Available, Expires Soon, etc
  • 24. Attribute Display Logic • Display “Free Shipping” graphic on the item if it has a free shipping attribute row 50 items per page
  • 25. Attribute Display - Traditional class Item { public function hasAttribute($name) { $sql = "SELECT 1 FROM items_attributes WHERE item_id = $this->id AND attr_name=$name LIMIT 1"; $result = $this->pdo->execute($sql); return $result != false; } }
  • 26. Denormalize data from MySQL to Redis• Smart caching• Define a consistent way to represent a relational object in Redis • I prefer [object class]:[object id]:[attribute] • ex: product:13445:num_comments • This prevents data collisions, and makes it easy to work with data on the command line
  • 27. Attribute Display - Redisclass Item{ public function hasAttribute($name) { return $this->redis->sismember(“item:$this->id:attributes”, $name); } public function addAttribute($name, $value) { //traditional mysql stuff here still $this->redis->sadd(item:45:attributes, $name); }}
  • 28. Advantages• The more items you have, the less MySQL will scale this solution for you on it’s own• Frequently updating your items kills the MySQL query cache• Checking existence of a set member is O(1) time• On a laptop, I can check roughly 10,000 attributes per second
  • 29. Session Clustering WEB - 1 WEB - 2 WEB - 3 PHP Sessions PHP Sessions PHP Sessions Inconsistent state
  • 30. Session Clustering WEB - 1 WEB - 2 WEB - 3 DB - 1 PHP SessionsSlow with many users (never cached), replication lag
  • 31. Session Clustering WEB - 1 WEB - 2 WEB - 3 REDIS - 1 DB - 1 PHP Sessions Constant time lookups, in-memory sessions
  • 32. Job Queues• Redis lists make great job queues• Offload your intensive workloads to some other process• Blocking I/O allows you to easily build long-running daemons
  • 33. Job Queuesclass Queue{ protected $name; protected $predis; public function push(Array $job) { $this->predis->lpush($this->name, json_encode($job)); } public function pop($block = false) { $job = null; if ($block) { $data = $this->predis->brpop($this->name, 0); $job = $data[1]; } else { $job = $this->predis->rpop($this->name); } if ($job) { return json_decode($job); } }}
  • 34. Queuing Jobs$q = new Queue(test_queue);$form = new My_Zend_Form();if ($form->isValid($_POST)) { $q->push($form->getValues()); $message = “Thanks for your submission”;} else { $message = “Error - something went wrong”;}echo “<h1>$message</h1>”;
  • 35. Processing Jobs - Crontab style function processJob(Array $job) { //...something really cool here // throw an exception on error } // process all pending jobs $q = new Queue(‘test_queue’); while ($job = $q->pop()) { try { processJob($job); } catch (Exception $e) { Echo “error processing job”; $q = new Queue(‘errors’); $q->push($job); } }
  • 36. Processing Jobs - Worker stylefunction processJob(Array $job){ //...something really cool here // throw an exception on error}// keep processing jobs as they become available$q = new Queue(‘test_queue’);while ($job = $q->pop(true)) { try { processJob($job); } catch (Exception $e) { Echo “error processing job”; $q = new Queue(‘errors’); $q->push($job); }}
  • 37. MVC Routing• Example: • Offers.com has about 3900 stores • Store pages live at /[store]/
  • 38. Old Store Routing class Store_Route { public function route($url) { $sql = “SELECT id, type FROM stores WHERE url=’$url’”; $store = $this->pdo->execute($sql); //... do logic to determine what action to use based on type ...// return array(“module” => “core”, “controller” => “store”, “action” => $action); } }• And then there’s offers.com/[holiday]/....
  • 39. New Routingclass Redis_Route{ public function route($url) { $p = $this->predis; if ($p->exists($url)) { list($module, $controller, $action) = $this->redis->hVals($url); return array(“module” => $module, “controller” => $controller, “action” => $action); } return false; }}
  • 40. Filling in the Redis keys class Store { public function create(Array $data) { // ... traditional SQL stuff to put store in the database ... // $route = array(“module” => “core”, “controller” => “store”, “action” => $data[“type”]); $this->predis->hmset($data[“url”], $route); } }
  • 41. Advantages• I can now create offers.com/[anything]/ and route it to the right place, in O(1) time• I’m only adding a few lines of code to my existing models
  • 42. Questions?
  • 43. http://joind.in/3783
  • 44. Vertive is Hiring• We help people save money• We’re looking for engineers in Austin, TX to work on: