Scaling php applications with redis

Scaling PHP Applications with RedisJosh ButtsZendcon 2011http://joind.in/3783

About Me• Director of Development at Vertive, LLC• Zend Certified in PHP 5 and ZF• Organizer of AustinPHP• Find me online: • Twitter: @jimbojsb • Github: jimbojsb

Agenda• Redis Overview• Types of keys• PHP Strategies

What is Redis?• Redis is an “Advanced” key-value store database• Backed by VMWare• Redis works like memcached, but better: • Atomic commands & transactions • Server-side data structures • In-memory + disk persistence

Redis in an in-memory database• You need as much RAM as you have data• There are no plans to improve support for “virtual” memory• Disk persistence is exactly that• Customize disk writes interval to suit your pain threshold• Something interesting happens when you run out of memory

How to get Redis• http://redis.io• Doesn’t run (well) on Windows • There are a few ports out there• Has virtually no compile dependencies• Most distros have a package• Make sure you’re running at least 2.2 • 2.4.1 became stable 10/17/11

How to explore• There aren’t any good GUI tools out there• redis-cli is your friend

A bit about organization• Redis can have multiple databases • Analogous to a MySQL database within a server instance • Theoretically, over 1000 databases per server • One data file on disk per instance• Within a database, namespace your keys • Ex: myapp:mykey • Keep them concise but useful. Keys take memory too!

Types of Keys

String Keys http://redis.io/commands#string• Simple key-value• Memcache equivalent• Common Commands • SET • GET • INCR • STRLEN

Hash Keys http://redis.io/commands#hash• Key + multiple fields / values • Common commands• Like an associative array in PHP • HSET• mykey => [field1 => value1, field2 => value2] • HGET• No nesting (unlike PHP) • HGETALL • HDEL • HVALS • HKEYS

Hash Keys

Set Keys http://redis.io/commands#set• key + unordered list of strings• myset => [item2, item5, item1,]• Common Commands • SADD • SMEMBERS • SISMEMBER • SREM

Set Keys

List Keys http://redis.io/commands#list• Like sets, except insertion order matters• Build queues or stacks• Optional blocking• Common commands • [B]LPUSH • [B]LPOP • LLEN

List Keys

Sorted Set Keys• Like sets, but sorted by a user-provided score value• Extremely fast access by score or range of scores, because it’s sorted in storage• Common commands • ZADD • ZRANGE • ZREVRANGE

Sorted Set Keys

Other commands that work on all keys• DEL - delete a key, regardless of type• KEYS - search for keys (usually with a wildcard)• EXPIRE / PERSIST - change expiration values for keys

Connecting from PHP

Connecting from PHP• Several libraries out there • Rediska (http://rediska.geometria-lab.net/) • PHP 5.2 / ZF 1.x friendly • Predis (https://github.com/nrk/predis) • The best in my experience • Requires PHP 5.3

Predis• All the examples here assume you’re using Predis• Connect to a localhost redis: $p = new PredisClient();• Redis commands implemented as magic methods on Client();• $p->set($key, $val);• $val = $p->get($key);

Attribute Display Logic items_attributes items id INT(11) id INT(11) item_id INT(11) attr_name VARCHAR(32) name VARCHAR(32) attr_value VARCHAR(32) 10k rows 100k rows• An item, a Dell Latitude Laptop, has: • Free Shipping, Financing Available, Expires Soon, etc

Attribute Display Logic • Display “Free Shipping” graphic on the item if it has a free shipping attribute row 50 items per page

Attribute Display - Traditional class Item { public function hasAttribute($name) { $sql = "SELECT 1 FROM items_attributes WHERE item_id = $this->id AND attr_name=$name LIMIT 1"; $result = $this->pdo->execute($sql); return $result != false; } }

Denormalize data from MySQL to Redis• Smart caching• Define a consistent way to represent a relational object in Redis • I prefer [object class]:[object id]:[attribute] • ex: product:13445:num_comments • This prevents data collisions, and makes it easy to work with data on the command line

Attribute Display - Redisclass Item{ public function hasAttribute($name) { return $this->redis->sismember(“item:$this->id:attributes”, $name); } public function addAttribute($name, $value) { //traditional mysql stuff here still $this->redis->sadd(item:45:attributes, $name); }}

Advantages• The more items you have, the less MySQL will scale this solution for you on it’s own• Frequently updating your items kills the MySQL query cache• Checking existence of a set member is O(1) time• On a laptop, I can check roughly 10,000 attributes per second

Session Clustering WEB - 1 WEB - 2 WEB - 3 PHP Sessions PHP Sessions PHP Sessions Inconsistent state

Session Clustering WEB - 1 WEB - 2 WEB - 3 DB - 1 PHP SessionsSlow with many users (never cached), replication lag

Session Clustering WEB - 1 WEB - 2 WEB - 3 REDIS - 1 DB - 1 PHP Sessions Constant time lookups, in-memory sessions

Job Queues• Redis lists make great job queues• Offload your intensive workloads to some other process• Blocking I/O allows you to easily build long-running daemons

Job Queuesclass Queue{ protected $name; protected $predis; public function push(Array $job) { $this->predis->lpush($this->name, json_encode($job)); } public function pop($block = false) { $job = null; if ($block) { $data = $this->predis->brpop($this->name, 0); $job = $data[1]; } else { $job = $this->predis->rpop($this->name); } if ($job) { return json_decode($job); } }}

Queuing Jobs$q = new Queue(test_queue);$form = new My_Zend_Form();if ($form->isValid($_POST)) { $q->push($form->getValues()); $message = “Thanks for your submission”;} else { $message = “Error - something went wrong”;}echo “<h1>$message</h1>”;

Processing Jobs - Crontab style function processJob(Array $job) { //...something really cool here // throw an exception on error } // process all pending jobs $q = new Queue(‘test_queue’); while ($job = $q->pop()) { try { processJob($job); } catch (Exception $e) { Echo “error processing job”; $q = new Queue(‘errors’); $q->push($job); } }

Processing Jobs - Worker stylefunction processJob(Array $job){ //...something really cool here // throw an exception on error}// keep processing jobs as they become available$q = new Queue(‘test_queue’);while ($job = $q->pop(true)) { try { processJob($job); } catch (Exception $e) { Echo “error processing job”; $q = new Queue(‘errors’); $q->push($job); }}

MVC Routing• Example: • Offers.com has about 3900 stores • Store pages live at /[store]/

Old Store Routing class Store_Route { public function route($url) { $sql = “SELECT id, type FROM stores WHERE url=’$url’”; $store = $this->pdo->execute($sql); //... do logic to determine what action to use based on type ...// return array(“module” => “core”, “controller” => “store”, “action” => $action); } }• And then there’s offers.com/[holiday]/....

New Routingclass Redis_Route{ public function route($url) { $p = $this->predis; if ($p->exists($url)) { list($module, $controller, $action) = $this->redis->hVals($url); return array(“module” => $module, “controller” => $controller, “action” => $action); } return false; }}

Filling in the Redis keys class Store { public function create(Array $data) { // ... traditional SQL stuff to put store in the database ... // $route = array(“module” => “core”, “controller” => “store”, “action” => $data[“type”]); $this->predis->hmset($data[“url”], $route); } }

Advantages• I can now create offers.com/[anything]/ and route it to the right place, in O(1) time• I’m only adding a few lines of code to my existing models

Questions?

http://joind.in/3783

Vertive is Hiring• We help people save money• We’re looking for engineers in Austin, TX to work on:

Scaling php applications with redis

by jimbojsb on Oct 18, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

Statistics

Scaling php applications with redis — Presentation Transcript