Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Blazing Data With Redis (and LEGOS!)

7,704 views

Published on

There are many fast data stores, and then there is Redis. Learn about this excellent NoSQL solution that is a powerful in-memory key-value store. Learn how to solve traditionally difficult problems with Redis, and how you can benefit from 100,000 reads/writes a second on commodity hardware. We’ll discuss how and when to use the different datatypes and commands to fit your needs. We’ll discuss the different PHP libraries with their pros and cons. We’ll then show some live examples on how to use it for a chatroom, and how Redis manages a billion data points for our dating matching system. Finally, we’ll discuss some of the upcoming features in the near future, such as clustering and scripting.

Published in: Technology
  • Be the first to comment

Blazing Data With Redis (and LEGOS!)

  1. 1. Blazing Data with Redis S!) By: Justin Carmony LE GO ( and
  2. 2. edis)About Me (I <3 R• Director of Development for DeseretNews.com• CIO of CEVO• I Make (and Break) Web Stuff• Focus on Scalable, Real-time Websites & APIs
  3. 3. About Presentation• We’ll ask for questions several times during presentation & the end.• I will post links, slides, resource, etc.• Goal: Educate & Inspire you on how, when, and why to use Redis!
  4. 4. by rt e S ta th ts ingLe ur eas enceM udi A
  5. 5. u ta abo em lk obl Ta Pr t’sLe monC o m
  6. 6. in ja N a e re m u’ soYo e A w oper evel D
  7. 7. to nt e a n W nli e W kO ac ers! Tr S U BosRea s: “W l-Ti e W me ant Da ta!”
  8. 8. in ja N a e re m u’ so Yo e A w oper evel DWe Want To Know Who Is Viewing Our Site!
  9. 9. “I’ll Just Add a Table to MySQl....”
  10. 10. Real-Time Data
  11. 11. Real-Time Data • High Volume Writes • High Volume Reads • Low Latency / Delay • Grows Exponentially compared to Rest of your Data
  12. 12. Traditional Data Stores• Examples: Databases (i.e. MySQL, Postgre)• Writing is Resource Intensive• Stored on Disk (aka Slower)• Slower as Volume Grows ds...) or the rw• Challenge for Replication (i no
  13. 13. olu me ighV ith H ites s w Wr wn s & el tdo eadM R
  14. 14. Many NoSQL Solutions
  15. 15. Redis - Built for Speed• Key-Value Storage• In-Memory•••• nd...) seco ai ta (W
  16. 16. Isn’t This Just Memcached?
  17. 17. o! NIsn’t This Just Memcached?
  18. 18. Redis - Built for Speed• Key-Value Storage• In-Memory• Persistence• Multiple Data Types• Transactions• Pub/Sub w/ Blocking ast ...) it ’s F (and
  19. 19. Extremely Fast...
  20. 20. Ludicrous Speed Fast
  21. 21. 0+ s ,00 rite 1 00 W s & ond ad ec Re r S PeLudicrous Speed Fast
  22. 22. It’s gone Plaid!
  23. 23. To w isH o R edU se
  24. 24. Step One: Install & Run• Linux & Mac: Compile from Source • No Dependencies • make && make install• Windows • Download Binaries • Compile? Best of Luck
  25. 25. Running Redis• Run: redis-server• Can use configuration file• Run as a “service”: http://redis.io/topics/ quickstart• redis-cli to try out commands
  26. 26. Step Two:Understand Keys
  27. 27. Redis Keys• Keys MUST be UNIQUE• Keys are Binary Safe Strings• Super Long Keys (i.e. 1024 bytes) are costly to look up• Cryptic Short Keys (i.e. u:1000:pwd) have little performance improvement over a descriptive key (i.e. user:1000:password)
  28. 28. Step Three:Thinking Key-Value
  29. 29. This is not an RMDBS• It’s All About the Keys, Not the Values• There is no “Querying”• There are no “Indexes”• There are no “Schemas”
  30. 30. e nt oc um ta!D D a Yo ur
  31. 31. Step Four: Get to Know your Data Types g Your in ! ow ces Kn Pie L ike go Le
  32. 32. Redis Strings• Store Simple Strings• Binary Safe• Great for JSON• Advanced Commands
  33. 33. Data-Type “Matrix” Sorted Unsorted SortedComparable Sets SetsStand Alone Lists Hashes
  34. 34. Quick Variable Guide• Hashes - Small in Size,Very Efficient• Lists - Awesome Queues, Size Doesn’t Affect Performance• Sets - Great for Intersecting with others• Sorted Sets - Use to Keep “Indexes”, Sorts using Scores
  35. 35. Step Five:Learn the Commands
  36. 36. Learning the Commands• Generic Commands for All Keys• Commands for Each Data Type• Each Command has Big O Notation for Performance• Simple,Yet Powerful
  37. 37. ive act s ter pleIn m E xa
  38. 38. g It tin her ut etP gA ll To
  39. 39. Using Redis With PHP• Predis • PHP Library • Easy to Use • Very Fast to Update New Features• phpredis • PHP Extension ing • Faster, but requires Be Us compiling module e ’ll dis W Pre
  40. 40. Example: Simple Caching
  41. 41. Simple Cache• Data Types: • Strings• Commands: • SETEX <key> <seconds to expire> <value> • GET <key> • EXPIREAT <key> <timestamp>
  42. 42. Connecting<?php// Include the Predis Autoloaderrequire predis/lib/Predis/Autoloader.php;// Register the AutoloaderPredisAutoloader::register();// Create a Client with defaults$redis = new PredisClient();// Create with Connection String$redis = new PredisClient(tcp://10.0.0.1:6379);/** Our Examples Will Assume This Is Already Done **/
  43. 43. Simple Cache$key = cache.user:justin;$data_str = $redis->get($key);if($data_str){! $data = unserialize($data_str);}else{! // Really Expensive Method of Getting This Data! $data = MyDatabase::GetExpensiveData();! $redis->setex($key, 60, serialize($data));}/* Do something with the $data */
  44. 44. Example: Online Users• Data Types: • Sets• Commands: • SADD <key> <value> • SUNION <key1> <key2> <key3> <key....> • EXPIRE <key> <timestamp>
  45. 45. Example: Users Online
  46. 46. Marking Users Online /* Store Current User */ // Current User ID $user_id = 1234;// Get Current Time$now = time();$min = date("i",$now);// Generate the Key$key = "online:".$min;// Adding user to online users$redis->sadd($key, $user_id);$redis->expire($key, 60 * 10); // Expire in 10 minutes
  47. 47. Getting Online Users/* Getting Onling Users */$keys = array();// Get Current Time$now = time();$min = date("i",$now);$count = 0;$minutes_ago = 5;while($count < $minutes_ago){! $keys[] = "online:".$min;!! $count++;! $min--;! if($min < 0)! {! ! $min = 59;! }}$scmd = $redis->createCommand("sunion",$keys);$online_ids = $redis->executeCommand($scmd);
  48. 48. Example: Friends Online• Data Types: • Sets• Additional Commands: • SUNIONSTORE <dest> <key1> <key2> <key....> • SINTER <key1> <key2> <key...>
  49. 49. My Friends Online/* My Friends Online */$keys = array(online_users);$user_id = 1234;// Get Current Time$min = date("i",time());$count = 0;$minutes_ago = 5;while($count < $minutes_ago){! $keys[] = "online:".$min;! $count++;! $min--;! if($min < 0) { $min = 59; }}// SUNIONSTORE online_users online:10 online:9 online:8 online:7 online:6$scmd = $redis->createCommand("sunionstore",$keys);$redis->executeCommand($scmd);$online_friend_ids = $redis->sinter(online_users! , user:.$user_id..friend_ids);
  50. 50. Under The Hood
  51. 51. Few Things About Redis• Single Threaded• Can “Shard” For More Capacity / Performance• All Commands are Atomic• Transactions for Multiple Atomic Commands• Pipelining for High Performance
  52. 52. Persistence• Snapshots on a Configurable Schedule • Will “fork” the process and write the DataSet to Disk• Append-Only File• Will Let You Survive a “Reboot”
  53. 53. New Stuff / On The Horizon• Lua Scripting (Kinda Querying)• Redis Clustering
  54. 54. aL ive ut bo ? AH ow emo D
  55. 55. One Redis Server
  56. 56. 50 Servers * 8 Clients = 400 Workers
  57. 57. Rackspace Cloud +Salt (saltstack.org) + Redis & PHP
  58. 58. Demo Time!
  59. 59. Questions?
  60. 60. Your HomeworkDownload & Play Around with Redis! It’s Awesome!
  61. 61. More Info?http://www.justincarmony.com/redis
  62. 62. Flickr Credits & Images: http://gim.ie/jsLJ
  63. 63. Thanks!Rate My Talk (PLEASE): https://joind.in/6486 Twitter: @JustinCarmony IRC: carmony #uphpu #phpc #salt Website: http://www.justincarmony.com/blog Email: justin@justincarmony.com

×