Your SlideShare is downloading. ×
Redis everywhere - PHP London
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Redis everywhere - PHP London


Published on

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. REDIS EVERYWHERE Ricard Clau (@ricardclau) PHP London - 1st August 2013
  • 2. HELLO WORLD • Ricard Clau, born and grown up in Barcelona • Software engineer at Hailo • Symfony2 lover and PHP believer • Open-source contributor, sometimes I give talks • Twitter @ricardclau / Gmail
  • 3. WHAT IS REDIS? • REmote DIctionary Server • Created in 2009 by Salvatore Sanfilipo (@antirez) • Open source • Advanced in-memory key-value data-structure server • Stop thinking SQL, think back to structures!
  • 5. AGENDA • REDIS introduction • Getting started with PHP / Symfony2: sessions, caching • Cookbooks: queues, statistics, leaderboards, users online, friends online, locking • War Stories from a 7.5M DAU project • Final thoughts
  • 6. INTRODUCTIONTO REDIS And some demystifications
  • 7. DATATYPES • Strings up to 512 Mb • Lists of elements sorted by insertion order (max 2^32 - 1) • Sets unordered collection of elements supporting unions, intersections and differences (max 2^32 - 1) • Hashes key-value pairs (max 2^32 - 1) • Sorted sets automatically ordered by score (max 2^32 - 1)
  • 8. ONLY IN-MEMORY? • Your data needs to fit in your memory • It has configurable persistance: RDB snapshots,AOF persistence logs, combine both or no persistance at all • Think like memcached on steroids with persistance • • “Memory is the new disk, disk is the new tape”
  • 9. INTERESTING FEATURES • Master-slave replication • Pipelining to improve performance • Transactions (kind of with MULTI ... EXEC) • Publisher / Subscriber • Scripting with LUA (~stored procedures) • Predictable performance (check commands complexity)
  • 10. SHARDING DATA • Redis cluster will be available in 3.0 (delayed every year...) • Project Twemproxy can help sharding memcached / redis • Client-side partitioning is enough for most cases • Range partitioningVS hash partitioning • Presharding can help scaling horizontally
  • 11. HARDWARETIPS • Redis is single-threaded, so a 2 core machine is enough • If using EC2, maybe m1.large (7.5Gb) is the most suitable, but if it is not enough you might consider any m2 memory optimized type • CPU is rarely the bottleneck with Redis • Read carefully the doc to maximize performance!
  • 12. GETTING STARTED WITH PHP It is easier than you may think!
  • 13. PHP CLIENTS • • Predis (PHP) vs phpredis (PHP extension) • Predis is very mature, actively maintained, feature complete, extendable, composer friendly and supports all Redis versions • Phpredis is faster, but not backwards compatible • Network latency is the biggest performance killer
  • 14. REDIS-CLI AND MONITOR • Redis-cli to connect to a Redis instance • With Monitor we can watch live activity!
  • 15. SESSIONS, CACHING • Super-easy to implement with commercial frameworks • Commands used: GET <session-id>, SETEX <session-id> <expires> <serialized-data> • Much better performance than PDO and also persistent! • Caching and temp-data storage work exactly equal • Be careful with temp-data storage and memory usage!
  • 16. SYMFONY2 INTEGRATION • There must be a bundle for that... • • Out of the box: Session management, Monolog handler, Swiftmailer spooling, Doctrine caching • Full integration with Symfony2 profiler
  • 18. SOME COOKBOOKS And comparisons with other technologies
  • 19. QUEUE PROCESSING • Using Lists and LPUSH / RPOP instructions • We can have different languages accessing data • We used that to send OpenGraph requests to Facebook (REALLY slow API) with Python daemons • ~80k messages/minute processed with 1 m1.large EC2 • If intensive, consider using RabbitMQ,ActiveMQ, ZeroMQ...
  • 20. REAL-TIME ANALYTICS • We can use hashes to group many stats under one key • Most of the times we have counters: HINCRBY “stats" <key> <increment> • HMGET “stats” <key1> <key2> ... to retrieve values and HMSET “stats”“stat1” <value1> “stat2” <value2> to set them • HGETALL to get the full hash
  • 21. LEADERBOARDS • Super-easy with Redis, heavy consuming with other systems • Each board with a single key, using sorted sets • ZINCRBY “rankXXX” <increment> <userId> • Top N ZREVRANGE “rankXXX” 0 N [WITHSCORES] • We stored several milions of members under 1 key
  • 22. WHO IS ONLINE? (I) • We can use SETS to achieve it! • One set for every minute, SADD <minute> <userId> Time +1m +2m +3m 2 2 7 2 3 1 4 1 1 1 3 5 SUNION 2 1 3 5 7 4
  • 23. WHO IS ONLINE? (II) • Online users == everyone active in the last N minutes • SUNION <now> <now-1> <now-2> ... • SUNIONSTORE “online” <now> <now-1> <now-2> ... • Number of users: SCARD “online” • Obtain online users with SMEMBERS “online”
  • 24. FRIENDS ONLINE? • If we store user´s friends in a SET with key friends-<user-id> • Friends online: SINTERSTORE “friends-<userid>-online” “online”“friends-<userid>” • Imagine how you would do it without Redis! 7 1 3 5 2 4 15 9 3 10 7 1 ONLINE FRIENDS-12 SINTER FRIENDS-12-ONLINE
  • 25. DISTRIBUTED LOCKING (I) • Problem in heavy-write applications: 2 almost concurrent requests trying to update same records • SELECT FOR UPDATE? Easy to create deadlocks! timeline R1 R2 R1read R2save R2read R1save R1 updates are lost!!!!!
  • 26. DISTRIBUTED LOCKING (II) • Locking can be solved with Redis: SETNX returns true only if key does not exist. • PseudoCode: try SETNX <lock> <time+expires>, if false check if existing lock has expired and overwrite if so. Otherwise, sleep some milliseconds and try to acquire again. Remember to release lock at the end! • 8 m1.xlarge instances to lock 7.5M Daily Active Users • Alternatives:Apache Zookeeper, perhaps better for scaling
  • 27. WAR STORIES (~7.5M DAU Facebook Game)
  • 28. PROGRESSIVELY • DragonCity had ~2M DAU when we introduced Redis • First usages: Queues PHP-Python and Temporary storage • Introduced Locking with Redis -> End of deadlocks • Used for temporary contests involving rankings • Session tracking, cheat detection and many others • About 7.5M DAU and similar amount of servers
  • 29. REDIS IS SUPER-FAST • Your network gets killed much before Redis is down • Always increase your ulimit values in Redis servers • We had Gigabit network connectivity between machines but it was not enough to handle the data sent and retrieved • Redis was just at 20% of CPU • Do some heavy-load tests before launching!
  • 30. SURPRISES WITH MEMORY • Redis documentation:“Sets of binary-safe strings”so, we changed values in sets from being just numbers to number + :<char> to add some info to values • We expected ~20% memory increase but in was 500% • • If value can be represented as long, it is stored more efficiently • values-are-strings.html
  • 31. FINALTHOUGHTS When and when not to consider Redis
  • 32. REDIS IS PERFECT FOR... • Intensive read-write data applications • Temporary stored data • Data that fits in memory • Problems that fit Redis built-in data types • Predictability in performance needed (all commands complexity documented)
  • 33. BUT IS NOT SUITABLE WHEN.. • Big data sets, archive data • Relational data (RDBMS are absolutely fine to scale) • We don´t know how we will access data • Reporting applications (no where clauses) • ALWAYS choose the right tool for the job!
  • 34. SPECIALTHANKS • Ronny López (@ronnylt) & AlonsoVidales (@alonsovidales) • All backend and systems engineers at @socialpoint • Of course, to all of you for coming the 1st of August! • Check our open-source project Redtrine, PR welcome:
  • 35. QUESTIONS? • Twitter: @ricardclau • E-mail: • Github: • Blog about PHP and Symfony2: • Hailo is hiring! If you are interested, talk to us or apply!