Flickr uses Redis extensively across many services for caching, queueing, and more. Some key uses of Redis include caching for the camera roll and activity feed, which each have multiple masters and slaves distributed across data centers. Notifications are also queued in Redis. In total, Flickr operates over 100 physical hosts running 500+ Redis instances to support over 600,000 operations per second across services like the camera roll, activity feed, notifications, and more.

1. @ flickr & tripod
Presented by Sean Perkins
Sr. Production Engineer

2. flickr & tripod
100+ physical hosts
500+ instances
600k Ops/s

3. Use Cases for Redis

4. Cameraroll
24 hosts (12 on each coast)
4 masters per host
cache
twemproxy client
ketama hash ring

5. Activityfeed
8 hosts
8 masters per host
cache
twemproxy client
ketama hash ring

6. Notifications
3 Hosts
1 Master
2 Slaves
Clients on both coasts
Redis only in one

7. And many many more…
45 redis-general hosts w/ ~200 instances
about 50% have HA

8. Why Redis

9. Redis Cluster
Manual builds at first
created package to automate setup
tested failed nodes
benchmarking

10. redis-cluster101
redis-cluster102
redis-cluster103
196GB RAM each
8 master instances x 3 hosts = 24
16 slave instances x 3 hosts = 48
tripod

11. Console (tripod on boarding site)
Java Spring Client
Cache
Smart Client

12. redis-cluster101
redis-cluster102
redis-cluster103
64GB RAM each
4 master instances x 3 hosts = 12
8 slave instances x 3 hosts = 24
flickr

13. Activitybot
Kue Library and Nodejs
Queue
Smart Client

14. Questions?