Handling Redis failover with ZooKeeper

Handling Redis failover with ZooKeeper Ryan LeCompte BaRuCo Lightning Talk September 2012

Me• Rubyist at Spreecast in San Francisco• redis_failover, method_locator, synchronizable• http://github.com/ryanlecompte• lecompte@gmail.com• @ryanlecompte

Typical Redis Setup

Master Dies :-(

What happens to your data andconnected clients?• Resque Jobs?• Sidekiq Jobs?• Cached pages?• Cached stats?• Rubygems?

Scramble to manuallyreconﬁgure slave as master

Update all Redis clients totalk to the new master

Restart production servicesso they can see the new conﬁg

What do we really want to do? NOTHING

Failover should be safe and automatic

Let’s talk aboutZooKeeper

Open source ApacheProject (originally from Yahoo)

Provides a set of simpleprimitives to help you build powerfuldistributed applications

Automatically handlesnetwork partitions &quorum management

Replicated, HighlyAvailable, Ordered,Battle-proven

Used by Yahoo, Netﬂix, Rackspace, Hadoop, Neo4j, etc

How can I useZooKeeper in my architecture?

Distributed Conﬁguration(update thread pool sizes & other settings without requiring a server restart)

Distributed Locking (only one process should beable to do something at a time)

Priority Queues

Leader Election, Group Membership & more

Practical application:Using ZooKeeper to build automatic failover for Redis

redis_failoverhttp://github.com/ryanlecompte/redis_failover

Provides a smart “failover-aware” client that wrapsexisting Ruby client forRedis

Clients automaticallyconnect to ZooKeeperand register for watchnotiﬁcations

Node Managers monitorRedis servers for availabilityand automatically informclients of failover viaZooKeeper

Architecture

RedisFailover::Client Example# create new client with list of ZooKeeper serversclient = RedisFailover::Client.new(:zkservers => localhost:2181,localhost:2182,localhost:2183)# write operations automatically routed to masterclient.set(‘foo’, ‘bar’)# read operations automatically routed to a slaveclient.smembers(‘data’)

Companies using redis_failover• Papertrail• Backupify• MediaPiston• IDT Corporation• others

Consider incorporatingZooKeeper into your overallarchitecture.Utilize the primitives it gives you forsolving your next distributedprogramming challenge.

Don’t let your Redis serversgo unmanaged in production.Consider usingredis_failover to sleepbetter at night.

Thanks for listening! http://github.com/ryanlecompte/redis_failover Questions? Ask me at the beach!

Handling Redis failover with ZooKeeper

by ryanlecompte on Sep 08, 2012

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 13

Statistics

Handling Redis failover with ZooKeeper Presentation Transcript

https://twitter.com	12
http://www.hanrss.com	1