Your SlideShare is downloading. ×
0
Riak at Posterous        Julio Capote  San Francisco Riak Meetup          1/18/2012
A/S/L?• Julio Capote• Backend Developer at Posterous• @capotej
• Allows anyone to create multiple private or               public spaces (blogs)           • Around since 2008           ...
How we store posts• Original post body goes into MySQL• Multiple variants are generated (nojs,  mobile, etc)• Expensive to...
Enter Variant Cache          • A generic read/write-through cache library          • Started with Memcache          • Move...
Redis is awesome, but           • Requires both the key and value go into               memory           • Terrible disk s...
The Dream
What we wanted• Key/Value store• Disk backed• Built in distribution• Use less boxes to serve more users• Consistent perfor...
Percona MySQL / HandlerSocket
MySQL /       HandlerSocket              The Good• Great performance• Can handle a huge number of rows• Mature / Safe (at ...
MySQL /                     HandlerSocket                           The Bad          • Sharding definitely not built in    ...
MongoDB               The Good• Crazy fast• Built in sharding support• ...did I mention it was fast?
MongoDB                                      The Bad           • 30% standard deviation on fetch times (!)           • Wou...
Riak + Bitcask             The Good• Distributed by default• Consistent and predictable performance• Highly concurrent, no...
Riak + Bitcask                                  The Bad          • Not crazy fast           • Stuck it behind memcache    ...
Riak in production• Started using our 3 node cluster for the  global production cache• Accidentally turned off a node• Key...
Stats• 3 nodes• 2600+ requests/second• 300+ GB• ~200 million keys• 10 GB memcache/host
#Protips• All nodes can serve all requests, so... • Use a vip, or... • Pass all cluster nodes to client driver    (thanks ...
Any Questions?
Thanks for listening!              Special thanks to              @twoism              @vincentchu              @kangchen ...
Riak at Posterous
Riak at Posterous
Riak at Posterous
Riak at Posterous
Riak at Posterous
Riak at Posterous
Upcoming SlideShare
Loading in...5
×

Riak at Posterous

9,355

Published on

Posterous recently deployed Riak to serve as their content cache. In this talk, Julio Capote will cover why the engineering team chose Riak for the use case. He'll also share some details on the old post cache and its problems, what solutions they evaluated, and how they settled on Riak.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
9,355
On Slideshare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
13
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Riak at Posterous"

  1. 1. Riak at Posterous Julio Capote San Francisco Riak Meetup 1/18/2012
  2. 2. A/S/L?• Julio Capote• Backend Developer at Posterous• @capotej
  3. 3. • Allows anyone to create multiple private or public spaces (blogs) • Around since 2008 • Millions of posts and users • Tons of long tail trafficSome of the first posts are still being accessed today due to search engines
  4. 4. How we store posts• Original post body goes into MySQL• Multiple variants are generated (nojs, mobile, etc)• Expensive to generate (sanitizers, expanders)
  5. 5. Enter Variant Cache • A generic read/write-through cache library • Started with Memcache • Moved to RedisAt the time disk store looked promising, so we moved from memcache to redis
  6. 6. Redis is awesome, but • Requires both the key and value go into memory • Terrible disk store performance • Even with 3 machines with 64gb ram, couldn’t fit entire working set • Forced to set a TTLredis wasn’t really designed to ever hit the disk
  7. 7. The Dream
  8. 8. What we wanted• Key/Value store• Disk backed• Built in distribution• Use less boxes to serve more users• Consistent performance over raw performance
  9. 9. Percona MySQL / HandlerSocket
  10. 10. MySQL / HandlerSocket The Good• Great performance• Can handle a huge number of rows• Mature / Safe (at least the mysql part)
  11. 11. MySQL / HandlerSocket The Bad • Sharding definitely not built in • HandlerSocket is pretty much abandonedNo support going forward
  12. 12. MongoDB The Good• Crazy fast• Built in sharding support• ...did I mention it was fast?
  13. 13. MongoDB The Bad • 30% standard deviation on fetch times (!) • Would falsely acknowledge a writeThis is probably tunable, but still
  14. 14. Riak + Bitcask The Good• Distributed by default• Consistent and predictable performance• Highly concurrent, no perf degradation• Ops guy loves it!
  15. 15. Riak + Bitcask The Bad • Not crazy fast • Stuck it behind memcache • Still way faster than generating • No multi get supportwrite and read through memcache
  16. 16. Riak in production• Started using our 3 node cluster for the global production cache• Accidentally turned off a node• Keys rebalanced, site didn’t skip a beat• No one even noticed till hours later
  17. 17. Stats• 3 nodes• 2600+ requests/second• 300+ GB• ~200 million keys• 10 GB memcache/host
  18. 18. #Protips• All nodes can serve all requests, so... • Use a vip, or... • Pass all cluster nodes to client driver (thanks @aphyr!)• Use curb instead of net/http• Use Keep Alive
  19. 19. Any Questions?
  20. 20. Thanks for listening! Special thanks to @twoism @vincentchu @kangchen @argv0 @pharkmillups @seancribbs @aphyr @jrecursive
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×