Lessons Learned at Wakoopa
Upcoming SlideShare
Loading in...5
×
 

Lessons Learned at Wakoopa

on

  • 2,185 views

The talk I gave at RubyEnRails 2008

The talk I gave at RubyEnRails 2008

Statistics

Views

Total Views
2,185
Views on SlideShare
2,111
Embed Views
74

Actions

Likes
0
Downloads
6
Comments
0

2 Embeds 74

http://inside.wakoopa.com 65
http://wakoopa.wordpress.com 9

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Lessons Learned at Wakoopa Lessons Learned at Wakoopa Presentation Transcript

  • Lessons Learned while building Wakoopa
  • Menno van der Sman Developer at Wakoopa since May 2007
  • Wakoopa is about software View slide
  • PC Mac View slide
  • 200 million
  • Does it scale?
  • Scaling the front is easy
  • Scaling the back is harder
  • Limits spawn creativity
  • One server at Railsmachine sliced into staging, production and db MySQL 4.1 Apache + Mongrel
  • Background queue
  • Lots of tools already BackgroundRB, DelayedJob, Daemons etc
  • Rolling our own database based using script/runner processing multiple items in one loop one cache update per loop
  • Smart queries UPDATE developers SET active_seconds = active_seconds + ... INSERT INTO hourly_usage (...) VALUES (...) ON DUPLICATE KEY UPDATE active_seconds = active_seconds + ..., idle_seconds = idle_seconds + ... Allows for concurrency
  • Size = rowsize x rows Make rowsize smaller
  • sometimes `id` shouldn’t be the Primary Key
  • Enter composite_primary_keys
  • Overrides default PK class HourlyUsage < ActiveRecord::Base set_primary_keys :user_id, :software_id, :used_at ... end Original idea by Dr Nic Enhanced for Rail 2.1 by Darrin Holst http://compositekeys.rubyforge.org/
  • Archive old data that’s not frequently used
  • HourlyUsage to HourlyUsageStorage SELECT ... FROM hourly_usage WHERE used_at < '...' INTO OUTFILE '/path/to/file' LOAD DATA INFILE '/path/to/file' REPLACE INTO TABLE hourly_usage_storage avoids long locks
  • Upcoming improvements
  • MySQL 5.1 Partitioning CREATE TABLE hourly_usage_storage ( ... ) PARTITION BY RANGE (software_id) ( PARTITION p0 VALUES LESS THAN (...), PARTITION p1 VALUES LESS THAN (...), PARTITION p2 VALUES LESS THAN (...), PARTITION p3 VALUES LESS THAN (...), .... PARTITION pn VALUES LESS THAN MAXVALUE ); Makes ‘WHERE software_id = ...’ very fast Available in MySQL 5.1
  • And many more plans Designing a new infrastructure MySQL 5.1 Memcached Sphinx Rails 2.1 Amazon EC2
  • Announcing a new blog Inside Wakoopa http://inside.wakoopa.com
  • Questions? or suggestions? Looking for C++ and frontend awesomeness http://wakoopa.com/jobs