A Small Talk on Getting Big.
Scaling a Rails App & all that Jazz.
Good Dog Picture.
Medium Dog.
Bad Dog Picture.
How did this Happen?
Mongrel Picture.
Too Much Time in the Application.
 Too Much Time in the Database.
A Really Cool Trick
class ApplicationController < ActionController:Base
    include ResponseManagementAdditions
    after_...
A Really Cool Trick
module ResponseManagementAdditions
   def show_runtimes
    return if response.headers['Status'] == '3...
Two Ways People Design Sites.

       Over-architect.
       Under-architect.
Move Fast. Scale Quickly.
Too Much Time in the Application.
Abstract Long Running Processes
          to Daemons.
An Ugly but Amazingly Simple Queuing
              System.
class FooDaemon < TwitterDaemon::Base
 before_startup :set_fugly_dist_idx

 def process
  unprocessed_content do |c|
   in...
...

def unprocessed_content(&block)
   loop do
     content = ContentModel.pending_content(quot;substring(truncate(id,
0)...
A Better Queuing System.
Starling.
Distributed Queuing.
   Transactional Playback.
            Fast.
           Simple.
Speaks Memcache's Language.
      100...
Not Open Source.
(yet ...)
Too Much Time in the Database.
The Basics. Database 101.
(I shouldn't need this slide in here.)
Index everything you will query on.
            Avoid complex joins.
Use joint indices when you must join tables.
     Avo...
Cache.
Cache.
Cache.
CACHE!
But How?
That Sounds Hard.
Turns Out it Isn't.
Serialize, Denormalize.
class User < ActiveRecord::Base
 serialize :following_ids

 def following_ids
  # this accessor is overwritten because we ...
def following_ids_add(the_id)
  ids = self.following_ids.dup
  ids << the_id
  write_attribute(:following_ids, ids)
 end

...
Oh yeah, and Cheat.
     (It's ok!)
Thing about your application.
How can you cheat and get away
           with it?
Is your data delivered in real time?
    Is your data static content?
      How do users interact?
Interestingness.
(Little things that don't deserve other space.)
It's OK to use Monit to kill processes
          if they get too big.
Ensure you can deploy frequently.
Ensure you can roll back easily.
Scale where it matters.
Some code is ugly. It's OK.
  (who needs a hug?)
Ensure your users can give feedback
              easily.
Use the Community.
Make an API.
(Scale your Developer-base.)
We run on Edge (but with Piston).
A Cool Trick. Gems in Vendor.
Rails::Initializer.run do |config|

 # Load Gems from the /vendor/gems folder first, if they...
Personal Pet Peeve.

It's 2007. Every spammer has your email address.
  Put it on your goddamn webpage so people can
     ...
Questions?
Britt Selvitelle

      IM & Email
anotherbritt@gmail.com
Upcoming SlideShare
Loading in...5
×

A Small Talk on Getting Big

17,363

Published on

Published in: Technology
4 Comments
16 Likes
Statistics
Notes
  • Nicely done. I’m Ana Mui Stanley, working on my latest site on lyrics, www.lyrics-search.org/ . I enjoy reading the slide.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Excellent presentation, enjoyed it! Congrats for the presentation!

    http://www.riding-mower.org/

    http://www.riding-mower.org/la175-john-deere-lawn-tractor/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Fantastic, with scary photos. :)

    John.
    www.freeringtones.ws/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hello.. sound good.. great sharing

    Regards
    Anisa
    http://phonehut.info
    http://www.jpolls.net
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
17,363
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
374
Comments
4
Likes
16
Embeds 0
No embeds

No notes for slide

A Small Talk on Getting Big

  1. 1. A Small Talk on Getting Big. Scaling a Rails App & all that Jazz.
  2. 2. Good Dog Picture.
  3. 3. Medium Dog.
  4. 4. Bad Dog Picture.
  5. 5. How did this Happen?
  6. 6. Mongrel Picture.
  7. 7. Too Much Time in the Application. Too Much Time in the Database.
  8. 8. A Really Cool Trick class ApplicationController < ActionController:Base include ResponseManagementAdditions after_filter :show_runtimes around_filter { |controller, action| controller.total_runtime = Benchmark.measure(&action).real } end
  9. 9. A Really Cool Trick module ResponseManagementAdditions def show_runtimes return if response.headers['Status'] == '304 Not Modified' || ! (response.body && response.body.respond_to?(:sub!)) db_runtime = ((0 + (@db_rt_before_render || 0) + (@db_rt_after_render || 0)) * 1000).truncate rendering_runtime = ((@rendering_runtime || 0) * 1000).truncate total_runtime = ((@total_runtime || 0) * 1000).truncate response.body.gsub!(/</body></html>$/, quot;<!-- served to you through a copper wire by #{HOSTNAME} at #{Time.now.to_s(:short)} in #{total_runtime} ms (d #{db_runtime} / r #{rendering_runtime}). thank you, come again. --> n</body></html>quot;) end end
  10. 10. Two Ways People Design Sites. Over-architect. Under-architect.
  11. 11. Move Fast. Scale Quickly.
  12. 12. Too Much Time in the Application.
  13. 13. Abstract Long Running Processes to Daemons.
  14. 14. An Ugly but Amazingly Simple Queuing System.
  15. 15. class FooDaemon < TwitterDaemon::Base before_startup :set_fugly_dist_idx def process unprocessed_content do |c| increment_counter(:total) # Do work ... break unless running? end end ...
  16. 16. ... def unprocessed_content(&block) loop do content = ContentModel.pending_content(quot;substring(truncate(id, 0),-2,1) = #{@fugly_dist_idx}quot;) messages.each { |message| yield message } sleep 1 if messages.nil? || messages.empty? end end def set_fugly_dist_idx @fugly_dist_idx = ARGV.find { |v| v.match(/[0-9]/) } raise quot;You need to specify a dist idx between 0 and 9.quot; unless @fugly_dist_idx @fugly_dist_idx = @fugly_dist_idx.to_i end end
  17. 17. A Better Queuing System.
  18. 18. Starling.
  19. 19. Distributed Queuing. Transactional Playback. Fast. Simple. Speaks Memcache's Language. 100% Pure Ruby.
  20. 20. Not Open Source.
  21. 21. (yet ...)
  22. 22. Too Much Time in the Database.
  23. 23. The Basics. Database 101. (I shouldn't need this slide in here.)
  24. 24. Index everything you will query on. Avoid complex joins. Use joint indices when you must join tables. Avoid scanning large sets of data.
  25. 25. Cache.
  26. 26. Cache.
  27. 27. Cache.
  28. 28. CACHE!
  29. 29. But How? That Sounds Hard.
  30. 30. Turns Out it Isn't.
  31. 31. Serialize, Denormalize.
  32. 32. class User < ActiveRecord::Base serialize :following_ids def following_ids # this accessor is overwritten because we want to lazily set the # friends_ids column, rather than running a gigantic slow migration. RAILS_DEFAULT_LOGGER.debug quot;loading following_idsquot; ids = read_attribute(:following_ids) if ids.nil? || !ids.kind_of?(Array) ids = connection.select_values(quot;SELECT DISTINCT followed_user_id FROM followed_users WHERE user_id = #{self.id}quot;).map(&:to_i).compact update_attribute(:following_ids, ids) end ids end ...
  33. 33. def following_ids_add(the_id) ids = self.following_ids.dup ids << the_id write_attribute(:following_ids, ids) end def following_ids_delete(the_id) ids = self.following_ids.dup ids.delete(the_id) write_attribute(:following_ids, ids) end end # End Class
  34. 34. Oh yeah, and Cheat. (It's ok!)
  35. 35. Thing about your application. How can you cheat and get away with it?
  36. 36. Is your data delivered in real time? Is your data static content? How do users interact?
  37. 37. Interestingness. (Little things that don't deserve other space.)
  38. 38. It's OK to use Monit to kill processes if they get too big.
  39. 39. Ensure you can deploy frequently.
  40. 40. Ensure you can roll back easily.
  41. 41. Scale where it matters.
  42. 42. Some code is ugly. It's OK. (who needs a hug?)
  43. 43. Ensure your users can give feedback easily.
  44. 44. Use the Community.
  45. 45. Make an API. (Scale your Developer-base.)
  46. 46. We run on Edge (but with Piston).
  47. 47. A Cool Trick. Gems in Vendor. Rails::Initializer.run do |config| # Load Gems from the /vendor/gems folder first, if they exist. config.load_paths += Dir[quot;#{RAILS_ROOT}/vendor/gems/**quot;].map do |dir| File.directory?(lib = quot;#{dir}/libquot;) ? lib : dir end ...
  48. 48. Personal Pet Peeve. It's 2007. Every spammer has your email address. Put it on your goddamn webpage so people can get ahold of you about interesting things.
  49. 49. Questions?
  50. 50. Britt Selvitelle IM & Email anotherbritt@gmail.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×