SlideShare a Scribd company logo
1 of 50
Download to read offline
A Small Talk on Getting Big.
Scaling a Rails App & all that Jazz.
Good Dog Picture.
Medium Dog.
Bad Dog Picture.
How did this Happen?
Mongrel Picture.
Too Much Time in the Application.
 Too Much Time in the Database.
A Really Cool Trick
class ApplicationController < ActionController:Base
    include ResponseManagementAdditions
    after_filter :show_runtimes

   around_filter { |controller, action| controller.total_runtime =
   Benchmark.measure(&action).real }
end
A Really Cool Trick
module ResponseManagementAdditions
   def show_runtimes
    return if response.headers['Status'] == '304 Not Modified' || !
(response.body && response.body.respond_to?(:sub!))

   db_runtime = ((0 + (@db_rt_before_render || 0) +
(@db_rt_after_render || 0)) * 1000).truncate
   rendering_runtime = ((@rendering_runtime || 0) * 1000).truncate
   total_runtime = ((@total_runtime || 0) * 1000).truncate

    response.body.gsub!(/</body></html>$/, quot;<!-- served to you
through a copper wire by #{HOSTNAME} at
#{Time.now.to_s(:short)} in #{total_runtime} ms (d #{db_runtime} / r
#{rendering_runtime}). thank you, come again. -->
n</body></html>quot;)
   end
end
Two Ways People Design Sites.

       Over-architect.
       Under-architect.
Move Fast. Scale Quickly.
Too Much Time in the Application.
Abstract Long Running Processes
          to Daemons.
An Ugly but Amazingly Simple Queuing
              System.
class FooDaemon < TwitterDaemon::Base
 before_startup :set_fugly_dist_idx

 def process
  unprocessed_content do |c|
   increment_counter(:total)

       # Do work ...

   break unless running?
  end
 end

 ...
...

def unprocessed_content(&block)
   loop do
     content = ContentModel.pending_content(quot;substring(truncate(id,
0),-2,1) = #{@fugly_dist_idx}quot;)
     messages.each { |message| yield message }
     sleep 1 if messages.nil? || messages.empty?
   end
 end

 def set_fugly_dist_idx
  @fugly_dist_idx = ARGV.find { |v| v.match(/[0-9]/) }
  raise quot;You need to specify a dist idx between 0 and 9.quot; unless
@fugly_dist_idx
  @fugly_dist_idx = @fugly_dist_idx.to_i
 end
end
A Better Queuing System.
Starling.
Distributed Queuing.
   Transactional Playback.
            Fast.
           Simple.
Speaks Memcache's Language.
      100% Pure Ruby.
Not Open Source.
(yet ...)
Too Much Time in the Database.
The Basics. Database 101.
(I shouldn't need this slide in here.)
Index everything you will query on.
            Avoid complex joins.
Use joint indices when you must join tables.
     Avoid scanning large sets of data.
Cache.
Cache.
Cache.
CACHE!
But How?
That Sounds Hard.
Turns Out it Isn't.
Serialize, Denormalize.
class User < ActiveRecord::Base
 serialize :following_ids

 def following_ids
  # this accessor is overwritten because we want to lazily set the
  # friends_ids column, rather than running a gigantic slow migration.
  RAILS_DEFAULT_LOGGER.debug quot;loading following_idsquot;
  ids = read_attribute(:following_ids)

  if ids.nil? || !ids.kind_of?(Array)
    ids = connection.select_values(quot;SELECT DISTINCT followed_user_id
FROM followed_users WHERE user_id =
#{self.id}quot;).map(&:to_i).compact
    update_attribute(:following_ids, ids)
  end

  ids
 end

 ...
def following_ids_add(the_id)
  ids = self.following_ids.dup
  ids << the_id
  write_attribute(:following_ids, ids)
 end

 def following_ids_delete(the_id)
  ids = self.following_ids.dup
  ids.delete(the_id)
  write_attribute(:following_ids, ids)
 end

end # End Class
Oh yeah, and Cheat.
     (It's ok!)
Thing about your application.
How can you cheat and get away
           with it?
Is your data delivered in real time?
    Is your data static content?
      How do users interact?
Interestingness.
(Little things that don't deserve other space.)
It's OK to use Monit to kill processes
          if they get too big.
Ensure you can deploy frequently.
Ensure you can roll back easily.
Scale where it matters.
Some code is ugly. It's OK.
  (who needs a hug?)
Ensure your users can give feedback
              easily.
Use the Community.
Make an API.
(Scale your Developer-base.)
We run on Edge (but with Piston).
A Cool Trick. Gems in Vendor.
Rails::Initializer.run do |config|

 # Load Gems from the /vendor/gems folder first, if they exist.
 config.load_paths += Dir[quot;#{RAILS_ROOT}/vendor/gems/**quot;].map do |dir|
  File.directory?(lib = quot;#{dir}/libquot;) ? lib : dir
 end

 ...
Personal Pet Peeve.

It's 2007. Every spammer has your email address.
  Put it on your goddamn webpage so people can
      get ahold of you about interesting things.
Questions?
Britt Selvitelle

      IM & Email
anotherbritt@gmail.com

More Related Content

What's hot

Painless Persistence in a Disconnected World
Painless Persistence in a Disconnected WorldPainless Persistence in a Disconnected World
Painless Persistence in a Disconnected WorldChristian Melchior
 
Java script at backend nodejs
Java script at backend   nodejsJava script at backend   nodejs
Java script at backend nodejsAmit Thakkar
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...Tom Croucher
 
A million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scaleA million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scaleTom Croucher
 
Why Task Queues - ComoRichWeb
Why Task Queues - ComoRichWebWhy Task Queues - ComoRichWeb
Why Task Queues - ComoRichWebBryan Helmig
 
Realm of the Mobile Database: an introduction to Realm
Realm of the Mobile Database: an introduction to RealmRealm of the Mobile Database: an introduction to Realm
Realm of the Mobile Database: an introduction to RealmMartin Grider
 
node.js, javascript and the future
node.js, javascript and the futurenode.js, javascript and the future
node.js, javascript and the futureJeff Miccolis
 
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015Matt Raible
 
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet Campmodern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet CampPuppet
 
Building an App with jQuery and JAXER
Building an App with jQuery and JAXERBuilding an App with jQuery and JAXER
Building an App with jQuery and JAXERBrian Moschel
 
Introduction to rest.li
Introduction to rest.liIntroduction to rest.li
Introduction to rest.liJoe Betz
 
Nodejs Event Driven Concurrency for Web Applications
Nodejs Event Driven Concurrency for Web ApplicationsNodejs Event Driven Concurrency for Web Applications
Nodejs Event Driven Concurrency for Web ApplicationsGanesh Iyer
 
&lt;x> Rails Web App Security Title
&lt;x> Rails Web App Security Title&lt;x> Rails Web App Security Title
&lt;x> Rails Web App Security Title'"><x> '"><x>
 
Getting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
Getting Started with Apache Cassandra by Junior Evangelist Rebecca MillsGetting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
Getting Started with Apache Cassandra by Junior Evangelist Rebecca MillsDataStax Academy
 
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac..."Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac...Fwdays
 
Server Side Event Driven Programming
Server Side Event Driven ProgrammingServer Side Event Driven Programming
Server Side Event Driven ProgrammingKamal Hussain
 
An Introduction to Windows PowerShell
An Introduction to Windows PowerShellAn Introduction to Windows PowerShell
An Introduction to Windows PowerShellDale Lane
 
Async servers and clients in Rest.li
Async servers and clients in Rest.liAsync servers and clients in Rest.li
Async servers and clients in Rest.liKaran Parikh
 
Scaling up task processing with Celery
Scaling up task processing with CeleryScaling up task processing with Celery
Scaling up task processing with CeleryNicolas Grasset
 

What's hot (20)

Painless Persistence in a Disconnected World
Painless Persistence in a Disconnected WorldPainless Persistence in a Disconnected World
Painless Persistence in a Disconnected World
 
Java script at backend nodejs
Java script at backend   nodejsJava script at backend   nodejs
Java script at backend nodejs
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...
 
A million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scaleA million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scale
 
Why Task Queues - ComoRichWeb
Why Task Queues - ComoRichWebWhy Task Queues - ComoRichWeb
Why Task Queues - ComoRichWeb
 
Realm of the Mobile Database: an introduction to Realm
Realm of the Mobile Database: an introduction to RealmRealm of the Mobile Database: an introduction to Realm
Realm of the Mobile Database: an introduction to Realm
 
node.js, javascript and the future
node.js, javascript and the futurenode.js, javascript and the future
node.js, javascript and the future
 
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
 
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet Campmodern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
 
Building an App with jQuery and JAXER
Building an App with jQuery and JAXERBuilding an App with jQuery and JAXER
Building an App with jQuery and JAXER
 
Introduction to rest.li
Introduction to rest.liIntroduction to rest.li
Introduction to rest.li
 
Nodejs Event Driven Concurrency for Web Applications
Nodejs Event Driven Concurrency for Web ApplicationsNodejs Event Driven Concurrency for Web Applications
Nodejs Event Driven Concurrency for Web Applications
 
&lt;x> Rails Web App Security Title
&lt;x> Rails Web App Security Title&lt;x> Rails Web App Security Title
&lt;x> Rails Web App Security Title
 
Getting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
Getting Started with Apache Cassandra by Junior Evangelist Rebecca MillsGetting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
Getting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
 
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac..."Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
 
Server Side Event Driven Programming
Server Side Event Driven ProgrammingServer Side Event Driven Programming
Server Side Event Driven Programming
 
An Introduction to Windows PowerShell
An Introduction to Windows PowerShellAn Introduction to Windows PowerShell
An Introduction to Windows PowerShell
 
Async servers and clients in Rest.li
Async servers and clients in Rest.liAsync servers and clients in Rest.li
Async servers and clients in Rest.li
 
Scaling up task processing with Celery
Scaling up task processing with CeleryScaling up task processing with Celery
Scaling up task processing with Celery
 
AJAX Transport Layer
AJAX Transport LayerAJAX Transport Layer
AJAX Transport Layer
 

Viewers also liked

Twitter - Architecture and Scalability lessons
Twitter - Architecture and Scalability lessonsTwitter - Architecture and Scalability lessons
Twitter - Architecture and Scalability lessonsAditya Rao
 
Improving Running Components at Twitter
Improving Running Components at TwitterImproving Running Components at Twitter
Improving Running Components at TwitterEvan Weaver
 
Evolution of The Twitter Stack
Evolution of The Twitter StackEvolution of The Twitter Stack
Evolution of The Twitter StackChris Aniszczyk
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to RedisDvir Volk
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsJonas Bonér
 

Viewers also liked (6)

Twitter - Architecture and Scalability lessons
Twitter - Architecture and Scalability lessonsTwitter - Architecture and Scalability lessons
Twitter - Architecture and Scalability lessons
 
Improving Running Components at Twitter
Improving Running Components at TwitterImproving Running Components at Twitter
Improving Running Components at Twitter
 
OAuth
OAuthOAuth
OAuth
 
Evolution of The Twitter Stack
Evolution of The Twitter StackEvolution of The Twitter Stack
Evolution of The Twitter Stack
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability Patterns
 

Similar to A Small Talk on Getting Big

[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...Functional Thursday
 
The Art Of Readable Code
The Art Of Readable CodeThe Art Of Readable Code
The Art Of Readable CodeBaidu, Inc.
 
Beyond php it's not (just) about the code
Beyond php   it's not (just) about the codeBeyond php   it's not (just) about the code
Beyond php it's not (just) about the codeWim Godden
 
Beyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeBeyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeWim Godden
 
"Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin "Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin Vasil Remeniuk
 
Crossing the Bridge: Connecting Rails and your Front-end Framework
Crossing the Bridge: Connecting Rails and your Front-end FrameworkCrossing the Bridge: Connecting Rails and your Front-end Framework
Crossing the Bridge: Connecting Rails and your Front-end FrameworkDaniel Spector
 
He stopped using for/while loops, you won't believe what happened next!
He stopped using for/while loops, you won't believe what happened next!He stopped using for/while loops, you won't believe what happened next!
He stopped using for/while loops, you won't believe what happened next!François-Guillaume Ribreau
 
Beyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the codeBeyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the codeWim Godden
 
Random Ruby Tips - Ruby Meetup 27 Jun 2018
Random Ruby Tips - Ruby Meetup 27 Jun 2018Random Ruby Tips - Ruby Meetup 27 Jun 2018
Random Ruby Tips - Ruby Meetup 27 Jun 2018Kenneth Teh
 
Intro to-rails-webperf
Intro to-rails-webperfIntro to-rails-webperf
Intro to-rails-webperfNew Relic
 
Rails Tips and Best Practices
Rails Tips and Best PracticesRails Tips and Best Practices
Rails Tips and Best PracticesDavid Keener
 
Zero, one, two, Freddy's coming for you
Zero, one, two, Freddy's coming for youZero, one, two, Freddy's coming for you
Zero, one, two, Freddy's coming for youAndrey Karpov
 
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..Mark Rackley
 

Similar to A Small Talk on Getting Big (20)

The Joy Of Ruby
The Joy Of RubyThe Joy Of Ruby
The Joy Of Ruby
 
[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...
 
The Art Of Readable Code
The Art Of Readable CodeThe Art Of Readable Code
The Art Of Readable Code
 
Beyond php it's not (just) about the code
Beyond php   it's not (just) about the codeBeyond php   it's not (just) about the code
Beyond php it's not (just) about the code
 
Beyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeBeyond php - it's not (just) about the code
Beyond php - it's not (just) about the code
 
"Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin "Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin
 
Rspec
RspecRspec
Rspec
 
Crossing the Bridge: Connecting Rails and your Front-end Framework
Crossing the Bridge: Connecting Rails and your Front-end FrameworkCrossing the Bridge: Connecting Rails and your Front-end Framework
Crossing the Bridge: Connecting Rails and your Front-end Framework
 
NodeJS for Beginner
NodeJS for BeginnerNodeJS for Beginner
NodeJS for Beginner
 
Migrating legacy data
Migrating legacy dataMigrating legacy data
Migrating legacy data
 
He stopped using for/while loops, you won't believe what happened next!
He stopped using for/while loops, you won't believe what happened next!He stopped using for/while loops, you won't believe what happened next!
He stopped using for/while loops, you won't believe what happened next!
 
Coding Naked 2023
Coding Naked 2023Coding Naked 2023
Coding Naked 2023
 
Beyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the codeBeyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the code
 
Random Ruby Tips - Ruby Meetup 27 Jun 2018
Random Ruby Tips - Ruby Meetup 27 Jun 2018Random Ruby Tips - Ruby Meetup 27 Jun 2018
Random Ruby Tips - Ruby Meetup 27 Jun 2018
 
Intro to-rails-webperf
Intro to-rails-webperfIntro to-rails-webperf
Intro to-rails-webperf
 
Rails Tips and Best Practices
Rails Tips and Best PracticesRails Tips and Best Practices
Rails Tips and Best Practices
 
Zero, one, two, Freddy's coming for you
Zero, one, two, Freddy's coming for youZero, one, two, Freddy's coming for you
Zero, one, two, Freddy's coming for you
 
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
 
Refactoring
RefactoringRefactoring
Refactoring
 
Refactoring
RefactoringRefactoring
Refactoring
 

Recently uploaded

New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 3652toLead Limited
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Wonjun Hwang
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 

Recently uploaded (20)

New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
Hot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort Service
Hot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort ServiceHot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort Service
Hot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort Service
 

A Small Talk on Getting Big

  • 1. A Small Talk on Getting Big. Scaling a Rails App & all that Jazz.
  • 5. How did this Happen?
  • 7. Too Much Time in the Application. Too Much Time in the Database.
  • 8. A Really Cool Trick class ApplicationController < ActionController:Base include ResponseManagementAdditions after_filter :show_runtimes around_filter { |controller, action| controller.total_runtime = Benchmark.measure(&action).real } end
  • 9. A Really Cool Trick module ResponseManagementAdditions def show_runtimes return if response.headers['Status'] == '304 Not Modified' || ! (response.body && response.body.respond_to?(:sub!)) db_runtime = ((0 + (@db_rt_before_render || 0) + (@db_rt_after_render || 0)) * 1000).truncate rendering_runtime = ((@rendering_runtime || 0) * 1000).truncate total_runtime = ((@total_runtime || 0) * 1000).truncate response.body.gsub!(/</body></html>$/, quot;<!-- served to you through a copper wire by #{HOSTNAME} at #{Time.now.to_s(:short)} in #{total_runtime} ms (d #{db_runtime} / r #{rendering_runtime}). thank you, come again. --> n</body></html>quot;) end end
  • 10. Two Ways People Design Sites. Over-architect. Under-architect.
  • 11. Move Fast. Scale Quickly.
  • 12. Too Much Time in the Application.
  • 13. Abstract Long Running Processes to Daemons.
  • 14. An Ugly but Amazingly Simple Queuing System.
  • 15. class FooDaemon < TwitterDaemon::Base before_startup :set_fugly_dist_idx def process unprocessed_content do |c| increment_counter(:total) # Do work ... break unless running? end end ...
  • 16. ... def unprocessed_content(&block) loop do content = ContentModel.pending_content(quot;substring(truncate(id, 0),-2,1) = #{@fugly_dist_idx}quot;) messages.each { |message| yield message } sleep 1 if messages.nil? || messages.empty? end end def set_fugly_dist_idx @fugly_dist_idx = ARGV.find { |v| v.match(/[0-9]/) } raise quot;You need to specify a dist idx between 0 and 9.quot; unless @fugly_dist_idx @fugly_dist_idx = @fugly_dist_idx.to_i end end
  • 19. Distributed Queuing. Transactional Playback. Fast. Simple. Speaks Memcache's Language. 100% Pure Ruby.
  • 22. Too Much Time in the Database.
  • 23. The Basics. Database 101. (I shouldn't need this slide in here.)
  • 24. Index everything you will query on. Avoid complex joins. Use joint indices when you must join tables. Avoid scanning large sets of data.
  • 30. Turns Out it Isn't.
  • 32. class User < ActiveRecord::Base serialize :following_ids def following_ids # this accessor is overwritten because we want to lazily set the # friends_ids column, rather than running a gigantic slow migration. RAILS_DEFAULT_LOGGER.debug quot;loading following_idsquot; ids = read_attribute(:following_ids) if ids.nil? || !ids.kind_of?(Array) ids = connection.select_values(quot;SELECT DISTINCT followed_user_id FROM followed_users WHERE user_id = #{self.id}quot;).map(&:to_i).compact update_attribute(:following_ids, ids) end ids end ...
  • 33. def following_ids_add(the_id) ids = self.following_ids.dup ids << the_id write_attribute(:following_ids, ids) end def following_ids_delete(the_id) ids = self.following_ids.dup ids.delete(the_id) write_attribute(:following_ids, ids) end end # End Class
  • 34. Oh yeah, and Cheat. (It's ok!)
  • 35. Thing about your application. How can you cheat and get away with it?
  • 36. Is your data delivered in real time? Is your data static content? How do users interact?
  • 37. Interestingness. (Little things that don't deserve other space.)
  • 38. It's OK to use Monit to kill processes if they get too big.
  • 39. Ensure you can deploy frequently.
  • 40. Ensure you can roll back easily.
  • 41. Scale where it matters.
  • 42. Some code is ugly. It's OK. (who needs a hug?)
  • 43. Ensure your users can give feedback easily.
  • 45. Make an API. (Scale your Developer-base.)
  • 46. We run on Edge (but with Piston).
  • 47. A Cool Trick. Gems in Vendor. Rails::Initializer.run do |config| # Load Gems from the /vendor/gems folder first, if they exist. config.load_paths += Dir[quot;#{RAILS_ROOT}/vendor/gems/**quot;].map do |dir| File.directory?(lib = quot;#{dir}/libquot;) ? lib : dir end ...
  • 48. Personal Pet Peeve. It's 2007. Every spammer has your email address. Put it on your goddamn webpage so people can get ahold of you about interesting things.
  • 50. Britt Selvitelle IM & Email anotherbritt@gmail.com