SlideShare a Scribd company logo
A Small Talk on Getting Big.
Scaling a Rails App & all that Jazz.
Good Dog Picture.
Medium Dog.
Bad Dog Picture.
How did this Happen?
Mongrel Picture.
Too Much Time in the Application.
 Too Much Time in the Database.
A Really Cool Trick
class ApplicationController < ActionController:Base
    include ResponseManagementAdditions
    after_filter :show_runtimes

   around_filter { |controller, action| controller.total_runtime =
   Benchmark.measure(&action).real }
end
A Really Cool Trick
module ResponseManagementAdditions
   def show_runtimes
    return if response.headers['Status'] == '304 Not Modified' || !
(response.body && response.body.respond_to?(:sub!))

   db_runtime = ((0 + (@db_rt_before_render || 0) +
(@db_rt_after_render || 0)) * 1000).truncate
   rendering_runtime = ((@rendering_runtime || 0) * 1000).truncate
   total_runtime = ((@total_runtime || 0) * 1000).truncate

    response.body.gsub!(/</body></html>$/, quot;<!-- served to you
through a copper wire by #{HOSTNAME} at
#{Time.now.to_s(:short)} in #{total_runtime} ms (d #{db_runtime} / r
#{rendering_runtime}). thank you, come again. -->
n</body></html>quot;)
   end
end
Two Ways People Design Sites.

       Over-architect.
       Under-architect.
Move Fast. Scale Quickly.
Too Much Time in the Application.
Abstract Long Running Processes
          to Daemons.
An Ugly but Amazingly Simple Queuing
              System.
class FooDaemon < TwitterDaemon::Base
 before_startup :set_fugly_dist_idx

 def process
  unprocessed_content do |c|
   increment_counter(:total)

       # Do work ...

   break unless running?
  end
 end

 ...
...

def unprocessed_content(&block)
   loop do
     content = ContentModel.pending_content(quot;substring(truncate(id,
0),-2,1) = #{@fugly_dist_idx}quot;)
     messages.each { |message| yield message }
     sleep 1 if messages.nil? || messages.empty?
   end
 end

 def set_fugly_dist_idx
  @fugly_dist_idx = ARGV.find { |v| v.match(/[0-9]/) }
  raise quot;You need to specify a dist idx between 0 and 9.quot; unless
@fugly_dist_idx
  @fugly_dist_idx = @fugly_dist_idx.to_i
 end
end
A Better Queuing System.
Starling.
Distributed Queuing.
   Transactional Playback.
            Fast.
           Simple.
Speaks Memcache's Language.
      100% Pure Ruby.
Not Open Source.
(yet ...)
Too Much Time in the Database.
The Basics. Database 101.
(I shouldn't need this slide in here.)
Index everything you will query on.
            Avoid complex joins.
Use joint indices when you must join tables.
     Avoid scanning large sets of data.
Cache.
Cache.
Cache.
CACHE!
But How?
That Sounds Hard.
Turns Out it Isn't.
Serialize, Denormalize.
class User < ActiveRecord::Base
 serialize :following_ids

 def following_ids
  # this accessor is overwritten because we want to lazily set the
  # friends_ids column, rather than running a gigantic slow migration.
  RAILS_DEFAULT_LOGGER.debug quot;loading following_idsquot;
  ids = read_attribute(:following_ids)

  if ids.nil? || !ids.kind_of?(Array)
    ids = connection.select_values(quot;SELECT DISTINCT followed_user_id
FROM followed_users WHERE user_id =
#{self.id}quot;).map(&:to_i).compact
    update_attribute(:following_ids, ids)
  end

  ids
 end

 ...
def following_ids_add(the_id)
  ids = self.following_ids.dup
  ids << the_id
  write_attribute(:following_ids, ids)
 end

 def following_ids_delete(the_id)
  ids = self.following_ids.dup
  ids.delete(the_id)
  write_attribute(:following_ids, ids)
 end

end # End Class
Oh yeah, and Cheat.
     (It's ok!)
Thing about your application.
How can you cheat and get away
           with it?
Is your data delivered in real time?
    Is your data static content?
      How do users interact?
Interestingness.
(Little things that don't deserve other space.)
It's OK to use Monit to kill processes
          if they get too big.
Ensure you can deploy frequently.
Ensure you can roll back easily.
Scale where it matters.
Some code is ugly. It's OK.
  (who needs a hug?)
Ensure your users can give feedback
              easily.
Use the Community.
Make an API.
(Scale your Developer-base.)
We run on Edge (but with Piston).
A Cool Trick. Gems in Vendor.
Rails::Initializer.run do |config|

 # Load Gems from the /vendor/gems folder first, if they exist.
 config.load_paths += Dir[quot;#{RAILS_ROOT}/vendor/gems/**quot;].map do |dir|
  File.directory?(lib = quot;#{dir}/libquot;) ? lib : dir
 end

 ...
Personal Pet Peeve.

It's 2007. Every spammer has your email address.
  Put it on your goddamn webpage so people can
      get ahold of you about interesting things.
Questions?
Britt Selvitelle

      IM & Email
anotherbritt@gmail.com

More Related Content

What's hot

Painless Persistence in a Disconnected World
Painless Persistence in a Disconnected WorldPainless Persistence in a Disconnected World
Painless Persistence in a Disconnected World
Christian Melchior
 
Java script at backend nodejs
Java script at backend   nodejsJava script at backend   nodejs
Java script at backend nodejs
Amit Thakkar
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...
Tom Croucher
 
A million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scaleA million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scale
Tom Croucher
 
Why Task Queues - ComoRichWeb
Why Task Queues - ComoRichWebWhy Task Queues - ComoRichWeb
Why Task Queues - ComoRichWeb
Bryan Helmig
 
Realm of the Mobile Database: an introduction to Realm
Realm of the Mobile Database: an introduction to RealmRealm of the Mobile Database: an introduction to Realm
Realm of the Mobile Database: an introduction to Realm
Martin Grider
 
node.js, javascript and the future
node.js, javascript and the futurenode.js, javascript and the future
node.js, javascript and the future
Jeff Miccolis
 
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
Matt Raible
 
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet Campmodern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
Puppet
 
Building an App with jQuery and JAXER
Building an App with jQuery and JAXERBuilding an App with jQuery and JAXER
Building an App with jQuery and JAXER
Brian Moschel
 
Introduction to rest.li
Introduction to rest.liIntroduction to rest.li
Introduction to rest.li
Joe Betz
 
Nodejs Event Driven Concurrency for Web Applications
Nodejs Event Driven Concurrency for Web ApplicationsNodejs Event Driven Concurrency for Web Applications
Nodejs Event Driven Concurrency for Web Applications
Ganesh Iyer
 
&lt;x> Rails Web App Security Title
&lt;x> Rails Web App Security Title&lt;x> Rails Web App Security Title
&lt;x> Rails Web App Security Title
'"><x> '"><x>
 
Getting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
Getting Started with Apache Cassandra by Junior Evangelist Rebecca MillsGetting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
Getting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
DataStax Academy
 
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac..."Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
Fwdays
 
Server Side Event Driven Programming
Server Side Event Driven ProgrammingServer Side Event Driven Programming
Server Side Event Driven Programming
Kamal Hussain
 
An Introduction to Windows PowerShell
An Introduction to Windows PowerShellAn Introduction to Windows PowerShell
An Introduction to Windows PowerShell
Dale Lane
 
Async servers and clients in Rest.li
Async servers and clients in Rest.liAsync servers and clients in Rest.li
Async servers and clients in Rest.li
Karan Parikh
 
Scaling up task processing with Celery
Scaling up task processing with CeleryScaling up task processing with Celery
Scaling up task processing with Celery
Nicolas Grasset
 
AJAX Transport Layer
AJAX Transport LayerAJAX Transport Layer
AJAX Transport Layer
Siarhei Barysiuk
 

What's hot (20)

Painless Persistence in a Disconnected World
Painless Persistence in a Disconnected WorldPainless Persistence in a Disconnected World
Painless Persistence in a Disconnected World
 
Java script at backend nodejs
Java script at backend   nodejsJava script at backend   nodejs
Java script at backend nodejs
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...
 
A million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scaleA million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scale
 
Why Task Queues - ComoRichWeb
Why Task Queues - ComoRichWebWhy Task Queues - ComoRichWeb
Why Task Queues - ComoRichWeb
 
Realm of the Mobile Database: an introduction to Realm
Realm of the Mobile Database: an introduction to RealmRealm of the Mobile Database: an introduction to Realm
Realm of the Mobile Database: an introduction to Realm
 
node.js, javascript and the future
node.js, javascript and the futurenode.js, javascript and the future
node.js, javascript and the future
 
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
 
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet Campmodern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
 
Building an App with jQuery and JAXER
Building an App with jQuery and JAXERBuilding an App with jQuery and JAXER
Building an App with jQuery and JAXER
 
Introduction to rest.li
Introduction to rest.liIntroduction to rest.li
Introduction to rest.li
 
Nodejs Event Driven Concurrency for Web Applications
Nodejs Event Driven Concurrency for Web ApplicationsNodejs Event Driven Concurrency for Web Applications
Nodejs Event Driven Concurrency for Web Applications
 
&lt;x> Rails Web App Security Title
&lt;x> Rails Web App Security Title&lt;x> Rails Web App Security Title
&lt;x> Rails Web App Security Title
 
Getting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
Getting Started with Apache Cassandra by Junior Evangelist Rebecca MillsGetting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
Getting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
 
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac..."Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
 
Server Side Event Driven Programming
Server Side Event Driven ProgrammingServer Side Event Driven Programming
Server Side Event Driven Programming
 
An Introduction to Windows PowerShell
An Introduction to Windows PowerShellAn Introduction to Windows PowerShell
An Introduction to Windows PowerShell
 
Async servers and clients in Rest.li
Async servers and clients in Rest.liAsync servers and clients in Rest.li
Async servers and clients in Rest.li
 
Scaling up task processing with Celery
Scaling up task processing with CeleryScaling up task processing with Celery
Scaling up task processing with Celery
 
AJAX Transport Layer
AJAX Transport LayerAJAX Transport Layer
AJAX Transport Layer
 

Viewers also liked

Twitter - Architecture and Scalability lessons
Twitter - Architecture and Scalability lessonsTwitter - Architecture and Scalability lessons
Twitter - Architecture and Scalability lessons
Aditya Rao
 
Improving Running Components at Twitter
Improving Running Components at TwitterImproving Running Components at Twitter
Improving Running Components at Twitter
Evan Weaver
 
OAuth
OAuthOAuth
OAuth
Blaine
 
Evolution of The Twitter Stack
Evolution of The Twitter StackEvolution of The Twitter Stack
Evolution of The Twitter Stack
Chris Aniszczyk
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
Dvir Volk
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability Patterns
Jonas Bonér
 

Viewers also liked (6)

Twitter - Architecture and Scalability lessons
Twitter - Architecture and Scalability lessonsTwitter - Architecture and Scalability lessons
Twitter - Architecture and Scalability lessons
 
Improving Running Components at Twitter
Improving Running Components at TwitterImproving Running Components at Twitter
Improving Running Components at Twitter
 
OAuth
OAuthOAuth
OAuth
 
Evolution of The Twitter Stack
Evolution of The Twitter StackEvolution of The Twitter Stack
Evolution of The Twitter Stack
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability Patterns
 

Similar to A Small Talk on Getting Big

The Joy Of Ruby
The Joy Of RubyThe Joy Of Ruby
The Joy Of Ruby
Clinton Dreisbach
 
[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...
Functional Thursday
 
The Art Of Readable Code
The Art Of Readable CodeThe Art Of Readable Code
The Art Of Readable Code
Baidu, Inc.
 
Beyond php it's not (just) about the code
Beyond php   it's not (just) about the codeBeyond php   it's not (just) about the code
Beyond php it's not (just) about the code
Wim Godden
 
Beyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeBeyond php - it's not (just) about the code
Beyond php - it's not (just) about the code
Wim Godden
 
"Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin "Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin
Vasil Remeniuk
 
Rspec
RspecRspec
Crossing the Bridge: Connecting Rails and your Front-end Framework
Crossing the Bridge: Connecting Rails and your Front-end FrameworkCrossing the Bridge: Connecting Rails and your Front-end Framework
Crossing the Bridge: Connecting Rails and your Front-end Framework
Daniel Spector
 
NodeJS for Beginner
NodeJS for BeginnerNodeJS for Beginner
NodeJS for Beginner
Apaichon Punopas
 
Migrating legacy data
Migrating legacy dataMigrating legacy data
Migrating legacy data
Patrick Huesler
 
He stopped using for/while loops, you won't believe what happened next!
He stopped using for/while loops, you won't believe what happened next!He stopped using for/while loops, you won't believe what happened next!
He stopped using for/while loops, you won't believe what happened next!
François-Guillaume Ribreau
 
Coding Naked 2023
Coding Naked 2023Coding Naked 2023
Coding Naked 2023
Caleb Jenkins
 
Beyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the codeBeyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the code
Wim Godden
 
Random Ruby Tips - Ruby Meetup 27 Jun 2018
Random Ruby Tips - Ruby Meetup 27 Jun 2018Random Ruby Tips - Ruby Meetup 27 Jun 2018
Random Ruby Tips - Ruby Meetup 27 Jun 2018
Kenneth Teh
 
Intro to-rails-webperf
Intro to-rails-webperfIntro to-rails-webperf
Intro to-rails-webperf
New Relic
 
Rails Tips and Best Practices
Rails Tips and Best PracticesRails Tips and Best Practices
Rails Tips and Best Practices
David Keener
 
Zero, one, two, Freddy's coming for you
Zero, one, two, Freddy's coming for youZero, one, two, Freddy's coming for you
Zero, one, two, Freddy's coming for you
Andrey Karpov
 
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
Mark Rackley
 
Refactoring
RefactoringRefactoring
Refactoring
Amir Barylko
 
Refactoring
RefactoringRefactoring
Refactoring
Amir Barylko
 

Similar to A Small Talk on Getting Big (20)

The Joy Of Ruby
The Joy Of RubyThe Joy Of Ruby
The Joy Of Ruby
 
[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...
 
The Art Of Readable Code
The Art Of Readable CodeThe Art Of Readable Code
The Art Of Readable Code
 
Beyond php it's not (just) about the code
Beyond php   it's not (just) about the codeBeyond php   it's not (just) about the code
Beyond php it's not (just) about the code
 
Beyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeBeyond php - it's not (just) about the code
Beyond php - it's not (just) about the code
 
"Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin "Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin
 
Rspec
RspecRspec
Rspec
 
Crossing the Bridge: Connecting Rails and your Front-end Framework
Crossing the Bridge: Connecting Rails and your Front-end FrameworkCrossing the Bridge: Connecting Rails and your Front-end Framework
Crossing the Bridge: Connecting Rails and your Front-end Framework
 
NodeJS for Beginner
NodeJS for BeginnerNodeJS for Beginner
NodeJS for Beginner
 
Migrating legacy data
Migrating legacy dataMigrating legacy data
Migrating legacy data
 
He stopped using for/while loops, you won't believe what happened next!
He stopped using for/while loops, you won't believe what happened next!He stopped using for/while loops, you won't believe what happened next!
He stopped using for/while loops, you won't believe what happened next!
 
Coding Naked 2023
Coding Naked 2023Coding Naked 2023
Coding Naked 2023
 
Beyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the codeBeyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the code
 
Random Ruby Tips - Ruby Meetup 27 Jun 2018
Random Ruby Tips - Ruby Meetup 27 Jun 2018Random Ruby Tips - Ruby Meetup 27 Jun 2018
Random Ruby Tips - Ruby Meetup 27 Jun 2018
 
Intro to-rails-webperf
Intro to-rails-webperfIntro to-rails-webperf
Intro to-rails-webperf
 
Rails Tips and Best Practices
Rails Tips and Best PracticesRails Tips and Best Practices
Rails Tips and Best Practices
 
Zero, one, two, Freddy's coming for you
Zero, one, two, Freddy's coming for youZero, one, two, Freddy's coming for you
Zero, one, two, Freddy's coming for you
 
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
 
Refactoring
RefactoringRefactoring
Refactoring
 
Refactoring
RefactoringRefactoring
Refactoring
 

Recently uploaded

Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
Azure API Management to expose backend services securely
Azure API Management to expose backend services securelyAzure API Management to expose backend services securely
Azure API Management to expose backend services securely
Dinusha Kumarasiri
 
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Tatiana Kojar
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Antonios Katsarakis
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process MiningTrusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
LucaBarbaro3
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
AstuteBusiness
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
Postman
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
alexjohnson7307
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
SAP S/4 HANA sourcing and procurement to Public cloud
SAP S/4 HANA sourcing and procurement to Public cloudSAP S/4 HANA sourcing and procurement to Public cloud
SAP S/4 HANA sourcing and procurement to Public cloud
maazsz111
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 

Recently uploaded (20)

Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
Azure API Management to expose backend services securely
Azure API Management to expose backend services securelyAzure API Management to expose backend services securely
Azure API Management to expose backend services securely
 
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process MiningTrusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
SAP S/4 HANA sourcing and procurement to Public cloud
SAP S/4 HANA sourcing and procurement to Public cloudSAP S/4 HANA sourcing and procurement to Public cloud
SAP S/4 HANA sourcing and procurement to Public cloud
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 

A Small Talk on Getting Big

  • 1. A Small Talk on Getting Big. Scaling a Rails App & all that Jazz.
  • 5. How did this Happen?
  • 7. Too Much Time in the Application. Too Much Time in the Database.
  • 8. A Really Cool Trick class ApplicationController < ActionController:Base include ResponseManagementAdditions after_filter :show_runtimes around_filter { |controller, action| controller.total_runtime = Benchmark.measure(&action).real } end
  • 9. A Really Cool Trick module ResponseManagementAdditions def show_runtimes return if response.headers['Status'] == '304 Not Modified' || ! (response.body && response.body.respond_to?(:sub!)) db_runtime = ((0 + (@db_rt_before_render || 0) + (@db_rt_after_render || 0)) * 1000).truncate rendering_runtime = ((@rendering_runtime || 0) * 1000).truncate total_runtime = ((@total_runtime || 0) * 1000).truncate response.body.gsub!(/</body></html>$/, quot;<!-- served to you through a copper wire by #{HOSTNAME} at #{Time.now.to_s(:short)} in #{total_runtime} ms (d #{db_runtime} / r #{rendering_runtime}). thank you, come again. --> n</body></html>quot;) end end
  • 10. Two Ways People Design Sites. Over-architect. Under-architect.
  • 11. Move Fast. Scale Quickly.
  • 12. Too Much Time in the Application.
  • 13. Abstract Long Running Processes to Daemons.
  • 14. An Ugly but Amazingly Simple Queuing System.
  • 15. class FooDaemon < TwitterDaemon::Base before_startup :set_fugly_dist_idx def process unprocessed_content do |c| increment_counter(:total) # Do work ... break unless running? end end ...
  • 16. ... def unprocessed_content(&block) loop do content = ContentModel.pending_content(quot;substring(truncate(id, 0),-2,1) = #{@fugly_dist_idx}quot;) messages.each { |message| yield message } sleep 1 if messages.nil? || messages.empty? end end def set_fugly_dist_idx @fugly_dist_idx = ARGV.find { |v| v.match(/[0-9]/) } raise quot;You need to specify a dist idx between 0 and 9.quot; unless @fugly_dist_idx @fugly_dist_idx = @fugly_dist_idx.to_i end end
  • 19. Distributed Queuing. Transactional Playback. Fast. Simple. Speaks Memcache's Language. 100% Pure Ruby.
  • 22. Too Much Time in the Database.
  • 23. The Basics. Database 101. (I shouldn't need this slide in here.)
  • 24. Index everything you will query on. Avoid complex joins. Use joint indices when you must join tables. Avoid scanning large sets of data.
  • 30. Turns Out it Isn't.
  • 32. class User < ActiveRecord::Base serialize :following_ids def following_ids # this accessor is overwritten because we want to lazily set the # friends_ids column, rather than running a gigantic slow migration. RAILS_DEFAULT_LOGGER.debug quot;loading following_idsquot; ids = read_attribute(:following_ids) if ids.nil? || !ids.kind_of?(Array) ids = connection.select_values(quot;SELECT DISTINCT followed_user_id FROM followed_users WHERE user_id = #{self.id}quot;).map(&:to_i).compact update_attribute(:following_ids, ids) end ids end ...
  • 33. def following_ids_add(the_id) ids = self.following_ids.dup ids << the_id write_attribute(:following_ids, ids) end def following_ids_delete(the_id) ids = self.following_ids.dup ids.delete(the_id) write_attribute(:following_ids, ids) end end # End Class
  • 34. Oh yeah, and Cheat. (It's ok!)
  • 35. Thing about your application. How can you cheat and get away with it?
  • 36. Is your data delivered in real time? Is your data static content? How do users interact?
  • 37. Interestingness. (Little things that don't deserve other space.)
  • 38. It's OK to use Monit to kill processes if they get too big.
  • 39. Ensure you can deploy frequently.
  • 40. Ensure you can roll back easily.
  • 41. Scale where it matters.
  • 42. Some code is ugly. It's OK. (who needs a hug?)
  • 43. Ensure your users can give feedback easily.
  • 45. Make an API. (Scale your Developer-base.)
  • 46. We run on Edge (but with Piston).
  • 47. A Cool Trick. Gems in Vendor. Rails::Initializer.run do |config| # Load Gems from the /vendor/gems folder first, if they exist. config.load_paths += Dir[quot;#{RAILS_ROOT}/vendor/gems/**quot;].map do |dir| File.directory?(lib = quot;#{dir}/libquot;) ? lib : dir end ...
  • 48. Personal Pet Peeve. It's 2007. Every spammer has your email address. Put it on your goddamn webpage so people can get ahold of you about interesting things.
  • 50. Britt Selvitelle IM & Email anotherbritt@gmail.com