SlideShare a Scribd company logo
Ruby Drink-up / January 2013
http://rivierarb.fr/2013/01/08/Drinkup/
Karan Malkani
ABOUT THESE SLIDES…
A Presentation of the Twitter / Ruby couple and why it did not last
Motivated by my own interest in Ruby, born 2 years ago when reading that
   Twitter was made using Ruby (or Ruby on Rails)
An Internet-based desk research




But…


Not so much / easy available information due to the excellent Twitter
   communication
TWITTER HISTORY WITH (A BIG) SHORTCUT
   Born in the postcasting company Odeo
   Initial idea: an SMS service within a small group / Original code name: twttr / First prototype
    was used as an internal service for Odeo employees / Full version introduced publicly in July
    2006
   October 2006: Jack Dorsey, Biz Stonen, and Evan Williams formed Obvious Corporation,
    acquired Odeo (including twitter.com).
   Twitter, as a company, was launched in April 2007



   Today: the 2nd social network in the world after Facebook:
            (Estimated Unique Monthly Visitors – January 2013)

               •   1 Facebook - 750,000,000
               •   2 Twitter - 250,000,000
               •   3 LinkedIn - 110,000,000
               •   4 Pinterest - 85,500,000
               •   5 MySpace - 70,500,000
               •   6 Google Plus+ - 65,000,000



                   Source:

                   How Twitter Was Born http://www.140characters.com/2009/01/30/how-twitter-was-born/

                   Twitter Development History in 3 minute and 53 seconds! http://www.shoutmeloud.com/twitter-development-history-in-3-
                         minute-and-53-seconds.html

                   Wikipedia: Twitter http://en.wikipedia.org/wiki/Twitter

                   eBizMBA (http://www.ebizmba.com/articles/social-networking-websites)
A FANTASTIC GROWTH – (> 100M NEW USERS IN 2010)




                        Source:

                        http://blog.twitter.com/2010/02/measuring-tweets.html

                        http://mashable.com/2010/12/16/twitter-stats-2010/
THE PROBLEM




  Source:

  http://www.whatisfailwhale.info/

  http://en.wikipedia.org/wiki/Twitter#Outages

  http://www.wired.com/underwire/2008/07/twitter-fans-tu/
WHY SCALING?
Number of users
Number of messages (impacted by the Retweet feature realeased end 2009)
Applications that can publish tweets / Applications that access tweets
Search Engine (Twitter acquired Summize – July 2008)


2009 : Some critical outages / Many FailWhales experiences
June 12, 2009 – In what was called a potential "Twitpocalypse", the unique numerical identifier associated with each
     tweet exceeded the limit of 32-bit signedintegers (2,147,483,647 total messages).[142] While Twitter itself was not
     affected, some third-party clients could no longer access recent tweets. Patcheswere quickly released, though
     some iPhone applications had to wait for approval from the App Store.[143]
June 25, 2009 – Twitter ran slowly for some time after over 50,000 tweets on Michael Jackson’s death were recorded
     in an hour.[144]
August 6, 2009 – Twitter and Facebook suffered from a denial-of-service attack, causing the Twitter website to go
    offline for several hours.[145] It was later confirmed that the attacks were directed at one pro-Georgian user around
    the anniversary of the 2008 South Ossetia War, rather than the sites themselves.[146]
September 22, 2009 – The identifier exceeded the limit for 32-bit unsigned integers (4,294,967,296 total messages)
     again breaking some third-party clients.[147]




                                                        Source:

                                                        http://blog.twitter.com/2009/11/retweet-limited-rollout.html

                                                        http://techcrunch.com/2008/07/15/confirmed-twitter-acquires-summize-search-
                                                                engine/

                                                        http://en.wikipedia.org/wiki/Twitter#Outages

                                                        http://www.businessinsider.com/twitter-doesnt-have-product-geniuses-so-its-
                                                                buying-them-2010-4
STARTING TWITTER WITH RUBY
An excellent choice to
 Quickly prototype
 Quickly launch a first real site
 Quickly deploy an API for third-party applications
 Tease and attract developers

Improving performances with Ruby:


Example - mid 2009: Evan Weaver (Twitter)


We recently migrated Twitter from a custom Ruby 1.8.6 build to a
     Ruby Enterprise Edition release candidate, courtesy of
     Phusion. Our primary motivation was the integration of
     Brent’s MBARI patches, which increase memory stability.


Some features of REE have no effect on our codebase, but we
    definitely benefit from the MBARI patchset, the Railsbench
    tunable GC, and the various leak fixes in 1.8.7p174. These
    are difficult to integrate and Phusion has done a fine job.




                     Source:

                     http://blog.evanweaver.com/2009/09/24/ree/

                     http://www.linkedin.com/in/evanweaver/
RUBY ISSUES KNOWN WHEN TWITTER WAS FOUNDED
FULL COPY/PASTE OF: HOW HAS RUBY ON RAILS BEEN HOLDING UP TO THE INCREASED LOAD? (5 QUESTION INTERVIEW WITH TWITTER
DEVELOPER ALEX PAYNE – APRIL 2007)

By various metrics Twitter is the biggest Rails site on the net right               All the convenience methods and syntactical sugar that makes
now.                                                                                Rails such a pleasure for coders ends up being absolutely
                                                                                    punishing, performance-wise.
Running on Rails has forced us to deal with scaling issues - issues
that any growing site eventually contends with – far sooner than I                  Once you hit a certain threshold of traffic, either you need to
think we would on another framework.                                                strip out all the costly neat stuff that Rails does for you (RJS,
                                                                                    ActiveRecord, ActiveSupport, etc.) or move the slow parts of
The common wisdom in the Rails community at this time is that                       your application out of Rails, or both.
scaling Rails is a matter of cost: just throw more CPUs at it.
                                                                                    It’s also worth mentioning that there shouldn’t be doubt in
The problem is that more instances of Rails (running as part of a                   anybody’s mind at this point that Ruby itself is slow.
Mongrel cluster, in our case) means more requests to your database.
                                                                                    It’s great that people are hard at work on faster implementations
At this point in time there’s no facility in Rails to talk to more than one         of the language, but right now, it’s tough. If you’re looking to
database at a time.                                                                 deploy a big web application and you’re language-agnostic,
The solutions to this are caching the hell out of everything and                    realize that the same operation in Ruby will take less time in
setting up multiple read-only slave databases, neither of which are                 Python.
quick fixes to implement.                                                           All of us working on Twitter are big Ruby fans, but I think it’s
So it’s not just cost, it’s time, and time is that much more precious               worth being frank that this isn’t one of those relativistic language
when people can[‘t] reach your site.None of these scaling                           issues. Ruby is slow.
approaches are as fun and easy as developing for Rails.




                                                                         Source:

                                                                         http://yasulab.tumblr.com/post/10271634919/5-question-interview-with-
                                                                                twitter-developer-alex-payne/

                                                                         The original article on Radical Behavior is no longer accessible
ABANDONNING RUBY ON RAILS
April 2008 - After 2 years of high scaling problems: decision is made to abandon
   RoR


Start to change the infrastructure: Middleware moved to a mixture of C and
    Scala/JVM to run a messaging model, asynchronous process, 3 levels of
    cache (Twitter kept Ruby for the front-end only)


April 2009 - Twitter on Scala


Spring 2010 – changing the back-end from MySQL to a real-time version of
   Lucene (Apache database)


2011 – RoR front-end for search replaced with Blender, a Java server




                                     Source:

                                     http://techcrunch.com/2008/05/01/twitter-said-to-be-abandoning-ruby-on-rails/

                                     http://www.artima.com/scalazine/articles/twitter_on_scala.html

                                     http://engineering.twitter.com/2011/04/twitter-search-is-now-3x-faster_1656.html
THE “GEEK” SLIDE (I DON’T UNDERSTAND THIS STUFF)
> Ruby side
Mongrel
Kiji (generational garbage collector)
Starling (light-weight persistent queue server speaking MemCache
   protocol)


> Java side
Modified Lucene (real-time version of the java database)
Optimized JSON fragment cache (to handle initial page load and
   every call from the client)
Mustache (for the rendering – client and server side)
oEmbed (standard to handle the inline media)




                              Source:

                              http://engineering.twitter.com/2010/09/tech-behind-new-twittercom.htm

                              http://engineering.twitter.com/2010/10/twitters-new-search-architecture.html

                              http://engineering.twitter.com/2011/04/twitter-search-is-now-3x-faster_1656.html

                              http://www.infoq.com/news/2012/11/twitter-ruby-to-java
HOW TO TERMINATE THIS SLIDE SET?
Sorry - This was not so much about Ruby
Yellowpages.com did exactly the contrary in 2008, moving from Java on Rails to Ruby on Rails


The “New Twitter” is fantastic – and survived the US Election end 2012. Some impressive numbers:
    327,452 tweets/minute, 31 M election-related tweets during the day, with a 15,107 tweets/second
    peak)


Back to June 2009 (copy/paste from Scaling Twitter: Making Twitter 10000 Percent Faster)


      Twitter started as a side project and blew up fast, going from 0 to millions of page views within
      a few terrifying months. Early design decisions that worked well in the small melted under the
      crush of new users chirping tweets to all their friends. Web darling Ruby on Rails was fingered
      early for the scaling problems, but Blaine Cook, Twitter's lead architect, held Ruby blameless:
      For us, it’s really about scaling horizontally - to that end, Rails and Ruby haven’t been
      stumbling blocks, compared to any other language or framework. The performance boosts
      associated with a “faster” language would give us a 10-20% improvement, but thanks to
      architectural changes that Ruby and Rails happily accommodated, Twitter is 10000% faster
      than it was in January.
      If Ruby on Rails wasn't to blame, how did Twitter learn to scale ever higher and higher?



                                                Source:

                                                http://techcrunch.com/2008/05/01/twitter-said-to-be-abandoning-ruby-
                                                        on-rails/

                                                http://highscalability.com/scaling-twitter-making-twitter-10000-percent-
                                                       faster

                                                http://www.infoq.com/news/2012/11/twitter-ruby-to-java

More Related Content

Similar to Ruby and Twitter at the Ruby drink-up of Sophia, January 2013

At&T Interactive: The Many Facets Of Ruby
At&T Interactive: The Many Facets Of RubyAt&T Interactive: The Many Facets Of Ruby
At&T Interactive: The Many Facets Of Ruby
Coby Randquist
 
Ruby, Rails, and the Open Source Community
Ruby, Rails, and the Open Source CommunityRuby, Rails, and the Open Source Community
Ruby, Rails, and the Open Source Community
Jim Myhrberg
 
DiUS Computing Lca Rails Final
DiUS  Computing Lca Rails FinalDiUS  Computing Lca Rails Final
DiUS Computing Lca Rails Final
Robert Postill
 
Rails Concept
Rails ConceptRails Concept
Rails Concept
Javed Hussain
 
Web 2.0 Application development with Ruby on Rails
Web 2.0 Application development with Ruby on RailsWeb 2.0 Application development with Ruby on Rails
Web 2.0 Application development with Ruby on Rails
Amit Mathur
 
Ror Seminar With agilebd.org on 23 Jan09
Ror Seminar With agilebd.org on 23 Jan09Ror Seminar With agilebd.org on 23 Jan09
Ror Seminar With agilebd.org on 23 Jan09
Shaer Hassan
 
Ruby Kaigi09 China Rubyupdate20090718
Ruby Kaigi09 China Rubyupdate20090718Ruby Kaigi09 China Rubyupdate20090718
Ruby Kaigi09 China Rubyupdate20090718
tengu
 
Ruby On Rails Presentation
Ruby On Rails PresentationRuby On Rails Presentation
Ruby On Rails Presentation
Paul Pajo
 
Viridians on Rails
Viridians on RailsViridians on Rails
Viridians on Rails
Viridians
 
Ruby Rails Web Development.pdf
Ruby Rails Web Development.pdfRuby Rails Web Development.pdf
Ruby Rails Web Development.pdf
Ayesha Siddika
 
Things you must know on ruby on rails single page application
Things you must know on ruby on rails single page applicationThings you must know on ruby on rails single page application
Things you must know on ruby on rails single page application
Andolasoft Inc
 
Massively maintained accessibility: WordPress
Massively maintained accessibility: WordPressMassively maintained accessibility: WordPress
Massively maintained accessibility: WordPress
Joseph Dolson
 
From Java to Ruby...and Back
From Java to Ruby...and BackFrom Java to Ruby...and Back
From Java to Ruby...and Back
Anil Hemrajani
 
Ruby Rails Web Development SEO Expert Bangladesh LTD.pdf
Ruby Rails Web Development  SEO Expert Bangladesh LTD.pdfRuby Rails Web Development  SEO Expert Bangladesh LTD.pdf
Ruby Rails Web Development SEO Expert Bangladesh LTD.pdf
Tasnim Jahan
 
The story of language development
The story of language developmentThe story of language development
The story of language development
Hiroshi SHIBATA
 
Introduction To Rails
Introduction To RailsIntroduction To Rails
Introduction To Rails
Eric Gruber
 
Enterprise Architectures with Ruby (and Rails)
Enterprise Architectures with Ruby (and Rails)Enterprise Architectures with Ruby (and Rails)
Enterprise Architectures with Ruby (and Rails)
Konstantin Gredeskoul
 
Building iOS Apps With RubyMotion
Building iOS Apps With RubyMotionBuilding iOS Apps With RubyMotion
Building iOS Apps With RubyMotion
Raymond T Hightower
 
IronRuby for the Rubyist
IronRuby for the RubyistIronRuby for the Rubyist
IronRuby for the Rubyist
Will Green
 
Ruby on Rails best resources for self
Ruby on Rails best resources for selfRuby on Rails best resources for self
Ruby on Rails best resources for self
Durga Prasad Tumu
 

Similar to Ruby and Twitter at the Ruby drink-up of Sophia, January 2013 (20)

At&T Interactive: The Many Facets Of Ruby
At&T Interactive: The Many Facets Of RubyAt&T Interactive: The Many Facets Of Ruby
At&T Interactive: The Many Facets Of Ruby
 
Ruby, Rails, and the Open Source Community
Ruby, Rails, and the Open Source CommunityRuby, Rails, and the Open Source Community
Ruby, Rails, and the Open Source Community
 
DiUS Computing Lca Rails Final
DiUS  Computing Lca Rails FinalDiUS  Computing Lca Rails Final
DiUS Computing Lca Rails Final
 
Rails Concept
Rails ConceptRails Concept
Rails Concept
 
Web 2.0 Application development with Ruby on Rails
Web 2.0 Application development with Ruby on RailsWeb 2.0 Application development with Ruby on Rails
Web 2.0 Application development with Ruby on Rails
 
Ror Seminar With agilebd.org on 23 Jan09
Ror Seminar With agilebd.org on 23 Jan09Ror Seminar With agilebd.org on 23 Jan09
Ror Seminar With agilebd.org on 23 Jan09
 
Ruby Kaigi09 China Rubyupdate20090718
Ruby Kaigi09 China Rubyupdate20090718Ruby Kaigi09 China Rubyupdate20090718
Ruby Kaigi09 China Rubyupdate20090718
 
Ruby On Rails Presentation
Ruby On Rails PresentationRuby On Rails Presentation
Ruby On Rails Presentation
 
Viridians on Rails
Viridians on RailsViridians on Rails
Viridians on Rails
 
Ruby Rails Web Development.pdf
Ruby Rails Web Development.pdfRuby Rails Web Development.pdf
Ruby Rails Web Development.pdf
 
Things you must know on ruby on rails single page application
Things you must know on ruby on rails single page applicationThings you must know on ruby on rails single page application
Things you must know on ruby on rails single page application
 
Massively maintained accessibility: WordPress
Massively maintained accessibility: WordPressMassively maintained accessibility: WordPress
Massively maintained accessibility: WordPress
 
From Java to Ruby...and Back
From Java to Ruby...and BackFrom Java to Ruby...and Back
From Java to Ruby...and Back
 
Ruby Rails Web Development SEO Expert Bangladesh LTD.pdf
Ruby Rails Web Development  SEO Expert Bangladesh LTD.pdfRuby Rails Web Development  SEO Expert Bangladesh LTD.pdf
Ruby Rails Web Development SEO Expert Bangladesh LTD.pdf
 
The story of language development
The story of language developmentThe story of language development
The story of language development
 
Introduction To Rails
Introduction To RailsIntroduction To Rails
Introduction To Rails
 
Enterprise Architectures with Ruby (and Rails)
Enterprise Architectures with Ruby (and Rails)Enterprise Architectures with Ruby (and Rails)
Enterprise Architectures with Ruby (and Rails)
 
Building iOS Apps With RubyMotion
Building iOS Apps With RubyMotionBuilding iOS Apps With RubyMotion
Building iOS Apps With RubyMotion
 
IronRuby for the Rubyist
IronRuby for the RubyistIronRuby for the Rubyist
IronRuby for the Rubyist
 
Ruby on Rails best resources for self
Ruby on Rails best resources for selfRuby on Rails best resources for self
Ruby on Rails best resources for self
 

Recently uploaded

EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
Jimmy Lai
 
How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...
DianaGray10
 
Keynote : AI & Future Of Offensive Security
Keynote : AI & Future Of Offensive SecurityKeynote : AI & Future Of Offensive Security
Keynote : AI & Future Of Offensive Security
Priyanka Aash
 
LeadMagnet IQ Review: Unlock the Secret to Effortless Traffic and Leads.pdf
LeadMagnet IQ Review:  Unlock the Secret to Effortless Traffic and Leads.pdfLeadMagnet IQ Review:  Unlock the Secret to Effortless Traffic and Leads.pdf
LeadMagnet IQ Review: Unlock the Secret to Effortless Traffic and Leads.pdf
SelfMade bd
 
The Path to General-Purpose Robots - Coatue
The Path to General-Purpose Robots - CoatueThe Path to General-Purpose Robots - Coatue
The Path to General-Purpose Robots - Coatue
Razin Mustafiz
 
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
OnBoard
 
Accelerating Migrations = Recommendations
Accelerating Migrations = RecommendationsAccelerating Migrations = Recommendations
Accelerating Migrations = Recommendations
isBullShit
 
Retrieval Augmented Generation Evaluation with Ragas
Retrieval Augmented Generation Evaluation with RagasRetrieval Augmented Generation Evaluation with Ragas
Retrieval Augmented Generation Evaluation with Ragas
Zilliz
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
alexjohnson7307
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
Brian Pichman
 
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
alexjohnson7307
 
Mastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for SuccessMastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for Success
David Wilson
 
Intel Unveils Core Ultra 200V Lunar chip .pdf
Intel Unveils Core Ultra 200V Lunar chip .pdfIntel Unveils Core Ultra 200V Lunar chip .pdf
Intel Unveils Core Ultra 200V Lunar chip .pdf
Tech Guru
 
Vulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive OverviewVulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive Overview
Steven Carlson
 
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
DianaGray10
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
shanihomely
 
Semantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software DevelopmentSemantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software Development
Baishakhi Ray
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
Matthias Neugebauer
 
Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
Google Developer Group - Harare
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Nicolás Lopéz
 

Recently uploaded (20)

EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
 
How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...
 
Keynote : AI & Future Of Offensive Security
Keynote : AI & Future Of Offensive SecurityKeynote : AI & Future Of Offensive Security
Keynote : AI & Future Of Offensive Security
 
LeadMagnet IQ Review: Unlock the Secret to Effortless Traffic and Leads.pdf
LeadMagnet IQ Review:  Unlock the Secret to Effortless Traffic and Leads.pdfLeadMagnet IQ Review:  Unlock the Secret to Effortless Traffic and Leads.pdf
LeadMagnet IQ Review: Unlock the Secret to Effortless Traffic and Leads.pdf
 
The Path to General-Purpose Robots - Coatue
The Path to General-Purpose Robots - CoatueThe Path to General-Purpose Robots - Coatue
The Path to General-Purpose Robots - Coatue
 
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
 
Accelerating Migrations = Recommendations
Accelerating Migrations = RecommendationsAccelerating Migrations = Recommendations
Accelerating Migrations = Recommendations
 
Retrieval Augmented Generation Evaluation with Ragas
Retrieval Augmented Generation Evaluation with RagasRetrieval Augmented Generation Evaluation with Ragas
Retrieval Augmented Generation Evaluation with Ragas
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
 
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
 
Mastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for SuccessMastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for Success
 
Intel Unveils Core Ultra 200V Lunar chip .pdf
Intel Unveils Core Ultra 200V Lunar chip .pdfIntel Unveils Core Ultra 200V Lunar chip .pdf
Intel Unveils Core Ultra 200V Lunar chip .pdf
 
Vulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive OverviewVulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive Overview
 
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
 
Semantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software DevelopmentSemantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software Development
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
 
Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
 

Ruby and Twitter at the Ruby drink-up of Sophia, January 2013

  • 1. Ruby Drink-up / January 2013 http://rivierarb.fr/2013/01/08/Drinkup/ Karan Malkani
  • 2. ABOUT THESE SLIDES… A Presentation of the Twitter / Ruby couple and why it did not last Motivated by my own interest in Ruby, born 2 years ago when reading that Twitter was made using Ruby (or Ruby on Rails) An Internet-based desk research But… Not so much / easy available information due to the excellent Twitter communication
  • 3. TWITTER HISTORY WITH (A BIG) SHORTCUT  Born in the postcasting company Odeo  Initial idea: an SMS service within a small group / Original code name: twttr / First prototype was used as an internal service for Odeo employees / Full version introduced publicly in July 2006  October 2006: Jack Dorsey, Biz Stonen, and Evan Williams formed Obvious Corporation, acquired Odeo (including twitter.com).  Twitter, as a company, was launched in April 2007  Today: the 2nd social network in the world after Facebook: (Estimated Unique Monthly Visitors – January 2013) • 1 Facebook - 750,000,000 • 2 Twitter - 250,000,000 • 3 LinkedIn - 110,000,000 • 4 Pinterest - 85,500,000 • 5 MySpace - 70,500,000 • 6 Google Plus+ - 65,000,000 Source: How Twitter Was Born http://www.140characters.com/2009/01/30/how-twitter-was-born/ Twitter Development History in 3 minute and 53 seconds! http://www.shoutmeloud.com/twitter-development-history-in-3- minute-and-53-seconds.html Wikipedia: Twitter http://en.wikipedia.org/wiki/Twitter eBizMBA (http://www.ebizmba.com/articles/social-networking-websites)
  • 4. A FANTASTIC GROWTH – (> 100M NEW USERS IN 2010) Source: http://blog.twitter.com/2010/02/measuring-tweets.html http://mashable.com/2010/12/16/twitter-stats-2010/
  • 5. THE PROBLEM Source: http://www.whatisfailwhale.info/ http://en.wikipedia.org/wiki/Twitter#Outages http://www.wired.com/underwire/2008/07/twitter-fans-tu/
  • 6. WHY SCALING? Number of users Number of messages (impacted by the Retweet feature realeased end 2009) Applications that can publish tweets / Applications that access tweets Search Engine (Twitter acquired Summize – July 2008) 2009 : Some critical outages / Many FailWhales experiences June 12, 2009 – In what was called a potential "Twitpocalypse", the unique numerical identifier associated with each tweet exceeded the limit of 32-bit signedintegers (2,147,483,647 total messages).[142] While Twitter itself was not affected, some third-party clients could no longer access recent tweets. Patcheswere quickly released, though some iPhone applications had to wait for approval from the App Store.[143] June 25, 2009 – Twitter ran slowly for some time after over 50,000 tweets on Michael Jackson’s death were recorded in an hour.[144] August 6, 2009 – Twitter and Facebook suffered from a denial-of-service attack, causing the Twitter website to go offline for several hours.[145] It was later confirmed that the attacks were directed at one pro-Georgian user around the anniversary of the 2008 South Ossetia War, rather than the sites themselves.[146] September 22, 2009 – The identifier exceeded the limit for 32-bit unsigned integers (4,294,967,296 total messages) again breaking some third-party clients.[147] Source: http://blog.twitter.com/2009/11/retweet-limited-rollout.html http://techcrunch.com/2008/07/15/confirmed-twitter-acquires-summize-search- engine/ http://en.wikipedia.org/wiki/Twitter#Outages http://www.businessinsider.com/twitter-doesnt-have-product-geniuses-so-its- buying-them-2010-4
  • 7. STARTING TWITTER WITH RUBY An excellent choice to  Quickly prototype  Quickly launch a first real site  Quickly deploy an API for third-party applications  Tease and attract developers Improving performances with Ruby: Example - mid 2009: Evan Weaver (Twitter) We recently migrated Twitter from a custom Ruby 1.8.6 build to a Ruby Enterprise Edition release candidate, courtesy of Phusion. Our primary motivation was the integration of Brent’s MBARI patches, which increase memory stability. Some features of REE have no effect on our codebase, but we definitely benefit from the MBARI patchset, the Railsbench tunable GC, and the various leak fixes in 1.8.7p174. These are difficult to integrate and Phusion has done a fine job. Source: http://blog.evanweaver.com/2009/09/24/ree/ http://www.linkedin.com/in/evanweaver/
  • 8. RUBY ISSUES KNOWN WHEN TWITTER WAS FOUNDED FULL COPY/PASTE OF: HOW HAS RUBY ON RAILS BEEN HOLDING UP TO THE INCREASED LOAD? (5 QUESTION INTERVIEW WITH TWITTER DEVELOPER ALEX PAYNE – APRIL 2007) By various metrics Twitter is the biggest Rails site on the net right All the convenience methods and syntactical sugar that makes now. Rails such a pleasure for coders ends up being absolutely punishing, performance-wise. Running on Rails has forced us to deal with scaling issues - issues that any growing site eventually contends with – far sooner than I Once you hit a certain threshold of traffic, either you need to think we would on another framework. strip out all the costly neat stuff that Rails does for you (RJS, ActiveRecord, ActiveSupport, etc.) or move the slow parts of The common wisdom in the Rails community at this time is that your application out of Rails, or both. scaling Rails is a matter of cost: just throw more CPUs at it. It’s also worth mentioning that there shouldn’t be doubt in The problem is that more instances of Rails (running as part of a anybody’s mind at this point that Ruby itself is slow. Mongrel cluster, in our case) means more requests to your database. It’s great that people are hard at work on faster implementations At this point in time there’s no facility in Rails to talk to more than one of the language, but right now, it’s tough. If you’re looking to database at a time. deploy a big web application and you’re language-agnostic, The solutions to this are caching the hell out of everything and realize that the same operation in Ruby will take less time in setting up multiple read-only slave databases, neither of which are Python. quick fixes to implement. All of us working on Twitter are big Ruby fans, but I think it’s So it’s not just cost, it’s time, and time is that much more precious worth being frank that this isn’t one of those relativistic language when people can[‘t] reach your site.None of these scaling issues. Ruby is slow. approaches are as fun and easy as developing for Rails. Source: http://yasulab.tumblr.com/post/10271634919/5-question-interview-with- twitter-developer-alex-payne/ The original article on Radical Behavior is no longer accessible
  • 9. ABANDONNING RUBY ON RAILS April 2008 - After 2 years of high scaling problems: decision is made to abandon RoR Start to change the infrastructure: Middleware moved to a mixture of C and Scala/JVM to run a messaging model, asynchronous process, 3 levels of cache (Twitter kept Ruby for the front-end only) April 2009 - Twitter on Scala Spring 2010 – changing the back-end from MySQL to a real-time version of Lucene (Apache database) 2011 – RoR front-end for search replaced with Blender, a Java server Source: http://techcrunch.com/2008/05/01/twitter-said-to-be-abandoning-ruby-on-rails/ http://www.artima.com/scalazine/articles/twitter_on_scala.html http://engineering.twitter.com/2011/04/twitter-search-is-now-3x-faster_1656.html
  • 10. THE “GEEK” SLIDE (I DON’T UNDERSTAND THIS STUFF) > Ruby side Mongrel Kiji (generational garbage collector) Starling (light-weight persistent queue server speaking MemCache protocol) > Java side Modified Lucene (real-time version of the java database) Optimized JSON fragment cache (to handle initial page load and every call from the client) Mustache (for the rendering – client and server side) oEmbed (standard to handle the inline media) Source: http://engineering.twitter.com/2010/09/tech-behind-new-twittercom.htm http://engineering.twitter.com/2010/10/twitters-new-search-architecture.html http://engineering.twitter.com/2011/04/twitter-search-is-now-3x-faster_1656.html http://www.infoq.com/news/2012/11/twitter-ruby-to-java
  • 11. HOW TO TERMINATE THIS SLIDE SET? Sorry - This was not so much about Ruby Yellowpages.com did exactly the contrary in 2008, moving from Java on Rails to Ruby on Rails The “New Twitter” is fantastic – and survived the US Election end 2012. Some impressive numbers: 327,452 tweets/minute, 31 M election-related tweets during the day, with a 15,107 tweets/second peak) Back to June 2009 (copy/paste from Scaling Twitter: Making Twitter 10000 Percent Faster) Twitter started as a side project and blew up fast, going from 0 to millions of page views within a few terrifying months. Early design decisions that worked well in the small melted under the crush of new users chirping tweets to all their friends. Web darling Ruby on Rails was fingered early for the scaling problems, but Blaine Cook, Twitter's lead architect, held Ruby blameless: For us, it’s really about scaling horizontally - to that end, Rails and Ruby haven’t been stumbling blocks, compared to any other language or framework. The performance boosts associated with a “faster” language would give us a 10-20% improvement, but thanks to architectural changes that Ruby and Rails happily accommodated, Twitter is 10000% faster than it was in January. If Ruby on Rails wasn't to blame, how did Twitter learn to scale ever higher and higher? Source: http://techcrunch.com/2008/05/01/twitter-said-to-be-abandoning-ruby- on-rails/ http://highscalability.com/scaling-twitter-making-twitter-10000-percent- faster http://www.infoq.com/news/2012/11/twitter-ruby-to-java