Successfully reported this slideshow.

Ruby and Twitter at the Ruby drink-up of Sophia, January 2013

0

Share

Upcoming SlideShare
Whats New In Roller5
Whats New In Roller5
Loading in …3
×
1 of 11
1 of 11

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Ruby and Twitter at the Ruby drink-up of Sophia, January 2013

  1. 1. Ruby Drink-up / January 2013 http://rivierarb.fr/2013/01/08/Drinkup/ Karan Malkani
  2. 2. ABOUT THESE SLIDES… A Presentation of the Twitter / Ruby couple and why it did not last Motivated by my own interest in Ruby, born 2 years ago when reading that Twitter was made using Ruby (or Ruby on Rails) An Internet-based desk research But… Not so much / easy available information due to the excellent Twitter communication
  3. 3. TWITTER HISTORY WITH (A BIG) SHORTCUT  Born in the postcasting company Odeo  Initial idea: an SMS service within a small group / Original code name: twttr / First prototype was used as an internal service for Odeo employees / Full version introduced publicly in July 2006  October 2006: Jack Dorsey, Biz Stonen, and Evan Williams formed Obvious Corporation, acquired Odeo (including twitter.com).  Twitter, as a company, was launched in April 2007  Today: the 2nd social network in the world after Facebook: (Estimated Unique Monthly Visitors – January 2013) • 1 Facebook - 750,000,000 • 2 Twitter - 250,000,000 • 3 LinkedIn - 110,000,000 • 4 Pinterest - 85,500,000 • 5 MySpace - 70,500,000 • 6 Google Plus+ - 65,000,000 Source: How Twitter Was Born http://www.140characters.com/2009/01/30/how-twitter-was-born/ Twitter Development History in 3 minute and 53 seconds! http://www.shoutmeloud.com/twitter-development-history-in-3- minute-and-53-seconds.html Wikipedia: Twitter http://en.wikipedia.org/wiki/Twitter eBizMBA (http://www.ebizmba.com/articles/social-networking-websites)
  4. 4. A FANTASTIC GROWTH – (> 100M NEW USERS IN 2010) Source: http://blog.twitter.com/2010/02/measuring-tweets.html http://mashable.com/2010/12/16/twitter-stats-2010/
  5. 5. THE PROBLEM Source: http://www.whatisfailwhale.info/ http://en.wikipedia.org/wiki/Twitter#Outages http://www.wired.com/underwire/2008/07/twitter-fans-tu/
  6. 6. WHY SCALING? Number of users Number of messages (impacted by the Retweet feature realeased end 2009) Applications that can publish tweets / Applications that access tweets Search Engine (Twitter acquired Summize – July 2008) 2009 : Some critical outages / Many FailWhales experiences June 12, 2009 – In what was called a potential "Twitpocalypse", the unique numerical identifier associated with each tweet exceeded the limit of 32-bit signedintegers (2,147,483,647 total messages).[142] While Twitter itself was not affected, some third-party clients could no longer access recent tweets. Patcheswere quickly released, though some iPhone applications had to wait for approval from the App Store.[143] June 25, 2009 – Twitter ran slowly for some time after over 50,000 tweets on Michael Jackson’s death were recorded in an hour.[144] August 6, 2009 – Twitter and Facebook suffered from a denial-of-service attack, causing the Twitter website to go offline for several hours.[145] It was later confirmed that the attacks were directed at one pro-Georgian user around the anniversary of the 2008 South Ossetia War, rather than the sites themselves.[146] September 22, 2009 – The identifier exceeded the limit for 32-bit unsigned integers (4,294,967,296 total messages) again breaking some third-party clients.[147] Source: http://blog.twitter.com/2009/11/retweet-limited-rollout.html http://techcrunch.com/2008/07/15/confirmed-twitter-acquires-summize-search- engine/ http://en.wikipedia.org/wiki/Twitter#Outages http://www.businessinsider.com/twitter-doesnt-have-product-geniuses-so-its- buying-them-2010-4
  7. 7. STARTING TWITTER WITH RUBY An excellent choice to  Quickly prototype  Quickly launch a first real site  Quickly deploy an API for third-party applications  Tease and attract developers Improving performances with Ruby: Example - mid 2009: Evan Weaver (Twitter) We recently migrated Twitter from a custom Ruby 1.8.6 build to a Ruby Enterprise Edition release candidate, courtesy of Phusion. Our primary motivation was the integration of Brent’s MBARI patches, which increase memory stability. Some features of REE have no effect on our codebase, but we definitely benefit from the MBARI patchset, the Railsbench tunable GC, and the various leak fixes in 1.8.7p174. These are difficult to integrate and Phusion has done a fine job. Source: http://blog.evanweaver.com/2009/09/24/ree/ http://www.linkedin.com/in/evanweaver/
  8. 8. RUBY ISSUES KNOWN WHEN TWITTER WAS FOUNDED FULL COPY/PASTE OF: HOW HAS RUBY ON RAILS BEEN HOLDING UP TO THE INCREASED LOAD? (5 QUESTION INTERVIEW WITH TWITTER DEVELOPER ALEX PAYNE – APRIL 2007) By various metrics Twitter is the biggest Rails site on the net right All the convenience methods and syntactical sugar that makes now. Rails such a pleasure for coders ends up being absolutely punishing, performance-wise. Running on Rails has forced us to deal with scaling issues - issues that any growing site eventually contends with – far sooner than I Once you hit a certain threshold of traffic, either you need to think we would on another framework. strip out all the costly neat stuff that Rails does for you (RJS, ActiveRecord, ActiveSupport, etc.) or move the slow parts of The common wisdom in the Rails community at this time is that your application out of Rails, or both. scaling Rails is a matter of cost: just throw more CPUs at it. It’s also worth mentioning that there shouldn’t be doubt in The problem is that more instances of Rails (running as part of a anybody’s mind at this point that Ruby itself is slow. Mongrel cluster, in our case) means more requests to your database. It’s great that people are hard at work on faster implementations At this point in time there’s no facility in Rails to talk to more than one of the language, but right now, it’s tough. If you’re looking to database at a time. deploy a big web application and you’re language-agnostic, The solutions to this are caching the hell out of everything and realize that the same operation in Ruby will take less time in setting up multiple read-only slave databases, neither of which are Python. quick fixes to implement. All of us working on Twitter are big Ruby fans, but I think it’s So it’s not just cost, it’s time, and time is that much more precious worth being frank that this isn’t one of those relativistic language when people can[‘t] reach your site.None of these scaling issues. Ruby is slow. approaches are as fun and easy as developing for Rails. Source: http://yasulab.tumblr.com/post/10271634919/5-question-interview-with- twitter-developer-alex-payne/ The original article on Radical Behavior is no longer accessible
  9. 9. ABANDONNING RUBY ON RAILS April 2008 - After 2 years of high scaling problems: decision is made to abandon RoR Start to change the infrastructure: Middleware moved to a mixture of C and Scala/JVM to run a messaging model, asynchronous process, 3 levels of cache (Twitter kept Ruby for the front-end only) April 2009 - Twitter on Scala Spring 2010 – changing the back-end from MySQL to a real-time version of Lucene (Apache database) 2011 – RoR front-end for search replaced with Blender, a Java server Source: http://techcrunch.com/2008/05/01/twitter-said-to-be-abandoning-ruby-on-rails/ http://www.artima.com/scalazine/articles/twitter_on_scala.html http://engineering.twitter.com/2011/04/twitter-search-is-now-3x-faster_1656.html
  10. 10. THE “GEEK” SLIDE (I DON’T UNDERSTAND THIS STUFF) > Ruby side Mongrel Kiji (generational garbage collector) Starling (light-weight persistent queue server speaking MemCache protocol) > Java side Modified Lucene (real-time version of the java database) Optimized JSON fragment cache (to handle initial page load and every call from the client) Mustache (for the rendering – client and server side) oEmbed (standard to handle the inline media) Source: http://engineering.twitter.com/2010/09/tech-behind-new-twittercom.htm http://engineering.twitter.com/2010/10/twitters-new-search-architecture.html http://engineering.twitter.com/2011/04/twitter-search-is-now-3x-faster_1656.html http://www.infoq.com/news/2012/11/twitter-ruby-to-java
  11. 11. HOW TO TERMINATE THIS SLIDE SET? Sorry - This was not so much about Ruby Yellowpages.com did exactly the contrary in 2008, moving from Java on Rails to Ruby on Rails The “New Twitter” is fantastic – and survived the US Election end 2012. Some impressive numbers: 327,452 tweets/minute, 31 M election-related tweets during the day, with a 15,107 tweets/second peak) Back to June 2009 (copy/paste from Scaling Twitter: Making Twitter 10000 Percent Faster) Twitter started as a side project and blew up fast, going from 0 to millions of page views within a few terrifying months. Early design decisions that worked well in the small melted under the crush of new users chirping tweets to all their friends. Web darling Ruby on Rails was fingered early for the scaling problems, but Blaine Cook, Twitter's lead architect, held Ruby blameless: For us, it’s really about scaling horizontally - to that end, Rails and Ruby haven’t been stumbling blocks, compared to any other language or framework. The performance boosts associated with a “faster” language would give us a 10-20% improvement, but thanks to architectural changes that Ruby and Rails happily accommodated, Twitter is 10000% faster than it was in January. If Ruby on Rails wasn't to blame, how did Twitter learn to scale ever higher and higher? Source: http://techcrunch.com/2008/05/01/twitter-said-to-be-abandoning-ruby- on-rails/ http://highscalability.com/scaling-twitter-making-twitter-10000-percent- faster http://www.infoq.com/news/2012/11/twitter-ruby-to-java

×