Facebook Technology Stack


Published on

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Dark launches are usually done two weeks before the actual launch. (without users noticing since there will be no corresponding UI elements). This acts as a real-world stress test and helps expose bottlenecks and other problem areas before a feature is officially launched.
  • The social graph is very important because it helps establish the relationships between people and now between people and the world around them. social graph" as "the global mapping of everybody and how they're related".
  • While logged in, users can connect with friends via these media and post information and updates to their Facebook profile. Developers can use these services to help their users connect and share with their Facebook friends on and off of Facebook and increase engagement for their website or application.
  • Facebook Technology Stack

    1. 1. FACEBOOK
    2. 2. Introduction <ul><li>Facebook  is  the “ social networking ”. </li></ul><ul><li>People have been “facebooking” each other for about 7 years now, making Facebook  the most used social network with over 500 million users worldwide. </li></ul><ul><li>50% of our active users log on to Facebook in any given day </li></ul><ul><li>Average user has 130 friends </li></ul><ul><li>People spend over 700 billion minutes per month on Facebook </li></ul><ul><li>There are over 900 million objects that people interact with (pages, groups, events and community pages) </li></ul><ul><li>Average user is connected to 80 community pages, groups and events </li></ul><ul><li>Average user creates 90 pieces of content each month </li></ul><ul><li>More than 30 billion pieces of content (web links, news stories, blog posts, notes, photo albums, etc.) shared each month. </li></ul>http://www.facebook.com/press/info.php?statistics
    3. 3. Facebook’s scaling challenge <ul><li>Here are a few factoids to give you an idea of the scaling challenge that Facebook has to deal with: </li></ul><ul><li>Facebook serves  570 billion page views per month  (according to Google Ad Planner). </li></ul><ul><li>There are more photos on Facebook than all other photo sites combined (including sites like Flickr). </li></ul><ul><li>More than  3 billion photos  are uploaded every month. </li></ul><ul><li>Facebook’s systems serve  1.2 million photos per second . This doesn’t include the images served by Facebook’s CDN. </li></ul><ul><li>More than  25 billion pieces of content  (status updates, comments, etc) are shared every month. </li></ul><ul><li>Facebook has more than  30,000 servers  (and this number is from last year!) </li></ul>All Data 2009
    4. 4. How Does Facebook Work? The Nuts and Bolts <ul><li>Linux & Apache </li></ul><ul><li>PHP </li></ul><ul><li>Memcache </li></ul><ul><li>Haystack </li></ul><ul><li>BigPipe </li></ul>The Front End
    5. 5. LAMP stack <ul><li>Facebook uses Linux, but has optimized it for its own purposes (especially in terms of network throughput). </li></ul><ul><li>Facebook still uses PHP, but it has built a compiler(HIP HOP) for it so it can be turned into native code on its web servers, thus boosting performance. </li></ul>
    6. 6. Haystack <ul><li>There are more than 20 billion uploaded photos on Facebook, and each one is saved in four different resolutions, resulting in more than 80 billion photos. </li></ul><ul><li>And it’s not just about being able to handle billions of photos, performance is critical. Facebook serves around 1.2 million photos per second. </li></ul><ul><li>Haystack  is Facebook’s high-performance photo storage/retrieval system, a highly scalable object store used to serve Facebook’s immense amount of photos. </li></ul><ul><li>Strictly speaking, Haystack is an object store, so it doesn’t necessarily have to store photos. </li></ul>http://www.niallkennedy.com/blog/2009/04/facebook-haystack.html
    7. 7. Haystack <ul><li>Haystack stores photo data inside 10 GB bucket with 1 MB of metadata for every GB stored. </li></ul><ul><li>The Haystack index stores metadata about the one needle it needs to find within the Haystack. </li></ul><ul><li>Incoming requests for a given photo asset are interpreted as before, but now contain a direct reference to the storage offset containing the appropriate data. </li></ul>
    8. 8. BigPipe <ul><li>Pipelining web pages for high performance </li></ul><ul><li>BigPipe  -dynamic web page serving system, Facebook has developed. </li></ul><ul><li>Facebook uses it to serve each web page in sections (called “pagelets”) for optimal performance. </li></ul><ul><li>BigPipe is a fundamental redesign of the dynamic web page serving system. The general idea is to pipeline pagelets through several execution stages inside web servers and browsers. </li></ul>
    9. 9. BigPipe <ul><li>BigPipe breaks the page generation process into several stages </li></ul><ul><li>The first three stages are executed by the web server, and the last four stages are executed by the browser. </li></ul><ul><li>Each pagelet must go through all these stages sequentially, but BigPipe enables several pagelets to be executed simultaneously in different stages. </li></ul>
    10. 10. Pagelets in Facebook home page. Each rectangle corresponds to one pagelet.
    11. 11. Memcache <ul><li>Free & open source, high-performance, distributed memory object caching system </li></ul><ul><li>Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering. </li></ul><ul><li>The system uses a client–server architecture. The servers maintain a key–value associative array; the clients populate this array and query it. </li></ul><ul><li>The servers keep the values in RAM; if a server runs out of RAM, it discards the oldest values. </li></ul><ul><li>Clients can read each other's cached data. </li></ul>
    12. 12. How Does Facebook Work ? <ul><li>Thrift (protocol) </li></ul><ul><li>Cassandra (database) </li></ul><ul><li>Scribe (log server) </li></ul><ul><li>HipHop for PHP </li></ul>The Back End
    13. 13. Thrift <ul><li>Facebook’s backend services are written in a variety of different programming languages including C++, Java, Python, and Erlang. </li></ul><ul><li>Thrift  allows for easy exchange of data (variables, objects) between applications written in different languages. </li></ul><ul><li>Thrift protocol offers cross language serialization. It combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, C#, Java, Perl, Python, PHP, Erlang, Haskel, Cocoa and Ruby . </li></ul>http://www.thrift.pl/Thrift-tutorial-how-it-works.html
    14. 14. The Back End <ul><li>Scribe   (log server) is a server for aggregating log data streamed in real-time from many other servers. Useful for logging a wide array of data. It is built on top of Thrift. </li></ul><ul><li>Cassandra  is a database management system designed to handle large amounts of data spread out across many servers. It powers Facebook’s Inbox Search feature and provides a structured key-value store with eventual consistency. </li></ul><ul><li>HipHop for PHP  is a source code transformer for PHP script code and was created to save server resources. HipHop transforms PHP source code into optimized C++. After doing this, it uses g++ to compile it to machine code. </li></ul>
    15. 15. Reliability <ul><li>Unlike other social networks like Friendster, MySpace, and Twitter – all of whom have run into serious scalability issues at different points during their growth. Facebook has been mostly reliable throughout its rise.  </li></ul><ul><li>In actuality, Facebook uses JavaScript heavily, relies on their own in-house PHP wrapper called XHP, HipHop (which optimizes PHP), and many more technologies. </li></ul><ul><li>A lot of technologies have been developed by Facebook in-house to serve their own needs, for example Cassandra. </li></ul>
    16. 16. <ul><li>Facebook has also widgetized large portions of their application, meaning that widgets can be written in an appropriate language instead of simply using PHP. These widgets interface with the other parts of the application through the use of internal APIs. </li></ul><ul><li>Like many other big sites, Facebook uses a Content delivery network (CDN) to help serve static content. </li></ul><ul><li>And then of course there is  the huge data center Facebook is building in Oregon to help it scale out with even more servers. </li></ul>
    17. 17. Facebook’s love affair with open source <ul><li>Not only is Facebook using (and contributing to) open source software such as Linux, Memcached, MySQL, Hadoop, Hive, and many others, it has also made much of its internally developed software available as open source. </li></ul><ul><li>Examples of open source projects that originated from inside Facebook include HipHop, Cassandra, Thrift, Cfengine, Varnish and Scribe. </li></ul><ul><li>Facebook has also open-sourced Tornado , a high-performance web server framework developed by the team behind FriendFeed (which Facebook bought in August 2009). </li></ul><ul><li>Check out - http://developers.facebook.com/opensource/ </li></ul>
    18. 18. Gradual releases and dark launches <ul><li>Facebook has a system, Gatekeeper that lets run different code for different sets of users. </li></ul><ul><li>This lets Facebook do gradual releases of new features, activate certain features only for Facebook employees, etc. </li></ul><ul><li>Gatekeeper also lets Facebook do something called “ dark launches ”, which is to activate elements of a certain feature behind the scenes before it goes live. </li></ul>
    19. 19. Facebook Platform <ul><li>The  Facebook Platform  provides a set of APIs and tools which enable 3rd party developers to integrate with the &quot;open graph“. </li></ul><ul><li>Graph API is the core of Facebook Platform, enabling developers to read and write data to Facebook. </li></ul>
    20. 20. The Graph API <ul><li>The Graph API presents a simple, consistent view of the Facebook social graph, uniformly representing objects in the graph (e.g., people ,  photos ,  events , and  pages ) and the connections between them (e.g., friend relationships, shared content, and photo tags). </li></ul><ul><li>Restful API for accessing data on the Facebook graph. </li></ul><ul><li>O auth 2.0 based authentication. </li></ul><ul><li>JSON Modeling of objects and connections. </li></ul><ul><li>Every object in the social graph has a unique ID. You can access the properties of an object by requesting - https://graph.facebook.com/ID </li></ul><ul><li>Alternatively, people and pages with usernames can be accessed using their username as an ID. All responses are JSON objects. </li></ul><ul><li>Specifications - http://developers.facebook.com/docs/api </li></ul>
    21. 21. Facebook Markup Language <ul><li>FBML is a variant-evolved subset of HTML with some elements removed. </li></ul><ul><li>It allows Facebook Application developers to customize the &quot;look and feel&quot; of their applications, to a limited extent. </li></ul><ul><li>It is the specification of how to encode content so that Facebook's  servers can read and publish it. </li></ul><ul><li>FBML plays an important role in building applications. FBML is used to tap in to various Facebook elements when building applications. </li></ul><ul><li>It operates a lot like HTML and it gives the ability to do various tasks with ease such as: </li></ul><ul><ul><li>sending a user e-mail </li></ul></ul><ul><ul><li>creating a two column form </li></ul></ul><ul><ul><li>embedding flash video </li></ul></ul><ul><ul><li>creating a dashboard </li></ul></ul><ul><ul><li>posting on a wall </li></ul></ul><ul><ul><li>displaying a header…etc </li></ul></ul>
    22. 22. FBML <ul><li>Facebook also allows the use of regular HTML tags, such as <a href=”#”></a>, which is used to generate a hyperlink. Facebook also allows the use of many more HTML tags for building applications </li></ul>http://developers.facebook.com/docs/reference/fbml/
    23. 23. Facebook features Facebook features Chat Status Updates Credits URL shortener Facebook Live Usernames Networks, Groups, and Like Pages Easter eggs News Feed Wall Notification Gifts Poke Lite
    24. 24. Facebook’s New Messages <ul><li>The new Messages interweaves your chats, texts and emails. It’s a central place to control all of your private communication, both on and off Facebook. </li></ul><ul><li>Simply put, it can be a single inbox for all of your messages, no matter how you choose to send them. </li></ul><ul><li>A facebook.com Email Address </li></ul><ul><li>SMS From Facebook </li></ul><ul><li>Chat History </li></ul>
    25. 25. Facebook Connect <ul><li>Facebook Connect is a set of  APIs  from Facebook that enable Facebook members to log onto third-party websites, applications, mobile devices and gaming systems with their Facebook identity. </li></ul>