Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Technology Stack Behind Airbnb
By Rohan Khude
https://in.linkedin.com/in/rohankhude
http://stackoverflow.com/users/4635388...
Contents

Application and Data

Utilities

Devops

Business Tools

What and Why – help you drive to deeper level of o...
Contents

Application and Data

Utilities

Devops

Business Tools

What and Why – help you drive to deeper level of o...
Technology – what?

What(Summary)

Why Airbnb is using this technology
Application and Data
Nginx – Web Server

nginx [engine x] is an HTTP and reverse proxy server, as well as a
mail proxy server

nginx accelera...

Ruby on Rails – Ruby language on Rails
Framework

It is well known for its quick iteration cycle(Plan-Run-
Review-Chang...
Redis – In Memory Database

in-memory data structure store, used as
database, cache

In short a datastore like MySQL

A...
Amazon S3,EBS – Cloud Storage

Airbnb migrated all of its cloud computing functions to
AWS.

Backup for house backups an...

Amazon EC2 – Cloud Hosting

Airbnb is using Elastic Load Balancing, which
automatically distributes incoming traffic
be...
Amazon EMR

To easily process and analyze 50 Gigabytes of
data daily, Airbnb uses Amazon Elastic
MapReduce (Amazon EMR)

Amazon RDS – Cloud Database

Airbnb moved its main MySQL database to Amazon
Relational Database Service.

Airbnb chose...
Amazon ElastiCache

ElastiCache improves the performance of web
applications by allowing you to retrieve information
fast...
SaSS – Syntactically Awsome Style Sheet

Sass is an extension of CSS3, adding nested
rules, variables, mixins, selector i...
React – Javascript UI Libraries

A declarative, efficient, and flexible JavaScript
library for building user interfaces.
Airflow – Workflow Manager

Airbnb, its own data workflow management
framework

Airflow is being used internally at Airb...
Hadoop - BigDatabase

Open-Source Framework that allows to store and process
big data a distributed environment across cl...
Presto, Druid – Big Data Tools

SQL query tool for hadoop.

Druid is an open-source analytics data store
designed for bu...
Airpal – Big Data Tools
Developed by Airbnb
A web-based query execution tool that leverages Presto to facilitate
data anal...
Utilities
Amazon CloudFront - CDN
A content delivery network (CDN) is a system that deliver webpages
and other Web content to a user...
Sendgrid – Transactional Email

Delivering your transactional and marketing email through
one reliable platform.
MixPanel – Funnel Analysis Analytics
•
It tracks user interactions with web and mobile
applications
•
Funnel analysis help...
Amazon Route 53 – DNS Management

DNS data is typically deployed on multiple physical servers.

The main purposes of DNS...
BrainTree – Payment Services

Subsidiary of Paypal
Twilio – Voice and SMS
•
Twilio offers developers a API for phone services to make and
receive phone calls, and send and r...
Nexmo – Voice and SMS

API for SMS, Voice and Phone Verification.
Urban Airship - Mobile Push Messaging
Improve customer engagement with push notifications, location-
based marketing and a...
Urban Airship - Mobile Push Messaging
Improve customer engagement with push notifications, location-
based marketing and a...

The push notifications services enables international users to receive
notifications regardless of their location or car...
Visual Website Optimizer - A/B Testing Analytics
• VWO is the easiest A/B, Split and Multivariate testing tool.
• To optim...
Aerosolve - A machine learning package built for humans.
Created by airbnb
Airbnb's Price Tips feature helps users figure ...
Fig. Learning to rank images. On the left,
image ordering trained from professional
photographer ratings. On the right, im...
Aerosolve - A machine learning package built for humans.
Seasonal demand
for Austin
DeepLinkDispatch - Deep Linking as a Service
A simple, annotation-based library for making deep link handling
better on A...
DeepLinkDispatch - Deep Linking as a Service
A simple, annotation-based library for making deep link handling
better on A...
DeepLinkDispatch - Deep Linking as a Service
New Relic – Performance Monitor
SaaS Application Performance Management for Ruby, PHP, .Net,
Java, Python, and Node.js App...
Vagrant – Virtual Machine Management

A tool for building and distributing development environments


Vagrant allows yo...
Enzyme – Javascript Testing Framework

Javascript Testing utilities for React, by Airbnb
SmartStack – Service Discovery
•
SmartStack is an automated service discovery and registration
framework.
•
It makes the l...
Scout – Performance Monitoring

Scout is a simple hosted server monitoring service.

Rather than installing and configur...
Sentry - Exception Monitoring

Real-time crash reporting for your web apps, mobile apps, and
games.

Sentry notifies you...
Chef - Server Configuration and Automation
•
Chef enables you to manage and scale cloud infrastructure
with no downtime or...
Chef - Server Configuration and Automation

Chef enables you to manage and scale cloud infrastructure with
no downtime or...
Mocha - Javascript Testing Framework
Simple, flexible, fun javascript test framework for node.js & the
browser.
Mocha test...
Datadog - Performance Monitoring

Datadog is the service for cloud-scale monitoring

Shows Performance monitoring about ...
Logstash - Log Management

Logstash is a tool for managing events and logs.

You can use it to collect logs, parse them,...
kibana - Monitoring Tools
Kibana is an open source, browser based analytics and search
dashboard for Elasticsearch.
Airbnb...
Google Apps - Productivity Suite

Web based email, calendar, storage, file sharing and video
conferencing that let you wo...
Slack - Group Chat & Notifications
Slack brings all your communication together in one place
Asana - Project Management
Asana is the easiest way for teams to track their work.
From tasks and projects to conversation...
Campaign Monitor- Email Marketing
Campaign Monitor makes it easy to attract new subscribers, send
them beautiful email new...
InVision- Interactive Mockups

InVision is a web-based prototyping tool created for designers,
by designers.


It allow...
Assemblage- Visual Collaboration
•
For more efficient projects, teams and meetings.
•
Collaborate directly in your browser...
Caravel- Business Intelligence
•
Data exploration and visualization platform, by Airbnb
•
Caravel's main goal is to make i...
2009
Engineer Count: 3-4
Core - Rails 2.x app
Backend - MySQL Database
Frontend - Vanilla CSS ( no preprocessors), Prototy...
References
www.zdnet.com/article/telstra-invests-in-nginx-web-server/
http://stackshare.io/
https://www.quora.com/What-bac...
References(cont.)
www.invisionapp.com
https://github.com/airbnb/caravel
•Thank You
Technology stack behind Airbnb
Technology stack behind Airbnb
Technology stack behind Airbnb
Technology stack behind Airbnb
Technology stack behind Airbnb
Upcoming SlideShare
Loading in …5
×

Technology stack behind Airbnb

12,116 views

Published on

How Airbnb managed to handle global payments, search & discovery, messaging & communication, trust & safety, user experience, and growth strategies.

Published in: Technology
  • Be the first to comment

Technology stack behind Airbnb

  1. 1. Technology Stack Behind Airbnb By Rohan Khude https://in.linkedin.com/in/rohankhude http://stackoverflow.com/users/4635388/rohan-khude
  2. 2. Contents  Application and Data  Utilities  Devops  Business Tools  What and Why – help you drive to deeper level of observation
  3. 3. Contents  Application and Data  Utilities  Devops  Business Tools  What and Why – help you drive to deeper level of observation
  4. 4. Technology – what?  What(Summary)  Why Airbnb is using this technology
  5. 5. Application and Data
  6. 6. Nginx – Web Server  nginx [engine x] is an HTTP and reverse proxy server, as well as a mail proxy server  nginx accelerates content and application delivery, improves security, facilitates availability and scalability for the busiest web sites on the Internet.  Half of the Internet's largest websites rely on NGINX, including Airbnb, Box, Instagram, Netflix, Pinterest, SoundCloud, Zappos, and more.
  7. 7.  Ruby on Rails – Ruby language on Rails Framework  It is well known for its quick iteration cycle(Plan-Run- Review-Change)  Speed up development  Simplify prototyping
  8. 8. Redis – In Memory Database  in-memory data structure store, used as database, cache  In short a datastore like MySQL  Airbnb - Redis maintained a cache of ticket trend results from elastic search for Node.js app to render on the web
  9. 9. Amazon S3,EBS – Cloud Storage  Airbnb migrated all of its cloud computing functions to AWS.  Backup for house backups and static files, including 10 terabytes of user pictures.
  10. 10.  Amazon EC2 – Cloud Hosting  Airbnb is using Elastic Load Balancing, which automatically distributes incoming traffic between multiple Amazon EC2 instances.
  11. 11. Amazon EMR  To easily process and analyze 50 Gigabytes of data daily, Airbnb uses Amazon Elastic MapReduce (Amazon EMR)
  12. 12.  Amazon RDS – Cloud Database  Airbnb moved its main MySQL database to Amazon Relational Database Service.  Airbnb chose Amazon RDS because it simplifies much of the time-consuming administrative tasks typically associated with databases
  13. 13. Amazon ElastiCache  ElastiCache improves the performance of web applications by allowing you to retrieve information fastly  Used to speeding dynamic web application by alleviating database load
  14. 14. SaSS – Syntactically Awsome Style Sheet  Sass is an extension of CSS3, adding nested rules, variables, mixins, selector inheritance, and more.
  15. 15. React – Javascript UI Libraries  A declarative, efficient, and flexible JavaScript library for building user interfaces.
  16. 16. Airflow – Workflow Manager  Airbnb, its own data workflow management framework  Airflow is being used internally at Airbnb to build, monitor and adjust data pipelines.  Data pipelines use to automate the movement and transformation of data.
  17. 17. Hadoop - BigDatabase  Open-Source Framework that allows to store and process big data a distributed environment across clusters of computers using simple programming models.   It is designed to scale up from single servers to thousands of machines, each offering local computation and storage.
  18. 18. Presto, Druid – Big Data Tools  SQL query tool for hadoop.  Druid is an open-source analytics data store designed for business intelligence (OLAP) queries on event data.  Druid provides low latency (real-time) data.
  19. 19. Airpal – Big Data Tools Developed by Airbnb A web-based query execution tool that leverages Presto to facilitate data analysis. For example, you can highlight syntax, export results to CSV for download, view query history, save queries, use a Table Finder to search for appropriate tables, and use Table Explorer to visualize the schema of a table.
  20. 20. Utilities
  21. 21. Amazon CloudFront - CDN A content delivery network (CDN) is a system that deliver webpages and other Web content to a user based on the geographic locations of the user. Amazon CloudFront is a global content delivery network (CDN) service that accelerates delivery of your websites, APIs, video content or other web assets.
  22. 22. Sendgrid – Transactional Email  Delivering your transactional and marketing email through one reliable platform.
  23. 23. MixPanel – Funnel Analysis Analytics • It tracks user interactions with web and mobile applications • Funnel analysis helps you identify where people drop off so you can increase your conversion rates • The funnel analyses "are an effective way to calculate conversion rates on specific user behaviors'
  24. 24. Amazon Route 53 – DNS Management  DNS data is typically deployed on multiple physical servers.  The main purposes of DNS management software are: to reduce human error when editing complex and repetitive DNS data.  Amazon Route 53 is a highly available and scalable cloud Domain Name System (DNS) web service.
  25. 25. BrainTree – Payment Services  Subsidiary of Paypal
  26. 26. Twilio – Voice and SMS • Twilio offers developers a API for phone services to make and receive phone calls, and send and receive text. • Airbnb uses Twilio to build Voice Connect click to call, bridging the communication gap between guests and would be hosts.
  27. 27. Nexmo – Voice and SMS  API for SMS, Voice and Phone Verification.
  28. 28. Urban Airship - Mobile Push Messaging Improve customer engagement with push notifications, location- based marketing and analytics.
  29. 29. Urban Airship - Mobile Push Messaging Improve customer engagement with push notifications, location- based marketing and analytics.
  30. 30.  The push notifications services enables international users to receive notifications regardless of their location or carrier.  Users then receive push notifications for both new messages received on the platform as well as new reservations, delivered directly to their mobile device
  31. 31. Visual Website Optimizer - A/B Testing Analytics • VWO is the easiest A/B, Split and Multivariate testing tool. • To optimizing your website for increased conversion rate and sales. • A/B testing (sometimes called split testing) is comparing two versions of a web page to see which one performs better. • You compare two web pages by showing the two variants (let's call them A and B) to similar visitors at the same time. The one that gives a better conversion rate, wins!
  32. 32. Aerosolve - A machine learning package built for humans. Created by airbnb Airbnb's Price Tips feature helps users figure out what to charge by using machine learning. n this dynamic pricing feature, we show hosts the probability of getting a booking (green for a higher chance, red for a lower chance), or predicted demand, and allow them to easily price their listings dynamically with a click of a button.
  33. 33. Fig. Learning to rank images. On the left, image ordering trained from professional photographer ratings. On the right, image ordering trained from organic books, clicks and impressions Aerosolve - Image Analysis Algorithms
  34. 34. Aerosolve - A machine learning package built for humans. Seasonal demand for Austin
  35. 35. DeepLinkDispatch - Deep Linking as a Service A simple, annotation-based library for making deep link handling better on Android, by Airbnb Mobile deep links let users share content that's within an app. If you want to send a friend a pair of shoes from the Cole Haan app, you can send them to those shoes inside the app using a deep link. Without a deep link they have to find the Cole Haan app, open the app to the homepage, then find the shoes (and hope they found the right ones). With deep linking, links to content within mobile apps can be created and shared just like the web. Developers can build better experiences, and users can easily discover interesting content.
  36. 36. DeepLinkDispatch - Deep Linking as a Service A simple, annotation-based library for making deep link handling better on Android, by Airbnb Mobile deep links let users share content that's within an app. If you want to send a friend a pair of shoes from the Cole Haan app, you can send them to those shoes inside the app using a deep link. Without a deep link they have to find the Cole Haan app, open the app to the homepage, then find the shoes (and hope they found the right ones). With deep linking, links to content within mobile apps can be created and shared just like the web. Developers can build better experiences, and users can easily discover interesting content.
  37. 37. DeepLinkDispatch - Deep Linking as a Service
  38. 38. New Relic – Performance Monitor SaaS Application Performance Management for Ruby, PHP, .Net, Java, Python, and Node.js Apps. A tool for building and distributing development environments
  39. 39. Vagrant – Virtual Machine Management  A tool for building and distributing development environments   Vagrant allows you to create portable work environments which can be easily reproduced on any system.   It's useful for developers and system administrators who want to mimic a server's configuration on their local machine.   It's useful for teams who want to quickly get designers and project managers up and running with a local installation.
  40. 40. Enzyme – Javascript Testing Framework  Javascript Testing utilities for React, by Airbnb
  41. 41. SmartStack – Service Discovery • SmartStack is an automated service discovery and registration framework. • It makes the lives of engineers easier by transparently handling creation, deletion, failure, and maintenance work of the machines running code within your organization
  42. 42. Scout – Performance Monitoring  Scout is a simple hosted server monitoring service.  Rather than installing and configuring monitoring scripts on servers  Scout features a point-and-click plugin setup  Scout currently monitors the servers of AirBnB and The New York Times.
  43. 43. Sentry - Exception Monitoring  Real-time crash reporting for your web apps, mobile apps, and games.  Sentry notifies you when your users experience errors in your web and mobile apps.
  44. 44. Chef - Server Configuration and Automation • Chef enables you to manage and scale cloud infrastructure with no downtime or interruptions. • Freely move applications and configurations from one cloud to another. • Chef is integrated with all major cloud providers including Amazon EC2, VMWare, IBM Smartcloud, Rackspace, OpenStack, Windows Azure, HP Cloud, Google Compute Engine, Joyent Cloud and others.
  45. 45. Chef - Server Configuration and Automation  Chef enables you to manage and scale cloud infrastructure with no downtime or interruptions.  Freely move applications and configurations from one cloud to another.  Chef is integrated with all major cloud providers including Amazon EC2, VMWare, IBM Smartcloud, Rackspace, OpenStack, Windows Azure, HP Cloud, Google Compute Engine, Joyent Cloud and others.
  46. 46. Mocha - Javascript Testing Framework Simple, flexible, fun javascript test framework for node.js & the browser. Mocha tests run serially, allowing for flexible and accurate reporting, while mapping uncaught exceptions to the correct test cases.
  47. 47. Datadog - Performance Monitoring  Datadog is the service for cloud-scale monitoring  Shows Performance monitoring about app, tools and services in one place  Used by thousands of companies
  48. 48. Logstash - Log Management  Logstash is a tool for managing events and logs.  You can use it to collect logs, parse them, and store them for later use.
  49. 49. kibana - Monitoring Tools Kibana is an open source, browser based analytics and search dashboard for Elasticsearch. Airbnb - Collect logs with logstash and visualize them with kibana.
  50. 50. Google Apps - Productivity Suite  Web based email, calendar, storage, file sharing and video conferencing that let you work from anywhere.  Business emails can me migrated to google app to get above features
  51. 51. Slack - Group Chat & Notifications Slack brings all your communication together in one place
  52. 52. Asana - Project Management Asana is the easiest way for teams to track their work. From tasks and projects to conversations and dashboards, Asana enables teams to move work from start to finish--and get results.
  53. 53. Campaign Monitor- Email Marketing Campaign Monitor makes it easy to attract new subscribers, send them beautiful email newsletters.
  54. 54. InVision- Interactive Mockups  InVision is a web-based prototyping tool created for designers, by designers.   It allows you to quickly and easily create interactive mockups for your designs.   Transform your Web & Mobile (iOS, Android) designs into clickable, interactive Prototypes and Mockups.
  55. 55. Assemblage- Visual Collaboration • For more efficient projects, teams and meetings. • Collaborate directly in your browser. • Supports 20+ filetypes. • Draw, annotate and edit with partners, co-workers or friends. • All changes happen real-time on all screens.
  56. 56. Caravel- Business Intelligence • Data exploration and visualization platform, by Airbnb • Caravel's main goal is to make it easy to slice, dice and visualize data. • Allows you to create and share beautiful charts and dashboards
  57. 57. 2009 Engineer Count: 3-4 Core - Rails 2.x app Backend - MySQL Database Frontend - Vanilla CSS ( no preprocessors), Prototype.js 2012 Engineer Count: 30+ Core - Rails 3.x, Java (backend services) Backend - Redis, HDFS (for data warehousing), Postgres (geospacial data), Dynamo DB Frontend - SASS (CSS pre-processing) O2 (bootstrap with AirBnB styles) (style kit) Query (instead of Prototype), Backbone & Handlebars (single page, rich client apps) 2015 Core - Rails 3.x, Java, Node.JS Backend - MySQL, Redis, HDFS, Postgres DynamoDB, Kafka, RabbitMQ Frotend - SASS (CSS pre-processing), O2 ( AirBnB bootstrap-based style kit) jQuery, Backbone.js, Handlebars.js, React.js, CommonJS (for modules), ES6 (ES2015), Flux Airbnb Era
  58. 58. References www.zdnet.com/article/telstra-invests-in-nginx-web-server/ http://stackshare.io/ https://www.quora.com/What-backend-language-does-AirBNB-use https://medium.com/airbnb-engineering/how-airbnb-manages-to-monitor- customer-issues-at-scale-b883301ca461#.26u8nf9w3 http://aws.amazon.com/solutions/case-studies/airbnb/ https://blogs.aws.amazon.com/bigdata/post/Tx1BF2DN6KRFI27/Analyze- Data-with-Presto-and-Airpal-on-Amazon-EMR Www.Slideshare.net/airbnb http://techstacks.io/airbnb https://www.quora.com/What-is-the-technology-stack-of-Airbnb https://gigaom.com/2013/07/29/airbnb-is-engineering-itself-into-a-data-driven- company/ https://www.quora.com/Why-does-Airbnb-use-Braintree-instead-of-PayPal http://nerds.airbnb.com/aerosolve/ https://branch.io/what-is-deep-linking/ http://nerds.airbnb.com/smartstack-service-discovery-cloud/ http://www.yandss.com/yanfilofyev/2013/08/airbnb-technology-stack/ http://www.programmableweb.com/api/scout-app
  59. 59. References(cont.) www.invisionapp.com https://github.com/airbnb/caravel
  60. 60. •Thank You

×