The Facebook Data team
Infrastructure and insight
Jeff Hammerbacher
Engineering Manager
January 16, 2008

Agenda
Motivations
1
People
2
Products
3
Philosophy
4

Motivations
Source data living on a horizontally partitioned MySQL tier
▪
Intensive historical analysis difficult
▪
Difficult to assess impact of changes to the site
▪
First try: Reporting and Analysis
▪
Second try: Data Infrastructure and Data Insight
▪
Third try: Data
▪

People
Infrastructure
Rama Ramasamy
▪
Suresh Antony
▪
Avinash Lakshman
▪
Hao Liu
▪
Joydeep Sen Sarma
▪
Ashish Thusoo
▪
Pete Wyckoff
▪

People
Insight
Rupen Parikh
▪
Itamar Rosenn
▪
Ravi Grover
▪
Roddy Lindsay
▪
Cameron Marlow
▪
Danny Ferrante
▪
Ding Zhou
▪

Products
Infrastructure
Hive
▪
Hive Shell
▪
Pyhive
▪
Apidictor
▪
Nest
▪
???
▪

Products
Insight
Experimentation Platform
▪
Argus
▪
Insights
▪
InvEst
▪
Growth Report
▪
More coming soon...
▪

Philosophy
Data management as important as data analysis
▪
Know your data center: compute, storage, interconnect
▪
Shared nothing means the interconnect is the bottleneck
▪
Engineers are not analysts
▪
You must write code
▪
Don’t collect data without a purpose
▪
Basic statistics is more useful than advanced machine learning
▪
Visualization is as important as basic statistics
▪
Analysis and not reporting
▪
Developing products might be better than YAS
▪

(c) 2008 Facebook, Inc. or its licensors. \"Facebook\" is a registered trademark of Facebook, Inc.. All rights reserved. 1.0