This document discusses Google's infrastructure and data centers. It describes Google's use of large data centers containing thousands of servers and petabytes of storage. It also summarizes Google's development of technologies like GFS, MapReduce, and BigTable to handle massive amounts of data across their infrastructure. Key details are provided on hardware specifications, network switches, reliability targets, and the engineers involved in developing Google's data-handling systems.

1. 03/28/11 Google Inc

2. •
•
•
•
03/28/11 Google Inc

3. •
–
•
– Gmail, Calendar, Picasa, Docs, Google Apps,...
– App Engine
•
–
–
–
–
03/28/11 Google Inc

4. ‐‐ Tera
Gega
Tera ‐‐ Peta
‐‐ Peta
03/28/11 Google Inc

5. vs
03/28/11 Google Inc

6. vs
03/28/11 Google Inc

7. vs
03/28/11 Google Inc

8. vs
03/28/11 Google Inc

9. •
•
•
•
03/28/11 Google Inc

10. •
– 16GB DRAM; 160 GB SSD; 5 x 1TB disk
• (Rack)
– 40
– 48 port Gigabit Ethernet switch
• (Warehouse)
– 10,000 (250 racks)
– 2K port Gigabit Ethernet switch
03/28/11 Google Inc

11. ‐‐‐
03/28/11 Google Inc

12. ‐‐‐
03/28/11 Google Inc

13. ‐‐‐
03/28/11 Google Inc

14. •
– , ,
– , ,
•
–
–
•
–
– Flash (SSD); GPU
03/28/11 Google Inc

15. 30
03/28/11 Google Inc

16. ( )
2000 – 2005
6–8
03/28/11 Google Inc

17. ‐‐
03/28/11 Google Inc

18. = 2.0 Google = 1.16
IT
03/28/11 Google Inc

19. •
– , ,
– , ,
•
–
–
•
–
– Flash (SSD); GPU
03/28/11 Google Inc

20. • 99.9% = 9 /
• 10,000
– 0.25
– 3
– 1,000
– 1,000s
– etc., etc., etc.
03/28/11 Google Inc

21. •
•
•
–
–
–
03/28/11 Google Inc

22. •
– , ,
– , ,
•
–
–
•
–
– Flash (SSD); GPU
03/28/11 Google Inc

23. •
•
•
•
03/28/11 Google Inc

24. (GFS)
GFS Master
Master Client
GFS Master
R
p
a
c
e
s
i Client
l
C0 C1 C1 C0 C3
C3 C4 C3 C5 C4
03/28/11 Google Inc

25. MapReduce
Data Block 1
map Results
Results
Data
Data Data Block 2 datadata
datadatada datadata
datadatada Reduce datadata
tadatadata datadata
tadatadata map datadata
datadatada datadata
datadatada
tadatadata
tadatadata
Data Block 3 Data Block 5
map
map
map
Data Block 4
GFS Mapreduce 2004 , Hadoop
, Facebook Hadoop
03/28/11 Google Inc

26. (Big Table)
•
•
– :
– :
– :
03/28/11 Google Inc

27. •
•
•
•
03/28/11 Google Inc

28. •
•
•
03/28/11 Google Inc

29. 03/28/11 Google Inc

30. •
•
•
•
03/28/11 Google Inc

31. •
– Peter Norvig
– Stuart Feldman
– Edward Chang
– Xuemei Gu
– Hai Fang
03/28/11 Google Inc

32. 03/28/11 Google Inc

33. 杭州站 · 2011年10月20日~22日
www.qconhangzhou.com（6月启动）
QCon北京站官方网站和资料下载
www.qconbeijing.com