The document discusses designing large scale file storage and serving systems. It covers key requirements like scalability, reliability, and cost-effectiveness. It then describes various approaches to storage, such as network attached storage (NAS), storage area networks (SAN), and caching. Specific examples like Google File System, MogileFS, and Flickr's file system are examined. Business continuity planning (BCP) through redundancy and replication is also discussed.

1. Beyond the File System Designing Large Scale File Storage and Serving Cal Henderson

2. Hello!

7. Four buckets Storage Serving BCP Cost

8. Storage

9. The storage stack File system Block protocol RAID Hardware ext, reiserFS, NTFS SCSI, SATA, FC Mirrors, Stripes Disks and stuff File protocol NFS, CIFS, SMB

12. DAS Direct attached storage Disk shelf, connected by SCSI/SATA HP MSA30 – 14 disks in 3U

20. NetApp Filers Heads and shelves, up to 500TB in 6 Cabs FC SAN with 1 or 2 NAS heads

34. Real Life Case Studies

37. GFS – Google File System 1(a) 2(a) 1(b) Master

42. MogileFS – OMG Files Tracker Tracker MySQL

46. Flickr File System SM SM SM

50. Amazon S3 Servers Amazon

51. Amazon S3 Servers Amazon Users

53. The cost S3

54. The cost S3 Regular Bandwidth

56. Serving

67. Edge networks Origin

68. Edge networks Origin Cache Cache Cache Cache Cache Cache Cache Cache

79. BCP

89. Cost

91. Hardware costs Cost of hardware Usable GB Single Cost

92. Power costs Cost of power per year Usable GB Recurring Cost

93. Power costs Power installation cost Usable GB Single Cost

94. Space costs Cost per U Usable GB [ ] U’s needed (inc network) x Recurring Cost

95. Network costs Cost of network gear Usable GB Single Cost

96. Misc costs Support contracts + spare disks Usable GB + bus adaptors + cables [ ] Single & Recurring Costs

97. Human costs Admin cost per node Node count x Recurring Cost Usable GB [ ]

99. (fin)