Your SlideShare is downloading. ×
0
Presentation slides
Presentation slides
Presentation slides
Presentation slides
Presentation slides
Presentation slides
Presentation slides
Presentation slides
Presentation slides
Presentation slides
Presentation slides
Presentation slides
Presentation slides
Presentation slides
Presentation slides
Presentation slides
Presentation slides
Presentation slides
Presentation slides
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Presentation slides

1,117

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,117
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Case Study - Storage Consolidation Steve Curry Yahoo Inc.
  • 2. <ul><li>Quick Stats </li></ul><ul><li>300+ million registered users </li></ul><ul><li>2 billion page request per day </li></ul><ul><li>25 countries, 14 languages </li></ul><ul><li>500TB data on disk </li></ul><ul><li>1PB data on tape </li></ul>About Yahoo !
  • 3. <ul><li>Responsibilities </li></ul><ul><li>All US storage administration </li></ul><ul><li>Data archiving / backups </li></ul><ul><li>US/Global storage architecture / standards </li></ul><ul><li>2 nd tier support for global operations </li></ul><ul><li>Tool development </li></ul><ul><li>24/7 global issue/outage response </li></ul><ul><li>Reporting </li></ul>Yahoo! Storage Operations
  • 4. Case Study #1 Case Study #1 – Y! Photos/Briefcase <ul><li>Online photo album </li></ul><ul><li>Online file storage </li></ul>
  • 5. <ul><li>Legacy Architecture </li></ul><ul><li>Cheap… *repeat* cheap JBOD’s </li></ul><ul><li>Single host support JBOD array </li></ul><ul><li>A/B mirror for redundancy </li></ul><ul><li>FreeBSD OS </li></ul><ul><li>150TB of content </li></ul><ul><li>Custom apps </li></ul>Case Study #1 – Y! Photos/Briefcase
  • 6. <ul><li>… Legacy Architecture </li></ul><ul><li>Advantages </li></ul><ul><ul><li>Low cost hardware </li></ul></ul><ul><ul><li>Extremely distributed </li></ul></ul><ul><li>Disadvantages </li></ul><ul><ul><li>Not very scalable </li></ul></ul><ul><ul><li>Management headache </li></ul></ul><ul><ul><li>No longer meets reliability requirements </li></ul></ul>Case Study #1 – Y! Photos/Briefcase
  • 7. <ul><li>… Legacy Architecture </li></ul><ul><li>Management Issues </li></ul><ul><ul><li>Management is per host (over 160 storage hosts) </li></ul></ul><ul><ul><li>Synchronous mirror between A/B pair </li></ul></ul><ul><ul><li>No “Hot-Swap” support </li></ul></ul><ul><ul><li>Single spindle performance </li></ul></ul>Case Study #1 – Y! Photos/Briefcase
  • 8. Case Study #1 – Y! Photos/Briefcase This… X 12!  Single tier, single spindle performance.
  • 9. <ul><li>Consolidation Plan </li></ul><ul><li>NAS or SAN? </li></ul><ul><li>Requirements </li></ul><ul><ul><li>Reliability </li></ul></ul><ul><ul><li>Scalability </li></ul></ul><ul><ul><li>Reduce management overhead </li></ul></ul><ul><li>Considerations </li></ul><ul><ul><li>Current hardware investment </li></ul></ul><ul><ul><li>Application support </li></ul></ul>Case Study #1 – Y! Photos/Briefcase
  • 10. <ul><li>Network Attached Storage Solution </li></ul><ul><li>Management </li></ul><ul><ul><li>Filers are heavily deployed </li></ul></ul><ul><ul><li>Smart appliance </li></ul></ul><ul><ul><li>Suite of tools already developed for filers </li></ul></ul><ul><li>Advantages </li></ul><ul><ul><li>RAID redundancy </li></ul></ul><ul><ul><li>Multi-spindle performance </li></ul></ul><ul><ul><li>Takes advantage of existing hardware </li></ul></ul><ul><ul><li>Ease of application port </li></ul></ul>Case Study #1 – Y! Photos/Briefcase
  • 11. <ul><li>… Network Attached Storage Solution </li></ul><ul><li>Disadvantages </li></ul><ul><ul><li>Initial cost of deployment (cutover, SCSI –vs- IDE) </li></ul></ul><ul><ul><li>Lot’s of JBOD’s to get rid of! ;-) </li></ul></ul>Case Study #1 – Y! Photos/Briefcase
  • 12. <ul><li>New Architecture </li></ul><ul><li>NAS solution </li></ul><ul><li>FreeBSD app servers </li></ul><ul><li>Load balanced </li></ul><ul><li>10 storage hosts </li></ul><ul><li>Point in time snapshots </li></ul><ul><li>Dedicated SAN backup fabric </li></ul><ul><li>Distributed-farm model </li></ul>Case Study #1 – Y! Photos/Briefcase
  • 13. Case Study #1 – Y! Photos/Briefcase Simple 2 tier model. Scalable, redundant, multi-spindle RAID performance, hot-swap support.
  • 14. <ul><li>Consolidation Wins! </li></ul><ul><li>Cost considerations </li></ul><ul><li>Performance </li></ul><ul><li>Backups </li></ul><ul><li>Management </li></ul><ul><li>High availability </li></ul><ul><li>Hot swap </li></ul>Case Study #1 – Y! Photos/Briefcase
  • 15. <ul><li>Case Study #2 </li></ul><ul><li>Global data mining </li></ul><ul><li>Global log collection </li></ul>Case Study #2 - Data Mining
  • 16. <ul><li>Current Architecture </li></ul><ul><li>DAS attached arrays </li></ul><ul><li>Custom scripts </li></ul><ul><li>Stacker type tape libraries </li></ul><ul><li>Single-tier disk storage </li></ul>Case Study #2 - Data Mining
  • 17. <ul><li>Management Issues </li></ul><ul><li>Large storage host count </li></ul><ul><li>Many small tape libraries </li></ul><ul><li>No redundancy </li></ul><ul><li>Does scale for future requirements </li></ul>Case Study #2 - Data Mining
  • 18. <ul><li>Storage Requirements </li></ul><ul><li>High write performance </li></ul><ul><li>Data growth 2TB per day!! </li></ul><ul><li>Store data on disk for 30 days </li></ul><ul><li>Archive to tape </li></ul><ul><li>Consolidation Considerations </li></ul><ul><li>Reduce host management </li></ul><ul><li>Create a multi-tier storage architecture </li></ul><ul><li>Consolidate to one large tape library </li></ul><ul><li>Increase write performance </li></ul>Case Study #2 - Data Mining
  • 19. Case Study #2 - Data Mining <ul><li>Common Y! model </li></ul><ul><li>Multi-tier storage </li></ul><ul><li>Scalable </li></ul>

×