Data Center Survey: How Many IOPS Is Enough. Compares to site capacity and system latency.

530 views

Published on

Presentation from the Silicon Valley Database Meets SSD Meetup (www.meetup.com/db-speed-sv). Database applications have become a key user of SSDs. Why is this? What does it mean to database admins? Objective Analysis and Coughlin Associates performed a survey over the past several months in which nearly 200 respondents told us about their storage needs. 40% of these users listed database programs as their main application, and these respondents shared their thoughts on speed and performance needs. In this presentation, Jim Handy will discuss this part of our survey and of the reason that SSDs are so useful in database applications.

Published in: Data & Analytics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
530
On SlideShare
0
From Embeds
0
Number of Embeds
14
Actions
Shares
0
Downloads
23
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Data Center Survey: How Many IOPS Is Enough. Compares to site capacity and system latency.

  1. 1. 1 OBJECTIVE ANALYSIS – Semiconductor Market Research How Many IOPS is Enough? Tom Coughlin Coughlin Associate & Jim Handy Objective Analysis OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Outline • The Survey • Application Distribution and Attributes • More Survey Results: IOPS, Capacity and Latency • Developing tiers of storage for enterprise (and client) applications • Implications/Projections • Authors & Sources 2
  2. 2. 2 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Outline • The Survey • Application Distribution and Attributes • More Survey Results: IOPS, Capacity and Latency • Developing tiers of storage for enterprise (and client) applications • Implications/Projections • Authors and Sources 3 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Our Survey • Five-minute survey asked end users – IOPS needs – Capacity – Latency – System bottleneck IOPS – Primary application • Nearly 200 respondents • Report analyzes and interprets the results in depth 4
  3. 3. 3 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Outline • The Survey • Application Distribution and Attributes • More Survey Results: IOPS, Capacity and Latency • Developing tiers of storage for enterprise (and client) applications • Implications/Projections • Authors and Sources 5 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Applications Breakdown 6 Archiving and backup Video Creation or Distribution Cloud storage or services OLTP Scientific or Engineering Databases Mail server and mail storage
  4. 4. 4 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Application Types • Cloud Storage/Service-Virtualization • Databases • On-Line Transaction Processing (OLTP) • Video Creation and Distribution • Science & Engineering • Exchange Servers 7 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Cloud Storage/Services & Virtualization • The “IO Blender” – Many streams – Scrambled I/O – Highly random • Suits SSDs better than HDDs for rapid access • Many VM and VDI systems using flash cache to meet demand speed needs Image courtesy of Waring Corp. 8
  5. 5. 5 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Databases • Large data sets • Random traffic • High I/O load • Early SSD adopter (and before that used DRAM-based SSDs) • Some users load their entire DB on flash memory 9 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com OLTP (On-Line Transaction Processing) • Verified writes – Write/read back – Doubles I/O load • No room for errors • Speed is imperative – Delays lose customers 10
  6. 6. 6 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Video Creation or Distribution • Large data sets • Multiple video streams – Randomizes access • High bandwidth required • Expensive talent – Don’t want them sitting around waiting Image courtesy of the US Library of Congress 11 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Recording Media Share in Professional Video Cameras 2012 Digital Storage for Media and Entertainment Report, Coughlin Associates 12
  7. 7. 7 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Science & Engineering • Complex problems – Genome sequencing – CAD/CAM – Natural Resources – Nuclear modeling • Large data sets • Expensive talent – Don’t want them sitting around waiting Image courtesy of Wikimedia Commons 13 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Exchange Server • Multiple tasks – e-mail – Scheduling/calendars – Data storage • Scads of users • e-mail chaos – Multiple mailboxes – Asynchronous sends & receives – Spam & virus filters Image courtesy of Dell Computer 14
  8. 8. 8 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Outline • The Survey • Application Distribution and Attributes • More Survey Results: IOPS, Capacity and Latency • Developing tiers of storage for enterprise (and client) applications • Implications/Projections • Authors and Sources 15 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Let’s Look At the Overall Results Report breaks analysis down by application – That’s 54 charts – Too much for this presentation! • We’ll just look at Database
  9. 9. 9 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com IOPS Required 0% 10% 20% 30% 40%Respondents 10 100 1K 10K 100K 1M 10M IOPS OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Storage Capacity Needed 0% 5% 10% 15% 20% 25% Respondents 1G B 10G B 50G B 100G B500G B 1TB 5TB 10TB 50TB 100TB Capacity
  10. 10. 10 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com How Do The Two Compare To Each Other? OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com IOPS vs. Capacity 1 10 100 1,000 10,000 100,000 1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 1.E+07 IOPS Capacity(GB) 20
  11. 11. 11 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com System Bottleneck IOPS What is the Fastest Storage Your System Can Use? 0% 10% 20% 30% 40%Respondents 10 100 1K 10K 100K 1M 10M IOPS OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Minimum Latency Requirement 0% 10% 20% 30% 40% Respondents 10ns 1µs 100µs 10ms 1 sec Latency
  12. 12. 12 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Outline • The Survey • Application Distribution and Attributes • More Survey Results: IOPS, Capacity and Latency • Developing tiers of storage for enterprise (and client) applications • Implications/Projections • Authors and Sources 23 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com DRAMs Now 6,000x HDD Speed! From : HDDs and Flash Memory: A Marriage of Convenience 24
  13. 13. 13 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Memory and Storage Price vs. Bandwidth 1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 1.E+07 1.E-01 1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 Price per Gigabyte Bandwidth(MB/s) Tape HDD NAND DRAM L3 L2 L1 10 1 10 0 10 2 10 3 10 4 10 5 10 6 10 7 10 1 10 2 10 3 10 4 10 5 10 6 10 0 10 -1 From Objective Analysis: Are Hybrid Drives Finally Coming of Age? 25 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Price/GB Roughly Follows IOPS $0.01 $0.10 $1.00 $10.00 $100.00 $1,000.00 0 200 400 600 800 1,000 1,200 IOPS $/GB 26
  14. 14. 14 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Device IOPS by Form Factor 102 103 104 105 106 HDD SATA SAS/FC 2-Hop 1-Hop 27 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Outline • The Survey • Application Distribution and Attributes • More Survey Results: IOPS, Capacity and Latency • Developing tiers of storage for enterprise (and client) applications • Implications/Projections • Authors and Sources 28
  15. 15. 15 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Implications/Projections (1) • SSDs adoption will increase – Usually more IOPS is better • Fast storage is changing – From short-stroked HDDs to SSDs • HDDs becoming a tier behind SSDs • Other system elements will become the bottleneck – Network, software, servers… 29 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Implications/Projections (2) • Users will focus more attention on IOPS – Understanding will be greater than it is today • Higher IOPS will support data/content growth – This means more storage • SSD • HDD • Even tape 30
  16. 16. 16 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Report Compiles Survey Results • An analysis of complete survey results by application • Report, published July 2014, can be purchased for immediate download at www.Objective-Analysis.com. • Orders can also be processed through Coughlin Assocaites at: http://www.tomcoughlin.com/techpapers.htm. • You can contact Coughlin Associates by calling Tom at 408-871-8808, or e-mailing: Tom@TomCoughlin.com. 31 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Analysts 32 Tom Coughlin, President, Coughlin Associates is a highly-respected storage analyst and consultant with over 30 years in the data storage industry in engineering and management at high profile companies. Jim Handy is a widely recognized semiconductor analyst, has over 30 years in the electronics industry. His background includes marketing and design positions at market-leading suppliers.Jim Handy Objective Analysis Thomas Coughlin Coughlin Associates
  17. 17. 17 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Sources • How Many IOPS do You Really Need? • 2012 Digital Storage for Media and Entertainment, Coughlin Associates: (www.tomcoughlin.com/techpapers) • HDDs and Flash Memory: A Marriage of Convenience, Coughlin Associates and Objective Analysis, 2011 (www.tomcoughlin.com/techpapers) • Two may be Better than One: Why HDD and Flash Belong • SNIA SSSI White Paper, Coughlin/Handy 2010 • Are Hybrid Drives Finally coming of Age?, Objective Analysis, 2010 33

×