Data Center Survey: How Many IOPS Is Enough. Compares to site capacity and system latency.

1
OBJECTIVE ANALYSIS – Semiconductor Market Research
How Many IOPS is
Enough?
Tom Coughlin
Coughlin Associate
&
Jim Handy
Objective Analysis
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Outline
• The Survey
• Application Distribution and Attributes
• More Survey Results: IOPS, Capacity and
Latency
• Developing tiers of storage for enterprise
(and client) applications
• Implications/Projections
• Authors & Sources
2

2
Outline
• The Survey
Latency
• Authors and Sources
3
Our Survey
• Five-minute survey asked end users
– IOPS needs
– Capacity
– Latency
– System bottleneck IOPS
– Primary application
• Nearly 200 respondents
• Report analyzes and interprets the results
in depth
4

3
Outline
• The Survey
Latency
5
Applications Breakdown
6
Archiving and backup
Video Creation or Distribution
Cloud storage or services
OLTP
Scientific or Engineering
Databases
Mail server and mail storage

4
Application Types
• Cloud Storage/Service-Virtualization
• Databases
• On-Line Transaction Processing (OLTP)
• Video Creation and Distribution
• Science & Engineering
• Exchange Servers
7
Cloud Storage/Services
& Virtualization
• The “IO Blender”
– Many streams
– Scrambled I/O
– Highly random
• Suits SSDs better than
HDDs for rapid access
• Many VM and VDI
systems using flash cache
to meet demand speed
needs Image courtesy of Waring Corp.
8

5
Databases
• Large data sets
• Random traffic
• High I/O load
• Early SSD adopter
(and before that
used DRAM-based
SSDs)
• Some users load
their entire DB on
flash memory 9
OLTP
(On-Line Transaction Processing)
• Verified writes
– Write/read back
– Doubles I/O load
• No room for errors
• Speed is
imperative
– Delays lose customers
10

6
Video Creation or
Distribution
• Large data sets
• Multiple video
streams
– Randomizes access
• High bandwidth
required
• Expensive talent
– Don’t want them sitting
around waiting
Image courtesy of the US Library of Congress
11
Recording Media Share in
Professional Video Cameras
2012 Digital Storage for Media and Entertainment Report, Coughlin Associates
12

7
Science & Engineering
• Complex problems
– Genome sequencing
– CAD/CAM
– Natural Resources
– Nuclear modeling
• Large data sets
• Expensive talent
– Don’t want them sitting
around waiting
Image courtesy of Wikimedia Commons
13
Exchange Server
• Multiple tasks
– e-mail
– Scheduling/calendars
– Data storage
• Scads of users
• e-mail chaos
– Multiple mailboxes
– Asynchronous sends &
receives
– Spam & virus filters
Image courtesy of Dell Computer
14

8
Outline
• The Survey
Latency
15
Let’s Look At the Overall Results
Report breaks analysis down by application
– That’s 54 charts
– Too much for this presentation!
• We’ll just look at Database

9
IOPS Required
0%
10%
20%
30%
40%Respondents
10 100 1K 10K 100K 1M 10M
IOPS
Storage Capacity Needed
0%
5%
10%
15%
20%
25%
Respondents
1G
B
10G
B
50G
B
100G
B500G
B
1TB
5TB
10TB
50TB
100TB
Capacity

10
How Do The Two
Compare To Each Other?
IOPS vs. Capacity
1
10
100
1,000
10,000
100,000
1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 1.E+07
IOPS
Capacity(GB)
20

11
System Bottleneck IOPS
What is the Fastest Storage Your System Can Use?
0%
10%
20%
30%
40%Respondents
10 100 1K 10K 100K 1M 10M
IOPS
Minimum Latency Requirement
0%
10%
20%
30%
40%
Respondents
10ns 1µs 100µs 10ms 1 sec
Latency

12
Outline
• The Survey
Latency
23
DRAMs Now 6,000x HDD Speed!
From : HDDs and Flash Memory: A Marriage of Convenience
24

13
Memory and Storage
Price vs. Bandwidth
1.E+00
1.E+01
1.E+02
1.E+03
1.E+04
1.E+05
1.E+06
1.E+07
1.E-01 1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06
Price per Gigabyte
Bandwidth(MB/s)
Tape
HDD
NAND
DRAM
L3
L2
L1
10
1
10
0
10
2
10
3
10
4
10
5
10
6
10
7
10
1
10
2
10
3
10
4
10
5
10
6
10
0
10
-1
From Objective Analysis: Are Hybrid Drives Finally Coming of Age?
25
Price/GB Roughly Follows IOPS
$0.01
$0.10
$1.00
$10.00
$100.00
$1,000.00
0 200 400 600 800 1,000 1,200
IOPS
$/GB
26

14
Device IOPS by Form Factor
102 103 104 105 106
HDD SATA SAS/FC 2-Hop 1-Hop
27
Outline
• The Survey
Latency
28

15
Implications/Projections (1)
• SSDs adoption will increase
– Usually more IOPS is better
• Fast storage is changing
– From short-stroked HDDs to SSDs
• HDDs becoming a tier behind SSDs
• Other system elements will become the
bottleneck
– Network, software, servers…
29
Implications/Projections (2)
• Users will focus more attention on IOPS
– Understanding will be greater than it is today
• Higher IOPS will support data/content
growth
– This means more storage
• SSD
• HDD
• Even tape
30

16
Report Compiles Survey Results
• An analysis of complete survey results by application
• Report, published July 2014, can be purchased for
immediate download at www.Objective-Analysis.com.
• Orders can also be processed through Coughlin
Assocaites at:
http://www.tomcoughlin.com/techpapers.htm.
• You can contact Coughlin Associates by calling Tom at
408-871-8808, or e-mailing: Tom@TomCoughlin.com.
31
Analysts
32
Tom Coughlin, President, Coughlin Associates is a
highly-respected storage analyst and consultant with
over 30 years in the data storage industry in
engineering and management at high profile
companies.
Jim Handy is a widely recognized semiconductor
analyst, has over 30 years in the electronics industry.
His background includes marketing and design
positions at market-leading suppliers.Jim Handy
Objective Analysis
Thomas Coughlin
Coughlin Associates

17
Sources
• How Many IOPS do You Really Need?
• 2012 Digital Storage for Media and Entertainment, Coughlin
Associates: (www.tomcoughlin.com/techpapers)
• HDDs and Flash Memory: A Marriage of Convenience,
Coughlin Associates and Objective Analysis, 2011
(www.tomcoughlin.com/techpapers)
• Two may be Better than One: Why HDD and Flash Belong
• SNIA SSSI White Paper, Coughlin/Handy 2010
• Are Hybrid Drives Finally coming of Age?, Objective
Analysis, 2010
33

Data Center Survey: How Many IOPS Is Enough. Compares to site capacity and system latency.

Recommended

Recommended

More Related Content

Similar to Data Center Survey: How Many IOPS Is Enough. Compares to site capacity and system latency.

Similar to Data Center Survey: How Many IOPS Is Enough. Compares to site capacity and system latency. (20)

Recently uploaded

Recently uploaded (20)

Data Center Survey: How Many IOPS Is Enough. Compares to site capacity and system latency.