This document evaluates using either NFS or Fibre Channel (FC) storage for a VMware vSphere and vCloud Director 5.1 infrastructure with the following requirements: 37TB storage capacity, 25,000 IOPS, response times between 5-15ms. NFC storage allows larger datastores but has performance issues, while FC has better performance but smaller datastore sizes. The design decision is pending further analysis of risks around relocating customer images between datastores with either option.

1. vSphere Storage Type for vCloud
Director 5.1
NFS versus FC Storage
Design Decision
Draft 0.2
Prepared by David Pasek
Infrastructure Architect
14 August 2013

2. Problem statement
• Choose optimal type of storage (NFS x FC) for
vCloud Director 5.1 (vCD) infrastructure

3. Requirements
• Storage for VMware vSphere and vCloud Director 5.1 (vCD)
• Datastore for Virtual Machines managed by vCD
• Available Storage Capacity 37 TB
• Available front-end performance 25,000 IOPS (R/W Ratio 70:30)
• Common storage average response times between 5 and 15 ms
• Storage response times always bellow 25 ms
• VAAI Support
• Linked clones
• Thin provisioning yes - on disk array

4. Constraints
• FC Datastore max size 3 TB, Max Datastores 64
• NFS mounts 4
• NFS Datastore max size 10 TB
• Storage physical space up to 32 Rack Units
• Problematic vCD customer images relocation among datastores
• Customer’s VM Tiers (SLAs) are not defined by business. All have same
SLAs the difference is just CPU, RAM and disk capacity.

5. Assumption
• Ethernet Network is ready for high performance/throughput
NFS traffic

6. Options
• Option 1: NFS Storage
• Option 2: FC Storage

7. Option 1 (NFS) - advantages
• Bigger single datastore is advantage for vCD
and simplify storage management
• Significantly simpler datastore connection to
ESX hosts (no FC zonning and LUN masking per
ESX)
• NAS VAAI support and bigger datastore
improve linked clone efficiency.

8. Option 1 (NFS) - drawbacks
• NAS VAAI requires vCenter plugin
• Special ethernet network settings for NAS network (flow control,
Jumbo Frames, STP)
• Less volumes = less volume queues.
• Known performance issues and NFS volumes disconnections. The
workaround is to use SIOC or decrease Volume queues (VMware KB
article 2016122 ). This is not just NetApp issue but looks like ESX
NFS client issue. AFAIK it is still investigated. VMware TAM can find
more information before final decision.
• No MPIO from ESX to NFS volume because single NFS session can
be established per single path. LACP/port channeling doesn’t help
for IO multi-pathing from single ESX host to single NFS mount.

9. Option 2 (FC) - advantages
• VAAI works over FC via SCSI T10 commands
• FC is designed for storage purposes
• More volumes and more paths = more queues
= better performance in case of IO burst

10. Option 2 (FC) - drawbacks
• Smaller datastore size = worse balance of
vmdks. Datastore Clusters and SDRS can be
used to avoid this drawback. Doesn’t help to
vCD images (ISO images, OVA images).
• Linked clones cannot be offloaded to storage
and are done by vSphere on particular
datastore.

11. Options comparison
• Availability
– both options are highly available
• Manageability
– Option 1 [NFS] has positive impact on manageability because less
datastores and simple mounting over IP
• Performance
– Option 2 [FC] is better from general storage performance perspective
(MPIO, queues, FC protocol)
– Option 1 [NFS] has better performance of linked clones because of
hardware offload
• Recoverability
– Option 2 [FC] is better because less eggs in the basket
• Security
– Neither option has impact on security

12. Design decision and Justification
• TBD as a home work 

13. Design Risks & Implications
• vCD cannot relocate customer image files for
any storage type. NFS will help little bit with
bigger datastore size but it is not resolution
because image migration procedure is
eventually needed for any datastore NFS or
FC.

14. End Of Decision Point
SOME TECHNICAL MAXIMUM
INFORMATION FOLLOWS
JUST TO SUPPORT AND JUSTIFY DECISION

15. Additional NFS Technical Information
• NetApp FAS
• EMC Celerra

16. NetApp NFS Storage maximums
• Max Number of Volumes – 64
• Max Volume size (64 bit aggregates required)
– FAS 3xxx - 50 TB
– FAS 3170, 6030, 6040 - 70 TB
– FAS 6070, 6080 – 100 TB
• Max Spindles per aggregate
– FAS 3xxx – SATA 2TB;30, 1TB;61, 0.75TB; 82; 0.5TB; 123 – SAS 600GB;93, 450GB;125, 300GB; 192
– FAS 3170, 6030, 6040 – SATA 2TB;43, 1TB;86, 0.75TB; 115; 0.5TB; 173 – SAS 600GB;131,
450GB;175, 300GB; 269
– FAS 6070, 6080 – 100 TB – SATA 2TB;61, 1TB;123, 0.75TB; 164; 0.5TB; 247 – SAS
600GB;187, 450GB;250, 300GB; 385
• Max Spindles per array – depends on model from 480 up to 1440

17. EMC Celerra NFS Storage maximums
• TBD