State of the Art Thin Provisioning

State of the Art Thin Provisioning
Stephen Foskett
stephen@fosketts.net
twitter.com/sfoskett

Storage Is Supposed To Be Getting Cheaper!
Disk cost is dropping rapidly
$250 buys:
1994: 2 GB
1999: 20 GB
2004: 200 GB
2009: 2000 GB
But enterprise storage costs keep rising!
2

Where Is The Cost?
Hardware and software make up a small percentage of total enterprise storage spending…
…and hard disk drive capacity makes up a small percentage of that!
Data center/environmental, administrative personnel, maintenance, and data protection are much bigger
The biggest opportunity is inefficiency, but this has always been hard to tackle
3

Over-Allocation and Under-Utilization
4
Raw Disk Capacity Purchased
Conventional storage provisioning is grossly inefficient
Usable Protected Storage
Capacity Allocated to Servers
Requested Capacity
Used by Files
Required Capacity

Thin Provisioning Simplified!
5
Traditional storage provisioning
Thin storage provisioning
Allocated but unused
Free for allocation
Actually Used
Used

Thin Provisioning: Potentially Problematic
Storage is commonly over-allocated to servers
Some arrays can “thinly” provision just the capacity that actually contains data
500 GB request for new project, but only 2 GB of initial data is written – array only allocates 2 GB and expands as data is written
What’s not to love?
Oops – we provisioned a petabyte and ran out of storage
Chunk sizes and formatting conflicts
Can it thin unprovision?
Can it replicate to and from thin provisioned volumes?

Are You Solving a Technical or Business Issue?
7

Ever Play the “Telephone” Game?
Application
IV
File/Record Layer
File System
Database
III
Each layer obscures the ones above and below it
IIc
Block
Aggregation
Host
IIb
Network
Device
IIa
Storage Devices
I
SNIA Shared Storage Model

File System
It’s (Relatively) Easy to Allocate on Write
9
As applications write data
Storage
Capacity is allocated
File system write requests pass through to storage systems
so they can wait to allocate as requested

File System
But What About De-Allocate on Delete?
10
Data is deleted
Storage
Capacity is freed up
Most file systems don’t send a consistent “de-allocate” message to storage
so many thin systems get fatter over time

Two Approaches To Thin
11

Server Smarts: Metadata Monitoring
File system/VM combos can handle thin provisioning on their own
ZFS, Veritas Volume Manager, VMware VMFS
Arrays can “watch” an operating system allocate and de-allocate storage
Perilous! Known file systems and volume formats only!
Data Robotics Drobo supports FAT32, NTFS, HFS+
12
Drobo watches the file allocation table for deletes
File System
Storage

Storage Smarts: Zero Page Reclaim
Storage arrays watch for “pages” containing all zeros and simply don’t write them
IBM XIV, 3PAR, NetApp (with dedupe), HDS, EMC V-Max
Some storage vendors rely on utilities to reclaim
NetApp SnapDrive for Windows 5.0
Compellent Free Space Recovery
Veritas Storage Foundation Thin Reclamation
Can also force it with sdelete
13

Zero Page Reclaim: Pros and Cons
Pro:
Straightforward to implement in storage
Some implementation: VMware eagerzeroedthick
Con:
Requires application/OS/file system to actually have written all zeroes - most just ignore unused space rather than zeroing
Most implementations are page-based
Drives more I/O
VMware thin/thick don’t work
14

The Lingo: WRITE_SAME
Facilitates zero page reclaim
“Write this block 1,000,000 times”
Pro:
Conserves I/O operations
Popular with array vendors
Exists and is even implemented (a little)
Con:
Depends on file system layer intelligence
Still introduces extra I/O
Could be very, very bad in a thin-unaware array
15

The Bridge: Veritas Thin API
Thin Reclamation API can communicate de-allocation to arrays by zeroing using WRITE_SAME/UNMAP
Introduced in 5.0 (UNIX) and 5.1 (Windows)
Supports 3PAR, EMC CLARiiON CX4, HDS USPV/VM, HP XP20k/24k, IBM XIV
Will also support Compellent, EMC Symmetrix DMX, Fujitsu Eternus, HP EVA, HDS AMS, IBM DS8k, NetApp
SmartMove copies only allocated blocks
Supports any/all storage systems
Works with thin-capable arrays
Speeds up migrations in all cases
16

What About TRIM?
TRIM (ATA) and TRIM/UNMAP/PUNCH (SCSI) can inform storage that a block is no longer needed
Designed for SSD architecture:
Cells grouped into 4 kB pages and 512 kB blocks
Only empty pages can be written to
Writing to empty pages is quick!
Writing to used pages requires a block erase
Read-erase-write is slow(er)
OS support for TRIM:
Windows 7 & Server 2008 R2
Linux 2.6.33, Open Solaris, FreeBSD 9
17

TRIM Isn’t For Thin
Not really a thin-provisioning command but could play one on TV
NetApp proposed a hole punching standard to INCITS T10 committee
HDS and EMC prefer UNMAP bit
A similar NetApp approach uses NFS and a Windows file system redirect

More Obstacles!

Large page – no thin provisioning
Granularity (Page Sizes)
20
Small page – thin even with fragmentation

Processing and Scheduling
21
Intensive
Ineffective

Fragmentation Kills Thin Provisioning
22
Fragmented file
system spans
thin pages
Defragmented
file system allows
thin provisioning

The Performance Crunch
How high can we drive utilization without killing performance?

Stephen’s Dream
Thin provisioning could be awesome, provided it is integrated at all levels of the stack
Smart applications that don’t spew data everywhere
Smart file systems and volume managers that communicate what is and isn’t used
Smart virtualization layers that don’t obscure usage
Smart storage systems that act on all of this information with granularity and without falling over dead
Smart monitoring systems to tie everything together and head off disaster

Thank You!
Stephen Foskett
stephen@fosketts.net
twitter.com/sfoskett
+1(508)451-9532
FoskettServices.com
blog.fosketts.net
GestaltIT.com
25

State of the Art Thin Provisioning

by Stephen Foskett on Dec 23, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

Statistics

State of the Art Thin Provisioning — Presentation Transcript