1. File Stream
Microsoft SQL Server
2008 R2
Kobi Cohen
System DBA Manager
Spiral Solutions

2. Objectives
 Familiarization with FILESTREAM
 Overview
 Storage
 Security & Access
 Implementation
 Application Considerations
 Combined with other SQL Server features
 Cluster Configuration
 RBS (Remote BLOB store)
©Spiral Solutions, Ltd. All rights reserved.

3. File Stream
Overview
 For unstructured data  docs, images, videos.
 not new data type  Storage attribute.
 Varbinary (max) only  Fully transactional.
 BLOBs saved on storage (NTFS file system)
 Needs coordination between DBA and System Administrator.
 Can be accessed by T-SQL (DMLs) or Win32 API (stream access).
 Use NT system cache:
 Much faster  Avoid double writing (Buffer cache + Disk).
 Reduce load from DB engine.
 SQL Server buffer pool isn’t used  more free memory.
 BLOB size limited only by the volume size (not limited to 2GB)
even in Express Edition.
©Spiral Solutions, Ltd. All rights reserved.

4. File Stream
Overview
 Store BLOBs in table? On separate files? or in FILESTREAM?
 Data size 
 Larger than 1 MB  NTFS FILESTREAM.
 Smaller than 256 KB  inside the database.
 Between 256 KB and 1MB  depend on read vs. write ratio + overwrite rate.
 Performance  Fast read access is important NTFS FILESTREAM.
 Transactional Consistency  BLOB + structured data  FILESTREAM.
 Security  sensitive data  FILESTREAM  SQL security over ACLs.
 Manageability  integrated solution  less complex  FILESTREAM.
 Data fragmentation  frequent updates  easier to handle on NTFS.
 Client access  Using DB access protocols (ODBC, OLEDB)  NTFS
FILESTREAM.
©Spiral Solutions, Ltd. All rights reserved.

5. File Stream
Overview
BLOB Storage Methods Comparison
Storage solution
Comparison point SQL Server (using
File server / file system FILESTREAM
varbinary(max))
Maximum BLOB size NTFS volume size 2 GB – 1 bytes NTFS volume size
Streaming performance of large
Excellent Poor Excellent
BLOBs
Security Manual ACLs Integrated Integrated + automatic ACLs
Cost per GB Low High Low
Manageability Difficult Integrated Integrated
Integration with structured data Difficult Data-level consistency Data-level consistency
Application development and
More complex More simple More simple
deployment
Recovery from data fragmentation Excellent Poor Excellent
Performance of frequent small
Excellent Moderate Poor
updates
©Spiral Solutions, Ltd. All rights reserved.

6. File Stream
Overview
Read Performance Comparison (Cold buffer pool)
©Spiral Solutions, Ltd. All rights reserved.

7. File Stream
Overview
Read Performance Comparison (Warm buffer pool)
©Spiral Solutions, Ltd. All rights reserved.

8. File Stream
Overview
Insert Performance Comparison (Cold buffer pool)
©Spiral Solutions, Ltd. All rights reserved.

9. File Stream
Overview
Update Performance Comparison (Cold buffer pool)
©Spiral Solutions, Ltd. All rights reserved.

10. File Stream
Storage
 Preparing for FILESTREAM:
 Set the proper Disk/RAID level  resilience, performance, cost, load.
 Use FSUTIL :
 Turn off short (8.3) file names.
 Disable last access time.
 Format volume cluster size  Use 64-KB NTFS clusters  4-KB for
compressed volumes.
 Compress volume  extra CPU.
 Regularly defrag FILESTREAM computer systems.
 Set antivirus scan of FILESTREAM volumes to avoid delete
infected files  quarantine if it’s a must.
©Spiral Solutions, Ltd. All rights reserved.

11. File Stream
Storage
 FILESTREAM stored in special file groups:
 File system filter per each volume  one for SQL Server
version  Integrated security is a must.
 File system directories (data containers)
instead of data files (mdf, ndf).
 FILESTREAM file group = FILESTREAM data container.
 Adding file to FILESTREAM file group = define data container:
 Each logical file name is the root directory name.
 Filestream.hdr  FILESTREAM metadata describing data container.
 $FSLOG  FILESTREM equivalent to transaction log.
 FILESTREAM operation  mapped to LSN.
 FILESTREAM garbage collection thread  triggered by CHECKPOINT.
 DBCC CHECKDB  for validates link-level consistency.
©Spiral Solutions, Ltd. All rights reserved.

12. File Stream
Storage
 File stream storage considerations:
 Separate FILESTREAM data containers from:
 OS volumes and paging file volumes.
 Tempdb volumes.
 Databases data & log files volumes.
 Each data container on separate volume  granular control.
 Data containers cannot be nested.
 All FILSTREAM columns data in a table must be stored in same
FILESTREAM file group  FILESTREAM_ON better performance.
 Each row must have non-null unique row id (uniqueidentifier
ROWGUID) and a unique/primary key index (clustered only for
non-random data to avoid index reordering on every insert/delete).
 Can use partitioning on uniqueidentifier column for load balancing.
©Spiral Solutions, Ltd. All rights reserved.

13. File Stream
Security & Access
 FILESTREAM is secured like other data:
 GRANT permission on DB, schema, table or column.
 NTFS permissions (ACLs) are ignored.
 Only SQL Server service account is granted NTFS access to data
containers to avoid outside data editing.
 Requires integrated security (Windows authentication) 
Windows token  pass client identity in file I/O operation.
 Support snapshot isolation level (only in 2008 R2).
 Remote access By Server Message Block (SMB) protocol.
 Firewall considerations:
 Enable both client & server to resolve DNS names.
 Open file-sharing TCP ports: 139, 445 (for remote access).
©Spiral Solutions, Ltd. All rights reserved.

14. File Stream
Security & Access
 Dual programing model to access BLOBs:
 T-SQL for DMLs:
 Insert  can be used with null value, empty (‘’) value or short inline data.
 Update  if FILESTREAM field is set to null, the related BLOB is deleted.
Partial field update can’t be done with T-SQL (use UPDATE.Write()) since
update = create new copy of FILESTREAM data file + delete old file.
Partial update only by: FSCTL_SQL_FILESTREAM_FETCH_OLD_CONTENT
 Delete or Truncate  underlined BLOB data is deleted in file system.
 File system streaming access (for read & write):
 Statement model  models T-SQL statements by using open & close
methods  File handle is live in that scope  open handle = open
transaction.
 Storage namespace  DB engine controls BLOB physical file  new
internal function column_name.PathName(@option) used by API to provide
the logical UNC path of the BLOB to get the Win32 handle.
©Spiral Solutions, Ltd. All rights reserved.

15. File Stream
Security & Access
 Transacted file system access  New function provides token for current
session’s transaction GET_FILESTREAM_TRANSACTION_CONTEXT () 
binds file streaming operations with a started transaction.
Return NULL if there is no explicitly started transaction.
 In C++/ODBC  SQL Native Client 10  OpenSqlFilestream  Win32 handle.
 In C#  .NET Framework 3.5 SP1  SqlFileStream Data Type  path name +
transaction token  handle can be passed to Win32 File API.
 Write-through from remote client  No write operation are cached on client
side  better to make fewer writes with larger data size.
©Spiral Solutions, Ltd. All rights reserved.

16. File Stream
Demo
Demo
©Spiral Solutions, Ltd. All rights reserved.

17. File Stream
Application Considerations
 Use NULL instead of 0x for non-initialized FILESTREAM
column to avoid file creation.
 Avoid insert & update in tables with NOT NULL FILESTREAM
columns  decrease application performance over time.
 In replication  use NEWSEQUENTIALID() instead of NEWID() 
for better performance.
 Avoid T-SQL for update, append, prepend data to FILESTREAM
BLOB  will be spooled into tempdb.
 Avoid appending small BLOB updates  Causes FILESTREAM
files to be copied  Write BLOB into varbinary(max) parameter
and do single write operation.
 Avoid getting lots of BLOB files length in application use
DATALENGTH() in T-SQL  does not open the BLOB.
©Spiral Solutions, Ltd. All rights reserved.

18. File Stream
Combined with other features
 Snapshots  Only by excluding FILESTREAM file groups 
FILESTREAM file groups will be marked as offline.
 Replication Supported by both transactional & merge
with many considerations  SQL 2008 or later on all subscribers.
 Log Shipping  supports FILESTREAM.
 Mirroring  Does not support FILESTREAM.
 Clustering  fully supported. Must enable FILESTREAM on all nodes
 Backup & Restore  Fully supported on all recovery models.
 Encryption  is not supported even when TDE is enabled.
 Indexes  cannot be a part of: index key / include column / statistics.
 Constraints (PK,FK,UQ) cannot be created on FILESTREAM column.
 Temp tables & TVP  cannot be used in local/global temp tables & TVP.
©Spiral Solutions, Ltd. All rights reserved.

19. File Stream
Cluster Configuration
 Data containers must be on shared disk resource.
 Separate LUN for FILESTREAM is recommended.
 Steps for enabling FILESTREAM:
 Enable file stream on service level  on all nodes.
 Windows Share Name must be the same on all nodes.
 Allow remote clients access  create a file-share cluster resource.
 Enable file stream on instance level (facets)  on all nodes.
 Add dependency of FILESTREAM LUN to SQL Service.
 Take SQL Service off & Bring SQL Service + dependencies on.
 NetApp Snap Manager requirements:
 Separate LUNs to data files, Log files & FILESTREAM files.
©Spiral Solutions, Ltd. All rights reserved.

20. File Stream
RBS
 Complementary approach  store BLOBs on remote storage.
 Provided with SQL Server 2008 R2 Feature pack.
 library API set  moving BLOBs from MS SQL to external storage.
 Set of SPs and SDK  RBS provider for File stream.
 Client-side DLL linked into a user application + stored procedures on
SQL Server.
©Spiral Solutions, Ltd. All rights reserved.

21. File Stream
Useful Links
 FILESTREAM Storage in SQL Server 2008:
http://msdn.microsoft.com/en-us/library/cc949109(v=sql.100).aspx
 To BLOB or Not To BLOB - Large Object Storage in a Database or
a File system:
http://research.microsoft.com/apps/pubs/default.aspx?id=64525
 How to enable FILESTREAM from the command line:
http://sqlsrvengine.codeplex.com/wikipage?title=FileStreamEnable&referri
ngTitle=Home&ProjectName=sqlsrvengine
 FSUTIL syntax:
http://technet.microsoft.com/en-us/library/cc785435(v=ws.10).aspx
 SQL Server 2008 R2 Remote Blob Store (RBS) Download :
http://www.microsoft.com/download/en/confirmation.aspx?id=16978
©Spiral Solutions, Ltd. All rights reserved.

22. File Stream
End
Thanks
©Spiral Solutions, Ltd. All rights reserved.