ISUG 113: File stream
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
2,053
On Slideshare
1,796
From Embeds
257
Number of Embeds
1

Actions

Shares
Downloads
8
Comments
0
Likes
0

Embeds 257

http://www.sqlserver.co.il 257

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. File StreamMicrosoft SQL Server 2008 R2 Kobi Cohen System DBA Manager Spiral Solutions
  • 2. Objectives  Familiarization with FILESTREAM  Overview  Storage  Security & Access  Implementation  Application Considerations  Combined with other SQL Server features  Cluster Configuration  RBS (Remote BLOB store)©Spiral Solutions, Ltd. All rights reserved.
  • 3. File Stream Overview  For unstructured data  docs, images, videos.  not new data type  Storage attribute.  Varbinary (max) only  Fully transactional.  BLOBs saved on storage (NTFS file system)  Needs coordination between DBA and System Administrator.  Can be accessed by T-SQL (DMLs) or Win32 API (stream access).  Use NT system cache:  Much faster  Avoid double writing (Buffer cache + Disk).  Reduce load from DB engine.  SQL Server buffer pool isn’t used  more free memory.  BLOB size limited only by the volume size (not limited to 2GB) even in Express Edition.©Spiral Solutions, Ltd. All rights reserved.
  • 4. File Stream Overview  Store BLOBs in table? On separate files? or in FILESTREAM?  Data size   Larger than 1 MB  NTFS FILESTREAM.  Smaller than 256 KB  inside the database.  Between 256 KB and 1MB  depend on read vs. write ratio + overwrite rate.  Performance  Fast read access is important NTFS FILESTREAM.  Transactional Consistency  BLOB + structured data  FILESTREAM.  Security  sensitive data  FILESTREAM  SQL security over ACLs.  Manageability  integrated solution  less complex  FILESTREAM.  Data fragmentation  frequent updates  easier to handle on NTFS.  Client access  Using DB access protocols (ODBC, OLEDB)  NTFS FILESTREAM.©Spiral Solutions, Ltd. All rights reserved.
  • 5. File Stream Overview BLOB Storage Methods Comparison Storage solution Comparison point SQL Server (using File server / file system FILESTREAM varbinary(max)) Maximum BLOB size NTFS volume size 2 GB – 1 bytes NTFS volume size Streaming performance of large Excellent Poor Excellent BLOBs Security Manual ACLs Integrated Integrated + automatic ACLs Cost per GB Low High Low Manageability Difficult Integrated Integrated Integration with structured data Difficult Data-level consistency Data-level consistency Application development and More complex More simple More simple deployment Recovery from data fragmentation Excellent Poor Excellent Performance of frequent small Excellent Moderate Poor updates©Spiral Solutions, Ltd. All rights reserved.
  • 6. File Stream Overview Read Performance Comparison (Cold buffer pool)©Spiral Solutions, Ltd. All rights reserved.
  • 7. File Stream Overview Read Performance Comparison (Warm buffer pool)©Spiral Solutions, Ltd. All rights reserved.
  • 8. File Stream Overview Insert Performance Comparison (Cold buffer pool)©Spiral Solutions, Ltd. All rights reserved.
  • 9. File Stream Overview Update Performance Comparison (Cold buffer pool)©Spiral Solutions, Ltd. All rights reserved.
  • 10. File Stream Storage  Preparing for FILESTREAM:  Set the proper Disk/RAID level  resilience, performance, cost, load.  Use FSUTIL :  Turn off short (8.3) file names.  Disable last access time.  Format volume cluster size  Use 64-KB NTFS clusters  4-KB for compressed volumes.  Compress volume  extra CPU.  Regularly defrag FILESTREAM computer systems.  Set antivirus scan of FILESTREAM volumes to avoid delete infected files  quarantine if it’s a must.©Spiral Solutions, Ltd. All rights reserved.
  • 11. File Stream Storage  FILESTREAM stored in special file groups:  File system filter per each volume  one for SQL Server version  Integrated security is a must.  File system directories (data containers) instead of data files (mdf, ndf).  FILESTREAM file group = FILESTREAM data container.  Adding file to FILESTREAM file group = define data container:  Each logical file name is the root directory name.  Filestream.hdr  FILESTREAM metadata describing data container.  $FSLOG  FILESTREM equivalent to transaction log.  FILESTREAM operation  mapped to LSN.  FILESTREAM garbage collection thread  triggered by CHECKPOINT.  DBCC CHECKDB  for validates link-level consistency.©Spiral Solutions, Ltd. All rights reserved.
  • 12. File Stream Storage  File stream storage considerations:  Separate FILESTREAM data containers from:  OS volumes and paging file volumes.  Tempdb volumes.  Databases data & log files volumes.  Each data container on separate volume  granular control.  Data containers cannot be nested.  All FILSTREAM columns data in a table must be stored in same FILESTREAM file group  FILESTREAM_ON better performance.  Each row must have non-null unique row id (uniqueidentifier ROWGUID) and a unique/primary key index (clustered only for non-random data to avoid index reordering on every insert/delete).  Can use partitioning on uniqueidentifier column for load balancing.©Spiral Solutions, Ltd. All rights reserved.
  • 13. File Stream Security & Access  FILESTREAM is secured like other data:  GRANT permission on DB, schema, table or column.  NTFS permissions (ACLs) are ignored.  Only SQL Server service account is granted NTFS access to data containers to avoid outside data editing.  Requires integrated security (Windows authentication)  Windows token  pass client identity in file I/O operation.  Support snapshot isolation level (only in 2008 R2).  Remote access By Server Message Block (SMB) protocol.  Firewall considerations:  Enable both client & server to resolve DNS names.  Open file-sharing TCP ports: 139, 445 (for remote access).©Spiral Solutions, Ltd. All rights reserved.
  • 14. File Stream Security & Access  Dual programing model to access BLOBs:  T-SQL for DMLs:  Insert  can be used with null value, empty (‘’) value or short inline data.  Update  if FILESTREAM field is set to null, the related BLOB is deleted. Partial field update can’t be done with T-SQL (use UPDATE.Write()) since update = create new copy of FILESTREAM data file + delete old file. Partial update only by: FSCTL_SQL_FILESTREAM_FETCH_OLD_CONTENT  Delete or Truncate  underlined BLOB data is deleted in file system.  File system streaming access (for read & write):  Statement model  models T-SQL statements by using open & close methods  File handle is live in that scope  open handle = open transaction.  Storage namespace  DB engine controls BLOB physical file  new internal function column_name.PathName(@option) used by API to provide the logical UNC path of the BLOB to get the Win32 handle.©Spiral Solutions, Ltd. All rights reserved.
  • 15. File Stream Security & Access  Transacted file system access  New function provides token for current session’s transaction GET_FILESTREAM_TRANSACTION_CONTEXT ()  binds file streaming operations with a started transaction. Return NULL if there is no explicitly started transaction.  In C++/ODBC  SQL Native Client 10  OpenSqlFilestream  Win32 handle.  In C#  .NET Framework 3.5 SP1  SqlFileStream Data Type  path name + transaction token  handle can be passed to Win32 File API.  Write-through from remote client  No write operation are cached on client side  better to make fewer writes with larger data size.©Spiral Solutions, Ltd. All rights reserved.
  • 16. File Stream Demo Demo©Spiral Solutions, Ltd. All rights reserved.
  • 17. File Stream Application Considerations  Use NULL instead of 0x for non-initialized FILESTREAM column to avoid file creation.  Avoid insert & update in tables with NOT NULL FILESTREAM columns  decrease application performance over time.  In replication  use NEWSEQUENTIALID() instead of NEWID()  for better performance.  Avoid T-SQL for update, append, prepend data to FILESTREAM BLOB  will be spooled into tempdb.  Avoid appending small BLOB updates  Causes FILESTREAM files to be copied  Write BLOB into varbinary(max) parameter and do single write operation.  Avoid getting lots of BLOB files length in application use DATALENGTH() in T-SQL  does not open the BLOB.©Spiral Solutions, Ltd. All rights reserved.
  • 18. File Stream Combined with other features  Snapshots  Only by excluding FILESTREAM file groups  FILESTREAM file groups will be marked as offline.  Replication Supported by both transactional & merge with many considerations  SQL 2008 or later on all subscribers.  Log Shipping  supports FILESTREAM.  Mirroring  Does not support FILESTREAM.  Clustering  fully supported. Must enable FILESTREAM on all nodes  Backup & Restore  Fully supported on all recovery models.  Encryption  is not supported even when TDE is enabled.  Indexes  cannot be a part of: index key / include column / statistics.  Constraints (PK,FK,UQ) cannot be created on FILESTREAM column.  Temp tables & TVP  cannot be used in local/global temp tables & TVP.©Spiral Solutions, Ltd. All rights reserved.
  • 19. File Stream Cluster Configuration  Data containers must be on shared disk resource.  Separate LUN for FILESTREAM is recommended.  Steps for enabling FILESTREAM:  Enable file stream on service level  on all nodes.  Windows Share Name must be the same on all nodes.  Allow remote clients access  create a file-share cluster resource.  Enable file stream on instance level (facets)  on all nodes.  Add dependency of FILESTREAM LUN to SQL Service.  Take SQL Service off & Bring SQL Service + dependencies on.  NetApp Snap Manager requirements:  Separate LUNs to data files, Log files & FILESTREAM files.©Spiral Solutions, Ltd. All rights reserved.
  • 20. File Stream RBS  Complementary approach  store BLOBs on remote storage.  Provided with SQL Server 2008 R2 Feature pack.  library API set  moving BLOBs from MS SQL to external storage.  Set of SPs and SDK  RBS provider for File stream.  Client-side DLL linked into a user application + stored procedures on SQL Server.©Spiral Solutions, Ltd. All rights reserved.
  • 21. File Stream Useful Links  FILESTREAM Storage in SQL Server 2008: http://msdn.microsoft.com/en-us/library/cc949109(v=sql.100).aspx  To BLOB or Not To BLOB - Large Object Storage in a Database or a File system: http://research.microsoft.com/apps/pubs/default.aspx?id=64525  How to enable FILESTREAM from the command line: http://sqlsrvengine.codeplex.com/wikipage?title=FileStreamEnable&referri ngTitle=Home&ProjectName=sqlsrvengine  FSUTIL syntax: http://technet.microsoft.com/en-us/library/cc785435(v=ws.10).aspx  SQL Server 2008 R2 Remote Blob Store (RBS) Download : http://www.microsoft.com/download/en/confirmation.aspx?id=16978©Spiral Solutions, Ltd. All rights reserved.
  • 22. File Stream End Thanks©Spiral Solutions, Ltd. All rights reserved.