To BLOB or Not to BLOB?         Dan Holme                                                                       Randy Will...
Agenda What are BLOBs? How BLOBs affect SharePoint storage Options for externalizing BLOBs Benefits & drawbacks of externa...
Content lifecycle                           Birth                        (Creation)                     Life              ...
BLOBs Documents, databases, and BLOBs                                   SharePoint                                   SQL S...
SharePoint Documents in SQL
BLOBs in the AllDocStreams table
The burden of BLOBs       Typically, as much as 80 percent of       data for an enterprise-scale       deployment of Share...
The SharePoint storage dilemma Documents, databases, and BLOBs Storage growth                                             ...
Planning for SharePoint storage Planning for implementation and future growth From: Physical Storage Recommendations (Shar...
Content scaling support & guidance SharePoint 2010 RTM              July 2011     Content Database               Content D...
How big are your  biggest content  databases?0-100 GB100-200 GB200-500 GBMore than 500 GBI have no idea
Should I externalizeBLOBs?
Advantages of keeping BLOBs in SQL One storage container to   Maintain   Monitor   Recover Tier I storage   Performance re...
Disadvantages of large CDBs & BLOBs Storage cost            SQL operations   Content                 Locking     Current v...
Overview of BLOB externalization                                              Pointer                                     ...
Options for externalizing External BLOB Storage (EBS)   Released with SharePoint 2007 SP1   Supported by SharePoint 2010 R...
SharePoint Documents in SQL
Externalized BLOBs in AllDocStreams
BLOB store
Externalized BLOBs are transparent Check-in/Check-out Versioning Office applications Search crawling (indexing) Workflows ...
Advantages of externalizing Storage Cost Performance Storage Cost                Performance
Why Are BLOBS a Problem?           Maintaining large quantities of BLOB           data in a SQL Server database is a      ...
Advantages of externalizing Storage Cost Performance        Read*                Large documents        Write*        All ...
What’s the performance gain?                                     SQL BLOB         RBS       Gain Database Size - 1 TB     ...
Advantages of externalizing Storage Cost Performance        Read*        Write*        All other activity by users of the ...
EBS and RBS                        EBS                  RBSAPI                     SharePoint           SQL + SharePoint c...
Choosing third-party externalization Performance BLOB store platform    File system, SAN, NAS    Shared folder    Cloud   ...
All things considered                              Storage Cost             Archiving and                                 ...
Content lifecycle                           Birth                        (Creation)                     Life              ...
Content lifecycle meets RBS RBS determines whether BLOB gets externalized to another storage platform on upload Archiving ...
Externalization: Support and storage                                                                      THIRDFEATURE    ...
Externalization: Backup, recovery, DR                                                              THIRDFEATURE           ...
Externalization: Content lifecycle support                                                                     THIRDFEATUR...
Resources SharePoint Team Blog: Data Storage Changes in SP1   http://tinyurl.com/3rlvfnp MSDN: External Storage of Binary ...
Dan Holme                Randy Williamsdan.holme@avepoint.com   randy.williams@avepoint.com     Submit your questions!
© 2011 AvePoint, Inc. All rights reserved.No part of this may be reproduced, stored in a retrieval system, or transmitted ...
Upcoming SlideShare
Loading in...5
×

To blob or not to blob

1,322

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,322
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
24
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • ABSTRACTSharePoint and SQL Server offer several methods with which you can externalize the BLOBs—the files that are stored in libraries and as attachments—to reduce the size of your content databases. BLOB externalization offers benefits beyond reduced storage cost—benefits that may surprise you! But there's a lot of hype and misunderstanding about BLOB externalization, which is neither a silver bullet nor evil incarnate. Join SharePoint MVPs Dan Holme and Randy Williams for a balanced, intelligent, detailed examination of the technologies and issues surrounding BLOB externalization. Dive deep into the performance impact, scalability implications, and critical considerations for backup and restore, high availability, and disaster recovery. To BLOB or not to BLOB? There are no easy answers, but after this session you will have what it takes to maximize the potential of BLOB externalization in your storage architecture.
  • IT DEPENDS!!!
  • Introduce concept of documents being stored as BLOBs in CDBBUILD: Diagram of architectureDiscuss storage growthBUILD: Bloat of data, mostly inactiveBUILD: Burden on CDBsDiscuss need to thin about storage holistically: lifecycle, compliance, SLAs, cost
  • SharePoint Content Databases leverage a few tables related to documents:dbo.AllUserDataContains metadata for all items (list and library) in the content databaseIdentifiers for any documentdbo.AllDocsContains references to a document along with metadata about document. This metadata is internal SharePoint metadata—not user metadata (which is in AllUserData)dbo.AllDocVersionsIf versioning is enabled, this table contains metadata about the version.Identifier for the streamdbo.AllDocStreamsContains the document itself (or a reference to the remoted BLOB if using RBS)Identifier for the streamReferencehttp://msdn.microsoft.com/en-us/library/dd303586(v=PROT.13).aspx
  • Introduce concept of documents being stored as BLOBs in CDBBUILD: Diagram of architectureDiscuss storage growthBUILD: Bloat of data, mostly inactiveBUILD: Burden on CDBsDiscuss need to thin about storage holistically: lifecycle, compliance, SLAs, cost
  • We can estimate the amount of storage space necessary by doing a few simple calculations.Let’s assume a medium sized organization with 1,000 users is rolling out SharePoint. Assuming the system is well-adopted, users are adding 1MB per day into the farm.After a year, that’s 250GB, and we haven’t even discussed Versioning.Don’t forget that 100 versions of a 10MB Word Document will ultimately take up 1GB of storage in your content database.Also, the central-admin recycle bin settings default to storing 50% of the live sites’ data for second-stage deleted itemsThen there’s the SQL Server transaction logsSee how SharePoint databases can grow so quickly?
  • Discuss the challenges of RTM guidance: what was “guidance” and what was “support”?CONDITIONS APPLYContent databases of up to 4 TB are supported when the following requirements are met:Disk sub-system performance:0.25 IOPs per GB minimum2.00 IOPs per GB recommended for optimal performanceTTFB of 20msArchitecture and tools must support performance expectations, future capacity, backup, restore, high availability, disaster recoveryDiscussion: Does anyone have more than a terabyte of data in their farm? Does anyone have a database larger than 200GB? Are there any negative performance impacts? Does anyone have 2GB / 1GB / 500MB files stored in SharePoint? How do they perform? How fast is your SharePoint farm growing? If you haven’t deployed SharePoint, how do you know how much storage you’ll need?
  • Out of the box, RBS monitors for file uploads into SharePoint and checks the file size. If it’s over a certain configurable threshold, the upload will be split apart:The metadata goes to the databaseThe file goes to a file shareThe end user never knows the difference. All normal SharePoint features (site quotas, workflows, etc) still apply to the content.
  • There was a note on this slide:Cannot have both providers in “write” modeWhat’s this? I’m not familiar with this, though it does kind of make sense… What’s the scope? Per WFE/CDB?
  • Optimizing SharePoint storage improves performance. Here are the results of a Microsoft Whitepaper on RBS performance that you can download from their website. I’ll provide the link at the end of this session.Database SizeBy optimizing the database, the database size was reduced by over 98% which reduced the backup size and consequently, the backup time.Index RebuildsOccasionally SharePoint will rebuild the database indexes to prevent fragmentation, when it does the farm typically becomes unavailable. Externalizing the BLOBs via RBS significantly helps alleviate this problem because the smaller database requires less time to rebuild the indexes.Avg. SharePoint Response TimeEnabling the RBS feature results in smaller SharePoint content databases that in turn require fewer resources on the SQL Server database server to execute the queries. The saved resources are freed up to process the existing queries faster and to service more queries.Given that productivity and satisfaction of SharePoint users are often dependent on the SharePoint transaction response times, a 25% reduction in response times would result in higher levels of productivity and satisfaction.Large File Uploads:People won’t use SharePoint for large files if the uploads take too long. They’ll stick to file shares (BAD). RBS significantly boosts the performance of file uploads for large files because it doesn’t store the BLOB in the database and therefore neither the database overhead nor the double write penalties apply.
  • To blob or not to blob

    1. 1. To BLOB or Not to BLOB? Dan Holme Randy Williams Chief SharePoint Evangelist Enterprise Trainer & Evangelist dan.holme@avepoint.com randy.williams@avepoint.com @danholme @tweetraw © 2011 AvePoint, Inc. All rights reserved.No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint, Inc.
    2. 2. Agenda What are BLOBs? How BLOBs affect SharePoint storage Options for externalizing BLOBs Benefits & drawbacks of externalization Understanding the Big Picture FILESTREAM provider vs third party Q&A
    3. 3. Content lifecycle Birth (Creation) Life (Use) Death (Disposition)
    4. 4. BLOBs Documents, databases, and BLOBs SharePoint SQL Server 2008/R2
    5. 5. SharePoint Documents in SQL
    6. 6. BLOBs in the AllDocStreams table
    7. 7. The burden of BLOBs Typically, as much as 80 percent of data for an enterprise-scale deployment of SharePoint Foundation consists of file-based data streams that are stored as BLOB data. These BLOB objects comprise data associated with SharePoint files.msdn.microsoft.com/en-us/library/bb802976.aspx
    8. 8. The SharePoint storage dilemma Documents, databases, and BLOBs Storage growth SharePoint SQL Server 2008/R2 Content Database Content Database Content Database Active Content Actual Content
    9. 9. Planning for SharePoint storage Planning for implementation and future growth From: Physical Storage Recommendations (SharePoint Server) http://technet.microsoft.com/en-us/library/cc298801.aspx
    10. 10. Content scaling support & guidance SharePoint 2010 RTM July 2011 Content Database Content Database 200 GB (collaboration) 200 GB (out-of-box) 1 TB (Records Center) 4 TB (collaboration)* Unlimited (archive)* Site Collection Site Collection 100 GB 100 GB (out-of-box) 200 GB (1 site in a CDB) Up to size of CDB* Items per CDB 60 million * Conditions apply10
    11. 11. How big are your biggest content databases?0-100 GB100-200 GB200-500 GBMore than 500 GBI have no idea
    12. 12. Should I externalizeBLOBs?
    13. 13. Advantages of keeping BLOBs in SQL One storage container to Maintain Monitor Recover Tier I storage Performance relative to lower tiers of storage benefits all content access SQL caching Performance of reads/writes of small documents SQL Caching benefits reads
    14. 14. Disadvantages of large CDBs & BLOBs Storage cost SQL operations Content Locking Current version Reindexing Previous versions Performance Transaction logs Slower reads with large Backups documents SLA timeframes BLOBs written twice Backup Recovery
    15. 15. Overview of BLOB externalization Pointer (stub) EBS/RBS Upload Database User Web Front-end Disk Storage
    16. 16. Options for externalizing External BLOB Storage (EBS) Released with SharePoint 2007 SP1 Supported by SharePoint 2010 Remote BLOB Store (RBS) Released in SQL Server 2008 R2 Feature Pack Can be installed on SQL Server 2008 SP1 Supported by SharePoint 2010 Both are interface specifications Need provider to communicate with BLOB store
    17. 17. SharePoint Documents in SQL
    18. 18. Externalized BLOBs in AllDocStreams
    19. 19. BLOB store
    20. 20. Externalized BLOBs are transparent Check-in/Check-out Versioning Office applications Search crawling (indexing) Workflows Alerts Anything using the SharePoint API Third party tools Custom code
    21. 21. Advantages of externalizing Storage Cost Performance Storage Cost Performance
    22. 22. Why Are BLOBS a Problem? Maintaining large quantities of BLOB data in a SQL Server database is a suboptimal use of SQL Server resources. You can achieve equal benefit at lower cost with equivalent efficiency by using an external data store to contain BLOB data.msdn.microsoft.com/en-us/library/bb802976.aspx
    23. 23. Advantages of externalizing Storage Cost Performance Read* Large documents Write* All other activity by users of the CDB and SQL server SLA timeframes * Dependent on performance characteristics of BLOB store Storage Cost Performance
    24. 24. What’s the performance gain? SQL BLOB RBS Gain Database Size - 1 TB 2292 GB 26 GB 98.9% Database Backup Size - 100 GB 217 GB 7 GB 96.8% Database Backup Time - 217 GB 2490 sec 38 sec 98.5% Database Defrag Time - 100 GB 120 sec 4 sec 96.7% Avg. SharePoint Response Time 28 msec 21 msec 25.0% Large File Upload - 500 MB 55 seconds 29 seconds 47.6% Source: SQL Server RBS Performance with SharePoint Microsoft Download Center
    25. 25. Advantages of externalizing Storage Cost Performance Read* Write* All other activity by users of the CDB and SQL server SLA timeframes Access to features of BLOB storage platform Efficient content restructure Storage Cost Performance
    26. 26. EBS and RBS EBS RBSAPI SharePoint SQL + SharePoint clientSharePoint version WSS v3 or MOSS SharePoint 2010, v.Next 2007 SP1 or laterSQL version Any SQL Server 2008 or R2Externalization rules File size and type File sizeScope Site collection Content databaseMicrosoft support Deprecated Likely to continueProvider Third party SQL FILESTREAM Third party
    27. 27. Choosing third-party externalization Performance BLOB store platform File system, SAN, NAS Shared folder Cloud Integration with your storage platform Externalization rules Manageability Backup, recovery High availability Disaster recovery Long term retention, archiving & tiered storage Cost
    28. 28. All things considered Storage Cost Archiving and Performance Disposition Maintenance Migration and Administration Backup & Updates and Recovery SLAs Upgrades (RTO/RPO) High Availability & Disaster Recovery SharePoint Uncensored
    29. 29. Content lifecycle Birth (Creation) Life (Use) Death (Disposition)
    30. 30. Content lifecycle meets RBS RBS determines whether BLOB gets externalized to another storage platform on upload Archiving & disposal Third party utilities can add business rules to externalization S Archiving and disposition
    31. 31. Externalization: Support and storage THIRDFEATURE FILESTREAM PARTYSharePoint 2010 (Server and Foundation)SharePoint 2007 (MOSS 2007 and WSS v3)All SQL versions (2000, 2005, 2008, 2008 R2)All SQL editions (Express/Standard/Enterprise)Externalize BLOBs to DAS, iSCSI NAS/SANExternalize BLOBs to file share, WORMExternalize to the Cloud (Azure, Amazon etc)Native compression and encryptionExternalize to multiple storage providers within 1 CDB
    32. 32. Externalization: Backup, recovery, DR THIRDFEATURE FILESTREAM PARTYSynchronous backup of BLOB store & SharePointBackup of content DB independent of BLOB storeItem level RecoveryPlatform Level RecoveryRestore without DB staging
    33. 33. Externalization: Content lifecycle support THIRDFEATURE FILESTREAM PARTYContent restructure (shallow copy) across Web appsContent replicationConnect to and manage file shares through SharePointConnect to and manage media shares through SharePointBusiness rule support (content type, metadata, access date)Externalize to hardware-based HSM
    34. 34. Resources SharePoint Team Blog: Data Storage Changes in SP1 http://tinyurl.com/3rlvfnp MSDN: External Storage of Binary Large Objects (BLOBs) in SharePoint Foundation http://tinyurl.com/yay545y White paper: SQL Server RBS Performance with SharePoint Server 2010… http://tinyurl.com/3ccucww
    35. 35. Dan Holme Randy Williamsdan.holme@avepoint.com randy.williams@avepoint.com Submit your questions!
    36. 36. © 2011 AvePoint, Inc. All rights reserved.No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint, Inc.
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×