Developer’s guideHow to enhance your SPperformance: Understand        caching
Shai PetelVP Research and DevelopmentMCPD, MCT, MVP: SharePointServershai@kwizcom.com | @shaibs | http://kwizcom.blogspot....
TopicsWhy do we need cache?Understand CachingOther performance coding tips
TopicsWhy do we need cache?Understand CachingOther performance coding tips
SharePoint performance                                      challengePages• Dynamic list forms• Dynamic pages and page lay...
SharePoint Page     Processing
What can be done better? Minimizing round trips to DB on every call Reduce ACL rules Pre-compiling user controls and pa...
What does SharePoint already                                    do? Ghost files on WFE Caches our .NET code in native co...
When is caching good for me?Information updatesSecurity Views & RiskSourcesSize
TopicsWhy do we need cache?Understand CachingOther performance coding tips
Understand CachingBLOB Cache   Ghosting files   Object Cache      Page output       Custom         cache       cache profi...
BLOB cache                                          What is? WFE local disk-based cache One per web application Reduce ...
BLOB cache                                            When to use? Ideal for publishing with many anonymous visitors. Si...
BLOB cache                                                   How to configure? Edit web.config, “<BlobCache … />” existin...
Ghosting files                                                   What Is? No content in content DB table Reduces load an...
Ghosting files                                          When to use? Template files that do not change much, such as mast...
Ghosting files                                            How to configure? Include the files you need in a module elemen...
Object cache                                                What is? Reduces traffic between WFE and DB Stores lists, li...
Object Cache                                     When to use? When performance is important, memory is cheap and  minor d...
Object Cache                                  How to configure? Requires “SharePoint Server Publishing” feature Site col...
Object Cache                         How to configure?Configure Super User (must have full control) and Super Reader (mus...
Object Cache                                         How to configure?Map users to service: Create users in central admin...
Object CacheHow to configure?
Object cache                                                        How to use? Add using to Microsoft.SharePoint.Publish...
Page output cache                                                    What is? Native ASP.NET technology Caches the HTML ...
Page output cache                                    When to use? To be used on content pages (publishing / wiki) When y...
Page output cache                                How to configure? Requires “SharePoint Server Publishing” feature “Chec...
Page output cache                                  How to configure? Configure on web application (in web.config)   <Outp...
Page output cache             How to configure?Sitecollectionsettings
Page output cache                     How to configure?Sub site settings:
Page output cache                        How to configure?Page layout settings:
Page output cache                                   How to configure?Create new cache profiles Use “Vary by X” to enable ...
Page output cache            How to configure?New cacheprofile:
Page output cache                                                       How to configure?ASP.NET cacheability:Member name ...
Custom cache profiles                                  What is?A custom class that inherits from IVaryByCustomHandlerDep...
Custom cache profiles                           When to use?When you want to use output cache, butyour application renders...
Custom cache profiles                                      How to configure? Register the event handler in global.asax fi...
Custom cache profiles                                                   How to use?Example, Vary by CSS availability, supp...
Custom cache profiles                              How to use?//supported strings in "Vary by custom string" tolook for in...
public string GetVaryByCustomString(HttpApplication app, HttpContextctx, string custom) {    //look for parameters specifi...
TopicsWhy do we need cache?Understand CachingOther performance coding tips
Other performance coding tipsSPList.Items performance trap Poor Performing Methods and                                    ...
Other performance coding tipsPaginate large list queriesSPWeb oWebsite = SPContext.Current.Web;SPList oList = oWebsite.Lis...
Other performance coding tipsQuery Throttling List items view threshold, for users and for site  administrators Develope...
Other performance coding tipsColumn indexing can help! Query throttling limits the number of DB rows that are  affected ...
Other performance coding tips Cache objects in memory   Use Page.Cache, per WFE   Only cache thread safe object    SPWe...
Other performance coding tipsMulti-threaded, use lockprivate static object _lock = new object();public void CacheData() { ...
Other performance coding tipsCache on client Use cookie to store settings instead of reading them  every time Use view s...
Other performance coding tips Use lazy loading Use scripting on demand<SharePoint:ScriptLink ID="scriptLink1" runat="ser...
Other performance coding tipsWork with script on demand SP.SOD.registerSod(key,url); To call function from this script, ...
Other performance coding tips Make sure your code runs after all dependency scripts  finished loading:   SP.SOD.executeO...
Other performance coding tipsEnable Bit Rate Throttling If you are steaming a lot of large media files Limit rate those ...
TopicsWhy do we need cache?Understand CachingOther performance coding tips
See also Documentation   http://technet.microsoft.com/en-us/library/cc261797.aspx   http://msdn.microsoft.com/en-us/lib...
Questions?=2B |! 2B ? Cache : Live;         ?
Upcoming SlideShare
Loading in …5
×

Developer’s guide understand caching

7,329 views

Published on

A developers and ITPros information about internal SharePoint 2010 caching capabilities.
Tips on how to make your SharePoint application run faster and scale up, and plan for caching.

Published in: Technology
  • Be the first to comment

Developer’s guide understand caching

  1. 1. Developer’s guideHow to enhance your SPperformance: Understand caching
  2. 2. Shai PetelVP Research and DevelopmentMCPD, MCT, MVP: SharePointServershai@kwizcom.com | @shaibs | http://kwizcom.blogspot.com
  3. 3. TopicsWhy do we need cache?Understand CachingOther performance coding tips
  4. 4. TopicsWhy do we need cache?Understand CachingOther performance coding tips
  5. 5. SharePoint performance challengePages• Dynamic list forms• Dynamic pages and page layoutsContent• Dynamic content structure• Extensible (field types)• Stored in Content DBCustomizations• Custom UI elements (Web parts, user controls, field controls)• Other API plugins (event handlers, workflows)
  6. 6. SharePoint Page Processing
  7. 7. What can be done better? Minimizing round trips to DB on every call Reduce ACL rules Pre-compiling user controls and pages Pushing supporting files from DB to each WFE Saving the output HTML of pages that do not change frequently Reusing DB connections and requests when possible
  8. 8. What does SharePoint already do? Ghost files on WFE Caches our .NET code in native code (If put in GAC) Pre-compiles user controls Shares context objects (SPContext.Current Site, Web, List) with all controls for a single request Apply some sort of object caching by default (good example is BCS)
  9. 9. When is caching good for me?Information updatesSecurity Views & RiskSourcesSize
  10. 10. TopicsWhy do we need cache?Understand CachingOther performance coding tips
  11. 11. Understand CachingBLOB Cache Ghosting files Object Cache Page output Custom cache cache profiles
  12. 12. BLOB cache What is? WFE local disk-based cache One per web application Reduce database load and network traffic Limit by file types Built on demand Configure cache size PublishingHTTPModule Only document library Security trimmed
  13. 13. BLOB cache When to use? Ideal for publishing with many anonymous visitors. Sites that contain lots of media assets that are read-only or rarely changes. Support site that has PDF and Doc user manuals for download Collaboration sites for creative team– don’t cache images For super heavy use, store BLOB on a separate hard drive DO NOT delete files directly from BLOB cache storage!
  14. 14. BLOB cache How to configure? Edit web.config, “<BlobCache … />” existing tag <BlobCache location=“D:BLOBWebApp1" path=".(gif|jpg|jpeg|jpe|jfif|bmp|dib|tif| tiff|ico|png|wdp|hdp|css|js|asf|avi|flv|m4v|mov|mp3|mp4|mpeg|mpg|rm|rmvb|wma|wmv)$" maxSize="10" enabled=“true" /> Flush BLOB cache PoSH: $webApp = Get-SPWebApplication “http://mywebapp:port“ [Microsoft.SharePoint.Publishing.PublishingCache]::FlushBlobCache($ webApp) Write-Host "Flushed the BLOB cache for:" $webApp STSADM: stsadm –o setproperty –propertyname blobcacheflushcount –propertyvalue 11 –url http://mywebapp::port Microsoft.SharePoint.Publishing.PublishingCache.FlushBlobCache( C#: Microsoft.SharePoint.Administration.SPWebApplication.Lookup( new Uri("http://mywebapp:port") ) );
  15. 15. Ghosting files What Is? No content in content DB table Reduces load and (ab)use of DB and network Ghost files != BLOB cache, yet similar Updates only need to be done to local template file When modifying a ghost file, only the differences will be stored (unghost) Reset back to definition (reghost) is possible Saves on expensive DB storage
  16. 16. Ghosting files When to use? Template files that do not change much, such as master pages, page layouts, forms, images, CSS files, JS files, ASPX files. Recommended to use whenever you deploy files using a feature. If you expect a lot of site-level customizations of the file, do not mark it as ghost.
  17. 17. Ghosting files How to configure? Include the files you need in a module element <File Path="SiteAssetsProjectFolderActionPage.aspx" Url="SiteAssets/ProjectFolder/ActionPage.aspx" Type="Ghostable" /> Set Type to Ghostable / GhostableInLibrary
  18. 18. Object cache What is? Reduces traffic between WFE and DB Stores lists, libraries, site settings and page layouts, as well as cross list queries Cost memory on WFE “Security tolerance” (“Cross list query multiplier”) Query DB 2 times! Portal super user and super reader Some API objects and OOB controls are cached automatically by using the standard API, but there is more! 
  19. 19. Object Cache When to use? When performance is important, memory is cheap and minor data inconsistency is acceptable* When your data does not update often* You can enhance performance even more by settingobject cache not to check for changes on every query
  20. 20. Object Cache How to configure? Requires “SharePoint Server Publishing” feature Site collection level setting, web application memory cap Set memory limit, duration, and “security tolerance” Flush object cache is done via site collection administration web UI.
  21. 21. Object Cache How to configure?Configure Super User (must have full control) and Super Reader (must have full read access)Default users are wrong!
  22. 22. Object Cache How to configure?Map users to service: Create users in central admin Next, use PoSH to specify super user and super reader account names: $wa = Get-SPWebApplication -Identity "<WebApplication>“ $wa.Properties["portalsuperuseraccount"] = "<SuperUser>“ $wa.Properties["portalsuperreaderaccount"] = "<SuperReader>“ $wa.Update() Perform IIS Reset to complete the operation.
  23. 23. Object CacheHow to configure?
  24. 24. Object cache How to use? Add using to Microsoft.SharePoint.Publishing Run the following code using (SPSite site = new SPSite("http://mywebapp:port")) { CrossListQueryInfo crossListQueryInfo = new CrossListQueryInfo(); crossListQueryInfo.Lists = "<Lists BaseType="5" />"; //Load all items that have "important" in their title crossListQueryInfo.Query = "<Where><Contains><FieldRef Name="Title" /><Value Type="Text">important</Value></Contains></Where>"; crossListQueryInfo.RowLimit = 100; crossListQueryInfo.UseCache = true; crossListQueryInfo.ViewFields = "<FieldRef Name="Title" />"; crossListQueryInfo.Webs = "<Webs Scope="SiteCollection" />"; crossListQueryInfo.WebUrl = site.ServerRelativeUrl; CrossListQueryCache crossListQueryCache = new CrossListQueryCache(crossListQueryInfo); SiteDataResults results = crossListQueryCache.GetSiteDataResults(site, false); }
  25. 25. Page output cache What is? Native ASP.NET technology Caches the HTML output of ASPX page Caches multiple versions When a page request is answered with a cached version:  No calls to any data source  No load/merge master page, page layout, field and web controls  Does not run any server side code Cache profiles  A more precise and customized output cache mechanism on top of ASP.NET  A list-style set of rules that can be applied to page layouts, site, site collection
  26. 26. Page output cache When to use? To be used on content pages (publishing / wiki) When you have few contributors and a lot of readers / visitors When pages do not change frequently If you plan to have many requests for the same pages (landing page)
  27. 27. Page output cache How to configure? Requires “SharePoint Server Publishing” feature “Check for changes” option in cache profile NLB - set persistent sessions (or sticky sessions) Monitoring cache behaviour: check the “enable debug cache information on pages”
  28. 28. Page output cache How to configure? Configure on web application (in web.config) <OutputCacheProfiles useCacheProfileOverrides="false" varyByHeader="" varyByParam="*" varyByCustom="" varyByRights="true" cacheForEditRights="false" /> Configure at site collection, allow publishing sub-sites & page layouts to override
  29. 29. Page output cache How to configure?Sitecollectionsettings
  30. 30. Page output cache How to configure?Sub site settings:
  31. 31. Page output cache How to configure?Page layout settings:
  32. 32. Page output cache How to configure?Create new cache profiles Use “Vary by X” to enable this same profile to store different versions of the cache based on specific criteria: Header: by language for example Query string parameter: http://site/page.aspx?ID=1 User rights: based on the ACL evaluation of each user. Users with same permissions will get the same cached content. Custom: “browser”, or custom settings (code)
  33. 33. Page output cache How to configure?New cacheprofile:
  34. 34. Page output cache How to configure?ASP.NET cacheability:Member name DescriptionNoCache Sets the Cache-Control: no-cache header. Without a field name, the directive applies to the entire request and a shared (proxy server) cache must force a successful revalidation with the origin Web server before satisfying the request. With a field name, the directive applies only to the named field; the rest of the response may be supplied from a shared cache.Private Default value. Sets Cache-Control: private to specify that the response is cacheable only on the client and not by shared (proxy server) caches.Public Sets Cache-Control: public to specify that the response is cacheable by clients and shared (proxy) caches.Server Specifies that the response is cached only at the origin server. Similar to the NoCache option. Clients receive a Cache-Control: no-cache directive but the document is cached on the origin server. Equivalent to ServerAndNoCache.ServerAndNoCache Applies the settings of both Server and NoCache to indicate that the content is cached at the server but all others are explicitly denied the ability to cache the response.ServerAndPrivate Indicates that the response is cached at the server and at the client but nowhere else. Proxy servers are not allowed to cache the response.
  35. 35. Custom cache profiles What is?A custom class that inherits from IVaryByCustomHandlerDeployed to GAC and registered in global.asaxBuilds a “cache key” string
  36. 36. Custom cache profiles When to use?When you want to use output cache, butyour application renders different HTMLbased on custom rules, like:User profile association (Like: HR info, VaryByLocation, VaryByLastName)Device/Environment rules (like: allow scripts? Allow CSS?)
  37. 37. Custom cache profiles How to configure? Register the event handler in global.asax file://Register the VaryByCustom string in the Global.asax file.<%@ Assembly Name="Microsoft.SharePoint"%><%@ Assembly Name=“CacheExample.VaryByCustomExample"%><%@ Application Language="C#" Inherits="CacheExample.VaryByCustomExample" %> Pass string parameters to “vary by custom parameter” to trigger your handler (Optional)
  38. 38. Custom cache profiles How to use?Example, Vary by CSS availability, support XMLHttp, and is user site admin. Parameter(s) to beset in profile “Vary by custom parameter”: “SupportsCSS;SupportsXmlHttp;SiteAdmin”using System;using System.Collections.Generic;using System.Linq;using System.Text;using Microsoft.SharePoint.ApplicationRuntime;using System.Web;using Microsoft.SharePoint;namespace CacheExamples{ public class VaryByCustomExample : SPHttpApplication, IVaryByCustomHandler { … }}
  39. 39. Custom cache profiles How to use?//supported strings in "Vary by custom string" tolook for in order for our logic to kick inpublic enum SupportedParams { SupportsCSS,SupportsXmlHttp, SiteAdmin }public override void Init(){ base.Init(); this.RegisterGetVaryByCustomStringHandler( (Microsoft.SharePoint.ApplicationRuntime. IVaryByCustomHandler)this );}
  40. 40. public string GetVaryByCustomString(HttpApplication app, HttpContextctx, string custom) { //look for parameters specified in the cache profile StringBuilder sb = new StringBuilder(); string[] strings = custom.Split(;); foreach (string str in strings) { SupportedParams param; if (!Enum.TryParse<SupportedParams>(str, out param)) continue;//Not in our enum... Skip switch (param) { case SupportedParams.SupportsCSS: sb.Append(ctx.Request.Browser.SupportsCss + ";"); break; case SupportedParams.SupportsXmlHttp: sb.Append(ctx.Request.Browser.SupportsXmlHttp + ";"); break; case SupportedParams.SiteAdmin: sb.Append(SPContext.Current .Web.UserIsSiteAdmin.ToString() + ";"); break; default: continue; } } return sb.ToString().TrimEnd(;);}
  41. 41. TopicsWhy do we need cache?Understand CachingOther performance coding tips
  42. 42. Other performance coding tipsSPList.Items performance trap Poor Performing Methods and Better Performing Alternatives Properties SPList.Items.Count SPList.ItemCount Create an SPQuery object to retrieve SPList.Items.XmlDataSchema only the items you want. SPList.GetItemByUniqueId(System.Guid SPList.Items[System.Guid] ) SPList.Items[System.Int32] SPList.GetItemById(System.Int32) SPList.Items.GetItemById(System.Int32) SPList.GetItemById(System.Int32) SPFolder.Files.Count SPFolder.ItemCount
  43. 43. Other performance coding tipsPaginate large list queriesSPWeb oWebsite = SPContext.Current.Web;SPList oList = oWebsite.Lists["Announcements"];SPQuery oQuery = new SPQuery();oQuery.RowLimit = 10; //Limit # of items to under 2000int intIndex = 1;do { Response.Write("<BR>Page: " + intIndex + "<BR>"); SPListItemCollection collListItems = oList.GetItems(oQuery); foreach (SPListItem oListItem in collListItems) Response.Write(oListItem["Title"].ToString() +"<BR>"); //Move to next page, same query – this is all that changes. oQuery.ListItemCollectionPosition = collListItems.ListItemCollectionPosition; intIndex++;} while (oQuery.ListItemCollectionPosition != null);
  44. 44. Other performance coding tipsQuery Throttling List items view threshold, for users and for site administrators Developers override (for privileged users) http://msdn.microsoft.com/en- us/library/microsoft.sharepoint.spquerythrottleoption(v=office.14).aspx “Happy hour” Limit # of lookup, person, or workflow status fields that can be included in a single database query
  45. 45. Other performance coding tipsColumn indexing can help! Query throttling limits the number of DB rows that are affected Query that filters or sorts on a non-indexed column affects all list items, even if only 10 items return By indexing that column, only 10 rows in the DB will be affected Indexing cost resources
  46. 46. Other performance coding tips Cache objects in memory Use Page.Cache, per WFE Only cache thread safe object SPWeb, SPSite, SPListItemCollection – No! SPListItemCollection.GetDataTable() – Yes! Consider security when caching Poor man’s cache dependency – use cache version number to invalidate your data once updated.
  47. 47. Other performance coding tipsMulti-threaded, use lockprivate static object _lock = new object();public void CacheData() { SPListItemCollection oListItems; oListItems = (SPListItemCollection)Cache["ListItemCacheName"]; if(oListItems == null) { lock (_lock) { //Ensure the data not loaded by a concurrent thread while waiting for lock. oListItems = (SPListItemCollection)Cache*“ListItemCacheName”+; if (oListItems == null) { oListItems = DoQueryToReturnItems(); Cache.Add("ListItemCacheName", oListItems, null, DateTime.Now.AddSeconds(60), Cache.NoSlidingExpiration, CacheItemPriority.High, onRemove); } } }}
  48. 48. Other performance coding tipsCache on client Use cookie to store settings instead of reading them every time Use view state to store data between postbacks Override SaveViewState/LoadViewState to “zip” large data or “encrypt” sensitive information (I reduced 8MB viewstate to 65KB)
  49. 49. Other performance coding tips Use lazy loading Use scripting on demand<SharePoint:ScriptLink ID="scriptLink1" runat="server" Name="My_Script.js"LoadAfterUI="true" OnDemand="false" />Not on demand:<script type="text/javascript">// <![CDATA[document.write(<script type="text/javascript" src="/_layouts/my_script.js"></ +script>);// ]]></script>On demand:<script type="text/javascript">RegisterSod("my_script.js","u002f_layoutsu002fmy_script.js");</script>
  50. 50. Other performance coding tipsWork with script on demand SP.SOD.registerSod(key,url); To call function from this script, use: SP.SOD.executeFunc(key, functionName, fn); Script dependency: SP.SOD.registerSodDep(key, dependOnKey);
  51. 51. Other performance coding tips Make sure your code runs after all dependency scripts finished loading: SP.SOD.executeOrDelayUntilScriptLoaded(func, scriptName) In your script files, notify when your classes finished registring: SP.SOD.notifyScriptLoadedAndExecuteWaitingJobs(" my_script.js");
  52. 52. Other performance coding tipsEnable Bit Rate Throttling If you are steaming a lot of large media files Limit rate those files are delivered to client to conserve network bandwidth A part of IIS Media Services WMV, MP4, SWF files bit rates are detected automatically Good for “you tube” type of solution
  53. 53. TopicsWhy do we need cache?Understand CachingOther performance coding tips
  54. 54. See also Documentation http://technet.microsoft.com/en-us/library/cc261797.aspx http://msdn.microsoft.com/en-us/library/aa589700.aspx * except for “BLOB” sub page http://technet.microsoft.com/en-us/library/ee424404.aspx http://msdn.microsoft.com/en- us/library/bb687949(office.12).aspx http://msdn.microsoft.com/en-us/library/ff798465.aspx Sean McDonough on BLOB Cache http://sharepointinterface.com/2009/06/18/we-drift- deeper-into-the-sound-as-the-flush-comes/
  55. 55. Questions?=2B |! 2B ? Cache : Live; ?

×