SharePoint is from Mars, SQL Server is from Venus (SQL Server for SharePoint Administrators)
•
2 likes•1,374 views
The SQL Server Database Engine is a prerequisite and fundamental part of any SharePoint deployment but the downside to all of this is the Database Server usually brings a Database Administrator with it. In this session we will explain how to configure SQL Server correctly with SharePoint deployments in mind, how the underlying architecture and mechanisms of SQL Server work and why you need to do all this! By the end of this session you will have learnt how to talk to your DBA to explain that they are doing it wrong!
Recommended
More Related Content
What's hot
What's hot (20)
Similar to SharePoint is from Mars, SQL Server is from Venus (SQL Server for SharePoint Administrators)
Similar to SharePoint is from Mars, SQL Server is from Venus (SQL Server for SharePoint Administrators) (20)
More from Mark Broadbent
More from Mark Broadbent (14)
Recently uploaded
Recently uploaded (20)
SharePoint is from Mars, SQL Server is from Venus (SQL Server for SharePoint Administrators)
- 1. Mark Broadbent Principal Consultant SQLCloud SQLCLOUD.CO.UK SQL Server is from Venus
- 2. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Disclaimer • I am not a SharePoint Professional (yet) • This session is not about SharePoint • This is not a comprehensive introduction • There is a danger that you might end up becoming a DBA
- 3. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Why you should stay • Many of the SharePoint considerations are counter intuitive which means that: • Your DBA hates you (he always has and always will) • Dedicated SharePoint Farm SQL Servers are required • Your DBA is (sometimes) an idiot • But it is not always his fault • He only speaks T-SQL and doesn’t understand SharePoint • You can communicate with him better
- 4. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk About Mark Broadbent. “30 billion times more intelligent than a live mattress” Microsoft Certified Master/ Certified Solutions Master: SQL Server Owner/ Principal at SQLCloud Email: mark.broadbent@sqlcambs.org.uk Twitter: retracement Blog: http://tenbulls.co.uk Event Lead to the UK’s first ever SQLSaturday (Cambridge) – now also SharePoint Saturday Event #3 on 10/ 11/ 12th Sept 2015 (pre-cons & Community day) http://sqlcambs.org.uk
- 5. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Agenda Installation 1 Databases and internals 2 Memory, CPU and Connectivity High Availability The best part of the presentation… SharePint 4 3 5
- 6. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk What *exactly* is SQL Server? • High performance structured relational database engine • Complex suite and engine provides its own (internal) operating system (SQLOS)! • Optimised to reduce IO • Intelligent Query Optimisation • Provides ACID based transactional processing • Offers highly available solutions AND is also Cluster Aware
- 7. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Pre-Installation Considerations • Which versions of Windows and SQL Server? • Server Collation (consider Latin1_General_CI_AS_KS_WS ) • Content Database default • Cannot change post install • Problematic for shared environments (TempDB conflicts) • File placement of system databases • Aim to place System DBs correctly at installation time • TempDB can be easily moved post install • Requires restart • If you get it wrong SQL wont start (run in single user mode) • Format storage using 64K cluster size including RAID stripe • Dedicated SQL installation • Clustered or standalone –co-existance considerations?
- 8. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk
- 9. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Open file and save Open file and save
- 10. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk • Standalone only, for Clusters use Command line for unattended
- 11. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Run installer on Full-Fat Edition (full build). Step through wizard (do not complete). Grab ConfigurationFile.ini from file path. 1 From Zero to Core Configuration File Deploy core ready installation to Windows Server Core via command line setup.exe /CONFIGURATIONFILE=“ConfigurationFile.ini” /QS 4 Replace Features with core compatible ones. Remove all lines below this line. 2 FEATURES=SQLENGINE,REPLICATION,FULLTEXT,AS,CONN,IS “Install based on configuration file” on Full-Fat Edition. Use configuration file modified in step 2. 3 Remove UIMODE="Normal“ Add SAPWD=“{password}“ and any other account passwords
- 12. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk
- 13. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Post Installation Considerations • MAXDOP 1 • Max Mem (default is 2147483647) / OS reservation –Approx 2GB • Page File Size (NOT 1.5x your memory! Page File usage is key) • TempDB • Location • Files, growth • Fill factor to 80% to avoid page splits • Default Paths for Log, Data and Backup • Default Backup Compression turned on • Traceflags, TCP/IP (+ Port)
- 14. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Databases • Add multiple data files • Set equal size (proportional fill algorithm) • Consider TF1117 (forces all files to grow) • Ensure Instant file initialisation is switched on • Only works for data files • Add SQL Service account to “Perform volume maintenance tasks” group policy • Use Quota Templates for your site collections Master (SQL server schema and meta data) Msdb (SQL Server Agent job repository Resourcedb (read only copy of system objects) Model (default template for all new databases Tempdb (public toilet of SQL Server and prone to page allocation contention PFS, GAM, SGAM) Search Content DBs
- 15. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Transaction Log Architecture Log file (.ldf) Virtual Log Files (VLFs) Log growth size <=64MB then 4 VLFs > 64MB AND <= 1GB 8 VLFs > 1GB 16 VLFS Think about how big your log needs to be from the beginning Log Buffer 1 Log Buffer n Log Writer Signal to flush Log writes are first written to log buffers. Flush on fill or on commit. async Control returned to spid on commit.
- 16. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Pages • Single logical unit of data storage in SQL Server • Are 8 KB (or 8192 Bytes) in size • Have 96-byte page header & row offset (or slot array) of 36 bytes • Page row capacity is 8192-96-36 = 8060 bytes • Different types including: • Index Pages • Data Pages • Allocation Pages
- 17. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Extents and Files • 8 physically contiguous pages (totalling 64 KB) • Extents can be: • Mixed (default) – and shared up to 8 different objects • Only for objects (index or table) < 8 pages • Turned off using TF1118 • Uniform – owned entirely by single object • One or more files can belong to filegroup. *1 A object (table or index) only exists within single filegroup *1 SharePoint does not support using secondary filegroups
- 18. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Buffer Cache BEGIN TRAN UPDATE orders SET qty = 10 WHERE … 2 1 spid 115 BEGIN TRAN INSERT order (id,qty) VALUES (8,10) INSERT order (id,qty) VALUES (9,2) COMMIT spid 162 changes (IO) to pages in buffer cache first regardless of commit Datafile “dirty” pages in buffer cache flushed to disk on CHECKPOINT (but not removed from cache) Automatic CHECKPOINT MAX MEMORY
- 19. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Connectivity • Default instance on 1433 by default • Browser service (if enabled listens on 1434tonysql01 TCP Port 9561 registry SharePoint Farm Group Policy 32 bit and 64 bit client aliases tonysql01 resolves through alias DNS Aliases tony resolves through alias Browser service required to resolve instance port number spfarm1 spfarm1sql01
- 20. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Scheduling (think CPU) SPID 71 SPID 80 SPID 81 SPID 85 SPID 100 SPID 150 SPID 80 LCK_M_X SPID 81 NETWORKIO SPID 85 CXPACKET SPID 100 CXPACKET SPID 150 CXPACKET Running – Scheduler 1 Runnable Queue– Scheduler 1 Wait type: Signal Wait Waiter List Wait type: Resource specific On resource wait resource wait complete sys.dm_os_waiting_tasks Cumulative waitstats found in sys.dm_os_wait_stats
- 21. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Max Degree of Paralellism (MAXDOP) • exec sp_configure ‘max degree of parallelism’ 0; • So ‘cost threshold for parallelism’ can be ignored • Statistics determine estimations which determine cost (which determines parallelism)
- 22. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Database Backup and Recovery • Full or differential backups do not “reseed” (or break) backup chain • Differential all changes since last full • Full resets differential bitmap (not on COPY_ONLY) • Restore starts with a Full (or Primary Filegroup backup) Full Diff FULL RECOVERY Diff tran tran Diff Fulltran tran SIMPLE Full Diff Diff Diff Full Full Bulk Logged recovery has no use for SharePoint • Ensure database recovery model matches database point in time requirements. Is that content database important?
- 23. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Backup best practices • Ensure backups on remote storage (RAID 5 is fine) • Ensure compression is used • EXEC ‘backup compression default’,1; --set a server wide default • BACKUP DATABASE <database> WITH COMPRESSION…. –per statement • Run regular transaction log backups to truncate your VLFs • Ensure page checksums are enabled • Use WITH CHECKSUM on all backup and restore operations • Creates and validates backup stream checksum • (If exist)page checksums validated AND backup stream checksum validated)
- 24. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Supported Upgrade Path • SharePoint 2013 supports 2008R2 and above • Upgrade can be performed “in place” or to upstream instance • Database upgrade steps run on RESTORE WITH RECOVERY • Use manual backups, Log Shipping or Database Mirroring SQL 2005 SP4 SQL 2008R2 SP1 SQL 2012 SQL 2008 SP2 SQL 2000 Requires staged Upgrade to SQL 2005, SQL 2008 /R2 SQL 2014
- 25. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk SharePoint 2013 Database HA Solutions Manual backups, log shipping and Database Mirroring are perfect to assist with SharePoint DB upgrades Log-shipping Transaction HA Availability Groups Database(s) level HA Database Mirroring Database level HA Failover Clustering SQL Instance HA
- 26. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Cluster Quorum • Windows Clusters required by SQL FCI and AGs • By default one node casts one vote • Voting disk/ Voting Share adds one vote (up to Windows 2012) • Majority requires a visible count > Vote Maximum/ 2 • Windows 2012 and Windows 2012 R2 Maximum is dynamic “I can see dead people X votes” Lost connectivity to other nodes
- 27. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Quorum Models and Functionalilty Quorum determines if a node can be considered “alive” in a cluster • Windows 2008/ R2 provides: • Disk Only Quorum (or No Majority) • Node Majority (or Majority Node Set) • Node and Disk Majority • Node and File Share Majority • Node Weighting via hotfix KB2494036 • Windows 2012 added Dynamic Quorum + Wizard vote assignment • Windows 2012 R2 added: (THE reason for adoption) • Dynamic Witness Weight • Split brain protection (or Tie Breaker) for 50% split • Force quorum resiliency
- 28. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Other best practices in case I forgot to say! • Dedicated server instance • Set max mem –min mem also important (especially when LPIM used) • Avoid shrinking databases –unless you like fragmention! • DBCC CHECKDB REPAIR_ALLOW_DATA_LOSS is unsupported use instead REPAIR_REBUILD • Secondary filegroups unsupported –piecemeal restore not possible • Pre-allocate space (and avoid disk and page fragmentation) • SQL can perform more efficient data-file read-aheads • Pre allocating log will reduce VLFs • Avoid autogrowths • Group (or separate) Site collections for performance and recovery considerations. • FULL recovery for point in time. SIMPLE otherwise • Place active contentdbs on high performance storage • Use Site Collection Quotas to cap space usage
- 29. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk In Summary… • It is important for SharePoint Pros to understand a little of SQL and for SQL Pros to understand a little of SharePoint – try and talk to each other regularly • There are lots of nuances to using SQL Server for SharePoint databases -make sure both you AND your DBA understand them • Avoid configuring SQL for SQL Server “Best Practices” and instead configure for SharePoint “Best Practices” in the 1st instance
- 30. Copyright © 2015, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk Thank you for listening! Email: mark.broadbent@sqlcambs.org.uk Twitter: retracement Blog: http://tenbulls.co.uk
Editor's Notes
- Session preliminary. E.g. VMs running…. Start Audio Recorder Session - SharePoint is from Mars, SQL Server is from Venus Level: 300 Track: IT Pro The SQL Server Database Engine is a prerequisite and fundamental part of any SharePoint deployment but the downside to all of this is the Database Server usually brings a Database Administrator with it. In this session we will explain how to configure SQL Server correctly with SharePoint deployments in mind, how the underlying architecture and mechanisms of SQL Server work and why you need to do all this! By the end of this session you will have learnt how to talk to your DBA to explain that they are doing it wrong!
- 1st time in the history of SharePoint conferences that the audience knows more about sharepoint than the presenter Warning that I may rush through some of the slides due to time Story about Microsoft and my first exposure to SharePoint Have always buried my head in the Sand whenever SharePoint comes into any equation but I have tried to start learning to love it.
- <2>
- <3> Hates you – GUIDs, numerous Databases, surrendering of rights, MAXDOP, horrible queries, an environment more complicated to understand than <insert>, he is jealous that your job sounds slightly more interesting than his Idiot – accidental dba, default blame acceptor Learn to communicate with him using his language and his acroynyms. Educate yourself to educate him. Core concepts –If you understand some of the capabilities of SQL Server this will help when it comes to troubleshooting or performing upgrades or ensuring your Farm is backed up correctly and highly available.
- <5> But first I ought to introduce myself… Contact SQLSaturday
- <7>
- <9>
- <10>
- <12> Standalone instance New SQL failover clustered instance Add Node to SQL failover cluster Uninstall components Add components
- <13>
- <15> Standalone
- <14>
- <16>
- <17>
- <20> On restart Tempdb will shrink back to defaults if you haven’t explicitly set Databases created via SharePoint will not use model defaults – remember to check or create through SQL Ensure Storage is formatted using 64K cluster size. Includes RAID stripe File locations (file share, LUNs, CSVs (only option to clusters) Filestream required if you are using remote blob storage in SharePoint 2013 Farm.
- <24> Captures database changes and written to first via log buffers in parallel. Allows for the D in ACID (durability) of transactions. Log buffers are written to file sequentially. Log file is cyclic in nature do only one log file assuming big enough drive –could be a question to the audience (in delayed durability buffers will only flush on full or via sp_flush_log_ What happens when the log fills up and can’t expand? Log block is the unit of physical IO to disk and can range between 512 bytes to 60K VLFs are created at each file growth OR when the log is created. Sizing the database log correctly is important not to have too many VLFs.
- <26> Pages logically create b-tree structures for indexes and clustered tables
- <28> Turn off mixed extents to avoid sgam allocation page contention http://blogs.msdn.com/b/psssql/archive/2008/12/17/sql-server-2005-and-2008-trace-flag-1118-t1118-usage.aspx Mixed extents are SGAM pages and track available shared extents with pages free. http://blogs.msdn.com/b/sqlserverstorageengine/archive/2006/07/08/under-the-covers-gam-sgam-and-pfs-pages.aspx GAM tracks extents that are available for allocation. Each track every 64000 extents (~ 4GB)
- <30> Why is this a good thing? Because it means lots of changes can occur in memory before disk is needed only transaction log will have guaranteed writes on commit
- <33> Using client aliases will help if you ever want to move your SQL databases or move to AlwaysOn Availability groups You may have company standards to run on 1433, or want to use dns resolution. Dns resolution resolves dns alias to server host name. DNS on non 1433 port, Instance only found if browser service running and instance specified in connection string OR port number provided. Perhaps you want to run multiple instances on a server. Only one could have 1433. Same default port number for each instance. Can be done because each instance runs under a unique IP address (hence different network name)TCP port combo. Look this up Benefits are that if you have a standards in place to accommodate potential circumstances, then having this flexibility to standardise your ports can work really well. Good for DNS resolution. Surface area attack of known port number Migration of multiple sql instances to single standalone server then forced to change the port number to coexist.
- <36> MAXDOP 1 gives our query exactly this story SQLOS – Waits and Queues (co-operative/non-preemptive) Slide needs to be changed to Scheduler. Need to talk about the difference between Windows thread scheduling and
- <39> Parallism, memory grants, query cost, statistics. Schedulers to
- <42> Piecemeal restore is not possible since secondary filegroups not supported. BULK LOGGED recovery has no use for SharePoint Differential backup records all changed extents since last full Compression, checksum, copy only Backup chain - There is a fallacy that taking Full backup breaks the backup chain and this (I believe is a remnant from SQL 2000) You can use the WITH CHECKSUM option when taking a backup (of any variety). This creates a checksum over the entire backup stream, which is stored in the backup itself. If the backup is a full or differential, and the database has page checksums enabled (ALTER DATABASE AdventureWorks2012 SET RECOVERY FULL PAGE_VERIFY CHECKSUM;), then this option will also cause all existing page checksums to be tested as the backup process reads the pages. If a bad page checksum is found, the backup operation will fail.
- <45>
- <48> Unlike SharePoint, SQL Server can be upgraded databases can upgrade back across from multiple versions Performing database upgrade using backup and restore is far better and the upgrade steps only occur once recovery occurs. Make sure you put operating system on this slide and the upgrade path for that!! Slipstreaming SQL 2008 http://support.microsoft.com/kb/955392
- <51> Virtualization is commonly thought of as ‘the’ replacement for failover clustering, but they are both offering something completely different. Whilst the both provide the ability to move hosts (or cluster nodes), one offers protection from Operating System corruption and the other offers protection at a lower granularity (the SQL instance). Are we adding new risks to the environment by adding an HA solution. What changes are made to the underlying system that we are trying to protect? I always think about three key points. Firstly what is the level of HA abstraction , secondly how closely coupled is the SQL Server Data in the offering and finally what are the risks presented by using it?
- <53> A node must see a majority for it to continue running in the cluster. A loss of visible majority will cause it to fail. The loss of a cluster node means that one less vote is cast towards the visible count for other nodes. On a loss between sites and cluster failure, Quorum can be forced to start using net start clussvc /fq. This overrides quorum majority but cluster will always search for more node votes to add (as they start up). Once majority is achieved it will move the cluster out of forced state. To prevent a cluster node from adding a vote use net start clussvc /pq which essentially starts the node up with a weight of 0.
- <54> (MIGHT BE GOOD DEMO?) The introduction of Dynamic Witness with Window 2012 R2 is a huge advance in Quorum configuration. Now it is recommended to always configure a voting disk and let the Cluster “worry about it!” (Get-Cluster).WitnessDynamicWeight – do not think that this exists in the GUI. Split brain protection (I think if there are no multi-sites then id set to 0) Get-ClusterNode | ft (Get-ClusterNode –Name "Node1").Id (Get-Cluster).LowerQuorumPriorityNodeID = 1 Force quorum Split multi-site 3+2. 3 keeps running but 2 fails. 2 has storage connectivity but 3 doesn’t. Must forcequorum on 2 site In 2012 after connectivity is restored must manually restart partitioned nodes not part of forced quorum using /pq switch to prevent quorum ideally as quickly as possible. In 2012R2 there is automatic reconciliation when connectivity is restored and 3 nodes automatically restart with /pq switch to prevent quorum. 2 nodes are deemed authoritive.
- 55 (1h06 – 7) Adhoc administrative checks Reporting on read-only data Benefits outway the Risk (good luck on that one) RCSI and SI still have overhead of temp db
- But first I ought to introduce myself… Contact SQLSaturday