Orders of magnitude:Scale-Out your SQL Server DataMark Broadbent@retracement
A bit more about me!•   More than 20 years in IT and more than 14 years using SQL Server.    Worked at many large global c...
Agenda•   Why Scale-Out and what should we scale?•   Benefits of Scale-Out?•   Wait!!! Before you Scale-Out, Scale-In…•   ...
Why Scale-Out“Gartner Group study, for example, predicted that theamount of data generated by enterprises will grow by ast...
Traditional Synchronous Activity                      User                    “Database”                    Request A     ...
Asynchronous Activity?                     User                  “Database”                   Request A           A       ...
Separation of Function…                      User                   “Database”                                        A   ...
Separation of Requirements…                        User                     “Database”                                    ...
What is truly Mission Critical?                                         Distribution of System Data Interactive (real-time...
DEMO: Scaling for READSSnapshotsRolling SnapshotsRolling Snapshots on Database MirrorDatabase Mirror Failover?How to make ...
What can Scale-Out give us? # 1• Availability   – Portions of data can go offline but doesnt effect the whole• Disaster   ...
What can Scale-Out give us? #2• Performance  – Load balance  – Mix and match (LOW consumers and BIG consumers)  – Separati...
Before you Scale-Out…Scale-In #1• Keep statistics accurate and up to date   – Avoid big temp vars, they have no indexes or...
Before you Scale-Out…Scale-In #2• Reduce PageIO   –   Filtered indexes   –   Sparse columns   –   Use correct data types  ...
Scalable Shared Database   Report                    Report   Server                    Server                            ...
DEMO: Reporting Services Scale-OutReporting from SnapshotsReporting Services Scale-Out deploymentScalable Shared Database ...
Partitioned tables, views and filegroups   Table     Partitioned    Partitioned                View          Tables  payme...
With Distributed Partitioned views   Table       Distributed      Partitioned             Partitioned View     Tables  pay...
Peer to Peer Replication        Read-Write for User                                                                  Read-...
Hybrid Scale-Out• Database Mirroring with rolling snapshots• SQL Failover Cluster using over-provisioned  failover node “h...
Bringing it all together                                        Report    Virtual Cluster Nodes               Server      ...
The SQL Server Scale-Out Toolkit•   Service Broker                   •   Full Text Indexing•   Integration Services       ...
Taming the Beast• Governance   – Policy Based Management   – Resource Governor or WRSM   – Source Control• Monitoring   – ...
In Summary• We discussed  –   Why we should start thinking about Scaling-Out?  –   Benefits from Scale-Out  –   Scaling in...
Further References•   Books     – Apress - Pro SQL Server 2008 Service broker – Klaus Aschenbrenner     – Apress - Pro SQL...
Please Complete the Evaluation FormPick up your evaluation form:• In each presentation roomDrop off your completed form• N...
THANK YOU!                       Presented by DellFor attending this session andPASS SQLRally Orlando, Florida            ...
Upcoming SlideShare
Loading in...5
×

Orders of-magnitude-scale-out-your-sql-server-data-slideshare

1,881

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,881
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
29
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Orders of-magnitude-scale-out-your-sql-server-data-slideshare

  1. 1. Orders of magnitude:Scale-Out your SQL Server DataMark Broadbent@retracement
  2. 2. A bit more about me!• More than 20 years in IT and more than 14 years using SQL Server. Worked at many large global corporations and SMEs such as Microsoft, Nokia, Hewlett Packard and Encyclopaedia Britannica.• Presented at SQLBits 7 and 8• MCITP Database Development SQL 2008• MCITP Database Administrator SQL 2008/ 2005 and MCDBA SQL 2000• Microsoft Certified Application Developer (C# .net)• Microsoft Certified Systems Engineer + Internet• Participate on #sqlhelp, MSDN Forums, Stackoverflow & Serverfault.• Used to be active in the MS newsgroups until their demise :(• Run the LinkedIn groups• Blog: tenbulls.co.uk• Linux Mint User Group http://www.linkedin.com/groups?gid=2989801• SQL Server Scripting http://www.linkedin.com/groups?gid=3033621 Orders of magnitude: Scale-Out your 2 SQL Server Data
  3. 3. Agenda• Why Scale-Out and what should we scale?• Benefits of Scale-Out?• Wait!!! Before you Scale-Out, Scale-In…• Lets look at some strategies and Scale-Out!• Hybrid Scale-Out• Taming the Beast Orders of magnitude: Scale-Out your 3 SQL Server Data
  4. 4. Why Scale-Out“Gartner Group study, for example, predicted that theamount of data generated by enterprises will grow by astaggering 650 percent over the next five years. Anotherstudy sponsored by IBM found that 83 percent of the globalCIOs surveyed believe that analyzing and leveragingenterprise data is critical to their companies’ long-termcompetitiveness.” - Divining the Future of ERP Softwarehttp://dell.to/jsoJikReference: http://bit.ly/k2kpwW (2009 Gartner ITInfrastructure, Operations & Management Summit) Orders of magnitude: Scale-Out your 4 SQL Server Data
  5. 5. Traditional Synchronous Activity User “Database” Request A A Request BWasted B EvenProcessor distributioncycles Request C of database Time C load fromduring waits Request D application D Request E E“A common error in client/server development is to prototypean application in a small two tier environment, and then scaleup by simply adding more users to the server” – Lloyd Taylor Orders of magnitude: Scale-Out your 5 SQL Server Data
  6. 6. Asynchronous Activity? User “Database” Request A A Request BUser Competingexperience Request C C loads onmore B database Time Request D fromproductiveapplication Request E D application Ethread more faster allactive Request x round execution “to increase the capacity or performance of a system, tuning can get up to 10% improvement. To get order or two magnitude of performance and capacity improvement a change in the architecture of the system is needed ” – W.H. Inmon Orders of magnitude: Scale-Out your 6 SQL Server Data
  7. 7. Separation of Function… User “Database” A Request A Request BIntensive Separated Request C C loads acrossreportingqueries B database Time Request D E D based on Request E function, even faster Request x execution Units of Scale“Building a system from ground up is an absolute requirementfor highly scalable systems” - P. Randal & K.L. Tripp Orders of magnitude: Scale-Out your 7 SQL Server Data
  8. 8. Separation of Requirements… User “Database” A Request A B Request BQueued, Impact on Request C C D databaseScheduled orGoverned lessoned Time Request D Eeither by the due toapplication Request E scheduledserver or B or governedinstance Request x query Units of Scale“in order to best implement a scaleout architecture, itmust be planned in advance.” – Bob Beauchemin Orders of magnitude: Scale-Out your 8 SQL Server Data
  9. 9. What is truly Mission Critical? Distribution of System Data Interactive (real-time) OLTP Integrated (24hrs - 1 month) Static Data Aggregates Near line Probability Denormalized (3 - 4 years) of access Archival (5 - 10 years) DW /OLAP “Scale out workloads depending upon what is truly mission critical.” - Larry Chestnut Orders of magnitude: Scale-Out your 9 SQL Server Data
  10. 10. DEMO: Scaling for READSSnapshotsRolling SnapshotsRolling Snapshots on Database MirrorDatabase Mirror Failover?How to make use of the Rolling Snapshots Orders of magnitude: Scale-Out your 10 SQL Server Data
  11. 11. What can Scale-Out give us? # 1• Availability – Portions of data can go offline but doesnt effect the whole• Disaster – Recovery time (reduce time to restore - reduced when less to recover) – Large Disk Partitions can take long time to fix – Limit the impact of total disaster (i.e. when your DR strategy does not work)• Cost – Reuse commodity hardware for less important data – Higher we Scale-Up, more expensive it becomes. Scale-Out can be cheaper Orders of magnitude: Scale-Out your 11 SQL Server Data
  12. 12. What can Scale-Out give us? #2• Performance – Load balance – Mix and match (LOW consumers and BIG consumers) – Separation of workload types (OLAP and OLTP) – Parallelise a System (separate system requests across multiple hardware) – Overcome contentious parts of the DB server such as TempDB• Capacity – Backup time (reduce time to backup) – Limited resources Orders of magnitude: Scale-Out your 12 SQL Server Data
  13. 13. Before you Scale-Out…Scale-In #1• Keep statistics accurate and up to date – Avoid big temp vars, they have no indexes or stats – Stats can get skewed, ensure they are maintained• Query Tuning – Avoid table scans – Use indexes correctly, and remove duplicates – Is parallelism right for your query (OLAP vs OLTP) – Reduce size of the result set – Always use a WHERE clause – DON’T use SELECT * replace with precise column list – Sensible clustered key, avoid large covered index and prefer include option Orders of magnitude: Scale-Out your 13 SQL Server Data
  14. 14. Before you Scale-Out…Scale-In #2• Reduce PageIO – Filtered indexes – Sparse columns – Use correct data types – Use table/ row and page compression – Remove your LOBs from tables • Use other technologies such as FILESTREAM • Vertically partition• Reduce contention on shared resources – Denormalize – Filegroups Orders of magnitude: Scale-Out your 14 SQL Server Data
  15. 15. Scalable Shared Database Report Report Server Server Database SQL Server Instance A Report Server Database SQL Server Application Instance B Server Instance C Orders of magnitude: Scale-Out your 15 SQL Server Data
  16. 16. DEMO: Reporting Services Scale-OutReporting from SnapshotsReporting Services Scale-Out deploymentScalable Shared Database Orders of magnitude: Scale-Out your 16 SQL Server Data
  17. 17. Partitioned tables, views and filegroups Table Partitioned Partitioned View Tables payments payments2011 Filegroup1 payments2011 Filegroup2 Filegroup3 payments2010 payments2010 Filegroup4 payments2009 payments2009 Filegroup5 Orders of magnitude: Scale-Out your 17 SQL Server Data
  18. 18. With Distributed Partitioned views Table Distributed Partitioned Partitioned View Tables payments payments2011 FilegroupA1 payments2011 FilegroupA2 SQL Server FilegroupA3 Instance A payments2010 payments2010 SQL Server FilegroupB1 Instance B payments2009 payments2009 FilegroupB2 Orders of magnitude: Scale-Out your 18 SQL Server Data
  19. 19. Peer to Peer Replication Read-Write for User Read-Write for ETL Applications SQL Server SSIS Instance B Application Database X Server SQL Server Instance A Database X Read-Only for Reporting SQL Server Report Instance C Server Database X Orders of magnitude: Scale-Out your 19 SQL Server Data
  20. 20. Hybrid Scale-Out• Database Mirroring with rolling snapshots• SQL Failover Cluster using over-provisioned failover node “hot-swap Scale-Out/Up”.• Use Hyper-Visor, migrate to over-provisioned host server.• Clusters, Peer to Peer, Mirroring on Hyper-Visor Orders of magnitude: Scale-Out your 20 SQL Server Data
  21. 21. Bringing it all together Report Virtual Cluster Nodes Server Virtual Cluster Nodes Cluster A Snapshot Cluster B SQL Failover Clusters Instance E SQL Failover Clusters Database Y Mirror Instance A Instance B Instance C Instance D Database X Database Y Database Y Database X Partition Views Hyper-Visor A Hyper-Visor B Peer to Peer Replication Orders of magnitude: Scale-Out your 21 SQL Server Data
  22. 22. The SQL Server Scale-Out Toolkit• Service Broker • Full Text Indexing• Integration Services • Vertical Partitioning• Replication • Powershell• Horizontally Partitioned Views • CLR• Federated Databases • Linked Servers• Partitioned Views • Filegroups• Log Shipping • Files• Scalable Shared Database • Clustering• Scalable Shared Database for • Backup and Restore Analysis Services • Mirroring• Reporting Services Scale-out • Database Snapshots Deployment • Processor Affinity• Synonyms • Triggers• Schemas • Analysis Services Load• Query Notifications Balancing Orders of magnitude: Scale-Out your 22 SQL Server Data
  23. 23. Taming the Beast• Governance – Policy Based Management – Resource Governor or WRSM – Source Control• Monitoring – MDW and Data Collection – Performance condition alerts – Extended Events – Profiler – DMVs• Naming – SQL Client Aliases w/ GP – DNS Orders of magnitude: Scale-Out your 23 SQL Server Data
  24. 24. In Summary• We discussed – Why we should start thinking about Scaling-Out? – Benefits from Scale-Out – Scaling in before you Scale-Out – Scale-Out strategies – Hybrid Scale-Out strategies – Keeping your scaled environment under control Orders of magnitude: Scale-Out your 24 SQL Server Data
  25. 25. Further References• Books – Apress - Pro SQL Server 2008 Service broker – Klaus Aschenbrenner – Apress - Pro SQL Server 2008 Replication - Sujoy Paul – Morgan Kaufman - DW 2.0 - The Architecture for the Next Generation of Data Warewhousing – William Inmon, Derek Strauss and Genia Neushloss – MS Press - Improving .NET Application Performance and Scalability• Blogs/ Websites – Partitioned Table & Index Strategies Using SQL Server 2008 http://bit.ly/g28zQa – VoltDB.NET: Synchronous vs. Asynchronous Request Processing http://bit.ly/k3rY2N – Data Warehousing 2.0 and SQL Server: Architecture and Vision http://bit.ly/4tRXB4 – Performance Considerations of Data Types – Michelle Ufford http://bit.ly/aq9Wyr• Video/ Webcasts – PASS Summit 2010: AD270S Database Design Fundamentals - Louis Davidson – MCM #17 SQL Server Partitioning – SQLSkills http://bit.ly/ea3C6e Orders of magnitude: Scale-Out your 25 SQL Server Data
  26. 26. Please Complete the Evaluation FormPick up your evaluation form:• In each presentation roomDrop off your completed form• Near the exit of each presentation room• At the registration area Presented by Dell Orders of magnitude: Scale-Out your 26 SQL Server Data
  27. 27. THANK YOU! Presented by DellFor attending this session andPASS SQLRally Orlando, Florida 28 Session Code | Session Title
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×