BLAZING-FAST PERFORMANCEMission Critical Confidence
THE FANTASTIC 12 OF 2012        1         2        3    4        5         6        7    8        9        10        11   12
BLAZING-FAST PERFORMANCEGAIN BREAKTHROUGH & PREDICTABLE PERFORMANCE BACKED BY INDUSTRY-LEADING BENCHMARKS
XVELOCITY MEMORY OPTIMIZED COLUMNSTORE INDEXBoosts query performance by up to 100x
WHAT IS A COLUMNSTORE INDEX?
HOW DOES IT WORK? – ROWS VS. COLUMNS                        Row Store                        1 Phillip WestID   Name      ...
EXAMPLE QUERY
INDEX CREATION AND STORAGE A   B   C   D
I/O AND CACHING
DATA REDUCTIONEarly segment elimination based on segmentmetadataMin and max values stored in metadata for each segmentSimp...
LET THE QUERY OPTIMIZER DO THE WORKOptimizer makes a cost-based decisionData access methodColumnstore indexClustered (row-...
COLUMNSTORE INDEX OBSERVATIONS
DOUBLING OF THROUGHPUT IN BENCHMARK TESTSQL Server 2012 Columnstore IndexesTest SystemFast Track Data Warehouse 3.0 HP DL3...
LOADING NEW DATAData can not be directly modified in a table with aColumnstore indexPartition switching is allowedWays to ...
BEST PRACTICES FOR LOADINGSwitch-in a partition for each load, orDisable, update, rebuild indexQuery on Columnstore tableQ...
PARTITION SWITCHING EXAMPLE (PART 4)
Build CS index on fact tablesConsider for large dimension tablesDon’t use to seek into a rowOrder of listed columns not im...
Star joinsInner joinsGroup By
COLUMNSTORE INDEX BEST PRACTICESBuilding Columnstore IndexesInclude all “relevant” table columns in the columnstoreindexAv...
COLUMNSTORE INDEX BEST PRACTICES (CONT.)Querying Columnstore IndexesStructure queries as *star* joins maximizing grouping/...
COLUMNSTORE INDEX BEST PRACTICES (CONT.)Settings and Memory ManagementMaximum Degree of Parallelism (MAXDOP)No higher than...
COLUMNSTORE INDEX BEST PRACTICES (CONT.)Settings and Memory Management (cont.)Maximum server memoryNo higher than 92% of t...
COLUMNSTORE INDEX BEST PRACTICES (CONT.)Settings and Memory Management (cont.)Trace Flag T834 and “lock pages in memory”Do...
ADDITIONAL RESOURCESColumnstore FAQ:http://social.technet.microsoft.com/wiki/contents/articles/sql-server-columnstore-inde...
SCENARIO: CONSISTENT PERFORMANCE WITH RESOURCEGOVERNOR
SCHEDULER LOAD BALANCING CHANGES
SCENARIO: MULTI-TENANCY WITH RESOURCEGOVERNOR
CONTINUED ENHANCEMENTSBlazing-fast queries via columnar architecture with high compression rates
FULL TEXT SEARCH IMPROVEMENTS IN SQL SERVER 2012• Improved Performance and Scale:  •   Scale-up to 350M documents  •   iFT...
FULL TEXT SEARCH PERFORMANCE & SCALE IMPROVEMENTS• Architectural Improvements  • Improved internal implementation  • Queri...
FTS CODE IMPROVEMENTS
FULLTEXT PROPERTY SCOPED SEARCHNew Search Filter for Document Properties         CONTAINS (PROPERTY ( { column_name }, pro...
FULL-TEXT CUSTOMIZABLE NEAROLD NEAR SYNTAXselect * from fttable where contains(*, test near Space)NEW NEAR USAGES• SPECIFY...
FTS THROUGHPUT & EXECUTION TIME
FTS THROUGHPUT ON REAL WORKLOAD2X Query performance improvement compared with SQL Server 2005
SCALE-UP: FULL-TEXT SEARCH                                                                     2005/8 vs. 2012            ...
SCALE-UP: FULL-TEXT SEARCH                                                                           2005/8 vs. 2012      ...
SERVER PLATFORMS – MANY CORE PROCESSORSCommodity hardware on x64 surpassed performance requirements of alot of customers a...
SERVER PLATFORMS – MANY CORE PROCESSORSExperience was that moving up with many more cores from processor generation to pro...
SERVER PLATFORMS – HYPERTHREADINGLearnings:Current implementation of Intel Simultaneous HT seems to provide 20-30% bettert...
SERVER PLATFORM - NUMAWhat is NUMA?Non-Uniform-Memory-AccessWhich hardware uses it?AMD and Intel based architectures of Op...
SERVER PLATFORM – NUMA
SERVER PLATFORM – NUMAWhat is so special?Usually ‘remote’ memory access is 3-5 times slower than ‘local’ memory accessEspe...
SERVER PLATFORM – NUMAHow does SQL Server recognize NUMA configuration?Hardware Vendors describe NUMA processor and memory...
MEMORYMemory for commodity is relatively cheapNo reason not to equip these servers with ample memoryIn typical scenarios 6...
DISK SUBSYSTEM – ROTATING MEDIAVolume over last 10 years grew by 100 times. Disk access times only decreased byfactor 10. ...
DISK SUBSYSTEM – SSD, ETCMajor difference between consumer products (Laptop/Desktop) and serverstorageLaptop/Desktop are M...
DISK SUBSYSTEM – SSDCommon capabilities of SSDs and other non-rotatingmedia:Read rate often factor 20-40 times higher than...
DISK SUBSYSTEM – SSDWhat to use them for in SQL Server?Log File: shows great effect in cases where limit of outstanding lo...
DISK SUBSYSTEM – SAN/NASConfigurations of different devices by different vendors always is different and requiresproprieta...
DISK SUBSYSTEM - VOLUMESIn Windows Server 2012, SMB got improved majorlySQL Server 2012 will support deployment of databas...
NETWORK - INTRODUCTION10GBit Ethernet adapters are commodity10GBit NICs are lowering demand for # of NICs. However keep ev...
PERFORMANCE
PERFORMANCE FEATURES
TOP STATISTICS
SQL SERVER PROOF POINTS
World’s biggest publicly listed onlinegaming platform15 million page views and up to980,000 unique users a dayEnvironment5...
HIGH AVAILABILITY USING SQL SERVER 2012
Scale UP and Scale OUTScale UP main financial transactionsScale OUT other application functionsHigh Availability3 data cen...
GEO-DR USING SQL SERVER 2012High-Volume OLTP applicationData Centers 500 miles apartBasis for the “Last Man Standing” whit...
LG ELECTRONICSCustomer ProfileLG Electronics is a manufacturing corporation with more than 115 businesses worldwide and mo...
KEY FACTSTechnologies used:Analysis Services 2008 R2 – writebackCustom application built by Wise, local ISVKey Facts:Datab...
SCALING OUT SQL SERVERHow do you:Manage an 80 TB databaseBack up a petabyteBuild an index on a trillion row tableAnswerBre...
PDW APPLIANCE   Compute Rack
INTEGRATION WITH PDW:“HUB AND SPOKE”
SCALABILITY CONCEPT: SODA – SERVICESORIENTED DATABASE ARCHITECTURESeparate your data by business functionExample: HR, Payr...
SERVICE BROKERService broker (SB) is asynchronous, transactional, message, queuing embeddedinto sqlservr.exeAsynchronous: ...
ENABLE LIVE-LIVE USING SERVICE BROKERSB bi-di data sync across DCsR/W on both sitestransparentlyApplication load balancing...
ACTIVE-ACTIVE 5 9’S – IN PRODUCTIONLocal LB: NetScalerSQLMonitor to loadbalance between localDB clustersMinimize chance of...
ACTIVE-ACTIVE 5 9’S – AGAlways On non-sharedstorageAdditional data protectionFaster failoverMulti-castNetScaler SQLMonitor...
NEW CAPABILITIES IN SQL SERVER 2012Message_enqueue_timeHow long a message has been waiting in a destination queueOption to...
INDUSTRY LEADERS RUN ON SQL SERVER
TEMENOS’S T24 SCALES TO 11,500 TRANSACTIONS/ SEC BENCHMARKTEMENOS T24 is a fully integrated, modular corebanking solution ...
OPENTEXT’S ECM SUITE SCALES TO 14.8 MILLIONMESSAGES BENCHMARKOPENTEXT ECM SUITE, the flagship product fromOpenText, is des...
MISYS’S SUMMIT FT SCALES TO 400,000TRANSACTIONS BENCHMARKSummit FT is the Misys cross-asset, front-to-backsolution that sp...
MISYS’S OPICS PLUS SCALES TO 50,000TRADES BENCHMARKOpics Plus is a single platform, fully integratedtreasury and capital m...
CSC’S INTEGRAL SCALES TO 1,000 CONCURRENTUSERS BENCHMARKCSC’S INTEGRAL is a comprehensive administrationsuite for life ins...
SIEMENS PLM SOFTWARE’S TEAMCENTER10,000 CONCURRENT USER BENCHMARKTeamcenter is the leading mission-criticalproduct managem...
REDKNEE TCB 250-MILLION SUBSCRIBERBENCHMARKRedknee is a leading global provider of real-timeconverged billing and customer...
A FEW CASE STUDIES…• New 300K employee Kronos       • New benchmark showing       • Successful 10M account         • Sieme...
TPC BENCHMARK LEADERhttp://www.microsoft.com/sqlserver/en/us/product-info/benchmarks.aspxhttp://www.tpc.org
NEXT STEPSSQL Server 2012 Case Studies:http://www.microsoft.com/casestudies/Case_Study_Advanced_Search.aspx(Search on SQL ...
© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be...
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck
Upcoming SlideShare
Loading in …5
×

Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck

2,792 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Sql server 2012_sp1_02_of_12_blazing-fast_performance_level300_dark_deck

  1. 1. BLAZING-FAST PERFORMANCEMission Critical Confidence
  2. 2. THE FANTASTIC 12 OF 2012 1 2 3 4 5 6 7 8 9 10 11 12
  3. 3. BLAZING-FAST PERFORMANCEGAIN BREAKTHROUGH & PREDICTABLE PERFORMANCE BACKED BY INDUSTRY-LEADING BENCHMARKS
  4. 4. XVELOCITY MEMORY OPTIMIZED COLUMNSTORE INDEXBoosts query performance by up to 100x
  5. 5. WHAT IS A COLUMNSTORE INDEX?
  6. 6. HOW DOES IT WORK? – ROWS VS. COLUMNS Row Store 1 Phillip WestID Name Region 2 Pam East1 Phillip West 3 Cindy West2 Pam East 4 Mohamed Canada3 Cindy West 5 Tim East4 Mohamed Canada5 Tim East Column Store 12345 Phillip Pam Cindy Mohamed Tim West East West Canada East
  7. 7. EXAMPLE QUERY
  8. 8. INDEX CREATION AND STORAGE A B C D
  9. 9. I/O AND CACHING
  10. 10. DATA REDUCTIONEarly segment elimination based on segmentmetadataMin and max values stored in metadata for each segmentSimple filters evaluated in storage engine during CSindex scanConjunctions of comparisons, in-listBitmap filtersEvaluated during index scanBuilt by Hash Table Build operator
  11. 11. LET THE QUERY OPTIMIZER DO THE WORKOptimizer makes a cost-based decisionData access methodColumnstore indexClustered (row-based) indexNonclustered (row-based) indexHeapProcessing modeBatch modeRow mode
  12. 12. COLUMNSTORE INDEX OBSERVATIONS
  13. 13. DOUBLING OF THROUGHPUT IN BENCHMARK TESTSQL Server 2012 Columnstore IndexesTest SystemFast Track Data Warehouse 3.0 HP DL380 reference architectureSQL Server 2008 R220 TB data warehouse2.4 GB/s throughput on 1TB benchmark schemaConfiguration ChangesUpgrade to SQL Server 2012Apply Columnstore indexes to benchmark schemaNo query changesNo hardware changes
  14. 14. LOADING NEW DATAData can not be directly modified in a table with aColumnstore indexPartition switching is allowedWays to load data:Disable index, update (add/delete data), rebuild indexSwitch in new partition
  15. 15. BEST PRACTICES FOR LOADINGSwitch-in a partition for each load, orDisable, update, rebuild indexQuery on Columnstore tableQuery on rowstore “delta” table
  16. 16. PARTITION SWITCHING EXAMPLE (PART 4)
  17. 17. Build CS index on fact tablesConsider for large dimension tablesDon’t use to seek into a rowOrder of listed columns not important
  18. 18. Star joinsInner joinsGroup By
  19. 19. COLUMNSTORE INDEX BEST PRACTICESBuilding Columnstore IndexesInclude all “relevant” table columns in the columnstoreindexAvoid completely unique fields Avoid Unique identifiers Avoid Comment fieldUse columnstore indexes on only larger fact anddimension tables
  20. 20. COLUMNSTORE INDEX BEST PRACTICES (CONT.)Querying Columnstore IndexesStructure queries as *star* joins maximizing grouping/aggregationbenefitsAvoid joins & filters using strings against columns of columnstoreindexed tablesMinimize queries using “Outer join”, “Union all”, and “Not in”Lean towards integers, surrogate keys, and *star schema*Leverage table partitions
  21. 21. COLUMNSTORE INDEX BEST PRACTICES (CONT.)Settings and Memory ManagementMaximum Degree of Parallelism (MAXDOP)No higher than 32, 16-24 sweet spotResource Governor Memory Grant for workload groupsHigher than normal, Columnstore queries need lots of memory
  22. 22. COLUMNSTORE INDEX BEST PRACTICES (CONT.)Settings and Memory Management (cont.)Maximum server memoryNo higher than 92% of total available RAM
  23. 23. COLUMNSTORE INDEX BEST PRACTICES (CONT.)Settings and Memory Management (cont.)Trace Flag T834 and “lock pages in memory”Do not use if you are using columnstore indexes. (Until Cumulative Update in November)
  24. 24. ADDITIONAL RESOURCESColumnstore FAQ:http://social.technet.microsoft.com/wiki/contents/articles/sql-server-columnstore-index-faq.aspx Includes a resource list http://social.technet.microsoft.com/wiki/contents/articles/sql-server-columnstore- index-faq.aspx#More_details_on_columnstore_technologyColumnstore Tuning Guide:http://social.technet.microsoft.com/wiki/contents/articles/sql-server-columnstore-performance-tuning.aspx
  25. 25. SCENARIO: CONSISTENT PERFORMANCE WITH RESOURCEGOVERNOR
  26. 26. SCHEDULER LOAD BALANCING CHANGES
  27. 27. SCENARIO: MULTI-TENANCY WITH RESOURCEGOVERNOR
  28. 28. CONTINUED ENHANCEMENTSBlazing-fast queries via columnar architecture with high compression rates
  29. 29. FULL TEXT SEARCH IMPROVEMENTS IN SQL SERVER 2012• Improved Performance and Scale: • Scale-up to 350M documents • iFTS query perf 7-10 times faster than in SQL Server 2008 • Worst-case iFTS query response times < 3 sec for corpus • At par or better than main database search competitors• New Functionality: • Property Search • customizable NEAR • New Word Breakers: update existing WB, add Czech and Greek• Innovation in Search: • Semantic Similarity Search
  30. 30. FULL TEXT SEARCH PERFORMANCE & SCALE IMPROVEMENTS• Architectural Improvements • Improved internal implementation • Queries no longer block Index updates • Improved Query Plans: o Better Plans for common queries o Fulltext predicate folding o Parallel Plan execution• Index and Query tested on scale up to 350Million documents with < ~2 Sec Response• ~3X better w/o DML and ~9X better with DML throughput• Scale easily with increasing number of connections
  31. 31. FTS CODE IMPROVEMENTS
  32. 32. FULLTEXT PROPERTY SCOPED SEARCHNew Search Filter for Document Properties CONTAINS (PROPERTY ( { column_name }, property_name ), ‘contains_search_condition’ )• Setup once per database instance to load the office filters exec sp_fulltext_service load_os_resources,1 go exec sp_fulltext_service restart_all_fdhosts go• Create a property list CREATE SEARCH PROPERTY LIST p1;• Add properties to be extracted ALTER SEARCH PROPERTY LIST [p1] ADD NSystem.Author WITH (PROPERTY_SET_GUID = f29f85e0-4ff9-1068-ab91-08002b27b3d9, PROPERTY_INT_ID = 4, PROPERTY_DESCRIPTION = NSystem.Author);• Create/Alter Fulltext index to specify property list to be extracted ALTER FULLTEXT INDEX ON fttable... SET SEARCH PROPERTY LIST = [p1];• Query for properties SELECT * FROM fttable WHERE CONTAINS(PROPERTY(ftcol, System.Author), fernlope);
  33. 33. FULL-TEXT CUSTOMIZABLE NEAROLD NEAR SYNTAXselect * from fttable where contains(*, test near Space)NEW NEAR USAGES• SPECIFY DISTANCE select * from fttable where contains(*, near((test, Space), 5,false))• REDUCE DISTANCE select * from fttable where contains(*, near((test, Space), 2,false))• ORDER OF WORDS IS SPECIFIED AS IMPORTANT select * from fttable where contains(*, near((test, Space), 5,true))
  34. 34. FTS THROUGHPUT & EXECUTION TIME
  35. 35. FTS THROUGHPUT ON REAL WORKLOAD2X Query performance improvement compared with SQL Server 2005
  36. 36. SCALE-UP: FULL-TEXT SEARCH 2005/8 vs. 2012 2005/8 2012Queries over 350M documents database and random DMLs running in background.Beating SQL Server 2005 with a scale factor more than 2x and with avg 60x times better throughput
  37. 37. SCALE-UP: FULL-TEXT SEARCH 2005/8 vs. 2012 2005/8 2012Query avgExecTime (ms) under various number of connections (50 ~ 2000 users) for customer playback benchmark
  38. 38. SERVER PLATFORMS – MANY CORE PROCESSORSCommodity hardware on x64 surpassed performance requirements of alot of customers and will continue to increase throughput dramaticallyMost recent generation of processors look like:Intel Xeon E5 26xx: 1 socket=8 cores=16 threadsIntel Xeon E7 87xx: 1 socket=10 cores=20 threadsAMD Opteron 617x: 1 socket=16cores=16threads2-socket server does have 32 CPU threads4-socket server does have 64 (AMD)/80 CPU threads8-socket server does have 160 CPU threads (Intel only)New Intel Ivybridge-EX expected next year. Expect up to 30 CPU threadsper socketExpected that in a few years we are going to look into 25-core processors
  39. 39. SERVER PLATFORMS – MANY CORE PROCESSORSExperience was that moving up with many more cores from processor generation to processorgeneration, single CPU thread performance remained the same or even fell backPerformance of a single CPU thread often vital for queries which are single threadedOther sensitive areas critical sections within SQL Server. E.g. Writing into the Log Buffer,Lockhash, etcLooking at Single CPU performance using an ISV benchmark which measures a unit of SAPSlooks like:Large proprietary Unix server even worse:Sun SPARC M9000, 32 sockets, 256 cores,512 CPUs: 340SAPS/CPUIBM Power 795, 32 sockets, 256 cores,1024 CPUs: 650SAPS/CPUIBM Flex System p460 , 4sockets, 32 cores, 128 CPUs: 730SAPS/CPUIBM Power 795, 16 sockets, 128 cores, 512 CPUs: 750SAPS/CPUHP DL980, 8 sockets, 80 cores, 160 CPUs: 860SAPS/CPUHP DL580, 4 sockets, 40 cores, 80 CPUs: 920SAPS/CPUHP DL380 Gen8, 2 sockets, 16 cores, 32 CPUs: 1340SAPS/CPU
  40. 40. SERVER PLATFORMS – HYPERTHREADINGLearnings:Current implementation of Intel Simultaneous HT seems to provide 20-30% betterthroughput under most types of workloadsNew Intel Hyperthreading did show throughput performanceincrease under all possible SQL Server workloadsCertainly does not speed up a single batch where none of thequeries can be executed in parallelThroughput increase usually in the area of 20-30% means per CPUthread performance drops
  41. 41. SERVER PLATFORM - NUMAWhat is NUMA?Non-Uniform-Memory-AccessWhich hardware uses it?AMD and Intel based architectures of Opteron and XeonWhat is the principle problem for applications like SQL Server withNUMA architecturesMemory Access time varies dependent on local memory access or remote memory accessApplications like SQL Server need to balance memory allocations over differentNUMA nodes
  42. 42. SERVER PLATFORM – NUMA
  43. 43. SERVER PLATFORM – NUMAWhat is so special?Usually ‘remote’ memory access is 3-5 times slower than ‘local’ memory accessEspecially in chipsets >4 sockets one can eventually get 2 steps to memorynode furthest awayConclusionSoftware needs to be aware of hardware architectureWindows and SQL Server adapted massively over the last 10 years to keep vitalstructure NUMA or even CPU aligned in order to provide local memory accessEvery release of both products gets improvements in this space – continuouswork in that space
  44. 44. SERVER PLATFORM – NUMAHow does SQL Server recognize NUMA configuration?Hardware Vendors describe NUMA processor and memory configurationin HAL via SRAT interface to WindowsSQL Server checks hardware configuration via Win32 APIs and configuresto NUMA accordinglyStandard by hardware vendors today:Intel Xeon E5: 1-socket = 1 NUMA nodeIntel Xeon E7: 1-socket = 1 NUMA nodeIntel Xeon 56xx: 1-socket = 1 NUMA nodeAMD Opteron 61xx: 1-socket = 2 NUMA nodesAMD Opteron 62xx: 1-socket = 2 NUMA nodes
  45. 45. MEMORYMemory for commodity is relatively cheapNo reason not to equip these servers with ample memoryIn typical scenarios 6-8GB per processor core is goodEventually more needed for specific OLAP purposes when data shouldremain in memoryE.g. SQL Server 2012 Column StoreColumn Store keeps the data in segments of 4-16MBOne segment at least per columnHence single I/Os to load a segment are taking longerDesire to keep the segments of column store as long as possible in the cache
  46. 46. DISK SUBSYSTEM – ROTATING MEDIAVolume over last 10 years grew by 100 times. Disk access times only decreased byfactor 10. E.g.:1990: Seagate Hard Disk could provide 37.5 IOPS/GB volume2007: Seagate Hard Disk could provide 2.4 IOPS/GB volume Sizing of # of disks in high-end systems is not a matter of predicted data volume butmatter of expected I/O workload and I/O volumeRotating 2.5’’ 15K RPM is good to deliver around 150 -200 random reads of 8KB size withresponse times of <10msSequential Read bandwidth around 10 times higher than random read bandwidth. E.g. Read3TB from one rotating spindle:Takes 31 days with 140 random IOPS of 8KBTakes 8.3h reading sequentiallyNo increase in RPM expected anymore – Major SAN vendors use 10K RPMOnly slow improvements in seek times in the futurePrice: <<USD 1/GB. Power: 7-15W
  47. 47. DISK SUBSYSTEM – SSD, ETCMajor difference between consumer products (Laptop/Desktop) and serverstorageLaptop/Desktop are Multi Layer Cells( MLC), less sophisticated write logic results in lower lifetime with costs of aroundUSD 1/GBServer Storage products are eMLC with more sophisticated write logic providing higher lifetime (MTB) with a price ofaround USD 5-6/GBServer Storage products are Single Layer Cell with extreme sophisticated write logic providing high lifetime (MTB) with aprice of around USD >10/GBConsume between 1-3 Watt per unitMostly NAND based Flash devicesLaptop/Desktop MLCs are NOT suitable for serversand applications usually running on a serverMinimum required is eMLC
  48. 48. DISK SUBSYSTEM – SSDCommon capabilities of SSDs and other non-rotatingmedia:Read rate often factor 20-40 times higher than rotating media with dramaticallylower latencyEspecially for SLCs write latency similar to latency of rotating media by SANCacheExercises with different workloads proofed that moving from rotating spindlesto SSD can reduce # of units by factor of 10 with same and often even betterI/O latencyReduction provides large boost in Power Savings
  49. 49. DISK SUBSYSTEM – SSDWhat to use them for in SQL Server?Log File: shows great effect in cases where limit of outstanding log I/Os (128 since SQL Server 2008)and/or amount of outstanding bytes (3.84MB since SQL Server 2008) is hit despite having latencies 3-4 ms – usually seen in workload with massive data changes in highly concurrent manner or massiveparallel inserts Ideal to be covered with Direct attached SSD, FusionIO or Violin type of storageTempDB: Usually shows great effects in workloads which either perform massive hash spill outs or loadtempDB with a lot of versioning traffic (read committed snapshot, snapshot isolation). Ideal to be covered with Direct attached SSD or FusionIOData files: shows great effect when massive amount of random I/O are requiring large amounts ofdisks which are in not ratio to the volume of data Ideal are all sorts of SSDFor SSDs in SAN still expect latencies for random I/O in the space of 2-3msReason is the stack of Hardware/SW before you hit the spindle
  50. 50. DISK SUBSYSTEM – SAN/NASConfigurations of different devices by different vendors always is different and requiresproprietary toolsDon’t overestimate huge SAN/NAS caches. Real life cache hit ratio around 30-40%Throughput can be impacted by SAN ReplicationTest I/O throughput carefully with any type of SAN replication. Especially sequential writes into SQL ServerTransaction LogCheck HBA Queue Depth – Default of 32 usually too low. Good values 128 to 256(dependent on vendor)Configure for multipathing and workloadbalancingScale volume throughput with # of adaptersStorage-Tiering by SAN providers using SSDs combined with different grade rotatingspindle media not yet proven to work great under all load profiles of business applications
  51. 51. DISK SUBSYSTEM - VOLUMESIn Windows Server 2012, SMB got improved majorlySQL Server 2012 will support deployment of database andtransaction log files over Windows Server 2012 SMBWindows Server 2012 SMB 3.0 doesn’t lose any in flightIOPS anymoreHence alternative model of storage backend could be:Two clustered servers connecting to shared JBODs acting as file serverSQL Server instance deployed on different nodes placing accessing their filesover SMB from file server
  52. 52. NETWORK - INTRODUCTION10GBit Ethernet adapters are commodity10GBit NICs are lowering demand for # of NICs. However keep eventual failureof NIC in mind (minimum 2 NICs)Be careful… Network can become a bottleneck even if bandwidth notexhausted. Possible factors:Excessively high interrupts/secDPCs localized to single processorProcessor handling interrupts/DPCs is maxed outSQL Server IO completion thread overburdenedWith Network intensive client/server applications we usually don’t see issues inregards to bandwidth, but more around interrupt/DPC handlingWith applications loading intensive data, the bandwidth is becoming important
  53. 53. PERFORMANCE
  54. 54. PERFORMANCE FEATURES
  55. 55. TOP STATISTICS
  56. 56. SQL SERVER PROOF POINTS
  57. 57. World’s biggest publicly listed onlinegaming platform15 million page views and up to980,000 unique users a dayEnvironment5 DBA’s & 1 Database Architect100+ SQL Server Instances120+ TB of data,1,400+ Databases1,600+ TB storage5,000+ GB RAM450,000+ SQL Statements per second on a single server500+ Billion database transactions per dayNo downtime allowed
  58. 58. HIGH AVAILABILITY USING SQL SERVER 2012
  59. 59. Scale UP and Scale OUTScale UP main financial transactionsScale OUT other application functionsHigh Availability3 data centers2 Synchronous Availability Group copies adds 1-4 ms per transactionReplication for reportingLog Shipping for DR replaced by Async Availability Group replicaBackup 2 TB per hour over the networkhttp://sqlcat.com/whitepapers/archive/2009/08/13/a-technical-case-study-fast-and-reliable-backup-and-restore-of-a-vldb-over-the-network.aspxCase studyhttp://www.microsoft.com/casestudies/Microsoft-SQL-Server-2012/bwin.party/Company-Cuts-Reporting-Time-by-up-to-99-Percent-to-3-Seconds-and-Boosts-Scalability/710000000087
  60. 60. GEO-DR USING SQL SERVER 2012High-Volume OLTP applicationData Centers 500 miles apartBasis for the “Last Man Standing” whitepaper
  61. 61. LG ELECTRONICSCustomer ProfileLG Electronics is a manufacturing corporation with more than 115 businesses worldwide and morethan 80,000 employeesBusiness SituationNew demands for analysis based on Demand Forecasting System, the heart of SCM are Increasing.SolutionTo meet this challenge, LG Electronics built demand forecasting related BI system based on SQL Server2008 R2, offering much improved performance and convenience than the legacy system.BenefitsImproved transaction performanceDeployed adaptable BI with effective investmentIncreased usage value of BI system
  62. 62. KEY FACTSTechnologies used:Analysis Services 2008 R2 – writebackCustom application built by Wise, local ISVKey Facts:Database Size: 5TBAvg query response time: 30 seconds.3,500 registered users with average 11 concurrent users.LGE used writeback for the bottom-up level changes when they used t-sql updates and MOLAPpartitions for the top-down changes to balance performance levels. The cubes were partitioned by divisions so each one can process their partitions frequently without affecting the rest.History is 5 years * 80 million * 6 = 2.4 billion recordsData is increasing daily by minimum of 100 thousand to maximum of 5 million cells .
  63. 63. SCALING OUT SQL SERVERHow do you:Manage an 80 TB databaseBack up a petabyteBuild an index on a trillion row tableAnswerBreak it into manageable size pieces
  64. 64. PDW APPLIANCE Compute Rack
  65. 65. INTEGRATION WITH PDW:“HUB AND SPOKE”
  66. 66. SCALABILITY CONCEPT: SODA – SERVICESORIENTED DATABASE ARCHITECTURESeparate your data by business functionExample: HR, Payroll, Accounting, etcOr by user functionExample: Login, chat, email, pictures, etcEach function goes in a different databaseA common database for shared tables
  67. 67. SERVICE BROKERService broker (SB) is asynchronous, transactional, message, queuing embeddedinto sqlservr.exeAsynchronous: parallel stored proceduresQueuing: handle peak workloadsDistributed: scalabilityDecoupled: availabilityFeaturesAutomatic activationSingle queue with multiple readersRelated messages locked and processed by the same readerCross database, instance and machine boundarieswithout changing application codeAny program issuing TSQL, (e.g., a C# middletier Windows Service), is a SB endpoint
  68. 68. ENABLE LIVE-LIVE USING SERVICE BROKERSB bi-di data sync across DCsR/W on both sitestransparentlyApplication load balancingcross DCsLocal HA using MSCSReal-time DR with zerodowntime if one DC goesoffline for plannedmaintenance or unplannedoutagesAvailability: ~99.999%
  69. 69. ACTIVE-ACTIVE 5 9’S – IN PRODUCTIONLocal LB: NetScalerSQLMonitor to loadbalance between localDB clustersMinimize chance of userlatency increase due to DCoffline..or mobile users fromdifference geoReduce the need to DC failoverfor planned maintenance 4 x 3 SB routesRemoved log shipping
  70. 70. ACTIVE-ACTIVE 5 9’S – AGAlways On non-sharedstorageAdditional data protectionFaster failoverMulti-castNetScaler SQLMonitorfor local load balancingSite resiliency
  71. 71. NEW CAPABILITIES IN SQL SERVER 2012Message_enqueue_timeHow long a message has been waiting in a destination queueOption to disable poison message handlingAllow application to CATCH the error without disabling the queue with its ownpoision message handlingMulticastSending message to multiple destinations without multiple SEND statements SEND ON CONVERSATION (@conv_handle_1 [,.. @conv_handle_n]) [ MESSAGE TYPE message_type_name ] [ ( message_body_expression ) ]AlwaysOn Support via AG listenerSSB route specifics a VIPVIP maps to the primary of an AGAutomatic re-connection after failover
  72. 72. INDUSTRY LEADERS RUN ON SQL SERVER
  73. 73. TEMENOS’S T24 SCALES TO 11,500 TRANSACTIONS/ SEC BENCHMARKTEMENOS T24 is a fully integrated, modular corebanking solution that covers a broad spectrum offunctional requirements for the retail, private,corporate, universal, and Islamic banking andmicrofinance sectors.Benchmark testing was completed in September, 2011Temenos’s T24SQL Server 2012Database server with Intel Xeon processor E7-2870 (80x2.4 GHz)The results beat the performance goals11,592 transactions / secBelow 75% CPU utilization during peak
  74. 74. OPENTEXT’S ECM SUITE SCALES TO 14.8 MILLIONMESSAGES BENCHMARKOPENTEXT ECM SUITE, the flagship product fromOpenText, is designed to help customers createvalue from content by empowering people,fostering process agility, and controlling the riskand cost of content.Benchmark testing was completed in February, 2012OpenText ECM SuiteSQL Server 2012The results beat the performance goalsPeak email ingestion: 14.8 million messages in 24 hoursConcurrent email ingestion: 82 messages per second sustained
  75. 75. MISYS’S SUMMIT FT SCALES TO 400,000TRANSACTIONS BENCHMARKSummit FT is the Misys cross-asset, front-to-backsolution that specializes in derivatives and structuredproducts for global investment banks.Benchmark testing was completed in January, 2012Misys’s Summit FTSQL Server 2012Database server with Intel Xeon processor E7-4870 (8x2.40 GHz)The results beat the performance goalsPeak loads: Burst 5,000 trades FX Spot and FX SwapSustained loads: 400,000 trades a day FX Spot and FX Swap
  76. 76. MISYS’S OPICS PLUS SCALES TO 50,000TRADES BENCHMARKOpics Plus is a single platform, fully integratedtreasury and capital markets solution with richstraight-through-processing (STP), treasury and back-office functionality.Benchmark testing was completed in February, 2012Misys’s Opics PlusSQL Server 2012NEC Express 5800a-E database server with Intel Xeon processorE7-4870 (2.4 GHz)The results beat the performance goalsBatch processing: 50,000 trades in 19 minutesTrade entry through Web: 260 trades per second
  77. 77. CSC’S INTEGRAL SCALES TO 1,000 CONCURRENTUSERS BENCHMARKCSC’S INTEGRAL is a comprehensive administrationsuite for life insurance and annuities/pensions,property and casualty (P&C), general insurance, andgroup insurance.Benchmark testing was completed in March, 2012CSC’s Integral property and casualty (P&C) softwareSQL Server 2012Database server with Intel Xeon 4-socket processor (32x2.0 GHz)The results beat the performance goalsAverage only 10 percent CPU utilization using less then 20 GB RAMLinear Scalability: 1 application server test (333 concurrent users with 497 business transactions per minute) 5 application servers test (1,000 concurrent users with 1,952 business transactions per minute)
  78. 78. SIEMENS PLM SOFTWARE’S TEAMCENTER10,000 CONCURRENT USER BENCHMARKTeamcenter is the leading mission-criticalproduct management lifecycle (PLM) solutionused by tier-1 enterprises around the worldBenchmark testing was run in Nov, 2011Teamcenter 8.3SQL Server 2012NEC Express 5800/A 1080a database server with Intel Xeon processorE7-8870 (2.4 GHz)The results were impressive!Teamcenter scaled to 10,000 concurrent users withexcellent performanceAverage CPU utilization was nearly linear with concurrent user count
  79. 79. REDKNEE TCB 250-MILLION SUBSCRIBERBENCHMARKRedknee is a leading global provider of real-timeconverged billing and customer care solutionsfor communications service providersBenchmark testing was completed in Dec, 2011Redknee Turnkey Converged Billing (TCB)SQL Server 2012NEC Express5800/A1080a database server with Intel Xeon processorE7-8870 (2.4 GHz)The results beat the performance goalsGenerated 26 million invoices within a 6 hour billing cycleMediated 4.9 billion call detail records (CDRs) within 12 hours,showing a nearly linear scalability as number of subscribers increasedProcessed an average of 1,249 invoices/sec at peakMediated an average of 113,402 CDRs/sec at peak
  80. 80. A FEW CASE STUDIES…• New 300K employee Kronos • New benchmark showing • Successful 10M account • Siemens PLM software • Benchmark results Work Force Central on SQL PTC Windchill up to 10% benchmark shows 3,100 performs scales to 5,000 show ENOVIA scales Sever 2008 benchmark faster on SQL Server than tps at significant cost concurrent users on SQL up to at least 5,000 published on Oracle savings over Oracle/IBM Server 2008 collaborative users on SQL Server 2008• Customer wins include worlds • Customer wins include • Customer wins include • Customer wins include largest retailers Costco Quanta Computers (largest SinoPac Holdings (largest Volvo Aero, Sandvik, • Customer wins include Wholesale and , Home Depot Taiwan computer maker), Taiwan bank), BSM Callaway Golf ITER, Hoffman Pentair, Penske Racing (Indonesia’s largest Islamic Piaggio Aero bank)
  81. 81. TPC BENCHMARK LEADERhttp://www.microsoft.com/sqlserver/en/us/product-info/benchmarks.aspxhttp://www.tpc.org
  82. 82. NEXT STEPSSQL Server 2012 Case Studies:http://www.microsoft.com/casestudies/Case_Study_Advanced_Search.aspx(Search on SQL Technologies)SQL Server 2012 Hands On Labs:http://www.microsoft.com/sqlserver/en/us/learning-center/virtual-labs.aspxSQL Server 2012 Certification:http://www.microsoft.com/learning/en/us/certification/cert-sql-server.aspxSQL Server 2012 Best Practices:http://technet.microsoft.com/en-us/sqlserver/bb671430
  83. 83. © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademar ks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accu racy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

×