Many People know FastTrack as a reference architecture for relational databases. The goal of this guideline is to provide a reference architecture for scalable and fast Analysis Services solutions.
3. TEAM
Thomas
Kejser
CTO EMEA at Fusion-
io
Alexei
Khalyako
SQL CAT Program
Manager at Microsoft
Marcel Franke
Practice Lead at pmOne
Gerhard Brückl
Practice Lead at pmOne
SSAS Maestro
Hannes Mayer
Practice Lead at pmOne
SSAS Maestro
4. Build scalable solutions on user demand
Choose of an OLAP optimized hardware configuration
Align SSAS configuration to hardware settings
CUSTOMER CHALLENGES
7. What makes a good hardware for SQL Server Analysis Services
CPU
More cores more parallelism for Storage Engine bound queries
High clock rate faster Formula Engine bound queries
I/O
SSAS uses random reads
No spinning disks - use Fusion-IO drives instead
Use local disks to avoid unnecessary fiber channel/iSCSI cost
Memory
More memory more data can be loaded into memory
In ideal case: entire cube could be loaded into memory
GENERAL HARDWARE RECOMMENDATIONS
8. Server: HP DL380 Gen8
CPU
2.90 GHz. 8 Cores. 16 Logical Processors
Good mix between speed and number of cores
Hyper threading to 32 logical cores to the OS
I/O
Local Fixed Disk (4,38 TB). 4 x Fusion IO Drive 2 Stripe Set (4x1,2TB)
Very high IOPs (random reads: 140.000/s)
Very high throughput (random reads: 4,3GB/s)
Memory
256 GB physical memory
OS: Windows Server 2012 Datacenter
REFERENCE HARDWARE CONFIGURATION
11. 2 SQL Server Analysis Services Databases
100 GB – fits into memory
1.000 GB – does not fit into memory
3 query-patterns
7 Standard queries for common business questions
Ratio-to-Total, Rolling-12-month, Year-over-Year growth, …
1 DistinctCount query
1 Many-to-Many query
Increasing number of users
Incrementally grow number of connected users – 1 every 10 sec
Up to 200 concurrent users
TEST OVERVIEW
13. “How many queries can be answered
within a given time period“
DEFINING CONCURRENCY
14. EXPECTED RESULTS
(1) Constant test duration
(2) Until saturation point is reached
(3) Linear increase of test duration together with concurrent users
(1) (3)(2)
Avg. query
response time
15. More than 100+ concurrent users (connections) supported
For standard queries
With Average response time <3 seconds
Regardless of database size
Complex queries impact average response time
DistinctCount and Many-to-Many queries may not finish before the
next user connects – especially for the 1,000 GB cube
Usually:
- not every query has the pattern of a complex query
- executed rarely compare to the standard queries
- can be avoided by appropriate cube design
Scalability depends on CPU resources. I/O is not a limiting factor
anymore
FINAL RESULTS – OVERVIEW
16. Constant query execution time till CPU limit reached with growing
numbers of concurrent users
FINAL RESULTS – OVERVIEW
* Query 20
17. An average of 285 concurrent users with a response time below 3
seconds for standard queries
Only 2 concurrent users for more complex queries
FINAL RESULTS – DETAILS 100 GB
Query-Pattern Query
AvgTests/s
(@SP)
Max
Avg Tests/s
Median
Avg Tests/s
Queries
/ Test
Supported Concurrent Users/Queries
3s Response Time 10s Response Time
Standard Query2 360.0 513.2 429.8 1 1.080 3,600
Standard Query3 4.5 9.4 4.0 1 14 45
Standard Query20 22.5 41.0 22.0 1 68 225
Standard Query21 14.0 17.2 13.0 1 42 140
Standard Query22 1.5 4.2 1.0 1 5 15
Standard Query52 250.0 505.6 323.4 1 750 2,500
Standard Query77 13.0 88.0 14.8 1 39 130
DistinctCount Query100 0.6 2.8 0.6 1 2 6
Many-to-Many Query101 6.8 112.0 76.2 1 20 68
AllQueries AllQueries 0.8 260.4 0.2 9 2 8
(@SP) = @saturation point
18. An average of 210 concurrent users with a response time below 3
seconds for standard queries
more complex queries run longer than our required response time
which means we cannot even satisfy 1 concurrent user
FINAL RESULTS – DETAILS 1000 GB
Query-Pattern Query
AvgTests/s
(@SP)
Max
Avg Tests/s
Median
Avg Tests/s
Queries
/ Test
Supported Concurrent Users/Queries
3s Response Time 10s Response Time
Standard Query2 250,0 409,8 373,2 1 750 2.500
Standard Query3 0,4 3,4 - 1 1 4
Standard Query20 10,0 16,6 10,5 1 30 100
Standard Query21 6,6 11,4 6,6 1 20 66
Standard Query22 1,0 4,4 0,8 1 3 10
Standard Query52 220,0 388,6 304,2 1 660 2.200
Standard Query77 0,1 5,4 0,2 1 0 1
DistinctCount Query100 - 0,2 - 1 - -
Many-to-Many Query101 - 0,6 - 1 - -
AllQueries AllQueries - 2,6 - 9 - -
(@SP) = @saturation point
19. Shown great linear, CPU bound scale for fast queries
Concurrency is related to the number of queries not to the number of
users
No user would run more than one query per second
Caching helps to avoid expensive IO operations
NUMBER OF USERS VS. QUERY RESPONSE TIME
20. Bigger cube more IO
slower query response times
Bigger cube not everything can be loaded into memory
Constant IO going on slower query response times
2 possible solutions to handle more data
Better IO system
More memory
(More CPU to handle additional threads)
QUERY RESPONSE TIME VS. DATA VOLUME
22. This sample configuration can be used as reference architecture for
building high scalable OLAP solutions.
With given configuration we satisfied the SLAs for the cube up to 1TB
in size and 200+ concurrent queries.
Capacity calculation
The average query per core ratio is 6,5.
If more than 200 concurrent queries are expected then this ratio can be
used to calculated necessary number of cores.
Sample: 400 concurrent queries = 400/6,5 = 64 cores
CONCLUSION
Accelerate Data Warehouse projects with scalable reference architecture for SQL Server Analysis ServicesReduce costs, save time and reduce risk with reliable reference architectures and best practices Suitable for typical workloads (OLAP-DB size, query pattern, number of users)Reference Architecture consists of:Best Practices for SQL Server Analysis Services settingsHardware recommendations for typical workloadsAligned configurations of software, hardware and OSShow Scalability of SSAS and reference architectureProvide test cases & results