Agenda
World Record Breaking Performance (TPC-H)
SQL Server Leading the TPC-H benchmark
SQL Server 2016: Columnstore Improvements
Performance
Functionality: Data Warehouse (DW)
Functionality: Real-Time Operational Analytics
World Record Breaking Performance (TPC-H)
SQL Server 2016 gives 40% improved
performance over SQL Server 2014
Customer – Daman (Health Insurance, UAE)
Scenario
• Traditional DW with ~100 target tables, ~200 source
tables several billion rows
• Multi-layered, holistic, dimensional model, used by all
users
• Queries involve multi-table joins
Customer – Daman Requirements/Challenges
Requirements
Challenges
Approach
JulJunMayAprMarFebJan
2015 2016
DecNov
Go: POC SQL Server 2016
POC SQL Server 2016 Go
Production
Go
In the cloud
6 contenders
Synthetic data
Same data model
Similar characteristics
On the premises
SQL Server 2016 vs.
Sybase IQ
Full productive data set Daily load on SQL Server 2016
Automated test framework
Full test end user tools
Customer – Daman: Benchmark Results
Solution - CCI
Results – testing & production
50 reports weekly
Scheduled Reports
60% improvement
SQL 2016: 10 h
Sybase IQ: 28 h
Weekly
Runs at EOM
Business Tool
5 x improvement
SQL 2016: 10h
Sybase IQ: 2 d
Monthly
100s with 30 concurrent users
AdHoc Analytics
Significant
Improvement
Daily
Many queries < 20
seconds
Demo – Screen Shots
Customer Daman: Learnings
ETL
Optimizer
Case Study
https://customers.microsoft.com/en-us/story/daman
First American - Business Problem
• Scope of Data:150 Million Properties in USA
• Querying
• Not a traditional Columnstore Index Scenario
• Failed Solution
First American - Requirements
Success Criteria
Examples of Slow SQLs
SELECT
FROM
WHERE
SELECT
FROM
WHERE
19
Relational Table (disk-based)
(Clustered Index)
20
Btree index
Rowstore Table (disk-based)
Hybrid queries are directed appropriately
Challenge
Solution
First American - Hybrid Query
SELECT <Select List>
FROM NormandySearch.Search.Property
WHERE propertyid in(SELECT propertyid
FROM NormandySearch.Search.PropertyCS
WHERE [NumericStreetNumber] = 416
and OwnerName like 'BOW%‘ And ( [City] = 'Bay Village'
Or [PlaceName] = 'Bay Village')
First American - Results with Columnstore Index
Data Compression
CPU Consumption
Query Performance with 100 concurrent users
With Rowstore (PAGE Compressed) With CCI
Database Size (including Indexes) 560 GB 44 GB (13x)
First American - Learnings
FIS – Business Overview and Challenges
Application Overview
Current Solution
FIS – Application Challenges
Challenge
No Columnstore Index Benefits
What to do?
FIS – Solution
Considered two configurations
Baseline with all 1.2 million rows compressed
Delivered solution with NCCI with force compression
FIS – Performance Numbers
Status - Application Live in Production
FIS Case Study
what something is instead of where it's stored.
Organize information based on
Proposal
ESTT Corporation
9/30/2016
Website Renewal
Y:
Proposals 2016
Projects
ESTT
Official docs
Website renewal
M-FILES – Document Management
Each document inserted
results in approximately
60 SQL inserts into
Metadata
Metadata change in a
document can result
updating metadata of
large number of other
documents
Document loading and Viewing
Metadata Table
Btree Index
Query Before (sec) After (sec)
Contracts by Pricing Type and Agency 6.5 0.5
Contracts by year and month 2.6 0.05
Thank you
for your time!

Columnstore Customer Stories 2016 by Sunil Agarwal

  • 2.
  • 4.
    World Record BreakingPerformance (TPC-H) SQL Server Leading the TPC-H benchmark
  • 5.
    SQL Server 2016:Columnstore Improvements Performance Functionality: Data Warehouse (DW) Functionality: Real-Time Operational Analytics
  • 6.
    World Record BreakingPerformance (TPC-H) SQL Server 2016 gives 40% improved performance over SQL Server 2014
  • 8.
    Customer – Daman(Health Insurance, UAE) Scenario • Traditional DW with ~100 target tables, ~200 source tables several billion rows • Multi-layered, holistic, dimensional model, used by all users • Queries involve multi-table joins
  • 9.
    Customer – DamanRequirements/Challenges Requirements Challenges
  • 10.
    Approach JulJunMayAprMarFebJan 2015 2016 DecNov Go: POCSQL Server 2016 POC SQL Server 2016 Go Production Go In the cloud 6 contenders Synthetic data Same data model Similar characteristics On the premises SQL Server 2016 vs. Sybase IQ Full productive data set Daily load on SQL Server 2016 Automated test framework Full test end user tools
  • 11.
    Customer – Daman:Benchmark Results Solution - CCI
  • 12.
    Results – testing& production 50 reports weekly Scheduled Reports 60% improvement SQL 2016: 10 h Sybase IQ: 28 h Weekly Runs at EOM Business Tool 5 x improvement SQL 2016: 10h Sybase IQ: 2 d Monthly 100s with 30 concurrent users AdHoc Analytics Significant Improvement Daily Many queries < 20 seconds
  • 13.
  • 14.
    Customer Daman: Learnings ETL Optimizer CaseStudy https://customers.microsoft.com/en-us/story/daman
  • 16.
    First American -Business Problem • Scope of Data:150 Million Properties in USA • Querying • Not a traditional Columnstore Index Scenario • Failed Solution
  • 17.
    First American -Requirements Success Criteria Examples of Slow SQLs SELECT FROM WHERE SELECT FROM WHERE
  • 18.
  • 19.
    20 Btree index Rowstore Table(disk-based) Hybrid queries are directed appropriately Challenge Solution
  • 20.
    First American -Hybrid Query SELECT <Select List> FROM NormandySearch.Search.Property WHERE propertyid in(SELECT propertyid FROM NormandySearch.Search.PropertyCS WHERE [NumericStreetNumber] = 416 and OwnerName like 'BOW%‘ And ( [City] = 'Bay Village' Or [PlaceName] = 'Bay Village')
  • 21.
    First American -Results with Columnstore Index Data Compression CPU Consumption Query Performance with 100 concurrent users With Rowstore (PAGE Compressed) With CCI Database Size (including Indexes) 560 GB 44 GB (13x)
  • 22.
  • 24.
    FIS – BusinessOverview and Challenges Application Overview Current Solution
  • 25.
    FIS – ApplicationChallenges Challenge No Columnstore Index Benefits What to do?
  • 26.
    FIS – Solution Consideredtwo configurations Baseline with all 1.2 million rows compressed Delivered solution with NCCI with force compression
  • 27.
    FIS – PerformanceNumbers Status - Application Live in Production FIS Case Study
  • 29.
    what something isinstead of where it's stored. Organize information based on Proposal ESTT Corporation 9/30/2016 Website Renewal Y: Proposals 2016 Projects ESTT Official docs Website renewal M-FILES – Document Management
  • 30.
    Each document inserted resultsin approximately 60 SQL inserts into Metadata Metadata change in a document can result updating metadata of large number of other documents Document loading and Viewing
  • 33.
  • 34.
    Query Before (sec)After (sec) Contracts by Pricing Type and Agency 6.5 0.5 Contracts by year and month 2.6 0.05
  • 36.

Editor's Notes

  • #31 M-Files efficiently organizes information and documents and makes that content easy to find and share. One of the things that makes M-Files unique is our approach to storing and organizing content. Everything stored in M-Files is organized by what it is instead of where it's stored, as is the case when using a traditional folder-based approach. Only a few details need to be entered when saving documents, such as its type or class (i.e., proposal, invoice, contract, etc.) and what it's related to (i.e., customer, project, contact, etc.). This metadata-driven approach is much more intuitive and precise as compared to guessing the folder where it should be stored.
  • #33 users modify and insert data (OLTP) point lookups users navigate views (OLAP)