SlideShare a Scribd company logo
1 of 26
Download to read offline
Clustered Columnstore -
Introduction
Niko Neugebauer
Sponsors
Niko Neugebauer
Microsoft Data Platform Professional
OH22 (http://www.oh22.net)
15+ years in IT
SQL Server MVP
Founder of 3 Portuguese PASS Chapters
Blog: http://www.nikoport.com
Twitter: @NikoNeugebauer
LinkedIn: http://pt.linkedin.com/in/webcaravela
Email: info@webcaravela.com
Explore Everything PASS Has to Offer
Free SQL Server and BI Web Events Free 1-day Training Events Regional Event
Local User Groups Around the
World
Free Online Technical Training
This is Community Business Analytics Training
Session Recordings PASS Newsletter
Today’s program:
 Principles of the Columnstore Indexes & Batch Mode
 History
 Columnstore in SQL Server 2012
 Clustered Columnstore in SQL Server 2014
 Batch Mode improvements for SQL Server 2014
Principles of the Columnstore Indexes:
 Using a-kind-of-in-memory™ engine
 X-Velocity aka Vertipaq (PowerPivot, SSAS Tabular)
 Verytically separated, extremely compressed & tuned for
processing large volumes of data
Principles of the Columnstore Indexes
topics:
 Clustered & Nonclustered Indexes
 RowStore vs ColumnStore
 Columnstore Index creation sequence
 Not-suitable for OLTP, unless you are processing whole
table and not altering data
RowStore vs ColumnStore
RowStore vs ColumnStore
Row Store Index:
1, Jody, Philips, 43.04; 2, Mark, Johnson, 37.08; …
ID First
Name
Last Name Salary
1 Jody Philips 43.03
2 Mark Johnson 37.08
3 Matt Markensen 16.81
4 Gail Lindberg 24.90
Column Store Index:
1, 2, 3, 4; Jody, Mark, Matt, Gail; …
RowStore vs ColumnStore
Columnstore Index creation sequence:
1. Run-length compression
2. Dictionary enconding
3. Binary compression
Run-length compression
ID
1
1
2
3
3
3
4
1
5
1
5
ID
1:2
2
3:3
4
1-5:2
Dictionary enconding
ID
Mark
Andre
John
Mark
John
Andre
John
Mark
ID
Mark 1
Andre 2
John 3
ID
1
2
3
1
3
2
3
1
Binary Compression
Super-secret compression turning data into LOB’s. 
Vertipac aka xVelocity
Row Groups
~ 1.000.000 Rows are grouped in Segments,
called Row Groups
Data is Encoded & Compressed
Data is Unsorted
Stored as LOB’s
When executed in batch mode the Segments are eliminated
Principles of the Batch mode:
 Vector processing
 1 row vs ~1000 rows
 Progammers, its like passing Array vs 1 param
 Think about it as if it would be a Factory processing vs
Manual Processing (19th vs 18th Century)
History of the Columnstore
 Concept exists since 1969 (or since the beginning of Database era)
 Sybase IQ (since 1990s)
 PDW v1 had the first implementation for Microsoft Databases
 SQL Server 2012 introduced Columnstore
 Besides SQL Server there are Vertica, C-Store, Volt, Terradata, …
 Michael Stonebraker’s vision – In 10 years there will be virtually no
Rowstore for Datawarehousing
Columnstore in SQL Server 2012
 Only Nonclustered (but supporting multiple indedexs)
 Non-Updateable (and no schema-modification)
 Very limited data types (Precision 18 and no LOB’s)
 Batch mode functions only in certain conditions (no
OUTER JOIN, UNION ALL, EXIST, IN, Scalar
Aggregates, Distinct Aggregates)
Columnstore in SQL Server 2014
 Clustered & Nonclustered
 Clustered Columnstore are updateable &
Schema can be modified
 Large range of data types (except LOB)
 Better Memory estimation & consumption
 Batch mode is significantly improved
 NEW! Archival Compression
Columnstore in SQL Server 2014
 Delta-Stores (open & close)
 Deleted Bitmap
 Delete & Update work as a
DELETE + INSERT
 Schema modification
 Constraints are supported
(Non-Unique)
Important Limitations in SQL Server 2014
 No NonClustered Indexes for Clustered
Columnstore
 No ONLINE operations
 Not every operator in Execution plans is
supporting Batch mode.
Batch Mode Improvements in SQL Server
2014
 Mixed Mode – Row & Batch mode can co-exist
 OUTER JOIN, UNION ALL, EXIST, IN, Scalar
Aggregates, Distinct Aggregates
 Some TempDB operations for Columnstore Indexes are
running in Batch mode.
Demo Time
Links
• http://www.nikoport.com/2013/07/05/clustered-
columnstore-indexes-part-1-intro/ – My series of blog
posts on Clustered Columnstore Indexes
• http://rusanu.com/2013/06/11/sql-server-clustered-
columnstore-indexes-at-teched-2013/ – Remus Rusanu
Introduction for Clustered Columnstore
• http://research.microsoft.com/pubs/193599/Apollo3%20
-%20Sigmod%202013%20-%20final.pdf – White Paper
on the Clustered Columnstore
Sponsors

More Related Content

What's hot

How QBerg scaled to store data longer, query it faster
How QBerg scaled to store data longer, query it fasterHow QBerg scaled to store data longer, query it faster
How QBerg scaled to store data longer, query it fasterMariaDB plc
 
Scylla Summit 2018: Scylla 3.0 and Beyond
Scylla Summit 2018: Scylla 3.0 and BeyondScylla Summit 2018: Scylla 3.0 and Beyond
Scylla Summit 2018: Scylla 3.0 and BeyondScyllaDB
 
Jss 2015 in memory and operational analytics
Jss 2015   in memory and operational analyticsJss 2015   in memory and operational analytics
Jss 2015 in memory and operational analyticsDavid Barbarin
 
Oracle GoldenGate for MySQL Overview
Oracle GoldenGate for MySQL OverviewOracle GoldenGate for MySQL Overview
Oracle GoldenGate for MySQL OverviewJinyu Wang
 
SQL Explore 2012: P&T Part 2
SQL Explore 2012: P&T Part 2SQL Explore 2012: P&T Part 2
SQL Explore 2012: P&T Part 2sqlserver.co.il
 
Scylla Summit 2022: How ScyllaDB Powers This Next Tech Cycle
Scylla Summit 2022: How ScyllaDB Powers This Next Tech CycleScylla Summit 2022: How ScyllaDB Powers This Next Tech Cycle
Scylla Summit 2022: How ScyllaDB Powers This Next Tech CycleScyllaDB
 
What is Change Data Capture (CDC) and Why is it Important?
What is Change Data Capture (CDC) and Why is it Important?What is Change Data Capture (CDC) and Why is it Important?
What is Change Data Capture (CDC) and Why is it Important?FlyData Inc.
 
Products.intro.forum version
Products.intro.forum versionProducts.intro.forum version
Products.intro.forum versionsqlserver.co.il
 
Introduction of MariaDB 2017 09
Introduction of MariaDB 2017 09Introduction of MariaDB 2017 09
Introduction of MariaDB 2017 09GOTO Satoru
 
MariaDB: Connect Storage Engine
MariaDB: Connect Storage EngineMariaDB: Connect Storage Engine
MariaDB: Connect Storage EngineKangaroot
 
An Introduction to Amazon’s DynamoDB
An Introduction to Amazon’s DynamoDBAn Introduction to Amazon’s DynamoDB
An Introduction to Amazon’s DynamoDBKnoldus Inc.
 
How we switched to columnar at SpendHQ
How we switched to columnar at SpendHQHow we switched to columnar at SpendHQ
How we switched to columnar at SpendHQMariaDB plc
 
Introduction of MariaDB AX / TX
Introduction of MariaDB AX / TXIntroduction of MariaDB AX / TX
Introduction of MariaDB AX / TXGOTO Satoru
 
In-memory ColumnStore Index
In-memory ColumnStore IndexIn-memory ColumnStore Index
In-memory ColumnStore IndexSolidQ
 
Scylla Summit 2018: Getting the Most Out of Scylla on Kubernetes
Scylla Summit 2018: Getting the Most Out of Scylla on KubernetesScylla Summit 2018: Getting the Most Out of Scylla on Kubernetes
Scylla Summit 2018: Getting the Most Out of Scylla on KubernetesScyllaDB
 
MariaDB TX 3.0 新機能 / ロードマップ
MariaDB TX 3.0 新機能 / ロードマップMariaDB TX 3.0 新機能 / ロードマップ
MariaDB TX 3.0 新機能 / ロードマップGOTO Satoru
 
Getting started in the cloud for developers
Getting started in the cloud for developersGetting started in the cloud for developers
Getting started in the cloud for developersMariaDB plc
 
Running Scylla on Kubernetes with Scylla Operator
Running Scylla on Kubernetes with Scylla OperatorRunning Scylla on Kubernetes with Scylla Operator
Running Scylla on Kubernetes with Scylla OperatorScyllaDB
 
How MariaDB is approaching DBaaS
How MariaDB is approaching DBaaSHow MariaDB is approaching DBaaS
How MariaDB is approaching DBaaSMariaDB plc
 

What's hot (20)

How QBerg scaled to store data longer, query it faster
How QBerg scaled to store data longer, query it fasterHow QBerg scaled to store data longer, query it faster
How QBerg scaled to store data longer, query it faster
 
Scylla Summit 2018: Scylla 3.0 and Beyond
Scylla Summit 2018: Scylla 3.0 and BeyondScylla Summit 2018: Scylla 3.0 and Beyond
Scylla Summit 2018: Scylla 3.0 and Beyond
 
Jss 2015 in memory and operational analytics
Jss 2015   in memory and operational analyticsJss 2015   in memory and operational analytics
Jss 2015 in memory and operational analytics
 
Oracle GoldenGate for MySQL Overview
Oracle GoldenGate for MySQL OverviewOracle GoldenGate for MySQL Overview
Oracle GoldenGate for MySQL Overview
 
SQL Explore 2012: P&T Part 2
SQL Explore 2012: P&T Part 2SQL Explore 2012: P&T Part 2
SQL Explore 2012: P&T Part 2
 
Scylla Summit 2022: How ScyllaDB Powers This Next Tech Cycle
Scylla Summit 2022: How ScyllaDB Powers This Next Tech CycleScylla Summit 2022: How ScyllaDB Powers This Next Tech Cycle
Scylla Summit 2022: How ScyllaDB Powers This Next Tech Cycle
 
What is Change Data Capture (CDC) and Why is it Important?
What is Change Data Capture (CDC) and Why is it Important?What is Change Data Capture (CDC) and Why is it Important?
What is Change Data Capture (CDC) and Why is it Important?
 
Products.intro.forum version
Products.intro.forum versionProducts.intro.forum version
Products.intro.forum version
 
Introduction of MariaDB 2017 09
Introduction of MariaDB 2017 09Introduction of MariaDB 2017 09
Introduction of MariaDB 2017 09
 
MariaDB: Connect Storage Engine
MariaDB: Connect Storage EngineMariaDB: Connect Storage Engine
MariaDB: Connect Storage Engine
 
An Introduction to Amazon’s DynamoDB
An Introduction to Amazon’s DynamoDBAn Introduction to Amazon’s DynamoDB
An Introduction to Amazon’s DynamoDB
 
MongoDB and DynamoDB
MongoDB and DynamoDBMongoDB and DynamoDB
MongoDB and DynamoDB
 
How we switched to columnar at SpendHQ
How we switched to columnar at SpendHQHow we switched to columnar at SpendHQ
How we switched to columnar at SpendHQ
 
Introduction of MariaDB AX / TX
Introduction of MariaDB AX / TXIntroduction of MariaDB AX / TX
Introduction of MariaDB AX / TX
 
In-memory ColumnStore Index
In-memory ColumnStore IndexIn-memory ColumnStore Index
In-memory ColumnStore Index
 
Scylla Summit 2018: Getting the Most Out of Scylla on Kubernetes
Scylla Summit 2018: Getting the Most Out of Scylla on KubernetesScylla Summit 2018: Getting the Most Out of Scylla on Kubernetes
Scylla Summit 2018: Getting the Most Out of Scylla on Kubernetes
 
MariaDB TX 3.0 新機能 / ロードマップ
MariaDB TX 3.0 新機能 / ロードマップMariaDB TX 3.0 新機能 / ロードマップ
MariaDB TX 3.0 新機能 / ロードマップ
 
Getting started in the cloud for developers
Getting started in the cloud for developersGetting started in the cloud for developers
Getting started in the cloud for developers
 
Running Scylla on Kubernetes with Scylla Operator
Running Scylla on Kubernetes with Scylla OperatorRunning Scylla on Kubernetes with Scylla Operator
Running Scylla on Kubernetes with Scylla Operator
 
How MariaDB is approaching DBaaS
How MariaDB is approaching DBaaSHow MariaDB is approaching DBaaS
How MariaDB is approaching DBaaS
 

Viewers also liked

Sql server 2014 x velocity – updateable columnstore indexes
Sql server 2014 x velocity – updateable columnstore indexesSql server 2014 x velocity – updateable columnstore indexes
Sql server 2014 x velocity – updateable columnstore indexesPat Sheehan
 
Sql server data store data access internals
Sql server data store data access internalsSql server data store data access internals
Sql server data store data access internalsMasayuki Ozawa
 
SQL Server 2014 データベースエンジン新機能
SQL Server 2014 データベースエンジン新機能SQL Server 2014 データベースエンジン新機能
SQL Server 2014 データベースエンジン新機能Masayuki Ozawa
 
Sql server 2012 の新機能を使ってみよう。db 管理者向け機能の紹介
Sql server 2012 の新機能を使ってみよう。db 管理者向け機能の紹介Sql server 2012 の新機能を使ってみよう。db 管理者向け機能の紹介
Sql server 2012 の新機能を使ってみよう。db 管理者向け機能の紹介Masayuki Ozawa
 
Columnstore indexes in sql server 2014
Columnstore indexes in sql server 2014Columnstore indexes in sql server 2014
Columnstore indexes in sql server 2014Antonios Chatzipavlis
 
[D35] インメモリーデータベース徹底比較 by Komori
[D35] インメモリーデータベース徹底比較 by Komori[D35] インメモリーデータベース徹底比較 by Komori
[D35] インメモリーデータベース徹底比較 by KomoriInsight Technology, Inc.
 
Sql server 2016 ctp 3.0 新機能
Sql server 2016 ctp 3.0 新機能Sql server 2016 ctp 3.0 新機能
Sql server 2016 ctp 3.0 新機能Masayuki Ozawa
 
SQL server 2016 New Features
SQL server 2016 New FeaturesSQL server 2016 New Features
SQL server 2016 New Featuresaminmesbahi
 

Viewers also liked (9)

Sql server 2014 x velocity – updateable columnstore indexes
Sql server 2014 x velocity – updateable columnstore indexesSql server 2014 x velocity – updateable columnstore indexes
Sql server 2014 x velocity – updateable columnstore indexes
 
Sql server data store data access internals
Sql server data store data access internalsSql server data store data access internals
Sql server data store data access internals
 
SQL Server 2014 データベースエンジン新機能
SQL Server 2014 データベースエンジン新機能SQL Server 2014 データベースエンジン新機能
SQL Server 2014 データベースエンジン新機能
 
DBTS2015_B35_SQLServer2016
DBTS2015_B35_SQLServer2016DBTS2015_B35_SQLServer2016
DBTS2015_B35_SQLServer2016
 
Sql server 2012 の新機能を使ってみよう。db 管理者向け機能の紹介
Sql server 2012 の新機能を使ってみよう。db 管理者向け機能の紹介Sql server 2012 の新機能を使ってみよう。db 管理者向け機能の紹介
Sql server 2012 の新機能を使ってみよう。db 管理者向け機能の紹介
 
Columnstore indexes in sql server 2014
Columnstore indexes in sql server 2014Columnstore indexes in sql server 2014
Columnstore indexes in sql server 2014
 
[D35] インメモリーデータベース徹底比較 by Komori
[D35] インメモリーデータベース徹底比較 by Komori[D35] インメモリーデータベース徹底比較 by Komori
[D35] インメモリーデータベース徹底比較 by Komori
 
Sql server 2016 ctp 3.0 新機能
Sql server 2016 ctp 3.0 新機能Sql server 2016 ctp 3.0 新機能
Sql server 2016 ctp 3.0 新機能
 
SQL server 2016 New Features
SQL server 2016 New FeaturesSQL server 2016 New Features
SQL server 2016 New Features
 

Similar to Clustered Columnstore Introduction

SQLSaturday#290_Kiev_AdHocMaintenancePlansForBeginners
SQLSaturday#290_Kiev_AdHocMaintenancePlansForBeginnersSQLSaturday#290_Kiev_AdHocMaintenancePlansForBeginners
SQLSaturday#290_Kiev_AdHocMaintenancePlansForBeginnersTobias Koprowski
 
Upgrade to MySQL 8.0!
Upgrade to MySQL 8.0!Upgrade to MySQL 8.0!
Upgrade to MySQL 8.0!Ted Wennmark
 
01 upgrade to my sql8
01 upgrade to my sql8 01 upgrade to my sql8
01 upgrade to my sql8 Ted Wennmark
 
NoSQL and MySQL: News about JSON
NoSQL and MySQL: News about JSONNoSQL and MySQL: News about JSON
NoSQL and MySQL: News about JSONMario Beck
 
Balamurugan msbi cv
Balamurugan msbi cvBalamurugan msbi cv
Balamurugan msbi cvbala murugan
 
Introduction to Columnstore Indexes
Introduction to Columnstore IndexesIntroduction to Columnstore Indexes
Introduction to Columnstore IndexesJason Strate
 
KoprowskiT_SQLRelay2014#2_Southampton_MaintenancePlansForBeginners
KoprowskiT_SQLRelay2014#2_Southampton_MaintenancePlansForBeginnersKoprowskiT_SQLRelay2014#2_Southampton_MaintenancePlansForBeginners
KoprowskiT_SQLRelay2014#2_Southampton_MaintenancePlansForBeginnersTobias Koprowski
 
Microsoft SQL server 2016 Partner Mini Case Study - Dimodelo Solutions
Microsoft SQL server 2016 Partner Mini Case Study - Dimodelo SolutionsMicrosoft SQL server 2016 Partner Mini Case Study - Dimodelo Solutions
Microsoft SQL server 2016 Partner Mini Case Study - Dimodelo Solutionsmikerabinovici
 
13 - Panorama Necto 14 building models - visualization & data discovery solu...
13  - Panorama Necto 14 building models - visualization & data discovery solu...13  - Panorama Necto 14 building models - visualization & data discovery solu...
13 - Panorama Necto 14 building models - visualization & data discovery solu...Panorama Software
 
SQL Server Workshop for Developers - Visual Studio Live! NY 2012
SQL Server Workshop for Developers - Visual Studio Live! NY 2012SQL Server Workshop for Developers - Visual Studio Live! NY 2012
SQL Server Workshop for Developers - Visual Studio Live! NY 2012Andrew Brust
 
Performance dreams of sql server 2014
Performance dreams of sql server 2014Performance dreams of sql server 2014
Performance dreams of sql server 2014Shehap Elnagar
 
My 2019 SQL Saturday In San Diego presentation
My 2019 SQL Saturday In San Diego presentationMy 2019 SQL Saturday In San Diego presentation
My 2019 SQL Saturday In San Diego presentationSteve Rezhener, MBA
 
Restlet: Building a multi-tenant API PaaS with DataStax Enterprise Search
Restlet: Building a multi-tenant API PaaS with DataStax Enterprise SearchRestlet: Building a multi-tenant API PaaS with DataStax Enterprise Search
Restlet: Building a multi-tenant API PaaS with DataStax Enterprise SearchDataStax Academy
 
MSBI with SQL Server T-SQL | Microsoft BI With Server T-SQL | Realtime Traini...
MSBI with SQL Server T-SQL | Microsoft BI With Server T-SQL | Realtime Traini...MSBI with SQL Server T-SQL | Microsoft BI With Server T-SQL | Realtime Traini...
MSBI with SQL Server T-SQL | Microsoft BI With Server T-SQL | Realtime Traini...SequelGate
 
Apoyo en la administración de bases de datos mediante microsoft data tools
Apoyo en la administración de bases de datos mediante microsoft data toolsApoyo en la administración de bases de datos mediante microsoft data tools
Apoyo en la administración de bases de datos mediante microsoft data toolsSpanishPASSVC
 
Create column store index on all supported tables in sql server 2014 copy
Create column store index on all supported tables in sql server 2014    copyCreate column store index on all supported tables in sql server 2014    copy
Create column store index on all supported tables in sql server 2014 copyMustafa EL-Masry
 
SSAS, MDX , Cube understanding, Browsing and Tools information
SSAS, MDX , Cube understanding, Browsing and Tools information SSAS, MDX , Cube understanding, Browsing and Tools information
SSAS, MDX , Cube understanding, Browsing and Tools information Vishal Pawar
 
Oracle OpenWorld 2016 Review - Focus on Data, BigData, Streaming Data, Machin...
Oracle OpenWorld 2016 Review - Focus on Data, BigData, Streaming Data, Machin...Oracle OpenWorld 2016 Review - Focus on Data, BigData, Streaming Data, Machin...
Oracle OpenWorld 2016 Review - Focus on Data, BigData, Streaming Data, Machin...Lucas Jellema
 

Similar to Clustered Columnstore Introduction (20)

SQLSaturday#290_Kiev_AdHocMaintenancePlansForBeginners
SQLSaturday#290_Kiev_AdHocMaintenancePlansForBeginnersSQLSaturday#290_Kiev_AdHocMaintenancePlansForBeginners
SQLSaturday#290_Kiev_AdHocMaintenancePlansForBeginners
 
Upgrade to MySQL 8.0!
Upgrade to MySQL 8.0!Upgrade to MySQL 8.0!
Upgrade to MySQL 8.0!
 
01 upgrade to my sql8
01 upgrade to my sql8 01 upgrade to my sql8
01 upgrade to my sql8
 
NoSQL and MySQL: News about JSON
NoSQL and MySQL: News about JSONNoSQL and MySQL: News about JSON
NoSQL and MySQL: News about JSON
 
Balamurugan msbi cv
Balamurugan msbi cvBalamurugan msbi cv
Balamurugan msbi cv
 
Introduction to Columnstore Indexes
Introduction to Columnstore IndexesIntroduction to Columnstore Indexes
Introduction to Columnstore Indexes
 
KoprowskiT_SQLRelay2014#2_Southampton_MaintenancePlansForBeginners
KoprowskiT_SQLRelay2014#2_Southampton_MaintenancePlansForBeginnersKoprowskiT_SQLRelay2014#2_Southampton_MaintenancePlansForBeginners
KoprowskiT_SQLRelay2014#2_Southampton_MaintenancePlansForBeginners
 
Microsoft SQL server 2016 Partner Mini Case Study - Dimodelo Solutions
Microsoft SQL server 2016 Partner Mini Case Study - Dimodelo SolutionsMicrosoft SQL server 2016 Partner Mini Case Study - Dimodelo Solutions
Microsoft SQL server 2016 Partner Mini Case Study - Dimodelo Solutions
 
13 - Panorama Necto 14 building models - visualization & data discovery solu...
13  - Panorama Necto 14 building models - visualization & data discovery solu...13  - Panorama Necto 14 building models - visualization & data discovery solu...
13 - Panorama Necto 14 building models - visualization & data discovery solu...
 
SQL Server Workshop for Developers - Visual Studio Live! NY 2012
SQL Server Workshop for Developers - Visual Studio Live! NY 2012SQL Server Workshop for Developers - Visual Studio Live! NY 2012
SQL Server Workshop for Developers - Visual Studio Live! NY 2012
 
Performance dreams of sql server 2014
Performance dreams of sql server 2014Performance dreams of sql server 2014
Performance dreams of sql server 2014
 
My 2019 SQL Saturday In San Diego presentation
My 2019 SQL Saturday In San Diego presentationMy 2019 SQL Saturday In San Diego presentation
My 2019 SQL Saturday In San Diego presentation
 
Restlet: Building a multi-tenant API PaaS with DataStax Enterprise Search
Restlet: Building a multi-tenant API PaaS with DataStax Enterprise SearchRestlet: Building a multi-tenant API PaaS with DataStax Enterprise Search
Restlet: Building a multi-tenant API PaaS with DataStax Enterprise Search
 
No SQL and MongoDB - Hyderabad Scalability Meetup
No SQL and MongoDB - Hyderabad Scalability MeetupNo SQL and MongoDB - Hyderabad Scalability Meetup
No SQL and MongoDB - Hyderabad Scalability Meetup
 
MSBI with SQL Server T-SQL | Microsoft BI With Server T-SQL | Realtime Traini...
MSBI with SQL Server T-SQL | Microsoft BI With Server T-SQL | Realtime Traini...MSBI with SQL Server T-SQL | Microsoft BI With Server T-SQL | Realtime Traini...
MSBI with SQL Server T-SQL | Microsoft BI With Server T-SQL | Realtime Traini...
 
Apoyo en la administración de bases de datos mediante microsoft data tools
Apoyo en la administración de bases de datos mediante microsoft data toolsApoyo en la administración de bases de datos mediante microsoft data tools
Apoyo en la administración de bases de datos mediante microsoft data tools
 
Create column store index on all supported tables in sql server 2014 copy
Create column store index on all supported tables in sql server 2014    copyCreate column store index on all supported tables in sql server 2014    copy
Create column store index on all supported tables in sql server 2014 copy
 
SSAS, MDX , Cube understanding, Browsing and Tools information
SSAS, MDX , Cube understanding, Browsing and Tools information SSAS, MDX , Cube understanding, Browsing and Tools information
SSAS, MDX , Cube understanding, Browsing and Tools information
 
Oracle OpenWorld 2016 Review - Focus on Data, BigData, Streaming Data, Machin...
Oracle OpenWorld 2016 Review - Focus on Data, BigData, Streaming Data, Machin...Oracle OpenWorld 2016 Review - Focus on Data, BigData, Streaming Data, Machin...
Oracle OpenWorld 2016 Review - Focus on Data, BigData, Streaming Data, Machin...
 
Oow2016 review-db-dev-bigdata-BI
Oow2016 review-db-dev-bigdata-BIOow2016 review-db-dev-bigdata-BI
Oow2016 review-db-dev-bigdata-BI
 

Recently uploaded

Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...ScyllaDB
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingScyllaDB
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentationyogeshlabana357357
 
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...SOFTTECHHUB
 
Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptx
Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptxCyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptx
Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptxMasterG
 
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdfMuhammad Subhan
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...ScyllaDB
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAnitaRaj43
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Paige Cruz
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxFIDO Alliance
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024Lorenzo Miniero
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)Wonjun Hwang
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxFIDO Alliance
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontologyjohnbeverley2021
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe中 央社
 
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?Paolo Missier
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc
 
الأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهالأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهMohamed Sweelam
 

Recently uploaded (20)

Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentation
 
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
 
Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptx
Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptxCyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptx
Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptx
 
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptx
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
 
الأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهالأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهله
 

Clustered Columnstore Introduction

  • 3. Niko Neugebauer Microsoft Data Platform Professional OH22 (http://www.oh22.net) 15+ years in IT SQL Server MVP Founder of 3 Portuguese PASS Chapters Blog: http://www.nikoport.com Twitter: @NikoNeugebauer LinkedIn: http://pt.linkedin.com/in/webcaravela Email: info@webcaravela.com
  • 4. Explore Everything PASS Has to Offer Free SQL Server and BI Web Events Free 1-day Training Events Regional Event Local User Groups Around the World Free Online Technical Training This is Community Business Analytics Training Session Recordings PASS Newsletter
  • 5. Today’s program:  Principles of the Columnstore Indexes & Batch Mode  History  Columnstore in SQL Server 2012  Clustered Columnstore in SQL Server 2014  Batch Mode improvements for SQL Server 2014
  • 6. Principles of the Columnstore Indexes:  Using a-kind-of-in-memory™ engine  X-Velocity aka Vertipaq (PowerPivot, SSAS Tabular)  Verytically separated, extremely compressed & tuned for processing large volumes of data
  • 7. Principles of the Columnstore Indexes topics:  Clustered & Nonclustered Indexes  RowStore vs ColumnStore  Columnstore Index creation sequence  Not-suitable for OLTP, unless you are processing whole table and not altering data
  • 9. RowStore vs ColumnStore Row Store Index: 1, Jody, Philips, 43.04; 2, Mark, Johnson, 37.08; … ID First Name Last Name Salary 1 Jody Philips 43.03 2 Mark Johnson 37.08 3 Matt Markensen 16.81 4 Gail Lindberg 24.90 Column Store Index: 1, 2, 3, 4; Jody, Mark, Matt, Gail; …
  • 11. Columnstore Index creation sequence: 1. Run-length compression 2. Dictionary enconding 3. Binary compression
  • 14. Binary Compression Super-secret compression turning data into LOB’s.  Vertipac aka xVelocity
  • 15. Row Groups ~ 1.000.000 Rows are grouped in Segments, called Row Groups Data is Encoded & Compressed Data is Unsorted Stored as LOB’s When executed in batch mode the Segments are eliminated
  • 16. Principles of the Batch mode:  Vector processing  1 row vs ~1000 rows  Progammers, its like passing Array vs 1 param  Think about it as if it would be a Factory processing vs Manual Processing (19th vs 18th Century)
  • 17. History of the Columnstore  Concept exists since 1969 (or since the beginning of Database era)  Sybase IQ (since 1990s)  PDW v1 had the first implementation for Microsoft Databases  SQL Server 2012 introduced Columnstore  Besides SQL Server there are Vertica, C-Store, Volt, Terradata, …  Michael Stonebraker’s vision – In 10 years there will be virtually no Rowstore for Datawarehousing
  • 18. Columnstore in SQL Server 2012  Only Nonclustered (but supporting multiple indedexs)  Non-Updateable (and no schema-modification)  Very limited data types (Precision 18 and no LOB’s)  Batch mode functions only in certain conditions (no OUTER JOIN, UNION ALL, EXIST, IN, Scalar Aggregates, Distinct Aggregates)
  • 19. Columnstore in SQL Server 2014  Clustered & Nonclustered  Clustered Columnstore are updateable & Schema can be modified  Large range of data types (except LOB)  Better Memory estimation & consumption  Batch mode is significantly improved  NEW! Archival Compression
  • 20. Columnstore in SQL Server 2014  Delta-Stores (open & close)  Deleted Bitmap  Delete & Update work as a DELETE + INSERT  Schema modification  Constraints are supported (Non-Unique)
  • 21. Important Limitations in SQL Server 2014  No NonClustered Indexes for Clustered Columnstore  No ONLINE operations  Not every operator in Execution plans is supporting Batch mode.
  • 22. Batch Mode Improvements in SQL Server 2014  Mixed Mode – Row & Batch mode can co-exist  OUTER JOIN, UNION ALL, EXIST, IN, Scalar Aggregates, Distinct Aggregates  Some TempDB operations for Columnstore Indexes are running in Batch mode.
  • 24. Links • http://www.nikoport.com/2013/07/05/clustered- columnstore-indexes-part-1-intro/ – My series of blog posts on Clustered Columnstore Indexes • http://rusanu.com/2013/06/11/sql-server-clustered- columnstore-indexes-at-teched-2013/ – Remus Rusanu Introduction for Clustered Columnstore • http://research.microsoft.com/pubs/193599/Apollo3%20 -%20Sigmod%202013%20-%20final.pdf – White Paper on the Clustered Columnstore
  • 25.