The document discusses the features and best practices of columnstore indexes, particularly focusing on their usage in large tables with column-oriented databases. It highlights advantages like reduced I/O through VertiPaq compression, improved parallelism, and batch query execution, while also identifying potential pitfalls like avoiding outer joins and filtering on string columns. Additional sections cover data loading methods, datatype limitations, and references for further information.

1. Who is afraid of Columnstore Index?
Michael Zilberstein
Michael@DBArt.co.il

2. DEMO
2

3. History
Column-oriented databases:
–Sybase IQ
–Vertica ColumnOrders
–Aster Data Doric
–Greenplum
Ionic
–…
Excel PowerPivot. Corinthian
VertiPaq.
xVelocity Columnstore index.
3

4. Storage internals
C1 C2 C3 C4 C5 C6
Uses VertiPaq
compression
4

5. Reduced IO
Fetches only needed columns
from disk
SELECT C2, SUM (C3) …
C2 C3 Columns are compressed
C1 C4 C5 C6
Less IO
Better buffer hit rates
5

6. New query execution technology
Batch mode execution of some operations
– processes rows in batches
– groups of batch operations in query plan
Better parallelism, better algorithms
6

7. Dictionary-based compression
Year of Code
Birth
1996 1
Fact - 60 distinct values 1975 15
Internal Dictionary
1948 50
Year of
1932 58
Birth
… 60
1996 On-the-fly build dictionary with
all distinct value.
1975 Substitute non-selective
1975 values with ID.
Index in our example – 6 bits
1948 Year of per row.
Birth
1932 Code
1 Compressed Fact
…
15
15
50
58
60
7

8. Segments
C1 C2 C3 C4 C5 C6 Column segment
Set of contains values from
about 1M one column for a set
rows of about 1M rows
Column segments
are compressed
Each column
segment stored in
separate LOB
Column segment is
Column unit of transfer from
Segment disk
8

9. Data Dictionary Views
9

10. New execution plan elements
10

11. Best practices / worst practices
Best practices:
–Put columnstore indexes on large tables only.
–Include every column of the table in the
columnstore index.
–Structure your queries as star joins with grouping
and aggregation as much as possible.
Worst practices:
–Avoid JOIN and/or filter on string columns in the
table with columnstore index.
–Avoid OUTER JOIN, UNION ALL, IN/NOT IN.
–Avoid JOIN between 2 Fact tables.
11

12. Datatype Limitations
Decimal > 18 digits
(Var)Binary
BLOB
(N)Varchar(max)
Uniqueidentifier
Date/Time types > 8 bytes (Datetime2 etc)
CLR
12

13. Issues and Workarounds: good, bad and ugly
Outer Joins
NOT IN
Scalar Aggregates
Multiple DISTINCT
UNION ALL
IN and EXISTS
13

14. Data loading and columnstore index
Columnstore index makes table read-only.
DEMO
14

15. Data loading and columnstore index
2 ways to load data:
– Partition switching
– Disable/drop index -> load data ->
create/rebuild index.
15

16. References
http://channel9.msdn.com/Events/TechEd/NorthAmerica/20
11/DBI312
http://social.technet.microsoft.com/wiki/contents/articles/49
95.sql-server-columnstore-performance-tuning.aspx
16

17. Q&A
17

18. Israel
Mob: +972-52-4767219
Tel/Fax: +972-3-9504130
E-Mail: contact@dbart.co.il
Office: 35 Vinik St. Rishon Lezion 75241
The Art of Database Administration