Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
In-Memory Columnstore Indexes-Make Your Data Warehouse Fly
Joe D’Antoni
Philadelphia SQL Server Business Intelligence
Grou...
About Me
Solution Architect, Anexinet
@jdanton – Twitter
jdanton1@yahoo.com
Joedantoni.wordpress.com – Blog, Slides
Agenda
Indexes—a basic overview
Columnstore—an introduction
Report Performance—Demo
2012 and 2014—What’s Changing?

2014—D...
Indexes
• Data Structure that allows us to
speed data retrieval, by
maintaining an extra copy of
data
• Can be filtered
• ...
Indexes in SQL Server
• Clustered vs Nonclustered
• Non-clustered index ―just
an index‖
Clustered Index
• Data is ordered as is inserted into
pages
• Data in clustered index is only
stored on disk once (it’s th...
Non-Clustered Index
• Duplicate copy of the data in table
• Provides point from index to table
data
• No specific order of...
So Why All This Talk About Indexes?
Data Warehouse Queries
• Data Warehouses have a lot of
data
• Querying lots of a data can
take a really long time
• Proces...
Traditional Approaches To Improving
Performance
• Partitioned Tables
• Indexed Views
• Data Compression
Introducing Columnstore Indexes (SQL
2012)
• Data is stored in columns, as
opposed to rows
• This allows a much higher rat...
Columnar Data
Storage
Columnstore 2012
Demo
So How is So Much Faster?
• Very good compression ratio for Column oriented
data
• Better use of Memory
• Segment Eliminat...
Columnstore All The Things?
• Awesome performance—so what’s
the negative?
• Can’t update/insert in 2012
• Can only be nonc...
So Where To Use Columnstore
Indexes?
• Only on Large Tables—Fact
tables and Dimension Tables >
3 Million Rows
• Include Ev...
Columnstore 2014
Columnstore in 2014
• Fewer Data Type Limitations
• Updateable
• Can be Clustered Index

• New Archival Compression Mode
•...
Columnstore Updates (2014)

Updates To
Index

Collected
until they
reach 1000
rows

Tuple Movers
Move into
Index
Columnstore Updates (2014)
• Bulk Inserts go through
special API
• Updates are processed
as inserts and deletes,
so expens...
Columnstore 2014
Demo
What Do We Do Differently in 2014
• Best Practices are mostly the
same
• Batch mode gets enhanced and
gains more query typ...
Questions
Contact
jdanton1@yahoo.com
Joedantoni.wordpress.com
@jdanton
In memory columnstore indexes--make your data warehouse
In memory columnstore indexes--make your data warehouse
In memory columnstore indexes--make your data warehouse
In memory columnstore indexes--make your data warehouse
In memory columnstore indexes--make your data warehouse
In memory columnstore indexes--make your data warehouse
Upcoming SlideShare
Loading in …5
×

In memory columnstore indexes--make your data warehouse

1,929 views

Published on

Presentation on SQL Server 2012 and 2014 Columnstore Indexing feature presented to Philadelphia SQL BI Usergroup on November 19, 2013

Published in: Technology
  • Well done, also...GROW YOU DOWNLINE OVERNIGHT - Works with any mlm. Have dozens joining whatever mlm your doing today! Go to: www.mlmrc.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

In memory columnstore indexes--make your data warehouse

  1. 1. In-Memory Columnstore Indexes-Make Your Data Warehouse Fly Joe D’Antoni Philadelphia SQL Server Business Intelligence Group 19 November 2013
  2. 2. About Me Solution Architect, Anexinet @jdanton – Twitter jdanton1@yahoo.com Joedantoni.wordpress.com – Blog, Slides
  3. 3. Agenda Indexes—a basic overview Columnstore—an introduction Report Performance—Demo 2012 and 2014—What’s Changing? 2014—Demo Questions
  4. 4. Indexes • Data Structure that allows us to speed data retrieval, by maintaining an extra copy of data • Can be filtered • Can be function based, or ordered • Penalty is that writes become more expensive • More storage required
  5. 5. Indexes in SQL Server • Clustered vs Nonclustered • Non-clustered index ―just an index‖
  6. 6. Clustered Index • Data is ordered as is inserted into pages • Data in clustered index is only stored on disk once (it’s the data from the tables) • Table without a clustered index is called a heap—no order at all
  7. 7. Non-Clustered Index • Duplicate copy of the data in table • Provides point from index to table data • No specific order of data in index
  8. 8. So Why All This Talk About Indexes?
  9. 9. Data Warehouse Queries • Data Warehouses have a lot of data • Querying lots of a data can take a really long time • Processing data row by row— may not be the most efficient way to perform aggregations
  10. 10. Traditional Approaches To Improving Performance • Partitioned Tables • Indexed Views • Data Compression
  11. 11. Introducing Columnstore Indexes (SQL 2012) • Data is stored in columns, as opposed to rows • This allows a much higher rate of compression • Columns not used in a query a simply not scanned, nor returned • Recommended practice is to add most columns in a table to a index
  12. 12. Columnar Data Storage
  13. 13. Columnstore 2012 Demo
  14. 14. So How is So Much Faster? • Very good compression ratio for Column oriented data • Better use of Memory • Segment Elimination Skips Large Chunks of Data • Batch Mode • Processes data in chunks of a 1000 row ―batches‖ rather than row by row • 7-40x CPU savings with batch mode “The key to getting the best performance is to make sure your queries process the large majority of data in
  15. 15. Columnstore All The Things? • Awesome performance—so what’s the negative? • Can’t update/insert in 2012 • Can only be nonclustered index— so we are storing more data on disk • Data types are somewhat limited • One index per table • Can’t be a sorted index
  16. 16. So Where To Use Columnstore Indexes? • Only on Large Tables—Fact tables and Dimension Tables > 3 Million Rows • Include Every Column • Structure Queries as star joins with grouping and aggregation More details here
  17. 17. Columnstore 2014
  18. 18. Columnstore in 2014 • Fewer Data Type Limitations • Updateable • Can be Clustered Index • New Archival Compression Mode • Batch Mode Improvements
  19. 19. Columnstore Updates (2014) Updates To Index Collected until they reach 1000 rows Tuple Movers Move into Index
  20. 20. Columnstore Updates (2014) • Bulk Inserts go through special API • Updates are processed as inserts and deletes, so expensive opertation
  21. 21. Columnstore 2014 Demo
  22. 22. What Do We Do Differently in 2014 • Best Practices are mostly the same • Batch mode gets enhanced and gains more query types • No need to worry about dropping and rebuilding indexes—just append data • Still focus on large tables where data is not frequently updated • Archival Compression Good for old unused data
  23. 23. Questions
  24. 24. Contact jdanton1@yahoo.com Joedantoni.wordpress.com @jdanton

×