1.
MORE THAN JUST SOME BITS…
Data Warehousing and Bitmap Indexes
Dani Schnider, Trivadis AG
Oracle Open World 2011,
San Francisco
BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
1

2.
Dani Schnider
 Principal Consultant und
DWH/BI Lead Architect
bei Trivadis in Zürich
 Teacher for courses about
Data Warehousing, SQL
Optimization and Oracle
Warehouse Builder
 Co-author of book «Data
Warehousing mit Oracle»
2014 © Trivadis
2
Fehlertolerante Ladeprozesse
18. April 2012

3.
Mit über 600 IT- und Fachexperten bei Ihnen vor Ort
2014 © Trivadis
11 Trivadis Niederlassungen mit
über 600 Mitarbeitenden
200 Service Level Agreements
Mehr als 4'000 Trainingsteilnehmer
Forschungs- und Entwicklungs-budget:
CHF 5.0 / EUR 4 Mio.
Finanziell unabhängig und
nachhaltig profitabel
Erfahrung aus mehr als 1'900
Projekten pro Jahr bei über 800
Kunden
Stand 12/2012
Hamburg
Düsseldorf
Frankfurt
Freiburg
München
Wien
Basel
Bern Zürich
Lausanne
3
Stuttgart
Data Warehouseing & Bitmap Indexes
02.10.2011
3

4.
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
4
SOME BITS ABOUT
BITMAP INDEXES
AND THEIR USAGE IN
DATA WAREHOUSES

5.
Bitmap Indexes – Concept
Oracle® Database Concepts 11g Release 2 (11.2)
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
5

6.
Bitmap Indexes – Concept
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
6

7.
Bitmap Indexes – Example
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
7

8.
Comparison B-tree Index / Bitmap Index
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
8

9.
Comparison B-tree Index / Bitmap Index
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
9

10.
Concatenated Bitmap Indexes
Bitmap Indexes on two or more columns are possible, but not useful
2014 © Trivadis
Why?
 Concatenated bitmap index can be used only when all columns are
filtered
 Two or more separate bitmap indexes are more flexible and can be used
for all combinations
Data Warehouseing & Bitmap Indexes
02.10.2011
10

11.
Index Strategy on Star Schema
 Dimension Tables:
 Unique b-tree index on primary key column
 Bitmap indexes on additional columns (optional)
 Fact Tables:
 Bitmap indexes on all dimension key (foreign key) columns
 Bitmap join indexes on often used filter columns (optional)
 Usually no primary key on fact tables
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
11

12.
Index Strategy on Star Schema
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
12

13.
Query Optimization on Star Schemas
 Typical Queries:
 Filter criteria on (multiple) dimension
2014 © Trivadis
tables
 Facts are selected by join with
dimensions
 Problem:
 Tables with restrictions should be
read first
 Only two tables can be joined at a
time
 No relationships between dimension
tables
Data Warehouseing & Bitmap Indexes
02.10.2011
13

14.
Star Transformation – Concept
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
14

15.
Star Transformation – Example
What is the total revenue for product “Shorts” for all customers in Germany,
grouped by male and female customers?
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
15

16.
Star Transformation – Example
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
16

17.
Bitmap Join Index – Concept
 Bitmap index is defined on fact
2014 © Trivadis
table
 Indexed column is part of
dimension table
 Possible for star and snowflake
schemas
Data Warehouseing & Bitmap Indexes
02.10.2011
17

18.
Bitmap Join Index – Example
Same example with Bitmap Join Indexes
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
18
Demo

19.
Bitmap Indexes and ETL
 Bitmap indexes are efficient for set-based ETL
 Index maintenance is deferred until end of each DML operation
 Efficient index maintenance for Parallel DML
 Avoid “hand-made” parallel ETL with multiple sessions
 Locking behavior of bitmap indexes
 For large data loads: Rebuild indexes after ETL operation
 Set all indexes to UNUSABLE
 Load data into table
 Rebuild all indexes
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
19

20.
Bitmap Indexes and Partitioning
 Bitmap Indexes must always be LOCAL
 Local indexes are recommended in Data Warehouse
 Only purpose of global indexes in DWH: unique indexes without partition key
  Restriction of local bitmap indexes is not a problem
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
20

21.
Bitmap Indexes and Compression
 Bitmap Indexes are always compressed
 Zeros are compressed, Ones are not compressed
 Bitmap contains number of zeros to next 1 bit
 Bitmap Indexes need less disk space than B-tree Indexes
 Even for columns with high number of distinct keys
 Detail description see «Bitmap Index Internals», © Julian Dyke
 http://www.juliandyke.com/Presentations/BitmapIndexInternals.ppt
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
21

22.
Bitmap Indexes and Compression
 Number of distinct keys has impact on size of index
 Bitmap index is usually smaller than b-tree index
 For high number of keys, size of bitmap index increases
 Size of b-tree index is more or less constant
 Sort order affects size of bitmap index, but not of b-tree index
2014 © Trivadis
350
300
250
200
150
100
50
Data Warehouseing & Bitmap Indexes
02.10.2011
22
Demo
0
10 100 1000 10000 100000
Bitmap (scattered) Bitmap (sorted) B-tree index

23.
Bitmap Indexes and Compression
 Enabling table compression
 When a table is enabled for compression, all bitmap indexes must be disabled
 Reason: Hakan factor of table
 Maximum number of rows that can be stored in a table block
 Hakan factor changes when table is enabled for compression (only first time)
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
23
Demo

24.
Bitmap Indexes – Conclusion
 Bitmap Indexes are a powerful Data Warehouse feature
 Can be combined in ad hoc queries
 Star Transformation
 Useful for selective and non-selective columns
 Bitmap Indexes are useful for large databases
 Efficient for set-based ETL and Parallel DML
 Local bitmap indexes on partitioned tables
 Compression of bitmap indexes
 There are almost no reasons to use b-tree indexes in a Data Warehouse
 B-tree indexes are mainly used for Primary Key or Unique Key constraints
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
24

25.
References
 Oracle® Database Data Warehousing Guide 11g Release 2 (11.2)
 Oracle Corporation, http://www.oracle.com/pls/db112/
 Oracle® Database Concepts 11g Release 2 (11.2)
 Oracle Corporation, http://www.oracle.com/pls/db112/
 Christian Antognini: Troubleshooting Oracle Performance
 Apress, http://www.apress.com/9781590599174
 Jonathan Lewis: Cost-Based Oracle Fundamentals
 Apress, http://www.apress.com/9781590596364
 Julian Dyke: Bitmap Index Internals
 http://www.juliandyke.com/Presentations/BitmapIndexInternals.ppt
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
25

26.
THANK YOU.
Trivadis AG
Dani Schnider
Europa-Strasse 5
CH-8152 Glattbrugg/Zürich
Switzerland
Tel. +41 44 808 70 20
Fax +41 44 808 70 21
info@trivadis.com
www.trivadis.com
BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
26