SlideShare a Scribd company logo
1 of 35
Partitioning Your Oracle Data Warehouse 
– Just a Simple Task? 
Dani Schnider 
Principal Consultant 
Business Intelligence 
dani.schnider@trivadis.com 
Oracle Open World 2009, 
San Francisco 
BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
1
Dani Schnider 
 Principal Consultant und 
DWH/BI Lead Architect 
bei Trivadis in Zürich 
 Kursleiter für Trivadis-Kurse 
über Data Warehousing, 
SQL Optimierung und Oracle 
Warehouse Builder 
 Co-Autor des Buches «Data 
Warehousing mit Oracle» 
2014 © Trivadis 
2 
Partitioning Your Oracle Data Warehouse 
2009
Mit über 600 IT- und Fachexperten bei Ihnen vor Ort 
2014 © Trivadis 
11 Trivadis Niederlassungen mit 
über 600 Mitarbeitenden 
200 Service Level Agreements 
Mehr als 4'000 Trainingsteilnehmer 
Forschungs- und Entwicklungs-budget: 
CHF 5.0 / EUR 4 Mio. 
Finanziell unabhängig und 
nachhaltig profitabel 
Erfahrung aus mehr als 1'900 
Projekten pro Jahr bei über 800 
Kunden 
Stand 12/2012 
Hamburg 
Düsseldorf 
Frankfurt 
Freiburg 
München 
Wien 
Basel 
Bern Zürich 
Lausanne 
3 
Stuttgart 
Partitioning Your Oracle Data Warehouse 
2009 
3
AGENDA 
1. Partitioning Concepts 
2. The Right Partition Key 
3. Large Dimensions 
4. Partition Maintenance 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
4
Partitioning – The Basic Idea 
 Decompose tables/indexes into smaller pieces 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
5
Partitioning Methods in Oracle 
 Additionally: Interval Partitioning, Reference Partitioning, Virtual 
Column-Based Partitioning 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
6
Benefits of Partitioning 
Partition Pruning 
 Reduce I/O: Only relevant partitions have to be accessed 
Partition-Wise Joins 
 Full PWJ: Join two equi-partitioned tables (parallel or serial) 
 Partial PWJ: Join partitioned with non-partitioned table (parallel) 
Rolling History 
 Create new partitions in periodical intervals 
 Drop old partitions in periodical intervals 
Manageability 
 Backups, statistics gathering, compressing on individual partitions 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
7
AGENDA 
1. Partitioning Concepts 
2. The Right Partition Key 
3. Large Dimensions 
4. Partition Maintenance 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
8
Partition Methods and Partition Keys 
 Important questions for Partitioning: 
 Which tables should be partitioned? 
 Which partition method should be used? 
 Which partition key(s) should be used? 
 Partition key is important for 
 Query optimization (partition pruning, partition-wise 
2014 © Trivadis 
joins) 
 ETL performance (partition exchange, rolling 
history) 
 Typically in data warehouses 
 RANGE partitioning of fact tables on DATE 
column 
 But which DATE column? 
Partitioning Your Oracle Data Warehouse 
2009 
9
Practical Example 1: Airline Company 
 Flight bookings are stored in partitioned table 
 RANGE partitioning per month, partition key: booking date 
 Problem: Most of the reports are based on the flight date 
 Flights can be booked 11 months ahead 
 11 partitions must be read of one particular flight date 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
10
Practical Example 1: Airline Company 
 Solution: partition key flight date instead of booking date 
 Data is loaded into current and future partitions 
 Reports based on flight date read only one partition 
 Reports based on booking date must read 11 (small) partitions 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
11
Practical Example 1: Airline Company 
 Better solution: Composite RANGE-RANGE partitioning 
 RANGE partitioning on flight date 
 RANGE subpartitioning on booking date 
 More flexibility for reports on flight date and/or on booking date 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
12
Practical Example 2: International Bank 
 Account balance data of international customers 
 Monthly files of different locations (countries) 
 Correction files replace last delivery for the same month/country 
 Original approach 
 Technical load id for each combination of month/country 
 LIST partitioning on load id 
 Files are loaded in stage table 
 Partition exchange with current partition 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
13
Practical Example 2: International Bank 
 Problem: partition key load id is useless for queries 
 Queries are based on balance date 
2014 © Trivadis 
 Solution 
 RANGE partitioning on balance date 
 LIST subpartitions on country code 
 Partition exchange with subpartitions 
Partitioning Your Oracle Data Warehouse 
2009 
14
AGENDA 
1. Partitioning Concepts 
2. The Right Partition Key 
3. Large Dimensions 
4. Partition Maintenance 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
15
Partitioning in Star Schema 
2014 © Trivadis 
 Fact Table 
 Usually „big“ (millions to billions of rows) 
 RANGE partitioning by DATE column 
 Dimension Tables 
 Usually „small“ (10 to 10000 rows) 
 In most cases not partitioned 
 But how about large dimensions? 
 e.g. customer dimension with millions of 
rows 
Partitioning Your Oracle Data Warehouse 
2009 
16
HASH Partitioning on Large Dimension 
 DIM_CUSTOMER 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
17 
 HASH Partitioning 
 Partition Key: CUSTOMER_ID 
 FCT_SALES 
 Composite RANGE-HASH Partitioning 
 Partition Key: SALES_DATE 
 Subpartition Key: CUSTOMER_ID
HASH Partitioning on Large Dimension 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
18
Practical Example 3: Telecommunication Company 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
19
LIST Partitioning on Large Dimension 
 DIM_CUSTOMER 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
20 
 LIST Partitioning 
 Partition Key: COUNTRY_CODE 
 FCT_SALES 
 Composite RANGE-LIST Partitioning 
 Partition Key: SALES_DATE 
 Subpartition Key: COUNTRY_CODE 
(denormalized column in fact table)
LIST Partitioning on Large Dimension 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
21
 New approach for partition pruning on join conditions 
 A bloom filter is created based on the dimension table restriction 
 Dynamic partition pruning based on that bloom filter 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
22
AGENDA 
1. Partitioning Concepts 
2. The Right Partition Key 
3. Large Dimensions 
4. Partition Maintenance 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
23
Practical Example 3: Monthly Partition Maintenance 
 Requirements 
 Monthly partitions on all fact tables, daily inserts into current partitions 
 3 years of history (36 partitions per table) 
 Table compression to increase full table scan performance 
 Backup of current partitions only 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
24
Practical Example 3: Monthly Partition Maintenance 
1. Set next tablespace to read-write 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
25
Practical Example 3: Monthly Partition Maintenance 
1. Set next tablespace to read-write 
2. Drop oldest partition 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
26
Practical Example 3: Monthly Partition Maintenance 
1. Set next tablespace to read-write 
2. Drop oldest partition 
3. Create new partition for next month 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
27
Practical Example 3: Monthly Partition Maintenance 
1. Set next tablespace to read-write 
2. Drop oldest partition 
3. Create new partition for next month 
4. Compress current partition 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
28
Practical Example 3: Monthly Partition Maintenance 
1. Set next tablespace to read-write 
2. Drop oldest partition 
3. Create new partition for next month 
4. Compress current partition 
5. Set tablespace to read-only 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
29
Interval Partitioning 
 In Oracle11g, the creation of new partitions can be automated 
2014 © Trivadis 
 Example: 
Partitioning Your Oracle Data Warehouse 
2009 
30
Gathering Optimizer Statistics 
 DBMS_STATS parameter GRANULARITY defines statistics level 
2014 © Trivadis 
 Example: 
Partitioning Your Oracle Data Warehouse 
2009 
31
Problem of Global Statistics 
 Global statistics are essential for good execution plans 
 num_distinct, low_value, high_value, density, histograms 
 Gathering global statistics is time-consuming 
 All partitions must be scanned 
 Typical approach 
 After loading data, only modified partition statistics are gathered 
 Global statistics are gathered on regular time base (e.g. weekends) 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
32
Incremental Global Statistics 
 Synopsis-based gathering of statistics 
 For each partition a synopsis is stored in SYSAUX tablespace 
 Statistics metadata for partition and columns of partition 
 Global statistics by aggregating the synopses from each partition 
 Activate incremental global statistics: 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
33
Partitioning Your Data Warehouse – Core Messages 
2014 © Trivadis 
 Oracle Partitioning is a powerful option 
– not only for data warehouses 
 The concept is simple, but the reality 
can be complex 
 Many new partitioning features added 
in Oracle Database 11g 
 New Composite Partitioning methods 
 Interval Partitioning 
 Join-Filter Pruning 
 Incremental Global Statistics 
Partitioning Your Oracle Data Warehouse 
2009 
34
Thank you. 
Dani Schnider 
Principal Consultant 
Business Intelligence 
dani.schnider@trivadis.com 
Oracle Open World 2009, 
San Francisco 
BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN 
2014 © Trivadis 
Partitioning Your Oracle Data Warehouse 
2009 
35

More Related Content

What's hot

Day 02 sap_bi_overview_and_terminology
Day 02 sap_bi_overview_and_terminologyDay 02 sap_bi_overview_and_terminology
Day 02 sap_bi_overview_and_terminology
tovetrivel
 

What's hot (19)

Using SSRS Reports with SSAS Cubes
Using SSRS Reports with SSAS CubesUsing SSRS Reports with SSAS Cubes
Using SSRS Reports with SSAS Cubes
 
From Raw Data to Analytics with No ETL
From Raw Data to Analytics with No ETLFrom Raw Data to Analytics with No ETL
From Raw Data to Analytics with No ETL
 
OLAP
OLAPOLAP
OLAP
 
Star schema my sql
Star schema   my sqlStar schema   my sql
Star schema my sql
 
Data Warehouse Best Practices
Data Warehouse Best PracticesData Warehouse Best Practices
Data Warehouse Best Practices
 
Data warehouse
Data warehouseData warehouse
Data warehouse
 
Data Warehouse Logical Design using Mysql
Data Warehouse Logical Design using MysqlData Warehouse Logical Design using Mysql
Data Warehouse Logical Design using Mysql
 
Oltp vs olap
Oltp vs olapOltp vs olap
Oltp vs olap
 
Oracle GoldenGate for Oracle DBAs
Oracle GoldenGate for Oracle DBAsOracle GoldenGate for Oracle DBAs
Oracle GoldenGate for Oracle DBAs
 
Getting the most out of your Oracle 12.2 Optimizer (i.e. The Brain)
Getting the most out of your Oracle 12.2 Optimizer (i.e. The Brain)Getting the most out of your Oracle 12.2 Optimizer (i.e. The Brain)
Getting the most out of your Oracle 12.2 Optimizer (i.e. The Brain)
 
Oracle
OracleOracle
Oracle
 
Teradata 13.10
Teradata 13.10Teradata 13.10
Teradata 13.10
 
Making MySQL Great For Business Intelligence
Making MySQL Great For Business IntelligenceMaking MySQL Great For Business Intelligence
Making MySQL Great For Business Intelligence
 
Oracle 12c
Oracle 12cOracle 12c
Oracle 12c
 
Data Warehouse
Data Warehouse Data Warehouse
Data Warehouse
 
OLAP
OLAPOLAP
OLAP
 
Part 3 - Data Warehousing Lecture at BW Cooperative State University (DHBW)
Part 3 - Data Warehousing Lecture at BW Cooperative State University (DHBW)Part 3 - Data Warehousing Lecture at BW Cooperative State University (DHBW)
Part 3 - Data Warehousing Lecture at BW Cooperative State University (DHBW)
 
Day 02 sap_bi_overview_and_terminology
Day 02 sap_bi_overview_and_terminologyDay 02 sap_bi_overview_and_terminology
Day 02 sap_bi_overview_and_terminology
 
Consolidate More: High Performance Primary Deduplication in the Age of Abunda...
Consolidate More: High Performance Primary Deduplication in the Age of Abunda...Consolidate More: High Performance Primary Deduplication in the Age of Abunda...
Consolidate More: High Performance Primary Deduplication in the Age of Abunda...
 

Similar to Partitioning your Oracle Data Warehouse - Just a simple task?

WSC Net App storage for windows challenges and solutions
WSC Net App storage for windows challenges and solutionsWSC Net App storage for windows challenges and solutions
WSC Net App storage for windows challenges and solutions
Accenture
 
introduction to datawarehouse
introduction to datawarehouseintroduction to datawarehouse
introduction to datawarehouse
kiran14360
 
galileo-white-paper-tier-0
galileo-white-paper-tier-0galileo-white-paper-tier-0
galileo-white-paper-tier-0
Tim Conley
 
Rando Veizi: Data warehouse and Pentaho suite
Rando Veizi: Data warehouse and Pentaho suiteRando Veizi: Data warehouse and Pentaho suite
Rando Veizi: Data warehouse and Pentaho suite
Carlo Vaccari
 
4 Infrastructure Data Analysis
4 Infrastructure Data Analysis4 Infrastructure Data Analysis
4 Infrastructure Data Analysis
Jeremiah Loscalzo
 

Similar to Partitioning your Oracle Data Warehouse - Just a simple task? (20)

FDMEE Can Do That?
FDMEE Can Do That?FDMEE Can Do That?
FDMEE Can Do That?
 
Db2 update day 2015 managing db2 with ibm db2 tools svenn aage
Db2 update day 2015   managing db2 with ibm db2 tools svenn aageDb2 update day 2015   managing db2 with ibm db2 tools svenn aage
Db2 update day 2015 managing db2 with ibm db2 tools svenn aage
 
Nw2008 tips tricks_edw_v10
Nw2008 tips tricks_edw_v10Nw2008 tips tricks_edw_v10
Nw2008 tips tricks_edw_v10
 
Overcoming Data Gravity in Multi-Cloud Enterprise Architectures
Overcoming Data Gravity in Multi-Cloud Enterprise ArchitecturesOvercoming Data Gravity in Multi-Cloud Enterprise Architectures
Overcoming Data Gravity in Multi-Cloud Enterprise Architectures
 
WSC Net App storage for windows challenges and solutions
WSC Net App storage for windows challenges and solutionsWSC Net App storage for windows challenges and solutions
WSC Net App storage for windows challenges and solutions
 
SQL Server 2017 - Mejoras Impulsadas por la Comunidad
SQL Server 2017 - Mejoras Impulsadas por la ComunidadSQL Server 2017 - Mejoras Impulsadas por la Comunidad
SQL Server 2017 - Mejoras Impulsadas por la Comunidad
 
EBS Answers Webinar Series - Chart of Accounts Transformation Master Class: T...
EBS Answers Webinar Series - Chart of Accounts Transformation Master Class: T...EBS Answers Webinar Series - Chart of Accounts Transformation Master Class: T...
EBS Answers Webinar Series - Chart of Accounts Transformation Master Class: T...
 
introduction to datawarehouse
introduction to datawarehouseintroduction to datawarehouse
introduction to datawarehouse
 
Part 1 - Data Warehousing Lecture at BW Cooperative State University (DHBW)
Part 1 - Data Warehousing Lecture at BW Cooperative State University (DHBW)Part 1 - Data Warehousing Lecture at BW Cooperative State University (DHBW)
Part 1 - Data Warehousing Lecture at BW Cooperative State University (DHBW)
 
galileo-white-paper-tier-0
galileo-white-paper-tier-0galileo-white-paper-tier-0
galileo-white-paper-tier-0
 
vVols and Your Cloud Operating Model with Tristan Todd
vVols and Your Cloud Operating Model with Tristan ToddvVols and Your Cloud Operating Model with Tristan Todd
vVols and Your Cloud Operating Model with Tristan Todd
 
Teradata Aggregate Join Indices And Dimensional Models
Teradata Aggregate Join Indices And Dimensional ModelsTeradata Aggregate Join Indices And Dimensional Models
Teradata Aggregate Join Indices And Dimensional Models
 
Performance Tuning Oracle's BI Applications
Performance Tuning Oracle's BI ApplicationsPerformance Tuning Oracle's BI Applications
Performance Tuning Oracle's BI Applications
 
Today's Need To Manage The Storage Polymorphism
Today's Need To Manage The Storage PolymorphismToday's Need To Manage The Storage Polymorphism
Today's Need To Manage The Storage Polymorphism
 
Prague data management meetup 2017-02-28
Prague data management meetup 2017-02-28Prague data management meetup 2017-02-28
Prague data management meetup 2017-02-28
 
Rando Veizi: Data warehouse and Pentaho suite
Rando Veizi: Data warehouse and Pentaho suiteRando Veizi: Data warehouse and Pentaho suite
Rando Veizi: Data warehouse and Pentaho suite
 
4 Infrastructure Data Analysis
4 Infrastructure Data Analysis4 Infrastructure Data Analysis
4 Infrastructure Data Analysis
 
ACIC Rome & Veritas: High-Availability and Disaster Recovery Scenarios
ACIC Rome & Veritas: High-Availability and Disaster Recovery ScenariosACIC Rome & Veritas: High-Availability and Disaster Recovery Scenarios
ACIC Rome & Veritas: High-Availability and Disaster Recovery Scenarios
 
Big Data & Information Management Channel Manager
Big Data & Information Management Channel ManagerBig Data & Information Management Channel Manager
Big Data & Information Management Channel Manager
 
Oracle Big Data Appliance and Big Data SQL for advanced analytics
Oracle Big Data Appliance and Big Data SQL for advanced analyticsOracle Big Data Appliance and Big Data SQL for advanced analytics
Oracle Big Data Appliance and Big Data SQL for advanced analytics
 

More from Trivadis

More from Trivadis (20)

Azure Days 2019: Azure Chatbot Development for Airline Irregularities (Remco ...
Azure Days 2019: Azure Chatbot Development for Airline Irregularities (Remco ...Azure Days 2019: Azure Chatbot Development for Airline Irregularities (Remco ...
Azure Days 2019: Azure Chatbot Development for Airline Irregularities (Remco ...
 
Azure Days 2019: Trivadis Azure Foundation – Das Fundament für den ... (Nisan...
Azure Days 2019: Trivadis Azure Foundation – Das Fundament für den ... (Nisan...Azure Days 2019: Trivadis Azure Foundation – Das Fundament für den ... (Nisan...
Azure Days 2019: Trivadis Azure Foundation – Das Fundament für den ... (Nisan...
 
Azure Days 2019: Business Intelligence auf Azure (Marco Amhof & Yves Mauron)
Azure Days 2019: Business Intelligence auf Azure (Marco Amhof & Yves Mauron)Azure Days 2019: Business Intelligence auf Azure (Marco Amhof & Yves Mauron)
Azure Days 2019: Business Intelligence auf Azure (Marco Amhof & Yves Mauron)
 
Azure Days 2019: Master the Move to Azure (Konrad Brunner)
Azure Days 2019: Master the Move to Azure (Konrad Brunner)Azure Days 2019: Master the Move to Azure (Konrad Brunner)
Azure Days 2019: Master the Move to Azure (Konrad Brunner)
 
Azure Days 2019: Keynote Azure Switzerland – Status Quo und Ausblick (Primo A...
Azure Days 2019: Keynote Azure Switzerland – Status Quo und Ausblick (Primo A...Azure Days 2019: Keynote Azure Switzerland – Status Quo und Ausblick (Primo A...
Azure Days 2019: Keynote Azure Switzerland – Status Quo und Ausblick (Primo A...
 
Azure Days 2019: Grösser und Komplexer ist nicht immer besser (Meinrad Weiss)
Azure Days 2019: Grösser und Komplexer ist nicht immer besser (Meinrad Weiss)Azure Days 2019: Grösser und Komplexer ist nicht immer besser (Meinrad Weiss)
Azure Days 2019: Grösser und Komplexer ist nicht immer besser (Meinrad Weiss)
 
Azure Days 2019: Get Connected with Azure API Management (Gerry Keune & Stefa...
Azure Days 2019: Get Connected with Azure API Management (Gerry Keune & Stefa...Azure Days 2019: Get Connected with Azure API Management (Gerry Keune & Stefa...
Azure Days 2019: Get Connected with Azure API Management (Gerry Keune & Stefa...
 
Azure Days 2019: Infrastructure as Code auf Azure (Jonas Wanninger & Daniel H...
Azure Days 2019: Infrastructure as Code auf Azure (Jonas Wanninger & Daniel H...Azure Days 2019: Infrastructure as Code auf Azure (Jonas Wanninger & Daniel H...
Azure Days 2019: Infrastructure as Code auf Azure (Jonas Wanninger & Daniel H...
 
Azure Days 2019: Wie bringt man eine Data Analytics Plattform in die Cloud? (...
Azure Days 2019: Wie bringt man eine Data Analytics Plattform in die Cloud? (...Azure Days 2019: Wie bringt man eine Data Analytics Plattform in die Cloud? (...
Azure Days 2019: Wie bringt man eine Data Analytics Plattform in die Cloud? (...
 
Azure Days 2019: Azure@Helsana: Die Erweiterung von Dynamics CRM mit Azure Po...
Azure Days 2019: Azure@Helsana: Die Erweiterung von Dynamics CRM mit Azure Po...Azure Days 2019: Azure@Helsana: Die Erweiterung von Dynamics CRM mit Azure Po...
Azure Days 2019: Azure@Helsana: Die Erweiterung von Dynamics CRM mit Azure Po...
 
TechEvent 2019: Kundenstory - Kein Angebot, kein Auftrag – Wie Du ein individ...
TechEvent 2019: Kundenstory - Kein Angebot, kein Auftrag – Wie Du ein individ...TechEvent 2019: Kundenstory - Kein Angebot, kein Auftrag – Wie Du ein individ...
TechEvent 2019: Kundenstory - Kein Angebot, kein Auftrag – Wie Du ein individ...
 
TechEvent 2019: Oracle Database Appliance M/L - Erfahrungen und Erfolgsmethod...
TechEvent 2019: Oracle Database Appliance M/L - Erfahrungen und Erfolgsmethod...TechEvent 2019: Oracle Database Appliance M/L - Erfahrungen und Erfolgsmethod...
TechEvent 2019: Oracle Database Appliance M/L - Erfahrungen und Erfolgsmethod...
 
TechEvent 2019: Security 101 für Web Entwickler; Roland Krüger - Trivadis
TechEvent 2019: Security 101 für Web Entwickler; Roland Krüger - TrivadisTechEvent 2019: Security 101 für Web Entwickler; Roland Krüger - Trivadis
TechEvent 2019: Security 101 für Web Entwickler; Roland Krüger - Trivadis
 
TechEvent 2019: Trivadis & Swisscom Partner Angebote; Konrad Häfeli, Markus O...
TechEvent 2019: Trivadis & Swisscom Partner Angebote; Konrad Häfeli, Markus O...TechEvent 2019: Trivadis & Swisscom Partner Angebote; Konrad Häfeli, Markus O...
TechEvent 2019: Trivadis & Swisscom Partner Angebote; Konrad Häfeli, Markus O...
 
TechEvent 2019: DBaaS from Swisscom Cloud powered by Trivadis; Konrad Häfeli ...
TechEvent 2019: DBaaS from Swisscom Cloud powered by Trivadis; Konrad Häfeli ...TechEvent 2019: DBaaS from Swisscom Cloud powered by Trivadis; Konrad Häfeli ...
TechEvent 2019: DBaaS from Swisscom Cloud powered by Trivadis; Konrad Häfeli ...
 
TechEvent 2019: Status of the partnership Trivadis and EDB - Comparing Postgr...
TechEvent 2019: Status of the partnership Trivadis and EDB - Comparing Postgr...TechEvent 2019: Status of the partnership Trivadis and EDB - Comparing Postgr...
TechEvent 2019: Status of the partnership Trivadis and EDB - Comparing Postgr...
 
TechEvent 2019: More Agile, More AI, More Cloud! Less Work?!; Oliver Dörr - T...
TechEvent 2019: More Agile, More AI, More Cloud! Less Work?!; Oliver Dörr - T...TechEvent 2019: More Agile, More AI, More Cloud! Less Work?!; Oliver Dörr - T...
TechEvent 2019: More Agile, More AI, More Cloud! Less Work?!; Oliver Dörr - T...
 
TechEvent 2019: Kundenstory - Vom Hauptmann zu Köpenick zum Polizisten 2020 -...
TechEvent 2019: Kundenstory - Vom Hauptmann zu Köpenick zum Polizisten 2020 -...TechEvent 2019: Kundenstory - Vom Hauptmann zu Köpenick zum Polizisten 2020 -...
TechEvent 2019: Kundenstory - Vom Hauptmann zu Köpenick zum Polizisten 2020 -...
 
TechEvent 2019: Vom Rechenzentrum in die Oracle Cloud - Übertragungsmethoden;...
TechEvent 2019: Vom Rechenzentrum in die Oracle Cloud - Übertragungsmethoden;...TechEvent 2019: Vom Rechenzentrum in die Oracle Cloud - Übertragungsmethoden;...
TechEvent 2019: Vom Rechenzentrum in die Oracle Cloud - Übertragungsmethoden;...
 
TechEvent 2019: The sleeping Power of Data; Eberhard Lösch - Trivadis
TechEvent 2019: The sleeping Power of Data; Eberhard Lösch - TrivadisTechEvent 2019: The sleeping Power of Data; Eberhard Lösch - Trivadis
TechEvent 2019: The sleeping Power of Data; Eberhard Lösch - Trivadis
 

Recently uploaded

Recently uploaded (9)

Understanding Poverty: A Community Questionnaire
Understanding Poverty: A Community QuestionnaireUnderstanding Poverty: A Community Questionnaire
Understanding Poverty: A Community Questionnaire
 
Breathing in New Life_ Part 3 05 22 2024.pptx
Breathing in New Life_ Part 3 05 22 2024.pptxBreathing in New Life_ Part 3 05 22 2024.pptx
Breathing in New Life_ Part 3 05 22 2024.pptx
 
Oracle Database Administration I (1Z0-082) Exam Dumps 2024.pdf
Oracle Database Administration I (1Z0-082) Exam Dumps 2024.pdfOracle Database Administration I (1Z0-082) Exam Dumps 2024.pdf
Oracle Database Administration I (1Z0-082) Exam Dumps 2024.pdf
 
Microsoft Fabric Analytics Engineer (DP-600) Exam Dumps 2024.pdf
Microsoft Fabric Analytics Engineer (DP-600) Exam Dumps 2024.pdfMicrosoft Fabric Analytics Engineer (DP-600) Exam Dumps 2024.pdf
Microsoft Fabric Analytics Engineer (DP-600) Exam Dumps 2024.pdf
 
The Influence and Evolution of Mogul Press in Contemporary Public Relations.docx
The Influence and Evolution of Mogul Press in Contemporary Public Relations.docxThe Influence and Evolution of Mogul Press in Contemporary Public Relations.docx
The Influence and Evolution of Mogul Press in Contemporary Public Relations.docx
 
Deciding The Topic of our Magazine.pptx.
Deciding The Topic of our Magazine.pptx.Deciding The Topic of our Magazine.pptx.
Deciding The Topic of our Magazine.pptx.
 
DAY 0 8 A Revelation 05-19-2024 PPT.pptx
DAY 0 8 A Revelation 05-19-2024 PPT.pptxDAY 0 8 A Revelation 05-19-2024 PPT.pptx
DAY 0 8 A Revelation 05-19-2024 PPT.pptx
 
ACM CHT Best Inspection Practices Kinben Innovation MIC Slideshare.pdf
ACM CHT Best Inspection Practices Kinben Innovation MIC Slideshare.pdfACM CHT Best Inspection Practices Kinben Innovation MIC Slideshare.pdf
ACM CHT Best Inspection Practices Kinben Innovation MIC Slideshare.pdf
 
ServiceNow CIS-Discovery Exam Dumps 2024
ServiceNow CIS-Discovery Exam Dumps 2024ServiceNow CIS-Discovery Exam Dumps 2024
ServiceNow CIS-Discovery Exam Dumps 2024
 

Partitioning your Oracle Data Warehouse - Just a simple task?

  • 1. Partitioning Your Oracle Data Warehouse – Just a Simple Task? Dani Schnider Principal Consultant Business Intelligence dani.schnider@trivadis.com Oracle Open World 2009, San Francisco BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 1
  • 2. Dani Schnider  Principal Consultant und DWH/BI Lead Architect bei Trivadis in Zürich  Kursleiter für Trivadis-Kurse über Data Warehousing, SQL Optimierung und Oracle Warehouse Builder  Co-Autor des Buches «Data Warehousing mit Oracle» 2014 © Trivadis 2 Partitioning Your Oracle Data Warehouse 2009
  • 3. Mit über 600 IT- und Fachexperten bei Ihnen vor Ort 2014 © Trivadis 11 Trivadis Niederlassungen mit über 600 Mitarbeitenden 200 Service Level Agreements Mehr als 4'000 Trainingsteilnehmer Forschungs- und Entwicklungs-budget: CHF 5.0 / EUR 4 Mio. Finanziell unabhängig und nachhaltig profitabel Erfahrung aus mehr als 1'900 Projekten pro Jahr bei über 800 Kunden Stand 12/2012 Hamburg Düsseldorf Frankfurt Freiburg München Wien Basel Bern Zürich Lausanne 3 Stuttgart Partitioning Your Oracle Data Warehouse 2009 3
  • 4. AGENDA 1. Partitioning Concepts 2. The Right Partition Key 3. Large Dimensions 4. Partition Maintenance 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 4
  • 5. Partitioning – The Basic Idea  Decompose tables/indexes into smaller pieces 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 5
  • 6. Partitioning Methods in Oracle  Additionally: Interval Partitioning, Reference Partitioning, Virtual Column-Based Partitioning 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 6
  • 7. Benefits of Partitioning Partition Pruning  Reduce I/O: Only relevant partitions have to be accessed Partition-Wise Joins  Full PWJ: Join two equi-partitioned tables (parallel or serial)  Partial PWJ: Join partitioned with non-partitioned table (parallel) Rolling History  Create new partitions in periodical intervals  Drop old partitions in periodical intervals Manageability  Backups, statistics gathering, compressing on individual partitions 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 7
  • 8. AGENDA 1. Partitioning Concepts 2. The Right Partition Key 3. Large Dimensions 4. Partition Maintenance 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 8
  • 9. Partition Methods and Partition Keys  Important questions for Partitioning:  Which tables should be partitioned?  Which partition method should be used?  Which partition key(s) should be used?  Partition key is important for  Query optimization (partition pruning, partition-wise 2014 © Trivadis joins)  ETL performance (partition exchange, rolling history)  Typically in data warehouses  RANGE partitioning of fact tables on DATE column  But which DATE column? Partitioning Your Oracle Data Warehouse 2009 9
  • 10. Practical Example 1: Airline Company  Flight bookings are stored in partitioned table  RANGE partitioning per month, partition key: booking date  Problem: Most of the reports are based on the flight date  Flights can be booked 11 months ahead  11 partitions must be read of one particular flight date 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 10
  • 11. Practical Example 1: Airline Company  Solution: partition key flight date instead of booking date  Data is loaded into current and future partitions  Reports based on flight date read only one partition  Reports based on booking date must read 11 (small) partitions 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 11
  • 12. Practical Example 1: Airline Company  Better solution: Composite RANGE-RANGE partitioning  RANGE partitioning on flight date  RANGE subpartitioning on booking date  More flexibility for reports on flight date and/or on booking date 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 12
  • 13. Practical Example 2: International Bank  Account balance data of international customers  Monthly files of different locations (countries)  Correction files replace last delivery for the same month/country  Original approach  Technical load id for each combination of month/country  LIST partitioning on load id  Files are loaded in stage table  Partition exchange with current partition 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 13
  • 14. Practical Example 2: International Bank  Problem: partition key load id is useless for queries  Queries are based on balance date 2014 © Trivadis  Solution  RANGE partitioning on balance date  LIST subpartitions on country code  Partition exchange with subpartitions Partitioning Your Oracle Data Warehouse 2009 14
  • 15. AGENDA 1. Partitioning Concepts 2. The Right Partition Key 3. Large Dimensions 4. Partition Maintenance 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 15
  • 16. Partitioning in Star Schema 2014 © Trivadis  Fact Table  Usually „big“ (millions to billions of rows)  RANGE partitioning by DATE column  Dimension Tables  Usually „small“ (10 to 10000 rows)  In most cases not partitioned  But how about large dimensions?  e.g. customer dimension with millions of rows Partitioning Your Oracle Data Warehouse 2009 16
  • 17. HASH Partitioning on Large Dimension  DIM_CUSTOMER 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 17  HASH Partitioning  Partition Key: CUSTOMER_ID  FCT_SALES  Composite RANGE-HASH Partitioning  Partition Key: SALES_DATE  Subpartition Key: CUSTOMER_ID
  • 18. HASH Partitioning on Large Dimension 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 18
  • 19. Practical Example 3: Telecommunication Company 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 19
  • 20. LIST Partitioning on Large Dimension  DIM_CUSTOMER 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 20  LIST Partitioning  Partition Key: COUNTRY_CODE  FCT_SALES  Composite RANGE-LIST Partitioning  Partition Key: SALES_DATE  Subpartition Key: COUNTRY_CODE (denormalized column in fact table)
  • 21. LIST Partitioning on Large Dimension 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 21
  • 22.  New approach for partition pruning on join conditions  A bloom filter is created based on the dimension table restriction  Dynamic partition pruning based on that bloom filter 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 22
  • 23. AGENDA 1. Partitioning Concepts 2. The Right Partition Key 3. Large Dimensions 4. Partition Maintenance 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 23
  • 24. Practical Example 3: Monthly Partition Maintenance  Requirements  Monthly partitions on all fact tables, daily inserts into current partitions  3 years of history (36 partitions per table)  Table compression to increase full table scan performance  Backup of current partitions only 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 24
  • 25. Practical Example 3: Monthly Partition Maintenance 1. Set next tablespace to read-write 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 25
  • 26. Practical Example 3: Monthly Partition Maintenance 1. Set next tablespace to read-write 2. Drop oldest partition 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 26
  • 27. Practical Example 3: Monthly Partition Maintenance 1. Set next tablespace to read-write 2. Drop oldest partition 3. Create new partition for next month 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 27
  • 28. Practical Example 3: Monthly Partition Maintenance 1. Set next tablespace to read-write 2. Drop oldest partition 3. Create new partition for next month 4. Compress current partition 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 28
  • 29. Practical Example 3: Monthly Partition Maintenance 1. Set next tablespace to read-write 2. Drop oldest partition 3. Create new partition for next month 4. Compress current partition 5. Set tablespace to read-only 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 29
  • 30. Interval Partitioning  In Oracle11g, the creation of new partitions can be automated 2014 © Trivadis  Example: Partitioning Your Oracle Data Warehouse 2009 30
  • 31. Gathering Optimizer Statistics  DBMS_STATS parameter GRANULARITY defines statistics level 2014 © Trivadis  Example: Partitioning Your Oracle Data Warehouse 2009 31
  • 32. Problem of Global Statistics  Global statistics are essential for good execution plans  num_distinct, low_value, high_value, density, histograms  Gathering global statistics is time-consuming  All partitions must be scanned  Typical approach  After loading data, only modified partition statistics are gathered  Global statistics are gathered on regular time base (e.g. weekends) 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 32
  • 33. Incremental Global Statistics  Synopsis-based gathering of statistics  For each partition a synopsis is stored in SYSAUX tablespace  Statistics metadata for partition and columns of partition  Global statistics by aggregating the synopses from each partition  Activate incremental global statistics: 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 33
  • 34. Partitioning Your Data Warehouse – Core Messages 2014 © Trivadis  Oracle Partitioning is a powerful option – not only for data warehouses  The concept is simple, but the reality can be complex  Many new partitioning features added in Oracle Database 11g  New Composite Partitioning methods  Interval Partitioning  Join-Filter Pruning  Incremental Global Statistics Partitioning Your Oracle Data Warehouse 2009 34
  • 35. Thank you. Dani Schnider Principal Consultant Business Intelligence dani.schnider@trivadis.com Oracle Open World 2009, San Francisco BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 35