SlideShare a Scribd company logo
Partitioning Design
For Performance and Maintainability



                                      Martin Cairns
                             http://sqlbyparts.com
             http://twitter.com/MartinCairnsSQL
Who am I?
Partitioning Defined

Partitioning (noun)

1. the action or state of dividing or being
   divided into parts
2. a structure dividing a space into two parts



                  Source: http://oxforddictionaries.com/definition/english/partition
Types of Partitioning

• Horizontal
  • Partition Table by Rows
• Vertical
  • Partition Table by Columns
• File Group
  • Partition Tables by File Group
Horizontal Partitioning

 Divides the rows into small sets by
  boundaries
 Year   PK        Order ID   Product ID   Qty    Cost   Line XML
 2010   1         1          1            5      1.5    <Line…
 2010   2         1          1            7      2      <Line…

 2011   1000000   20000      345          1000   5      <Line…
 2011   1000001   20000      347          3000   12     <Line…

 2012   2000000   40000      705          8000   3      <Line…
 2012   2000001   40001      706          3000   7      <Line…
Vertical Partitioning

 Divides columns from one table into multiple
  tables
 Year   PK        Order ID   …   Year   PK        Line XML
 2010   1         1          …   2010   1         <Line…
 2010   2         1          …   2010   2         <Line…
 2011   1000000   345        …   2011   1000000   <Line…
 2011   1000001   347        …   2011   1000001   <Line…
 2012   2000000   705        …   2012   2000000   <Line…
 2012   2000001   706        …   2012   2000001   <Line…
File Group Partitioning

 File group partitioning is separating the storage of tables
  and indexes onto separate database files
 During a Primary File Group restore the whole database
  will be offline
 Separate the system & user tables to allow the quickest
  restore time
 Since the partitioned database is made up of smaller
  parts it is easier to manage the storage location of the
  files
Using Table Partitioning

 Supports Horizontal Partitioning
 Partition Function defines boundaries based
  on a single column
 Partition Scheme defines the File Group
 Inserts automatically supported
 All partitions share the same definition as the
  table (indexes, columns, fill factor etc)
Table Level Limitations

 Only ~200 steps for each statistics across the
  whole table
 Online Index rebuilds are for the whole table
  not individual partitions
 Fill Factor
 Lock settings (Row, Page, Escalation, etc)
 Indexes are defined for the whole table rather
  than partitions
Using View Partitioning

 View Partitioning allows you to overcome
  Table Level Constraints
 Check Constraints used to define partitions
 Combine all tables with a UNION ALL View
 Can be used together with Table Partitioning
 Trigger required to allow INSERT with Identity
  column
 Allows more complex partitioning schemes
Candidates for Partitioning

 Large vs. Small Tables
 Replicated vs. Non-Replicated Tables
 Normal vs. BLOB columns
 Write Heavy Current data vs. Heavily
  Read Historic data
 Read  Write vs. Read Only Tables
Partial Database Availability

 Also referred to as Piecemeal Restore
 Full Recovery Model is Required
 Individual File & File Groups Restore
 Only the Primary File Group is required to
  restore a database
 Allows restoring a subset of a correctly
  partitioned database for quicker recovery
  from a disaster
Tipping Point  Why is it Table Scanning

 As the number of rows in a table increase the
  depth of the B+Tree increases
 The number of Page Reads for a lookup is
  equal to the depth of the B+Tree
 The Tipping Point is the point where a Full
  Table Scan requires less Page Reads than
  the lookups
What Does Partition Give Us

 Choices
   Using different Tiers of Storage
   Allows quicker recovery strategies
   Allows partial restores of the database
 Performance
   Allows reduction of overhead of backups
   Allows better query plans due to more
    accurate statistics on large tables
Demos

 Moving Tables to File Groups
 Partial Restore + Using Partial Restore to
  Initialise Replication
 Moving Partition Between Tables For Current
  vs. Historic
 Show how View Partitioning can present a
  single view of all tables & still support
  partition elimination

More Related Content

Viewers also liked

History n Working Of Laser Printer
History n Working Of Laser PrinterHistory n Working Of Laser Printer
History n Working Of Laser PrinterAwais Alam
 
7 state model in Operating System
7 state model in Operating System7 state model in Operating System
7 state model in Operating SystemMohammad jawad khan
 
Oracle Insert Statements for DBAs and Developers
Oracle Insert Statements for DBAs and DevelopersOracle Insert Statements for DBAs and Developers
Oracle Insert Statements for DBAs and Developers
Guatemala User Group
 
Usability issues in google chrome & its solutions.
Usability issues in google chrome & its solutions.Usability issues in google chrome & its solutions.
Usability issues in google chrome & its solutions.Mohammad jawad khan
 
Laser printers
Laser printersLaser printers
Laser printers
Abhijit Jadhav
 
Printer Presentation - H.Toor
Printer Presentation - H.ToorPrinter Presentation - H.Toor
Printer Presentation - H.Toorhst77
 
11. Hashing - Data Structures using C++ by Varsha Patil
11. Hashing - Data Structures using C++ by Varsha Patil11. Hashing - Data Structures using C++ by Varsha Patil
11. Hashing - Data Structures using C++ by Varsha Patil
widespreadpromotion
 
Concept of hashing
Concept of hashingConcept of hashing
Concept of hashingRafi Dar
 
7. Tree - Data Structures using C++ by Varsha Patil
7. Tree - Data Structures using C++ by Varsha Patil7. Tree - Data Structures using C++ by Varsha Patil
7. Tree - Data Structures using C++ by Varsha Patil
widespreadpromotion
 
Hashing Techniques in Data Structures Part2
Hashing Techniques in Data Structures Part2Hashing Techniques in Data Structures Part2
Hashing Techniques in Data Structures Part2
SHAKOOR AB
 
6. Linked list - Data Structures using C++ by Varsha Patil
6. Linked list - Data Structures using C++ by Varsha Patil6. Linked list - Data Structures using C++ by Varsha Patil
6. Linked list - Data Structures using C++ by Varsha Patil
widespreadpromotion
 
NFA or Non deterministic finite automata
NFA or Non deterministic finite automataNFA or Non deterministic finite automata
NFA or Non deterministic finite automata
deepinderbedi
 
kerberos
kerberoskerberos
kerberos
sameer farooq
 
Types of printer
Types of printerTypes of printer
Types of printer
steven hasting
 
Finite Automata
Finite AutomataFinite Automata
Finite Automata
Mukesh Tekwani
 

Viewers also liked (20)

History n Working Of Laser Printer
History n Working Of Laser PrinterHistory n Working Of Laser Printer
History n Working Of Laser Printer
 
7 state model in Operating System
7 state model in Operating System7 state model in Operating System
7 state model in Operating System
 
Oracle Insert Statements for DBAs and Developers
Oracle Insert Statements for DBAs and DevelopersOracle Insert Statements for DBAs and Developers
Oracle Insert Statements for DBAs and Developers
 
Usability issues in google chrome & its solutions.
Usability issues in google chrome & its solutions.Usability issues in google chrome & its solutions.
Usability issues in google chrome & its solutions.
 
Laser printers
Laser printersLaser printers
Laser printers
 
Printers
 Printers Printers
Printers
 
Hashing
HashingHashing
Hashing
 
Printer Presentation - H.Toor
Printer Presentation - H.ToorPrinter Presentation - H.Toor
Printer Presentation - H.Toor
 
Probing
ProbingProbing
Probing
 
11. Hashing - Data Structures using C++ by Varsha Patil
11. Hashing - Data Structures using C++ by Varsha Patil11. Hashing - Data Structures using C++ by Varsha Patil
11. Hashing - Data Structures using C++ by Varsha Patil
 
PRINTERS
PRINTERSPRINTERS
PRINTERS
 
Concept of hashing
Concept of hashingConcept of hashing
Concept of hashing
 
Deterministic Finite Automata
Deterministic Finite AutomataDeterministic Finite Automata
Deterministic Finite Automata
 
7. Tree - Data Structures using C++ by Varsha Patil
7. Tree - Data Structures using C++ by Varsha Patil7. Tree - Data Structures using C++ by Varsha Patil
7. Tree - Data Structures using C++ by Varsha Patil
 
Hashing Techniques in Data Structures Part2
Hashing Techniques in Data Structures Part2Hashing Techniques in Data Structures Part2
Hashing Techniques in Data Structures Part2
 
6. Linked list - Data Structures using C++ by Varsha Patil
6. Linked list - Data Structures using C++ by Varsha Patil6. Linked list - Data Structures using C++ by Varsha Patil
6. Linked list - Data Structures using C++ by Varsha Patil
 
NFA or Non deterministic finite automata
NFA or Non deterministic finite automataNFA or Non deterministic finite automata
NFA or Non deterministic finite automata
 
kerberos
kerberoskerberos
kerberos
 
Types of printer
Types of printerTypes of printer
Types of printer
 
Finite Automata
Finite AutomataFinite Automata
Finite Automata
 

Similar to Partitioning Design

EstlGd.DataPartitioning
EstlGd.DataPartitioningEstlGd.DataPartitioning
EstlGd.DataPartitioningDaren Bieniek
 
Teched03 Index Maint Tony Bain
Teched03 Index Maint Tony BainTeched03 Index Maint Tony Bain
Teched03 Index Maint Tony Bain
Tony Bain
 
Sql server lesson7
Sql server lesson7Sql server lesson7
Sql server lesson7
Ala Qunaibi
 
SQL Server 2016 novelties
SQL Server 2016 noveltiesSQL Server 2016 novelties
SQL Server 2016 novelties
MSDEVMTL
 
Tuning and Optimizing U-SQL Queries (SQLPASS 2016)
Tuning and Optimizing U-SQL Queries (SQLPASS 2016)Tuning and Optimizing U-SQL Queries (SQLPASS 2016)
Tuning and Optimizing U-SQL Queries (SQLPASS 2016)
Michael Rys
 
Getting to know oracle database objects iot, mviews, clusters and more…
Getting to know oracle database objects iot, mviews, clusters and more…Getting to know oracle database objects iot, mviews, clusters and more…
Getting to know oracle database objects iot, mviews, clusters and more…Aaron Shilo
 
Optimizing Data Accessin Sq Lserver2005
Optimizing Data Accessin Sq Lserver2005Optimizing Data Accessin Sq Lserver2005
Optimizing Data Accessin Sq Lserver2005
rainynovember12
 
Database Performance
Database PerformanceDatabase Performance
Database Performance
Boris Hristov
 
Partitioning kendralittle
Partitioning kendralittlePartitioning kendralittle
Partitioning kendralittle
ngupt28
 
Pptofdistributeddb
PptofdistributeddbPptofdistributeddb
Pptofdistributeddb
Mahavir Devmane
 
Windows azure table storage – deep dive
Windows azure table storage – deep diveWindows azure table storage – deep dive
Windows azure table storage – deep dive
Sundararajan Subramanian
 
Best Practices and Performance Tuning of U-SQL in Azure Data Lake (SQL Konfer...
Best Practices and Performance Tuning of U-SQL in Azure Data Lake (SQL Konfer...Best Practices and Performance Tuning of U-SQL in Azure Data Lake (SQL Konfer...
Best Practices and Performance Tuning of U-SQL in Azure Data Lake (SQL Konfer...
Michael Rys
 
MS-SQL SERVER ARCHITECTURE
MS-SQL SERVER ARCHITECTUREMS-SQL SERVER ARCHITECTURE
MS-SQL SERVER ARCHITECTURE
Douglas Bernardini
 
初探AWS 平台上的 NoSQL 雲端資料庫服務
初探AWS 平台上的 NoSQL 雲端資料庫服務初探AWS 平台上的 NoSQL 雲端資料庫服務
初探AWS 平台上的 NoSQL 雲端資料庫服務
Amazon Web Services
 
Data warehouse physical design
Data warehouse physical designData warehouse physical design
Data warehouse physical design
Er. Nawaraj Bhandari
 
OVERVIEW OF NEW SQL,COMPARING SQL,NOSQL AND NEWSQL,B.Vinithamani,II-M.sc(Comp...
OVERVIEW OF NEW SQL,COMPARING SQL,NOSQL AND NEWSQL,B.Vinithamani,II-M.sc(Comp...OVERVIEW OF NEW SQL,COMPARING SQL,NOSQL AND NEWSQL,B.Vinithamani,II-M.sc(Comp...
OVERVIEW OF NEW SQL,COMPARING SQL,NOSQL AND NEWSQL,B.Vinithamani,II-M.sc(Comp...
vinithabalasubramani1
 
Ms sql server architecture
Ms sql server architectureMs sql server architecture
Ms sql server architecture
Ajeet Singh
 
2 extreme performance - smaller is better
2   extreme performance - smaller is better2   extreme performance - smaller is better
2 extreme performance - smaller is bettersqlserver.co.il
 

Similar to Partitioning Design (20)

EstlGd.DataPartitioning
EstlGd.DataPartitioningEstlGd.DataPartitioning
EstlGd.DataPartitioning
 
Teched03 Index Maint Tony Bain
Teched03 Index Maint Tony BainTeched03 Index Maint Tony Bain
Teched03 Index Maint Tony Bain
 
Sql server lesson7
Sql server lesson7Sql server lesson7
Sql server lesson7
 
SQL Server 2016 novelties
SQL Server 2016 noveltiesSQL Server 2016 novelties
SQL Server 2016 novelties
 
Tuning and Optimizing U-SQL Queries (SQLPASS 2016)
Tuning and Optimizing U-SQL Queries (SQLPASS 2016)Tuning and Optimizing U-SQL Queries (SQLPASS 2016)
Tuning and Optimizing U-SQL Queries (SQLPASS 2016)
 
Getting to know oracle database objects iot, mviews, clusters and more…
Getting to know oracle database objects iot, mviews, clusters and more…Getting to know oracle database objects iot, mviews, clusters and more…
Getting to know oracle database objects iot, mviews, clusters and more…
 
Optimizing Data Accessin Sq Lserver2005
Optimizing Data Accessin Sq Lserver2005Optimizing Data Accessin Sq Lserver2005
Optimizing Data Accessin Sq Lserver2005
 
Database Performance
Database PerformanceDatabase Performance
Database Performance
 
Partitioning kendralittle
Partitioning kendralittlePartitioning kendralittle
Partitioning kendralittle
 
Pptofdistributeddb
PptofdistributeddbPptofdistributeddb
Pptofdistributeddb
 
Windows azure table storage – deep dive
Windows azure table storage – deep diveWindows azure table storage – deep dive
Windows azure table storage – deep dive
 
Best Practices and Performance Tuning of U-SQL in Azure Data Lake (SQL Konfer...
Best Practices and Performance Tuning of U-SQL in Azure Data Lake (SQL Konfer...Best Practices and Performance Tuning of U-SQL in Azure Data Lake (SQL Konfer...
Best Practices and Performance Tuning of U-SQL in Azure Data Lake (SQL Konfer...
 
MS-SQL SERVER ARCHITECTURE
MS-SQL SERVER ARCHITECTUREMS-SQL SERVER ARCHITECTURE
MS-SQL SERVER ARCHITECTURE
 
初探AWS 平台上的 NoSQL 雲端資料庫服務
初探AWS 平台上的 NoSQL 雲端資料庫服務初探AWS 平台上的 NoSQL 雲端資料庫服務
初探AWS 平台上的 NoSQL 雲端資料庫服務
 
Data warehouse physical design
Data warehouse physical designData warehouse physical design
Data warehouse physical design
 
OVERVIEW OF NEW SQL,COMPARING SQL,NOSQL AND NEWSQL,B.Vinithamani,II-M.sc(Comp...
OVERVIEW OF NEW SQL,COMPARING SQL,NOSQL AND NEWSQL,B.Vinithamani,II-M.sc(Comp...OVERVIEW OF NEW SQL,COMPARING SQL,NOSQL AND NEWSQL,B.Vinithamani,II-M.sc(Comp...
OVERVIEW OF NEW SQL,COMPARING SQL,NOSQL AND NEWSQL,B.Vinithamani,II-M.sc(Comp...
 
Achievements
AchievementsAchievements
Achievements
 
Ms sql server architecture
Ms sql server architectureMs sql server architecture
Ms sql server architecture
 
2 extreme performance - smaller is better
2   extreme performance - smaller is better2   extreme performance - smaller is better
2 extreme performance - smaller is better
 
R2 roadshows
R2 roadshowsR2 roadshows
R2 roadshows
 

Recently uploaded

Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
OnBoard
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
KatiaHIMEUR1
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
Product School
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Ana-Maria Mihalceanu
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
Cheryl Hung
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
Dorra BARTAGUIZ
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Inflectra
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdfFIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Product School
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
Prayukth K V
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
Elena Simperl
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
Product School
 

Recently uploaded (20)

Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
 
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdfFIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdf
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
 

Partitioning Design

  • 1. Partitioning Design For Performance and Maintainability Martin Cairns http://sqlbyparts.com http://twitter.com/MartinCairnsSQL
  • 3. Partitioning Defined Partitioning (noun) 1. the action or state of dividing or being divided into parts 2. a structure dividing a space into two parts Source: http://oxforddictionaries.com/definition/english/partition
  • 4. Types of Partitioning • Horizontal • Partition Table by Rows • Vertical • Partition Table by Columns • File Group • Partition Tables by File Group
  • 5. Horizontal Partitioning  Divides the rows into small sets by boundaries Year PK Order ID Product ID Qty Cost Line XML 2010 1 1 1 5 1.5 <Line… 2010 2 1 1 7 2 <Line… 2011 1000000 20000 345 1000 5 <Line… 2011 1000001 20000 347 3000 12 <Line… 2012 2000000 40000 705 8000 3 <Line… 2012 2000001 40001 706 3000 7 <Line…
  • 6. Vertical Partitioning  Divides columns from one table into multiple tables Year PK Order ID … Year PK Line XML 2010 1 1 … 2010 1 <Line… 2010 2 1 … 2010 2 <Line… 2011 1000000 345 … 2011 1000000 <Line… 2011 1000001 347 … 2011 1000001 <Line… 2012 2000000 705 … 2012 2000000 <Line… 2012 2000001 706 … 2012 2000001 <Line…
  • 7. File Group Partitioning  File group partitioning is separating the storage of tables and indexes onto separate database files  During a Primary File Group restore the whole database will be offline  Separate the system & user tables to allow the quickest restore time  Since the partitioned database is made up of smaller parts it is easier to manage the storage location of the files
  • 8. Using Table Partitioning  Supports Horizontal Partitioning  Partition Function defines boundaries based on a single column  Partition Scheme defines the File Group  Inserts automatically supported  All partitions share the same definition as the table (indexes, columns, fill factor etc)
  • 9. Table Level Limitations  Only ~200 steps for each statistics across the whole table  Online Index rebuilds are for the whole table not individual partitions  Fill Factor  Lock settings (Row, Page, Escalation, etc)  Indexes are defined for the whole table rather than partitions
  • 10. Using View Partitioning  View Partitioning allows you to overcome Table Level Constraints  Check Constraints used to define partitions  Combine all tables with a UNION ALL View  Can be used together with Table Partitioning  Trigger required to allow INSERT with Identity column  Allows more complex partitioning schemes
  • 11. Candidates for Partitioning  Large vs. Small Tables  Replicated vs. Non-Replicated Tables  Normal vs. BLOB columns  Write Heavy Current data vs. Heavily Read Historic data  Read Write vs. Read Only Tables
  • 12. Partial Database Availability  Also referred to as Piecemeal Restore  Full Recovery Model is Required  Individual File & File Groups Restore  Only the Primary File Group is required to restore a database  Allows restoring a subset of a correctly partitioned database for quicker recovery from a disaster
  • 13. Tipping Point Why is it Table Scanning  As the number of rows in a table increase the depth of the B+Tree increases  The number of Page Reads for a lookup is equal to the depth of the B+Tree  The Tipping Point is the point where a Full Table Scan requires less Page Reads than the lookups
  • 14. What Does Partition Give Us  Choices  Using different Tiers of Storage  Allows quicker recovery strategies  Allows partial restores of the database  Performance  Allows reduction of overhead of backups  Allows better query plans due to more accurate statistics on large tables
  • 15. Demos  Moving Tables to File Groups  Partial Restore + Using Partial Restore to Initialise Replication  Moving Partition Between Tables For Current vs. Historic  Show how View Partitioning can present a single view of all tables & still support partition elimination

Editor's Notes

  1. Today I will be talking about partitioning which tends to be thought about after the lack of it has caused a problem.
  2. SQL Server MCM one of roughly 100 in the worldBI Architect/Consultant with 15 years experience of designing large scale and complex database systems
  3. A slightly different definition of partitioning than the marketing releases from MS
  4. Moving out BLOB columns removes SQL 2008 restrictions on Online Index RebuildsInfrequently access columns add overhead to queries which don’t require them, another good reason to avoid SELECT *
  5. It’s best not to separate Tables and their indexes as both will have to restored to use the tables
  6. Fill Factors waste space in the buffer pool and are only required for the active part of a tableRow Locks might be wasteful on the static historic data but required for active data
  7. Grouping the smaller tables away from the large allows a quicker recovery from data lossInitialise Replication from a backupOnline Index Rebuilds in SQL Server 2008 don’t allow LOBsRead-Only file group is the only guaranteed way to make a table read-only, Column Stores will be Read/Write at some point
  8. A single file from a file group can be restored while the database is online as long as it’s not from the Primary File GroupThe transaction log from the earliest file LSN to the current LSN will need to be restored to return the database into a consistent state
  9. Mention Kimberly Trip’s blog on the Tipping Point
  10. Today we will be covering the choices which should be considered when designing a new database and when deal with an existing database which has got new requirements like a higher uptime or scaling to larger volumes of data