This document discusses techniques for managing very large databases (VLDBs). It begins by defining what constitutes a VLDB and how that definition changes over time as database sizes increase. It then discusses some of the challenges of working with VLDBs, such as long load, update, and indexing times. The document proposes techniques for addressing these challenges, including database partitioning, which breaks large tables into smaller, more manageable partitions, and data compression techniques to reduce the physical size of databases. It provides examples of how partitioning and compression can be implemented in popular database systems like SQL Server and Oracle. The overall aim is to discuss strategies for optimizing performance and maintenance of extremely large databases.

1. In The Name Of God
large
1
Kazem Taghandiky
University of Isfahan
Taghandiky@gmail.com
Mobile:09335816144

2. 1. Introduction
A very large database, or VLDB, is a database that contains an extremely high
number of tuples (database rows), or occupies an extremely large physical file
system storage space. The most common definition of VLDB is a database that
occupies more than 1 terabyte or contains several billion rows, although naturally
this definition changes over time
If I updated this list in 3-5 years, all of the fixed measurements would probably increase
by a factor of 10X to 100X, maybe more. Probably the most relevant description of a
“Large Database” is one that is causing you pain and forcing you to consider other
The definition of large database is always changing,
options
2
>>>See More

3. 1. Introduction
Importan
ce
VLDB is a premier annual international forum for data management and database researchers, vendors, practitioners, application
developers, and users. The conference will feature research talks, tutorials, demonstrations, and workshops. It will cover current
issues in data management, database and information systems research. Data management and databases remain among the main
technological cornerstones of emerging applications of the twenty-first century
3

4. 1. Introduction
The Factors that Define a
Large Database
Hardware
Throughput
Data volume
Software
4
>>>See More

5. 2. DBMS vs. VLDB
5

6. 2-1. Examples of VLDB
6

7. 3. Challenges
Some of the top
challenges I hear:
 Loading a big database for a takes a long time.
 Adding rows to an existing database is slow.
 Adding or removing a column takes my table offline
for a long time.
 Adding an index takes my table offline long time.
 Backup is difficult. (Not in this talk.)
These activities are painless for small data, but can be
painful for big data.
7

8. 3-1. Techniques for challenge
Partitioning
1.Partitioning addresses key issues in supporting very large tables and indexes
by letting you decompose them into smaller components called partitions. SQL
queries and DML statements do not need to be modified in order to access
partitioned tables.
Each partition of a table or index must have the same logical attributes, such as
column names,
1.Data types, and
2.constraints,
8
but each partition can have separate physical attributes such as table spaces.

9. 3-1. Techniques for challenge
Overview of Partitioning Methods
Range partitioning divides a table into partitions based on a range of values. You can
specification for the partitions.
9

10. 3-1. Techniques for challenge
Overview of Partitioning Methods
List Partitioning this by specifying a list of discrete values for the partitioning key in
You do
the description for each partition
10

11. 3-1. Techniques for challenge
Overview of Partitioning Methods
HASH partitioning With this type of partitioning, a partition is selected based on the v
expression that operates on column values in rows to be inserted into the table
11

12. 3-1. Techniques for challenge
SQL Server :
Oracle :
Tables can be partitioned into up to 64,000 separate partit
12

13. 3-1. Techniques for challenge
Table partitioning delivers advantages in three main areas
• performance
• maintenance
• availability.
13
>>>See More

14. 3-1. Techniques for challenge
Data Compression
You can use the data compression feature to help compress the data inside a
database, and to help reduce the size of the database. In addition to saving space,
data compression can help improve performance of I/O intensive workloads
because the data is stored in fewer pages and queries need to read fewer pages
from disk. However, extra CPU resources are required on the database server to
compress and decompress the data,
14

15. 3-1. Techniques for challenge
Implementation
Row Compression
•It reduces the metadata overhead that is associated with the
record. This metadata is information about columns, their lengths
and offsets. In some cases, the metadata overhead might be larger
than the old storage format.
•It uses variable-length storage format for numeric types (for
example integer, decimal, and float) and the types that are
based on numeric (for example date time and money).
•It stores fixed character strings by using variable-length format by
not storing the blank characters
15

16. 3-1. Techniques for challenge
Implementation
Page Compression
Compressing the leaf level of tables and indexes with page
compression consists of three operations in the following order:
1.Row compression
2.Prefix compression
3.Dictionary compression
16

17. 3-1. Techniques for challenge
17

18. 4. Practical
18
>>>See More

19. 4. Practical
19
>>>See More

20. 4. Practical
20
>>>See More

21. Thank You For Your Attention
21