Access Methods - Lecture 9 - Introduction to Databases (WE-DINF-9552)

2 December 2005Introduction to DatabasesAccess MethodsProf. Beat SignerDepartment of Computer ScienceVrije Universiteit Brusselhttp://www.beatsigner.com

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 2April 26, 2013Context of Todays LectureAccessMethodsSystemBuffersAuthorisationControlIntegrityCheckerCommandProcessorProgramObject CodeDDLCompilerFileManagerBufferManagerRecoveryManagerSchedulerQueryOptimiserTransactionManagerQueryCompilerQueriesCatalogueManagerDMLPreprocessorDatabaseSchemaApplicationProgramsDatabaseManagerDataManagerDBMSProgrammers Users DB AdminsBased on Components of a DBMS, Database Systems,T. Connolly and C. Begg, Addison-Wesley 2010Data, Indices andSystem Catalogue

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 3April 26, 2013Basic Index Concepts An index is used to efficiently access specific data e.g. index in a textbook Often database queries reference only a small number ofrecords in a file indices can be used to enhance the search for these records most DBMSs automatically create an index for primary keys A search key is an attribute (or set of attributes) that isused to lookup records in a file An index file consists of (searchKey, pointer) indexentries normally much smaller than the original file pointer identifies a disk block and a record offset within the block

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 4April 26, 2013Basic Index Concepts ... There are two basic types of indices ordered indices- based on a sorted ordering of search keys hash indices- search keys are uniformely distributed to different buckets based on a hashfunction Indexing techniques have to be evaluated based on access types- e.g. records with a given attribute value or an attribute value in a specific range access time insertion and deletion time- if the underlying data is changed, all indices have to be updated which mayresult in a significant overhead for modifications space overhead

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 5April 26, 2013Ordered Indices An ordered index stores the search key values in sortedorder e.g. author catalogue in a library A primary index (clustering index) is an index whosesearch key also defines the sequential order of the file the primary key is often used as a search key of a primary index files with a clustering index on the search key are calledindex-sequential-files A secondary index (non-clustering index) is an indexwhose search key has a different order than thesequential file order

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 6April 26, 2013Dense Index A dense index contains an entry for each search key inthe file An index record of a dense primary index points to thefirst file record with the given search key An index record of a dense secondary index stores a listof pointers to the records with the same search key valuede RoverFreiJonesMeierMichelin2 de Rover Pieter Brussels1 Frei Urs Zurich8 Frei Urs Zurich14 Jones Andrew Paris53 Meier Beat Zurich5 Michelin Robert Parisdense indexindex-sequential file

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 7April 26, 2013Dense Index Insertionfind the search key of the record to be inserted in the indexif (index contains search key) {if (index stores pointers to all records) {add a pointer to the new record to the index entry}else { // index stores pointer to the first record onlyensure that the record is inserted after all other records with thesame search key}}else { // index does not contain the search keyinsert an index record with the search key at the appropriate position}

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 8April 26, 2013Dense Index DeletionD Update can be modelled as a delete followed by an insertlook up the record to be deletedif (deleted record is the only one with the search key value) {delete index record}else { // there are other records with the same search keyif (index stores pointers to all records) {delete the corresponding pointer from the index record}else { // index stores pointer to the first record onlyif (deleted record was the first record with the search key value) {update the index record to point to the next record}}}

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 9April 26, 2013Sparse Index Contains only some of the search key values can only be used for primary indices! To search a given record in a sparse index the followingsteps have to be performed1) find the index record ri with the largest search key value that issmaller or equal to the search key value to be found2) start a linear search at record ri until the record is foundde RoverJonesMichelin2 de Rover Pieter Brussels1 Frei Urs Zurich8 Frei Urs Zurich14 Jones Andrew Paris53 Meier Beat Zurich5 Michelin Robert Parissparse index

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 10April 26, 2013Sparse Index A sparse index consumes less space and produces lessmaintenance overhead for insertions and deletions On the other hand it takes more time to locate a recordbased on a sparse index A good trade-off is a sparse index with one index recordfor each block...sparse index...block 1block 2

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 11April 26, 2013Sparse Index Insertion// assumption: one index entry for each blockif (new block is created) {insert first search key value in the new block into the index}else { // no new blockif (new record has the smallest search key value in the block) {update the index record pointing to the block}}

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 12April 26, 2013Sparse Index Deletion Update can be modelled as a delete followed by an insertif (index contains the search key value of deleted record) {if (deleted record is the only one with the search key value) {if (search key value of the next record is not in the index {update the index record to point to the next record}else { // search key value of next record is already in the indexdelete index entry}}else { // there are other records with the same search keyif (deleted record was the first record with the search key value) {update the index entry to point to the next record with the samesearch key value}}

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 13April 26, 2013Multilevel Index A multilevel index can be used if theindex grows too large to be kept in memory inner index froms the primary index file outer index is a sparse index on the inner index...inner index...datablock 1datablock 2......outer index

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 14April 26, 2013Multilevel Index ... If even the outer index grows too large to be kept inmemory, the process can be repeated note that instead of this type of multilevel index it is preferred touse B+-trees The update of the multilevel index (on insertion anddeletion) is an extension of the updates described for thesingle level indices

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 15April 26, 2013Secondary Index A secondary index has to be a dense index and mustpoint to all the records with the same search key value e.g. indirection via buckets that contain the pointers The use of a secondary index normally requires moreI/O operations than a primary indexBrusselsParisZurich2 de Rover Pieter Brussels1 Frei Urs Zurich8 Frei Urs Zurich14 Jones Andrew Paris53 Meier Beat Zurich5 Michelin Robert Parissecondary indexbuckets

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 16April 26, 2013Index-Sequential File Problems Index-sequential files have a number of disadvantages degradation and loss of performance when the file grows sincemany overflow blocks have to be created the entire file has to be periodically reorganised Alternative index structures that do not show a loss ofperformance after updates can be used e.g. the B+-tree index file structure is a widely used multilevelindex structure that is based on a B+-tree (balanced and sorted )

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 17April 26, 2013B+-Tree Properties of a balanced B+-tree every path from the root to a leaf node has the same length for a given n, each leaf node has between n-1)/2 and n-1 values- leaf nodes must always be at least half full each non-leaf (and non-root) node has between n/2 and nchildren (fanout) typical B+-tree node- K1,..., Km-1 are the ordered search key values (Ki < Ki+1)- a non-leaf node has pointers P1,..., Pm to child nodes- a leaf node has pointers P1,..., Pm to file records or to buckets of pointers- typically the size of a disk block if the root node is not a leaf node it has at least 2 children;otherwise it has between 0 and n-1 valuesP1 K1 P2 ... Pm-1 Km-1 Pm ...

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 18April 26, 2013B+-Tree Leaf Nodes Leaf node example (n=3) for 1  i < n the pointers Pipoint to a file record with thesearch key value Ki or to a bucketof pointers for search keys that arenon-ordered non-candidate keys the pointer Pn points to the next leaf node in search key orderde Rover Frei2 de Rover Pieter Brussels1 Frei Urs Zurich8 Frei Urs Zurich...next leaf nodeindex-sequential file

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 19April 26, 2013B+-Tree Non-Leaf (Internal) Nodes The non-leaf nodes form a multilevel sparse index on theleaf nodes For a non-leaf node with m pointers the following holds the values of the search keys in the subtree that P1 points to areall smaller than K1 for 2  i < m the search key values in the subtree to which Pipoints to have values greater than or equal to Ki-1 and smallerthan Ki the search keys in the subtree to which Pm points to have valuesgreather than or equal to Km-1P1 K1 P2 ... Pm-1 Km-1 Pm ...

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 20April 26, 2013B+-Tree Example Example B+-tree for n=3 root node must have at least two children non-leaf nodes must have between 2 and 3 children (for n=3) leaf nodes must have between 1 and 2 values (for n=3) note that the node size is normally defined by the block size andthe fanout is typically in the range of 50-100 (e.g. n=100)Bush Frei Jones Meier Otlet PriceJones OtletMeier

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 21April 26, 2013B+-Tree Properties Logically close blocks do not have to be physically closesince the inter-node connections are realised via pointers B+-trees generally have a small height if there are k search key values, the B+-tree tree height is notgreater than log n/2 (k)- e.g. for 5000000 search keys and n = 100 we get a height not greater than 4(note that the height would be 23 for a binary tree) search operations can be performed efficiently (without manyblock accesses)

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 22April 26, 2013B+-Tree Lookup Find all records for a given search key vBush Frei Jones Meier Otlet PriceJones OtletMeierc = root nodewhile (c is not a leaf node) {let Ki = smallest search key value in c greater than vif (Ki exists) {c = node pointed to by Pi

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 23April 26, 2013B+-Tree Lookup ... The lookup requires O(lognk) I/O operations in worst case}else { // no search key greater than vc = node pointed to by Pm // where m is the number of pointers}}if (a key value Ki in c with the value v exists) {pointer Pi leads to the desired record or bucket}else {a record with the given search key value v does not exist}

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 24April 26, 2013B+-Tree Insertion(1)Find the leaf node in which the search key value wouldappear using the lookup algorithm presented before andadd the new record to the file(2)If the search key is already present in the leaf node if necessary add a new pointer to the bucket(3)If the search key value is not yet present if there is room in the leaf node then insert a new index record if there is no room then split the node- take the n index records (inlcuding the one to be inserted) in sorted order andplace the first n/2 records in the original node and the rest in a new node- let the new node be p and the smallest value in p be k. Insert (p,k) in the parentnode (if there is no space in the parent node, the split is propagated upwards)

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 25April 26, 2013B+-Tree Insertion ... In worst case, the root node has to be split and theheight of the tree is increased by 1 The insertion requires O(lognk) I/O operations in worstcase

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 26April 26, 2013B+-Tree Insertion Example insertion of CoddBush Frei Jones Meier Otlet PriceJones OtletMeierBush Codd Meier Otlet PriceFrei Jones OtletMeierJonesFrei

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 27April 26, 2013B+-Tree Deletion(1)Find the record to be deleted by using the lookupalgorithm and delete it from the file(2)Remove the search key and value from the leaf indexnode if there is no bucket or if the bucket is empty(3)If the node has no longer enough entries due to thedeleted index record and the entries in the node and asibling fit into a single node then merge the siblings insert the search key values of the two nodes into the left nodeand delete the right node recursively delete the entry for the deleted node from the parentnode (propagation)

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 28April 26, 2013B+-Tree Deletion ...(4)If the node has no longer enough entries due to thedeleted index record but the entries in the node and asibling do not fit into a single node then redistribute thepointers redistribute the pointers between the node and a sibling such thatboth have more than the minimum number of entries update the search key values in the parent node If the root node has only one pointer after deletion, it isdeleted and its child becomes the root The deletion of a previously located record requiresO(lognk) I/O operations in worst case note that there are variants where leaf nodes are not merged

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 29April 26, 2013B+-Tree Deletion Example deletion of FreiBush Codd Meier Otlet PriceFrei Jones OtletMeierJonesFreiBush Codd Meier Otlet PriceJones OtletMeierJones

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 30April 26, 2013B+-Tree Deletion Example ... deletion of MeierBush Codd Otlet PriceJonesBush Codd Meier Otlet PriceJones OtletMeierJonesJones Otlet

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 31April 26, 2013B+-Tree Deletion Example ... deletion of MeierBush Codd Meier Otlet PriceFrei Jones OtletMeierJonesFreiBush Codd Otlet PriceFrei OtletJonesJonesFrei

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 32April 26, 2013Features of B+-Tree Index Files Advantages automatic reorganisation with small local changes for insertionsand deletions no reorganisation of the file is required to maintain theperformance (solves index-sequential file problem) Disadvantages insertion and deletion overhead space overhead The advantages of B+-trees outweigh the disadvantageswhich makes them an extensively used data structure

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 33April 26, 2013B+-Tree File Organisation We can use the same B+-tree structure to organise therecords in a file and thereby solve the problem of datafile degradation In a B+-tree file organisation, the leaves of the tree storerecords instead of pointers to records number of records in the leaf nodes normally smaller than thenumber of pointers on non-leave nodes leaf nodes still have to be at least half full two siblings can be used in splits/merges for space optimisation adjacent nodes in the tree may not be continuous on the disk

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 34April 26, 2013Indexing Strings The use of strings as search keys introduces two majorproblems the strings can be of variable length which leads to differentfanouts the split and merge operations should no longer be based on thenumber of search keys but on the fraction of the used space The fanout of nodes can be increased by using a prefixcompression technique we no longer store the entire search key in non-leaf nodes butonly the part that is necessary to distinguish between the keyvalues in the subtrees

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 35April 26, 2013B+-Trees vs. B-Trees B-trees store every search key value only once(no redundancy) additional pointers to the file records have to be added for non-leaf nodes Advantages of B-trees may use less nodes than the corresponding B+-trees may find the search key before reaching the leaf node Disadvantages of B-trees non-leaf nodes get larger which results in a deeper trees insertion and deletion gets more complicated Normally the advantages of B-trees do not outweigh theirdisadvantages

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 36April 26, 2013Multiple-Key Access For queries that need fast search on the combination ofmultiple attributes we can define composite search keys e.g. search key (name, city) for customers The ordering of composite search keys is thelexicographic ordering e.g. (a1, a2) < (b1, b2) if- a1 < b1, or- a1 = b1 and a2 < b2 An index with composite search keys can be used toefficiently answer queries of the form WHERE name = Max Frisch AND city = Zurich WHERE name = Max Frisch AND city < Brussels

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 37April 26, 2013Static Hashing A bucket is a storage unit containing one or more records typically a disk block In a hash file organisation we get the bucket in a file byusing a hash function on the search key value A hash function h is a function from the set of search keyvalues K to the set of buckets B distribution should be uniform and random The hash function can be used to locate, insert anddelete records A bucket may contain records with different search keys bucket has to be sequentially searched

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 38April 26, 2013Hash File Organisation Example example of a hash function: binary representation of search keymodulo 614 Jones Andrew Paris2 Rover Pieter Brussels53 Meier Beat Zurich1 Frei Urs Zurich8 Frei Urs Zurich5 Michelin Robert Parisbucket 0 bucket 1bucket 2 bucket 3bucket 4 bucket 5

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 39April 26, 2013Bucket Overflows Buckets can overflow due to different reasons insufficent number of buckets a skew in the distribution of records- many records with the same search key- non-uniform hash function The possibility for bucket overflows can be reduced butwe cannot prevent overflows bucket overflows are handled by a linked list of additionaloverflow buckets this form of hashing with overflow buckets is called closedhashing Open hashing does not allow overflow buckets other existing buckets have to be "misused" (linear probing)

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 40April 26, 2013Hash Index Hashing cannot only be used for file organisation butalso for the creation of an index A hash index manages the search keys with their recordpointers in a hash file structure

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 41April 26, 2013153Hash Index Example2 de Rover Pieter Brussels1 Frei Urs Zurich8 Frei Urs Zurich14 Jones Andrew Paris53 Meier Beat Zurich5 Michelin Robert Paris5bucket 0bucket 114bucket 22bucket 38overflow bucket

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 42April 26, 2013Static Hashing Problems The hash function maps the search key values to a fixednumber of buckets B but the database will grow or shrinkover time if the file grows and the initial number of buckets is too small, theperformance decreases since overflow buckets are necessary if space is preallocated for future growth, some space will bewasted initially A possible solution would be to reorganise the file with anew hash function from time to time too expensive and has to be performed exclusively An alternative is to dynamically modify the number ofbuckets (dynamic hashing)

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 43April 26, 2013Ordered Indexing vs. Hashing The database designer has to choose which form ofindex should be used based on different criteria cost of periodic reorganisation of the index frequency of update operations optimised average or worst case access time expected types of queries- hashing is generally better for retrieving records with a specific attribute value• constant average lookup time• worst case lookup time proportional to the number of attribute values!- ordered indices are preferred for range queries where the attribute value lieswithin a range of values (e.g. WHERE A > C1 AND A < C2) In practice most DBMSs support an ordered B+-tree index but not all of themsupport hash indices

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 44April 26, 2013Bitmap Index A bitmap index can be used to efficiently query onmultiple keys/attributes The following assumptions must hold records in a relation must be numbered sequentially (e.g. startingfrom 0) given a number i, it must be easy to find the ith record- easy if records have fixed size Bitmaps are applicable on attributes with a relativelysmall number of distinct values e.g. gender, city, ... A bitmap is an array with as many bits as records the ith bit is 1 if record i has the desired attribute value

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 45April 26, 2013Bitmap Index ... Queries can be answered by using bitmap operations and (intersection), or (union) and not (complementation)- e.g. find all males in Zurich: 101101 AND 011010 = 001000- e.g. find females not living in Paris: 010010 AND 111010 = 010010 Retrieve records based on the resulting bit array can also be used to count the number of records with a given condition2 de Rover Pieter m Brussels21 Brown Kathy f Zurich8 Frei Urs m Zurich14 Jones Andrew m Paris9 Jones Lea f Zurich5 Michelin Robert m Paris101101010010100000000101011010mfBrusselParisZurichbitmaps for genderbitmaps for location

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 46April 26, 2013Index Definition in SQL The create index command can be used to create anindex on a relation different DBMSs support different types of indices (e.g. B+-tree,hash index, etc.) ExampleCREATE INDEX nameIndex ON Customer USING BTREE (name);

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 47April 26, 2013Summary Different types of indices can be used to speed up thesearch of specific records in a file The number and types of indices have to be designedcarefully since each index adds additional maintenancecosts on update operations The structures that are used for the indices (e.g. B+-treeor hash index) can also be used to manage the recordsin a file and reduce data file degradation

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 48April 26, 2013Homework Study the following chapter of theDatabase System Concepts book chapter 11- sections 11.1-11.11- Indexing and Hashing

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 49April 26, 2013Exercise 9 Functional Dependencies and Normalisation

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 50April 26, 2013References A. Silberschatz, H. Korth and S. Sudarshan,Database System Concepts (Sixth Edition),McGraw-Hill, 2010

2 December 2005Next LectureQuery Processing and Optimisation

Access Methods - Lecture 9 - Introduction to Databases (WE-DINF-9552)

by Beat Signer

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 17

Statistics

Access Methods - Lecture 9 - Introduction to Databases (WE-DINF-9552) Presentation Transcript

http://www.inf.ethz.ch	7
http://www.slideshare.net	5
http://wise.vub.ac.be	5