Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Part 6: Index Structures Junping Sun Database Systems 6-1 Index Structures for Files Why Index Structures? • Indexes make the search for records based on certain fields, called indexing fields, more efficient. Index Field: • It is usually defined on a single field of a file. Index Entry: • It includes two items or two fields, one is the key (index) field, and another is a pointer to the block which contains the record of that key value. Index File: • Index entries or index records are stored in a file called index file. • The values of indexes in the index file are ordered so that binary search can be applied on the sequential storage of index entries. • The index file size is smaller than the data file, so the searching on index using binary search is reasonably efficient. Classifications of Index Structures: Single Level Index vs. Multi-Level Index: Primary Index, Cluster Index, Secondary Index ISAM, Search Tree, B - Tree, B+ -Tree, B* - Tree Dense vs. Nondense Junping Sun Database Systems Page 1 6-2 Primary Index Primary Index: • It is an ordered file whose records are of fixed length with two fields. • The first field is of the same data type as the ordering key field of the data file. • The second field is a pointer to a disk block - a block address. • The ordering key field is called the primary key of the data file. • There is one index entry or index record in an index file for each block in the data file. The number of index entries is equal to the number of data blocks. Each index entry in an index is a two item tuple. Ex. < K(i), P(i) > K(i) is an index value, and P(i) is the pointer to point a block. Anchor Record or Block Anchor: • The first record in each block of the data file. A record with primary key value K will be in the block whose address is P(k) where K(i) d K d K(i+1), i is the index entry number, also the block number (relative) Junping Sun Database Systems 6-3 Primary Index on the Ordering Key Field Junping Sun Database Systems Page 2 6-4 Primary Index Junping Sun Database Systems 6-5 Search on Data File with Primary Index Search or Retrieve a Record Based on the Primary Key Value: 1. Do a binary search on index file to find the appropriate index entry i, 2. Retrieve the data file block whose address is p(i). Advantages of Primary Index: 1. The index file contains fewer index entries than the records in file. 2. The index entries contains less index fields than records. Nondense Index: • The index file does not contain index entries for every data records of data file. Dense Index: • The index file contains index entries for every data records. • The index file (nodense index) need substantially fewer blocks than the data file. Junping Sun Database Systems Page 3 6-6 Example of Primary Index Example: An ordered file with r=30,000 (record), B=1024 Bytes (block size) File record are of fixed size and unspanned with record length R=100 bytes Block Factor bfr = ¬(B/R)¼ ¬(1024/100)¼ = 10 records per block The number of blocks needed for the file b = ª(r/bfr)º ª(30,000/10)º= 3000 blocks A binary search on the data file would need approximately ªlog2 bº=ªlog2 3000)º= 12 block accesses Junping Sun Database Systems 6-7 Now if the ordering key field of the index file is V= 9 bytes, and a block pointer is P = 6 bytes • The size of each index entry is Ri = (9 + 6) = 15 bytes • The blocking factor for index is bfri = ¬(B/Ri)¼= ¬(1024/15)¼= 68 entries • The number of blocks needed for index is bi = ª(ri/bfri )º= ª(3000/68 )º= 45 • To perform a binary search on the index file would need ª(log2 bi )º= ª(log2 45 )º= 6 block accesses • To search a record using the index, one additional data block access needed total 6 + 1 = 7 block accesses are needed. • The improvement on data access is 5 block access less by using primary index than nonindex method. Junping Sun Database Systems Page 4 6-8 Discussions on Primary Index 1. Insertions and deletion will cause the same problem as in the ordered file. 2. Besides moving records to make space for new inserted record, index entries will also be changed correspondingly records of some blocks. 3. a. b. c. Methods to solve the above problems are use unordered overflow file. use a linked list of overflow records for each block in data file. keep the records within each block and its overflow linked list sorted to improve retrieval time. d. record deletion can be done by deletion mark, and reorganization can be done periodically. Junping Sun Database Systems 6-9 Cluster Index Clustering Field: • If records of a file are physically ordered on a nonkey field that does not have a distinct value for each record, that field called the clustering field. Clustering Index: • A clustering index is also an ordered file with two fields, the first field is of the same type of as the clustering field of the data file, and the second field is block pointer. • There is one index entry in the clustering index for each distinct value of the clustering field, containing that value and a pointer to the first block in the data file that has a record with that value for its clustering field. • It is used to speed up the retrieval of records that have the same value for the clustering field. Insertions and Deletions cause the maintenance problems • Method to solve the problem is to reserve a whole block for each distinct value of the clustering field, all records with the same value are placed in the block, if more than one block is needed to store the records for a particular value, additional blocks are allocated and linked together. Junping Sun Database Systems Page 5 6-10 Clustering Index Junping Sun Database Systems 6-11 Clustering Index Junping Sun Database Systems Page 6 6-12 Secondary Index Secondary Index: • It is an ordered file with two fields The first field is of the same data type as some nonordering field of the data file. The second field is a pointer to a disk block. The field on which the secondary index is constructed is called an index field of the file, whether its values are distinct for every record or not. Many secondary indexes can be built up since many index fields for the same file. Secondary Index on a Key Field (Secondary Key) • The nonordering key field has a distinct value for each record. A secondary index on a key field is a dense index because it contains one entry for each record in the data file. The reason is that the records in data file are not physically ordered by values of the secondary key field. The block anchor technology can not be used. Junping Sun Database Systems 6-13 Secondary Index on Non-Ordering Key Junping Sun Database Systems Page 7 6-14 Discussions on Secondary Index (on Nonordering Key Field) 1. Secondary Index needs more storage space than a primary index because each index entry corresponding to a record. 2. The improvement of search time for an arbitrary record on nonordering key filed is much greater for a secondary index than it is for a primary index, because a linear search on the data file is necessary if the secondary index did not exist. 3. For a primary index, the binary search still can be utilized on the main file even if the index does not exist because the records are physically ordered by the primary key field. Junping Sun 6-15 Database Systems Example Example: Record # r = 30,000 (fixed-length) Block Size B = 1024 bytes Record Size R = 100 bytes Number of Blocks needs is 3000 1. To do the linear search on nonordered key field of a file average b/2 = 3000/2 = 1500 block accesses are needed 2. If nonordering key field of the file that is V = 9 bytes, and a block pointer is P = 6 bytes The size of each index entry is Ri = (9 + 6) = 15 bytes The blocking factor for index is bfri = ¬(B/ Ri)¼= ¬(1024/15)¼= 68 entries The number of blocks needed for index is bi = ª(ri/bfri )º ª(30000/68 )º= 442 blocks (nondense index need 45 blocks) To perform a binary search on the index file would need ª(log2bi )º= ª(log2442 )º= 9 block accesses Total 9 + 1 = 10 block accesses are needed for retrieving a record Junping Sun Database Systems Page 8 6-16 Secondary Index on a Nonkey Field In the case numerous records in the data file can have the same value for the indexing field. A secondary index on a nonkey field of a file can be created. There are several options to implement such an index: Option 1: • To include several index entries with the same K(i) value - one for each record. This is a dense index. Option 2: • To have variable length records for the index entries, with a repeating field for the pointer. • A list of pointers < P(i,1), P(i,2), ... ,P(i,k) > in the index entry for K(i) - one pointer to each block that contains a record whose indexing field value equals K(i). K(i) is the indexing field value for that index entry i. k is the number of repeating field for the pointer, and also the number of records which have the same indexing field values. • Both Option 1 and Option 2 methods need the appropriate modifications of the binary search algorithm on the index. Junping Sun Database Systems 6-17 Option 3: • To keep index entries themselves at a fixed length and have a single entry for each index field value, but create an extra level of indirection to handle the multiple pointers. In this schema: • The pointer P(i) in the index entry < K(i), P(i) > points to a block of record pointers . • Each record pointer in that block (at the indirection level) points to one of the data file records with a value K(i) for the indexing field. • In the case, if some value K(i) has too many records, so that records can not fit in a single disk block, a linked list of blocks can be used. • This technique for retrieval via index needs an additional block access because of the extra level, but the algorithms for searching the index, insertion of new records and deletion of records in the data file are straightforward. • Retrievals on the complex selection conditions may be handled by referring to the pointers without having to retrieve many unnecessary file records. • A secondary index provides a logical ordering on the records by the indexing field. If the access of records is in the order of the entries in the secondary index, the order of records in the indexing field is obtained. This will reduce the sorting time from O(n*n) to O(n). Junping Sun Database Systems Page 9 6-18 Secondary Index on Non-Ordering Non-Key Field Junping Sun 6-19 Database Systems Summary on Types of Indexes Ordering Field Nonordering Field Key Field Primary Index Secondary Index (key) Nonkey Field Clustering Index Secondary Index (Nonkey) Junping Sun Database Systems Page 10 6-20 Properties of Index Types Index Type Number of Index Entries Dense or Nondense Primary Number of blocks in data file Nondense Yes Clustering Number of distinct index field value Nondense Yes/no Secondary (key) Number of records in data file Dense No Secondary (nonkey) Number of records or Number of distinct index field value Dense No Nondense No Junping Sun Database Systems Block Anchor 6-21 Multilevel Indexes Objective: • To try reduce the index block access times by building and using (primary) index on index blocks. Comparison of Search Times on Index Block Part: Single Level Index (log2bi) Multilevel Index (logfobi) • The blocking factor for index is bfri = ¬(B/Ri)¼, which is also called the fan-out of multilevel index, fo. • Searching on multilevel index also requires approximately (logfobi) block accesses, which is less than for the binary search if the fo is larger than 2. Junping Sun Database Systems Page 11 6-22 Multi-Level Index Junping Sun Database Systems 6-23 Structure of a Multilevel-level Primary Index • A multilevel index considers the index file as the first (or base) level of a multilevel index, as an ordered file with a distinct value for each K(i). The primary index is created at the first level. • A primary index for the first level index is called the second level of the multilevel index. • The second level index is a primary index, block anchors can be used so that the second level has one entry for each index block of the first level. • The block factor bfri for the second level and all subsequent levels, is the same as that for the first level index because all index entries are the same size, each having one field value and one block address. • If the first level has r1 entries and the blocking factor (fan-out, or fo), then the first level will need ª(r1/fo)ºblocks, which will be the number of entries r2 needed at the second level of the index. ( for the index, bfri = fo) • The third level, which is a primary index for the second level, has an entry for each secondary-level block, so number of third level entries is r3 = ª(r2/fo)º Junping Sun Database Systems Page 12 6-24 More on Multilevel-level Primary Index In a Multilevel Index: • A second level index is needed if only the first level index uses more than one block of disk storage, and similarly, a third level index only if the second level uses more than one block. • The repeat process of constructing multilevel index will not stop until all the entries of some index level t fit in a single block. • The block at t-th level is called top index level. • Each level reduce number of entries at the previous level by approximately a factor of fo. A multilevel index with r1 first-level entries will have approximately t levels, where t = ª(logfo(r1))º • Multilevel index scheme can be used on any type of index, whether it is a primary, clustering, or secondary index, as long as the first level index has distinct values for K(i) and fixed-length of entries. • One level of indirection blocks for record pointers can be used if the first level does not have the distinct values for K(i). Junping Sun 6-25 Database Systems Example Suppose the previous dense secondary index converted into a multilevel index Record # r = 30,000 (fixed-length) Block Size B = 1024 bytes Record Size R = 100 bytes Number of data blocks needs is 3000 The size of each index entry is Ri = (9 + 6) = 15 bytes The blocking factor for index is bfri = ¬(B/ Ri)¼= ¬ (1024/15) ¼= 68 entries • The number of blocks needed for the first level index is bi = ª(ri/bfri )º ª(30000/68 )º= 442 blocks • The number of the second level index blocks is b2 = ª(b1/fo)º ª(442/68)º = 7 blocks • The number of the third level index block is b3 = ª(b2/fo)º ª(7/68º = 1 block • The third level is top level and t = 3 The total number of block accesses is t + 1 = 3 + 1 = 4 instead of 10 Junping Sun Database Systems Page 13 6-26 Search Tree - Dynamic Multilevel Indexes Tree: • It is formed of nodes Root node is the node with no parent node. Leaf node is the node with no children nodes. Internal node - is a nonleaf node. Parent node is a node with children node. Children node - is a node with parent node. Formal Definition of a Tree: 1. A single node by itself is a tree. This node is also the root of the tree. 2. Suppose n is a node and T1, T2, ... , Tk are trees with roots n1, n2, ... , nk, respectively. We can construct a new tree by making n be the parent of nodes n1, n2, ... , nk. In this tree n is the root and T1, T2, ... , Tk are the subtrees of the root. Nodes n1, n2, ... , nk are called the children of node n. Junping Sun Database Systems 6-27 A Tree Data Structure Junping Sun Database Systems Page 14 6-28 Multilevel Index as Variation of Search Tree • A search tree is a special tree used to guide the search for a record given the value of one of its field. • The multilevel indexes can be thought of as a variation of a search tree data structure. • Each node in the multilevel index has as many as fo pointers and fo key values, where fo is the index fan-out. • The index field values in each node guide us to the next node, until we reach the data file block that contains the required records. • By following a pointer, we restrict our search at each level to a subtree of the search tree and ignore all nodes not in this subtree. Junping Sun Database Systems 6-29 Search Tree Structure Junping Sun Database Systems Page 15 6-30 Search Tree Example Junping Sun Database Systems 6-31 Search Tree Definition • A search tree of order p is a tree such that each node contains at most p-1 search values and p pointers in the order as follow: <P1, K1, P2, K2, ... , Pq-1, Kq-1, Pq>, where qd p, • Each Pi is a pointer to a child node (or a null pointer), and each Ki is a search value from some ordered set of values. • All search values are assumed to be unique. Two constraints must hold at all times on the search tree: • Within each node, K1 < K2 < ... < Kq-1. • For all values of X in the subtree pointed at by Pi , we have Ki-1 < X < Ki for 1 < i < q, X < Ki for i = 1, and Ki-1 < X for i = q Junping Sun Database Systems Page 16 6-32 More on Search Tree Search Field: • The value in the tree can be the values of one of the fields of the file. • It is the same as the index field if a multilevel index guides the search • Each value in the tree is associated with a pointer to the record in the data file having that value. • The pointer could be to the disk block containing that record. • The search tree itself can be stored on disk by assigning each tree node to a disk block. Disadvantages of Search Tree: • Search tree is not guaranteed balanced when there are insertions and deletions. • Record deletion may leave the some nodes of in the tree nearly empty, it causes wasting storage space and unnecessarily increasing the number of levels Junping Sun Database Systems 6-33 B-Tree B-Tree : • It is a search tree with some constraints to solve the problems of search tree. These constraints ensure that • the tree is balanced • space waste is not excessive B-Tree Operation and Maintenance: • algorithms for insertions and deletions become more complex • insertion becomes complicated when inserting a record into a node that is already full. • deletion becomes complicated when a deletion from a node that makes it less than half full. • B-Tree is used as an access structure on a key field to search for records in a data file Junping Sun Database Systems Page 17 6-34 B-Tree Structure Junping Sun Database Systems 6-35 B-Tree Definition 1. Each internal node in the B-Tree of order p is of the form <P1,<K1,Pr1>, P2,<K2,Pr2>, ... , Pq-1,< Kq-1,Prq-1>, Pq> where q dp. • Each Pi is a tree pointer - a pointer to another node in the B-tree. • Each Pri is a data pointer - a pointer to a data block in the data file that contains a record with search key field value equal to Ki. 2. Within each node, K1 < K2 < ... < Kq-1. 3. For all search key field values X in the subtree pointed at by Pi, we have Ki-1 < X < Ki for 1 < i < q, X < Ki for i = 1, and Ki-1 < X for i = q 4. Each node has at most p tree pointers Junping Sun Database Systems Page 18 6-36 5. Each node, except the root and leaf nodes, has at least ª(p/2)ºtreepointers. The root node has at least two tree pointers unless it is the only node in the tree. 6. A node with q tree pointers, q dp, has q-1 search key field values (also q-1 data pointers. 7. All leaf nodes are at the same level. Leaf nodes have the same structure as internal nodes except all their tree pointers are nil. • If B-tree is used on a nonkey field of a file, where numerous records can have the same value for the search field, we have to change the definition of the file pointers. • In this case, rather than point to the data block, each pointer Pri points to a block - or linked list of blocks - that contains pointers to the file records. Junping Sun Database Systems 6-37 Insertions in B-Tree 1. B-tree starts at a single root node (which is also a leaf node, at level 0). 2. Once the root node is full with p-1 search key values and another entry is to be inserted, the root node split into two nodes at level 1. Only the middle value is kept in the root node, and rest of the values are split as evenly as possible and moved to the other two nodes. 3. When a nonroot node is full and a new entry is inserted into it, that node is split two nodes at the same level, and middle entry is moved into the parent node along with two pointers to the split nodes. 4. If the parent node is full, it is also split in the same way. 5. Splitting can propagate all the way to the root node, creating a new level whenever the root is split. Junping Sun Database Systems Page 19 6-38 Deletions in B-Tree 1. If the deletion of a value makes a node less than half full, it may be combined with its neighboring nodes, and this can also propagate all the way to the root. 2. Deletion can cause the reduction of the tree levels. Observation on Insertions and Deletions of B-Tree: • Analysis and simulation show that after performing numerous random insertions and deletions on a B-tree, the nodes are approximately 69% full when the number of values in the tree stabilizes. • If this happens, then node splitting and combining will occur only rarely, so insertion and deletion become quite efficient. Junping Sun Database Systems 6-39 Example Suppose the search field is V=9 bytes long, the disk block size is B=512 bytes, a record (data) pointer is Pr = 7 bytes and a block pointer is P = 6 bytes. Each B-tree node can have at most p tree node pointers, p-1 data (record) pointers, and p-1 search key field values. These must fit into a single disk block if each B-tree node is to correspond to a disk block. Hence we must have (p * P) + ((p -1) * (Pr + V) d B (p*6) + ((p - 1) * (7 + 9)) d512 or (22 * p) d 528 • We can choose p to be the largest value that satisfies the above inequality, which gives p = 23 (instead of 24). • B-tree node may contain additional information needed by the algorithms that manipulate the tree, such as the number of entries q in the node and a pointer to the parent node. Junping Sun Database Systems Page 20 6-40 Example Suppose the search field of the previous example is a nonordering key field, and we construct a B-tree on this field. Assume that each node of the B-tree is 69% full. Each node, on the average, will have p * 0.69 = 23 * 0.69 | 16 pointers and 15 search key fields We can start at the root level and see how many values and pointers exist on the average at each subsequent level. root: 1 node level 1: level 2: level 3: 15 entries 16 nodes 256 nodes 4096 nodes 16 pointers 240 entries 3840 entries 61440 entries 256 pointers 4096 pointers Two level B-tree holds 3840 + 240 + 15 = 4095 entries Three level B-tree holds 61440 + 3840 + 240 + 15 = 65535 entries Junping Sun Database Systems 6-41 If all nodes in B-tree is full One level B-tree will hold 23*22 + 22 = 528 entries Two level B-tree holds 23*23*22 + 528 = 12166 entries Three level B-tree holds 23*23*23*22 + 12166 = 279840 entries Junping Sun Database Systems Page 21 6-42 B+-Tree Difference between B-Tree and B+-Tree: • In a B+-Tree, data pointers are stored only in leaf nodes of the tree, the structure of leaf nodes is different from the internal nodes. • The leaf nodes have an entry for every value of the search field, along with data pointer to the block that contains this record if the search field is key field. • For a nonkey search field, the pointer will point to a block containing pointers to the data file records, creating an extra level of indirection. • The leaf nodes of the B+-Tree are usually linked together to provide ordered access on the search field to the records. These leaf nodes are similar to the first (base ) level of an index. • The internal nodes of the B+-Tree correspond to the other levels of the index in the multilevel index scheme. • Some search field values from the leaf nodes are repeated in the internal nodes of the B+-Tree to guide the search. Junping Sun Database Systems 6-43 B+-Tree Junping Sun Database Systems Page 22 6-44 The Structure of The Internal Nodes of B+Tree 1. Each internal node is of the form <P1, K1, P2, K2, ... , Pq-1, Kq-1, Pq> where q d p (p is the order of B+-Tree) and each Pi is a tree pointer. 2. Within each internal node, K1 < K2 < ... < Kq-1. 3. For all search key field values X in the subtree pointed at by Pi, we have Ki-1 < X dKi for 1 < i < q, X dKi for i = 1, and Ki-1 < X for i = q. 4. Each internal node has at most p pointers. 5. Each internal node, except the root, has at least ª(p/2)º tree pointers. The root node has at least two tree pointers if it is a internal node. 6. An internal node with q pointers, q d p, has q-1 search field values. Junping Sun Database Systems 6-45 The Structure of The Leaf Nodes of B+Tree 1. Each leaf node is of the form <<K1,Pr1>, <K2,Pr2>, ... , < Kq-1,Prq-1>, Pnext>. where q d p, each Pri is a data pointer, and Pnext points to the next leaf node of the B+-Tree. 2. Within each leaf node, K1 < K2 < ... < Kq-1. 3. Each Pri is a data pointer that points to a file block containing the record whose search field value is Ki ( or to a block of record pointers that point to records whose search field value is Ki if the search key field is not a key). 4. Each leaf node has at least ¬(p/2)¼values. 5. All leaf nodes are at the same level. • Both internal nodes and leaf nodes have the same number of pointers and values. Junping Sun Database Systems Page 23 6-46 Differences Between Internal Nodes and Leaf Nodes • Pointers in internal nodes are tree pointers that point to blocks that are tree nodes. • Pointers in leaf nodes are data (record) pointers that point to the data file blocks. Except for Pnext pointer, which is a tree pointer that points to the next leaf node. • Sequential linear search can be done along with the pointer Pnext. The major advantage of the B+-Tree over the B-Tree: • The B+-Tree includes search values and tree pointers without any data pointers at internal node level, more entries can be packed into an internal node of a B+-Tree than for a similar B-Tree. • For the same block size, the order p value will be larger for the B+-Tree than for the B-Tree. This can lead to fewer B+-Tree levels, improving the search time. Junping Sun Database Systems 6-47 Example To Calculate the order p of a B+-Tree, suppose the • search field is V = 9 bytes, the block size B = 512, • a record pointer Pr = 7 bytes, and a block pointer is P = 6 bytes. • An internal node of B+-Tree can have up to p tree pointers and p-1 search field values, these must fit in a single block, hence we have the order p for internal node of B+-Tree (p * P) + ((p - 1) * V d B (p * 6) + ((p - 1) * 9) d512 or (15 * p) d521 • The largest value satisfying the above inequality is p = 34. • This is larger than the value of 23 for the B-tree. • The order of pleaf for the leaf of B+-Tree can be calculated as follows: (pleaf * (Pr + V)) + P d B (pleaf * (7 + 9)) + 6 d 512 (16 * pleaf) d 506 pleaf = 31 key value/data pointer combinations Junping Sun Database Systems Page 24 6-48 Suppose each node is 69% full and we have 34*0.69 approximately 23 pointers and 22 values, and four level B+-Tree will give the following average entries number at each level. root: level 1: level 2: level 3: 1 node 23 nodes 529 nodes 12167 nodes 22 entries 23 pointers 506 entries 529 pointers 11638 entries 12167 pointers 255,507 record pointers • B+-tree can hold 12167 * (31*.69%) = 255,507 record pointers at level 3. Junping Sun Database Systems 6-49 Insertions in B+-Tree Junping Sun Database Systems Page 25 6-50 Insertions in B+-Tree Junping Sun Database Systems 6-51 Insertions in B+-Tree Junping Sun Database Systems Page 26 6-52 Deletions in B+Tree Junping Sun Database Systems 6-53 Deletions in B+Tree Junping Sun Database Systems Page 27 6-54