Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Processing Data in External Storage CS 302 - Data Structures Mehmet H Gunes Modified from authors’ slides Contents • A Look at External Storage • Sorting Data in an External File • External Dictionaries Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 A Look at External Storage • External storage exists after program execution • Generally, there is more external storage than internal memory • Direct access files are essential for external dictionaries • A file contains records that are organized into blocks Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 A Look at External Storage • A file partitioned into blocks of records Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 A Look at External Storage • Direct access input and output involves blocks instead of records • A buffer stores data temporarily Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 A Look at External Storage • Updating a portion of a record within a block Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 A Look at External Storage • Time required to read or write a block of data – Longer than the time required to operate on the block’s data – Implication: reduce the number of required block accesses • File access time dominant factor in an algorithm’s efficiency Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 Sorting Data in an External File Example Problem • Fille contains 1,600 employee records • Sorted by Social Security number • Each block contains 100 records • Program can access only enough internal memory to manipulate 3 blocks at a time Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 Sorting Data in an External File • (a) Sixteen sorted runs, one block each, in file F1 ; (b) Eight sorted runs, two blocks each, in file F2 ; Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 Sorting Data in an External File • (c) Four sorted runs, four blocks each, in file F1 ; (d) Two sorted runs, eight blocks each, in file F2 Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 Sorting Data in an External File • (a) Merging single blocks; (b) merging long runs Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 Sorting Data in an External File • Algorithm for merging arbitrary-sized sorted runs Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 External Dictionaries • Records in order by search key • Algorithm to traverse file in sorted order • Retrieval could be done with binary search Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 External Dictionaries • Shifting across block boundaries Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 Indexing an External File Benefits of indexing with catalog 1.Index record will be much smaller than a data record 2.Do not need to maintain the data file in any particular order, insert new records in any convenient location 3.Maintain several indexes simultaneously Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 Indexing an External File • A data file with an index Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 Indexing an External File • A data file with a sorted index file Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 External Hashing • Similar to the internal scheme described in Chapter 18 • Hash the index file instead of data file • Hash table — contains a pointer to beginning of chain of items that hash into that location Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 External Hashing • A hashed index file Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 External Hashing • A single block with a pointer Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 External Hashing Insertion under external hashing 1.Insert the data record into the data file 2.Insert a corresponding index record into the index file Removal under external hashing 1.Search index file for corresponding index record 2.Remove data record from the data file Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 B-Trees • (a) Blocks organized into a 2-3 tree; Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 B-Trees • (b) a single node of the 2-3 tree Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 B-Trees • (a) A node with two children; (b) a node with three children; (c) a node with m children Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 B-Trees • (a) A full tree whose internal nodes have five children; (b) the format of a single node Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 B-Trees • A B-tree of degree 5 Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 B-Trees Insertion into a B-tree 1.Insert the data record into the data file 2.Insert a corresponding index record into the index file Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 B-Trees • (a through d) The steps for inserting 55 Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 B-Trees • (a through d) The steps for inserting 55 Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 B-Trees • (e) splitting the root Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 B-Trees Removal from a B-tree 1.Locate the index record in the index file 2.Remove the data record from the data file Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 B-Trees • (a through e) The steps for removing 73 ; Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 B-Trees • (a through e) The steps for removing 73 ; Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 B-Trees • (a through e) The steps for removing 73 ; Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 B-Trees • (a through e) The steps for removing 73 ; Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 B-Trees • (f) removing the root; Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 Traversals • Accessing only the search key of each record, but not the data file Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 Traversals • Accessing data file also Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013 Multiple Indexing • Multiple index files Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013