Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
I/O-Algorithms Lars Arge Aarhus University February 9, 2006 I/O-algorithms Random Access Machine Model R A M • Standard theoretical model of computation: – Infinite memory – Uniform access cost Lars Arge 2 I/O-algorithms Hierarchical Memory L 1 L 2 R A M • Modern machines have complicated memory hierarchy – Levels get larger and slower further away from CPU – Levels have different associativity and replacement strategies – Large access time amortized using block transfer between levels • Bottleneck often transfers between largest memory levels in use Lars Arge 3 I/O-algorithms I/O-Bottleneck • I/O is often bottleneck when handling massive datasets – Disk access is 106 times slower than main memory access – Large transfer block size (typically 8-16 Kbytes) track read/write head read/write arm magnetic surface • Important to obtain “locality of reference” – Need to store and access data to take advantage of blocks Lars Arge 4 I/O-algorithms Massive Data • Massive datasets are being collected everywhere • Storage management software is billion-$ industry Examples: • Phone: AT&T 20TB phone call database, wireless tracking • Consumer: WalMart 70TB database, buying patterns • WEB: Web crawl of 200M pages and 2000M links, Akamai stores 7 billion clicks per day • Geography: NASA satellites generate 1.2TB per day Lars Arge 5 I/O-algorithms I/O-Model D Block I/O M • Parameters N = # elements in problem instance B = # elements that fits in disk block M = # elements that fits in main memory K = # output size in searching problem • We often assume that M>B2 P Lars Arge • I/O: Movement of block between memory and disk 6 I/O-algorithms Fundamental Bounds • • • • Scanning: Sorting: Permuting Searching: Internal N N log N N log 2 N External N B N B log M B NB min{ N , NB log M B log B N N } B • Note: – Linear I/O: O(N/B) – Permuting not linear – Permuting and sorting bounds are equal in all practical cases – B factor VERY important: NB NB log M B NB N – Cannot sort optimally with search tree Lars Arge 7 I/O-algorithms Merge Sort • Merge sort: – Create N/M memory sized sorted runs – Merge runs together M/B at a time O(log M B N M ) phases using O( N B) I/Os each • Distribution sort similar (but harder – partition elements) Lars Arge 8 I/O-algorithms External Search Trees • Binary search tree: – Standard method for search among N elements – We assume elements in leaves (log 2 N ) – Search traces at least one root-leaf path – If nodes stored arbitrarily on disk Search in O(log 2 N ) I/Os Rangesearch in O(log 2 N T ) I/Os Lars Arge 9 I/O-algorithms External Search Trees (log 2 B) (B) • BFS blocking: – Block height O(log 2 N ) / O(log 2 B) O(log B N ) – Output elements blocked Rangesearch in (log B N T B) I/Os • Optimal: O(N/B) space and (log B N T B) query Lars Arge 10 I/O-algorithms External Search Trees • Maintaining BFS blocking during updates? – Balance normally maintained in search trees using rotations x y y x • Seems very difficult to maintain BFS blocking during rotation – Also need to make sure output (leaves) is blocked! Lars Arge 11 I/O-algorithms B-trees • BFS-blocking naturally corresponds to tree with fan-out (B) • B-trees balanced by allowing node degree to vary – Rebalancing performed by splitting and merging nodes Lars Arge 12 I/O-algorithms (a,b)-tree • T is an (a,b)-tree (a≥2 and b≥2a-1) – All leaves on the same level (contain between a and b elements) – Except for the root, all nodes have degree between a and b – Root has degree between 2 and b (2,4)-tree • (a,b)-tree uses linear space and has height O(log a N ) Choosing a,b = (B) each node/leaf stored in one disk block (N/B) space and (log B N T B) query Lars Arge 13 I/O-algorithms (a,b)-Tree Insert • Insert: Search and insert element in leaf v DO v has b+1 elements/children Split v: make nodes v’ and v’’ with b21 b and b21 a elements insert element (ref) in parent(v) (make new root if necessary) v=parent(v) v b 1 v’ v’’ b21 b21 • Insert touch (log a N ) nodes Lars Arge 14 I/O-algorithms (a,b)-Tree Insert Lars Arge 15 I/O-algorithms (a,b)-Tree Delete • Delete: Search and delete element from leaf v DO v has a-1 elements/children Fuse v with sibling v’: move children of v’ to v delete element (ref) from parent(v) (delete root if necessary) If v has >b (and ≤ a+b-1<2b) children split v v=parent(v) v a -1 v 2a - 1 • Delete touch O(log a N ) nodes Lars Arge 16 I/O-algorithms (a,b)-Tree Delete Lars Arge 17 I/O-algorithms (a,b)-Tree • (a,b)-tree properties: – If b=2a-1 one update can cause many rebalancing operations (2,3)-tree insert delete – If b≥2a update only cause O(1) rebalancing operations amortized – If b>2a O( b 1-a ) O( 1 a) rebalancing operations amortized 2 * Both somewhat hard to show – If b=4a easy to show that update causes O( 1a log a N ) rebalance operations amortized * After split during insert a leaf contains 4a/2=2a elements * After fuse during delete a leaf contains between 2a and 5a elements (split if more than 3a between 3/2a and 5/2a) Lars Arge 18 I/O-algorithms Summary/Conclusion: B-tree • B-trees: (a,b)-trees with a,b = (B) – O(N/B) space – O(logB N+T/B) query – O(logB N) update • B-trees with elements in the leaves sometimes called B+-tree • Construction in O( NB log M B NB ) I/Os – Sort elements and construct leaves – Build tree level-by-level bottom-up Lars Arge 19 I/O-algorithms Summary/Conclusion: B-tree • B-tree with branching parameter b and leaf parameter k (b,k≥8) – All leaves on same level and contain between 1/4k and k elements – Except for the root, all nodes have degree between 1/4b and b – Root has degree between 2 and b • B-tree with leaf parameter k (B) – O(N/B) space – Height O(log b NB ) – O( 1 k ) amortized leaf rebalance operations – O( b1k log b NB ) amortized internal node rebalance operations • B-tree with branching parameter Bc, 0<c≤1, and leaf parameter B – Space O(N/B), updates O(log B N ), queries O(log B N T B) Lars Arge 20 I/O-algorithms Secondary Structures • When secondary structures used, a rebalance on v often require O(w(v)) I/Os (w(v) is weight of v) – If (w(v)) inserts have to be made below v between operations O(1) amortized split bound O(log B N ) amortized insert bound • Nodes in standard B-tree do not have this property (2,4)-tree Lars Arge 21 I/O-algorithms BB[]-tree • In internal memory BB[]-trees have the desired property • Defined using weight-constraints – Ratio between weight of left child an weight of right child of a node v is between and 1- Height O(log N) • If 211 1 - 1 2 2 rebalancing can be performed using rotations x y y x • Seems hard to implement BB[]-trees I/O-efficiently Lars Arge 22 I/O-algorithms Weight-balanced B-tree • Idea: Combination of B-tree and BB[]-tree – Weight constraint on nodes instead of degree constraint – Rebalancing performed using split/fuse as in B-tree • Weight-balanced B-tree with parameters b and k (b>8, k≥8) – All leaves on same level and contain between k/4 and k elements l 1 l b k ... b k 4 – Internal node v at level l has w(v) < b l k l -1 1 l -1 b k ... b k 4 – Except for the root, internal node v at level l have w(v)> 14 b l k – The root has more than one child Lars Arge level l level l-1 23 I/O-algorithms Weight-balanced B-tree • Every internal node has degree between l -1 l 1 l 1 1 l -1 b k / b k b b k / b k 4b and 4 4 4 Height O (log b 1 l b k...b l k 4 N ) k 1 l -1 b k...b l -1k 4 level l level l-1 • External memory: – Choose 4b=B (or even Bc for 0 < c ≤ 1) – k=B O(N/B) space, O(log B N T B) query Lars Arge 24 I/O-algorithms Weight-balanced B-tree Insert • Search for relevant leaf u and insert new element • Traverse path from u to root: – If level l node v now has w(v)=blk+1 l 1 1 l 1 b k ... b k 4 then split into nodes v’ and v’’ with w(v' ) 12 (b l k 1) - b l -1k and bl k 1 w(v' ' ) 12 (b l k 1) b l -1k 1 l -1 b k...b l -1k 4 l -1 l • Algorithm correct since b k 18 b k such that w(v' ) 83 b l k and w(v' ' ) 85 b l k – touch O (log b Nk ) nodes • Weight-balance property: – (b l k ) updates below v’ and v’’ before next rebalance operation Lars Arge 25 I/O-algorithms Weight-balanced B-tree Delete • Search for relevant leaf u and delete element • Traverse path from u to root: – If level l node v now has w(v) 14 b l k - 1 then fuse with sibling into node v’ with 24 bl k - 1 w(v' ) 54 bl k - 1 – If now w(v' ) 78 b l k then split into nodes 7 l 5 l b k - 1 - b l -1k 16 b k -1 with weight 16 and 85 b l k b l -1k 86 b l k 1 l 1 b k...b l 1k 4 1 l bk 4 -1 l -1 1 l -1 b k ... b k 4 • Algorithm correct and touch O (log b Nk ) nodes • Weight-balance property: – (b l k ) updates below v’ and v’’ before next rebalance operation Lars Arge 26 I/O-algorithms Summary/Conclusion: Weight-balanced B-tree • Weight-balanced B-tree with branching parameter b and leaf parameter k=Ω(B) – O(N/B) space – Height O (log b Nk ) – O(log b N ) rebalancing operations after update – Ω(w(v)) updates below v between consecutive operations on v • Weight-balanced B-tree with branching parameter Bc and leaf parameter B – Updates in O(log B N ) and queries in O(log B N T B) I/Os • Construction bottom-up in O( NB log M B Lars Arge N ) B I/O 27 I/O-algorithms Persistent B-tree • In some applications we are interested in being able to access previous versions of data structure – Databases – Geometric data structures (later) • Partial persistence: – Update current version (getting new version) – Query all versions • We would like to have partial persistent B-tree with – O(N/B) space – N is number of updates performed – O(log B N ) update – O(log B N T B) query in any version Lars Arge 28 I/O-algorithms Persistent B-tree • East way to make B-tree partial persistent – Copy structure at each operation – Maintain “version-access” structure (B-tree) update i i+1 i+2 i i+1 i+2 i+3 • Good O(log B N T B) query in any version, but – O(N/B) I/O update – O(N2/B) space Lars Arge 29 I/O-algorithms Persistent B-tree • Idea: Elements augmented with “existence interval” and stored in one structure • Persistent B-tree with parameter b (>16): – Directed graph * Nodes contain elements augmented with existence interval * At any time t, nodes with elements alive at time t form B-tree with leaf and branching parameter b – B-tree with leaf and branching parameter b on indegree 0 node If b=B: – Query at any time t in O(log B N T B) I/Os Lars Arge 30 I/O-algorithms Persistent B-tree: Updates • Updates performed as in B-tree • To obtain linear space we maintain new-node invariant: – New node contains between 3 8 B and 7 8 B alive elements and no dead elements 1 4 Lars Arge 1 8 B B 3 8 1 2 B 1 8 B 7 8 B B B 31 I/O-algorithms Persistent B-tree Insert • Search for relevant leaf u and insert new element • If u contains B+1 elements: Block overflow – Version split: Mark u dead and create new node u’ with x alive element – If x 7 8 B: Strong overflow – If x 3 8 B: Strong underflow – If 3 8 B x 7 8 B then recursively update parent(l): Delete reference to u and insert reference to u’ 1 4 Lars Arge B 3 8 B 7 8 B B 1 4 B 3 8 B 7 8 B B 32 I/O-algorithms Persistent B-tree Insert • Strong overflow ( x 7 8 B) – Split v into u’ and u’’ with x 2 elements each ( 3 8 B x 2 – Recursively update parent(u): Delete reference to l and insert reference to v’ and v’’ 1 4 B 3 8 B 171 B 33B B BB 484 B 88 771 BB 884 B B83BB 1 2 B) 7 8 B B • Strong underflow ( x 3 8 B) – Merge x elements with y live elements obtained by version split on sibling ( 1 2 B x y 118 B) – If x y 7 8 B then (strong overflow) perform split into nodes with (x+y)/2 elements each ( 716 B ( x y) / 2 1116 B ) – Recursively update parent(u): Delete two insert one/two references Lars Arge 33 I/O-algorithms Persistent B-tree Delete • Search for relevant leaf u and mark element dead • If u contains x 1 4 B alive elements: Block underflow – Version split: Mark u dead and create new node u’ with x alive element – Strong underflow ( x 3 8 B ): Merge (version split) and possibly split (strong overflow) – Recursively update parent(u): Delete two references insert one or two references 1 4 Lars Arge 1 8 B B 3 8 1 2 B 1 8 B 7 8 B B B 34 I/O-algorithms Persistent B-tree Insert Delete done Block underflow 0,0 Block overflow Version split done Version split Strong overflow Strong underflow Split Merge -1,+1 Strong overflow done -1,+2 1 4 Lars Arge 1 8 B B 3 8 1 2 B 1 8 B 7 8 B B B done -2,+1 Split done -2,+2 35 I/O-algorithms Persistent B-tree Analysis • Update: O(log B N ) – Search and “rebalance” on one root-leaf path • Space: O(N/B) – At least 18 B updates in leaf in existence interval – When leaf u dies * At most two other nodes are created * At most one block over/underflow one level up (in parent(l)) 1 1 1 B B B – During N updates we create: 8 8 2 * O( N B) leaves 7 * O( N Bi ) nodes i levels up 1 B B B 83 B 8 4 O( N B i ) O( N B ) blocks i Lars Arge 36 I/O-algorithms Summary/Conclusion: Persistent B-tree • Persistent B-tree – Update current version – Query all versions • Efficient implementation obtained using existence intervals – Standard technique • During N operations – O(N/B) space – O(log B N ) update – O(log B N T B) query Lars Arge 37 I/O-algorithms Other B-tree Variants • Level-balanced B-trees – Global instead of local balancing strategy – Whole subtrees rebuilt when too many nodes on a level – Used when parent pointers and divide/merge operations needed • String B-trees – Used to maintain and search (variable length) strings Lars Arge 38 I/O-algorithms B-tree Construction • In internal memory we can sort N elements in O(N log N) time using a balanced search tree: – Insert all elements one-by-one (construct tree) – Output in sorted order using in-order traversal • Same algorithm using B-tree use O( N log B N ) I/Os log MB – A factor of O( B log B ) non-optimal • As discussed we could build B-tree bottom-up in O( NB log M B NB ) I/Os – But what about persistent B-tree? – In general we would like to have dynamic data structure to use in O( NB log M B NB ) algorithms O( B1 log M B NB ) I/O operations Lars Arge 39 I/O-algorithms References • External Memory Geometric Data Structures Lecture notes by Lars Arge. – Section 1-4 Lars Arge 40