Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Data Structures and Algorithms in Parallel Computing Lecture 2 Parallel algorithmic techniques • Techniques used in parallel algorithm design • Some are extensions of sequential algorithms’ design approaches • Others are specific to parallel algorithms • Methods: – Divide-and-conquer – Randomization – Parallel pointer manipulation Divide-and-conquer • Split problem in subproblems – Easier to solve than original problem • Merge solutions to construct global solution • Improves program modularity • May lead to simple but efficient algorithms • Powerful tool in sequential algorithm design Parallel divide-and-conquer • Parallelize the divide and merge steps • A trivial example is the MapReduce model • Independent map jobs followed by one or more reduce jobs • Examples: – Computational geometry – Sorting – Fast Fourier Transforms Example: mergesort • Takes a sequence of n keys and returns the ordered list • Split sequence in two sequences of n/2 keys • Recursively sort the two sequences • Merge the two sorted n/2 keys sequence • O(n) time complexity • Runtime Parallel mergesort • Recursive calls are independent Complexity analysis • Hence the complexity is W(n)/D(n) = O(n log n)/O(n) = O(log n) • Merge phase is sequential bottleneck – Need to improve the depth of the merge phase Improvements • Parallel merge C. P. Kruskal. Searching, merging, and sorting in parallel computation. IEEE Trans. Comput., C–32(10):942–946, Oct. 1983 • Pipelined divide-and-conquer : O(log n) – Start the merge phase at the top level before the recursive calls complete R. Cole. Parallel merge sort. SIAM Journal of Computing, 17(4):770–785, Aug. 1988 Randomization • Random numbers used in parallel algorithms to ensure that processors make local decisions which with high probability add up to global decisions • Techniques – Sampling – Symmetric breaking – Load balancing Sampling • Select a representative sample from a set of elements • Solve the problem on that sample • Use the solution to guide the solution for the original set • Example: sorting Number sorting • Partition the keys into non-overlapping intervals and sort each interval • Each interval must contain approximatively the same number of keys • Random sampling used to determine the interval boundaries – Select a random sample of keys – All selected keys are sorted together – Sorted keys are used as boundaries • Examples: computational geometry, graphs, string matching Symmetry breaking • Phenomenon in which small fluctuations acting on a system crossing a critical point decide the system’s fate • Example: – Select a large number of independent graph vertices in parallel – Vertices are not neighbors • If one vertex joins the set all of its neighbors must deny to join – Difficult if the structure of each vertex is the same – Randomness is used to break the symmetry between vertices Load balancing • Evenly distribute a large number of items across processors • Randomly assign each element to a subset • Good results if the average size of a subset is at least logarithmic in the size of the original set Parallel pointer techniques • Sequential techniques for manipulating lists, trees, graphs, do not translate easily into parallel techniques • Examples: – Traversing the elements of a linked list – Visiting the list of nodes in postorder – Performing a depth first search • Solution: replace them with equivalent parallel techniques Pointer jumping • In each step, each node in parallel replaces its pointer with that of its successor (or parent for trees) • Example – Label each node of an n-node list with the label of the last node (or root) – Log n steps for each node to point to the same node Euler tour • A path through the graph in which every edge is traversed exactly once • Example – Euler tour of an undirected tree is computed by following the perimeter of the tree visiting each edge twice (on the way down and then up) • Keeping the Euler tour it is possible to compute many functions on the tree • O(n) - independent on the depth of the tree Graph contraction • Reduce size of a graph while keeping some of its original structure • Example – Graph partitioning • Collapse high density vertices recursively • Find connected components • Expand the connected components to original graph Ear decomposition • Partition a graph into an ordered collection of paths – – – – First path is a cycle The rest are called ears End-points of each ear are anchored on previous paths Can be used to determine of 2 edges are on a common cycle • Example – Determining biconnectivity, triconnectivity, 4-connectivity, planarity, replace depth first search • Logarithmic depth and linear work What’s next? • Graph parallel algorithms • BSP model – SSSP, connected components, pagerank • Vertex centric vs. subgraph centric • Load balancing – Importance of partitioning and graph type • ...