Download Algorithm - SSUET - Computer Science Department

CS405 Parallel Computing
BS-Computer Science (SSUET)
LECTURE 12: Parallel Algorithm Analysis
Algorithm:
1. An algorithm is a precise prescription of how to accomplish a task.
2. Two important issues determine the character of an algorithm:
3. Which operations are available to us?
4. In which order can the operations be performed?
5. One at a time (sequentially).
6. Several at once (in parallel).
A Simple Model for Parallel Processing:
1. Parallel Random Access Machine (PRAM) model
a. a number of processors all can access
b. a large share memory
c. all processors are synchronized
d. all processor running the same program
i. Each processor has an unique id, pid. and may instruct
PRAM models:
1. PRAM models vary according
a. how they handle write conflicts
b. The models differ in how fast they can solve various problems.
2. Concurrent Read Exclusive Write (CREW)
a. only one processor are allow to write to
b. one particular memory cell at any one step
3. Concurrent Read Concurrent Write (CRCW)
4. Algorithm works correctly for CREW
a. will also works correctly for CRCW
b. but not vice versa
Approaches to the design of parallel algorithms:
1. Modify an existing sequential algorithm
a. Exploiting those parts of the algorithm that are naturally parallelizable.
2. Design a completely new parallel algorithm that
Lecturer: Engr. Muhammad Nadeem
Page 1
CS405 Parallel Computing
BS-Computer Science (SSUET)
a. May have no natural sequential analog.
Speedup and Efficiency of parallel algorithms:
1. Let T*(n) be the time complexity of a sequential algorithm to solve a problem P of input
size n
2. Let Tp(n) be the time complexity of a parallel algorithm to solves P on a parallel
computer with p processors
3. Speedup
a. Sp(n) = T*(n) / Tp(n)
b. Sp(n) <= p
c. Best possible, Sp(n) = p
i. when Tp(n) = T*(n)/p
4. Efficiency
a. Ep(n) = T1(n) / (p Tp(n))
i. where T1(n) is when the parallel algorithm run in 1 processor
b. Best possible, Ep(n) = 1
A simple parallel algorithm:
Adding n numbers in parallel
1. Example for 8 numbers: We start with 4 processors and each of them adds 2 items in the first
step.
2. The number of items is halved at every subsequent step. Hence log n steps are required for
adding n numbers. The processor requirement is O(n).
How do we analyze a parallel algorithm?
A parallel algorithm is analyzed mainly in terms of its time, processor and work complexities.
Lecturer: Engr. Muhammad Nadeem
Page 2
CS405 Parallel Computing
BS-Computer Science (SSUET)
1. Time complexity T(n) : How many time steps are needed?
2. Processor complexity P(n) : How many processors are used?
3. Work complexity W(n) : What is the total work done by all the processors?
Hence, for our example:
T(n) = O(log n)
P(n) = O(n)
W(n) = O(n log n)
Map Reduce Algorithm:
What is MapReduce?
MapReduce is a programming model and an associated implementation for manipulating very large
data sets. It provides a framework in which a programmer can write two simple functions, a mapper
and a reducer that perform the actual data manipulation. Applications written using the MapReduce
framework are automatically parallelized and suitable for execution on a large infrastructure of
connected machines.
Lecturer: Engr. Muhammad Nadeem
Page 3