Download Parallelizing Multiscale and Multigranular Spatial Data Mining

Parallelizing
Spatial Data Mining Algorithms:
A case study with Multiscale and
Multigranular Classification
PGAS 2006
Vijay Gandhi, Mete Celik, Shashi Shekhar
Army High Performance Computing and Research Center (AHPCRC)
University of Minnesota
Overview





PGAS Relevance, Application Domain
Problem Definition
Approach
Experimental Results
Conclusion & Future Work
PGAS Relevance


How effective is UPC in parallelizing
spatial applications?
How effective is UPC in improving
productivity of researchers in spatial
domain?
Spatial Applications: An Example

Multiscale Multigranular
Image Classification
Input
Output Images at
Multiple Scales
MSMG Classification Formulation

Model

Mˆ  arg max{ l ( x | M )  2 pen( M ) }
M





=
=
M
l(x | M ) =
pen =
x
observations
a classification model
log-likelihood (Quality Measure) of M
Penalty function
Calculation of log-likelihood of M


Uses Expectation Maximization
Computationally Expensive

7 hours of Computation time for an input image of size
512 x 512 pixels with 4 Classes
Spatial Application: Multiscale
Multigranular Image Classification

Applications




Land-cover change Analysis
Environmental Assessment
Agricultural Monitoring
Challenges



Expensive computation of Quality Measure i.e. likelihood
Large amount of data
Many dimensions
Pseudo-code : Serial Version









1. Initialize parameters.
2. for each Class
3.
for each Spatial Scale
4.
for each Quad
5.
Calculate Quality Measure
8.
end for Quad
9.
end for Spatial Scale
10. end for Class
11. Post-processing
Q? What are the options for parallelization?
Parallelization –
Problem Definition

Given





Find


Parallel formulation of the algorithm
Objective


Serial version of a Spatial Data Mining Algorithm
Likelihood of each specific class at each pixel
Class-hierarchy
Maximum Spatial Scale
Scalability e.g. Isoefficiency
Constraints

Parallel Platform “UPC”
Challenges in Parallelization
Description of work

Compute Quality Measure for combinations of
Class-label, Scale, Quad (Spatial Unit)
Challenges
a) Variable workload across computations of
quality measure
b) Many dimensions to parallelize
i.e. Class-label, Scale, Quad
c) Dependency across scales
Class-level Parallelization









1.
2.
3.
4.
5.
8.
9.
10.
11.
Initialize parameters and memory
upc_forall Class
for each Spatial Scale
for each Quad
Calculate Quality Measure
end for Quad
end for Spatial Scale
end upc_forall Class
Post-processing
Class-level Parallelization

Disadvantages:

Workload distribution is uneven
(Cost of Quality measure changes with Class)

Number of parallel processors is restricted to
number of classes
Examples
Quad-level Parallelization










1.
2.
3.
4.
5.
6
7.
8.
9.
11.
Initialize parameters and memory
for each Spatial Scale
upc_forall Quad
for each Class
Calculate Quality Measure
end for Class
end upc_forall Quad
upc_barrier
end for Spatial Scale
Post-processing
Quad-level Parallelization

Advantages:



Workload distribution is more even
Greater number of processors can be used
Number of Quads = f (Number of pixels)

Example
 Input: 4 Classes, Scale of 6
Input Image Size
Number of Quads
64 x 64
98,304
128 x 128
393,216
512 x 512
6,291,456
1024 x 1024
25,165,824
Experimental Design
Input
64 x 64 pixels image
(Plymouth County, Massachusetts)
 4 class labels
(Everything, Woodland, Vegetated,
Suburban)

Language
UPC
Hardware
Platform
Cray X1
Number of
Processors
1-8
Workload
Scale: 64 x 64
Input class hierarchy
Scale: 2 x 2
Output Images at Multiple scales
Effect of Number of
Processors
cy.
7
6
5
4
3
2
1
0


1.2
1
0.8
0.6
0.4
0.2
0
2

Efficiency Plot
Efficiency
Speedup
u
Speedup
4
8
Number of Processors
Class-level Quad-level
1
2
4
Number of Processors
Class-level
Quad-level
Quad-level parallelization gives better speed-up
Room for Speed-up for both approaches
Q? Class-level << Quad-level. Why?
8
Workload Distribution
Fixed Parameter - Number of processors: 4


Quad-level parallelization provides better load-balance
Probably because of large number of Quads (~100,000)
Conclusions

How effective is UPC in parallelizing Spatial
applications?

Quad-level parallelization



Speed-up of 6.65 on 8 processors
Large number of Quads (98,304)
Class-level parallelization

Speed-ups are lower

Smaller number of Classes (4)
Conclusions

How effective is UPC in improving productivity
of researches in spatial domain?

Coding effort was reduced



20 lines of new code in program with base size of 2000 lines
1 person-month
Analysis effort refocused



Identify units of parallel work i.e. Quality Measure
Identify dimensions to parallelize i.e. Quad, Class, Scale
Selecting dimension(s) to parallelize




Dependency Analysis (Ruled out Scale)
Number of Units (Larger the better)
Load Balancing
6 person-month
Future Work

Improve Efficiency


Explore Dynamic Load Balancing
Other parallel formulations
Acknowledgements



Spatial Databases / Spatial Data Mining Group
AHPCRC
 Richard Welsh, NCS
University of Boston
 Junchang Ju, Eric D. Kolaczyk, Sucharita Gopal
References



E. D. Kolaczyk, J. J., and G. S. Multiscale, Multigranular
Statistical Image Segmentation. Journal of the American
Statistical Association, 100, 1358-1369, 2005.
Z. Kato, M. Berthod, and J. Zerubia. A hierarchical Markov
random field model and multi-temperature annealing for
parallel image classification. Graphical Models and Image
Processing, 58(1):18–37, January 1996.
A. Y. Grama, A. Gupta, V. Kumar. Isoefficiency: Measuring
the Scalability of Parallel Algorithms and Architectures.
IEEE Parallel & Distributed Technology: Systems & Technology,
1, 12-21, 1993.