Download Multiresolution Methods for Image Processing

Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
Multiresolution Methods for Image Processing
Chiara Olivieri
Department of Computer and Information Science
[email protected]
Matematica, Forme, Immagini, 19/03/2010
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
Outline
1
Human Vision Model
2
Multiresolution Representations
3
The Steerable Pyramid Transform
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
A Model for Human Vision System
The center of the human vision is the
primary visual cortex.
It is located in the posterior pole of the
occipital cortex;
It is highly specialized for processing
information about static and moving objects
and is excellent in pattern recognition.
The neurons of the primary visual cortex (V1
neurons) can discriminate small changes
in visual orientations, spatial frequencies
and colors.
Seems to be that early responses of V1
neurons act as sets of selective
spatiotemporal filters.
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
A Top/Down Approach in Human Vision
Let’s think for a moment we are
looking at a beautiful sight out of the
windows...
we always look first at the scene
in its totality;
after we focus on the details of
the scene.
This is the intuitive approach of a
multiresolution transform.
A multiresolution transform can be
seen as a simplification and
restriction of the complex and high
non-linear functioning of V1 neurons.
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
A Multiresolution Framework
A multiresolution representation bases its main
concepts on the system of the sight of human being.
It provides a simple hierarchical framework for interpreting the
image information.
At different resolutions, the details of an image generally
characterize different physical structures of the scene.
At coarse resolution, the details correspond to larger structures.
At high resolution, we obtain the finer details.
The basis functions of these representations are well localized
in spatial position, orientation, and scale.
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
A Sparse Representation
The basis functions subdivide the information of the original
image in a set of subband of coefficients.
The structural information of the image is then subdivided in few
large-magnitude coefficients spread in several subbands.
The intuitive explanation is that images contain smooth areas
interspersed with occasional sharp transitions (e.g., edges).
The smooth regions produce small-amplitude coefficients, and
the transitions produce sparse large-amplitude coefficients.
As a rule of thumb...
Large-magnitude coefficients tend to lie along ridges with
orientation matching that of the subband.
Large-magnitude coefficients also tend to occur at the same
relative spatial locations in subbands at adjacent scales, and
orientations.
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
Histogram of the Coefficients
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
General Model for Decomposition
Given a starting image I, a typical multiresolution algorithm for
decomposition has the following steps:
Algorithm
1
filter I with a set of band-pass filters;
every band-pass filter should select a specific orientation;
2
filter I with a low pass filter;
obtaining a new image of reducted resolution;
3
4
since now we have more information than needed,
downsample I, obtaining an new image I 0 of half size;
repeat the first step on the downsampled image I 0 .
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
Example: the Steerable Pyramid Transform
First level of resolution
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
Example: the Steerable Pyramid Transform
Second level of resolution
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
Example: the Steerable Pyramid Transform
Third level of resolution
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
A Completely Adaptive Representation
The steerable pyramid transform is a multiresolution
representation that is a little different from classical wavelet.
A filter is called ”steerable” if it can be written as a linear sum of
rotated versions of itself.
The steerable pyramid can be more formally described with the
frame theory.
The formulation of the filters of the steerable pyramid allows a
complete control on the number of orientations we want to
analyze.
The maximum number of scales is fixed by the size of the image.
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
Good and Not so Good Properties...
The steerable filters provide an invertible and aliasing-free
reconstruction and can be easily adapted to create sets of odd
and even filters.
Problems
Advantages
invariant for translation;
invariant for rotation;
perfect-reconstruction with
Fourier domain
implementation.
the space-domain
implementation is not
perfect-reconstruction;
overcompleteness by a factor
of 4K /3, where K is the
number of orientation bands;
tricky 3D extension.
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
Pyramidal Decomposition
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
Set of Basis Filters
In order to preserve the invertibility of the reconstruction we
must set some constraints on the set of steerable filters.
Requisites
|H0 (r )|2 + |L0 (r )|2 = 1
PK −1
|H0 (r )|2 + |L0 (r )|2 [|L1 (r )|2 + k =0 Bk (r )2 ] =
1
r L
=
2r P
L
|L(r )|2 + K −1 |Bk (r , φ)|2
k =0
2
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
Steerable Filters
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
Denoising with Steerable Pyramid
A denoising algorithm with the Steerable Pyramid Transform
(and in general with every wavelet transform) consists of three
main stages:
Denoising
1
decomposition of the starting image in a set of
coefficient images;
2
estimation of the noise-free coefficients from the noisy
ones;
3
reconstruction of the image from the updated
coefficients.
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
Denoising through Thresholding
In the wavelet domain, the noise is uniformly spread
throughout the coefficients, while most of the image
information is concentrated in the few largest ones.
The most straightforward way to distinguish noise from
information consists of the thresholding of the wavelet
coefficients.
Thresholding functions
hard thresholding f (x) = 0 if |x| ≤ T ,
x elsewhere
soft thresholding
f (x) = 0 if |x| ≤ T ,
x + T if x < −T
x − T if x > T ,
”our” thresholding f (x) = x/(1 + exp(−S(|x| − T )))
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
Thresholding Functions
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
Results
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
Results
C. Olivieri
Multiresolution Methods
Human Vision Model
Multiresolution Representations
The Steerable Pyramid Transform
Results
C. Olivieri
Multiresolution Methods
Appendix
For Further Reading
For Further Reading I
S. Mallat.
A Wavelet Tour of Signal Processing, Third Edition: The
Sparse Way.
Academic Press, 2008.
J. Portilla, V. Strela, M.J. Wainwright, E.P. Simoncelli.
Image denoising using scale mixtures of Gaussians in the
wavelet domain.
IEEE Transactions on Image Processing, 2003.
C. Olivieri
Multiresolution Methods