Download Leon-Garcia, A.

Statistical Modeling of Natural Images
in the Wavelet Space
Parametric models of wavelet
coefficients
Univariate i.i.d. models
Spatially adaptive models
Application into texture synthesis
Pyramid-based scheme
(Heeger&Bergen’1995)
Projection-based scheme
(Portilla&Simoncelli’2000)
EE565 Advanced Image Processing Copyright
Xin Li 2008
1
Recall: Transform Facilitates Modeling
x2
y1
y2
x1
x1 and x2 are highly correlated
p(x1x2)  p(x1)p(x2)
y1 and y2 are less correlated
p(y1y2)  p(y1)p(y2)
EE565 Advanced Image Processing Copyright
Xin Li 2008
2
Empirical Observation
H1
1000
1000
900
900
800
800
700
700
600
600
500
500
400
400
300
300
200
200
100
100
0
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
A single peak at zero
EE565 Advanced Image Processing Copyright
Xin Li 2008
3
Univariate Probability Model
Laplacian:
Gaussian:
EE565 Advanced Image Processing Copyright
Xin Li 2008
4
Gaussian Distribution
EE565 Advanced Image Processing Copyright
Xin Li 2008
5
Laplacian Distribution
EE565 Advanced Image Processing Copyright
Xin Li 2008
6
Statistical Testing
 How do we know which parametric model better
fits the empirical distribution of wavelet
coefficients?
 In addition to visual inspection (which is often
subjective and less accurate), we can use
various statistical testing tools to objectively
evaluate the closeness of an empirical
cumulative distribution function (ECDF) to the
hypothesized one
 One of the most widely used techniques is
Kolmogorov-Smirnov Test (MATLAB function:
>help kstest).
EE565 Advanced Image Processing Copyright
Xin Li 2008
7
Kolmogorov-Smirnov Test*
The K-S test is based on the maximum distance between empirical CDF
(ECDF) and hypothesized CDF (e.g., the normal distribution N(0,1)).
EE565 Advanced Image Processing Copyright
Xin Li 2008
8
Example
Usage: [H,P,KS,CV] = KSTEST(X,CDF)
If CDF is omitted, it assumes pdf of N(0,1)
Accept the hypothesis
Reject the hypothesis
x: computer-generated samples
(0<P<1, the larger P, the more likely)
d: high-band wavelet coefficients
of lena image (note the normalization
by signal variance)
EE565 Advanced Image Processing Copyright
Xin Li 2008
9
Generalized Gaussian/Laplacian
Distribution
P: shape parameter
: variance parameter
where
Laplacian
Gaussian
EE565 Advanced Image Processing Copyright
Xin Li 2008
10
Model Parameter Estimation*
Maximum Likelihood Estimation
Method of moments
Linear regression method
Ref.
[1] Sharifi, K. and Leon-Garcia, A.
“Estimation of shape parameter for generalized Gaussian
distributions in subband decompositions of video,”
IEEE T-CSVT, No. 1, February 1995, pp. 52-56.
[2] www.cimat.mx/reportes/enlinea/I-01-18_eng.pdf
EE565 Advanced Image Processing Copyright
Xin Li 2008
11
I.I.D. Assumption Challenged
If wavelet coefficients of each subband
are indeed i.i.d., then random
permutation of pixels should produce
another image of the same class
(natural images)
The fundamental question here: does
WT completely decorrelate image
signals?
EE565 Advanced Image Processing Copyright
Xin Li 2008
12
Image Example
High-band
coefficients
permutation
You can run the MATLAB demo to check this experiment
EE565 Advanced Image Processing Copyright
Xin Li 2008
13
Another Experiment
5
4
3
2
1
Y
0
-1
-2
-3
-4
-4
-3
-2
-1
0
X
1
2
3
4
Joint pdf of two uncorrelated random variables X and Y
EE565 Advanced Image Processing Copyright
Xin Li 2008
14
Joint PDF of Wavelet Coefficients
Y=
X=
Joint pdf of two correlated
random variables X and Y
Neighborhood I(Q): {Left,Up,cousin and aunt}
EE565 Advanced Image Processing Copyright
Xin Li 2008
15
Heeger&Bergen’1995: Histogram-based
Basic idea: two visually similar textures will also have similar statistics
 Pyramid-based (using steerable pyramids)
Facilitate the statistical modeling
 Histogram matching
Enforce the first-order statistical constraint
 Texture matching
Alternate histogram matching in spatial and wavelet
domain
 Boundary handling: use periodic extension
 Color consistency: use color transformation
EE565 Advanced Image Processing Copyright
Xin Li 2008
16
Histogram Matching
Generalization of histogram equalization (the target is the histogram
of a given image instead of uniform distribution)
EE565 Advanced Image Processing Copyright
Xin Li 2008
17
Histogram Equalization
x
y  L   h(t )
t 0
Uniform
Quantization
L
x
y s   h(t )
t 0
L
0
1
Note:
 h(t )  1
t 0
cumulative probability function
L
EE565 Advanced Image Processing Copyright
Xin Li 2008
x
18
MATLAB Implementation
function y=hist_eq(x)
[M,N]=size(x);
for i=1:256
h(i)=sum(sum(x= =i-1));
End
y=x;s=sum(h);
for i=1:256
I=find(x= =i-1);
y(I)=sum(h(1:i))/s*255;
end
Calculate the histogram
of the input image
Perform histogram
equalization
EE565 Advanced Image Processing Copyright
Xin Li 2008
19
Histogram Equalization Example
EE565 Advanced Image Processing Copyright
Xin Li 2008
20
Histogram Specification
histogram1
histogram2
S-1*T
T
S
?
EE565 Advanced Image Processing Copyright
Xin Li 2008
21
Texture Matching
Objective: the histogram of both subbands and synthesized image
matches the given template
Basic hypothesis: if two texture images visually look similar, then they
have similar histograms in both spatial and wavelet domain
EE565 Advanced Image Processing Copyright
Xin Li 2008
22
Image Examples
EE565 Advanced Image Processing Copyright
Xin Li 2008
23
Portilla&Simoncelli’2000: Parametric
Basic idea: two visually similar textures will also have similar statistics
Instead of matching histogram (nonparametric models), we can build
parametric models for wavelet coefficients and enforce the synthesized
image to inherit the parameters of given image
Model parameters: 710 parameters were used in Portilla and Simoncelli’s
experiment (4 orientations, 4 scales, 77 neighborhood)
EE565 Advanced Image Processing Copyright
Xin Li 2008
24
Statistical Constraints
Four types of constraints
Marginal Statistics
Raw coefficient correlation
Coefficient magnitude statistics
Cross-scale phase statistics
Alternating Projections onto the four
constraint sets
Projection-onto-convex-set (POCS)
EE565 Advanced Image Processing Copyright
Xin Li 2008
25
Convex Set
x  , y  
ax  (1  a) y  , 0  a  1
A set Ω is said to be convex if for any two point
We have
Convex set examples
Non-convex set examples
EE565 Advanced Image Processing Copyright
Xin Li 2008
26
Projection Operator
Projection onto convex set C
f
g
C
g  Pf  {x  C || x  f || min || x  f ||}
xC
In simple words, the projection of f onto a convex set C is the
element in C that is closest to f in terms of Euclidean distance
EE565 Advanced Image Processing Copyright
Xin Li 2008
27
Alternating Projection
C1
X1
X∞
X0
X2
C2
Projection-Onto-Convex-Set (POCS) Theorem: If C1,…,Ck are
convex sets, then alternating projection P1,…,Pk will converge
to the intersection of C1,…,Ck if it is not empty
Alternating projection does not always converge in the case
of non-convex set. Can you think of any counter-example?
EE565 Advanced Image Processing Copyright
Xin Li 2008
28
Convex Constraint Sets
● Non-negative set
{ f | f  0}
● Bounded-value set
{ f | 0  f  255}
or
{ f | A  f  B}
● Bounded-variance set
{ f || f  g ||2  T }
A given signal
EE565 Advanced Image Processing Copyright
Xin Li 2008
29
Non-convex Constraint Set
Histogram matching used in
Heeger&Bergen’1995
Bounded Skewness and Kurtosis
skewness
kurtosis
The derivation of projection operators onto constraint sets are tedious
are referred to the paper and MATLAB codes by Portilla&Simoncelli.
EE565 Advanced Image Processing Copyright
Xin Li 2008
30
Image Examples
original
synthesized
EE565 Advanced Image Processing Copyright
Xin Li 2008
31
Image Examples (Con’d)
original
synthesized
EE565 Advanced Image Processing Copyright
Xin Li 2008
32
When Does It Fail?
original
synthesized
EE565 Advanced Image Processing Copyright
Xin Li 2008
33
Summary
Textures represent an important class of
structures in natural images – unlike
edges characterizing object boundaries,
textures often associate with the
homogeneous property of object surfaces
Wavelet-domain parametric models
provide a parsimonious representation of
high-order statistical dependency within
textural images
EE565 Advanced Image Processing Copyright
Xin Li 2008
34