Download ppt1

Advanced Seminar – University of Trento – June 2006
Michela Lecca - TeV
What is COMPASS
COMPASS is a distributed application for
content-based image retrieval using remoted
databases.
It has been developed in 2000 by
Roberto Brunelli and Ornella Mich by the
group of Technology of Vision, ITC – irst,
Trento, Italy
Michela Lecca - TeV
The problem
The considerable amount of multimedia
database requires sophisticated indices for its
effective use.
Manual indexing is the most effective method
to do this, but it is also the lowest and the most
expensive. Automated methods have to be
developed.
Michela Lecca - TeV
Multimedia Data Management
The traditional methods based on the textual
description and searches are no more
practicable for two reasons:
1. associating textual description to multimedia
data can be very expensive;
2. textual description may be not adequate for
multimedia retrieval.
Michela Lecca - TeV
COMPASS
COMPASS can be used for two main activities:
1. to search database images similar to a query
image (query by example);
2. to browse image databases .
Michela Lecca - TeV
Query by example
COMPASS is a client-server architecture in
which a client application submits a user query
to multiple image servers.
The answers are then merged and proposed
to the user as a single results.
Michela Lecca - TeV
Query by example
Image
Databases
One or more
query images
Description of the
database images by
low-level features
Description
and Comparison
with the Described
Image Databases
Described
Image
Databases
Ranked List
of Responses
Michela Lecca - TeV
Image Description
The images in the database, such as the user
query, are described by low-level features.
Michela Lecca - TeV
Image Description (2)
Simple General Descriptors: to describe the
general visual appearance of the image or of a
region of interest;
Texture Descriptors: to quantify and qualify
properties such as smoothness, coarseness
and regularity (pattern replication);
Shape Descriptors: to describe the shape of
an interest image region.
Michela Lecca - TeV
Simple General Descriptors (1)
The color properties are described by
1. Hue
2. Saturation
3. Luminance
Michela Lecca - TeV
Simple General Descriptors (2)
Color Space HSI
The color properties are
described by using the
space HSI.
Michela Lecca - TeV
Simple General Descriptors (3)
Hue
Color perceived as ranging
from red through yellow,
green and blue, as
determined by the dominant
wavelenght of the light.
In HSL space it is measured
in degrees.
green (120o)
cyan
(180o)
blue (270o)
yellow (60o)
red (90o)
violet (300o)
Michela Lecca - TeV
Simple General Descriptors (4)
green
yellow
Saturation
Hue
It defines how gray the color
is and it ranges in [0, 1]: 0 is cyan
gray, 1 is a pure color.
red
blue
violet
Michela Lecca - TeV
Simple General Descriptors (5)
Intensity
It defines the lightness of the colors. Variability range:
[0,1].
intensity = 0.3
intensity = 0.4
intensity = 0.5
Michela Lecca - TeV
Simple General Descriptors (6)
Edges Distribution
The edgeness is defined as
dx I + dy I
where I is the intensity of the image. So it is related to
the image gradient.
Michela Lecca - TeV
Simple General Descriptors (7)
The distribution of the simple general
descriptors are represented by 16 bins
histograms.
[R. Brunelli – O.Mich, On the Use of Histograms for Image
Retrieval, Proc. of IEEE ICCM 1999, Florence, Italy]
Michela Lecca - TeV
Texture Descriptors (1)
Texture means properties such as smoothness,
coarseness, and regularity (pattern replication).
Michela Lecca - TeV
Texture Descriptors (2)
Intensity and Hue Co-occurrence Histograms:
2-D histograms of distributions of hue and
intensity, of two pixels related each to other by a
positional operator T.
In COMPASS, indicated by (x, y) and (x', y') the
position in the image of the pixels p and p' resp.,
T is defined as
T(x, y) = (x + 2, y + 2).
p and p' are T-related iff (x', y') = T(x, y).
Michela Lecca - TeV
Texture Descriptors (3)
Intensity and Hue Co-occurrence Histograms:
Intensity co-occurrence : to describe textural
information of grey level image;
Hue co-occurrence: to describe the textural
properties of color image.
Michela Lecca - TeV
Texture Descriptors (4)
Wavelets:
Texture is strictly related with the scale factor at
which an image (or a portion of it) is observed.
Wavelets are a powerful approach to multiresolution image processing.
Michela Lecca - TeV
Texture Descriptors (5)
Wavelets:
The wavelet transform is a decomposition of the
signal into different frequency components by
means of a family of real orthonormal bases
fa, b(x) obtained through translation and dilation of a
kernel function f(x) called mother wavelet:
fa,b(x) = a-1/2 f((x-b)/a)
Michela Lecca - TeV
Texture Descriptors (6)
Wavelets:
The wavelet transform of a continuous signal
f(x) is
Wa, b f = ʃ f(x) f*a, b(x) dx
where f* indicates the analyzed wavelet.
[S. Mallat, A Theory for Multiresolution Signal Decomposition: The Wavelet
Representation, Proc. IEEE PAMI, Vol. 11, N. 7, 1989]
Michela Lecca - TeV
Shape Descriptors (1)
For region shape descriptors:
52 Li Moments: to describe the shape of a region;
these moments are computed by using the FourierMellin transformation;
Fourier Coefficients: to describe the external
contour of a region; they represent the shape in the
frequency domain (low frequency -> general
features, high frequency -> details).
Michela Lecca - TeV
Shape Descriptors (2)
52 Li Moments:
Y. Li, Reforming the theory of invariant moments for
pattern recognition, Pattern Recognition, 25(7),
1992
Fourier Coefficients:
A lot of Books of Mathematical Analysis ...
Michela Lecca - TeV
Descriptor Invariance and
Coding
All the used features are normalized to be
invariant by translation, rotation, rescaling and
combination by thereof.
The low-level features are then encoded in a
vector (96-dimensional vector in original
version, 208 in the extended version), whose
entries vary in the range [0, 255].
Michela Lecca - TeV
Image Comparison (1)
The similarity between the query image and
each item of the remoted databases is defined
as L1-distance between the corresponding
feature vectors v = [v1, ..., vn], w = [w1, ..., wn]:
d(v, w) = Si=1, ..., n | vi - wi|
[R. Brunelli – O.Mich, On the Use of Histograms for Image
Retrieval, Proc. of IEEE ICCM 1999, Florence, Italy]
Michela Lecca - TeV
Image Comparison (2)
In the case of queries with multiple images,
COMPASS uses warpable metrics to
emphasize (or minimize) the impact of the
most (or less) relevant features of the image
query set.
[See references for details.]
Michela Lecca - TeV
Performances
The search process is very efficient: retrieving
the closest item in a database of 1 million
elements takes less than a second on a
standard Pentium4 with 2GHz CPU !!!
http://compass.itc.it/compareIt.html
Michela Lecca - TeV
Browsing Databases
Database images are clustered in groups of
images similar to each other according to their
L1- distance. Each cluster is then represented
by a key image, the closest to the center, and
acts as hyperlink to the elements of the
cluster.
Michela Lecca - TeV
COMPASS Interface
[http://compass.itc.it/demos.html]
Michela Lecca - TeV
References
●
●
●
R. Brunelli, O. Mich, COMPASS: an Image
Retrieval System for Distributed Databases, Proc.
of ICME 2000, New York, USA
R. Brunelli, O. Mich, Image Retrieval by Example,
IEEE Transaction on Multimedia, Vol. 2, N. 3, 2000
C. Andreatta, CBIR Techniques for Object
Recognition, Tech. Rep. ITC-irst 04-12-01, Dec.
2004
http://compass.itc.it/papers.html
Michela Lecca - TeV