Download High-Dimensional Feature Descriptors to Characterize Volumetric

High-Dimensional Feature Descriptors to Characterize Volumetric Data
Julia EunJu Nam, Mauricio Mauer, and Klaus Mueller
Center for Visual Computing, Computer Science Department, Stony Brook University
ABSTRACT
Our feature vectors, however, are too high-dimensional for a
visual analysis on this low level, and also, visual clustering is
counter to our goal of model-driven volume data classification.
Volumetric data offer considerably more low-level information
than the density and gradient values traditionally employed in
transfer function-guided volume visualization. We propose the use
of a richer set of low-level feature descriptors to allow a more
sensitive and refined characterization of the data. The ensemble of
these feature descriptors then forms a distinct data signature that
can describe, classify, and categorize a dataset at levels of detail.
1
2
APPROACH
We chose to use two statistical low-level feature descriptors, and
not descriptors based on patterns such as [3]. Our descriptors are
(i) density histograms to represent the overall texture statistics and
(ii) gradient histograms to capture the variation, perceptual effects
of these. Both feature descriptors are also designed to be
insensitive to affine transformations, such as scale, rotation, and
translation, to support feature and object mining in large datasets.
A global histogram is not descriptive enough as a classifier,
and this motivated the use of more feature-oriented histograms.
We did not attempt automated segmentation which is still an open
research area, to confine the extent of histograms within object
boundaries. Instead, we aimed for a representation that has good
potential to capture the essence of an object, and yet is automatic
to compute. This led us to calculate the density signatures in local
histograms at a hierarchy of window sizes, to enable the detection
of feature density statistics at multiple levels of scales. For this,
we normalize the histograms. We also perform optional
smoothing at the histogram level (not the image level) to reach
independence of the sampling process that generated the data.
Finally, rotation invariance is achieved by constructing the
histograms within slightly overlapping radial regions.
To encode the gradient
signatures we used the SIFT
feature descriptor [5]. The SIFT
algorithm consists of two phases:
(i) the detection of critical points
(the keypoints) in scale-space and
(ii) the encoding of these into
keypoint
descriptors.
The
keypoints are local extrema in a
difference-of-Gaussians
multiscale pyramid. They are detected
by comparing a point’s neighbors
Figure 1. The 2D SIFT
on the current scale and the scales
keypoint descriptor.
above and below. Points having
low contrast or poorly localized along an edge are then rejected
limiting the set of keypoints to the most descriptive and salient
ones, preserving manageability. The keypoint (feature) descriptor
is an orientation histogram of local gradients, quantized into bins.
It is constructed by computing the magnitude and orientation at
each image sample point in a region around the keypoint location,
weighted by a Gaussian function with σ equal to one half the
width of the descriptor window to achieve a certain level of
smoothing. The samples are then aggregated into 8-bin orientation
histograms describing the neighborhood over a 4×4 matrix of
subregions (Figure 1 shows an example). This histogram is stored
into a 128-long feature vector, along with the scale ID and
location. Further, the SIFT feature descriptor can be made
orientation independent by aligning the descriptor and the gradient
orientations by the keypoint orientation. SIFT has traditionally
been used only in 2D in the context of images, but recently [1]
have generalized the framework into higher dimensions, for
INTRODUCTION
Extracting knowledge from data requires an effective
characterization and integration of the embedded low-level
features. In volume visualization, this feature integration is often
done explicitly by the user, aided by appropriate graphical
rendering techniques and implemented within a responsive user
interface. One measure of distinguishing these many rendering
techniques is by the underlying feature model, whose parameters
are captured in a feature vector. Here, the simplest model is
purely sample-based, that is, just the sample value (we shall
assume scalar densities for this paper, without loss of generality),
which can be visually enhanced via suitable RGB mappings in
transfer functions. The next step up is to assist the user in making
the conjunctive feature integration. Shape is a strong cue, and to
infer it, one can calculate gradients from the sample
neighborhood. More advanced local computations can emphasize
higher-level shape features, such as curvature, suggestive
contours, and others. All of these features have in common that
they have direct visual manifestations which can be readily
brought out by their interaction with light. Thus, the underlying
models have direct correspondences to familiar real-life visual
artifacts. This limits the scope of the features that can be used for
characterizing the data. The feature vectors are simply not
expressive enough to classify the data autonomously at the
required level of detail, thus requiring the sophisticated perceptual
and cognitive capabilities of the human user. This is particularly
troublesome in less coherent data, such as amorphous phenomena.
We aim to make progress in this area by first deriving a set of
rich feature vectors from the data and then using cluster analysis
to learn the feature vectors relevant to the model describing the
data. Here we are looking for features that are less visually
intuitive, requiring computer analysis for their derivation and
management. Once these rich feature vectors have been learned,
they can be used for subsequent classification, mining and
selection. In that regard, unlike other works in volume graphics
based on machine learning [1] our focus is not segmentation, but
model-based object-level classification, detection, categorization,
and visual feature content communication. A further anticipated
outcome is that such a model can then provide an ontology of the
data, to be matched with a suitable volume rendering algorithm
ontology for automated image generation. A mapping of this
nature was recently successfully demonstrated by [1] within an
information visualization context composed of graphs and plots.
Our method uses computational cluster analysis, followed by
an illustration of the results in a 2D plot obtained via MultiDimensional Scaling (MDS). Most visualization research is based
on visual cluster analysis, where feature vectors produce scatter
plots that are coupled with a transfer function interface [1][4][7].
1
determining accurate feature point correspondences between
medical volume datasets for object alignment. A 3D-SIFT feature
is comprised of a 163 cubic voxel region around the keypoint, with
64 subregions and each summarized by an 8×8 histogram of voxel
orientations. It uses two spherical coordinates (θ, φ) for
parameterization to form the 3D-SIFT feature descriptor.
Having captured the essential perceptual features using the two
low-level feature descriptors, we now assemble these into highlevel constructs. Our goal is to learn a high-level representation of
the sampled object, using its low-level features. Using amorphous
phenomena data as an example, we distinguish between three
levels. Level-1: categories (e.g., smoke vs. fire), level-2: category
instances (e.g., turbulent smoke vs. laminar smoke), and level-3:
groups of similar individual features. We note that the top two
levels may overlap in terms of the third level, that is, theses
features may be part of separate categories and instances.
We use k-means clustering at all levels to form a 3-level
hierarchy and visualize the clusters (at all three levels) via MDS.
For this, we compute the distances between all cluster pairs using
various distance metrics and then apply MDS to flatten the N-D
into 2D space. The closer the distance the more similar are the
clusters. In this paper we only report results from the Euclidian
distance similarity metric to calculate distances for an MDS map.
flows fall very close to each other, while more different elements
are placed farther from the rest. As the difference increases from
one dataset to the other, the distance in the MDS plot also
increases. It is interesting to note that 3D SIFT below correctly
increases the distance among flow 6 and the other members of the
group. Flows 4 and 5, although slightly different from other
elements, are in between flow 6 and the more similar flows 2 and
3. So, 3D SIFT succeeds in detecting this similarity in more detail
and places the flows closer to the main group of the family.
3
4
(a)
(b)
Figure 3. MDS plot of the Water Jet family, Euclidean of Centers
similarity metric. (a) 2D SIFT (b) 3D SIFT keypoint descriptors.
RESULTS
CONCLUSIONS
We have described a framework that more faithfully characterizes
volumetric data to support classification tasks. We demonstrated
these capabilities for synthetic flow data. Although not shown, we
have also successfully applied our framework to distinguish other
types of amorphous phenomena – within a category (level 2) and
across categories (level 1). In the latter we also detected common
groups of features (level 3). Finally, we have also characterized
real volumetric flow data (jet, shockwave, and vortex) and
medical data of various modalities and anatomies (where we used
a combination of SIFT / density histograms), with similar success.
So far we have built the models only by ways of k-means
clustering, which already worked quite well. A richer set of object
dichotomies could likely be obtained via probabilistic techniques,
such as EM. We are currently extending our framework to
associate the learned object and feature models with information
on appropriate rendering algorithms and parameters, learned from
user studies or automated observers, or a combination of these.
We have used our framework with 2-dimensional synthetic flow
data of four different families (synonymous to categories) of
flows: water jet, smoke, fire and flame. Each family has several
members, each one different from the others with respect to scale,
orientation and other characteristics such as speed, randomness,
weight, strength, etc. All series were generated with the program
Wondertouch Particle Illusion 3.0 [9]. We used this program since
it allowed us to generate a highly diverse set of amorphous
phenomena, which we recorded as a sequence of images, one for
each frame. 2D SIFT was then applied to each frame separately
and all keypoint descriptors were concatenated forming one single
set of feature descriptors for the entire sequence. In addition, 3D
SIFT was also performed on the volume assembled by stacking all
frames of the flow in order. In such cases, the depth of the volume
reflected the number of frames, while the width and height
correspond to the width and height of the flow sequence.
Figure 2 depicts all flows used in our experiments. Although
almost imperceptible, the differences between flows 1, 2 and 3 of
the Fire family rely on the speed and angle of emission of the
particles source. The same applies to flows 1, 3 and 4 of the
Smoke family and flows 1, 2, 3 and 4 of the Water Jet family.
Space constraints do not permit to show all results we have
obtained with our system – we restrict ourselves here to the water
jet data. Figure 3 shows the MDS plots of our feature vector
clustering (flows 1 and 7 were discarded to unclutter the picture
since flow 1 is very similar to flows 2, 3 and flow 7 is somewhat
different all other flows). Applying 2D SIFT to each flow and
visualizing the MDS map for each family, one notices that similar
REFERENCES
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
Figure 2. Flow data used in our experiments.
2
W. Cheung, G. Hamarneh, "N-SIFT: N-dimensional scale invariant
feature transform for matching medical images," IEEE Intern. Symp.
on Biomedical Image Processing (ISBI), pp. 720-723, 2007.
O. Gilson, N. Silva, P. Grant, M. Chen, "From web data to
visualization via ontology mapping" Computer Graphics Forum,
27(3):959-966, 2008.
H. Jänicke, A. Wiebel, G. Scheuermann, W. Kollmann, "Multifield
Visualization Using Local Statistical Complexity," IEEE Trans. on
Visualization and Computer Graphics, 13(6):1384-1391, 2007.
J. Kniss, G. Kindlmann, C. Hansen, "Multidimensional Transfer
Functions for Interactive Volume Rendering," IEEE Transactions on
Visualization and Computer Graphics, 8(3):270-285, 2002.
D. Lowe, "Distinctive Image Features from Scale-Invariant
Keypoints," Intern. Journal of Computer Vision, 60(2):91-110, 2004.
F. Pinto, C. Freitas, "Design of Multidimensional Transfer Functions
Using Dimensional Reduction," EuroVis, pp. 130-137, 2007.
P. Sereda, A. Vilanova Bartrolí, I. Serlie, F. Gerritsen, "Visualization
of Boundaries in Volumetric Data Sets Using LH Histograms,” IEEE
Trans. on Visualization Computer Graphics, 12(2): 208-218, 2006.
F. Tzeng, E. Lum, K.-L. Ma, "An Intelligent System Approach to
Higher-Dimensional Classification of Volume Data," IEEE Trans.
on Visualization and Computer Graphics, 11(3):273-284, 2005.
Wondertouch Particle Illusion 3.0. http://www.wondertouch.com/