Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
High-Dimensional Feature Descriptors to Characterize Volumetric Data Julia EunJu Nam, Mauricio Mauer, and Klaus Mueller Center for Visual Computing, Computer Science Department, Stony Brook University ABSTRACT Our feature vectors, however, are too high-dimensional for a visual analysis on this low level, and also, visual clustering is counter to our goal of model-driven volume data classification. Volumetric data offer considerably more low-level information than the density and gradient values traditionally employed in transfer function-guided volume visualization. We propose the use of a richer set of low-level feature descriptors to allow a more sensitive and refined characterization of the data. The ensemble of these feature descriptors then forms a distinct data signature that can describe, classify, and categorize a dataset at levels of detail. 1 2 APPROACH We chose to use two statistical low-level feature descriptors, and not descriptors based on patterns such as [3]. Our descriptors are (i) density histograms to represent the overall texture statistics and (ii) gradient histograms to capture the variation, perceptual effects of these. Both feature descriptors are also designed to be insensitive to affine transformations, such as scale, rotation, and translation, to support feature and object mining in large datasets. A global histogram is not descriptive enough as a classifier, and this motivated the use of more feature-oriented histograms. We did not attempt automated segmentation which is still an open research area, to confine the extent of histograms within object boundaries. Instead, we aimed for a representation that has good potential to capture the essence of an object, and yet is automatic to compute. This led us to calculate the density signatures in local histograms at a hierarchy of window sizes, to enable the detection of feature density statistics at multiple levels of scales. For this, we normalize the histograms. We also perform optional smoothing at the histogram level (not the image level) to reach independence of the sampling process that generated the data. Finally, rotation invariance is achieved by constructing the histograms within slightly overlapping radial regions. To encode the gradient signatures we used the SIFT feature descriptor [5]. The SIFT algorithm consists of two phases: (i) the detection of critical points (the keypoints) in scale-space and (ii) the encoding of these into keypoint descriptors. The keypoints are local extrema in a difference-of-Gaussians multiscale pyramid. They are detected by comparing a point’s neighbors Figure 1. The 2D SIFT on the current scale and the scales keypoint descriptor. above and below. Points having low contrast or poorly localized along an edge are then rejected limiting the set of keypoints to the most descriptive and salient ones, preserving manageability. The keypoint (feature) descriptor is an orientation histogram of local gradients, quantized into bins. It is constructed by computing the magnitude and orientation at each image sample point in a region around the keypoint location, weighted by a Gaussian function with σ equal to one half the width of the descriptor window to achieve a certain level of smoothing. The samples are then aggregated into 8-bin orientation histograms describing the neighborhood over a 4×4 matrix of subregions (Figure 1 shows an example). This histogram is stored into a 128-long feature vector, along with the scale ID and location. Further, the SIFT feature descriptor can be made orientation independent by aligning the descriptor and the gradient orientations by the keypoint orientation. SIFT has traditionally been used only in 2D in the context of images, but recently [1] have generalized the framework into higher dimensions, for INTRODUCTION Extracting knowledge from data requires an effective characterization and integration of the embedded low-level features. In volume visualization, this feature integration is often done explicitly by the user, aided by appropriate graphical rendering techniques and implemented within a responsive user interface. One measure of distinguishing these many rendering techniques is by the underlying feature model, whose parameters are captured in a feature vector. Here, the simplest model is purely sample-based, that is, just the sample value (we shall assume scalar densities for this paper, without loss of generality), which can be visually enhanced via suitable RGB mappings in transfer functions. The next step up is to assist the user in making the conjunctive feature integration. Shape is a strong cue, and to infer it, one can calculate gradients from the sample neighborhood. More advanced local computations can emphasize higher-level shape features, such as curvature, suggestive contours, and others. All of these features have in common that they have direct visual manifestations which can be readily brought out by their interaction with light. Thus, the underlying models have direct correspondences to familiar real-life visual artifacts. This limits the scope of the features that can be used for characterizing the data. The feature vectors are simply not expressive enough to classify the data autonomously at the required level of detail, thus requiring the sophisticated perceptual and cognitive capabilities of the human user. This is particularly troublesome in less coherent data, such as amorphous phenomena. We aim to make progress in this area by first deriving a set of rich feature vectors from the data and then using cluster analysis to learn the feature vectors relevant to the model describing the data. Here we are looking for features that are less visually intuitive, requiring computer analysis for their derivation and management. Once these rich feature vectors have been learned, they can be used for subsequent classification, mining and selection. In that regard, unlike other works in volume graphics based on machine learning [1] our focus is not segmentation, but model-based object-level classification, detection, categorization, and visual feature content communication. A further anticipated outcome is that such a model can then provide an ontology of the data, to be matched with a suitable volume rendering algorithm ontology for automated image generation. A mapping of this nature was recently successfully demonstrated by [1] within an information visualization context composed of graphs and plots. Our method uses computational cluster analysis, followed by an illustration of the results in a 2D plot obtained via MultiDimensional Scaling (MDS). Most visualization research is based on visual cluster analysis, where feature vectors produce scatter plots that are coupled with a transfer function interface [1][4][7]. 1 determining accurate feature point correspondences between medical volume datasets for object alignment. A 3D-SIFT feature is comprised of a 163 cubic voxel region around the keypoint, with 64 subregions and each summarized by an 8×8 histogram of voxel orientations. It uses two spherical coordinates (θ, φ) for parameterization to form the 3D-SIFT feature descriptor. Having captured the essential perceptual features using the two low-level feature descriptors, we now assemble these into highlevel constructs. Our goal is to learn a high-level representation of the sampled object, using its low-level features. Using amorphous phenomena data as an example, we distinguish between three levels. Level-1: categories (e.g., smoke vs. fire), level-2: category instances (e.g., turbulent smoke vs. laminar smoke), and level-3: groups of similar individual features. We note that the top two levels may overlap in terms of the third level, that is, theses features may be part of separate categories and instances. We use k-means clustering at all levels to form a 3-level hierarchy and visualize the clusters (at all three levels) via MDS. For this, we compute the distances between all cluster pairs using various distance metrics and then apply MDS to flatten the N-D into 2D space. The closer the distance the more similar are the clusters. In this paper we only report results from the Euclidian distance similarity metric to calculate distances for an MDS map. flows fall very close to each other, while more different elements are placed farther from the rest. As the difference increases from one dataset to the other, the distance in the MDS plot also increases. It is interesting to note that 3D SIFT below correctly increases the distance among flow 6 and the other members of the group. Flows 4 and 5, although slightly different from other elements, are in between flow 6 and the more similar flows 2 and 3. So, 3D SIFT succeeds in detecting this similarity in more detail and places the flows closer to the main group of the family. 3 4 (a) (b) Figure 3. MDS plot of the Water Jet family, Euclidean of Centers similarity metric. (a) 2D SIFT (b) 3D SIFT keypoint descriptors. RESULTS CONCLUSIONS We have described a framework that more faithfully characterizes volumetric data to support classification tasks. We demonstrated these capabilities for synthetic flow data. Although not shown, we have also successfully applied our framework to distinguish other types of amorphous phenomena – within a category (level 2) and across categories (level 1). In the latter we also detected common groups of features (level 3). Finally, we have also characterized real volumetric flow data (jet, shockwave, and vortex) and medical data of various modalities and anatomies (where we used a combination of SIFT / density histograms), with similar success. So far we have built the models only by ways of k-means clustering, which already worked quite well. A richer set of object dichotomies could likely be obtained via probabilistic techniques, such as EM. We are currently extending our framework to associate the learned object and feature models with information on appropriate rendering algorithms and parameters, learned from user studies or automated observers, or a combination of these. We have used our framework with 2-dimensional synthetic flow data of four different families (synonymous to categories) of flows: water jet, smoke, fire and flame. Each family has several members, each one different from the others with respect to scale, orientation and other characteristics such as speed, randomness, weight, strength, etc. All series were generated with the program Wondertouch Particle Illusion 3.0 [9]. We used this program since it allowed us to generate a highly diverse set of amorphous phenomena, which we recorded as a sequence of images, one for each frame. 2D SIFT was then applied to each frame separately and all keypoint descriptors were concatenated forming one single set of feature descriptors for the entire sequence. In addition, 3D SIFT was also performed on the volume assembled by stacking all frames of the flow in order. In such cases, the depth of the volume reflected the number of frames, while the width and height correspond to the width and height of the flow sequence. Figure 2 depicts all flows used in our experiments. Although almost imperceptible, the differences between flows 1, 2 and 3 of the Fire family rely on the speed and angle of emission of the particles source. The same applies to flows 1, 3 and 4 of the Smoke family and flows 1, 2, 3 and 4 of the Water Jet family. Space constraints do not permit to show all results we have obtained with our system – we restrict ourselves here to the water jet data. Figure 3 shows the MDS plots of our feature vector clustering (flows 1 and 7 were discarded to unclutter the picture since flow 1 is very similar to flows 2, 3 and flow 7 is somewhat different all other flows). Applying 2D SIFT to each flow and visualizing the MDS map for each family, one notices that similar REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] Figure 2. Flow data used in our experiments. 2 W. Cheung, G. Hamarneh, "N-SIFT: N-dimensional scale invariant feature transform for matching medical images," IEEE Intern. Symp. on Biomedical Image Processing (ISBI), pp. 720-723, 2007. O. Gilson, N. Silva, P. Grant, M. Chen, "From web data to visualization via ontology mapping" Computer Graphics Forum, 27(3):959-966, 2008. H. Jänicke, A. Wiebel, G. Scheuermann, W. Kollmann, "Multifield Visualization Using Local Statistical Complexity," IEEE Trans. on Visualization and Computer Graphics, 13(6):1384-1391, 2007. J. Kniss, G. Kindlmann, C. Hansen, "Multidimensional Transfer Functions for Interactive Volume Rendering," IEEE Transactions on Visualization and Computer Graphics, 8(3):270-285, 2002. D. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Intern. Journal of Computer Vision, 60(2):91-110, 2004. F. Pinto, C. Freitas, "Design of Multidimensional Transfer Functions Using Dimensional Reduction," EuroVis, pp. 130-137, 2007. P. Sereda, A. Vilanova Bartrolí, I. Serlie, F. Gerritsen, "Visualization of Boundaries in Volumetric Data Sets Using LH Histograms,” IEEE Trans. on Visualization Computer Graphics, 12(2): 208-218, 2006. F. Tzeng, E. Lum, K.-L. Ma, "An Intelligent System Approach to Higher-Dimensional Classification of Volume Data," IEEE Trans. on Visualization and Computer Graphics, 11(3):273-284, 2005. Wondertouch Particle Illusion 3.0. http://www.wondertouch.com/