Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
153 Y. Zhang, A. Koschan, and M. Abidi, "Superquadrics Based 3D Object Representation of Automotive Parts Utilizing Part Decomposition," Proc. of SPIE 6th International Conference on Quality Control by Artificial Vision, TN, Vol. 5132, pp. 241-251, Gatlinburg, TN, May 2003. Superquadrics based 3D object representation of automotive parts utilizing part decomposition Yan Zhang∗ , Andreas Koschan, and Mongi Abidi 334 Ferris Hall, Dept. Electrical and Computer Engineering University of Tennessee, Knoxville, TN 37996-2100 ABSTRACT We present a new superquadrics based object representation strategy for automotive parts in this paper. Starting from a 3D watertight surface model, a part decomposition step is first performed to segment the original multi-part objects into their constituent single parts. Each single part is then represented by a superquadric. The originalities of this approach include first, our approach can represent complicated shapes, e.g., multi-part objects, by utilizing part decomposition as a preprocessing step. Second, superquadrics recovered using our approach have the highest confidence and accuracy due to the 3D watertight surfaces utilized. A novel, generic 3D part decomposition algorithm based on curvature analysis is also proposed in this paper. The proposed part decomposition algorithm is generic and flexible due to the popularity of triangle meshes in the 3D computer community. The proposed algorithms were tested on a large set of 3D data and experimental results are presented. The experimental results demonstrate that our proposed part decomposition algorithm can segment complicated shapes, in our case automotive parts , efficiently into meaningful single parts. And our proposed superquadric representation strategy can then represent each part (if possible) of the complicated objects successfully. Keywords: Superquadrics, 3D object representation, automotive parts, 3D part decomposition 1. INTRODUCTION Object representation denotes representing real world objects with known graphic or mathematic primitives that can be recognized by computers. This research has numerous applications for object-related tasks in areas including computer vision, computer graphics, reverse engineering, etc. The shape of an object can be represented by three levels of primitives in terms of the dimensional complexity: volumetric primitives, surface elements, and contours. The primitive selected to describe the object depends on the complexity of the object and the tasks involved. As the highest level primitives, volumetric primitives can better represent global features of an object with a significantly reduced amount of information compared with surface elements and contours. In addition, they have the ability to achieve the highest data compression ratio without losing the accuracy of the raw data. For these reasons volumetric primitives are the most efficient features to represent objects in 3D. The primarily used volumetric primitives include generalized cylinders, geons, superquadrics, etc 1 . As a subclass of generalized cylinders, superquadrics are a family of geometric solids, which can be interpreted as a generalization of basic quadric surfaces and solids. With only a few parameters, they can represent a large variety of standard geometric solids as well as smooth shapes. This makes superquadrics very convenient for object representation. Superquadrics are also very efficient for representing three-dimensional surface data. Opposed to a mesh representation of an object with thousands of triangles, the same object may be represented by a single superquadric or a small set of superquadrics which are uniquely defined by 11 parameters each. This compact object representation can be efficiently used for object recognition to aid, for example, automated depalletizing of industrial parts or robot-guided bin picking of mixed nuclear waste in a hazardous environment. The quality control of both tasks mentioned above can be enhanced by employing superquadrics. Furthermore, the registration of multi-view data is indispensable to measure the size of partially occluded objects or their distances from each other in several image-based quality control tasks. Superquadrics can be used to efficiently register range data of multi-object scenes with small overlap 2 . ∗ [email protected]; phone: (865) 974-9213; fax: (865) 974-5459 Proc. of SPIE Vol. 5132 241 Most early research on superquadric representation concentrated on representing single-part objects from single-view intensity or range images by assuming that the image has been pre-segmented into single-part simple objects 3-13 . This category of research focused on the data fitting processes including objective functions, fitting criteria measurements, convergence speed, etc. For complex, multi-part objects or scenes, there are two major types of approaches. The first type of methods incorporates an image segmentation step priori to the superquadric representation 11-15 . The other type of methods directly recovers superquadrics from a range image without pre-segmentation 16-19 . Compared with the early work on single-part object representation, these types of methods can represent more complex objects and have wider applications in related tasks including robotic navigation, object recognition, etc. However, there are several weaknesses for existing superquadric representation methods for complex, multi-part objects. First, exis ting methods cannot handle arbitrary shapes or occlusions in the scene. Fig.1 shows an example of the most complicated object that can be represented by superquadrics using existing methods 18 . Fig. 1. A real range image of a multi-part object obtained from 18. It can be observed that the range image shown in Fig.1 contains very few occlusions between different parts (objects) because the viewpoint from which the image was acquired has been very well selected. When an automotive part, a complex, multi-part object as shown in Fig.2, is of interest no existing methods can represent this part correctly because heavy occlusions cannot be avoided from any single viewpoint. (a) (b) Fig. 2. Images of a distributor cap. (a) A color picture and (b) a set of 3D calibrated range data from a single view. The second weakness of existing methods is they only utilize single-view images. Again, for the automotive part shown in Fig. 2 (a), it is too difficult to choose an optimal viewpoint from which all the parts are visible due to self-occlusions 242 Proc. of SPIE Vol. 5132 and occlusions as shown in Fig. 2 (b). In addition, the confidence of recovered superquadrics is low due to incomplete single-view data utilized and the accuracy of the recovered models highly depends on the viewpoint used to acquire the data. How complicated, multi-part objects can be represented by superquadrics with high confidence and accuracy remains unknown from the literature. In this paper, we propose an efficient strategy to represent multi-part objects with superquadrics utilizing part decomposition as a preprocessing step. We also present a novel 3D part decomposition algorithm based on curvature analysis. Experiments were conducted on a large number of 3D triangulated surfaces. The remainder of this paper proceeds as follows. In Section 2, a superquadric representation method is proposed for multi-part objects. In Section 3, the 3D part decomposition algorithm is proposed. The experimental results are presented in Section 4. Finally, conclusions are presented in Section 5. 2. SUPERQUADRIC REPRESENTATION OF MULTI-PART OBJECTS UTILIZING PART DECOMPOSITION The diagram for this algorithm is illustrated in Fig. 3. Part 1 A multi-part object 3D part decomposition Single SQ recovery SQ 1 Single SQ recovery SQ 2 …… Part N Single SQ recovery SQ N Fig. 3. Diagram of the proposed superquadric representation strategy utilizing part decomposition. Beginning with a complete 3D watertight surface composed of triangle meshes, we propose a part decomposition algorithm to segment the model into single parts. Next, each single part is fitted with a superquadric model. The reasons that we use complete, watertight 3D triangulated models as input data include first, the complete 3D model can provide sufficient information for superquadric fitting and recovered superquadrics, therefore, have higher confidence when compared with single-view images. Second, both the proposed representation algorithm and the proposed part decomposition algorithm are generic and very flexible due to the triangle meshes used since triangle meshes have been the standard surface representation elements in many computer-related areas. A triangulation step is needed if only unstructured 3D point clouds are provided. 2.1 Introduction to superquadrics Shapes superquadrics can represent are shown in Fig. 4. The implicit definition of superquadrics is expressed as 20 ε1 2 2 2 ε 2 ε2 ε2 ε1 x y z F ( x , y , z) = + + = 1, ε 1 , ε 2 ∈ (0,2). a a2 a3 1 (1) Proc. of SPIE Vol. 5132 243 Fig. 4. Shapes represented by superquadrics with various shape parameters. Where ( x, y, z) represents a surface point of the superquadric, ( a1, a2 , a3 ) represent sizes in the ( x, y, z) directions, and (ε1 , ε 2 ) represent shape factors. To represent a superquadric model in the world coordinate system, 11 parameters are needed. They are summarized as ∧ = ( a1 , a2 , a3 , ε1 , ε 2 , φ ,θ , ϕ , p x , p y , p z ) (2) Most approaches define an objective function and find the parameters of superquadrics through minimizing this objective function. The mostly commonly used objective function is 1 N G ( ∧) = a1a2 a3 ∑ ( F ε1 ( xc , y c , z c ) − 1) 2 . i =1 The Levenberg-Marquardt method efficiency. 20 (3) has been primarily used to minimize the objective function due to its stability and 3. CURVATURE BASED 3D PART DECOMPOSITION Many tasks in computer vision, computer graphics, and reverse engineering are performed based on objects or models. Those object-based tasks become extremely difficult when the object is complicated, e.g., it contains multiple parts. Part decomposition can simplify the original task performed on multi-part objects into several subtasks each dealing with their constituent single, much simpler parts 21, 22 . While a significant amount of research for part decomposition of 2D intensity or 2.5D range images has been conducted over the last two decades 23-25 , little effort has been made on part segmentation of 3D data 26, 27 . Therefore, a novel 3D part decomposition algorithm is proposed in this paper. Fig. 5 illustrates the difference between region segmentation and part decomposition. The cylinder is seen as segmented into three surfaces by a region segmentation algorithm but remains one part for a part decomposition algorithm. Part decomposition is more appropriate for high-level tasks such as object recognition, especially for 3D models. The diagram of the proposed part decomposition algorithm is shown in Fig. 6. 244 Proc. of SPIE Vol. 5132 Fig. 5. The difference between region segmentation and part decomposition. 3D triangulated surface Curvature estimation Boundary detection Region growing Postprocessing Fig. 6. Diagram of the proposed 3D part decomposition algorithm. The proposed algorithm consists of four major steps, Gaussian curvature estimation, boundary detection, region growing, and postprocessing. Boundaries between two articulated parts are composed of points with highly negative curvature based on the transversality regularity 21, 22 . These boundaries are therefore detected by thresholding estimated Gaussian curvatures for each vertex. A component labeling operation is then performed to grow non-boundary vertices into parts. A postprocessing step is performed to assign non-labeled vertices to one of the parts according to the sign of curvature and the smallest distance to neighbor vertices. Additionally, a threshold, that is the minimum number of vertices of a part, is used to eliminate parts composed of too few vertices. This part decomposition algorithm is summarized as follows. {Algorithm 1 (3D Part decomposition)} {Input:} Triangulated 3D surface. {Step 1.} Compute Gaussian curvature for each vertex on the surface. {Step 2.} Label vertices of highly negative curvature as boundaries using a pre-specified threshold. Label the remaining vertices as seeds. {Step 3.} Eliminate isolated vertices. {Step 4.} Perform iterative region growing on each seed vertex. {Step 5.} Assign non-labeled vertices to parts, and eliminate parts having less than a pre-specified number of vertices. {Output:} Point clouds (3D unstructured data points) of various decomposed single parts written to separate 3D files. The major steps of this part decomposition algorithm are described respectively in the following sections. Proc. of SPIE Vol. 5132 245 3.1 Gaussian curvature estimation and boundary detection The mesh-based curvature estimation method proposed in 28 is implemented in this work to estimate Gaussian curvature for each vertex of a triangle mesh. Since only the sign of surface curvature is used in our proposed decomposition, this simple curvature estimation is sufficient for our application. As shown in Fig. 7, Gaussian curvature of the vertex p is computed as 28 N K ( p) = 3( 2π − ∑ θ i ) i =1 N δ 2 ( p − pi ) , (4) ∑ Ai i =1 where p represents the point of interest, pi represents one of the mesh neighbors of the point p, and Ai represents the area of the corresponding triangle. θ i represents the interior angle of the triangle at p, and δ is the Dirac delta function. Fig. 7. Curvature calculation for the vertex p utilizing triangle mesh information. After Gaussian curvature is obtained for each vertex on the surface, a specified threshold is applied to label vertices as boundary or seed. Vertices of highly negative curvature are labeled as boundaries between two parts while the rest are labeled as seeds belonging to potential object parts according to the transversality regularity 21 . The threshold is critical and affects the performance of later region growing. This threshold is determined in a heuristic way depending on object and mesh resolution. According to the label of each vertex, isolated vertices are then removed. As noted previously, two types of isolated vertices defined in this work include: (i) a point which is labeled as boundary while all of its neighbors are labeled as seeds and (ii) a point which is labeled as a seed while all of its neighbors are labeled as boundary. The labels of isolated vertices are changed to be the same as their neighbors. 3.2 Region growing and postprocessing After the vertices are labeled, a region-growing operation is performed on each vertex labeled as seed. Fig. 8 shows triangle meshes around the point p. To illustrate the region growing process, a two-ring neighborhood of the mesh around point p is shown in this figure. Region growing is performed as follows. Starting from a seed vertex p, the unique region label is first assigned to the vertex. Second, all the neighbors pi labeled as seeds initially are then labeled with the same region number as the point p. The same labeling process is performed for each neighbor pi to label vertices pij . This process terminates when the grown region is surrounded by boundary vertices, i.e., the neighbors of the edge vertices of the region are all labeled as boundaries. This process is repeated for each seed vertex, but not for a vertex which has been grown and already labeled uniquely. After the seed vertices are assigned new labels, a post-processing step is performed for each boundary vertex. Given a seed point x, its mesh neighbors xi are first sorted in ascending order based on their Euclidean distance to the point x. Next, the vertex x k is selected from the ordered neighbors where x k is the first vertex in the list with a region label and not a boundary label. The boundary vertex x is then labeled the same as the vertex x k , i.e., the label of x is changed from boundary to the unique region label of x k . Finally, with the exception of a few missing vertices, each vertex should now have a region label and thus assigned to different object parts. 246 Proc. of SPIE Vol. 5132 Fig. 8. Region growing process for the vertex p. Finally, a post-processing step is performed to assign the remaining vertices that have not been labeled. For example, the vertex p is an unlabeled vertex and needs further post-processing. Assuming that pi (i = 1, 2, ..., N) represents a neighbor vertex of the point p, the neighbor vertices are first selected if they have the same sign of curvature as that of the vertex p and belong to one of the segmented parts. Next, among those neighbor vertices, the vertex which has the smallest Euclidean distance to the vertex p is selected as a target vertex. For example, the vertex p1 is assumed to be the target vertex of the vertex p. Finally, the vertex p is assigned the same label of the vertex p1 , i.e., the same segmented part. Furthermore, parts that are composed of fewer vertices than a specified threshold are merged with adjacent regions. 4. EXPERIMENTAL RESULTS Experimental results on both part decomposition and subsequent superquadric representation are shown in this section. Fig. 9 shows a 3D triangulated teapot model, the model with labeled negative Gaussian curvatures, and the decomposed parts of the teapot. Boundary vertices with negative Gaussian curvature are labeled blue as shown in Fig. 9 (b). Fig. 9 (c) labels decomposed parts in different colors and it can be observed that all the parts including the cap, the handle, the body, the spout, and the small block at the bottom are correctly decomposed from the teapot. (a) (b) (c) Fig. 9. Part decomposition of a teapot. (a) The 3D triangulated surface of a teapot, (b) the model with curvature labeled as blue, and (c) decomposed parts. Proc. of SPIE Vol. 5132 247 Fig. 10 shows results of both part decomposition and subsequent superquadric representation for the distributor cap. The 3D triangulated surface for the object was reconstructed from multi-view range images using the RANGER scanner 29 . Fig. 10 (c) shows that the five small cylinders, the base, two small screws of the distributor cap are all correctly segmented from the original distributor cap model. The only part that is not decomposed from the distributor cap is the long small cylinder linked to the base. Instead, this small cylinder is assigned to the base part after the decomposition. This is because the mesh smoothing step during the surface reconstruction step also smoothes out the boundary between the small cylinder and the base of the distributor cap. Therefore, the small cylinder is decomposed into the same part as the base instead of a single part of its own. Fig. 10 (d) shows five recovered and rendered superquadrics for the small cylindrical parts on the top of the distributor cap. The recovered superquadric parameters and the ground truth values of one of the small cylinders are shown in Table 1 as an example. It can be observed that the superquadrics have correct size and shape information when compared with the ground truth parameters of the object. (a) (b) (c) (d) Fig. 10. Part decomposition and superquadric representation of an automotive part . (a) The 3D triangulated surface of a distributor cap, (b) the model with curvature labeled as blue, (c) decomposed parts, and (d) recovered and rendered superquadrics. 248 Proc. of SPIE Vol. 5132 Table 1. Recovered superquadric parameters and ground truth values for one of the small cylinders shown in Fig. 10 (d). GT: ground truth values. RP: recovered parameters. Units: mm. a1 a2 a3 ε1 ε2 GT 15.2 15.6 20.1 0.1 1.0 RP 16.45 15.67 20.42 0.12 0.96 Fig. 11 shows a reconstructed 3D model of a water neck, the model with labeled negative Gaussian curvatures, the decomposed parts, and superquadric representation. Fig. 11 (c) shows that the cylindrical handle, the ball, the base, the small cylinder next to the handle, and several small screws of the water neck are all correctly segmented from the original model although the boundaries are detected redundantly. This demonstrates that the proposed part decomposition algorithm can handle 3D triangulated models with various smoothness and resolutions very well. And it is insensitive to detected boundaries. Fig. 11 (d) shows three recovered and rendered superquadrics for the handle, the ball, and the small cylinder. The recovered superquadric parameters and the ground truth values for the handle of the water neck are shown in Table 2. Again, we observe correct size and orientation information for the recovered superquadrics. (a) (c) (b) (d) Fig. 11. Part decomposition and superquadric representation of an automotive part. (a) The 3D triangulated surface of a waterneck, (b) the model with curvature labeled as blue, (c) decomposed parts, and (d) recovered and rendered superquadrics. Proc. of SPIE Vol. 5132 249 Table 2. Recovered superquadric parameters and ground truth values for handle shown in Fig. 11 (d). GT: ground truth values. RP: recovered parameters. Units: mm. a1 a2 a3 ε1 ε2 GT 39.7 39.4 17.6 0.1 1.0 RP 40.23 40.58 66.83 0.13 0.98 5. CONCLUSIONS This paper proposed an effective approach to represent multi-part objects with superquadrics utilizing part decomposition as a preprocessing step. The advantages of our method include: (i) the proposed representation approach can represent very complicated, multi-part objects by decomposing them into their constituent single parts, and (ii) superquadrics recovered using the proposed representation method have the highest confidence and accuracy due to the reconstructed 3D triangulated surface utilized. The ambiguities contained in single-view data have been eliminated during the surface reconstruction step by utilizing multi-view data sets . A novel 3D part decomposition algorithm is also proposed to decompose compound objects represented by triangle meshes into their constituent parts based on curvature analysis. Considering the fact that the surface polygonal mesh has been a standard representation form in computer vision and computer graphics, the proposed part decomposition algorithm is generic, flexible, and has wide application in high level computer vision tasks such as shape description, object recognition, and 3D reconstruction. Furthermore, the algorithm can handle a large number of triangle meshes (over 100,000) in only seconds when implemented on a SGI Octane workstation. Even though not every decomposed part of the original object can be represented by a superquadric due to its representation limitation, the superquadric parameters recovered for some parts of the object will significantly benefit the object recognition task. 6. ACKNOWLEDEGMENTS This work was supported by the University Research Program in Robotics under grant DOE-DE-FG02-86NE37968, by the DOD/TACOM/NAC/ARC Program, R01-1344-18, and by FAA/NSSA Program, R01-1344-48/49. 7. REFERENCES 1. 2. 3. 4. 5. 6. 7. 8. 250 F. Solina and R. Bajcsy, “Recovery of Parametric Models from Range Images: The Case for Superquadrics with Global Deformations”, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 12, No.2, pp. 131-147, 1990. Y. Zhang, J. Paik, A. Koschan, and M. Abidi, “3-D Object Representation from Multi-view Range Data Applying Deformable Superquadrics”, Int’l Conf. Pattern Recognition, Vol. III, pp. 273-276, 2002. A. Gross and T. Boult, “Error of Fit Measures for Recovering Parametric Solids”, 2nd Int’l Conf. Computer Vision, pp. 690-694, 1988. A. P. Pentland and S. Sclaroff, “Closed-form Solutions for Physically Based Shape Modeling and Recognition”, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 13, No.7, pp. 715-729, 1991. A. P. Pentland, “Automatic Extraction of Deformable Part Models”, International Journal of Computer Vision, Vol. 4, pp. 107-126, 1990. M. Leyton, “A Process-grammar for Shape”, Artificial Intelligence, Vol. 34, 99. 213-247, 1988. D. Terzopoulos, A. Witkin, and M. Kass, “Constraints on Deformable Models: Recovering 3D Shape and Nongrid Motion”, Artificial Intelligence, Vol. 36, pp. 91-123, 1988. D. Metaxas and D. Terzopoulos, “Constrained Deformable Superquadrics and Nongrid Motion Tracking”, Int’l Conf. Computer Vision and Pattern Recognition, pp. 337-343, 1991. Proc. of SPIE Vol. 5132 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. D. Terzopoulos and D. Metaxas, “Dynamic 3-D Models with Local and Global Deformations: Deformable Superquadrics”, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 13, No.7, pp. 703-714, 1991. Eric Bardinet, L. D. Cohen, and N. Ayache, “Fitting 3D Data Using Superquadrics and Free-form Deformations”, the 12th IAPR Int’l Conf. on Pattern Recognition, Vol. 1, pp. 79-83, 1994. D. Metaxas, E. Koh, and N. I. Badler, “Multi-level Shape Representation Using Global Deformations and Locally Adaptive Finite Elements”, International Journal of Computer Vision, Vol. 25, No.1, pp. 49-61, 1997. D. Metaxas, “Physics-based Deformable Models: Applications to Computer Vision, Graphics and Medical Imaging”, Kluwer Academic Press, 1997. M. Pilu, A. W. Fitzgibbon, and R. B. Fisher, “Training PDMs on Models: the Case of Deformable Superellipses”, the British Machine Vision Conference, pp. 373-382, 1996, Edinburgh. K. Wu and M. D. Levine, “Recovering Parametric Geons from Multiview Range Data”, Int’l Conf. Computer Vision and Pattern Recognition, pp. 159-166, 1994. H. Zha, T. Hoshide, and T. Hasegawa, “A Recursive Fitting-and-splitting Algorithm for 3D Object Modeling Using Superquadrics”, 14th Int’l Conf. Pattern Recognition, pp. 658-662, 1998. Y. Hu and W. Wee, “Robust 3-D Part Extraction from Range Images with Deformable Superquadric Models”, SPIE Conf. signal processing, sensor fusion and target recognition, pp. 524-535, 1995. A. Leonardis, A. Jaklic and F. Solina, "Superquadrics for Segmentation and Modeling Range Data", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 19, pp. 1289-1295, 1997. A. Jaklic, A. Leonardis, and F. Solina, “Segmentation and Recovery of Superquadrics”, Kluwer Academic Publishers, 2000. A. H. Barr, “Superquadrics and Angle-preserving Transformation”, IEEE Trans. Computer Graphics and Applications, Vol. 1, pp. 11-23, 1981. W. Press, W. Vetterling, S. Teukolsky and B. Flannery, “Numerical Recipes in C: the Art of Scientific Computing”, Cambridge Press, U.S, 1992. D. Hoffman and W. Richards, "Parts of Recognition", Cognition, Vol. 18, pp.65-96, 1984. A. Pentland, "Part Segmentation for Object Recognition", Neural Computation, Vol. 1, pp. 82-91, 1981. H. Rom and G. Medioni, "Part Decomposition and Description of 3D Shapes", Int'l Conf. Pattern Recognition, pp. 629-632, 1994. M. Bennamoun, “A Contour-based Part Segmentation Algorithm”, Int'l Conf. Acoustics, Speech, and Signal Processing, Vol. 5, pp. 41-44, 1994. K. Koara and A. Nishikawa and O. Miyazaki, “Hierarchical Part Decomposition Method of Articulated Body Contour”, Int'l Conf. Intelligent Robots and systems, Vol. 3, pp. 2055--2060, 2000. K. Wu and M. D. Levine, “3D Part Segmentation using Simulated Electrical Charge Distributions”, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 19, pp. 1223-1235, 1997. A. Mangan and R. Whitaker, “Partitioning 3D Surface Meshes Using Watershed Segmentation”, IEEE Trans. Visualization and Computer Graphics, Vol. 5, No. 4, pp. 308-321, 1999. C. Lin and M. Perry, "Shape Description Using Surface Triangulation", Conf. Comp uter Vis ion: Representation and Control, pp.38-43, 1982. User Documentation: MAPP2500 Ranger PCI System, Version 1.6, Integrated Vision Products, Sweden, 2000. Proc. of SPIE Vol. 5132 251