Download Untitled

Computer-aided Interactive Object Delineation
Using an Intelligent Paintbrush Technique
Frederik Maes? , Dirk Vandermeulen, Paul Suetens, Guy Marchal
Laboratory for Medical Imaging Research??
Katholieke Universiteit Leuven, Belgium.
Department of Electrical Engineering, ESAT,
Kardinaal Mercierlaan 94, B-3001 Heverlee, Belgium.
Department of Radiology, University Hospital Gasthuisberg,
Herestraat 49, B-3000 Leuven, Belgium.
Abstract. A method for fast generic object segmentation is presented
that allows the user to quickly paint the object of interest in the image using an intelligent paintbrush. This intelligence is based on a partitioning of the image in segmentation primitives, which are computed
automatically by merging watershed regions with similar image intensity
distribution using the Minimum Description Length principle. We show
results for Magnetic Resonance images of the heart and of the brain and
for Computerized Tomography images of the abdomen.
1 Introduction
Segmentation of medical images is a necessary requirement for three-dimensional
(3D) visualization and quantication of the anatomical structures of interest.
There are numerous clinical applications, for instance in surgery and radiotherapy treatment planning and in quantitative diagnosis. Because completely manual object delineation on large 3D datasets is very tedious and time-consuming,
computer-assistance is needed to facilitate the segmentation task in clinical routine. Fully automatic methods, relying on a priori knowledge such as shape and
intensity models, are precluded by the complexity of the images, the diversity of
the objects of interest and the occurrence of pathological deformations.
In this paper, we present a semi-automatic segmentation method, whereby
the user, assisted by the computer, interactively segments the object of interest
by colouring it in the image using an intelligent paintbrush [1]. This approach is
based on a partitioning of the image in primitive regions and assumes that the
object of interest is the union of one or more of these regions. In that case, only
one point of each region that belongs to the object needs to be selected to completely segment it, which dramatically speeds up the interactive segmentation
process.
Frederik Maes is Aspirant of the Belgian National Fund of Scientic Research
(NFWO). Corresponding author. Email: [email protected]
?? Directors: Andr
e Oosterlinck & Albert L. Baert
?
78
Section 2 describes how a relevant partitioning is computed automatically
using a fast watershed-like procedure. In order to reduce the need for user interaction in the paintbrush tool, regions that are likely to belong to the same object
need to be merged selectively while maintaing the relevant object boundatries.
Section 3 describes how this is achieved using the Minimum Description Length
(MDL) criterion [9]. Section 4 illustrates the power of the method by showing
results for Magnetic Resonance (MR) images of the heart and of the brain and
for Computerized Tomography (CT) images of the abdomen.
2 Construction of primitive image regions
Boundaries between dierent objects in an image are often characterised by a
signicant change in image intensity across the boundary. Such boundaries correspond to ridges in the gradient magnitude image and the contours of the objects
coincide with the crest lines of these ridges. Therefore, a partitioning of the image in meaningful segmentation entities can be obtained by detecting these crest
lines using the watershed of the gradient magnitude image. A true watershed
transformation, however, is intractable, because it is a computationally expensive operation [10]. Instead, we construct pseudo-watershed regions by tracking
downhill Maximum Gradient Paths (MGP) [6], starting from each point in the
image and terminating in a local gradient minimum.
The gradient of the image intensity is computed using the Sobel [4] operator,
after Gaussian smoothing of the original image in order to reduce the inuence of
noise on the gradient computation. The downhill MGP of each pixel p is tracked
by recursively selecting the pixel q in the 8-connected (in 2D, or 26-connected in
3D) neighbourhood of p for which the gradient magnitude is minimal. If more
than one such pixel q exists, we systematically select the rst one found. The
MGP terminates in the pixel m if there is no pixel in its neighbourhood with
strictly smaller gradient magnitude. Each such pixel m is marked as a local
minimum of the gradient magnitude and given a distinct label. If a minimum
is reached in whose neighbourhood another minimum with the same gradient
magnitude value has already been detected, the new minimum is disregarded by
giving it the same label as the neighbouring one in order to reduce the number
of regions. Each pixel in the image is assigned the label of the minimum its
downhill MGP terminates in.
A pixel p is a boundary pixel of the region R(p) it belongs to if there is at least
one pixel q in the 4-connected (in 2D, or 6-connected in 3D) neighbourhood of p
that does not belong to R(p). This pixel q is a boundary pixel of its region R(q)
and the common pixel boundary of p and q is a boundary line element (in 2D,
or surface element in 3D) of R(p) and R(q). The boundary between two regions
is the union of all their common boundary elements. We now argue that the sodened downhill region boundaries coincide with the crest lines of the gradient
magnitude. Indeed, if the pixel p is located on the crest line of a gradient ridge,
the gradient of the image gradient magnitude points in opposite directions on
both sides of the ridge, ipping direction in p. Therefore, the neighbourhood of
79
p will contain at least two pixels that are mapped on two dierent local minima
on opposite sides of the crest and p is a boundary pixel of its downhill region.
The result of this algorithm is an oversegmentation of the image, as illustrated
in Fig. 1. The correspondence of the watershed boundaries to object contours
can be compared with the result of an edge detector, such as the Canny edge
detector [2] (Fig. 1c). A typical problem of Canny edge detection is that the
edges found are not closed and that additional processing for explicit detection
of junctions and linking of edge segments is usually required. The watershed
method on the contrary, has no problem detecting junctions and, because of the
region based approach, all boundaries are closed implicitly, regardless of noise
or low intensity contrast.
[t]
a
b
c
Fig. 1. (a) Short-axis MR cross-section of the heart at diastole (200 times 200). (b)
Watershed regions of the gradient magnitude of the smoothed image ( = 2). (c) Canny
edges thresholded at 10% of the maximum edge strength (sigma = 2).
3 Selectively merging primitive regions
Because the image intensity is often rather homogeneous within one object and
signicantly dierent between objects, the oversegmentation of the watershed
can be reduced while preserving the relevant object boundaries by selectively
merging regions with similar intensity distributions. Apart from intensity similarity, merging small regions that have a large common boundary compared to
their size should be preferred, because this reduces the oversegmentation more
than merging regions that only share a small common boundary.
These criteria can be formulated mathematically as an optimization problem
using the Minimum Description Length (MDL) principle: the optimal image
partitioning is the one for which the total number of bits required to describe
the image data in terms of a specied descriptive language is minimal [8, 9]. In
our case, such a description consists of the number of bits needed to code the
80
image intensity of each pixel in each region and the number of bits needed to
code the region boundaries. In this section, we only discuss the 2D case, but
extension to 3D regions is straightforward.
We code the image data using separate code alphabets for each region Ri by
treating the original image intensity x of all pixels in Ri as a stochastic variable
with distribution pR (x). According to information theory [3], the number of bits
BI (Ri ) required to code the image intensity of all pixels in Ri using an optimal
coding scheme is given by BI (Ri ) = n(Ri )H (Ri ), with n(Ri ) being the number
of pixels in Ri and H (Ri ) the entropy of pR (x). The image intensity distribution
pR (x) is estimated from the histogram of the original image in the region Ri as
the convolution of with a Gaussian Parzen-window [4].
The boundary line of each region is coded by specifying one starting point
and a chain code of boundary line elements, such that the total number of bits
needed to code the region boundary information of the partitioning R is BB (R) =
NR (R) b1 + Nb (R) b2, with NR (R) being the number of regions in R, Nb (R)
the total boundary length, b1 the number of bits required to code the starting
point and b2 the number of bits required to code each element of the boundary
chain code. We take b1 = 2 8 = 16 for a 2D 256 256 image and b2 = log2 4 = 2,
as each 2D boundary element corresponds to one of 4 possible directions. The
total number of bits to code the image data from the
P image partitioning R using
this coding scheme is B (R) = BI (R) + BB (R) = i BI (Ri ) + BB (R).
If two neighbouring regions R1 and R2 are merged into a new region R3 , more
bits will be required to code the image intensity information of R3 that of R1
and R2 separately. However, the total description length might decrease because
no bits are needed to code the common boundary segment of R1 and R2 . The
description length gain B (R1 ; R2 ) of merging R1 and R2 into R3 is given by
B (R1 ; R2 ) = n(R1 )H (R1 ) + n(R2 )H (R2 ) , n(R3 )H (R3 ) + b1 + nb(R1 ; R2 ) b2 ,
with nb (R1 ; R2 ) being the length of the common boundary of R1 and R2 .
The merging process proceeds iteratively by selecting at each step the couple
of neighbouring regions with the largest positive description length gain, merging
these into a new region and updating the neighbour relations. This procedure is
repeated as long as regions can be merged with positive description length gain
and merging terminates at a local minimum of the description length function.
Figure 2 shows the result of this procedure on the partitioning of Fig. 1b. This
partitioning is used as input to the intelligent paintbrush tool.
i
i
i
4 Results obtained with the intelligent paintbrush tool
The original image and the region label image of the primitive region partitioning
are used as input to the intelligent paintbrush tool. Only the original image is
presented to the user, while the region label image remains hidden. When the
user selects a point on the screen by clicking the left mouse button, the region
the selected point belongs to is extracted from the region label image and shown
in colour overlay on the screen. The user can erase coloured regions by clicking
81
[t]
a
b
c
Fig. 2. (a) Merged region boundaries for the image of gure 1(b). (b) Overlay of (a)
on the original image. (c) The myocardium segmented with the intelligent paintbrush
tool.
the right mouse button. Dragging the mouse with the left or right mouse button
pressed, colours or erases all regions the mouse pointer hits.
In the current implementation of this tool, the colour overlay is opaque and
hides the underlying image, which might make it dicult for the user to assess
the result of the segmentation. For clinical use, the colourings should be transparent and the user should be oered the choice of displaying the lled coloured
regions or only the boundary of their union. Manual editing of the contour of
the delineated region might also be needed to correct segmentation errors.
[t]
a
b
c
Fig. 3. (a) Original MR image of the brain (400 400). (b) Merged watershed partitioning. (c) Tumor segmented with the intelligent paintbrush.
Figure 2c shows the segmentation of the myocardium in the image of Fig. 1,
using the partitioning of Fig. 2a. This partitioning was computed automatically
from the original image in less than 5 seconds on an IBM RS/6000 workstation. It
82
takes the user about 5 seconds to complete the segmentation with the intelligent
paintbrush tool. Figure 3 shows the segmentation of a tumor in one slice of a
transversal 3D MR image of the brain. Slice by slice segmentation of the tumor
on all 33 slices takes the user less than 3 minutes. Figure 4 shows one slice of a
3D 512 512 CT image of the abdomen and the segmentation of the prostate
in this image.
[t]
a
b
c
Fig. 4. (a) Original CT image of the abdomen (512 512). (b) Merged watershed
partitioning. (c) Prostate segmentation with the intelligent paintbrush.
5 Discussion
The accuracy of the intelligent paintbrush segmentation depends on the overlap
of boundaries in the image partitioning with the contours of the object of interest. First results show that the pseudo-watershed procedure is able to detect
small intensity dierences in the image, as long as these correspond to a ridge
in the gradient magnitude image. This might not be the case for the boundaries
of thin line-like (or plate-like) structures, which are likely to be blurred with the
background due to the smoothing needed for the gradient computation. Other
image partitionings can be obtained that are based on another ridge denition.
Grin [6] describes a method to partition the image in districts by growing
MGP's both uphill and downhill from the saddle points of the gradient magnitude image, computed at dierent scales from a continuous representation of the
orginal image. Eberly [5] uses a ridge ow model, whereby ridges, corresponding
to medial structures, are computed from the image intensity itself.
The MDL criterion requires estimation of the image intensity distribution in
each region. For small regions, the number of samples might be too small to obtain reliable statistics. However, the image intensity is likely to be rather homogeneous within each region, such that the average number of samples per histogram
bin might still be signicant. Region merging is favored by histogram binning,
83
because all region intensity distibutions will be more compact and therefore more
similar. Other optimization strategies to minimize the description length function, such as simulated annealing [], might result in more optimal partitionings
with lower description length, but are computationally more expensive than the
procedure used here.
Instead of using one single image partitioning as input to the paintbrush
tool, several partitionings of the same original image could be used with dierent
granularity. A hierarchy of dierent image partitionings is constructed implicitly during merging. One partitioning with larger granularity could be used for
fast, but rough, colouring of the object of interest, while another with smaller
granularity could be used for erasing and more detailed editing.
6 Conclusion
Partitioning of the image in segmentation primitives is a powerful technique. It
allows fast and robust computer-aided interactive segmentation of the object of
interest using the intelligent paintbrush tool. Further work includes extension of
the merging procedure to 3D with the purpose of true 3D object segmentation,
and evaluation of the accuracy of the method on clinical applications by medical
experts.
Acknowledgements
This research is supported by the Belgian National Fund for Scientic Research
(NFWO). The algorithms were developed with AVS (Application Visualization System) on an IBM RS/6000 workstation. We thank Dr. J. Bogaert and
Dr. F. Demaerel for providing the images and D. Delaere and E. Bellon for their
contribution to the implementation of the intelligent paintbrush tool.
References
1. Beard, D. V., Eberly, D., Hemminger, B., Pizer, S., Faith, R., Kurak, C., Livingston, M.: Interacting with image hierarchies for fast and accurate object segmentation. Proc. SPIE Newport Beach, CA, 2167 (1994) 10{17.
2. Canny, J.: A computational approach to edge detection. IEEE Trans. PAMI 11
(1986) 679{698.
3. Cover, T.M., Thomas, J.A.: Elements of information theory. John Wiley & Sons
(1991).
4. Duda, R.O., Hart, P.E.: Pattern classication and scene analysis. John Wiley &
Sons (1973).
5. Eberly. D., Pizer, S.M.: Ridge ow models for image segmentation. Proc. SPIE
Newport Beach, CA, 2167 (1994) 54{64.
6. Grin, L.D., Colchester, A.C.F., Robinson, G.P.: Scale and segmentation of greylevel images using maximum gradient paths. Image and Vision Computing 10
(1992) 389{402.
84
7. Kirkpatrick, S., Gelatt, C.D, Vecchi, M.P.: Optimization by simulated annealing.
Science 220 (1982) 671{680.
8. Leclerc, Y.G.: Constructing simple stable descriptions for image partitioning. Intern. J. Computer Vision 3 (1987) 73{102.
9. Rissanen, J.: Minimum-description-length principle. Encyclopedia of Statistical
Sciences. Wiley, New York 5 (1987) 425{527.
10. Vincent, L., Soille,P.: Watersheds in digital spaces: an ecient algorithm based on
immersion simulations. IEEE Trans. PAMI 13 (1991) 583{598.
This article was processed using the LATEX macro package with LLNCS style