Phương pháp chẩn đoán hình ảnh medical image analysis methods (phần 7)

2089_book.fm Page 271 Tuesday, May 10, 2005 3:38 PM

7

Three-Dimensional
Multiscale Watershed
Segmentation
of MR Images
Ioannis Pratikakis, Hichem Sahli, and Jan Cornelis

CONTENTS
7.1
7.2

7.3

7.4

Introduction
Watershed Analysis
7.2.1 The Watershed Transformation
7.2.1.1 The Continuous Case
7.2.1.2 The Discrete Case
7.2.1.3 The 3-D Case
7.2.1.4 Algorithms about Watersheds
7.2.2 The Gradient Watersheds
7.2.3 Oversegmentation: A Pitfall to Solve in Watershed Analysis
Scale-Space and Segmentation
7.3.1 The Notion of Scale
7.3.2 Linear (Gaussian) Scale-Space
7.3.3 Scale-Space Sampling

7.3.4 Multiscale Image-Segmentation Schemes
7.3.4.1 Design Issues
7.3.4.2 The State of the Art
The Hierarchical Segmentation Scheme
7.4.1 Gradient Magnitude Evolution
7.4.2 Watershed Lines during Gradient Magnitude Evolution
7.4.3 Linking across Scales
7.4.4 Gradient Watersheds and Hierarchical Segmentation in
Scale-Space
7.4.5 The Salient-Measure Module
7.4.5.1 Watershed Valuation in the Superficial
Structure-Dynamics of Contours
7.4.5.2 Dynamics of Gradient Watersheds in Scale-Space
7.4.6 The Stopping-Criterion Stage
7.4.7 The Intelligent Interactive Tool

Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 272 Tuesday, May 10, 2005 3:38 PM

272

Medical Image Analysis

7.5

Experimental Results
7.5.1 Artificial Images
7.5.2 Medical Images

7.6 Conclusions
References

7.1 INTRODUCTION
The goal of image segmentation is to produce primitive regions that exhibit homogeneity and then to impose a hierarchy on those regions so that they can be grouped
into larger-scale objects. The first requirement concerning homogeneity can be very
well fulfilled by using the principles of watershed analysis [1]. Specifically, our
primitive regions are selected by applying the watershed transform on the modulus
of the gradient image. We argue that facing an absence of contextual knowledge,
the only alternative that can enrich our knowledge concerning the significance of
our segmented pixel groups is the creation of a hierarchy, guided by the knowledge
that emerges from the superficial and deep image structure. The current trends about
the creation of hierarchies among primitive regions that have been created by the
watershed transformation consider either the superficial structure [1–4] or the deep
image structure [5, 6] alone. In this chapter, we present the novel concept of dynamics
of contours in scale-space, which integrates the dual-image structure type into a
single one. Along with the incorporation of a stopping criterion, the proposed integration
embodies three different features, namely homogeneity, contrast, and scale. Application will be demonstrated in a medical-image analysis framework. The output of
the proposed algorithm can simplify scenarios used in an interactive environment
for the precise definition of nontrivial anatomical objects. Specifically, we present
an objective and quantitative comparison of the quality of the proposed scheme
compared with schemes that construct hierarchies using information either from the
superficial structure or the deep image structure alone. Results are demonstrated for
a neuroanatomical structure (white matter of the brain) for which manual segmentation is a tedious task. Our evaluation considers both phantom and real images.

7.2 WATERSHED ANALYSIS
7.2.1 THE WATERSHED TRANSFORMATION
In the field of image processing, and more particularly in mathematical morphology,
gray-scale images are considered as topographic reliefs, where the numerical value
of a pixel stands for the elevation at this point. Taking this representation into account

we can provide an intuitive description of the watershed transformation as in geography, where watersheds are defined in terms of the drainage patterns of rainfall. If
a raindrop falls on a certain point of the topographic surface, it flows down the
surface, following a line of steepest descent toward some local surface minima. The
set of all points that have been attracted to a particular minimum defines the catchment basin for that minimum. Adjacent catchment basins are separated by divide
lines or watershed lines. A watershed line is a ridge, a raised line where two sloping
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 273 Tuesday, May 10, 2005 3:38 PM

Three-Dimensional Multiscale Watershed Segmentation of MR Images

raindrops

273

Watershed Line

Regional Minima

FIGURE 7.1 (Color figure follows p. 274.) Watershed construction during flooding in two
dimensions (2-D).

surfaces meet. Raindrops falling on opposite sides of a divide line flow into different
catchment basins (Figure 7.1).
Another definition describes the watershed line as the connected points that lie
along the singularities (i.e., creases or curvature discontinuities) in the distance
transform. It can also be considered as the crest line, which consequently can be
interpreted by two descriptions: firstly, as the line that consists of the local maxima
of the modulus of the gradient, and secondly, as the line that consists of the zeros

of the Laplacian. These intuitive descriptions for the watershed-line construction
have been formalized in both the continuous and discrete domain.
7.2.1.1 The Continuous Case
In the continuous domain, formal definitions of the watershed have been worked
out by Najman [7] and Meyer [8]. The former definition is based on a partial ordering
relation among the critical points that are above several minima.
Definition 1: A critical point b is above a if there exists a maximal descending
line of the gradient linking b to a.
Definition 2: A path γ: ] −∞, +∞ [ → R2 is called a maximal line of the
gradient if
∀s ∈] − ∞, +∞[, γ (s ) = ±∇f [ γ (s )] ≠ 0
and
lim γ (s) = lim γ (s) = 0

s→−∞

s→−∞

Definition 3: A maximal line is descending if
∀s ∈] − ∞, +∞[, γ (s ) = −∇f [ γ (s )]
Definition 4: Let P(ƒ) be the subset of the critical points a of ƒ that are above
several minima of ƒ. Then the watershed of ƒ is the set of the maximal
lines of the gradient linking two points of P(ƒ). This definition of Meyer
[8] is based on a distance function that is called topographical distance.
Let us consider a function ƒ: Rn→R and let supp(ƒ) be its support. The
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 274 Tuesday, May 10, 2005 3:38 PM

274

Medical Image Analysis

topographical distance between two points p and q can be easily defined
by considering the set Γ(p,q) of all paths between p and q that belong to
supp(ƒ).
Definition 5: If p and q belong to a line of steepest slope between p and q
(ƒ(q) > ƒ(p)), then the topographical distance is equal to
TD(p,q) = ƒ(q) − ƒ(p)
Definition 6: We define a catchment basin of a regional minimum mi, CB(mi),
as the set of points x ∈ supp(ƒ) that are closer to mi than to any other
regional minimum with respect to the topographical distance
j ≠ i ⇒ TD(x, mi) < TD(x, mj)
Definition 7: The watershed line of a function ƒ is the set of points of the
support of ƒ that do not belong to any catchment basin

Wsh ( f ) = supp ( f ) ∩ 


∪
i


[CB(mi )]


7.2.1.2 The Discrete Case
Meyer’s definition [8] can also be applied for the discrete case if we replace the
continuous topographical distance TF by its discrete counterpart. Another definition

is given by Beucher [1] and Vincent [9]. The basic idea of the watershed construction
is to create an influence zone to each of the regional minima of the image. In that
respect, we attribute a one-to-one mapping between the regional minima and the
catchment basin.
Definition 8: The geodesic influence zone IZA(Bi) of a connected component
Bi of B in A is the set of points of A for which the geodesic distance to Bi
is smaller than the geodesic distance to any other component of B.
IZA(Bi) = {p ∈ A, ∀j ∈ [1,k]\{i}, dA(p,Bi) < dA(p,Bj)}
Definition 9: The skeleton by influence zones of B in A, denoted as SKIZAB,
is the set of points of A that do not belong to any IZA(Bi).
SKIZA(B) = A/IZA (B)
with
IZA(B) = ∪i∈[1,k] IZA(Bi)
Definition 10: The set of catchment basins of the gray-scale image I is equal
to the set X hmax obtained after the following recursion (Figure 7.2)
X hmin = Thmin ( I )
∀h ∈[ hmin , hmax − 1], X h +1 = min h +1 ∪ IZ Th +1 ( I ) ( X h )
where
hmin, hmax are the minimum and maximum gray level of the image, respectively
Th(I) is the threshold of the image I at height h
minh is the set of the regional minima at height h
Definition 11: The set of points of an image that do not belong to any
catchment basin correspond to the watersheds.
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 275 Tuesday, May 10, 2005 3:38 PM

Three-Dimensional Multiscale Watershed Segmentation of MR Images

Level 1

Level 2

275

Level 3

X1

X2

X3

Watershed lines

FIGURE 7.2 (Color figure follows p. 274.) Illustration of the recursive immersion process.

7.2.1.3 The 3-D Case
A brief but explicit discussion about watersheds in three-dimensional (3-D) space
was initiated by Koenderink [10], who considered watersheds as a subset of the
density ridges. According to his definition, “the density ridges are the surfaces
generated by the singular integral curves of the gradient, that is, those integral curves
that separate the families of curves going to distinct extrema.” In cases where we
consider only families of curves that go to distinct minima, then the produced density
ridges are the watersheds. For a formal definition of the watersheds in 3-D, the
reader can straightforwardly extend the definitions in Sections 7.2.1.1 and 7.2.1.2.
For the definition of Najman [7] in the 3-D case, we have to consider that the points
in P(ƒ) are the maxima and that the two types of hypersaddles are connected to two
distinct minima. These points have, in one of the three principal curvature directions,

slope lines descending to the distinct minima; the two slope lines run in opposite
directions along the principal curvature direction. These points make the anchor points
for a watershed surface defined by these points and the slope lines that connect them.
7.2.1.4 Algorithms about Watersheds
The implementation of the watershed transformation has been done by using the
following methods: iterative, sequential, arrowing, flow-line oriented, and flooding.
The iterative methods were initiated by Beucher and Lantuéjoul [11], who
suggested an algorithm based on the immersion paradigm. The method
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 276 Tuesday, May 10, 2005 3:38 PM

276

Medical Image Analysis

expands the influence zones around local minima within the gray-scale
levels via binary thickenings until idempotence is achieved.
The sequential methods rely on scanning the pixels in a predefined order, in
which the new value of each pixel is taken into account in the processing
of subsequent pixels. Friedlander and Meyer [12] have proposed a fast
sequential algorithm based on horizontal scans.
The arrowing method was presented by Beucher [1] and involves description
of the image with a directed graph. Each pixel is a node of the graph, and
the node is connected to those neighbors with lower gray value. The word
“arrowing” comes from the directed connections of the pixels.
The flow-line oriented methods are those that make an explicit use of the
flow lines in the image to partition it by watersheds [5].
The flooding methods are based on immersion simulations. In this category,

there are two main algorithms. The algorithm of Vincent and Soille [9] and
the algorithm of Meyer [13]. For an extensive analysis and comparisons of
the algorithms that are based on flooding, the interested reader can refer to
the literature [14, 15].

7.2.2 THE GRADIENT WATERSHEDS
Whenever the watershed transformation is used for segmentation, it is best to apply
it only on the gradient magnitude of an image, because then the gradient-magnitude
information will guide the watershed lines to follow the crest lines, and the real
boundaries of the objects will emerge. It has no meaning to apply it on the original
image. Therefore, from now on, we will refer to gradient watersheds, thus explicitly
implying that we have retrieved the watershed lines from the modulus of the gradient
image. Examples of gradient watersheds in two dimensions (2-D) and 3-D can be
seen in Figure 7.3 and Figure 7.4–7.5, respectively. The singularities of the gradient
squared in 2-D occur in the critical points of the image and in the points where the
second-order structure vanishes in one direction. This can be formulated as:
L2x + L2y = 0, ( L x = 0 ∧ Ly = 0 )
L2x + L2y ≠ 0 ∧ Lww = 0 ∧ Lwv = 0

FIGURE 7.3 Gradient watersheds in 2-D.
Copyright 2005 by Taylor & Francis Group, LLC

(7.1)
(7.2)


2089_book.fm Page 277 Tuesday, May 10, 2005 3:38 PM

Three-Dimensional Multiscale Watershed Segmentation of MR Images

277

(a)

(b)

FIGURE 7.4 (a) The cross-sections of the 3-D object and (b) their 3-D gradient watersheds.

where x, y denote Cartesian coordinates and w, v denote gauge coordinates [16].
The gradient can be estimated in different ways. It can be computed as (a) the
absolute maximum difference in a neighborhood, (b) a pixelwise difference between
a unit-size morphological dilation of the original image and a unit-size morphological erosion of the original image, and (c) a computation of horizontal and vertical
differences of local sums guided by operators such as the Roberts, Prewitt, Sobel,
or isotropic operators. The application of gradient operators as in case c reduces the
effect of noise in the data [17]. In the current study, the computation of the gradient
magnitude is done by applying the Sobel operator. Accordingly, in the case of 3-D,
the singularities of the gradient squared occur due to the following conditions
L2x + L2y + L2z = 0, ( L x = 0 ∧ Ly = 0 ∧ Lz = 0 )
L2x + L2y + L2z ≠ 0 ∧ Lww = 0 ∧ Lwv = 0 ∧ Lwu = 0

(7.3)
(7.4)

where x, y, z denote Cartesian coordinates and w, v, u denote gauge coordinates with
w in the gradient direction and (u, v) in the perpendicular plane to w (the tangent
plane to the isophote).
Similar to the 2-D case, the gradient magnitude in 3-D can be estimated in
different ways. All of the existing approaches issue from a generalization of 2-D
Copyright 2005 by Taylor & Francis Group, LLC

2089_book.fm Page 278 Tuesday, May 10, 2005 3:38 PM

278

Medical Image Analysis

FIGURE 7.5 A rendered view of the 3-D gradient watershed surface and the orthogonal
sections.

edge detectors. Lui [18] has proposed to generalize the Roberts operator in 3-D by
using a symmetric gradient operator. Zucker and Hummel [19] have extended to 3D the Hueckel operator [20]. They propose an optimal operator that turns out to be
a generalization of the 2-D Sobel operator. The morphological gradient in 2-D has
been extended to 3-D by Gratin [21]. Finally, Monga [22] extends to 3-D the optimal
2-D Deriche edge detector [23]. For the implementation of the gradient watersheds
in 3-D, the current study has adopted the 3-D Zucker operator for the 3-D gradientmagnitude computation.

7.2.3 OVERSEGMENTATION: A PITFALL

TO

SOLVE

IN

WATERSHED ANALYSIS

The use of the watershed transformation for segmentation purposes is advantageous
due to the fact that
•

•
•

Watersheds form closed curves, providing a full partitioning of the image
domain; thus, it is a pure region-based segmentation that does not require
any closing or connection of the edges.
Gradient watersheds can play the role of a multiple-point detector, thus
treating any case of multiple-region coincidence [7].
There is a one-to-one relationship between the minima and the catchment
basins. Therefore, we can represent a whole region by its minima.

Those advantages can be useful provided that oversegmentation, which is inherent to the watershed transformation, can be eliminated. An example of oversegmentation is shown in Figure 7.6. This problem can be treated by following two different
strategies. The first strategy considers the selection of markers on the image and
their introduction to the watershed transformation, and the second considers the
construction of hierarchies among the regions that will guide a merging process.
The next sections of this chapter are dedicated to the study of methods following
the second strategy.
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 279 Tuesday, May 10, 2005 3:38 PM

Three-Dimensional Multiscale Watershed Segmentation of MR Images

279

FIGURE 7.6 Example of an oversegmented image.

7.3 SCALE-SPACE AND SEGMENTATION

7.3.1 THE NOTION

OF

SCALE

As Koenderink mentions [24], in every imaging situation one has to face the problem
of scale. The extent of any real-world object is determined by two scales: the inner
and the outer scale. The outer scale of an object corresponds to the minimum size
of a window that completely contains the object and is consequently limited by the
field of view. The inner scale corresponds to the resolution that expresses the pixel
size and is determined by the resolution of the sampling device.
If no a priori knowledge for the image being measured is available, then we
cannot decide about the right scale. In this case, it makes sense to interpret the image
at different scales simultaneously. The same principle has been followed by the
human visual front-end system. Our retina typically has 108 rods and cones, and a
weighted sum of local groups of them make up a receptive field (RF). The profile
of such an RF takes care of the perception of the details in an image by scaling up
to a larger inner scale in a very specific way. Numerous physiological and psychophysical results support the theory that the cortical RF profiles can be modeled by
Gaussian filters (or their derivatives) of various widths [25].

7.3.2 LINEAR (GAUSSIAN) SCALE-SPACE
Several authors [24, 26–35] have postulated that the blurring process must essentially
satisfy a set of hypotheses like linearity and translation invariance, regularity, locality,
causality, symmetry, homogeneity and isotropy, separability, and scale invariance.
These postulates lead to the family of Gaussian functions as the unique filter for
scale-space blurring. It has been shown that the normalized Gaussian Gσ(x) is the
only filter kernel that satisfies the conditions listed above:
Gσ (x ) =
Copyright 2005 by Taylor & Francis Group, LLC

 x⋅x
1
exp  2 
 2σ 
(2 πσ 2 ) d / 2

(7.5)


2089_book.fm Page 280 Tuesday, May 10, 2005 3:38 PM

280

Medical Image Analysis

(a)

(b)

(c)

(d)

FIGURE 7.7 An MR brain image blurred at different scales (a) σ = 1, (b) σ = 4, (c) σ = 8,
(d) σ = 16.

Here x⋅x is the scalar product of two vectors, and d denotes the dimension of
the domain. The extent of blurring or spatial averaging is defined by the standard
deviation σ of the Gaussian, which represents the scale parameter. An example of

this spatial blurring can be seen in Figure 7.7. From this example, it can clearly be
seen how the level of detail in the image decreases as the level of blurring increases
and how the major structures are retained.
The scale-space representation of an image is denoted by the family of derived
images L(x,σ) and can be obtained as follows: let L(x) be an image acquired by
some acquisition method. Because this image has a fixed resolution determined by
the acquisition method, it is convenient to fix the inner scale as zero. The linear
scale-space L(x,σ) of the image is defined as
L(x,σ) = L(x) ⊗ Gσ(x)

(7.6)

where ⊗ denotes spatial convolution. Note that the family of derived images L(x,σ)
depends only on the original image and the scale parameter σ.
Lindeberg [29] has pointed out that the scale-space properties of the Gaussian
kernel hold only for continuous signals. For discrete signals, it is necessary to blur
with a modified Bessel function, which, for an infinitesimal pixel size, approaches
the Gaussian function.
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 281 Tuesday, May 10, 2005 3:38 PM

Three-Dimensional Multiscale Watershed Segmentation of MR Images

281

Koenderink [24] has also shown that the generation of the scale-space as defined
in Equation 7.6 can be viewed as solving the heat equation or diffusion equation
∂L

= c ∆L
∂t

(7.7)

The conductance term c controls the rate of blurring at each time step. If c is a
constant, the diffusion process is called linear diffusion, and the Gaussian kernel is
the Green’s function of Equation 7.7. In this case, the time parameter replaces the
scale parameter in Equation 7.6 with t = σ2/2c, given the initial condition L(x,0) =
L(x). The diffusion flow is a local process, and its speed depends only on the intensity
difference between neighboring pixels and the conductance c. The diffusion process
reaches a state of equilibrium at t → ∞ when all pixels approach the same intensity value.

7.3.3 SCALE-SPACE SAMPLING
The scale-space representation is a continuous representation. In practice, however,
it is necessary to sample the scale-space at some discrete values of scale. An
equidistant sampling of scale-space would violate the important property of scale
invariance [30]. The basic argument for scale invariance has been taken from physics
expressing the independence of physical laws from the choice of fundamental parameters. This corresponds to what is known as dimensional analysis, which defines that
a function that relates physical observables must be independent of the choice of
dimensional units. The only way to introduce a dimensionless parameter is by introducing a logarithmic measure [30]. Thus, the sampling should follow a linear and
dimensionless scale parameter δτ, which is related to σ according to the following:
σ n = εe τ0 + nδτ

(7.8)

where n denotes the quantization level. A convenient choice for τ0 is zero, which
implies that the inner scale σ0 of the initial image is taken to be equal to the linear
grid measure ε. At coarse scales, the ratio between successive scales will be about
constant, while at fine scales the differences between successive scales will be

approximately equal.

7.3.4 MULTISCALE IMAGE-SEGMENTATION SCHEMES
The concept of scale-space has numerous applications in image analysis. For a
concise overview, the interested reader can refer to the literature [16]. In this paper,
scale-space theory concepts are used for image-segmentation purposes.
7.3.4.1 Design Issues
For the implementation of a multiscale image-segmentation scheme, a number of
considerations must be kept in mind. A general recipe for any multiscale segmentation algorithm consists of the following tasks:
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 282 Tuesday, May 10, 2005 3:38 PM

282

Medical Image Analysis

1. Select a scale-space generator that will build the deep structure and
govern the simplification process for the image structure.
2. Determine the linking scheme that will connect the components (features)
in the deep image structure. Naturally, an immediate question arises about
which features will be the ones that will be linked. The answer is one of
the components that are apparent for the linking-scheme description. The
other components are the rules that will steer the linking and the options
that will be examined for the linkages (bottom-up, top-down, doubly
linked lists).
3. Attribute a significance measure of the scale-space segment. This implies
that a valuation has to be introduced at the localization level, for either
the region or the segmentation contours, by retrieving information from

their scale-space hierarchical tree.
All the above considerations have been combined in different ways and led
different authors to advocate their own multiscale segmentation scheme. In the
following section, the state of the art is presented.
7.3.4.2 The State of the Art
In Lifshitz and Pizer’s work [36], a multiscale “stack” representation was constructed
considering isointensity paths in scale-space. The gray level at which an extremum
disappears is used to define a region in the original image by local thresholding on
that gray level. The same authors observed that this approach can be used to meet
the serious problem of noncontainment. This problem refers to the case that a point,
which at one scale has been classified as belonging to a certain region (associated
to a local maximum), can escape from that region when the scale parameter increases.
Moreover, the followed isointensity paths can be intertwined in a rather complicated
way.
Lindeberg [37] has based his approach on formal scale-space theory to construct
his scale-space primal sketch. This representation is achieved by applying a linking
among light or dark blobs. Because a blob corresponds to an extremum, he used
catastrophe theory to describe the proposed linking as interactions between saddles
and extremum. To attribute a significance measure for the scale-space blob, he
considered three features: spatial extent, contrast, and lifetime in the scale-space.
Correspondence between two blobs of consecutive scale is attributed by measuring
the degree of overlap.
Multiscale segmentation of unsharp blobs has also been reported by Gerig et al.
[38]. They applied Euclidean shortening flow, which progressively smoothes the
level curves and lets them converge to circles before they disappear at singularity
points. Object detection is interleaved with shape computation by analyzing the
continuous extremum paths of singularities in scale-space. Assuming radially symmetric structures, the singularity traces are attributed to the evolution time.
Eberly [39] constructed a hierarchy based on annihilation of ridges in scalespace. He segmented each level of the scale-space by decomposing the ridges into
Copyright 2005 by Taylor & Francis Group, LLC

2089_book.fm Page 283 Tuesday, May 10, 2005 3:38 PM

Three-Dimensional Multiscale Watershed Segmentation of MR Images

283

curvilinear segments, followed by labeling. Using a ridge flow model, he made a
one-to-one correspondence of each ridge segment to a region. At each pixel in the
image, the flow line is followed until the flow line intersects a ridge. Every pixel
along the path is assigned the label of the terminal ridge point. The links at the
hierarchical tree are inserted, based on how primitive regions at one scale become
blurred into single regions at the next scale. The latter single primitive region is
considered to be the parent of the original two regions because it overlaps those two
regions more than any other region at the current scale.
The segmentation scheme of Vincken [40, 41] and Koster [42] is based on the
hyperstack, which is a generalization to 3-D of the stack proposed by Koenderink
[24]. Between voxels at adjacent scale levels, child-parent linkages are established
according to a measure of affection [42]. This measure is a weighted sum of different
measures such as intensity difference, ground volume size, and ground volume mean
intensity. A ground volume is the finest-scale slice of a 4-D scale-space segment.
This linking-model-based segmentation scheme has been applied not only for the
linear scale-space, but experiments have also been reported [43] for gradient-dependent diffusion and Euclidean shortening flow. Vincken et al. [40, 41] used the
hyperstack in combination with a probabilistic linking, wherein a child voxel can
be linked to more than one parent voxel. The multiparent linkage structure is
translated into a list of probabilities that also indicate the partial-volume voxels and
to which extent these voxels belong to the partial-volume class of voxels. Thus, an
explicit solution for the treatment of partial-volume effects caused by the limited
resolution, due to the acquisition method and leading to multiple object voxels, is
proposed.

Using linear-scale evolution of gray-scale images, Kalitzin et al. [44] proposed
a hierarchical segmentation scheme where, for each scale, segments are generated
as Voronoi diagrams, with a distance measure defined on the image landscape. The
set of centers of the Voronoi cells is the set of the local extrema of the image. This
set is localized by using the winding number distribution of the gradient-vector field.
The winding number represents the number of times that the normalized gradient
turns around its origin, as a test point circles around a given contour. The process
is naturally described in terms of singularity catastrophes within the smooth scale
evolution. In short, this approach is a purely topological segmentation procedure,
based on singular isophotes.
Griffin et al. [45] proposed a multiscale n-ary hierarchy. The basic idea is to
create a hierarchical description for each scale and then link these hierarchies across
scale. In a hierarchical description of the structure, the segments are ordered in a
tree structure. A segment is either the sum of its subobjects or a single pixel. This
hierarchy is built by iteratively merging adjacent objects. The order of merging is
based on an edge-strength measure that combines pure edge strength along with
perceptual significance of the edge, determined by the angle of the edge trajectory. The
linking of the hierarchies proceeds from coarse to fine scales and from the top of the
hierarchies to the bottom. First, the roots in the hierarchies are linked, then the subobjects
of the roots are matched, etc. This results in a multiscale n-ary hierarchy.
The multiscale segmentation framework presented in this chapter deals with regions
produced after the application of the watershed transformation and its subsequent
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 284 Tuesday, May 10, 2005 3:38 PM

284

Medical Image Analysis

tracking in scale-space. In a similar spirit, other authors have produced works in
this field:
Jackway [46] applied morphological scale-space theory to control the number
of extrema in the image, and by subsequent homotopy-linking of the gradient extrema to the image extrema, he obtained a scale-space segmentation
via the gradient-watershed transformation [46]. In this case, the watershed
arcs that are created at different scales move spatially and are not a subset
of those at zero scale.
Gauch and Pizer [5] presented an association of scale with watershed boundaries after a gradual blurring with a series of Gaussians. When an intensity
minimum annihilates into a saddle point, the water that drains towards the
annihilated minimum now drains to some other intensity minimum in the
image. This defines the parent-child relationship between these two watershed regions. By continuing this process for all intensity minima in the
image, a hierarchy of watershed regions is defined.
Olsen [6] analyzed the deep structure of segmentation using catastrophe
theory. In this way, he advocated a correspondence between regions produced by the gradient watersheds at different scales.

7.4 THE HIERARCHICAL SEGMENTATION SCHEME
The relationship between watershed analysis and scale-space can be attributed to
the simplification process that is offered by the scale-space. On the one hand, a
decreasing number of singularities occur during an increasing smoothing of the
image. On the other hand, the duality of the watershed segments increases with their
respective minima in the gradient image. Both contribute to an attractive framework
for the examination of a merging process in a segmentation task. A detailed explanation of this relationship, along with the produced results, will be given in the
following.

7.4.1 GRADIENT MAGNITUDE EVOLUTION
As discussed in Section 7.3.4.1, when we think about the implementation of a
multiscale segmentation scheme, certain considerations have to draw our attention.
The very first consideration is the selection of the image-evolution scheme. In this
work, we have studied the gradient-magnitude evolution. The basic motivation is

that treating a problem of an uncommitted front-end, contrast and scale are the only
useful information. Gradient magnitude provides the contrast information, and scale
is inherent to the evolution itself.
During the image evolution according to the heat equation Lt = Ln, the gradientsquared image follows an evolution according to the following:
Tensor notation
∂( Li Li )
= 2 Li Lit = 2 Li Lti = 2 Li Lkki
∂t
Copyright 2005 by Taylor & Francis Group, LLC

(7.9)


2089_book.fm Page 285 Tuesday, May 10, 2005 3:38 PM

Three-Dimensional Multiscale Watershed Segmentation of MR Images

285

Using a Cartesian coordinate system
( L2x + L2y ) t = 2 L x ( L xxx + L xyy ) + 2 Ly ( Lyxx + Lyyy )

(7.10)

Computing the Laplacian for the gradient squared
∇ 2 ( L2x + L2y ) = ∂ xx ( L2x + L2y ) + ∂ yy ( L2x + L2y )
= (2 L2xx + 2 Lx Lxxx + 2 L2xy + 2 L y Lxxy )
+ (2 L2xy + 2 Lx Lxyy + 2 L2yy + 2 L y L yyy )
= 2 L2xx + 2 L2yy + 4 L2xy + 2 Lx ( Lxxx + Lxyy )

(7.11)

+ 2 L y ( L yxx + L yyy )
= 2 L2xx + 2 L2yy + 4 L2xy + ( L2x + L2y ) t
We observe that the gradient-squared evolution is not governed by the diffusion
equation, and subsequently the corresponding singular points or regional areas evolve
in a different manner.

7.4.2 WATERSHED LINES

DURING

GRADIENT MAGNITUDE EVOLUTION

The second consideration (see Section 7.3.4.1) for building up a multiscale segmentation scheme is the determination of the linking scheme for the selected features
in the deep image structure. In a watershed-analysis framework, the selected features
are the regions that are produced by the gradient watersheds, each of them corresponding to a singularity (regional minimum) of the gradient-magnitude image.
Because the proposed segmentation scheme relies on the behavior of singularities
in time, we have used catastrophe theory to study an explicit classification of the
topological changes that occur during evolution and to explain their linking in scalespace. In this study, we have drawn the conclusion that two types of catastrophes
(fold and cusp) occur during the gradient-magnitude evolution. The detailed algebraic analysis can be found in a work by Pratikakis [47].
Using the duality between the regional minima and the regions produced by the
gradient watersheds, we can observe how the watershed lines behave during this
evolution. Figure 7.8, Figure 7.9, and Figure 7.10 give a clear insight of this behavior.
Looking at Figure 7.9, we can observe both catastrophe types. The fold catastrophe
is perceived as an annihilation of the regional minimum, and the cusp catastrophe
is perceived as merging between two regional minima to give one minimum. This
behavior is reflected on the watershed-line construction by an annihilation of watershed-line segments. Obviously, this demonstrates why the placement of watershed
Copyright 2005 by Taylor & Francis Group, LLC

2089_book.fm Page 286 Tuesday, May 10, 2005 3:38 PM

286

Medical Image Analysis

FIGURE 7.8 Successive blurring of the original image.

analysis into a scale-space framework makes it an attractive merging process. Nevertheless, there is a major pitfall. In Figure 7.10, it is clearly evident that, during
the evolution of the gradient magnitude, the watershed lines become increasingly
delocalized. This situation does not permit us to have a merging process by only
considering the critical-point events and retrieving the produced segments at the
desired scale. This also explains why the deep image structure has to be viewed as
one body and not as a collection of separated scaled versions of the image under
study. To achieve a single-body description of the deep image structure, we need to
link (connect) all the components or features of this structure. For segmentation
purposes, this linking is useful because it guides us to achieve a segmentation at the
localization scale. This is feasible by describing all the spatial changes and interactions of the singularities that also influence the saliency measure of the localized
watershed segments. The next section of this chapter provides a detailed description
of the proposed linking scheme.
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 287 Tuesday, May 10, 2005 3:38 PM

Three-Dimensional Multiscale Watershed Segmentation of MR Images

287

FIGURE 7.9 Behavior of the regional minima during the gradient-magnitude evolution.

7.4.3 LINKING

ACROSS

SCALES

We have already explained that interaction between singularities during the magnitude-gradient evolution is attached to behaviors of either a fold or a cusp catastrophe.
The critical points disappear with increasing scale, and this event is the generic way
in which it happens. The term generic means that if the image is changed slightly,
the event may change position in scale-space, but it will still be present. Apart from
the disappearance, another event is also generic. This is the appearance of two critical
points [36, 48, 49]. In a more detailed way, the generic events of the minima in the
gradient magnitude are as follows:
No interaction with other singularities (Figure 7.11a)
Creation in a pair with a saddle (Figure 7.11b)
Splitting into saddle and two minima (Figure 7.11c)
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 288 Tuesday, May 10, 2005 3:38 PM

288

Medical Image Analysis

FIGURE 7.10 Watershed segment merging and delocalization of the watershed lines during
the gradient-magnitude evolution.

Annihilation with a saddle (Figure 7.11b)
Merging with a saddle and another minimum into one minimum (Figure
7.11c)
In Figure 7.11, all the generic events are schematically described, using broken
lines to indicate linking between the minima of two adjacent regions in scale-space.
As scale increases from bottom to top, one can observe how interactions between
critical points can lead to merging of two adjacent regions due to the underlying
one-to-one correspondence between a minimum and a region.
Linking of the minima (parent–child relationship) for successive levels is applied
by using the proximity criterion [24]. This criterion checks the relative distance for
all the minima at scale σi that have been projected on the same influence zone
IZA(Bj)i+1 at scale σi+1 with respect to the original minimum of this influence zone.
An example can be seen in Figure 7.12, which represents the linking for two
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 289 Tuesday, May 10, 2005 3:38 PM

Three-Dimensional Multiscale Watershed Segmentation of MR Images

289

Annihilation
Creation

(a)

(b)

(c)

FIGURE 7.11 (Color figure follows p. 274.) Generic events for gradient-magnitude evolution.

(a)

(b)

Scale N + 1

Scale N
(c)

FIGURE 7.12 Linking in two successive levels: (a) scale N and (b) scale N + 1.
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 290 Tuesday, May 10, 2005 3:38 PM

290

Medical Image Analysis

successive levels of the evolution example that is depicted in Figure 7.8, Figure 7.9,
and Figure 7.10. Figure 7.12a shows the regional minima at scale σi that have been
spatially projected onto level σi+1. The watershed lines at level σi+1 are also shown,
and these delimit the influence zones at this level. The regional minima at scale σi+1
can be seen in Figure 7.12b.
For the sake of clarity, for each regional minimum in Figure 7.12a and Figure
7.12b, there is a marker of different shape and gray value that makes them distinct.
A linking for the minima (mj)σi at scale σi and the minima (m′p )σi+1 at scale σi+1

appears in Figure 7.12c. After the linking stage, we have for each quantization scale
level a labeling for the minima with respect to their linking ability. These labels are
of two types. Either the minimum is annihilated/merged and will not be considered
in the next levels, or the minimum does not interact with other singularities and
takes up the role of the father label for all the minima that were annihilated or
merged and were situated at the same influence zone. This labeling is guided by the
proximity criterion. The projected minimum (m j )σi (∈ IZA(Bp)i+1), which is closest
to the minimum (m′p )σi+1, is considered the father, and the rest of the projected minima
onto the same influence zone are considered annihilated. Closeness is defined with
respect to the topographic distance (see Section 7.2.1.1), which is a natural distance
measure following the steepest gradient path inside the catchment basin. From the
implementation point of view, we have to mention that we use ordered queues to guide
m σi toward mσ′ i+1 . In that way, we avoid problems caused by the presence of plateaus.
Being consistent with the theory, we have to keep in mind that a generic event
in gradient-magnitude evolution is also the creation/splitting of minima. In practice,
this event can be understood as an increasing of minima in successive levels in the
evolution. Due to the quantization of scale, such an increase in the amount of minima
rarely occurs, and even if it happens, its scale-space lifetime is very short. This
motivated us to keep the same linking scheme for this event, too. In the case that a
creation contributes to an increasing amount of minima, then linking is done with
the closest minimum of the two new ones, while the other is ignored.
The proposed linking scheme is a single-parent scheme that links the regional
minima and their respective dual watershed regions in successive evolution levels.
This region-based approach is chosen to avoid problems of a pixel-based linking
caused by the noncontainment problem (see Section 7.3.4). An additional advantage
of a region-based approach, specifically when a watershed-analysis framework is
used, is the inherent definition of a search area for the linking process, namely the
influence zones, that otherwise, in a pixel-based approach, has to be defined in an
ad hoc manner.
The aim of the proposed linking is to determine which contours (common

borders) can be regarded as significant, without any a priori information about scale,
spatial location, or the shape of primitives.

7.4.4 GRADIENT WATERSHEDS AND HIERARCHICAL SEGMENTATION
IN SCALE-SPACE
Once the linking between the regional minima in the deep image structure has been
obtained, an explicit hierarchy is attributed to these minima. The currently obtained
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 291 Tuesday, May 10, 2005 3:38 PM

Three-Dimensional Multiscale Watershed Segmentation of MR Images

291

SALIENT MEASURE
MODULE

Gaussian
blurring

Child-Parent
Linking
Dynamics of
minima

Yes
No Down Projection
Continue

&
blurring?
Contour valuation

STOPPING CRITERION
MODULE
Ranking
Yes
No
Adjacent
regions
to merge?

Merge
2 regions

FALSE

Hierarchical Segmented
Levels HLi
i

Null
Hypothesis

END

TRUE

i+1

FIGURE 7.13 Dynamics of contours in scale-space algorithm.

hierarchy is only based on scale. At this point, we go toward the description of how
to enrich this hierarchy and make it more consistent. Consistency will be obtained
because the hierarchy is based on more features than only scale. A hierarchical
segmentation of an image is a tree structure by inclusion of connected image regions.
The construction of the tree structure follows a model consisting of two modules.
The first module is dedicated to evaluate each contour arc with a salient measure,
and the second module identifies the different hierarchical levels by using a stopping
criterion. A schematic diagram can be seen in Figure 7.13. As mentioned in Section
7.3.4.1, the third consideration for constructing a multiscale segmentation scheme
is the significance measure. The following subsection explains this measure and how
we attribute it to the watershed segments at the localized scale.

7.4.5 THE SALIENT-MEASURE MODULE
7.4.5.1 Watershed Valuation in the Superficial StructureDynamics of Contours
The principle of dynamics of contours [3] uses the principle of dynamics of minima
[2] as an initial information for the common contour valuation of adjacent regions
(see Figure 7.14). The additional information that is used is based on the tracking
of the flooding history. In such a way, a contour valuation can be found by the
comparison of dynamics between the regions that have reached the contour of interest
during a flooding. The dynamics of a minimum m1 is easily defined with a flooding
scenario. Let h be the altitude of the flood when, for the first time, a catchment basin
with a deeper minimum m2 (m2 < m1) is reached. The dynamics of m1 is then simply
equal to h-altitude (m1). Each catchment basin is attributed the value of the dynamics
of its minimum. The contour valuation that is attributed to each common border of
the regions at a certain scale σ = σ0 (superficial structure) is denoted as
Copyright 2005 by Taylor & Francis Group, LLC

2089_book.fm Page 292 Tuesday, May 10, 2005 3:38 PM

292

Medical Image Analysis

P

5

4

3
2
1
4

5

I
P
II

1

2

3

FIGURE 7.14 Flooding list for point P, the basis for computing dynamics of contours that
correspond to P.

DC(a)σ = min max{ f (a) − f (ai )}
i

ai ∈Bi ,σ

(7.12)

where a denotes the lower point (saddle point) of the arc that belongs to the
watershed, and Bi,σ denotes an open connected component that belongs to the topological open set Bas(a)σ, which is defined as
Bas(a)σ = {b|∃γ, γ(0,σ) = a, γ(1,σ) = b, ƒ(γ(s,σ)) < ƒ(a)}, ∀s ∈ [0,1]

(7.13)

This contour valuation is based on contrast and is characterized by a high degree
of noise sensitivity. Experimental results have been reported in the literature [50].
Motivated by the shortcoming in noise sensitivity, it came as a natural consequence
to obtain a contour valuation by considering the behavior of the catchment basins
in scale-space.
7.4.5.2 Dynamics of Gradient Watersheds in Scale-Space
Once the parent–child linkages have been completed, the next step is to valuate the
gradient watersheds at the localization scale σ0. Let us assume that we want to
valuate the gradient watershed that separates regions A and B (see Figure 7.15).
From the linking step, we have created a linkage list, Λ(m,n) (see Figure 7.15a),
where m and n denote the regions at the localization scale.
In our example, m = A and n = B. This list provides the following information:
region F is attributed to a root for regions A and B at scale S4. This has occurred
because (a) region A has been linked to region D at scale S2, and region D has been

Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 293 Tuesday, May 10, 2005 3:38 PM

S4

F

S3
S2

E
D

S1
S0

Scale Quantization Levels

Scale Quantization Levels

Three-Dimensional Multiscale Watershed Segmentation of MR Images

C

D

S2

D

S1

(a)

DC3

F

S3

S0

B

A

S4

DC2

E
C
C

A

293

DC1
DC0

B

A
(b)

FIGURE 7.15 Down-projection and contour valuation (The linkage list Λ(A,B)).

linked to region F; and (b) region B has been linked to region C at scale S1, region
C has been linked to region E at scale S3, and region E has been linked to region F.
Based upon this linkage list, we compute the multiscale contour dynamics for the
contour arc that separates region A and B at localization scale σ0. As demonstrated
in Figure 7.15b, at each scale quantization level l in the linkage list Λ(m,n), where
no root creation has occurred, we compute the dynamics of contours DCl for the
m ,n )
m ,n )
region couple RlΛ(
, RlΛ(
, which constitute part of the branches in the linkage
,m
,n
list at the scale quantization level l. This computation answers the question of how
m ,n )
m ,n )
contrasted the regions RlΛ(
, RlΛ(
are, rather than attributing a valuation to a
,m

,n
contour arc that separates those regions at a scale quantization level Sl. The reason
m ,n )
m ,n )
is that regions RlΛ(
, RlΛ(
may not be adjacent at a certain quantization scale
,m
,n
other than S0. Nevertheless, all the valuations DCk during the evolution of regions
R0,Λ(mm ,n ) , R0,Λ(nm ,n ) in scale-space will be integrated to provide the dynamics of the
localized gradient watersheds at σ0. Taking into account Equation 7.12, the dynamics
of contours in scale-space are denoted as
S a −1

DCS( a ) =

∑ DC(k)

σi

|B j ,σi ∈ Λ( p, q ) ∧ k ∉ IZ A ( B j ,σi )

(7.14)

i = S0

where p and q denote the regions that have the common contour a at the localization
scale, S0 denotes the localization-scale quantization level, and Sa denotes the annihilation-scale quantization level. The difference ∆t = Sa − S0 denotes the scale-space
lifetime.

The determination of a salient measure with respect to the dynamics of contours
in scale-space is described by the flow diagram in Figure 7.13. To retrieve the
different hierarchical levels HLi, a second phase follows that we call the stoppingcriterion phase. This phase involves a merging criterion of region couples whose
semantics have been sorted according to their dynamics of contours in scale-space
valuation.
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 294 Tuesday, May 10, 2005 3:38 PM

294

Medical Image Analysis

7.4.6 THE STOPPING-CRITERION STAGE
When the calculation of the saliency measure has been completed, we sort these
values. As a result, we obtain the ranking of the adjacent region couples that are to
be merged. From there on, the consecutive hierarchical levels can be constructed
based on the ability of each merged couple to satisfy the null hypothesis during a
hypothesis testing as we scan the ranked values. For our problem, the hypothesis
set is defined as:
H 0k : Two adjacent regions belong to the same label at level k
H1k : Two adjacent regions belong to different labels at level k
Upon that definition, it is critical to say that for every level transition from k to
k + 1, we update the statistics used to formulate the hypothesis. This update is
expressed by the error term σ 2( err )k . Considering that (a) our initial partitioning of the
image (oversegmented image) consists of homogeneous regions and (b) objects at
a certain hierarchical level k are expected to exhibit a certain homogeneity, we choose
as the most suitable statistic the variance σ2. Therefore, we formulate our hypothesis
testing as follows:

H 0k : σ 2(Oi ∪O j )k = σ (2ij )k

(7.15)

H1k : σ 2 (Oi ∪O j )k > σ 2(ij )k

(7.16)

where σ O2 i ∪O j denotes the variance of the merged region as a result of the updated
regions Oi and Oj, and where σ 2(ij )k is expressed as
σ 2(ij )k = σ (2Oi ∪O j )k + σ (2err )k −1

(7.17)

where σ (2Oi ∪O j )k denotes the estimated variance of the merged region as a result of
the updated regions Oi and Oj. It is a result of weighting the variances of the updated
constituent regions by the region area, and it is equal to ni σ i2 + n j σ 2j ni + n j .
The term σ 2( err )k−1 denotes the error in variance at level k − 1, due to which the
alternative hypothesis was validated, and consequently the hierarchical level k − 1
was created. Initially, σ 2( err )0 is set to zero. Applying the chi-squared test to the
calculated variance s2 of the merged regions Oi and Oj, we get

(

s(2ij )k <

s(2ij )k ≥
Copyright 2005 by Taylor & Francis Group, LLC

σ (2ij )k

n −1
σ (2ij )k
n −1

)(

)

χ2a ( n − 1) ⇒ P (i, j ) = true

(7.18)

χ2a ( n − 1) ⇒ P (i, j ) = false

(7.19)


2089_book.fm Page 295 Tuesday, May 10, 2005 3:38 PM

Three-Dimensional Multiscale Watershed Segmentation of MR Images

295

P(i,j) is a decision function that is true if updated partitions i and j belong to
the same object at level k; otherwise it is false, and n is the number of pixels in the
merged region. In our case, n >> 0, so we introduce the following approximation
of the chi-square [51]
χ2a ( n − 1) ≈ ( n − 1) + ua 2 ( n − 1)

(7.20)

where ua is the right critical value for a one-tailed hypothesis test carried out on a
standard normal distribution [N(1,0)]. Substituting the approximation of Equation
7.20 in Equation 7.18 and Equation 7.19 leads to Equation 7.21
 ( n − 1) + u 2 ( n − 1) 
2
a
σ 2(ij )k 
 ≈ σ (ij )k
n
−
1




ua 2 
1+

n 


(7.21)

Therefore, our test is expressed as

u 2
s(2ij )k < σ (2ij )k  1 + a
 ⇒ P (i, j ) = true
n 



(7.22)


u 2
s(2ij )k ≥ σ (2ij )k  1 + a
 ⇒ P (i, j ) = false
n 


(7.23)

During all the experimental work, the confidence interval used was a = 0.05,
which corresponds to ua = 1.96. As a last note, it must be mentioned that when we
update the variances of the merged regions, we do not have to compute them, because
statistics for a merged region can be computed by using statistics of the constituent
partitions, which already have been computed at the very beginning.

7.4.7 THE INTELLIGENT INTERACTIVE TOOL
Automatic segmentation methods are known to be unreliable due to the complexity and
variability of medical images, and they cannot be applied without supervision by the
user. On the other hand, manual segmentation is a tedious and time-consuming process,
lacking precision and reproducibility. Moreover, it is impractical when applied to extensive temporal and spatial sequences of images. Therefore, to perform an image-segmentation task in a generic and efficient way, a compromise has to be found between
automatic and manual modes. Under this principle, a hybrid scheme has to be constructed that minimizes the user input and allows the presence of appropriate tools to
provide fast and accurate manual correction. The user input is minimized by constructing an image description rich in meaningful regions with low cardinality, and an
interactive tool ensures accuracy and reproducibility without requiring any special
Copyright 2005 by Taylor & Francis Group, LLC