Phương pháp chẩn đoán hình ảnh medical image analysis methods (phần 9)

2089_book.fm Page 341 Tuesday, May 10, 2005 3:38 PM

9

Landmark-Based
Registration of
Medical-Image Data
J. Ruiz-Alzola, E. Suarez-Santana,
C. Alberola-Lopez, and Carl-Fredrik Westin

CONTENTS
9.1
9.2
9.3
9.4
9.5
9.6

Introduction
Deformation Maps
Landmark-Based Image Analysis
Landmark Detection and Location
Our Approach to Landmark-Based Registration
Deformation Model Estimation
9.6.1 Intensity-Based Registration
9.6.1.1 Template Matching
9.6.1.2 Multiresolution Pyramid
9.6.1.3 Local Structure
9.6.1.4 Entropy-Based Similarity Measure
9.6.2 Variogram Estimation
9.7 Landmark-Based Local Registration

9.7.1 Displacement Field Model
9.7.2 Ordinary Kriging Prediction of Displacement Fields
9.8 Results
9.9 Conclusions
Appendix 9.1 Geostatistical Spatial Modeling
Acknowledgment
References

9.1 INTRODUCTION
Image registration consists of finding the geometric (coordinate) transformation that
relates two different images, source and target. Hence, when the transformation is
applied to the source image, an image with the same geometry as the target one is
obtained. Should both images be obtained with the same acquisition modality and
illumination conditions, the transformed source image would ideally become identical

Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 342 Tuesday, May 10, 2005 3:38 PM

342

Medical Image Analysis

to the target one. Image registration is a crucial element of computerized medicalimage analysis that is also present in other nonmedical applications of image processing and computer vision. In computer vision, for example, it appears as the socalled correspondence problem for stereo calibration [1] and for motion estimation
[2], which is also of paramount importance in video coding [3]. In remote sensing,
registration is needed to equalize image distortion [4], and in the broader area of
geographic information systems (GIS), registration is needed to accommodate different maps in a common reference system [5].
In this chapter we propose a geostatistical framework for the registration of
medical images. Our motivation is to provide the highest possible accuracy to

computer-aided clinical systems in order to estimate the geometric (coordinate)
transformation between two multidimensional, possibly multimodal, datasets.
Hence, in addition to being accurate, the approach must be fast if it is to operate in
clinically acceptable times. Even though the framework presented here could be
applied to several fields, such as the ones mentioned above, this chapter focuses on
the application of image registration to the medical field. Registration of medical
(both two- and three-dimensional) images, from the same or different imaging
modalities, is needed by computer-aided clinical systems for diagnosis, preoperative
planning, intraoperative procedures, and postoperative follow-up. Registration is also
needed to perform comparisons across a population, for deterministic and statistical
atlas construction, and to embed anatomic knowledge in segmentation algorithms.
A good review of the current state of the art for medical-image registration can be
found in the literature [6].
Our framework is based on the reconstruction of a dense arbitrary displacement
field by interpolating the displacements measured from control points [7]. To this
extent, the statistical second-order characterization of the displacement field is estimated from the result of a general-purpose intensity-based registration algorithm,
and it is used to make the best linear unbiased estimation of the displacement in
every point using a fast implementation of universal Kriging, an optimal estimation
scheme customarily used in geostatistics.
Several schemes have been proposed in the past to interpolate sparse displacement fields for medical-image registration. Most of them fit in one of the two next
categories, i.e., PDE- and spline-based. As for PDE-based approaches [8, 9], they
rely on a mechanical dynamic model stated as a set of partial differential equations,
where the sparse displacements are associated with actuating forces. The mechanical
model provides an ad hoc regularization to the interpolation problem that produces
a physically feasible result. However, the assumption that the physical difference
between the source and the target image can actually be represented by some specific
model is by no means evident. Moreover, mechanical properties must also be
endowed to the anatomic structures in order to obtain a proper model. With respect
to spline-based approaches, they usually make an independent interpolation for each
of the components of the vector field. Interpolating or smoothing thin-plate splines

[10–12] are used, depending on whether the sparse displacements are considered to
be noiseless or not. The former need the order of the spline to be specified in advance,
while the latter also need the regularization parameter to be specified. Adaptiveness
can be obtained by spatially changing the spline order and the regularization term.
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 343 Tuesday, May 10, 2005 3:38 PM

Landmark-Based Registration of Medical-Image Data

343

The bending term in the spline energy functional could, in principle, also be modified
to account for nonisotropic behavior, and even a set of covariables could also be
added to the coupling term of the functional. None of these improvements are usually
implemented, possibly because of the difficulty of obtaining an objective design
from data.
Our framework departs from the previous two approaches by adopting a geostatistical approach. Related work in the field of statistical shape analysis has been
previously reported by Dryden and Mardia [13]. The underlying idea is to use an
experimental approach that makes the fewest a priori assumptions by statistically
analyzing the available data, i.e., the displacement field obtained from approximate
intensity-based image registration. Our method consists of locally applying the socalled universal Kriging estimator [14] to obtain the best linear unbiased estimator
(BLUE) of the displacement at every point from the displacements initially obtained
at the control points. Central to this approach is the estimation of the second-order
characterization of the displacement field, now modeled as a vector random process
model. The estimated variogram [14] (a statistics related to the spatial covariance
function or covariogram) plays the role of the spline kernel, though now they are
directly obtained from data and not from an a priori dynamic model. Remarkably,
thin-plate splines can be considered as a special case of universal Kriging estimation [15].

9.2 DEFORMATION MAPS
Consider two multidimensional images I1(x) (source) and I2(x′) (target). Registration
consists of finding the mapping
Y : ℜD → ℜD
x

→ x′ = Y ( x)

(9.1)

that geometrically transforms the source image onto the target image. The components of the mapping can be made explicit as
 x ′1   Y 1 ( x1, …, x D ) 


x ′ =   = 

 x ′   Y D ( x1, …, x D )
D



(9.2)

The vector field Y(x) is commonly termed deformation or warp. Sometimes the
displacement field is considered instead, i.e.,
D(x) = Y(x) − x
A deformation mapping should count on two basic properties:
Copyright 2005 by Taylor & Francis Group, LLC

(9.3)


2089_book.fm Page 344 Tuesday, May 10, 2005 3:38 PM

344

Medical Image Analysis

1. Bijective: a one-to-one and onto mapping, which means that the inverse
mapping exists
2. Differentiable: continuous and smooth, ideally a diffeomorphism, so that
the inverse mapping is also differentiable, thus ensuring that no foldings
are present
In addition, the construction method of the mapping must be equivariant with
respect to some global transformations. For example, to be equivariant with respect
to affine transformations, if both (source and target) images are affinely transformed,
the mapping should be consistently affinely transformed too.
Any deformation must also accommodate both global and local differences, i.e.,
the mapping can be decomposed in a global and a local component. Global differences are large-scale trends, such as an overall polynomial, affine, or rigid transformation. Local differences are on a smaller scale, highlighting changes in a local
neighborhood, and are less smooth. Local differences are the reminder of the deformation once the global difference has been compensated. The definition of global
and local components depends on whether they are composed or added to form the
total map
Y(x) = YG(x) + YL(x)
= TL[TG(x)]

(9.4)

where YG, YL and TG, TL refer to the global and local components of the mapping,
in the addition and in the composition forms, respectively*.

Most commonly, the global deformation consists of a polynomial map (of
moderate order to avoid oscillations). Translations, rotations (i.e., Euclidean maps),
and affine maps are the most usual global maps. The global polynomial map can be
expressed as
YG(x) = c0 + C1x + xtC2x …
= Λ(x)a

(9.5)

where a contains all the unknown coefficients in c0, C1, C2, etc.
Registration algorithms must estimate the deformation from the corresponding
source and target images. This process can be done in one step by obtaining directly
both the global and the local deformation, usually decomposed as an addition.
Alternatively, many registration algorithms use a two-step approach by which the
global map is first obtained and, then, the local map is obtained from the globally
transformed source image and the target one, leading to the composition formulation.

* Both forms are equivalent, and it is possible to switch easily between them, i.e.:
YG ( x) = TG ( x)
YL ( x ) = TL [TG ( x )] − TG ( x )
Copyright 2005 by Taylor & Francis Group, LLC

TG ( x) = YG ( x)
TL ( x ) = x + YL YG−1 ( x ) 


2089_book.fm Page 345 Tuesday, May 10, 2005 3:38 PM

Landmark-Based Registration of Medical-Image Data

345

9.3 LANDMARK-BASED IMAGE ANALYSIS
Landmarks are singular points of correspondence in objects with a highly descriptive
power. They are commonly used in morphometrics [13] to describe shape and to
analyze intra- and interpopulation statistical differences. In particular, local differences of shape between two objects are commonly studied by reconstructing a
deformation that maps both objects from their homologous landmark correspondences. The most popular approach to making the reconstruction of the deformation
is based on independent interpolating thin-plate splines for each coordinate [10].
Approximating thin-plate splines can also be used when a trade-off between actual
confidence on landmark positions and smoothness of the deformation is desired.
The trade-off is controlled with a smoothing parameter that can be either estimated
by cross-validation (something usually involving a nonstraightforward optimization)
or just by an ad hoc guess [13].
The former approach has also been applied to image registration [10, 12]. In
this case two two-dimensional (2-D) or three-dimensional (3-D) images contain the
corresponding objects, and the deformation is the geometric mapping between both
images. Hence, registration consists of finding this mapping from both images. In
this case, landmarks are extracted from the images.
Landmarks are referred to in the literature in different ways, e.g., control points,
fiducials, markers, vertices, sampling points, etc. Different applications and communities, as ever, usually have different jargons. This is also true for different
classifications on landmark types. For example:
A usual classification
Anatomical landmark: point assigned by an expert that corresponds between organisms in some biologically meaningful way
Mathematical landmark: points located on an object according to some
mathematical property (e.g., curvature maximum)
Pseudo-landmark: points located in between anatomical or mathematical
landmarks to complete a description (They can also lie along outlines.
Continuous curves and surfaces can be approximated by a large number
of pseudo-landmarks.)
Another usual classification

Type I landmark: a point whose location is supported by the strongest evidence, such as the joins of tissue/bone or a small patch of some unusual
histology
Type II landmark: a point whose location is defined by a local geometric
property
Type III landmark: a landmark having at least one deficient coordinate, for
instance, either end of a longest diameter, or the bottom of a concavity
(Type III landmarks characterize more than one region.)
A useful classification for image registration
Normal landmark: point with a unique position or with an approximately
isotropic uncertainty around a mean position
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 346 Tuesday, May 10, 2005 3:38 PM

346

Medical Image Analysis

Quasi- (or semi-) landmark: point with one or more degrees of freedom,
i.e., it can slide along some direction or, with a highly anisotropic location uncertainty, around a mean position
Yet another classification
Unlabeled landmark: a point for which no natural labeling is available
Labeled landmark: a point for which a natural and unique identification
exists

9.4 LANDMARK DETECTION AND LOCATION
Before any deformation map can be reconstructed, landmarks must be detected and
located. These are not easy tasks, even for human experts. On the one hand, no
general detection paradigm (i.e., answering the question: is there any landmark

around?) can be used because the definition of landmarks varies from application
to application. On the other hand, locating landmarks accurately (once a landmark
has been detected it is necessary to estimate its exact position) on images is extremely
difficult because digital images are defined on discrete grids, and quite often they
are quasi-landmarks defined on smooth boundaries (and consequently with a high
uncertainty along these boundaries). For a human expert, things become even more
complicated when the images are 3-D, no matter what interaction approach with the
data is implemented to click-point on the landmark locations.
Therefore, it is important to count on reconstruction schemes of the deformation
map that are able to deal with the uncertainty in the extracted landmark positions.
A first step toward this goal is the use of approximating thin-plate splines mentioned
previously. Nevertheless, this scheme only considers isotropic noise models for the
landmark positions. A remarkable extension due to Rohr [16, 17] allows the incorporation of anisotropic noise models and, hence, quasi-landmarks, something important in order to deal with the registration of smooth boundaries. Anisotropic noise
models correspond to nondiagonal covariance matrices, with the obvious consequence of coupling the thin-plate splines formerly acting on each coordinate independently.
The location of N landmarks, extracted by any means from both images, can be
modeled as realizations of independent Gaussian random vectors (Xl and Xl′, l = 1,
…, N) with means equal to the correct landmark positions and covariance matrices
C Xl and C X ′ . Notice that nondiagonal covariance matrices account for anisotropic
l
uncertainty.
Another remarkable achievement of Rohr, which will be used in this chapter
extensively, is the derivation of the Cramer-Rao lower bound for the estimation of
a point landmark position [12] from discrete images of arbitrary dimensionality in
additive white Gaussian noise,
σ 2N I
Σ I ( x) =
m
Copyright 2005 by Taylor & Francis Group, LLC




∇I ( xk ) ⋅ ∇I ( xk )t 



k ∈M ( m )

∑

−1

(9.6)


2089_book.fm Page 347 Tuesday, May 10, 2005 3:38 PM

Landmark-Based Registration of Medical-Image Data

347

where σ 2N I denotes the variance of the noise, and M(m) is a neighborhood around
the landmark with m elements. We will also assume this result to model the covariance of the manually extracted landmarks directly from the image data.

9.5 OUR APPROACH TO LANDMARK-BASED REGISTRATION
We will consider that the deformation that puts into correspondence the source and
target images is a realization of a vector random field. The global component of the
deformation corresponds to the trend (mean) of the random field, whereas the local
component of deformation is modeled by an intrinsically stationary random field.
The field is sampled by means of landmark correspondences, i.e., to each landmark
in the source image corresponds a landmark in the target one, which are then used

to reconstruct the whole realization of the random deformation field.
The geostatistical method tries to honor actual observations by estimating the
model spatial variability directly from available data. This essentially consists of
estimating the variogram of the field, which is a difficult problem, especially if it is
to be done from landmarks displacements, because there are usually just a few. This
has possibly prevented Kriging’s method from being used in landmark-based registration. Here we propose a practical way to circumvent these difficulties by splitting
the approach into three steps:
1. Image-based global registration: Estimating the variogram of the displacement field requires detrending of the data. To avoid introducing any
subsequent bias into the variogram estimation, we propose to make an
intensity-based global (i.e., rigid or affine) registration first to remove the
trend effect, with a variety of algorithms being available. For example,
rigid registration by maximization of mutual information is a well-known
algorithm [18] that can be used when image intensities in both images
are different.
2. Model estimation: Estimating the variogram structure of the detrended
displacement field is still a difficult task. The number of available landmarks in most practical applications is almost never enough to make good
variogram estimations, and trying to extract a significant number from the
images would render the method impractical. We propose to use a fast,
general-purpose, nonrigid registration algorithm to obtain an approximate
dense displacement field. Again, a number of algorithms are available,
although we are using, with excellent results, a regularized block-matching scheme with mutual information (and others) similarity measure that
was developed by our team [19]. The variogram is then readily estimated
from this field.
3. Landmark-based local registration: Landmarks are extracted from the
registered image pair and used to reconstruct a realization of a zero-mean
random deformation field using ordinary Kriging, with the variogram
structure just estimated.

Copyright 2005 by Taylor & Francis Group, LLC

2089_book.fm Page 348 Tuesday, May 10, 2005 3:38 PM

348

Medical Image Analysis

9.6 DEFORMATION MODEL ESTIMATION
9.6.1 INTENSITY-BASED REGISTRATION
The model estimation, as noted previously, relies on a fast, general-purpose, intensity-based, nonrigid registration algorithm to obtain an approximate dense displacement field. This registration framework is presented in the following subsections.
To understand the design criteria of our algorithm, general properties of registration algorithms are shown. To simplify the exposition, we will restrict the discussion to three-dimensional medical images.
Let I1 and I2 be two medical images, i.e., two scalar functions defined on two
regions of the space. We will use two different coordinate systems x and x′ for each
one. The registration problem consists of finding the transformation x′ = Y(x) that
relates every point x in the coordinate system of I1 with a point x′ in the coordinate
system of I2.
The criteria of correspondence are usually set by means of high-level information, for example anatomical knowledge. However, when coding the correspondence
into a registration algorithm, some properties should be satisfied.
Invertibility of the solution: A registration algorithm should provide an invertible solution. Invertibility implies the existence of an inverse transformation
x = Y*(x′) that relates every point on I2 back to a point on I1, where Y* =
Y−1. It is satisfied if the Jacobian of the transformation is positive.
No boundary restriction: A registration algorithm should not impose any
boundary condition. Boundary restrictions, sometimes in the model, sometimes in the representation of the warping, are usually set to help either
implementation or convergence of the search technique. However, boundaries are acquisition dependent, not data dependent, so they are a fictitious
matching in the solution. Thus, ideal registration should provide free-form
warpings.
Intensity channel insensitivity: Another desirable property of a registration
algorithm is the insensitivity to noise or to a bias field in the acquisitions.
These variations are usually dealt with by an entropy-based similarity
measure.

Possibility of large deformations: Some registration schemes are based on
models such as linear elastic models, which are not thought to be useful
for large deformations. The theory of linear elasticity is successful whenever
relative displacements are small. Hence, mechanical models should be used
with care when trying to register tissue deformations.
9.6.1.1 Template Matching
Intensity-based registration methods, i.e., those using directly the full content of the
image and not simplifying it to a set of features to steer the registration, usually
correspond to one of two important families: template matching and variational. The
former was popular years ago because of its conceptual simplicity [20]. Nevertheless,
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 349 Tuesday, May 10, 2005 3:38 PM

Landmark-Based Registration of Medical-Image Data

349

in its conventional formulation, it is not powerful enough to address the challenging
needs of medical-image registration. Variational methods rely on the minimization
of a functional (energy) that is usually formulated as the addition of two terms: data
coupling and regularization, the former forcing the similarity between both data sets
(target and source deformed with the estimated field) to be high, and the latter
enforcing the estimated field to fulfill some constraint (usually enforcing spatial
coherence-smoothness). As opposed to variational methods, template matching does
not impose any constraint on the resulting fields, which, moreover, due to the discrete
movement of the template, turn out to be discrete as well. These facts have led to
an increasing popularity of variational methods for registration, while template
matching has been losing ground in this arena.

Template matching finds the displacement for every voxel in a source image by
minimizing a local cost measure that is obtained from a small neighborhood of the
source image and a set of potential correspondent neighborhoods in a target image.
The main disadvantage of template matching is that it estimates the displacement
field independently in every voxel, and no spatial coherence is imposed to the
solution. Another disadvantage of template matching is that it needs to test several
discrete displacements to find a minimum.
There are several optimization-based template-matching solutions that provide
a real solution for every voxel, although they are slow [21]. Therefore, most
template-matching approaches render discrete displacement fields. Another problem associated with template matching is commonly denoted as the aperture
problem in the computer-vision literature [22]. This essentially consists of the
inability of making a good match when no discriminant structure is available, such
as in homogeneous regions, surfaces, and edges. When this fact is not taken into
account, the matching process is steered by noise and not by the local structure,
because it is not available.
The model-estimation registration algorithm that we present here maintains the
simplicity of template matching while addressing its drawbacks. It consists of a
weighted regularization of the template-matching solution, where weights are
obtained from the local structure, to render spatially coherent real deformation fields.
Thanks to the multiscale nature of our approach, only displacements of one voxel
on every scale are necessary when matching the local neighborhoods.
9.6.1.2 Multiresolution Pyramid
The algorithm works in a way that is similar to the Kovaˇciˇc and Bajcsy elastic
warping [23], in which images are decomposed on Gaussian multiresolution pyramids. On the highest level, the deformation field is estimated by regularized template
matching steered by local structure (details in the following subsections). On the
next level, the source data set is deformed with a deformation field obtained by
spatial interpolation of the one obtained on the first level. The deformed source and
the target data sets on the current level are then registered to obtain the deformation
field corresponding to the current level of resolution. This process is iterated on
every level. The algorithm implementation is summarized in Figure 9.1.

Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 350 Tuesday, May 10, 2005 3:38 PM

350

Medical Image Analysis

(i1)

Previous scale level
(i)

Image 1

(i)

Image 1
Transformed

Data
Matching
1 2 Step
Deformation(i)

Image 2

(i)

Local
Structure

1 2 Global
Deformation(i)

(i+1)

Next scale level

FIGURE 9.1 Algorithm pipeline for pyramidal level (i).

FIGURE 9.2 (Color figure follows p. 274.) MRI T1-weighted axial slice of human brain
and its structure tensors. (Hot color represents high structure.)

9.6.1.3 Local Structure
Local structure measures the quantity of discriminant spatial information on every
point of an image, and it is crucial for template-matching performance: the higher
the local structure, the better is the result obtained on that region with template
matching.
To quantify local structure, a structure tensor is defined as T(x) = (∇I(x)⋅⋅∇I(x)t)σ,
where the subscript σ indicates a local smoothing. The structure tensor consists of
a symmetric positive-semidefinite 3×3 matrix that can be associated with ellipsoids,
i.e., eigenvectors and eigenvalues correspond to the ellipsoids’ axes directions and
lengths, respectively. A scalar measure of the local structure can be obtained as [16,
17, 24].
structure( x) =

det T( x)
trace T( x)

(9.7)

Figure 9.2 shows an MRI T1-weighted axial slice of the brain and the estimated
structure tensors overlaid as ellipsoids. Small eigenvalues indicate a lack of gradient
variation along the associated principal direction, and therefore, high structure is
indicated by big (large eigenvalues), round (no eigenvalue is small) ellipsoids. The
color coding represents the scalar structure measure, with hot colors indicating higher
structure.
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 351 Tuesday, May 10, 2005 3:38 PM

Landmark-Based Registration of Medical-Image Data

351

FIGURE 9.3 (Top) MRI T1-weight cross-sections; (bottom) local structure measure (arrows
point at higher structure regions).

Figure 9.3 shows cross-sections of a T1-weighted MRI dataset of a human brain
(top row) and the scalar measure of local structure obtained from them, represented
with a logarithmic histogram correction (bottom row). Note how anatomical landmarks have the highest measure of local structure, corresponding to the points
indicated by the arrows on the top row. Curves are detected with lower intensity
than points, and surfaces have even lower intensity. Homogeneous areas have almost
no structure.
Template matching provides a discrete deformation field where no spatial coherence constraints have been imposed. In the discussion in this subsection, this field
is regularized so as to obtain a mathematically consistent continuous mapping. We
will consider the deformation field to be a diffeomorphism, i.e., an invertible continuously differentiable mapping. To be invertible, the Jacobian of the deformation

field must be positive. On every scale level, the displacement is small enough to
guarantee such a condition. For every level of the pyramid, the mapping is obtained
by composing the transformation on a higher level than the one on the current level,
so that the positive Jacobian condition is preserved.
Spatial regularization is achieved by locally projecting the deformation field
provided by template matching on an appropriate signal subspace, and simultaneously taking into account the quality of the matching as indicated by the scalar
measure of local structure. We propose here to use normalized convolution [25, 26],
a popular refinement of weighted-least squares that explicitly deals with the socalled signal/certainty philosophy. Essentially, the scalar measure of structure is
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 352 Tuesday, May 10, 2005 3:38 PM

352

Medical Image Analysis

incorporated as a weighting function in a least squares fashion. The field obtained
from template matching is then projected onto a vector space described by a nonorthogonal basis, i.e., the dot products between the field and every element of the
basis provide covariant components that must be converted into contravariants by
an appropriate metric tensor. Normalized convolution provides a simple implementation of this operation. Moreover, an applicability function is enforced on the basis
elements to guarantee a proper localization and avoid high-frequency artifacts. This
essentially corresponds to weighting each basis element with a Gaussian window.
The desired transformation is related to the displacement field by the simple relation
shown in Equation 9.3.
Because the transformation is differentiable, we can write the function in different orders of approximation
Y ( x ) ≈ Y ( x0 )

(9.8)

Y ( x ) ≈ Y ( x0 ) + J ( x0 ) ⋅ ( x − x0 )

(9.9)

Equation 9.8 and Equation 9.9 consist of linear decompositions of bases of size 3
and 12 basis elements, respectively. We have not found relevant experimental
improvement of the registration algorithm by using the linear approximation instead
of the zero-order one, probably due to the local nature of the algorithm. The basis
set used is then
 Y 1 ( x) = 1
Y 1 ( x) = 0
Y 1 ( x) = 0
 2
 2

b1 = Y ( x) = 0 b2 =  Y ( x)) = 1 b3 = Y 2 ( x) = 0
 Y 3 ( x) = 1
Y 3 ( x) = 0
Y 3 ( x) = 0




(9.10)

Figure 9.4 shows a 2-D discrete deformation field that has been regularized using
the certainty on the left side and a 2-D Gaussian applicability function with σ = 0.8.

FIGURE 9.4 (Left) certainty, (center) discrete matching deformation, (right) weight-filtered
deformation.

Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 353 Tuesday, May 10, 2005 3:38 PM

Landmark-Based Registration of Medical-Image Data

353

9.6.1.4 Entropy-Based Similarity Measure
In a work by Suarez et al. [19], the registration framework was tested using square
blocks that were matched using the sum of squared differences and correlation
coefficient as similarity measures. In the current work, we introduce entropy-based
similarity measures into this framework, although it can be used by any algorithm
based on template matching.
A similarity measure can be interpreted as a function defined on the joint
probability space of two random variables to be matched. In the case of block
matching, each block represents a set of samples from each random variable. When
this probability density function (PDF) is known, mutual information can be computed as
MI ( I1 , I 2 ) =

∫

Ω

p (i1 , i2 ) log

p (i1 , i2 )
di1 di2
p (i1 ) p (i2 )

(9.11)

where I1, I2 are the images to register, and Ω is the joint probability function space.
A discrete approximation is to compute the mutual information from the PDF
and a small number N of samples (i1[k], i2[k])
N

MI ( I1, I 2 )

∑
k =1

log

p(i1[k ], i2 [k ])
=
p(i1[k ]) p(i2 [k ])

N

∑ f (i [k], i [k])
p

1

2

(9.12)

k =1

where ƒp is a coupling function defined on Ω. Therefore, the local evaluation of the
mutual information for a displaced block containing N voxels can be computed just
by summing the coupling function ƒp on the k samples that belong to this block.
We propose to compute a set of multidimensional images, each of them containing at each voxel the local similarity measure corresponding to a single displacement applied to the whole target image. A decision will be made for each voxel,
depending on which displacement renders the greatest similarity.
A problem associated with local entropy-based similarity measures is the local
estimation of the joint PDF of both blocks, because there are never enough samples
available. We propose to overcome this problem by using the joint PDF corresponding to the whole displaced source image and the target one. The PDF to be used for
a given displacement is the global joint-intensity histogram of the reference image
with the displaced target image. This is crucial for higher pyramidal levels, where
one voxel displacement drastically changes the PDF estimation.
It is straightforward to compute the local mutual information for a given discrete
displacement in the whole image. This requires only the convolution of a square
kernel representing the block window and the evaluation of the coupling function
for every pair of voxels. Furthermore, because the registration framework only needs
discrete deformation fields, no interpolation is needed in this step. Any similarity
measure that can be computed as a kernel convolution can be implemented this way.
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 354 Tuesday, May 10, 2005 3:38 PM

354

Medical Image Analysis
Image 1

Image 1

Image 2
Image 2

FIGURE 9.5 (Left) target image to be matched, (center) reference image where similarity
measure is going to be estimated for every discrete displacement, (right) for every discrete
displacement, the similarity measure is computed for every voxel by performing a convolution.

A small sketch of this technique is shown in Figure 9.5. For smoothness and locality
reasons, we have chosen to convolve using Gaussian kernels instead of square ones.
To achieve a further computational saving, Equation 9.12 can be written as
N

MI ( I1, I 2 )

∑ (log p(i [k], i [k]) − log p(i [k]) − log p(i [k]))
1

2

1

2

(9.13)

k =1

The displacement field defines the displacement of a voxel in the source image.
The similarity measure will be referred to as the source-image reference system

(image 1). For a given voxel in the source image, the comparison of Equation 9.13
for different displacement will always contain the same terms, depending on p(i1[k]).
Thus, we can take this term off and modify accordingly the coupling function to
reduce computational cost. Any other entropy-based similarity measure can be
estimated in a similar way. The computational cost is then very similar to any other
similarity measure not based on entropy.

9.6.2 VARIOGRAM ESTIMATION
The variogram is estimated under the assumption of intrinsic stationarity (i.e., the
mean of the displacement field must be constant) from the displacement field
obtained by intensity-based image registration. Should intrinsic stationarity not be
the case, a trend model must be pre-estimated so that it can be substrated from the
field prior to estimating the variogram. This process is undesirable because it introduces bias in the variogram estimation due to its inherent circularity: the probabilistic
characterization of the random component of the field must be known to estimate
the trend, but the trend must also be known to estimate the probabilistic characterization of the random component. Nevertheless, this issue is present in any model
with a trend and a random component, and, in fact, estimating the sample variogram
instead of the sample autocovariance has several advantages [14] from this point of view:
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 355 Tuesday, May 10, 2005 3:38 PM

Landmark-Based Registration of Medical-Image Data

355

If the mean value of the field is an unknown constant, it is not necessary to
pre-estimate it because the variogram sample estimator is based on differences. Hence, in this case, the sample variogram can be estimated unbiasedly.
The sample variogram estimator is more robust against mean model mismatch
than the sample autocovariance one.

The sample variogram estimator is less biased than the sample autocovariance
one when the mean model is pre-estimated and subtracted from the field
realization to make the spatial-dependence model estimation.

9.7 LANDMARK-BASED LOCAL REGISTRATION
9.7.1 DISPLACEMENT FIELD MODEL
The reconstruction of the local displacement field DL(x), can be cast as the optimal
prediction of the displacement at every location x from our set of observations*.
These observations are obtained by measuring the displacement between pairs of
point landmarks extracted from both images. The observation process is then
Z(x) = X′(x) − X(x)
= D(x) + Nz(x)

(9.14)

where X, X′ are the landmark position random processes, D is the stochastic characterization of the local displacement field, and NZ consists of a zero-mean Gaussian
random noise field with autocovariance independent of D.
From the model, it follows that
µZ(x) = µD(x)

(9.15)

CZ(x) = CX′(x) + CX(x)

(9.16)

CZ(xi, xj) = CD(xi, xj)

(9.17)

Furthermore, Equation 9.16 can be rewritten for the sampled landmarks (xl, x′l) as
C Z ( xl ) = C Xl + C Xl = Σ I1 ( xl ) + Σ I 2 ( xl )

(9.18)

where the Cramer-Rao lower bound introduced in Section 9.4 has been used.

* Hereinafter, the L subscript will be omitted.

Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 356 Tuesday, May 10, 2005 3:38 PM

356

Medical Image Analysis

9.7.2 ORDINARY KRIGING PREDICTION

OF

DISPLACEMENT FIELDS

The mean for each component of the displacement field, µD(x), is assumed to be an
unknown constant. We have found that this is a very convenient model, even after
the global preregistration that should render zero-mean values for the resulting
displacement components. The reason is that usually a locally varying mean structure
can model much of the local deformation. Therefore, in this case we will not use
all the samples but a limited number around the prediction location. This has the

added benefit of reducing the computational burden.
For the sake of simplicity, positions of the observed landmarks will be denoted
by the set O = {x1, …, xN}, and the observation vector is denoted
Zr(O) = [Zr(x1) … Zr(xN)]t

(9.19)

The ordinary co-Kriging (i.e., multivariate Kriging) predictor takes the form
t
 Dˆ 1( x)   k11 ( x, O )
 
ˆ ( x) ≡ 
D

 =
 ˆ d ( x)  d t ( x, O )
D
  k1

…
…

kd1 ( x, O )   Z1 (O ) 




  Zd (O )
dt




kd ( x, O )
t

 k1 t ( x, O ) 


=
 Z(O )
t
 d ( x, O )
k

= K ( x, O ) Z(O )

(9.20)

If there is no second-order probabilistic dependence among the field components,
each of them is dealt with independently, leading to a block-diagonal K(x,O) matrix
and resulting in the conventional ordinary Kriging predictor for each component.
The ordinary Kriging coefficients must minimize the mean square prediction error
MSPE r ( x, O ) = E[( Dr ( x) − k r t Z(O ))2 ]
t

= E[( Dr ( x) − µ Dr ( x) − k r (Z(O ) − µ Z (O )))2 ]
t

t

= σ 2Dr ( x) − 2 k r C Z Dr (O, x) + k r C Z (O ) k r ,
Copyright 2005 by Taylor & Francis Group, LLC

(9.21)


2089_book.fm Page 357 Tuesday, May 10, 2005 3:38 PM

Landmark-Based Registration of Medical-Image Data

357

subject to the unbiasedness constraint
ˆ ( x)] = E[D( x)]
E[D

(9.22)

Closed-form equations for the coefficients and for the achieved squared error
can be readily obtained after some algebra (see, for example, Cressie [14]). Because
of space constraints, we only present the coefficients’ equation, expressed in terms
of covariances. The matrix Λ is block diagonal, with each diagonal block equal to
a column vector of ones, and the vector λr is a zero row vector with a single 1 in
the r position:
k r = C −Z1 (O )[C ZDr (O , x) − Λ( Λ t C −Z1 (O ) Λ)−1 ( Λ t C−Z1 (O )C ZDr (O , x) − λ r )]

(9.23)

Extensions of ordinary Kriging are possible by incorporating more complex
mean structure models. Though this could seem in principle appealing, it has the

serious drawback of hindering the estimation of the spatial variability model, because
the mean structure has to be filtered out before the covariance structure can be
estimated. Notice that estimating the variogram does not require pre-estimation of
the mean, as this is constant.

9.8 RESULTS
We are currently using the proposed framework in a number of applications. To
better illustrate its behavior, we have selected two simple experiments. Figure 9.6(a)
shows a T1w MRI axial slice of a multiple sclerosis patient, and Figure 9.6(b) a
corresponding T2w axial slice of a different patient. Ellipsoids representing landmark
covariances have been overlaid (seven landmarks in the brain and four on the skull).
Figure 9.6(d) and Figure 9.6(e) show two T1w mid-sagittal slices of MS patients,
also with covariance landmark ellipsoids overlaid (11 landmarks in the brain and 3
on the skull). In each case, the second image is to be warped onto the first one.
In both cases the images are first globally registered. Then a forward displacement field is obtained for each one using our general-purpose general registration
scheme [19] to estimate the variograms. Sample variograms and their weighted-least
squares fit to theoretical models (linear combination of Gaussian and power models)
are shown in Figure 9.6(g) and Figure 9.6(h). For this purpose, 5000 displacements
were sampled, which makes the estimation highly accurate.
Registration results are shown in Figure 9.6(c) and Figure 9.6(f) by ordinary
Kriging prediction of the displacement field, using only the displacements from the
landmarks on the images. Notice how even with so few landmarks, a good result is
achieved, especially in areas closer to the landmarks, because of the proper estimation of the random displacement field. The open-source software Gstat [27] was
used in these experiments.

Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 358 Tuesday, May 10, 2005 3:38 PM

358

45

semivariogram

40

Medical Image Analysis

(a)

(b)

(c)

(d)

(e)

(f)

dy: sample variogram
dy: 6.78018 Gau(16.3546) + 0.0254245 Pow(1.32512)
dx: sample variogram
dy: 27.886 Gau(32.4634) + Pow 0.380956 Pow(0.788065)

90
dx: sample variogram
dx: 10.2934 Gau(18.4780) + 0.0355 Pow(1.2746)

dy: 24.7626 Gau(33.4645) + 0.2789 Pow(1.1992)
dy: sample variogram

80

35

70

30

60
50

25

40

20

30
15
20
10
10
5
0
0
0

10

20

30

40

h

50

(g)

60

70

80

90

–10

0

10

20

30

40

50

60

70

80

90

(h)

FIGURE 9.6 Experimental results: (a) axial T1, (b) axial T2, (c) warped axial T2, (d) first
T1 sagittal, (e) second T1 sagittal, (f) warped second sagittal, (g) displacement variograms
(axial), and (h) displacement variograms (sagittal).

9.9 CONCLUSIONS
We have presented a practical approach to the statistical prediction of displacement
fields from pairs of landmarks. The method is grounded on the solid theory of
ordinary Kriging, and it also provides a way of estimating the spatial-dependence
models from image data, thus circumventing some of the hurdles found when using
Kriging. The fact that the statistical relation between both geometries is successfully
used makes the method highly accurate and particularly well suited for imageCopyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 359 Tuesday, May 10, 2005 3:38 PM

Landmark-Based Registration of Medical-Image Data

359

registration and shape-analysis applications. It is remarkable to note that thin-plate
splines can be considered a particular case of Kriging, and in this sense, our approach
generalizes this popular registration method.

APPENDIX 9.1

GEOSTATISTICAL SPATIAL MODELING

Consider a random field Zr(x) (the superscript r is meant to consider several random
fields, such as the components of a vector random field) such that
var[ Z r ( xi ) − Z r ( x j )] = 2 γ Z r ( xi − x j ), ∀xi , x j ∈Ω

(9.24)

The function 2γ Z r ( h) , with h = xi − xj, is called the variogram of the random
field Zr(x) and, assuming it exists, is the central parameter to model the spatial
dependence of the random field in the geostatistical method. The variable γ Z r ( h)
(without the 2 factor) is usually called the semivariogram. The variogram can be
easily related to the variance and covariance from the relation
var[ Z r ( xi ) − Z r ( x j )] = σ 2Z r ( xi ) + σ 2Z r ( x j ) − 2C Z r ( xi , x j )

(9.25)

The shape of a variogram is summarized by the following parameters:
Nugget: it is the size of the discontinuity of the semivariogram at the origin.

Note that the presence of a nugget other than zero indicates that the random
field is not continuous. The presence of a nugget effect is usually attributed
to measurement noise and to a very local random component of the field
that appears as uncorrelated at the working resolution. Both effects are
usually superimposed and modeled with white noise.
Sill: if the variogram is bounded, the sill is the value of the bound. A sill
indicates total noncorrelation as, for example, with white noise. Usually,
random fields become uncorrelated for big lags, reaching a sill.
Partial sill: it is the difference between the sill and the nugget.
Range: it is the lag for which the sill is reached, of course assuming there is
a sill in the variogram.
Various approaches for constructing valid theoretical variogram models are
available [14, 27–30]. Most often, existing variogram models such as nugget (white
field), spherical, linear, exponential, power, etc. are used as building blocks in a
linear combination of valid variogram models, making use of the convexity of the
set of valid variograms.
The variogram can be extended for the multivariate case [14]. The pseudo-crossvariogram function is defined as
2 γ Z r Z s ( xi − x j ) = var[ Z r ( xi ) − Z s ( x j )]
Copyright 2005 by Taylor & Francis Group, LLC

(9.26)


2089_book.fm Page 360 Tuesday, May 10, 2005 3:38 PM

360

Medical Image Analysis

A9.1.1

INTRINSIC STATIONARITY

The scalar random field Zr(x) is said to be intrinsically stationary if it has a constant
mean E[ Z r ( x)] = µ Z r and its variogram exists. Moreover, any conditionally negativedefinite function 2γ(h) is the variogram of an intrinsically stationary random field.
The variogram of an intrinsic random field Zr(x) is
2 γ Z r ( h) = E[( Z r ( x + h) − Z r ( x))2 ]

A9.1.2

RELATION

BETWEEN INTRINSIC AND

(9.27)

SECOND-ORDER STATIONARITIES

Note that the family of intrinsic stationary fields is larger than the second-order
stationary one. In particular, unbounded valid variograms, i.e., variograms without
a sill, do not have a corresponding autocovariance function. For second-order stationary fields, there is a simple relation between the variogram and the autocovariance, i.e.,
2 γ Z r ( h) = 2(C Z r (0) − C Z r ( h))

(9.28)

It is clear that in the common situation for second-order stationary fields where
the covariance approaches zero for large space lags, the sill of the variogram is
2C Z r (0) .

ACKNOWLEDGMENT

This work has been partially funded by the Spanish Government (MCyT) under
research grant TIC-2001-3808-C02-01.

REFERENCES
1. Faugeras, O., Three-Dimensional Computer Vision: a Geometric Viewpoint, MIT
Press, Cambridge, MA, 1993.
2. Shah, M. and Jain, R., Eds., Motion-Based Recognition, Vol. 9, Computational Imaging and Vision, Kluwer, Dordrecht, Netherlands, 1997.
3. Tekalp, A.M., Digital Video Processing, Signal Processing Series, Prentice Hall,
Upper Saddle River, NJ, 1995.
4. Lillesand, T.M. and Kiefer, R.W., Remote Sensing and Interpretation, 4th ed., John
Wiley & Sons, New York, 1999.
5. Burrough, P.A. and McDonell, R.A., Principles of Geographic Information Systems
(Spatial Information Systems and Geostatistics), 2nd ed., Oxford University Press,
Oxford, U.K., 1988.
6. Maintz, J.B.A. and Viergever, M.A., A survey of medical-image registration, Medical
Image Anal., 2, 1–36, 1998.
7. Ruiz-Alzola, J., Suárez, E., Alberola-López, C., Warfield, S.K., and Westin, C.-F.,
Geostatistical medical-image registration, in Lecture Notes in Computer Science, no.
2879, Springer-Verlag, New York, 2003, pp. 894–901.
Copyright 2005 by Taylor & Francis Group, LLC


2089_book.fm Page 361 Tuesday, May 10, 2005 3:38 PM

Landmark-Based Registration of Medical-Image Data

361

8. Bajcsy, R. and Kovacˇiˇc, S., Multiresolution elastic matching, Computer Vision,
Graphics, Image Process., 46, 1–21, 1989.

9. Christensen, G.E., Joshi, S.C., and Miller, M.I., Volumetric transformation of brain
anatomy, IEEE Trans. Medical Imaging, 16, 864–877, 1997.
10. Bookstein, F.L., Principal warps: thin-plate splines and the decomposition of deformations, IEEE Trans. Pattern Anal. Machine Intelligence, 11, 567–585, 1989.
11. Rohr, K., Image registration based on thin-plate splines and local estimates of anisotropic landmark localization uncertainties, in Lecture Notes in Computer Science, no.
1496, Springer-Verlag, Heidelberg, 1998, pp. 1174–1183.
12. Rohr, K., Landmark-based image analysis (using geometry and intensity models),
Vol. 21, Computational Imaging and Vision, Kluwer, Dordrecht, Netherlands, 2001.
13. Dryden, I.L. and Mardia, K.V., Statistical Shape Analysis, Wiley Series in Probability
and Statistics, John Wiley & Sons, New York, 1998.
14. Cressie, N.A.C., Statistics for Spatial Data, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, New York, 1993.
15. Matheron, G., Splines and Kriging: their formal equivalence, in Down-to-Earth Statistics: Solutions Looking for Geological Problems, Syracuse University Geological
Contributions, Syracuse, NY, 1981, pp. 77–95.
16. Rohr, K., Differential operators for detecting point landmarks, Image Vision Computing, 15, 219–233, 1997.
17. Harris, C. and Stephens, M., A combined corner and edge detector, in Proc. Fourth
Alvey Vision Conference, 1988, pp. 147–151.
18. Wells, W.M., Viola, P., Atsumi, H., Nakajima, S., and Kikinis, R., Multimodal volume
registration by maximization of mutual information, Medical Image Anal., 1, 35–51, 1996.
19. Suárez, E., Westin, C.-F., Rovaris, E., and Ruiz-Alzola, J., Nonrigid registration using
regularized matching weighted by local structure, in Lecture Notes in Computer
Science, no. 2489, Springer-Verlag, Heidelberg, 2002, pp. 581–589.
20. Duda, R.O. and Hart, P.E., Pattern Classification and Scene Analysis, John Wiley &
Sons, New York, 1973.
21. Suárez, E., Cárdenes, R., Alberola, C., Westin, C.-F., and Ruiz-Alzola, J., A general
approach to nonrigid registration: decoupled optimization, in 23rd Ann. Int. Conf.
IEEE Eng. Med. Biol. Soc., IEEE, Washington, DC, 2000.
22. Poggio, T., Torre, V., and Koch, C., Computational vision and regularization theory,
Nature, 317, 314–319, 1985.
23. Kovacic, S. and Bajcsy, R.K., Multiscale/multiresolution representations, in Brain
Warping, Academic Press, New York, 1999, pp. 45–65.
24. Ruiz-Alzola, J., Kikinis, R., and Westin, C.-F., Detection of point landmarks in

multidimensional tensor data, Signal Process., 81, 2243–2247, 2001.
25. Westin, C.-F., A Tensor Framework for Multidimensional Signal Processing, Ph.D.
Thesis, Linköping University, Sweden, 1994.
26. Knutsson, H. and Westin, C.-F., Normalized and differential convolution: methods
for interpolation and filtering of incomplete and uncertain data, in Proc. Computer
Vision and Pattern Recognition, IEEE, New York, 1993, pp. 515–523.
27. Pebesma, E.J. and Wesseling, C.G., Gstat: a program for geostatistical modelling,
prediction and simulation, Comput. Geosci., 24, 17–31, 1998.
28. Chiles, J.-P. and Delfiner, P., Geostatistics: Modeling Spatial Uncertainty, Wiley
Series in Applied Probability and Statistics, Wiley-Interscience, New York, 199.
29. Ripley, B.D., Statistical Inference for Spatial Processes, repr., Cambridge University
Press, Cambridge, U.K., 1991.
30. Arlinghaus, S.L. and Griffith, D.A., eds., Practical Handbook of Spatial Statistics,
rev. ed., CRC Press, Boca Raton, FL, 1995.
Copyright 2005 by Taylor & Francis Group, LLC