Extraction of man made features from high resolution satellite imagery

EXTRACTION OF MAN-MADE FEATURES FROM
HIGH RESOLUTION SATELLITE IMAGERY

SOWMYA SELVARAJAN
(B.E., Anna University, India)

A THESIS SUBMITTED
FOR THE DEGREE OF MASTER OF ENGINEERING
DEPARTMENT OF CIVIL ENGINEERING
NATIONAL UNIVERSITY OF SINGAPORE
2002


ACKNOWLEDGEMENTS

The author would like to thank her supervisor, Associate Professor Chan Weng
Tat for initiating and encouraging her interest in application-based remote sensing and for
providing advice and direction.

The author is also thankful to her parents and all her friends who have been
providing moral support all along the way.

i


TABLE OF CONTENTS

Acknowledgements

i

Table of Contents

ii

List of Figures

vi

List of Tables

ix

List of Acronyms

x

Chapter

Chapter

1

Introduction

1

1.1

Background of the research

1

1.2

Scope

3

1.3

Objective of the study

5

1.4

Organization of the thesis

6

2

Literature Review

7

2.1

Introduction

7

2.2

Satellite remote sensing of urban areas

8

2.2

Wavelet approaches to image processing

9

2.3

Current edge detection methods

11

2.4

Template matching methods

14

2.5

Summary

16

ii


Chapter

3

Existing methods and underlying theories for urban

remote sensing

18

3.1

Urban remote sensing

18

3.1.1

Fundamentals of remote sensing

18

3.1.2

Extraction of information through digital

image processing

3.2

3.3

19

Wavelet Theory

21

3.2.1

Introduction to wavelets

21

3.2.2

Comparison of Fourier and wavelet transforms

23

3.2.3

Discrete Wavelet Transform

26

3.2.4

Multi-resolution Wavelet Analysis

27

Edge detection approach

29

3.3.1

Purpose of edge enhancement

29

3.3.2

Types of edge operators

29

3.3.3

The Canny edge algorithm

30

3.4

Template matching

3.5

Proposed wavelet-edge template matching technique for man-made

object extraction

32

33

3.5.1

Detection of local intensity variation

34

3.5.2

Edge based segmentation approach

36

3.5.3

Shape classification

37

iii


Chapter

4

Details of implementation

38

4.1

Focus of the study

38

4.2

Imagery and Study Area

38

4.3

Parameters for wavelet analysis

41

4.3.1

Choice of wavelet type

41

4.3.2

Approximations and details of the wavelet analysis 42

4.3.3

Wavelet resolution

4.4

Chapter

44

Parameters for Canny edge detection

46

4.4.1

Threshold parameter

47

4.4.2

Standard Deviation

51

4.5

Morphological operations

54

4.6

Parameters for template matching

55

4.5.1. Shape models

55

4.5.2

Block Matching

56

4.5.3

Edge-based Matching

58

5

Results and discussions

61

5.1

Results of feature extraction

61

5.2

Assessment of accuracy

64

5.3

Key features of the proposed method

66

5.4

Validation of the algorithm

73

iv


Chapter

6

Conclusions

79

6.1

Conclusions

79

6.2

Future Improvements

81

References

82

Appendix 1

87

v


LIST OF FIGURES

Fig. 3.1

Fourier basis functions, time-frequency tiles,
and coverage of the time-frequency plane

Fig. 3.2

26

Daubechies wavelet basis functions,
time-frequency tiles, and coverage of the
time-frequency plane

27

Fig. 3.3

Flow chart depicting the proposed algorithm

34

Fig. 4.1

IKONOS 1m PAN imagery

41

Fig. 4.2

Training site

42

Fig. 4.3

The Daubechies family

43

Fig. 4.4.a

Approximation Xa1

44

Fig. 4.4.b

Horizontal Detail Xh

44

Fig. 4.4.c

Vertical Detail Xv1

45

Fig. 4.4.d

Diagonal Detail Xd1

45

Fig. 4.5.a

Level 1 image

46

Fig. 4.5.b

Edges detected at level 1

46

Fig. 4.6.a

Level 2 image

46

Fig. 4.6.b

Edges detected at level 2

46

Fig. 4.7.a

Level 3 image

47

Fig. 4.7.b

Edges detected at level 3

47

Fig. 4.8

Decomposed 2nd level Wavelet image

49

Fig. 4.9

Edge map at 0.25 threshold

49

vi


Fig. 4.10

Edge map from Canny edge algorithm (threshold = 0.2)

50

Fig. 4.11

Edge map from Canny edge algorithm (threshold = 0.01)

50

Fig. 4.12

Edge map from Canny edge algorithm (threshold = 0.07)

51

Fig. 4.13

Threshold plot

51

Fig. 4.14

Edge map (threshold = 0.13)

52

Fig. 4.15

Edge map (standard deviation = 4)

52

Fig. 4.16

Edge map (standard deviation = 2)

53

Fig. 4.17

Edge map (standard deviation = 0.2)

53

Fig. 4.18

Edge map (standard deviation = 0.05)

54

Fig. 4.19

Standard deviation plot

54

Fig. 4.20

Edge map

55

Fig. 4.21

Final edge map

56

Fig. 4.22

Block template matching (accuracy parameter: 92%)

58

Fig. 4.23

Block template matching (accuracy parameter: 75%)

58

Fig. 4.24

Edge template matching (accuracy parameter: 95%)

59

Fig. 4.25

Edge template matching (accuracy parameter: 90%)

60

Fig. 4.26

Edge template matching (accuracy parameter: 80%)

60

Fig. 4.27

Edge template matching (accuracy parameter: 80%)

61

Fig. 5.1

Training Site

62

Fig. 5.2

Schematic map of the training site with house plots

63

Fig. 5.3

Block Template Matching

64

Fig. 5.4

Edge template matching

64

Fig. 5.5

Prewitt Edge Detector

68

Fig. 5.6

Sobel Edge Detector

68
vii


Fig. 5.7

Zero-crossing Edge Detector

69

Fig. 5.8

Canny Edge Detector without wavelet analysis

69

Fig. 5.9

Canny Edge Detector with wavelet analysis

70

Fig. 5.10

Sobel Edge Map

7

Fig. 5.11

Wavelets + Sobel Edge Map

72

Fig. 5.12

Sobel Edge Detector + Template Matching

73

Fig. 5.13

Wavelets + Sobel Edge Detector + Template Matching

73

Fig. 5.14

Canny Edge Detector + Template Matching

74

Fig. 5.15

Experiment Site I

75

Fig. 5.16

Edge Map of Experiment Site I

75

Fig. 5.17

Final Map of Experiment Site I

75

Fig. 5.18

Experiment Site II

76

Fig. 5.19

Schematic map of Experiment Site II

77

Fig. 5.20

Edge Map of Experiment Site II

78

Fig. 5.21

Final Map of Experiment Site II

79

viii

LIST OF TABLES
Table 5.1

Accuracy assessment of buildings

65

Table 5.2

Comparison between Canny & Sobel edge detectors

71

ix


LIST OF ACRONYMS

DN

Digital Number

DWT

Discrete Wavelet Transform

EMR

Electro Magnetic Radiation

FFT

Fast Fourier Transform

GIS

Geographic Information Systems

MRA

Multi Resolution Analysis

x


Chapter 1 Introduction

CHAPTER 1. INTRODUCTION

1.1 Background

Earth scientists actively collect resource data to test hypotheses and simulate or
model the environment. The data is collected using either in situ or remote sensing
methods. Measurements from the sensors using either method provide much of the data
for physical geography and other earth science research.

Remote Sensing is defined as the technique of obtaining information about objects
through the analysis of data collected by special instruments that are not in physical

contact with the objects of investigation. This relatively new science is gaining popularity
in a number of applications, starting from geology and soil mapping to archaeology, from
weather forecasting to mineral extraction. Other application areas include, land use/land
cover analysis, agriculture, forestry, rangeland, water resource, urban and regional
planning, wetland, wildlife ecology and environmental assessment, to name a few.

Another area of research, and one that is relevant to this thesis is urban monitoring
and assessment. Urban and regional planners require continuous acquisition of data to
formulate governmental policies and programs. These policies and programs might arise
from domains which range from social, economic, and cultural to environmental and
natural resource planning. The role of urban and government planning agencies is

1


Chapter 1 Introduction

becoming increasingly more complex and, as such, they are increasingly acting to a wider
range of timely, accurate, and cost effective sources of data of various forms in order to
plan/design the urban and semi-urban fringe to curb/expand the related activities. The
challenge today in order to detect the change in an urban environment by detecting manmade features from imagery

Modern remote sensing satellites are equipped with high-resolution sensors and
electronic cameras that enable ground stations to receive highly detailed images in digital
format and instantaneously relay them to designated locations continuously. Moreover,
being in digital format, the information can be easily stored, retrieved and made available
for post processing analysis. The automation of photographic analysis is one of the
research topics in remote sensing today.

In the past, civilian satellite sensors had not been able to provide data of

sufficiently high spatial resolution (i.e., large mapping scale) to identify the features of
interest in urban areas, such as individual buildings, roads and areas of open space. The
situation has changed with the advent of a number of commercial satellite sensors that
provide image data with a spatial resolution as fine as 1m (i.e., roughly speaking, they are
able to detect objects approximately 1m by 1m in size).

In the field of remote sensing, the rapid development of technology and the
opening of the image market have made high spatial resolution data available. The new
high spatial resolution satellite imagery from IKONOS with a pixel size of 1 meter (PAN

2


Chapter 1 Introduction

band) offers a new quality of detailed information about the earth’s surface, and objects
such as house plots, streets, and trees. Therefore, the extraction of these objects is of great
interest. Man-made feature extraction is typically based on high-resolution aerial imagery
and has been widely studied in photogrammetric, as well as the computer vision
community, for many years. Different techniques and methods combine geometric and
chromatic attributes as well as stereo information about contours and their flanking
regions from multi-view imagery.

The problem is that while the sensor technology available for civilian use has
effectively `advanced' overnight, this has not been mirrored by an equivalent improvement
in the techniques used to process the resultant data. A multitude of research effort is now
concentrated on improving segmentation, more specific aspects of it, like edge detection.
It is well known that much information regarding image structure, even in gray-scale
images, is provided by means of extracting edge information. Recent developments in
multi-resolution analysis such as wavelet transform help to overcome this difficulty of

scaling by analyzing data at various levels/resolutions. Shape classification helps to
group/cluster the extracted objects.

1.2 Scope

Singapore is a highly urbanized city with high-rise public housing estates,
commercial buildings and low-rise estates. Less attention has been on low-rise units in the
semi- urban fringe. The study is to investigate the feasibility of auto-detection of manmade features like buildings in the local context using recently available high resolution

3


Chapter 1 Introduction

IKONOS imagery.

Satellite imagery and aerial photographs have been used for a long time for the
extraction of urban features. The classical classification using spectral information at
present is supplemented by more recent spatial analysis techniques. This helps to take into
account the geometric and contrast information as opposed to the purely spectral
classification of urban features. The main consideration is the resolution with which the
urban features are classified. As the resolution increases, the spectral homogeneity of the
pixels decreases and so normal spectral classification less accurate. The alternative is to
use the edge classification scheme. Edges of an image mostly reflect the information of
the image and contain the basic character of the image. Traditional edge detection
methods are divided into two categories: time field and frequency field. In time field
methods are detectors like the Sobel edge detector, Prewitt edge detector, Canny edge
detector, etc. In the frequency field are various kinds of bandwidth-based filters. Methods
that only enhance edges like the Sobel edge detector cannot be applied in the cases
somewhere parts of the image are homogenous and some parts are sharp variations. In the

frequency field, the normal Fourier transform has no resolution in the time field. Any
modification of any frequency will influence the whole time zone leading to poor results.

The wavelet transform has made much impact as a new method of information
processing. Wavelet transformation is sometimes known popularly as the mathematical
microscope as it has resolution both in the time and frequency domains, and is localized. It
is therefore able to focus onto any detail of the analyzed object by taking finer steps in the
time field or space field. This technique can give good results with respect to edge

4


Chapter 1 Introduction

detection.

Edges obtained by any of the edge detection methods above are grouped to
reconstruct meaningful object shapes by matching to pre-defined object models. Shape
classification plays a major part in the image segmentation approach. Like pixel-based
supervised classification techniques, this classification method consists of two parts – the
training stage with representative data (the object models and their shape features) and the
classification stage with classifier model.

The above-explained techniques were grouped together to achieve optimum
output, rather than the application of any one particular technique. The proposed technique
described contributes to the eventual goal of using image-processing techniques to
generate maps of urban land use directly from digital remotely sensed images with the
minimum of manual intervention. The grouping and its advantages are clearly explained
in Chapter 2.

1.3 Objective

The underlying goal of the research is to develop a feasible, accurate and rapid
method of determining building outlines that combines remote sensing interpretation with
image processing techniques. An important technical objective is the application of
digital image processing technology such as Canny edge detection technique as the best
type for information extraction due to the double threshold and good localization

5


Chapter 1 Introduction

properties; multi-resolution wavelet analysis for decomposition of the image to greater
levels of details; and template matching for shape classification The method developed
will be a step towards the goal of semi-automating the procedure to maintain the data
derived from remote sensing methods.

1.4 Organization of the thesis

The organization of the thesis is as follows. The literature review chapter surveys
previous similar attempts to develop applications similar and the techniques used in this
study.

Urban remote sensing techniques are discussed in detail in the third chapter. This
chapter provides the reader with essential background knowledge in satellite remote
sensing and digital image processing. The chapter also provides details of the wavelet
theory, edge detection and shape classification and describes the proposed technique.

Details of the implementation of the proposed algorithm are discussed in the fourth

chapter, which also provides details of the parametric study to determine good operating
values for some of the parameters in the algorithm.

The fifth chapter presents the results of the feature extraction and assess the
accuracy of the study, together with the advantages and shortcomings of the method
proposed in the study.

The final chapter presents the conclusions and suggests areas for future
improvement.

6


Chapter 2 Literature Review

CHAPTER 2. LITERATURE REVIEW

2.1 Introduction

Utilization of remote sensing data for urban studies is crucial in certain aspects
like urban monitoring, land use feature extraction, etc. to name a few. Due to the recent
surge in the acquisition of satellite data, there have been many attempts in the study of
these in the urban arena. IKONOS, which is the first commercial 1m-resolution satellite,
has been successfully launched in 1999 and, many 1m-resolution satellites are being
launched within few years. The availability of such high-resolution satellite images may
change the mapping, photogrammetry and remote sensing world. Because the
shortcomings of airborne photogrammetry, such as small coverage, difficulties in
periodical acquisition may be overcome and many objects that were not identifiable in
low-resolution (10-30m) may be detected in high-resolution images. One of the major
objects that can be extracted from the images are buildings. Extracting buildings is so far

done using high-resolution airborne images or low-resolution satellites images. Hence,
algorithms for man made feature extractions from 1m resolution images can hardly be
found in literatures. As mentioned in the previous chapter, image processing techniques
have been used in this study to extract building features. There are many algorithms and
techniques available in the market of digital image processing; choosing the ones

7


Chapter 2 Literature Review

required for this study is the challenging task. It is very important to review the previous
studies and the nature of various algorithms before starting the actual study.

2.2 Related Research
2.2.1 Application related studies – satellite remote sensing of urban areas

Remote sensing of urban areas reflects the power of using multiple sources of
satellite imagery with geographic information and the expertise of urban areas. With the
rapid growth in population throughout the world, it is crucial to have a well-concerted
plan for urban expansion.

Satellite images give urban planners synoptic views of large areas, which allow
them to lay plans for urban expansion effectively [6]. Li Yingcheng et al [37] in their
paper described satellite remote sensing as a means of monitoring with high speed,
accurate and efficient. It not only gives the amount and location of the change
information of land use, but can also be used to check the data supplied by local
government. It gives the administrators with scientific assistance in the macro managing,
planning and utilizing the land resource. It is also an important means in constructing the
system of national land use dynamic monitoring. Jothimani [23] describes this subject of

urban sprawl mapping and monitoring as one of the operational applications of satellite
remote sensing data, irrespective of its spatial and spectral resolution of the satelliteborne sensors. From the earliest data (Landsat-MSS) with comparatively coarse

8


Chapter 2 Literature Review

resolution to the present high spatial resolution data (IKONOS - panchromatic), detecting
the changes in land cover and its use, especially the delineation of built up environment
has been proved efficient. The visual interpretation technique which has an edge over the
digital analysis and interpretation of the built-environment has been advocated in
conjunction with the topographical maps to operationalize the techniques. Multi-temporal
and repetitive satellite data offer unique opportunities for mapping and monitoring some
of the elements of urban core, its dynamics and the resultant urban structure. The
complexities and elements of urban dynamics as well as the required satellite data
characteristics are controlling factors in urban inventory and analysis.

2.2.2 Wavelet Approaches to Image Processing

A fundamental property of wavelets is their ability to describe detail both in time
and in frequency [26] proposed a wavelet approach due to its scale aspect. They
discussed about texture and its properties and such that texture analysis methods extract
the characteristics that are believed to be most important in particular texture
characterization problems. The normal analysis methods highlight different aspects of
texture, which is a too general and vague concept to encompass in a single description.
But the shortcoming is that, the wavelet approach has not been utilized to the highest
extent in this paper and only applied for frequency splitting.

9

Chapter 2 Literature Review

Fatemi-Ghomi et al [10] discussed at length about the wavelet based texture
segmentation. They reported that a good textured image segmentation scheme should
consist of a method for estimating texture features taking the non-stationary nature of the
feature image planes into account. Essentially, they have described textures in the image
by a series of features derived from the wavelet transform at different levels of transform.
This leads to a classical problem in pattern recognition – that of choosing appropriate
features to use when performing a clustering analysis. Therefore, there is a need to be
able to not only quantify the performance of the final segmentation of the images by the
various methods, but also examine the choice of features that have been used. Friha et al
[28] experimented with the same and have concluded that multi-resolution decomposition
can be exploited to detect edges within images and make a clear distinction between
useful information data and noise. Kun Wardana Abyoto [16] added to the above reports
of wavelets as a tool for scale characterization and included that the general weakness of
all the texture analysis procedures is that they primarily focus on the coupling between
image pixels on a single scale [5] [30] [31] [2].

Several important problems in the areas of image processing can be seen as
learning problems [24], where a set of data is to be split into a set if classes as in edge
detection. Since image regions of different frequency content have different perceptual
relevance, it is often the case that the task of learning can be handled best in a multiresolution framework. For some of these problems, a scale-dependent learning algorithm
is combined with a multi-resolution image decomposition to achieve scale independence.
This study shows the application of wavelet based neural network on edge detection.

10

Chapter 2 Literature Review

Designing neural network architecture is a very complicated task especially for problems
like edge detection where the number of layers and units are to be specified. This study is
feasible in applications, which require less number of edges to be detected, but in an
environment like a remote sensing, this study is difficult.

Laure et al [3] reported the approach of wavelet transform in image fusion. When
images are merged in wavelet space, the data can be processed at different frequency
ranges. A number of applications have been listed in this paper but not much in detail has
been discussed. The paper shows that wavelets can be used for image fusion applications
for applications such as image misregistration and processing artifacts.

Karras et al [15] reported a method for improving the recognition for abrupt
image changes without increasing the noise levels too much. This was achieved by a
combination of local and global methods. These considerations made them employ the
wavelet transform, aiming at extracting informative features for quite different and much
more demanding task of detecting the critical edges for image structure identification.
The Sobel edge detector has been used in this study. Through this approach the main
objective was to reduce the noise level but at the cost of image structure information.

2.2.3 Current Edge Detection Methods

Marr and Hildreth [4] introduced the theory of edge detection and

11


Chapter 2 Literature Review

described a method for determining the edges using the zero crossings of the Laplacian of
Guassian of an image. Haralick [25] determined edges by fitting polynomial functions to
local image intensities and finding the zero-crossings of the second directional derivative
of the functions. Canny [12] determined edges by an optimization process and proposed
an approximation to the optimal detector as the maxima of gradient magnitude of a
Gaussian smoothed image. Clark [14] and Ulupinar and Medioni [9] independently found
a method to filter out false edges obtained by the Laplacian of Gaussian operator.
Bergholm [8] introduced the concept of edge focusing and tracked edges from coarse to
fine, to mask weak and noisy edges. A curve-fitting approach to edge detection was
proposed by Goshtasby and Shyu [1] in which edge contours were represented by
parameters curves that fitted to high-gradient image pixels with weights proportional to
the gradient magnitudes of the pixels. Recent advances in edge detection include a
method by Elder and Zucker [13] to determine edges at multitudes of scales, and an
adaptive smoothing method by Li [29] to remove noisy details in an image without
blurring the edges. Many other edge detection techniques have been developed in the
recent past. Among the edge detection methods proposed and developed so far, the
Canny edge detector is the most rigorously defined operator and widely used [18]. The
popularity of the Canny edge detector can be attributed to its optimality according to the
three criteria of good detection, good localisation and single response to an edge.

Luiz Alberto Lisoba da Silva Cardoso [19] experimented with the Canny
algorithm for edge detection in his study of computer aided recognition of man-made
structures in aerial photographs. This was chosen after excelling in tests he did against

12


Chapter 2 Literature Review

multi-level thresholding edge detectors. According to him, it meets the optimality criteria

concerning non-spurious edge detection, accuracy on the edge location and avoidance of
double edge detection. Neil Rowe et al [22] used the Canny edge detector for the
automatic change detection of linear features in aerial photographs. The reason for
choosing the particular detector is that it produced longer connected segments than the
other algorithms which help in matching features. The edge cells were found using two
thresholds.

The next step in the field of image vision was the wavelet based edge detection.
As single resolution edge detection did not result in much valuable information, in late
1990s, the research in this field became popular. Several images of the same area from
multi-sensor are fused together to maintain most of details, restrain noise of the images
and enhance the rate of detection. Edges are extracted from the fused result image. This
method of image fusion and edge detection based on wavelet transform is discussed in
detail by Wu Xiuqing [35]. The result of the experiment showed that, it not only can
suppress the noise effectively but also was adapted to detecting distinct edges and weak
edges.

Most edge detection methods operate on an image at a single resolution and
output a binary edge map. Edges within an image, however, generally occur ay various
resolutions, or scales, and represent transitions of different degrees, or gradient levels.
Therefore, single resolution edge detection methods do not always yield satisfactory
results. The study by Junaid I. Siddique et al [33] developed a multi-resolution method

13


Chapter 2 Literature Review

that utilized a multi-rate wavelet decomposition to generate a series of images with
progressively lower edge resolution. The series of edge maps is restricted to form a

stacking edge map pyramid. This approach is shown to have advantages over the normal
edge detection methods.

An image pyramid with multi-resolution is needed for coarse-to-fine image
matching such as pattern recognition and template matching. A modern GIS shows a
multi-scaling function and an image analysis system should effectively analyze the multiresolution information of images. According to Li Deren et al [17], the wavelet theory is
that which concentrates on the processing of multi-resolution of images.

2.2.4 Review of approaches in template matching

Before the advent of panchromatic imageries and photographs, only the multispectral information was used. The early object matching methods include the simple
classification algorithms. The process of multi-spectral classification may be performed
using either of two methods: supervised or unsupervised. Sameer Singh et al [34] have
used the supervised classification techniques such as linear discriminant analysis, mean
classifier and K-nearest neighbor methods to identify natural scenes in a FLIR (Forward
Looking Infra-Red) images. Regarding the latter technique of classification, Deba Prasad
Mandal et al [20] proposed a method on the theory of fuzzy sets which provides suitable
tools in analyzing complex systems and decision processes where pattern indeterminacy

14