Spatio temporal approaches to denoising and feature extraction in rapid image triage

SPATIO-TEMPORAL APPROACHES TO
DENOISING AND FEATURE
EXTRACTION IN RAPID IMAGE
TRIAGE
YU KE
(B.Eng., Zhejiang University)
A THESIS SUBMITTED
FOR THE DEGREE OF DOCTOR OF PHILOSOPHY
DEPARTMENT OF BIOENGINEERING
NATIONAL UNIVERSITY OF SINGAPORE
2012
i
Acknowledgments
I would like to express my sincere gratefulness to my supervisor Professor Li
Xiaoping, Director of Neuroengineering Laboratories, for his generous sharing,
encouraging attitude, insightful vision and enlightening guidance.
It is my pleasure to enjoy a wonderful 4-year study with so many amazing lab
mates, Dr. Shen Kaiquan, Dr. Ng Wu Chun, Dr. Fan Jie, Dr. Ning Ning,
Dr. Shao Shiyun, Dr. Wu Xiang, Mr. Khoa Wei Long Geoffrey, Mr. Wu Ji,
Mr. Rohit Tyagi, Mr. Bui Ha Duc, Miss Wang Yue, Miss Ye Yan and Mr. Wu
Tiecheng. I benefit a lot from their selfless support and valuable suggestions,
and would like to take this opportunity to show my deep thankfulness.
Special acknowledgments are given to my parents. Their love accompanies me
whenever and wherever I am.
Last but not the least, I am very grateful to the National University of Singapore
for granting me the financial support, with which I can endeavor myself to this
doctoral research.
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
ii
Table of Contents

Acknowledgments i
Summary vii
List of Tables xi
List of Figures xvii
List of Symbols xviii
Acronyms xxi
1 Introduction 1
1.1 A Snapshot of Image Screening Strategies . . . . . . . . . . . . 2
1.1.1 Artificial intelligence based . . . . . . . . . . . . . . . 2
1.1.2 Human intelligence oriented . . . . . . . . . . . . . . . 4
1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
TABLE OF CONTENTS iii
2 Literature Review 10
2.1 EEG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.1.1 Physiological background . . . . . . . . . . . . . . . . 11
2.1.2 Technical background . . . . . . . . . . . . . . . . . . 12
2.1.3 Event-related potentials . . . . . . . . . . . . . . . . . 17
2.2 Brain Computer Interface . . . . . . . . . . . . . . . . . . . . . 23
2.2.1 Invasive BCI . . . . . . . . . . . . . . . . . . . . . . . 23
2.2.2 Noninvasive BCI . . . . . . . . . . . . . . . . . . . . . 25
2.3 EEG Signal Processing Methods . . . . . . . . . . . . . . . . . 29
2.3.1 Signal modeling . . . . . . . . . . . . . . . . . . . . . 30
2.3.2 Denoising . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.3.3 Feature extraction . . . . . . . . . . . . . . . . . . . . . 36
2.3.4 Classification . . . . . . . . . . . . . . . . . . . . . . . 39
2.4 Rapid Image Triage . . . . . . . . . . . . . . . . . . . . . . . . 41
2.4.1 Rationale of RIT . . . . . . . . . . . . . . . . . . . . . 41
2.4.2 Past work on RIT . . . . . . . . . . . . . . . . . . . . . 43

2.5 Mathematical Supplement . . . . . . . . . . . . . . . . . . . . 50
2.5.1 Common spatial pattern . . . . . . . . . . . . . . . . . 50
2.5.2 Weighted support vector machine . . . . . . . . . . . . 52
3 Rapid Image Triage System 59
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
TABLE OF CONTENTS iv
3.1 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.2 RIT for Online Broad-area Imagery Screening . . . . . . . . . . 61
3.2.1 Image preparation . . . . . . . . . . . . . . . . . . . . 61
3.2.2 Experimental procedure . . . . . . . . . . . . . . . . . 64
3.2.3 Data processing . . . . . . . . . . . . . . . . . . . . . . 67
3.2.4 Broad-area imagery screening . . . . . . . . . . . . . . 67
3.3 Real-life experiments . . . . . . . . . . . . . . . . . . . . . . . 68
3.3.1 Participants . . . . . . . . . . . . . . . . . . . . . . . . 69
3.3.2 Experimental paradigm . . . . . . . . . . . . . . . . . . 69
3.3.3 Data acquisition . . . . . . . . . . . . . . . . . . . . . 70
4 A Spatio-Temporal Filtering Approach to Denoising of Single-Trial
ERP in Rapid Image Triage 72
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
4.2 Proposed Spatio-Temporal Filtering Approach . . . . . . . . . . 76
4.2.1 Background . . . . . . . . . . . . . . . . . . . . . . . . 76
4.2.2 Spatial filtering . . . . . . . . . . . . . . . . . . . . . . 77
4.2.3 Spatio-temporal filtering . . . . . . . . . . . . . . . . . 78
4.2.4 Estimating the ERP latency difference between channels 79
4.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.3.1 Simulation tests . . . . . . . . . . . . . . . . . . . . . . 81
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
TABLE OF CONTENTS v
4.3.2 Real-life RIT experiments . . . . . . . . . . . . . . . . 82
4.4 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . 84

4.4.1 Simulation test I . . . . . . . . . . . . . . . . . . . . . 84
4.4.2 Simulation test II . . . . . . . . . . . . . . . . . . . . . 85
4.4.3 Simulation test III . . . . . . . . . . . . . . . . . . . . 88
4.4.4 Real RIT experiments . . . . . . . . . . . . . . . . . . 90
4.4.5 Future work . . . . . . . . . . . . . . . . . . . . . . . . 95
4.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . 96
5 Common Spatio-Temporal Pattern for Single-Trial Detection of ERP
in Rapid Image Triage 97
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.2 Single-Trial ERP Detection . . . . . . . . . . . . . . . . . . . . 100
5.2.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . 100
5.2.2 Common spatio-temporal pattern method . . . . . . . . 101
5.2.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . 105
5.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
5.3.1 Filters and patterns . . . . . . . . . . . . . . . . . . . . 106
5.3.2 Classification . . . . . . . . . . . . . . . . . . . . . . . 109
5.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
5.4.1 Scenario I . . . . . . . . . . . . . . . . . . . . . . . . . 111
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
TABLE OF CONTENTS vi
5.4.2 Scenario II . . . . . . . . . . . . . . . . . . . . . . . . 115
5.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . 116
6 Bilinear Common Spatial Pattern for Single-Trial Detection of ERP
in Rapid Image Triage 117
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
6.2 Proposed Method . . . . . . . . . . . . . . . . . . . . . . . . . 119
6.2.1 Common spatial pattern . . . . . . . . . . . . . . . . . 119
6.2.2 Bilinear common spatial pattern . . . . . . . . . . . . . 121
6.2.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . 125
6.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

6.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
6.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . 132
7 Conclusions and Recommendations 134
7.0.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . 134
7.0.2 Recommendations . . . . . . . . . . . . . . . . . . . . 137
Author’s Publications 139
Bibliography 141
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
vii
Summary
Nowadays, along with the advances in imaging and storage technology, there
has been an accentuated contradiction between the fast increasing sets of large-
volume imagery and limited number of skilled image analysts. Rapid image
triage (RIT) which leverages split-second human perceptual judgement via the
interpretation of electroencephalogram (EEG) signals, can effectively improve
the efficiency of imagery screening. This thesis is mainly concerned with devel-
oping novel single-trial EEG signal processing methods which are the backbone
of RIT system, to augment visual target object detection. These novel single-
trial methods are characterized by explicitly exploiting the spatio-temporal prop-
agation of event related potentials (ERP) across the scalp, which are particularly
informative for ERP detection. Improvements regarding the RIT protocol are
also taken into account.
The measured scalp EEG signals are always contaminated by physiological arti-
facts and environmental artifacts. These artifacts are of much stronger amplitude
than the EEG signals and thus significantly deteriorate the decoding of infor-
mative cerebral signals. In this work, a non-sophisticated and highly effective
denoising approach is put forward to strengthen the signal-to-noise ratio (SNR).
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
SUMMARY
viii

The approach performs spatial smoothing in a temporally adjusted space, in
which noises become less correlated and can be easily suppressed, while re-
taining inherent signals. The results from simulation experiments and real-life
experiments indicate that the proposed approach is well suited for RIT.
Single-trial feature extraction serves as an important mean of counteracting
“curse of dimensionality” which refers to the situation that a large volume of
data is required to achieve statistical significant result in a high-dimensional
space. By solely preserving underlying meaningful features, the RIT system
is less vulnerable to irrelevant and misleading information. Hence the opti-
mization problem in RIT can be greatly simplified. This work implements two
single-trial feature extraction methods, extending the common spatial pattern
analysis (CSP) to accommodate additional temporal structures. The incorpora-
tion of discriminative temporal information has been proven to be meaningful
and very effective as demonstrated in the comparison with competing methods
on real-life experiments.
The real-life experiments were conducted on the developed near real-time RIT
system, which is primarily designed for online broad-area imagery screening.
The RIT system integrates software platform with hardware devices. It stream-
lines the procedures and is characterized by an image analyst-centric protocol:
1) centering visual objects for convenient observation; 2) maintaining the spatial
information flow of imagery so as to avoid eliciting interfering brain signals.
The present work enriches conventional EEG signal processing toolbox with
several novel single-trial spatio-temporal denoising and feature extraction ap-
proaches, which lend RIT system significant discriminating capability. Further
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
SUMMARY
ix
investigation of the properties and implementation of time-delayed CSP is very
promising because time-delayed CSP will be less vulnerable to the noise. In
addition, substantial field tests are also necessary for a full evaluation of RIT.

NATIONAL UNIVERSITY OF SINGA PORE SING APORE
x
List of Tables
5.1 The performance of the proposed CSTP method, CSP, CTP,
KDJ and CSSP. Channel selection (Krusienski et al., 2007) has
been done prior to KDJ. For CSSP, the time delay
τ
is 15 (60 ms)
and 4 features (first 2 and last 2) are used. The last row presents
the average accuracies and standard deviations (in parentheses)
for all methods. The best performance is highlighted in bold. . . 110
5.2 The performance of CSSP and CSTP
∗
in test session. The time
delay
τ
is 15 (60 ms). The last column presents the average ac-
curacies and standard deviations (in parentheses) for two meth-
ods. The better performance is highlighted in bold. . . . . . . . 115
6.1 The performance of CSP, CSSP, CSTP, CSTP
∗
, CSSSP and BCSP
in test session. Among them, the results of CSP, CSSP, CSTP
and CSTP
∗
have been reported in Chapter 5. For CSSSP, the
length of finite impulse filter is 10 time points and regularization
constraint is set by cross validation. The last column presents
the average accuracies and standard deviations (in parentheses)
for these methods. The best performance is highlighted in bold. . 127

NATIONAL UNIVERSITY OF SINGA PORE SING APORE
xi
List of Figures
2.1 The international 10-20 system of electrode placement (Web-
ster, 1997). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2 The standard 10-10 system of electrode placement (Oostenveld
and Praamstra, 2001). Black circles stand for sites of the origi-
nal 10-20 system and gray circles indicate additional sites intro-
duced in the 10-10 extension. . . . . . . . . . . . . . . . . . . . 15
2.3 The adaptive filter can be used for noise cancellation. . . . . . . 33
2.4 The common concept of BSS methods . . . . . . . . . . . . . . 35
2.5 Rapid image triage system. . . . . . . . . . . . . . . . . . . . . 42
3.1 Schematic of the RIT system . . . . . . . . . . . . . . . . . . . 60
3.2 The software developed for image preparation . . . . . . . . . . 62
3.3 The raster scan order of image preparation. Target is the image
containing point of interest (POI). . . . . . . . . . . . . . . . . 63
3.4 Ordinary image preparation. The rectangular in color stands for
the image boundary. . . . . . . . . . . . . . . . . . . . . . . . . 63
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
LIST OF FIGURES xii
3.5 Image preparation with overlapping. Adjacent images share a
portion of imagery. . . . . . . . . . . . . . . . . . . . . . . . . 64
3.6 Checking impedance by ASA. a) Injecting gel in the cap. b)
Impedance is shown in ASA - the darker the blue color, the
better the impedance. . . . . . . . . . . . . . . . . . . . . . . . 65
3.7 Eye blinking and movement calibration. (a) The image ana-
lyst blinked upon the disappearance of the white cross. (b) The
image analyst made eye movements while the white cross alter-
nated repeatedly from left to right, up to down . . . . . . . . . . 66
3.8 The pseudo-colored layer overlaid on the original broad-area

imagery. The layer was generated by the developed RIT system.
The star-shaped symbol illustrates the actual position of target
objects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.9 The standard RSVP paradigm, fifty images were presented in
fast bursts of 7.5 seconds, with each image lasting for 150 ms . . 70
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
LIST OF FIGURES xiii
4.1 The EEG potential topographies before and after filtering in
Simulation test I. The first row is the benchmark target ERP
which is noise-free. The second row is the noise-contaminated
version of target ERP. The noise artificially added is the spatially
and temporally white noise. The third row and the fourth row
are the outputs of the proposed spatio-temporal filtering and 2D-
G, respectively.
σ
s
was set to 0.04 m. All topographies are un-
der the same scale [-8 8]
µ
V , and interpolated by an EEGLAB
function ‘topoplot’ (Delorme and Makeig, 2004). . . . . . . . . 84
4.2 The EEG potential topographies before and after filtering in
Simulation test II. The first row is the benchmark target ERP
which is noise-free. The second row is the noise-contaminated
version of target ERP. The noise artificially added is spatially
correlated but temporally white noise. The third row and the
fourth row are the outputs of the proposed approach and 2D-G,
respectively.
σ
s

was set to 0.04 m. All topographies are un-
der the same scale [-8 8]
µ
V , and interpolated by an EEGLAB
function ‘topoplot’ (Delorme and Makeig, 2004). . . . . . . . . 86
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
LIST OF FIGURES xiv
4.3 The EEG potential topographies before and after filtering in
Simulation test III. The first row is the benchmark target ERP
which is noise-free. The second row is the noise-contaminated
version of target ERP. The noise added is real EEG noise. The
third row and the fourth row are the outputs of the proposed
spatio-temporal filtering and 2D-G, respectively.
σ
s
was set to
0.04 m. All topographies are under the same scale [-8 8]
µ
V ,
and are interpolated by an EEGLAB function ‘topoplot’ (De-
lorme and Makeig, 2004). . . . . . . . . . . . . . . . . . . . . . 89
4.4 Target ERP in neighbouring channels have stronger correlation
and less difference in latency. (a) depicts the mean and standard
variance of correlation coefficients versus the spatial distance
over 62 channels. (b) shows the relationship among latency, cor-
relation coefficient and spatial distance of channels. The blue
line in (a) and all the color points in (b) are obtained in the cir-
cumstance when the maximal correlation coefficient is achieved
in 0-15 time-point delay for every pair of channels. Correla-
tion coefficients that are less than 0.5 are not plotted in (b), as

no time delay has been applied to them in this work (see Section
4.2.4). Electrode coordinates follow cross-registrations between
spherical and realistic head geometry (Towle et al., 1993). . . . . 92
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
LIST OF FIGURES xv
4.5 The balanced error rates across 20 subjects achieved with the
proposed approach (PA), raw signals without denoising (RS),
2D Gaussian (2D-G), DSS, ICA and SCP respectively. For
2D Gaussian and the proposed approach,
σ
s
was set to 0.04
m. For DSS, the first 10 components were used for signal re-
construction. For ICA, the EEGLAB function ‘RUNICA’ (De-
lorme and Makeig, 2004) was used and independent compo-
nents were selected according to the evoked-to-total power ra-
tio (de Cheveign
´
e and Simon, 2008). For SCP, spatio-temporal
screening template (Miwakeichi et al., 2004) corresponding to
different component numbers, i.e. 3, 5, 10, 15 and 20 were
tested for every subject, and the best results achieved were pre-
sented. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.1 Spatial filters (left column) and their corresponding filtered tem-
poral patterns (middle and right columns for target condition
and non-target condition, respectively). Spatial filters presented
here are the first and last columns of V
V
V . Filtered temporal pat-
terns presented here are obtained by v

v
v
′
X
X
X and ensemble aver-
ages over epochs in the test session. (a1) and (a2) are results
of target condition and non-target condition projected by (a),
respectively. (b1) and (b2) are results of target condition and
non-target condition projected by (b), respectively. . . . . . . . 106
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
LIST OF FIGURES xvi
5.2 Temporal filters (left column) and their corresponding filtered
spatial patterns (middle and right columns for target condition
and non-target condition, respectively). Temporal filters pre-
sented here are the first and last columns of
˜
V
V
V . Filtered spatial
patterns presented here are obtained by X
X
X
˜
v
v
v and ensemble av-
erages over trials in the test session. (a1) and (a2) are the re-
sults of target condition and non-target condition filtered by (a),
respectively. (b1) and (b2) are results of target condition and

non-target condition filtered by (b), respectively. . . . . . . . . . 107
5.3 Common spatial patterns in (a1) and (a2) are corresponding to
spatial filters in Figs. 5.1(a) and 5.1(b), respectively. They are
the columns of (V
V
V
′
)
−1
. Common temporal patterns in (b1) and
(b2) are corresponding to temporal filters in Figs. 5.2(a) and
5.2(b), respectively. They are the columns of (
˜
V
V
V
′
)
−1
. . . . . . . 109
6.1 The top highest power ratio obtained at each iteration step for
each subject. . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
6.2 Common spatial patterns that are rows of W
W
W
−1
are presented at
each iteration step. Common spatial pattern 1 and common spa-
tial pattern 2 correspond to the two most discriminative spatial
filters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

NATIONAL UNIVERSITY OF SINGA PORE SING APORE
LIST OF FIGURES xvii
6.3 Common temporal patterns that are the rows of V
V
V
−1
are pre-
sented at each iteration step. Common temporal patterns 1, 2,
3 and 4 correspond to the four most discriminative temporal fil-
ters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
6.4 The topographies of average target ERP at 472 ms and 360 ms. . 130
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
xviii
List of Symbols
· transpose operator
C
C
Co
o
ov
v
v(·,·) covariance
det(·) determinant
E[·] expectation
||·|| norm operator
ϕ
(·) nonlinear mapping
< ·> ensemble averaging
g(·,·) 1D-Gaussian
K(·,·) kernel mapping

L(·,·,·, ·) Lagrange function
tr ace(·) matrix trace
A
A
A common spatial pattern matrix
˜
A
A
A common temporal pattern matrix
a
a
a Lagrange multipliers
B
B
B whitening matrix
b bias
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
LIST OF SYMBOLS
xix
C regulation parameter
d
i
geometric margin of the i
th
sample
I
I
I identity matrix
n number of epochs
P

P
P rotation matrix
p
p
p
i
spatial position of the i
th
position
r(t) reference signal
T
T
T (i, j) estimated ERP latency difference between i
th
channel and j
th
channel
V
V
V bilinear temporal filter matrix or spatial filter matrix
V
V
V
i
bilinear temporal filter matrix at the i
th
iteration
v
v
v bilinear temporal filter or spatial filter

W
W
W bilinear spatial filter matrix
W
W
W
i
bilinear spatial filter matrix at the i
th
iteration
w
w
w bilinear spatial filter or filter coefficient vector
X
X
X EEG signal epoch
X
X
X
c
EEG epoch of class c
X
X
X
τ
EEG signal epoch with time delay
τ

X
X

X CSSP data matrix
x
x
x feature vector
¯x(i, j) EEG amplitude at channel i and time j
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
LIST OF SYMBOLS
xx
Σ
Σ
Σ composite spatial covariance matrix
Σ
Σ
Σ
a
normalized spatial correlation matrix of class a
˜
Σ
Σ
Σ
a
normalized temporal correlation matrix of class a
Σ
Σ
Σ
c
normalized spatial covariance matrix of class c or spatial
composite matrix
Σ
Σ

Σ
d
spatial discriminative matrix
ξ
slack variable
ρ
i, j
spatial correlation between i
th
channel and j
th
channel
Λ
Λ
Λ diagonal matrix
λ
power ratio
λ
i
product of a number of power ratios at i
th
iteration
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
xxi
Acronyms
AR autoregressive
ARMA autoregressive moving average
BA balanced accuracy
BCI brain-computer interface
BCSP bilinear common spatial pattern

BER balanced error rate
BOLD blood oxygen level dependent
BP Bereitschaftspotential
BSS blind source separation
CSP common spatial pattern
CSSP common spatio-spectral pattern
CSSSP common sparse spectral spatial pattern
CSTP common spatio-temporal pattern
DSS denoising source separation
ECG electrocardiogram
ECoG electrocorticography
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
ACRONYMS
xxii
EEG electroencephalogram
EMG electromyography
EOG electrooculogram
ERP event related potentials
fMRI functional magnetic resonance imaging
HDCA hierarchical discriminant component analysis
ICA independent component analysis
KDJ Krusienski
KNN k-nearest-neighbor
KTT Karush-Kuhn-Tucker
LDA linear discriminant analysis
MEG magnetoencephalography
MI motor imagery
MVAR multivariate autoregressive
PARAFAC parallel factor analysis
PCA principle component analysis

POI point of interest
RIT rapid image triage
RSVP rapid serial visual presentation
SCP shifted CANDECOMP/PARAFAC model
SNR signal-to-noise ratio
SVM support vector machine
WSVM weighted support vector machine
NATIONAL UNIVERSITY OF SINGA PORE SING APORE
ACRONYMS
xxiii
2D-G 2-dimensional Gaussian smoothing
NATIONAL UNIVERSITY OF SINGA PORE SING APORE