Cognitive learning and memory systems using spiking neural networks

COGNITIVE LEARNING AND MEMORY
SYSTEMS USING SPIKING NEURAL
NETWORKS
HU JUN
NATIONAL UNIVERSITY OF SINGAPORE
2014
COGNITIVE LEARNING AND MEMORY
SYSTEMS USING SPIKING NEURAL
NETWORKS
HU JUN
B. Eng., Nanjing University of Aeronautics and Astronautics
A THESIS SUBMITTED
FOR THE DEGREE OF DOCTOR OF PHILOSOPHY
DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING
NATIONAL UNIVERSITY OF SINGAPORE
2014
Acknowledgments
Acknowledgments
This work was done in the computational intelligence group led by Dr. Tan
Kay Chen at the Department of Electrical and Computer Engineering, National
University of Singapore and financially supported by Agency for Science, Tech-
nology and Research (A*STAR) and National University of Singapore.
First of all, I would like to express deepest appreciation to my supervisor Dr.
Tan Kay Chen for introducing me into the splendid research field of computa-
tional intelligence. His valuable guidance and support help me to accomplish my
research.
I wish to thank Dr. Tang Huajin for his patient and consistent technical
advisory and encouraging support. His enthusiasm for studying and dedication
to research have inspired me throughout my Ph.D. course.
I would like to express gratitude to Dr. Tan Chin Hiong, Dr. Yu Jiali, Dr.

Huang Weiwei, and Dr. Cheu Eng Yeow in Institute for Infocomm Research,
A*STAR, with whom I worked together and from whom I learned how to work
professionally as a researcher.
My thanks also go to my colleagues of our Computational Intelligence Re-
search Group. Dr. Shim Vui Ann for being my senior who kindly shared his
research experience and encouraged me from time to time, Yu Qiang for accom-
panying me during the last three and half years, Gee Sen Bong for sharing his
excellent coding skills, Willson Amalraj A. for demonstrating how to convert
research achievements into applications, Arrchana Muruganantham for teaching
me website design, Lim Pin for sharing his work experience, Qiu Xin for keeping
i
Acknowledgments
our lab full of joy, Zhang Chong and Goh Sim Kuan for being the replacements. I
also want to thank lab officers of Control & Simulation Lab, Mr. Zhang Hengwei
and Ms. Sara K. for their continuous assistance.
I would like to thank A/Prof. Dipti Srinivasan and A/Prof. Xiang Cheng
at National University of Singapore, who provide me suggestive critiques and
encouraging support.
Last but not least, I would like to dedicate this thesis to my parents for their
constant support and unconditional love.
ii
Contents
Acknowledgments i
Contents iii
Summary vii
List of Tables x
List of Figures xi
Nomenclature xiv
1 Introduction 1
1.1 Background and Basic Concepts . . . . . . . . . . . . . . . . . . 2

1.1.1 Cognitive Learning and Memory in the Brain . . . . . . . 2
1.1.2 Artificial Neural Networks . . . . . . . . . . . . . . . . . . 5
1.2 Research Scope and Contributions . . . . . . . . . . . . . . . . . 8
1.3 Organization of the Thesis . . . . . . . . . . . . . . . . . . . . . . 9
2 Literature Review 12
2.1 Spiking Neuron Models . . . . . . . . . . . . . . . . . . . . . . . 12
iii
Contents
2.2 Spiking Neural Networks . . . . . . . . . . . . . . . . . . . . . . . 15
2.2.1 Neural Coding in Spiking Neural Networks . . . . . . . . 16
2.2.2 Learning in Spiking Neural Networks . . . . . . . . . . . . 20
2.2.3 Memory Models Using Spiking Neural Networks . . . . . 27
3 A Spike-Timing Based Integrated Model for Pattern Recogni-
tion 30
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.2 The Integrated Model . . . . . . . . . . . . . . . . . . . . . . . . 35
3.2.1 Neuron Model and General Structure . . . . . . . . . . . 35
3.2.2 Latency-Phase Encoding . . . . . . . . . . . . . . . . . . . 35
3.2.3 Supervised Spike-Timing Based Learning . . . . . . . . . 39
3.3 Numerical Simulations . . . . . . . . . . . . . . . . . . . . . . . . 42
3.3.1 Network Architecture and Encoding of Grayscale Images 42
3.3.2 Learning Performance . . . . . . . . . . . . . . . . . . . . 44
3.3.3 Generalization Capability . . . . . . . . . . . . . . . . . . 45
3.3.4 Parameters Evaluation . . . . . . . . . . . . . . . . . . . . 48
3.3.5 Capacity of the Integrated System . . . . . . . . . . . . . 52
3.4 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4 A Computationally Efficient Associative Memory Model of Hip-
pocampus CA3 by Spiking Neurons 59
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

4.2 CA3 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
iv
Contents
4.2.1 Spike Response Neurons . . . . . . . . . . . . . . . . . . . 64
4.2.2 SRM Based Pyramidal Cells and Interneurons . . . . . . 65
4.3 Synaptic Modification . . . . . . . . . . . . . . . . . . . . . . . . 66
4.4 Experimental Results and Discussions . . . . . . . . . . . . . . . 69
4.4.1 Associative Memory Storage and Recall . . . . . . . . . . 69
4.4.2 Computational Efficiency . . . . . . . . . . . . . . . . . . 75
4.5 Discussion and Conclusion . . . . . . . . . . . . . . . . . . . . . . 79
5 A Hierarchical Organized Memory Model with Temporal Pop-
ulation Codes 81
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.2 The Hierarchical Organized Memory Model . . . . . . . . . . . . 85
5.2.1 Pyramidal Cells and Theta/Gamma Oscillations . . . . . 86
5.2.2 Temporal Population Coding . . . . . . . . . . . . . . . . 87
5.2.3 The Spike-timing Based Learning and NMDA Channels . 90
5.3 Numerical simulation . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.3.1 Network Behavior . . . . . . . . . . . . . . . . . . . . . . 94
5.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
5.4.1 Information Flow and Emergence of Neural Cliques . . . 107
5.4.2 Storage, Recall and Organization of Memory . . . . . . . 108
5.4.3 Temporal Compression and Information Binding . . . . . 109
5.4.4 Related Works . . . . . . . . . . . . . . . . . . . . . . . . 110
5.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
6 Hierarchical Organized Memory Model with Spike-driven Learn-
v
Contents
ing of Visual Features 115
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

6.2 The Hierarchical Organized Memory Model . . . . . . . . . . . . 118
6.2.1 Network Architecture . . . . . . . . . . . . . . . . . . . . 118
6.2.2 Temporal Population Coding in Encoding Layer . . . . . 119
6.2.3 Spike-timing Based Learning . . . . . . . . . . . . . . . . 120
6.3 Numerical Simulation . . . . . . . . . . . . . . . . . . . . . . . . 124
6.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
6.5 Conclusion and Future Works . . . . . . . . . . . . . . . . . . . . 127
7 Conclusions and Future Works 128
7.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
7.2 Future Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
Bibliography 133
Appendix: Author’s Publications 147
vi
Summary
Summary
Neural networks have been studied for many years in efforts to mimic many
aspects of biological neural systems. Remarkable progress has been made in
solving problems such as vehicle control navigation, decision making, financial
applications, and data mining using neural networks. However, humans can
thoroughly defeat artificial intelligence with little difficulty when facing with
cognitive tasks such as pattern recognition. Moreover, with the increasing de-
mand of our modern life, cognitive function becomes more and more important
in intelligent systems.
Rate coding is a traditional coding scheme used in neural networks. However,
the b ehavioral response of a neuron may be too fast that makes it is impossible
to describe its activity relying on the firing rate. With the development of
instruments and experimental techniques, increasing findings suggest that spike
times make sense in encoding information. The idea that information could be
encoded by precisely timed spikes has drawn increasing attention over the past
20 years. By incorporating the concept of time, spiking neural networks (SNNs)

is compatible with the temporal code rather than the rate code. The goal of
this thesis is to investigate aspects of theories of spiking neural networks in an
attempt to develop cognitive learning and memory models for computational
intelligence.
Firstly, a spike-timing-based integrated model is devised for solving pattern
recognition problem. We attempt to build an integrated model based on SNNs,
which performs sensory neural encoding and supervised learning with precisely
vii
Summary
timed sequences of spikes. Sensory information is encoded by explicit firing times
of action potentials rather than mean firing rates using a latency-phase encoding
method and subsequently transmitted to a consecutive network for learning. The
results show that when a supervised spike-timing-based learning is used, different
spatiotemporal patterns are recognized by different spike patterns with a high
time precision in milliseconds.
Inspired by the hippocampal CA3 region, a computationally efficient auto-
associative memory model using spiking neurons is proposed. The spiking neural
network is able to encode different memory items by different subsets of neurons
with a recurrent network structure. After activation of neurons coding for a
specific memory, they can be kept firing repetitively in the following gamma
cycles by the short term memory (STM) mechanism. By the synaptic modifi-
cations of recurrent collateralsby fast N-methyl-D-aspartate (NMDA) channels
with a deactivation time shorter than a gamma subcycle, accurate formation of
auto-associative memory can be achieved.
The recent identification of network-level memory coding units in hippocam-
pus suggests that memory could be represented by population of neurons and
organized in a categorical and hierarchical manner. A hierarchically organized
memory (HOM) model using spiking neurons is proposed, in which the temporal
population code is considered as the neural representation of information and
spike-timing-based learning methods are employed to train the network. We

demonstrate that the proposed computational model can store patterns in forms
of both associative memory and episodic memory using temporal population
codes.
viii
Summary
Finally, a hierarchical structured feed-forward spiking neural network is pro-
posed to develop invariant response patterns through spike-timing based learning
rules. Internal representations of external stimuli in the brain are achieved by
generating selectivities of neural assemblies. A spike-timing-based learning al-
gorithm is employed to develop hetero-association b etween afferent spikes with
postsynaptic spikes, while STDP learning abstracts knowledge from repetitive
patterns. The results demonstrate that neurons are able to generate selectivities
to visual features with different specificity levels along the hierarchy through
spike-timing based neural computation.
ix
List of Tables
4.1 Comparative results of mean simulation times . . . . . . . . . . . 79
x
List of Figures
1.1 Multi-store model . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Diagram of the hippocampal circuit . . . . . . . . . . . . . . . . 4
1.3 Information flow in an artificial neural network . . . . . . . . . . 4
1.4 Diagrams of artificial neuron models . . . . . . . . . . . . . . . . 7
2.1 Comparison of the neuro-computational properties of spiking neu-
ron models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2 Spike trains in response to moving bars in monkey striate cortex. 16
2.3 Spike-timing-dependent plasticity . . . . . . . . . . . . . . . . . . 23
3.1 General structure and information process of the integrated model 36
3.2 Flowchart of the latency-phase encoding scheme . . . . . . . . . . 39
3.3 Learning rule of ReSuMe . . . . . . . . . . . . . . . . . . . . . . 41

3.4 The latency-phase encoding . . . . . . . . . . . . . . . . . . . . . 44
3.5 Illustration of the learning process and performance . . . . . . . 46
3.6 Testing results with different type of noises . . . . . . . . . . . . 47
3.7 Sample images from the LabelMe database . . . . . . . . . . . . 49
3.8 Encoding error with different encoding cycles and phase shift con-
stants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
xi
List of Figures
3.9 The encoded patterns with a different phase shift constant . . . . 51
3.10 Influence of the complexity of target patters. . . . . . . . . . . . 52
3.11 Memory capacity of the integrated model . . . . . . . . . . . . . 53
4.1 Block diagram of hippocampus . . . . . . . . . . . . . . . . . . . 60
4.2 Network architecture of CA3 . . . . . . . . . . . . . . . . . . . . 61
4.3 Kernel functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.4 Synaptic weight learning window . . . . . . . . . . . . . . . . . . 68
4.5 Mechanism of short term memory (STM) in the model . . . . . . 71
4.6 Illustration of limited capacity of short-term network . . . . . . . 73
4.7 Synaptic weights between pyramidal cells represented in a weight
matrix. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.8 Boxplot of log for varying network size . . . . . . . . . . . . . . . 78
4.9 Plot of log against simulated time . . . . . . . . . . . . . . . . . . 79
5.1 Network architecture of the hierarchical model . . . . . . . . . . 85
5.2 Encoding of a receptive field . . . . . . . . . . . . . . . . . . . . . 89
5.3 Illustration of determining the firing time of a input neuron coding
for a RF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.4 Illustration of the tempotron learning rule . . . . . . . . . . . . . 91
5.5 LTP induced by STDP learning for different memory items . . . 93
5.6 An example of encoding visual stimulation into spatiotemporal
patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
5.7 Neural activity propagates through the system . . . . . . . . . . 96

5.8 Input images and their corresponding neural responses . . . . . . 97
xii
List of Figures
5.9 Typical neural responses of pyramidal neurons . . . . . . . . . . 98
5.10 Evolution of the neural connectivity of the network . . . . . . . . 99
5.11 Weight matrices of lateral connections . . . . . . . . . . . . . . . 99
5.12 Robustness in presence of random jitter noise of different level . . 103
5.13 Illustration of neural cliques and testing results of associative
memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
5.14 Illustration of generated connectivity in Layer II and neural ac-
tivities during recall . . . . . . . . . . . . . . . . . . . . . . . . . 105
5.15 Recall of neural clique activities induced by accumulated EPSPs 106
6.1 Architecture of the feedforward spiking neural network . . . . . . 119
6.2 Illustration of the tempotron learning rule . . . . . . . . . . . . . 122
6.3 Sample images from the face image dataset . . . . . . . . . . . . 124
6.4 Encoding of a grayscale image . . . . . . . . . . . . . . . . . . . . 124
6.5 Reconstruction for extracted features . . . . . . . . . . . . . . . . 125
xiii
Nomenclature
v
i
(t) membrane potential of neuron i
V
rest
resting membrane potential
V
thr
threshold of the membrane potential
τ
m

membrane time constant
R
m
leak resistance
C
m
membrane capacitance
I
syn
injected current from synaptic inputs
h
ext
injected current from external input
I
n
injected current from background noise
η(t) action potential and after-potential function of SRM neuron
ϵ(t) post-synaptic potential function of SRM neuron
w
i
synaptic weight from the ith input neuron to the post-synaptic neuron
t
(f)
i
arrival time of the fth input spike from ith input neuron
ˆ
t firing time of the last spike of a spike train
r
i
(t) firing rate of neuron i

T length of the encoding time window
n
sp
spike count
S
i
(t) spike train of the i th input
xiv
Nomenclature
S
d
(t) desired output spike train
S
o
(t) actual output spike train
δ(t) PSP function of an action potential
w
ji
synaptic weight from neuron i to neuron j
s time difference between pre- and postsynaptic spikes
s
i
intensity of analog stimulation
ϕ
0
reference initial phase of oscillation
∆ϕ constant phase difference
N
RF
number of receptive fields

N
inp
number of input neurons
N
i
number of hidden neurons of the ith hidden layer
N
out
number of output neurons
C correlation between spike trains
F
i
set of all firing times of neuron i
Γ
i
set of presynaptic neurons which the neuron receives input from
τ
delay
time delay for a spike to travel from the soma to the synapses
V
IPSP
feedback inhibition
η
ADP
response of pyramid neuron after firing
A
ADP
amplitude of after-depolarization potential
i
θ

(t) current of theta oscillation
A
θ
amplitude of theta oscillation
H(t) Heaviside step function
xv
Chapter 1
Introduction
To understand how the human brain works is a fascinating challenge that
requires comprehensive scientific research in collaboration with multidisciplinary
fields such as biology, chemistry, computer science, engineering, mathematics,
psychology, etc. Mimicking brain functions such as perception, learning, memory
and cognition is the major goal of an artificial intelligent system.
Neural networks have contributed to theoretical advances in the scientific
study of nervous system during the last few decades. An artificial neural net-
work refers to a system of interconnected “neurons”, which processes informa-
tion by their activities in response to external inputs. Although the capability
of artificial neural networks depends to a large extent on our current under-
standing of biological neural systems, they provide us powerful techniques in
solving real-world problems such as pattern recognition, time series prediction,
data processing robotics, etc.
Spiking neural networks have attracted increasing interests over the past
twenty years, and a great deal of theoretical and practical achievements have
1
Chapter 1. Introduction
been made. This thesis focuses on advancing existing theories and developing
innovative cognitive learning and memory models using spiking neural networks.
1.1 Background and Basic Concepts
1.1.1 Cognitive Learning and Memory in the Brain
In a biological nervous system, learning and memory are two indispensable

components of all mental processes, especially for cognition functions. Learning
is the acquisition of knowledge or skills through study and experience. And
memory is the process in which information is encoded, stored, and retrieved.
Therefore, an intelligent machine is supposed to be able to obtain knowledge from
external stimulation and store them in the form of memory. When encountering
new problems, it would response relying on the stored knowledge.
From the perspective of psychology, memory can be generally divided into
sensory memory, short-term memory (STM) and long term memory (LTM).
Memory models proposed by psychologists provide abstract descriptions of how
memory works. For example, Atkinson-Shiffrin model simplifies the memory
system as shown in Figure 1.1.
In order to explore the memory function in biological systems, different parts
of biological nervous system have been studied (Hawkins & Blakeslee, 2004; He,
2011). Researches on sensory system, especially vision system, advance our
understanding of sensory encoding. Indicated by the study on patient H.M., the
hippo campus is believed to be the most essential part involved in consolidation
of memory.
The hippocampus is one of the most widely studied regions of the brain.
2
Chapter 1. Introduction
Sensory Memory
(millisecond – 1 second)
Short-term
Memory
(< 1 minute)
Long-term
Memory
(days, months, years)
consolidaƟon
retrieval

aƩenƟon
rehearsal
Figure 1.1: Multi-store model of Memory (Atkinson & Shiffrin, 1968).
It resides within the medial temporal lobe of the brain and is believed to be
responsible for the storage of declarative memories. At the macroscopic level,
highly processed neocortical information from all sensory inputs converges onto
the medial temporal lobe. These processed signals enter the hippocampus via
the entorhinal cortex (EC). Within the hippocampus, there are connections from
the EC to all parts of the hippocampus, including the dentate gyrus (DG), CA3
and CA1 through perforant pathway. Connections from the DG are connected
to CA3 through mossy fibers, from CA3 to CA1 through schaffer collaterals,
and then from CA1 back to EC. In addition, there are also strong recurrent
connections within the DG and CA3 regions. Figure 1.2 depicts the overview of
hippo campus based on its anatomic structure.
A few computational models simulating different regions of the hippocampus
were proposed and studied (Jensen & Lisman, 2005; Kunec, Hasselmo, & Kopell,
3
Chapter 1. Introduction
DG
CA3
CA1
EC
Figure 1.2: Diagram of the hippo campal circuit.
2005; Cutsuridis & Wennekers, 2009). Inspired by the structure of hippocam-
pus, memory function have been demonstrated in these models. However, due
to insufficient knowledge about mechanisms underlying neural computation in
biological systems, limited function of the brain can be artificially reproduced
with current technology. By simulating artificial neuron models, neural networks
are inherently close to the nature of biological nervous systems and possible to
mimic their functions. Figure 1.3 illustrates the information flow in typical ar-

tificial neural networks. Similar to biological nervous systems, encoding and
learning are the most important components of a memory model using neural
networks. Encoding defines the form of neural representation of information,
while learning plays a pivotal role in the development of neural systems and
formation of memory.
Real World
Stimulus
(Continuous Signal)
Neural Spikes
(Discrete Signal)
Sensory
Encoding
Learn/Test
Neural
Patterns
Figure 1.3: Information flow in an artificial neural network.
4
Chapter 1. Introduction
1.1.2 Artificial Neural Networks
The first artificial neuron was proposed by McCulloch & Pitts (1943). A
biological neuron is simplified to a threshold logic unit (TLU), which receives
inputs from other neurons (representing the dendrites) and produces an output
by summing inputs (representing the axon). Since each afferent synaptic connec-
tion has a different efficacy (Figure 1.4), the summation of inputs is weighted.
The state of a neuron is defined to be either “active” or “inactive” depending
on whether the weight sum exceeds the predefined threshold or not.
Shortly after this, the perceptron proposed by Rosenblatt (1958) and mul-
tilayer perceptron (MLP) equipped with backpropagation algorithm (Werbos,
1974) advances neural network research to a new level. Generally, for a typical
artificial neuron, the output value is usually the outcome of a transfer func-

tion f(x) which applies to the weighted sum. The mathematical description is
formulated as follows:
y = f(
∑
i
w
i
x
i
) (1.1)
However, the discovery of the incapability of solving linear inseparable prob-
lem (Minsky & Papert, 1969) stagnated neural network research for over ten
years. In 1990s, support vector machine (SVM) drew more attention due to
its remarkable performance on classification problems, although it is a bit more
far away from biological neural networks. Recent hot spot of deep learning has
renewed interests in neural networks, however, its stochastic nature weakens its
biological plausibility.
Spiking neural networks (SNNs), which fall into the third generation of neural
5
Chapter 1. Introduction
network models, elevate the level of biological realism of artificial neurons by in-
corporating the concept of time into the computational neuron model. Different
from neuron models of the previous two generations, a spiking neuron initializes
an action potential when its membrane potential reaches the threshold, which
integrates postsynaptic potentials it receives.
Schematic diagrams of computational units of three generations are presented
in Figure 1.4. The membrane potential of the neuron is modeled as the weighted
sum of all afferent inputs, and the output is described as the outcome of the
activation function.
The computational units evolves from binary input/output to continuous

input/output and finally becomes spiking input/output. This is mainly caused
by defining the computational units with different types of activation function.
The McCulloch-Pitts neuron uses the Heaviside step function as the activation
function, so that the input and output values are binary (only 0 or 1). An
analogue neuron (second generation) adopts a sigmoid function that enables it
to receive and generate analogue input and output. The spiking neuron models
incorporate a temporal integration module so that the spiking neural networks
are compatible with spike patterns.
Due to their more biological realistic properties, spiking neural networks have
enhanced our understanding of information processing in the biological system
and advanced research of computational neuroscience over the past few decades.
Spiking neural networks have been shown to be more efficient to solve non-linear
classification task such as the classical XOR-problem with less neurons than the
first two generations (Bohte, Kok, & Poutr´e, 2002). Moreover, increasing ex-
6
Chapter 1. Introduction
. . .
∑
Input vector
Output
McCulloch-PiƩs neuron
Perceptron
Step funcƟon
. . .
∑
Input spike train
Output spike train
Spiking neuron
. . .
∑

Input vector
Output
Sigmoidal neuron
Sigmoidal funcƟon
Figure 1.4: Generations of artificial neurons.
perimental findings in neurobiology and research achievements in computational
intelligence using spiking neural networks lead to growing research interests in
designing learning and memory models with spiking neurons.
7