proceedings of the iapr international

Proceedings of the IAPR International
Conference on Pattern Recognition and
Information Processing – PRIP’97,
vol. 1, 1997, pp.39-47
Education Aspects:
Handwriting Recognition-
Neural Networks-
Fuzzy Logic
J. Gilewski
1)
, Phil Phillips
2)
,
S.Yanushkevich
1)
, D.Popel
3)
1)
Institute of Computer Science & Information Systems, Technical University, Zovnierska 49,
Szczecin 71-210, Poland, Fax::(+4891)4876439, E-mail: ,

2)
IEE C3 Committee, Netherlaw 1, North Berwick, United Kingdom, Fax: (**44) 017-1250-1290,
E-mail:
3)
Belarussian State University of Infomatics and Radioelectronics, P.Brovky 6, Minsk 220069,
Republic of Belarus, Fax: (+375)2-310914, 495106
.
Abstract
- A laboratory work on courses Pattern Recognition and Image Processing and Artificial
Intelligence is introduced. The originality of authors’ approaches is that (i) one of the approaches to

handwriting recognition (ii) elements of fuzzy logic, and (iii) neural networks, are simultaneously
studied in the work. Other aim of the paper is to review such directions of biometric technologies as
signature and handwriting authorization (identification).
The offered laboratory work has a character of research work and frequently requires non-standard
decision-making of the students. The efficiency of the offered approach for educational process is
estimated by such factors as interest of the students, expansion of their outlook, ability for complex
using their knowledge and more complete preparation for examinations.
The proposed technology is easy to be implemented and allow flexibly reacting onto changing and
adding in training courses
1
.
Index terms
- handwriting recognition, fuzzy logic, feature extraction, neuron network, biometric
technologies, education
1
This work was supported in part by Technical University of Szczecin (Poland) and in part by State
University of Informatics & Radioelectronics (Belarus).
1. Introduction
It is well known that one of the main goal for every university is to use modern scientific
achievements in education process. This way is not simple because new scientific results can mapped
into education process only by using so called methodical pre-processing and approbation.Some
principals of this mapping was formulated by Society for Research in Higher Education (UK) and we
agree with them. So, methodical approach based on next principals: orienting, motivating, presenting,
clarifying, confirming, consolidating (opportunities to develop and test personal understanding), and
elaborating (introducing additional materials to develop more detailed knowledge).
As is the case for most computer Science departments, the Institute of Computer Science &
Information Systems (Technical University of Szczecin) is regularly faced with the need to revise its
courses and curriculum in order to remain current in a rapidly changing field.
This paper is devoted to a problem of handwriting recognition by using non-classical approaches,
and we have followed these principles when developing this laboratory work.

Rosenblatt’s perceptron (1950) was one of the first neural network (NN) and it was able to
recognize a fixed-font character set. Zadeh proposed the fuzzy logic theory [Zadeh], which changed
many our conceptions about methods to solve the problem. It is well known today that NN and fuzzy
logic are complementary tools for solving handwritting recognition problem.
When composing this task for education goal we have aimed to reach: i) simplicity of
implementation, ii) flexibility, i.e. ability to make quickly changing and adding in according with
requirements of the courses, and (iii) integrated using of inter-disciplinary knowledge.
In this paper we present original technology for simultaneously studying next aspects: (i) methods of
handwriting recognition, (ii) elements of fuzzy logic, (iii) NN architecture, and (iv) training methods for
NN.
It is clear that students study these questions, at least, in two university courses: Pattern Recognition
and Image Processing and Artificial Intelligence.
In other words, to fulfil the given work, very good preparation of the students is required. Moreover,
it has character of research work.
Our approach for working out this laboratory work was under influencing other additional factors
too, namely: to realise in the work the possibility to learn such directions of biometric technologies as
signature and handwriting authentication and recognition [Shme], [Zhou]. It is caused by that the
significant attention when training the experts in area of Banking Information Technologies, is paid to
problems of information safety, including biometrics methods [ShYa], [Sold]. It should be indicated,
that biometric methods include voice and speech recognition, dynamic signature and handwriting
capture, eyes (iris and retinal) identification, hand geometry, fingerprint identification, face
recognition and keystroke dynamics [Sold]. Therefore it is necessary to relate also at least two
functions to the listed above ones, namely: (i) studying the methods of signature identification, and( ii)
studying the methods of handwriting verification.
Conspectus of lectures on the specified above courses, textbooks, numerous scientific papers on the
given direction and methodical literature were used when to compose the laboratory task [Amin],
[Bern], [GadMoh], [ShmYan], and others.
Our approach is explained as follows. At the beginning, we present the common structure of the
laboratory work, the main components of which are pre-processing, fuzzy feature extraction, and
feedforward NN. The functions of the components of the handwriting characters processing and

recognition can be changed. It means that is possible to „follow” lectures and take into account
changing and additions without changing of the work structure.
Then we concentrate attention on realising the fuzzy feature extraction algorithm. This algorithm
allow modify, and students have opportunities to explore enough carefully the mathematics for fuzzy
logic and features of the applications for a concrete task.
Further we explain some methodical questions of fulfilling the work by students and share our
experience.
2. The general structure of the system
The main components of the work, or stages to learn the theme, are pre-processing, fuzzy feature
extraction, and feedforward NN (Fig.1). The handwriting model combines both functions: segmentation
and classification of a target handwritten letter. The character feature of such organizing consists of the
fact that functions of the components can be essentially changed.
Let us assume that a binary image of a character be the input
data. Before we start the segmentation process we have to
realise the pre-processing of the image which includes the
following steps (Fig. 3):
- filtering: the aim is to reduce noise and make easier
extracting the structural features,
- thinning (or skeletonizing): this task removes outer pixels
by iterative boundary erosion process until a skeleton of pixel
chains only remains,
- searching vertices: at this step we extract a line junctions
and the ends of the lines called vertices. Every vertex has a
number of branches which meet each other in this point.
After the pre-processing we get the thinned image with a set
of vertices and its number of branches. We can perform a fuzzy
feature extraction. The task of our feature extraction module is
to process a binary image so that to obtain a vector of features.
The main features are: a kind of segment, its orientation and size
related to the character frame and position.

The features coding block includes a complex structure for
input layer of a NN classifier. The input vector contains details
for our character extracted from previous step.
The last step is to classify the character. We apply the
classic three-layer feedforward NN [Fu].
3.
Fuzzy feature extraction algorithm
The pre-processing of the image allows students studying a
quite enough class of algorithms and estimate the influence of the obtained results onto quality of the
recognition. Then the fuzzy feature extraction algorithm is learned in the work.
The process of feature extraction of a character consists of the distinguishing typical elements
(vertices) and branches or segments (Fig. 2).
Fig. 2. Example of a character
Some different kinds of segments can be indicated, but taking into account the requirements we
defined the simple segments shown in Table 1. It is clear that the number of segments in Table 1 can be
extended, what is necessary, in particular, for signature authentication [Shme], [ShmYan]. Every vertex
has its own co-ordinates Vx and Vy and the number of branches Vn which are crossed in point (Vx, Vy).
The exception is for the vertex which lies on the end of the segment where Vn =1(seeFig.2for
vertex 3).
The aim of this algorithm is to extract the defined segments from the character image. Before we
start extraction, the pre-processing of the image should be perform first. As a result we achieve a
thinned image I, a vertices structure V and the number of vertices Vc. The vertices structure V contains:
a) vertex co-ordinates Vx
i
,Vy
i
b) the number of branches Vn
i,
,wherei is an index of vertex, i =0 Vc-1.
Preprocessing

Image filtering
Thining
Search vertices
Fuzzy feature extraction
Features coding
Neural Network
Input Image
Output
Fig. 1. The structure of handwriting
recognition system
Table 1.
Basic segments of a character with typical elements
line right left loop
horizontal
vertical
right sloped
left sloped
3
2
curve branch
stright branch
vertex
The exact algorithm includes the
following steps.
Input data:
a) m x n binary
image matrix I, b) vertices structure
V, c) number of vertices Vc,
Output data:
fuzzy data

structure F of: a) symbols of
classified segments, b) bound boxes
of segments, c) position of
segments, d) co-ordinates of the
segment (start and end points), e)
sizes of segments.
Step 1:
Initialize the following variable: index of vertices i =0,indexofthebranchj =0forthe
vertex V
i
,mxn temporary image matrix I’ with all zero elements, segment points co-ordinates matrix P
of the current segment.
Step 2:
Trace the segment of the branch j, starting from vertex co-ordinates Vx
i
, Vy
i
to the next
found vertex:
(i) Check both the image matrix I and the temporary image matrix I’ for the pixel If the pixel is set
on I’ it indicates that this pixel was previously traced and it
is skipped. The tracing of the segment is stopped when
tracer has no way to go
(ii) All traced points are set on the temporary image
matrix I’ and their co-ordinates are saved to the matrix P.
Save the start and the end point of the segment and the
bound box which contains the path to the Fuzzy Data F.
(iii) Note the number of segment pixels to k .
Step 3:
To qualify the segment as a loop, the segment

must keep the condition d<=(k+d)*
α
/ 360, where d is a
distance from the beginning to the end of the segment, k is
number of the segment points (number of elements in
matrix P),
α
is the angle threshold (maximum width of an
angle to classify a segment as a loop):
(i) If the below condition is checked the segment is
classified as a loop. The size of the segment is calculated
as at the step 6 (Fig. 8).
(ii) Go to step 7.
Step 4:
Calculate coefficients a and b of the line (from
the start point to the end). We call it as a base line of the
segment. Calculate the average deviation D of the distance
for every segment point to the line y = ax+b,
Dd
l
2
l1
k2
=
=
−
∑
,wherek is the number of segment
points, d
l

- distance from
point l to base line
y=ax+b, D is the average
deviation of the distance.
To realise the
classification of the segment
orientation we define the
following membership
functions for fuzzy sets
Horizontal (H), Vertical (V),
Right Slope (R), Left Slope
(L).
Fig. 3. Main steps of pre-processing.
sDX
sDY
start point
bound box
end point
Fig. 4. Properties of the segment
d
α
Fig. 5. Classifying the segment as a loop
d
1
d
2
d
3
d
k-2

d
k-3
d
l
1
2
3
l
k
-3
k
-2
k
-1
0
θ
base line
y=ax+b
Fig. 6. Calculating the average deviation D of the distance
for every segment point for the line y =
ax+b.
Orginal noisy
image
Filtered
image
Thined
image
Image with
found vertices
45 90

1
135 180 225 270 315 360
0
H
R
VLHR
VLH
θ
F
Fig. 7. Membership function for fuzzy
orientation sets H, R, V, L.
Step 5:
If
D <=
β
then the segment represents a line, where
β
is a curve threshold. This membership
function of the fuzzy sets which has a maximum value indicates what orientation has the line segment.
If
D>
β
(there is a curve) go to step 6.
Size of the segment is defined by fuzzy membership function shown below:
1
0
size
small
medium
large

Ms
SM L
F
Ms/2
Fig. 8. Membership function for
fuzzy size sets S, M, L
where, size = k, Ms = n and k -numberofpoints
belongs to the segment, n - height of the character.
Go to
step 7.
Step 6:
Check what kind of curve corresponds to
the segment.
(i) Calculate the number of intersections of the line LL,
LR with the segment
(ii) Calculate coefficients of the two parallel line LL
and RL tothebaselineBL of the segment
(iii) The following curve Horizontal Left Curve (HLC),
Horizontal Right Curve (HRC), Vertical Left Curve
(VLC), Vertical Right Curve (VRC), Right Slope Right
Curve (RR), Right Slope Left Curve (RLC), Left Slope Right Curve (LRC), Left Slope Left Curve (LRC)
are defined by by the conditions:
if b
∈
H and IL>=2 and IR=0 then b
∈
HLC if b
∈
H and IL=0 and IR>=2 then b
∈

HRC
if b
∈
V and IL>=2 and IR=0 then b
∈
VLC if b
∈
V and IL=0 and IR>=2 then b
∈
VRC
if b
∈
R and IL>=2 and IR=0 then b
∈
RLC if b
∈
R and IL=0 and IR>=2 then b
∈
RRC
if b
∈
L and IL>=2 and IR=0 then b
∈
LLC if b
∈
L and IL=0 and IR>=2 then b
∈
LRC
where b-branch, IL - number of the left line intersections with segment, IR -numberoftherightline
intersections with segment, H, V, R, L - kind of base line define by membership functions FH, FV, FL,

FR.
(iv) Calculate the size of the segment. The size of the segment is defined by the same way as at step 5
(see fig. 8) but size = sDX * sDY, Ms = m * n,wheresDX, sDY is the width and the height of the bound
box (see fig. 3.4), m is the width of the character, n is the height of the character.
Step 7:
If j < Vn
i
(there is another branch to check) go to step 2 else if i < Vc (there is another
vertex to start from) increase index of vertices (i = i + 1) and go to step 2. If i >= Vc that means there is
no vertex left and stop the algorithm.
FH(
θ
)=1-min{ min[|
θ
|, |180 -
θ
|, |360 -
θ
|]/45,1}
FV(
θ
)=1-min{ min[|90-
θ
|, |270 -
θ
|]/45, 1}
FR(
θ
)=1-min{ min[|45-
θ

|, |225 -
θ
|]/45, 1}
FL(
θ
)=1-min{ min[|135-
θ
|, |315 -
θ
|]/45, 1}
where
θ
= arctag(a),
F is a value of the above membership functions
0forsize >0,5Ms
FS(size)=
-2/Ms*size +1forsize <= 0,5Ms
FM(size)=-|2/Ms*size -1|+1
0forsize <0,5Ms
FL(size)=
-2/S*size +2 forsize >0,5Ms
BL
LL
R
L
line
intersections
Fig. 9. Specifying the kind of curve by
calculating the intersection with
parallel line to the base line BL

4.
Features coding
This stage supposes preparing the initial data for the NN. Students are requested to learn and
research some methods of feature coding when handwriting character recognizing. The choice of the
feature coding method is essentially influences onto the learn characteristics of the NN and the quality
of recognition. Let us consider briefly one of the methods, that we suggest to students
. First of all we
need to determine the maximal number of components because of fixed number of neurones in input
layer. All features extracted from character are coded into the input vector. In this vector we can
distinguish two parts. The first part include a kind of segment, its orientation and size. The second part
encodes the relationships between the components. For each pair of distinct components the segment
relative position record is defined (with information about what direction has to go from the first to
reach the second and whether the objects touch).
5.
Multilayer feed-forward NN
The classic three-layer
feedforward NN is used in the
laboratory work (Fig. 10). The
activation function of every input unit
is the linear function
f(x) = x,and
activation function of every hidden
and output unit is the sigmoid function
f(f) = 1/(1 + e
-x
). In our case, the
inputs of the neural NN will be
components of the vector which
represent a collection of segments,
together with information about how

the segments are related. To train a
NN classifier, the preparing of a set of
training data and test data is required.
The number of input nodes depends
on the number of a maximal number
of the components. The number of
hidden nodes is a variable that can be
adjusted by user but there is no special
rules to do that. We will propose
several tests to find convenient values
for the number of hidden neurones.
To have the NN to perform the algorithm of the recognition we must adjust weight values in order
to obtain a desired network performance. For this architecture, a backpropagation training algorithm is
convenient. We use standard Kohonen self-organization feature map, which accurately represents the
fuzziness of the character classes. Student can see the weight vectors of an map that trained using 24x18
binary images of characters.
6.
Educational process
The mentioned above system represented as laboratory work requires for students knowledge in
methods of pattern recognition and image processing, fuzzy logic, and NN. When extending the system
and its using in the course on Banking Information technologies, students must know some bases of
handwriting and signature identification.
6.1 Studying of the fuzzy feature extractor
The aim of this subject is to acquaint students with a main steps of this algorithm and to produce a
set of training and test data. To start the laboratory work we need a lot of binary images of every
character that we want to learn the classifier. Student wants to experiment with the feature extractor
should get to know about working rules of this algorithm. By experimenting we can set the following
properties:
Complex structure of the features:
number of neurons are dependent

on number of the fuzzy feature
Hidden layer:
number of
neurons is chosen
by experiments
One neuron for each posible
character
Threshold
Output layer
Input layer
Fig. 10. The structure of the NN
vt - vertex threshold: factor of minimal distance between vertices; minDV = ((m * n)/2 * vt)/100,
where minDV is a minimal distance between vertices and m, n - size of the image. If the found vertex is
closer than
minDV than the vertex is skipped.
st - segment threshold: factor of minimal length of a path to classify as a segment to analyse; minDS
=st*minDV
,whereminDS is a minimal length of a path. If the traced path is shorter than minDS than
the path is not analyse (it’s too short);
α
- angle threshold: minimal width of an angle to classify a segment as a loop (see step 3 of the
algorithm, Fig. 5);
β
- curve threshold: minimal average deviation of distance to classify a segment as a line (see step 4
of the algorithm, Fig. 6);
N - number of components for NN classifier.
6.2
Studying of NN classifier
A component of the presented system can be any software realization of NN (NN Professional or
Studgard NN Simulator etc.). For creating a NN students must specify a number of input nods

(dependant on
N) number of nodes in hidden layer and number of output nodes. The number of output
nodes is the number of recognized character.
The most important step in the recognition system is the learning of the NN. The starting point for
training procedure is data which students produce in the first part. Training data consists of input-output
pairs that have been generated by the fuzzy feature extraction process. In this part students can change
parameters of the created network and depending to the used simulator they can get many kinds of
graph, chart and tables.
6.3
Training tasks
We consider below two examples of training tasks for students. These examples allow estimating
how to do this laboratory work and its complexity.
Task 2 consists in estimating of influence of the number of neurones in hidden layer onto the
training process of the network, and then onto the recognition. A family of simplest graphics is
illustrated in Fig.11. Students are invited to explain these results of the experiments.
Task 1
Subject:
Influence of learning coefficient onto a
learning progress.
•
Prepare a learning data using fuzzy feature
extractor with chosen settings and chosen
learning set of character images
•
Create a three-layer NN with proper number
of input and output node (depending on how
many feature components are used and how
many kinds of character images we recognize)
•
For each chosen learning coefficient NN pass

a learning task, note the output error in every
learning step
• Make a output error graph (put learning step
on a horizontal axis and the output error on
vertical axis)
• Make a conclusions
Task 2
Subject: Influence of number of nodes in hidden
layer onto a learning progress
• Prepare a learning data using fuzzy feature
extractor with chosen settings and chosen
learning set of character images
• Create a three-layer NN with proper number
of input node and output node (depending on
how many feature components we are using
and how many kind of character images we
recognize).
• For each NN pass a learning task, note the
output error chart
• Make a conclusions
7. Conclusion and comments
This paper is addressed to our colleagues which are related the training process at universities. The
main goal of the paper is to exchange the experiences of using the modern achievements in pattern
recognition and image processing, NN, fuzzy logic and biometric technologies.
Preparing the laboratory works for learning new scientific achievements is always difficult. These
difficulties have to classify as „technological”, they have methodological character. This problem in
special literature on education is referred as
mapping the new achievements in educational process.
That’s why the subject of the paper can be interpreted as
a way to map new achievements in Pattern recognition and

image processing into didactic process of university, by
means of the laboratory work. The initial state of the problem
considered in the paper, was to create a laboratory work on
one of chapters of the course ”Pattern recognition and image
processing”. As a result, it has become possible to essentially
extend the functional abilities of the created system. It
allowed using the system for other courses: ”Artificial
Intelligence” and ”Banking Information Technologies”. It
was reached owing to the fact that the system is implemented
as a set of modules based on typical programme packages,
and allows training all its components and also fast
modification. It means that numerous methods of image pre-
processing can be learned in the course ”Pattern recognition and image processing” basing on such real
data as handwriting symbols. Its main goal - to learn handwriting recognition technology based on
fuzzy logic and NN - is kept.
Fuzzy logic and NN can be applied at many levels in complex handwriting recognition process.
Fuzzy logic systems can represent high-level knowledge and it is very importand point of our laboratory
task.
When we have been prepared this paper, we got to know the paper [Gader]. It stimulated us to
develop the supposed system. We would like to make the work more complex in future, and to propose
students to study the technology of character recognition considered in this paper.
The originality of authors’ approach is in that the system allows for students to get to know better
one of directions of biometric technologies - handwriting and signature identification. Biometric
technologies are one of principal components of the lectures and practical works on Banking
Information Technologies.
Unfortunately, we can not estimate the efficiency of this laboratory work with respect to its using in
educational process. We have only indirect estimations. So, for example, during task executing, it was
possible notice easily the most prepared students, and also to reveal the students which are able for
research work. Besides this, the results of the work were frequently resulted as examples on
examinations.

The presented here system is used in didactic process of the Institute Computer Science &
Information Systems (Technical University of Szczecin, Poland). Students universally found this task
interesting and useful, and they were able to complete the contents and propose different versions of the
exercises.We think, this experience can be used at other universities too.
References
1. Amin A., and Wilson W.A., Hand-Printed Character Recognition System Using Artificial Neural Networks,
Proc. of the 2
nd
Int.Conf. on Pattern Recognition, pp. 943-946, 1993
2. Bernard G., Multilayer Perceptron and Uppercase Handwrtitten Character Recognition,Proc.ofthe2
nd
Int.
Conf. on Pattern Recognition, 1993 pp. 935-938
3. Gader P.D., Keller J.M., Krishnapuram R., Chiang J-H., Mohamed M.A. Neural and Fuzzy Methods in
Handwriting Recognition, Computer, Feb, 1997, pp.79-86
4. Gader P.D., Mohamed M.A., and Chiang J-H.,Comparison of Crisp and Fuzzy Character Neural Networks in
Handwritten Word Recognition, IEEE Trans. Fuzzy Systems, Aug., 1995, pp.357-364
5. Pepe S., and Chen C.S., Fuzzy Logic for Handwritten Numeral Character Recognition in Fuzzy Models for
Pattern Recognition, ed. J.C Bezdek, S.K. Pal IEEE Press 1992
6. Shmerko V., and Yanushkevich S., Script of lectures on Artificial Inelligence (In Polish)
7. Shmerko V. Experience of Designing and Using Signature and Handwriting Person Indetification Systems.
The British Computer Society, Computer Security Specialist Group Annual Conference, March, London,
1995
8. Shmerko V. and Yanushkevich S. (Eds.), Banking Information Systems, Technical University of Szczecin,
Poland, 1997
9. Soldek J. , Shmerko V., Phillips Phil, Kukharev G., Rogers W., and Yanushkevich S., Image Analysis and
Pattern Recognition in Biometric Technologies, this issue
10. Zhou R.W., and Quck C., An Automatic Fuzzy Neural Network Driven Signature Verification System, The
IEEE Int. Conf. on Neural Networks, Washngton, USA, vol.2, 1996, pp.1034-1039
11. Fu LiMin, Neural Networks in Computer Intelligence, McGraw-Hill, Inc.,1994

12. Zadeh L.A. Outline of a New Approach to the Analysis of Complex Systems and Decision Processes. IEEE
Trans.Syst.Man Cybern. 3(1), 1973, pp.28-44
Fig. 11. Influence of number (from 5 to
25) of nodes in hidden layer on a
learning progress
5
15