Lecture Notes in Computer Science- P72 docx

F. Li et al. (Eds.): ICWL 2008, LNCS 5145, pp. 344–355, 2008.
© Springer-Verlag Berlin Heidelberg 2008
Automated Chinese Handwriting Error Detection Using
Attributed Relational Graph Matching
Zhihui Hu
1,2,3
, Howard Leung
2,3
, and Yun Xu
1,2

1
Department of Computer Science and Technology,
University of Science & Technology of China, Hefei, China
2
Joint Research Lab of Excellence, CityU-USTC Advanced Research Institute,
Suzhou, China
3
Department of Computer Science, City University of Hong Kong, Hong Kong S.A.R.
, ,
Abstract. Due to the complex shapes and various writing styles of Chinese
characters, it is a challenge to automatically detect the errors in people’s hand-
writing. In this paper, we use attributed relational graph to represent a Chinese
character. To model the spatial relationships between the strokes in a Chinese
character, a refined interval relationship that considers more granular levels is
proposed. A novel interval neighborhood graph is also proposed to compute the
distances among the refined interval relationships. Error-tolerant graph match-
ing is used to locate the stroke production errors, sequence error as well as the
spatial relationship errors. We also propose a pruning strategy in order to speed
up the graph matching. Experiment results show that our proposed method out-
performs existing approaches in terms of accuracy as well as its ability to han-

dle more kinds of handwriting errors in less computational time.
Keywords: Chinese handwriting error detection, attributed relational graph,
stroke spatial relationship error, stroke spatial relationship error, error-tolerant
graph matching.
1 Introduction
A Chinese character is an ideogram composed of many strokes. The correct handwrit-
ing should follow the correct position, proportion and order of each stroke. Law et al.
[1] shows the following handwriting errors children may often make: 1) stroke pro-
duction errors that include missing, extra, broken, and concatenated strokes; 2) stroke
sequence errors. Besides, there exist other handwriting errors such as spatial relation-
ship errors resulting from problems in the relative length or position between strokes.
When a student makes a handwriting mistake, he/she often does not even realize it. It
is thus essential for the student to receive feedback about his/her handwriting in order
to correct any mistakes.
Traditionally, the teacher can help the student find out their handwriting errors in
class however the teacher’s available time for each student is limited. As a result, we
are motivated to build a Chinese handwriting education system for assisting the
teacher when the teacher is absent. In this system, a student can first write a Chinese
Automated Chinese Handwriting Error Detection 345
character by following a template character from teacher then the system can auto-
matically check the handwriting and give feedback to indicate whether and where
there are any errors.
The existing handwriting education systems can be divided into two categories.
The first one is the view-only system. The student can see how a Chinese character
should be written but they cannot practice handwriting through the system [2, 3]. The
other category allows the student to practice handwriting and gives some feedback to
indicate if there are errors in their handwriting. These systems can be further divided
into four main streams. The first one is focused on locating the production errors [4,
5, 6]. The second stream can only evaluate the stroke sequence errors [7]. The third
stream can detect the spatial relationship error among strokes [8]. The last one is the

combination of the previous types. In [9] the system can find out both the stroke pro-
duction and sequence errors but without considering the spatial relationship errors. As
a result, we are motivated to explore a method that can identify the stroke sequence,
production and spatial relationship errors at the same time.
In this paper, we propose a method that can not only identify the stroke production
errors and sequence error but also the spatial relationship errors between strokes given
an input online Chinese handwriting. This is achieved by using the attributed rela-
tional graph (ARG) matching. Attributed relational graph is a powerful tool to repre-
sent the relational structure of a pattern. It has been used in 2D recognition [10, 11] as
well as Chinese handwriting education [6]. In our application, the Chinese character is
represented by a complete ARG. The nodes in the ARG are used to describe the
strokes of the character and the edges denote the relations between any two strokes.
As the relations between the Chinese characters are rather complex, we propose to
extend the existing interval relationship to refine its granularity. The optimal detailed
matching between the two ARGs is the mapping between corresponding strokes. In
order to find this detailed matching, the error-tolerant graph matching [13, 14] is used
with the graph edit operations: deletion, insertion, substitution, merging and splitting
of the nodes and the edges. A* algorithm is applied to perform the state-space search-
ing of such a graph matching. The resulting operations can reflect the graph distor-
tions. On the other hand the operation of the edges can show the spatial relationship
between strokes. However, we should not ignore the computational complexity of
graph matching thus we propose a pruning strategy to reduce the matching time.
The main contributions of this paper is as follows: 1) we propose an algorithm that
can analyze an input online Chinese handwriting and determine stroke production
error, stroke sequence error and stroke spatial relationship error at the same time; 2)
we define a refined interval relationship to model the spatial relationship between
strokes and extend the interval neighborhood graph to obtain the distance measures
for the refined interval relationships; 3) we propose a pruning strategy in order to
reduce the state-space searching time while we apply the error-tolerant graph match-
ing. The remainder of this paper is organized as follows: In Section 2, the proposed

ARG matching method incorporating the spatial relationships is described. Experi-
ments and results are discussed in Section 3. Conclusions and future work are pro-
vided in Section 4.
346 Z. Hu, H. Leung

, and Y. Xu
2 Our Proposed Method
2.1 Overview
The flowchart of our method is illustrated in Figure 1. First, the sample handwriting
inputted by the student and the template character with which the student should fol-
low are both represented as ARGs. Then the error-tolerant graph matching is applied
on the two ARGs in order to find out the stroke production and sequence error in the
sample handwriting. Afterwards, the post processing can detect the stroke relationship
error. Finally the feedback that locates all the errors is provided to the student.
Representation
Representation
Character
matching
Post
p
rocessin
g
Sample
handwritin
g
Template
handwritin
g
Feedback

Fig. 1. Flowchart of our method
2.2 Spatial Relationship in Chinese Character
A Chinese character consists of many strokes that form a particular structure unique
to that Chinese character. The spatial relationship between strokes is one important
factor in determining whether a student’s Chinese handwriting is written correctly. In
object recognition, people have studied the spatial relationship between objects. Allen
firstly shows 13 interval relationships in [15] and the spatial relationships between
objects have been described in [16, 17].Nevertheless, it is not sufficient to use these
interval relationships to fully describe the spatial relationship between strokes. This
can be illustrated by the example in Figure 2. The strokes in Figure 2(a), (b) and (c)
all have the same ‘during d’ relation as defined in Allen’s interval relationship mean-
ing that the duration of stroke a is within the duration of stroke b. However, only
Figure 2(b) shows the standard handwriting of this character. The handwritings in
Figure 2(a) and (c) are non-standard because stroke a in Figure 2(a) is too long
whereas the one in Figure 2(c) is too short.
(a) Non-standard handwriting (b) Standard handwriting (c) Non-standard handwriting

Fig. 2. Example of spatial relationships in Chinese character
As illustrated in Figure 2, it can be observed that the relationship between the strokes
is not only the topological relationship but also the relative distance between the strokes.
A more granular definition of the interval relationship is able to distinguish among the
Automated Chinese Handwriting Error Detection 347
three cases in Figure 2. In particular, we propose to further refine the interval relation-
ship into three levels (f, m, l) by considering the distance information. The refined inter-
val relationships of the strokes in Figure 2(a), (b) and (c) become ‘dl’, ‘dm’ and ‘df’
respectively. The refined relationship with three additional levels based on the distance
can also be applied to other existing interval relationships. The resulting refined rela-
tionships are summarized in Figure 3.
Relation Symbol Symbol for inverse Example

Fig. 3. Refined interval relationships with more granular levels
2.3 Complete ARG Representation of Chinese Character
ARG was first described in [10] to represent the structure information of a pattern as
g=(V,E,α,β). In our application, the set of nodes V describe the strokes of the Chinese
character, and the set of edges E describes the relationships between any two strokes
as defined in Figure 3. The ARG representation is given as follows.
Nodes in the ARG. Each node stores the x and y coordinates of a stroke. The node
labeling function
V
LV →:
α
returns n data points for each stroke [6].
Edges in the ARG. Each edge stores the relation of the two nodes (strokes) which are
connected by this edge. The edge labeling function
E
LE →:
β
returns (μ, λ) where μ, λ
are the refined interval relationship along the x-axis and y-axis respectively.
As an example, a Chinese character and its stroke spatial relationships are shown in
Figure 4(a). The ARG representation of this character is shown in Figure 4(b). The strokes
a, b and c in the character are represented by the nodes a, b and c in the ARG. The term
r
s1s2
is the relationship between strokes s1 and s2, and
21),,,(2,1 sscbass ≠∈
.

348 Z. Hu, H. Leung

, and Y. Xu
a
c b
r
ac: (df, mi)
r
ca: (dif, m)
r
ab: (df, dif)
r
ba: (dif, df)
r
bc: (dm, >m)
r
cb: (dim, <m)
(a) A Chinese character (b) Corresponding ARG

Fig. 4. ARG representation of a Chinese character
In this example, r
ac
is denoted by (df ,mi), r
ab
is denoted by (df, dif), and r
bc
is denoted
by (dm,>m). Note that the r
ca
is formed simply by taking the inverse of each compo-
nent of the relationship used to represent r
ac

and is denoted by (dif, m).
2.4 Error-Tolerant Graph Matching
As illustrated in Figure 1, the input (sample) handwriting is represented as an ARG
g
1
=(V
1
,E
1
,α
1
,β
1
) and the template handwriting is represented as another ARG
g
2
=(V
2
,E
2
,α
2
,β
2
). In order to decide whether the two ARGs have some differences, we
find an error–tolerant graph matching from g
1
to g
2
which is a transformation denoted

by the function f [13, 14].This function f consists of many edit operations performed
on both nodes and edges. The node operations have been defined by the authors in [6]
with node substitution, merging, splitting, deletion and insertion. On the other hand,
we extend the work in [6] by adding the edge operations defined as follows: 1) edge
substitution implying that both nodes sharing this edge are correct; 2) edge deletion
implying that one of the nodes/both nodes sharing this edge is an extra or broken
stroke; 3) edge insertion implying that one of the node/both nodes sharing this edge is
a missing or concatenated stroke.

Edge substitution. The cost for the edge substitution is the matching cost between
an edge in the sample character and an edge in the template. We use Rt to denote
the set of edges in the template and Rs to denote the set of edges in the sample.
Note that an edge represents the spatial relationship between two strokes in a hand-
writing. The i-th template edge Rt
i
can be denoted by (μt
i
, λt
i
) and the j-th sample
edge Rs
j
can be denoted by (μs
j
, λs
j
). The dissimilarity between (μt
i
, λt
i

) and (μs
j
, λs
j
)
is defined as D(Rt
i
, Rs
j
) which is derived from the idea of the interval neighborhood
graph [16]. Two interval relationships are neighbors, if they can be transformed into
one another by continuous deformation (shortening, lengthening, and moving) [17].
We construct a new interval neighborhood graph in Figure 5 which considers our
proposed refined relationship with three levels (f, m, l) in each relationship defined
in Figure 3. Note that the three levels with the same interval relationship are close
to each other in the refined interval neighborhood graph since they can be trans-
formed from one to another by shortening or lengthening the distance between the
two strokes.