A Vietnamese Text-based Conversational Agent

Table of Contents
1 Introduction 1
1.1 A Semantic Approach for Question Analysis . . . . . . . . . . . . . . 1
1.2 A Vietnamese Text-based Conversational Agent . . . . . . . . . . . . 2
1.3 Thesis Organisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Literature review 4
2.1 Text-based conversational agents . . . . . . . . . . . . . . . . . . . . 4
2.1.1 Using keywords for pattern matching . . . . . . . . . . . . . . 4
2.1.2 Using the sentence similarity measure
for pattern matching . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 FrameScript Scripting Language . . . . . . . . . . . . . . . . . . . . . 9
2.3 Question answering systems . . . . . . . . . . . . . . . . . . . . . . . 12
3 Our Question Answering System Architecture 15
3.1 Vietnamese Question Answering System . . . . . . . . . . . . . . . . 15
3.1.1 Natural language question analysis component . . . . . . . . . 16
3.1.1.1 Intermediate representation of an input question . . 16
3.1.1.2 Question analysis . . . . . . . . . . . . . . . . . . . . 17
3.1.2 Answer retrieval component . . . . . . . . . . . . . . . . . . . 18
3.2 Using FrameScript for question analysis . . . . . . . . . . . . . . . . . 19
3.2.1 Preprocessing module . . . . . . . . . . . . . . . . . . . . . . . 19
3.2.2 Syntactic analysis module . . . . . . . . . . . . . . . . . . . . 20
3.2.3 Semantic analysis module . . . . . . . . . . . . . . . . . . . . 22
4 Text-based Conversational Agent for Vietnamese 24
4.1 Overview of architecture . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.2 Determining separate contexts . . . . . . . . . . . . . . . . . . . . . . 25
4.3 Identifying hierarchical contexts . . . . . . . . . . . . . . . . . . . . . 27
vi
TABLE OF CONTENTS vii
5 Evaluation and Discussion 29
5.1 Experimental results
for Vietnamese text-based conversational agent . . . . . . . . . . . . 29

5.2 Question Analysis for English . . . . . . . . . . . . . . . . . . . . . . 31
5.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
6 Conclusion 34
A Scripting patterns
for English question analysis 36
B Definitions of question-class types 38
C Definitions of question-structures 40
List of Figures
2.1 O’Shea et al.’s conversational agent framework. . . . . . . . . . . . . 7
2.2 Aqualog’s architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.1 Architecture of our question answering system. . . . . . . . . . . . . . 16
3.2 Architecture of the natural language question analysis component
using FrameScript. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.1 Architecture of our Vietnamese text-based conversational agent. . . . 25
viii
List of Tables
4.1 Script examples of “subjects” . . . . . . . . . . . . . . . . . . . . . . . 26
4.2 Transformations between contexts . . . . . . . . . . . . . . . . . . . . 27
4.3 Order of transformation rules . . . . . . . . . . . . . . . . . . . . . . 28
4.4 Ordered transformation between contexts . . . . . . . . . . . . . . . . 28
5.1 List of transformations among contexts . . . . . . . . . . . . . . . . . 30
5.2 Unsatisfying analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.3 The satisfied degree of students . . . . . . . . . . . . . . . . . . . . . 31
5.4 Number of rules corresponding with each question-structure type . . . 31
5.5 Number of rules with conditional responses . . . . . . . . . . . . . . . 32
5.6 Number of questions corresponding with each question-structure type 32
5.7 Error results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
ix
List of Abbreviations
CA Conversational Agent

QA Question Answering
IR Information Retrieval
IE Information Extraction
GATE General Architecture for Text Engineering
JAPE Java Annotation Patterns Engine
NLIDB Natural Language Interface to DataBase
POS Part-of-Speech
NLP Natural Language Processing
GUI Graphic User Interface
x
Chapter 1
Introduction
1.1 A Semantic Approach for Question Analysis
The goal of question answering systems is to give answers to the user’s questions
instead of ranked lists of related documents as used by most current search engines
(Hirschman and Gaizauskas, 2001). Natural language question analysis component
is the first component in any question answering systems. This component creates
an intermediate representation of the input question, which is expressed in natural
language, to be utilized in the rest of the system.
For the task of translating a natural language question into an explicit intermedi-
ate representation of the complexity in question answering systems, most published
works so far use rule-based approach to the best of our knowledge. Some question
answering systems such as (Lopez et al., 2007; Phan and Nguyen, 2010) manually
defined a list of sequence rule structures to analyze questions. However, in these
rule-based approaches, manually creating the rules is error-prone and expensive in
time and effort.
In this thesis, we present an approach to return an intermediate representation
of question via FrameScript scripting language (McGill et al., 2003). Natural lan-
guage questions will be transformed into intermediate representation elements which
include the construction type of question, question class, keywords in question and

semantic constraints between them. Framescript allows users to intuitively write
rules to directly extract the output tuple.
1
2 Chapter 1. Introduction
1.2 A Vietnamese Text-based Conversational Agent
A text-based conversational agent is a program allowing the conversational inter-
actions between human and machine by using natural language through text. The
text-based conversational agent uses scripts organized into contexts comprising hier-
archically constructed rules. The rules consist of patterns and associated responses,
where the input is matched based on patterns and the corresponding responses are
sent to user as output.
We focus on the analysis of input text in building a conversational agent. Re-
cently, the input analysis over user’s statements have been developed following two
main approaches: using keywords (ELIZA (Weizenbaum, 1983), ALICE (Wallace,
2001), ProBot (Sammut, 2001)) and using similarity measures (O’Shea et al., 2010;
Graesser et al., 2004; Traum, 2006) for pattern matching. The approaches using
keywords usually utilize a scripting language to match the input statements, while
the other approaches measure the similarity between the statements and patterns
from the agent’s scripts.
In this thesis, we introduce a Vietnamese text-based conversational agent ar-
chitecture on a specific knowledge domain. Our system aims to direct the user’s
statement into an appropriate context. The contexts are structured in a hierarchy of
scripts consisting of rules in FrameScript language (McGill et al., 2003). In addition,
our text-based conversational agent was constructed to integrate in a Vietnamese
question answering system. Our conversational agent provides not only information
related to user’s statement but also provides necessary knowledge to support our
question answering system when it is unable to find an answer.
The knowledge domain we used to build our text-based conversational agent is
the academic regulation at Vietnam National University, Hanoi (VNU). The aca-
demic regulation book helps students to know the course programs, the regulation of

examinations, the discipline at VNU However, most students don’t prefer reading
the academic regulation book. Therefore, our contribution creates an interaction
channel to offer the necessary information to students. Once students give their
statements that they are interested in the academic regulation, our text-based con-
versational agent responses these statements by providing the related information in
detail. Furthermore, our conversation agent also interacts with students by offering
the option to ask if students want to know other information.
1.3. Thesis Organisation 3
1.3 Thesis Organisation
This dissertation consists of 6 chapters. In chapter 2, we provide some literature re-
views and describe our Vietnamese question answering system architecture, in which
we present a method for converting a natural language question into an intermediate
representation, in chapter 3. We propose our Vietnamese text-based conversational
agent architecture in chapter 4. We describe our experiments and discussions in
chapter 5, and conclusion will be presented in chapter 6.
Chapter 2
Literature review
In this chapter, we review related works using text-based approaches for conversa-
tional agent (CA). Section 2.1 describes the approaches constructing rules to match
user’s natural language utterances in the ways of using keywords (in section 2.1.1)
and using a sentence similarity measure (in section 2.1.2). In addition, section 2.2
covers the basic knowledge background about FrameScript scripting language that
we have been working on, while section 2.3 presents reviews about the question
answering systems driving specific-domains.
2.1 Text-based conversational agents
2.1.1 Using keywords for pattern matching
ELIZA (Weizenbaum, 1983) was one of the earliest text-based conversational agents
based on a simple pattern matching by using the identification of keywords from
user’s statement. Then ELIZA transforms the user’s statement to an appropriate
rule and generates output response. The procedure that ELIZA responds to an user

input to give an appropriate output consists of five steps.
• Identify the important keywords appearing in user’s statement.
• Define some minimal context within which selected keyword occurs.
• Determine an appropriate transformation rule.
• Generate the responses when the input text contained no keywords.
4
2.1. Text-based conversational agents 5
• Provide a facilitate editing for scripts on the script writing level.
Transformation rules are used to serve decomposing a data string according to
certain criteria and reassembling a decomposed string according to certain assembly
specifications. Therefore, the input are analyzed based on the decomposition rules
triggered by keywords, and responses are generated against the reassembly rules
associated with selected decomposition rules. For example, encountering the input
sentence:
“It seems that you like me”
this sentence is decomposed into the four parts:
(1) It seems that (2) you (3) like (4) me
by using the decomposition rule:
(0 YOU 1 ME)
The associated response might then be:
“What makes you think I like you”
by using the reassembly rule:
(WHAT MAKES YOU THINK I 3 YOU)
An integer 0 in the decomposition rule will match more words and a non-zero integer
“n” appearing in a decomposition rules indicates that exactly “n” words will be
matched, while an integer 3 in the above reassembly rule shows that the third part
of the decomposed sentence is inserted in its place to reply the input sentence. If
each word is defined in a dictionary of keywords by scanning an input sentence from
left to right, then only decomposition rules containing that keyword need to be tried.
An ELIZA script consists mainly of a set of list structures as following:

(K ((D
1
) (R
1, 1
) (R
1, 2
) (R
1, m1
))
((D
2
) (R
2, 1
) (R
2, 2
) (R
2, m2
))
.
.
.
((D
n
) (R
n, 1
) (R
n, 2
) (R
n, mn
)))

where K is the keyword, D
i
the ith decomposition rule associated with K and R
i, j
the jth reassembly rule associated with the ith decomposition rule. Any number
of decomposition rules may be associated with a given keyword and any number of
reassembly rules with any specific decomposition rule since having no predetermined
ordering limitations.
6 Chapter 2. Literature review
ALICE (Wallace, 2001) is a text-based conversational agent as chat robot uti-
lizing an XML language called Artificial Intelligence Markup Language (AIML).
AIML files consist of category tags representing rules; each category tag contains a
pair of pattern and template tag. The entire category is stored in a tree. The system
searches the pattern according with an user input by using depth-first search in the
tree, and produces the appropriate template as a response. For example, a category
below:
<category>
<pattern>DO YOU KNOW WHO * IS?</pattern>
<template><srai>WHO IS <star/></srai></template>
</category>
AIML uses the * wild-card character in creating patterns to match any non-zero
number of words. When an input matched this pattern, the portion bound to the
* wild-card may be placed into the response with the <star/> markup. This above
category reduces any input of the form “Do you know who X is?” to “Who is X”.
AIML allows two types of optional context called “that” and “topic”. The that
tag appearing inside the category matches the robot’s previous utterance, while the
topic tag occurring outside the category indicates a group of categories together and
the topic may be set inside any template. Observing a sample topic, like:
<topic name=“MOVIES”>
<category>

<pattern>YES</pattern>
<that>DO YOU LIKE ROMANTIC MOVIES</that>
<template>What is your favourite romantic movie?</template>
</category>
<category>
<pattern>YES</pattern>
<that>DO YOU LIKE ACTION MOVIES</that>
<template>What is your favourite action movie?</template>
</category>
When the client says yes, the program must discover the robot’s previous utterance.
If the robot asked “Do you like romantic movies?”, the response sent to reply is
“What is your favourite romantic movie?”.
AIML is clever and simple, and easy for implementation and a good start for
beginners writing simple bots. However, it is difficult to write and debug more
2.1. Text-based conversational agents 7
discriminating patterns, and it is very hard to know all the transformations available
because AIML depends on self-modifying the input.
Sammut (Sammut, 2001) presented a text-based CA called ProBot that is able
to extract data from users. ProBot’s scripts are typically organized into hierarchi-
cal contexts consisting of a number of organized rules to handle unexpected inputs.
Concurrently, McGill et al. (McGill et al., 2003) derived from ProBot’s scripts (Sam-
mut, 2001) build the rule system in FrameScript scripting language (in section 2.2).
FrameScript (McGill et al., 2003) provides for the rapid prototyping of conversa-
tional interfaces and simplifies the writing of scripts.
2.1.2 Using the sentence similarity measure
for pattern matching
O’Shea et al. (O’Shea et al., 2008, 2010) proposed a text-based conversational agent
framework (shown in figure 2.1) using semantic analysis. All patterns in scripts are
the natural language sentences. The pattern matching uses a sentence similarity
measure (Li et al., 2006) to calculate the similarity between sentences from scripts

and user input. The highest ranked sentence is selected and its associated response
is sent as output.
Figure 2.1: O’Shea et al.’s conversational agent framework.
Scripts used in framework consist of contexts relating to a specific topic of conver-
sation. Each context contains one or more rules, and each rule uses “s” to represent
8 Chapter 2. Literature review
a natural language sentence and “r” to represent a response statement. For example,
considering a following rule:
<Rule_01>
s: I’m a student
r: Which university do you study?
With a user’s statement:
“I am a master student” or
“I am a phd student”
This input and the natural language sentences from the scripts are received in order
to send the sentence similarity measure. Then sentence similarity measure calculates
a firing strength for each sentence pair to rank the sentences. In this above example,
the highest ranked sentence selected is “I’m a student” and its associated response
sent to user is “Which university do you study?”.
The advantages of using a sentence similarity measure for pattern-matching is
that rule structures are simplified and reduced in size and complexity. By contrast,
this approach can’t retrieve some information from an input to insert into response
like using keywords for presented section 2.1.1.
Graesser et al. (Graesser et al., 2004) presented a conversational agent called
AUTOTUTOR matching input statements in the use of Latent Semantic Analysis.
Traum (Traum, 2006) adapted the effective question answering characters (Leuski
et al., 2006) to build a conversational agent also employing Latent Semantic Analysis
for pattern matching.
2.2. FrameScript Scripting Language 9
2.2 FrameScript Scripting Language

FrameScript (McGill et al., 2003) is a language for creating multi-modal user in-
terfaces. It employs from Sammut’s Probot (Sammut, 2001) to enable rule-based
programming, frame representations and simple function evaluation. The Frame-
Script scripting language also proposes a set of tools to represent knowledge and
interacting with users and external devices.
Each script in FrameScript (McGill et al., 2003) includes a list of rules matched
against user input and used to give the appropriate response. Rules are grouped into
particular contexts of the form: context_name :: rule_set. The scripting rules in the
FrameScript language consist of patterns and responses with the form:
pattern ==> response.
Pattern expressions may contain 2 wild-cards characters which are * and
∼
. * will
match 0 or more words and
∼
within a word indicates that 0 or more characters may
be matched. Pattern expressions also allow the use of the alternatives by constructing
of the form:
{ alternative 1 | alternative 2 | }.
Moreover, patterns use non-terminals to reuse other pattern expressions by writing
the name of the non-terminal surrounded by ‘<’ and ‘>’. Non-terminals are often
declared as list of alternatives followed by ;;.
For example:
Number:: ==>
{1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 10 | 11 | 12 | 13 | 14 | 15
| 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25};;
Response expressions contain two different types which are sequences and alter-
natives. Sequence of responses has the form surrounded by brackets:
[response 1 | response 2 | | another response],
where each response is given in turn every time the pattern is matched and the

sequence repeats when the last response is output. Alternatives have the form sur-
rounded by braces:
{response 1 | response 2 | | another response},
in which any response may be chosen randomly for user output.
In addition, responses utilize the ‘#’ to perform some action such as chang-
ing the current context. For example, #goto(a_script) transforms a conversation or
interaction from one context to another. Similarly, ‘
∧
’ is used to perform actions, ex-
10 Chapter 2. Literature review
cept that when the following expression is evaluated it is inserted into response not
thrown away. And some response expression may be dependent on some conditions
holding true in the constructed form below:
* ==>
[
∧
(condition) –>
response if true
|
∧
(! condition) –>
response if false ]
Furthermore, some pattern elements create a numbered match component when
a pattern matches. These component are segments of the input that can be referred
to in a response using ‘
∧
’. Pattern elements that identify match components are
wild-card (*, and
∼
), alternatives and non-terminals. When ‘

∧
’ is followed by an
integer then the numbered pattern component associated with that integer is placed
in the output response. Encountering an example as following:
{My name is | I’m} * ==>
[ Hello
∧
2. How old are you? ]
I am <Number> years old ==>
[
∧
(
∧
1 <= 20) –> Are you a student?
| How do you do? ]
The transcript of dialogue is shown below illustrating the above example:
User: My name is X
CA: Hello X. How old are you?
User: I am 19 years old
CA: Are you a student?
An input received from user is given to a domain in order to ensure that the input
is matched against the correct scripts. Script can be registered as topic in a domain
to become the current script and process the input. When a script is registered as
a topic, the domain uses the script’s trigger to determine whether or not an input
activates that topic. If a topic doesn’t have a trigger, any input will activate it.
When a topic’s trigger matches the input, it becomes the current context and the
current topic.
2.2. FrameScript Scripting Language 11
Example ::
domain example

trigger{* {Hi | hi | Hello | hello} *}
* {Hi | hi | Hello | hello} * ==> [Hi there!]
When writing complex scripts where scripts have similar behaviours, FrameScript
is possible to use inheritance to enable rule to be shared between scripts. Moreover,
FrameScript allows defining failsafes for scripts. A failsafe is another script whose
rules would be used if an input matches incorrectly any of rules for a script.
The order in which domains attempt to determine rules that the input should
be matched is:
1. triggers of the topics
2. the current context
3. the failsafe of the current context
4. the current topic
5. the failsafe of the current topic
6. the failsafe for the domain
When an input is compared to the rules of a script, the input is first compared to
the rules specifically defined by the script. If none of these rules match, the input is
matched against the rules of the script’s parents. The rules of the scripts are tried
in top to bottom order.
12 Chapter 2. Literature review
2.3 Question answering systems
Kinds of question answering systems range from closed-domain systems (aiming to
answer questions in a specific domain) to open-domain systems (aiming to answer all
of asked questions). In our experiment, the open-domain systems focus on retrieving
and ranking related documents corresponding with the input, while the close-domain
systems focus on analysis natural language questions to extract reliable terms.
Additionally, natural language question analysis component is the first compo-
nent in any question answering systems. This component creates an intermediate
representation of the input question, which is expressed in natural language, to be
utilized in the rest of the system. The basis of the question parser is question clas-
sification that can be defined as the task of mapping a given question to one of

k classes based on the possible types of the answers (Li and Roth, 2002b). Subse-
quently, natural language questions analysis techniques are used to identify keywords
and semantic relations in input questions.
Therefore, our related works come from reviewing question answering systems
against the question analysis approaches in specific domain driven ones.
Pattern-matching based systems
Close-domain question answering systems are usually linked to relational databases
and called natural language interfaces to databases. A natural language interface to
a database (NLIDB) is a system that allows the users to access information stored in
a database by typing questions using natural language expressions (Androutsopoulos
et al., 1995).
Early NLIDB systems used pattern-matching technique to process user’s ques-
tions and generate corresponding answers. (Sneiders, 2002) presented a NLIDB sys-
tem by using question patterns covering conceptual model of the database. The
input is converted into SQL query by using defined templates that contain entity
slots – free space for data instances representing the primary concepts of the ques-
tion. Some other open-domain systems presented in (Wu et al., 2003; Saxena et al.,
2007) used pattern-matching techniques to respond user’s requests.
The main advantage of pattern-matching approach is its simplicity, and the sys-
tem can be able to perform well in certain applications. However, the one’s shallow-
ness would often lead to bad results.
2.3. Question answering systems 13
Semantic-based systems
Later NILDBs respond user’s question by using semantic grammar to parse the
input into syntax tree and mapping the tree to a database query. In semantic-based
systems, the grammar’s categories (i.e. the non-leaf nodes appearing in the parse
tree) have not to correspond to syntactic concepts (Androutsopoulos et al., 1995).
Semantic constraints are usually enforced by choosing semantic grammar categories,
in which the grammar’s categories can also be chosen to ease the mapping from the
syntax tree to database objects.

Nguyen and Le (Nguyen and Le, 2008) introduced a NLIDB question answering
system in Vietnamese employing semantic grammars. Their system includes two
main modules: QTRAN and TGEN. QTRAN (Query Translator) maps a natural
language question to an SQL query while TGEN (Text Generator) generates answers
based on the query result tables. QTRAN uses limited context-free grammars to
analyze user’s question into syntax tree via CYK algorithm. The syntax tree is
then converted into an SQL query by using a mapping dictionary to determine
names of attributes in Vietnamese, names of attributes in the database and names
of individuals stored in these attributes.
The PRECISE system (Popescu et al., 2003) maps the natural language ques-
tion to a unique semantic interpretation by analyzing some lexicons and semantic
constraints. (Stratica et al., 2003) described a template-based system to translate
English question into SQL query by matching the syntactic parse of the question to
a set of fixed semantic templates. Some other systems based on semantic grammar
rules such as Planes (Waltz, 1978), Eufid (Templeton and Burger, 1983). Semantic
grammar-based approaches were considered as an engineering methodology, which
allows semantic knowledge to be easily included in the system.
Annotation-based systems
Recently, some question answering systems that used semantic annotations gener-
ated high results in natural language question analysis. A well known annotation
based framework is GATE (General Architecture for Text Engineering) (Cunning-
ham et al., 2002) which have been used in many question answering systems like
Ontology-based AquaLog (Lopez et al., 2007) and QuestIO (Damljanovic et al.,
2008) systems, and Galea’s open-domain system (Galea, 2003), especially for the
natural language question analysis component.
14 Chapter 2. Literature review
Aqualog (Lopez et al., 2007) shown in figure 2.2 is an ontology-based question
answering system for English and is the basis for the development of our system. A
natural language question is mapped to a set of representation based on the inter-
mediate triple that is called a Query-Triple through the Linguistic Component by

using Java Annotation Patterns Engine (JAPE) grammars in GATE (Cunningham
et al., 2002). The Relation Similarity Service takes a Query-Triple and processes
it to provide queries with respect to the input ontology called Onto-Triple. Then
Aqualog uses Onto-Triple to return an answer for users.
Figure 2.2: Aqualog’s architecture.
In our experiment, we reported an approach to convert Vietnamese natural lan-
guage questions into intermediate representation element in query-tuples (Question-
structure, Question-class, Term
1
, Relation, Term
2
, Term
3
) based on semantic annota-
tions via JAPE grammars (Nguyen et al., 2009). The selected query-tuple type is
more complex aiming to cover a wider variety of question types in different languages.
In addition, we proposed a language-independent approach to acquire JAPE rules
in a systematic manner which avoids unintended interaction among rules (Nguyen
et al., 2011). (Phan and Nguyen, 2010) presented an approach to syntactically and
semantically map Vietnamese questions into triple-like of Subject, Verb and Object
in also utilizing JAPE grammars.
The START (Katz, 1997; Katz et al., 2006) question answering system also
used natural language annotations (Katz, 1997) without utilizing GATE. A lexical
database WordNet (Fellbaum, 1998) is important natural language application. After
the appearance of WordNet, almost question answering systems used it to provide
information for analyzing questions.
Chapter 3
Our Question Answering System
Architecture
In this chapter, we introduce the overview of our first Ontology-based question an-

swering system for Vietnamese (in section 3.1). Our system contains a front-end
component that performs syntactic and semantic analysis on natural language ques-
tions. The back-end component is responsible for making sense of the user’s query
with respect to a target ontology using various concept-matching techniques between
a natural language phrase and elements in the ontology. The communication between
the front-end and back-end is an intermediate representation of the question, which
captures the semantic structure of the users’ question.
Furthermore, we focus on describing a rule-based approach to directly extract an
intermediate representation elements of question via FrameScript scripting language
(McGill et al., 2003) (in section 3.2).
3.1 Vietnamese Question Answering System
The architecture of our question answering system is shown in figure 3.1. It includes
two components: the Natural language question analysis and the Answer retrieval.
The question analysis component takes the user’s question as an input and re-
turns a query-tuple representing the question in a compact form. The role of this
intermediate representation is to provide structured information of the input ques-
tion for later processing such as retrieving answers.
The answer retrieval component includes two main modules: Ontology mapping
15
16 Chapter 3. Our Question Answering System Architecture
and Answer extraction. It takes an intermediate representation produced by the
question analysis component and an ontology as its input to generate semantic
answers.
Figure 3.1: Architecture of our question answering system.
3.1.1 Natural language question analysis component
3.1.1.1 Intermediate representation of an input question
The intermediate representation used in our approach aims to cover a wider variety
of question types. It consists of a question-structure and one or more query-tuple in
the following format:
( question-structure, question-class, Term

1
, Relation, T erm
2
, Term
3
)
where T erm
1
represents a concept (object class), Term
2
and T erm
3
, if exist,
represent entities (objects), Relation (property) is a semantic constraint between
terms in the question. This representation is meant to capture the semantics of the
question.
Simple questions corresponding to basic constructions only have one query-tuple
3.1. Vietnamese Question Answering System 17
and its question-structure is the query-tuple’s question-structure. More complex
questions such as composite questions are constructed by several sub-questions,
each sub-question is described by a separate question-structure, and the question-
structure capture this composition attribute. This representation is chosen so that
it can represent a richer set of question types. Therefore, some terms or relation in
the query-tuple can be missed. Composite questions such as:
“list all students in the Faculty of Information Technology whose hometown is
Hanoi?”
has question structure of type And with two query-tuples where ? represents a
missed element: ( UnknRel , List , students , ? , Faculty of Information Technology,
? ) and ( Normal , List , students, hometown, Hanoi, ? ).
The definitions of the following question categories of HowWhy, YesNo, What,

When, Where, Who, Many, ManyClass, List and Entity, and question-structures of
Normal, UnknTerm, UnknRel, Definition, Compare, ThreeTerm, Clause, Combine,
And, Or, Affirm, Affirm_3Term, Affirm_MoreTuples could be found in appendixes
B and C respectively.
3.1.1.2 Question analysis
We wrapped existing linguistic processing modules for Vietnamese such as Word
Segmentation, Part-of-speech tagger (Pham et al., 2009) as GATE plug-ins. Results
of the modules are annotations capturing information such as sentences, words,
nouns and verbs. Each annotation has a set of feature-value pairs. For example, a
word has a feature category storing its part-of-speech tag. This information can then
be reused for further processing in subsequent modules. New modules are specifi-
cally designed to handle Vietnamese questions using JAPE grammars over existing
linguistic annotations.
There are three modules that we use to get an intermediate representation of
user’s question including: preprocessing, syntactic analysis and semantic analysis.
The preprocessing module generates TokenVn annotations representing a Viet-
namese word with features such as part-of-speech to identify question-words and
comparing-phrases or special-words by using JAPE rules.
The syntactic module is responsible for identifying noun phrases, question
phrases and relation phrases between noun phrases or noun phrases and question
phrases. The different modules communicate through the annotations, for example,
18 Chapter 3. Our Question Answering System Architecture
this module uses the TokenVn annotations, which is the result of the preprocessing
module.
The semantic analysis module identifies the question structure and produces
the query-tuples as the intermediate representation (question-structure, question-
class, Term
1
, Relation, Term
2

, Term
3
) of the input question using the annotations
generated by the previous modules. Using results from two front modules, we use
JAPE grammars to detect the question structure and corresponding terms and re-
lations.
We create the intermediate representation of input question in hard-wire manner
linking every detected pattern via JAPE grammars to Java source codes to extract
corresponding elements. It takes a lot of time and effort when appearing new pat-
terns. Therefore, we will present a knowledge acquisition approach to directly get
an intermediate representation of question by using FrameScript scripting language
(McGill et al., 2003) in the section 3.2 to resolve above mentioned problems.
3.1.2 Answer retrieval component
The answer retrieval component includes two main modules: Ontology Mapping and
Answer Extraction as shown in figure 3.1. It takes an intermediate representation
produced by the question analysis component and an ontology as its input to gen-
erate a semantic answer.
The task of the Ontology Mapping module is to map terms and relations in
the query-tuple to concepts, instances and relations in the ontology by using string
names. If exact match is not possible, we use string distance algorithm (Cohen
et al., 2003; Vargas-Vera and Motta, 2004; Lopez et al., 2007) to find near-matched
elements in the Ontology with the similarity measure above a certain threshold. In
case ambiguity is still present, the system interacts with the users by presenting
different options to get the correct ontology element.
For each query-tuple, the result of the Mapping Ontology module is an ontology-
tuple where the terms and relations in the query-tuple are now their corresponding
elements in the ontology. How the Mapping Ontology module finds corresponding
elements in the ontology depends on the question structure. For example, when the
query-tuple contains Term
1

, Term
2
and Relation with Term
3
missing. It first tries to
match Term
1
and Term
2
with concepts or instances in the target Ontology. After that,
the set of potential relations in the Ontology contains only relations between the
3.2. Using FrameScript for question analysis 19
two mapped conceptsinstances. The ontology Relation is then identified in a similar
manner as mapping term to a concept or an instance. With the ontology-tuple, the
Answer Extraction module find all individuals of the corresponding ontology concept
of Term
1
, having the ontology Relation with the individual corresponding to Term
2
.
Depending on the question-structure and question-class, the best semantic answer
will be returned.
3.2 Using FrameScript for question analysis
Unlike existing approaches for English (Lopez et al., 2007) and Vietnamese (in
section 3.1.1) where the intermediate representation couldn’t be extracted directly
in rules, we will describe an approach to directly extract the representation of user’s
question using FrameScript scripting language (McGill et al., 2003).
To set the context for the discussion of question analysis, our approach presented
in figure 3.2 also consists of three sub-modules: preprocessing, syntactic analysis and
semantic analysis.

Figure 3.2: Architecture of the natural language question analysis component using
FrameScript.
3.2.1 Preprocessing module
The preprocessing module identifies part-of-speech tags in a user’s question. After
that, we use part-of-speech tags to create basic scripts for detecting words. The ba-
sic scripts Noun, Verb, Determiner, Adjective, Adverb, Conjunction and Preposition
are used to identify corresponding nouns, verbs, determiners, adjectives, adverbs,
20 Chapter 3. Our Question Answering System Architecture
conjunctions and prepositions. In fact, these scripts will be used in creating rules in
the syntactic and semantic analysis modules at later stages.
Noun ::
{ NN | NNS | NNP | NNPS | NP | NPS | CD } ;;
Verb ::
{ FVG | VBN | VBZ | VBG | VBD | VBP | VB } ;;
Determiner ::
{ DT | PRP } ;;
Adjective ::
{ JJ | JJR | JJS } ;;
Adverb ::
{ RB | RBR | RBS } ;;
Conjunction ::
{ CC } ;;
Preposition ::
{ PREP | TO | IN } ;;
3.2.2 Syntactic analysis module
This module is responsible for determining noun phrases, question phrases and re-
lation phrases between noun phrases or noun phrases and question phrases.
Concepts and entities are normally expressed in noun phrases. Therefore, it is
important that we can reliably identify noun phrases in order to generate the query-
tuple. Hence we build the script called NounPhrase (such as a sample script below) to

specify patterns of noun phrases by utilizing scripts generated from the preprocessing
module.
Composite ::
{ <Noun> | <Conjunction> | <Adjective>
| <Adverb> <Adjective> } ;;
NounPhrase ::
{ <Noun> | <Determiner> <Noun>
| <Composite> <Noun>
| <Determiner> <Composite> <Noun> } ;;
For example, given the following question: “which projects are about ontologies and