Coordination and control of multi agent systems

CONTROL & COORDINATION OF
MULTI-AGENT SYSTEMS
CHENG-HENG FUA
B. Eng (Hons.), National University of Singapore
A THESIS SUBMITTED
FOR THE DEGREE OF DOCTOR OF PHILOSOPHY
NUS GRADUATE SCHOOL FOR INTEGRATIVE SCIENCES & ENGINEERING
NATIONAL UNIVERSITY OF SINGAPORE
2008
Acknowledgments
One phase of my life ends and another begins. I look upon the road before me with
hope and with excitement, and as I face the future that is ahead, I look back upon the
way I have come, and am comforted by the fact that I am not alone; that I am blessed
with the great fortune of having the companionship of several distinct individuals. Like
Frodo Baggins and the fellowship of the Ring, or Dorothy and her companions in the
land of Oz, my own adventures would not have been possible without their support,
encouragement, guidance and wisdom, and I would like to take this opportunity to
express my gratitude to all of them.
To my thesis supervisor, Professor Shuzhi Sam Ge, for his inspiring presence, for
his constant, patient guidance, and for his selfless sharing of experiences in all things
research and more. Thanks also go to Professor Khiang Wee Lim, my thesis co-
supervisor, for his guidance and help on all matters concerning my research despite his
busy schedule. I would also like to thank Dr Javier-Ibanez Guzman, for his insightful
advice and guidance on shaping my research direction and goals.
To my friends. I have been extremely fortunate to have worked with many, many
brilliant people during my study, and who have always been willing and generous with
their time and friendship. Special thanks to Mr Keng Peng Tee and Mr Pey Yuen Tao,
my fellow intrepid adventurers, for the endless hours of discussions and brainstorming
that are always filled with creativity, inspiration and crazy ideas, for their moral sup-
port and for always being there to help. To Dr Xuecheng Lai, Dr Zhuping Wang and
Dr Feng Guan, for their friendship, help and guidance since the day I first joined the

research team. Thanks also go to Professor Khac Duc Do for his patience, guidance
and wonderful advice in my research on formation control. I would also like to thank
ii
Mr Chenguang Yang, Ms Chen Wang, Mr Hooman Samani, and many other fellow col-
leagues and researchers at the Social Robotics Laboratory for their help and friendship.
To the pillars of my life – my family – I would never be where I am today without
their unquestioning trust, support and encouragement. They have always been there for
me, stood by me through the good times and the bad, and have always given me their
full support in whatever choices I made. To my inner circle of friends – Mr Konghui
Kay, Mr Khiam Boon Png, Mr Keng Chuan Ong and Mr Thian Khoon Ng – who have
shown me what true friendship is, for their unwavering friendship and moral support,
and for always being there in times of need.
Finally, I am immensely grateful to the Agency of Science, Technology and Re-
search (A*STAR), as well as the NUS Graduate School of Integrative Sciences and
Engineering (NGS), for their funding and support, without which, this great adventure
might never have taken place.
iii
Abstract
This thesis considers in detail the technical issues associated with the effective con-
trol and coordination of Multi-Agent Systems (MAS), with particular emphasis on tech-
niques that increase the robustness of the team of physical agents, when subjected to
uncertainties, such as malfunctions and imperfect communications. For physical agent
teams which are to be deployed within uncertain and dynamic environments, it is im-
portant for the team to continue functioning even in the event of unforeseen disruptions
such as single-agent breakdowns and spurious communication losses, and continue be-
ing driven toward its collective goals. Such abilities are crucial for the success of au-
tonomous teams, and would constitute the main motivation for the work presented in
this thesis.
The thesis is organized according to the two main decision making levels where
coordination between members within an agent team can occur, namely on the Macro-

and Micro-levels of decision making, following the general flow of decisions, from
mission specification, assignment, to actual task accomplishment. Agent cooperation
on the macro-level concerns a more general management protocol that, in combination
with a planning and representation framework, manages the resources (i.e. robots) and
tries to arrive at a suitable distribution of tasks to either individual robots or sub-teams of
robots. This level of decision making focuses on mission and task representations, and
team organization algorithms. Actual task accomplishment by robot sub-teams require
further, more explicit, cooperation between individual members, and this falls into the
realm of micro-level coordination. Such forms of coordination is investigated in the
context of representing and cooperative accomplishment of multi-agent formations.
Under this framework, and with the above objectives in mind, the technical con-
iv
tributions of this thesis provide a step towards increasing the robustness of physical
agent systems operating within dynamic, uncertain operating domains. The proposed
solutions include:
(i) A general mission/ task representation framework based on the concept of basis
tasks, that is amenable to analysis, for the efficient portrayal, subdivision, and
allocation of tasks to agent teams on-the-fly.
(ii) A robust, cooperative task allocation scheme, the Cooperative Back-Off Scheme
(COBOS), for instantaneous task (re)distribution between spatially distant task
locations which are subjected to limited communications.
(iii) A representation framework for task allocation schedules over an extended time
period, based on the transformation of task schedules into an agent-formation
space, where stable, self-organizing, convergence algorithms are introduced to
form a dynamic time-extended allocation.
(iv) An efficient formation representation scheme, the Q-structure, to facilitate scal-
able and flexible agent formations. The Q-structure allows the representation of
a wide variety of formations, and coupled with a behavior based algorithms for
each agent, enables decentralized, robust formation tracking with automatic scal-
ing.

(v) A decentralized and reactive potential field based method is used for stably guid-
ing agents into formation based only on local communications, and further subdi-
viding the decision making process into the fast and slow time scales. Theoretical
analysis have also been carried out to verify the convergence properties of the nav-
igation controls. To further improve performance in scenarios involving limited
communications, methods for dynamically adapting the short term Q-structure
representation are also used.
v
Nomenclature
R the field of real numbers;
R
n×m
the set of n ×m-dimensional real matrices;
ˆx the unit vector of a vector x;
x
(f)
a vector x expressed in the frame f. Taken to be in
the world frame if unspecified;
g
(f)
(·) a function g(·), expressed in the frame f;
x the Euclidean norm of a vector x;
T
(f
2
)
(f
1
)
the transformation matrix from frame f

1
to f
2
;
R
jk
the jk-th element of a matrix R;
⊗ the vector product operator;
|X| the cardinality of a set X;
r
i
the robot i;
x
k
the k-th element of a vector X ∈ R
n
;
|X| the number of elements in a set X;
n
j
the number of robots r
j
has communication links with;
TSM(j) the Task Utility Matrix compiled by r
j
;
TSM
ij
(j) the (i, j)-th element of TSM(j);
TSM

−
(j) a sub-matrix made up of selected rows and columns
of TSM(j);
B, B the set, or vector form, of Basis Tasks;
b
i
(
i
)(b
i
∈B) the i-th basis task with the list of arguments
it accepts, 
i
;
n
b
the number of basis tasks in B;
L the arguments accepted by basis tasks in B;
T
i
the i-th macro task;
L
i
the physical location associated with T
i
;
φ
i
the number of robots required by T
i

;
T
i,k
the k-th sub-macro task of T
i
;
T
i
the Task Specification Matrix of T
i
;
vi
T
i,dp
the set of tasks that must be completed before T
i
can start;
|T
i
| the number of sub-macro tasks in T
i
;
|T
i,j
| the number of basis tasks in T
i,j
;
T
p
the ordered set of high priority tasks;

T
atv
the ordered set of active, but not high priority tasks;
T
ach
the ordered set of archived tasks;
T
ls
the ordered set of all the tasks given to the robots;
n
ls
the number of tasks given to the robots;
n
atv
the number of tasks in a subnetwork that can be considered,
given the number of robots in that subnetwork, and the total
number of robots required for each of these tasks;
A
j
the vector containing the utility level r
j
has with each of
the n
b
basis tasks;
T
rj
the task r
j
is currently performing;

W the physical workspace of the team;
N the set of disjoint subspaces in W ;
n
sn
the number of disjoint networks in W ;
S
j
(Z
n
b
×1
) the Task Success Matrix of r
j
.
t
dd,j
the time at which task T
j
must be completed, the task due date;
t
sd,j
the earliest start time for task T
j
;
t
d,j
the approximate task duration for task T
j
;
A

j
the set of agents representing the virtual-agent equivalent of task T
j
;
a
jk
the agent associated with task T
j
on robot r
k
;
n
jr
the number of robots that can service a task T
j
;
RS
i
the Roam-Space on a robot r
i
;
RW the Roam-World;
I
jk
the area of influence of an agent a
jk
;
UC
i
the set of uncovered points in RS

i
;
F, F
N
the desired formation consisting of N robots;
Q the set of all the queues in a formation F;
G the set of all the formation vertices;
Q
j
the j-th queue in the set Q;
V
j
a list of (either one or two) formation vertices
that influences Q
j
;
S
j
the set of points describing the shape of Q
j
;
C
j
the capacity ∈ [0, 1] of Q
j
;
E
j
the encapsulating region of Q
j

;
O
j
the set of functions that describe the orientation of agents along Q
j
χ
i
(t) is the queue status of r
i
at t;
E
j
excess length (the number of excess robots in Q
j
);
q
i
,q
tg,i
,q
t
position of robot r
i
, its target, and team’s target respectively;
v
i
,v
tg,i
,v
t

velocity of robot r
i
, its target, and team’s target respectively;
κ
i
,κ
tg,i
,κ
t
(topside-vector) vector normal to the plane of robot r
i
, its target, and team’s target
respectively;
vii
q
(vj),i,nr
the vector from a robot r
i
at q
(vj),i
to the nearest point on its queue Q
j
at q
(vj),nr
;
x
(f),i,t
the relative position (x = q), velocity (x = v) or topside-vector (x = κ) of target
w.r.t. r
i

in frame f;

j,i,nr
the shortest distance between r
i
and a queue Q
j
;
N
tot
the number of robots currently in the team;
N
v
,N
q
the number of formation vertices and queues respectively;
ρ
sf
the safety distance between r
i
and an obstacle;
ρ
adp
the distance a deformed queue is from an obstacle;
ρ
0
the influence range of an obstacle;
R
max
the maximum range of r

i
’s range sensor;
R
act
(≤ R
max
), the active range of the Instant Goal behavior;
n
R,t
the unit vector of the ray from the range readings that is
closest to ˆq
(ri),i,t
;
(φ, θ) a pair representing the direction of an arbitrary point in the
frame of r
i
.
U
b
, F
b
the potential function and force derived for a behavior b;
F
b
the magnitude of the force F
b
;
a
b
the weighting parameter for a behavior b;

δ, δ
a
the average distance of robots from their queues and
deformed queues (if applicable) respectively;
c
ig,on
ON(1)/OFF(0) status of Instant Goal Behavior;
v
max
,ω
max
the maximum speed and turnrate of a robot respectively.
viii
Contents
Acknowledgments ii
Abstract iv
Nomenclature vi
Table of Contents xi
List of Figures xi
List of Tables xiv
1 Introduction 1
1.1 Motivation of Research: Multi-Agent Coordination . . . . . . . . . . . 1
1.2 Macro-Level Planning & Inter-Task Coordination . . . . . . . . . . . . 5
1.2.1 Task Representation & Short Term Allocation . . . . . . . . . . 6
1.2.2 Plan Representation & Long Term Allocation . . . . . . . . . . 8
1.3 Micro-Level Coordination . . . . . . . . . . . . . . . . . . . . . . . . 10
1.3.1 Formation Representation & Control . . . . . . . . . . . . . . . 10
1.4 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.5 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2 Task Representation and Short Term Allocation 17

2.1 The Cooperative BackOff Adaptive Scheme . . . . . . . . . . . . . . . 18
2.1.1 Disjoint Broadcast Networks . . . . . . . . . . . . . . . . . . . 19
2.1.2 Formal Description of Tasks . . . . . . . . . . . . . . . . . . . 20
2.1.3 Task Suitability Matrices (TSM) . . . . . . . . . . . . . . . . . 26
2.1.4 Fault Tolerance and Coping with Uncertain Task Specifications 28
2.1.5 Adaptation of Internal Robot Model . . . . . . . . . . . . . . . 30
2.2 Task Prioritization and Allocation . . . . . . . . . . . . . . . . . . . . 32
2.3 Analysis and Comparisons . . . . . . . . . . . . . . . . . . . . . . . . 38
2.3.1 Domain of Operation . . . . . . . . . . . . . . . . . . . . . . . 39
2.3.2 Communication Complexity . . . . . . . . . . . . . . . . . . . 41
2.3.3 Computation Complexity . . . . . . . . . . . . . . . . . . . . . 41
2.3.4 Quality of Solutions . . . . . . . . . . . . . . . . . . . . . . . 42
2.4 Simulation Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . 44
ix
Contents
2.4.1 Tasks and Mission Statement . . . . . . . . . . . . . . . . . . . 44
2.4.2 Tasks in Connected Communication Networks . . . . . . . . . 45
2.4.3 Tasks in Disjoint Communication Networks . . . . . . . . . . . 50
2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3 Plan Representation and Long Term Allocation 54
3.1 Decentralized Task Scheduling . . . . . . . . . . . . . . . . . . . . . . 55
3.1.1 Self-Organizing Schedules . . . . . . . . . . . . . . . . . . . . 55
3.2 Agent Dynamics and Behavior . . . . . . . . . . . . . . . . . . . . . . 60
3.3 Simulation Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.3.1 Convergence of Agents with Known Task Durations . . . . . . 65
3.3.2 Tasks with High Degree of Uncertainty . . . . . . . . . . . . . 67
3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4 Formation Representation with Q-Structures 70
4.1 Queues and Artificial Potential Trenches . . . . . . . . . . . . . . . . . 71
4.1.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

4.1.2 Formations and Queues . . . . . . . . . . . . . . . . . . . . . . 72
4.1.3 Changing Queues . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.1.4 Potential Trench Functions . . . . . . . . . . . . . . . . . . . . 77
4.2 Robot Behaviors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.2.1 Target Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.2.2 Instant Goal Behavior . . . . . . . . . . . . . . . . . . . . . . 82
4.2.3 Obstacle Avoidance . . . . . . . . . . . . . . . . . . . . . . . . 85
4.2.4 Overall Robot Behavior . . . . . . . . . . . . . . . . . . . . . 86
4.3 Analysis of Parameter Values . . . . . . . . . . . . . . . . . . . . . . . 86
4.4 Simulation Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . 89
4.4.1 Convergence to Formations and Scaling . . . . . . . . . . . . . 91
4.4.2 Maneuvers in Confined Spaces . . . . . . . . . . . . . . . . . . 93
4.4.3 Reaction of Formations to obstacles . . . . . . . . . . . . . . . 97
4.4.4 Disruption of Wireless Communications . . . . . . . . . . . . . 97
4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
5 Q-Structures and Formation Convergence with Limited Communication 103
5.1 Formation Representation and Dynamic Target Determination . . . . . 104
5.1.1 Division of Information Flow . . . . . . . . . . . . . . . . . . 105
5.1.2 Properties of the Q-structure . . . . . . . . . . . . . . . . . . . 106
5.1.3 Determination of Target on Queue . . . . . . . . . . . . . . . . 110
5.2 Navigation of Robots to Positions in Formations . . . . . . . . . . . . . 111
5.3 Simulation Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
5.3.1 Formation Convergence and Scaling . . . . . . . . . . . . . . . 121
5.3.2 Moving formations . . . . . . . . . . . . . . . . . . . . . . . . 124
5.3.3 Changing Formations . . . . . . . . . . . . . . . . . . . . . . . 124
5.3.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
5.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
x
Contents
6 Q-Structures and Formation Convergence with Motion Limitations 128

6.1 Q-Struture Representation . . . . . . . . . . . . . . . . . . . . . . . . 129
6.1.1 Incorporation of Orientation Information . . . . . . . . . . . . 129
6.2 Target Generation and Determination of Robot Behavior . . . . . . . . 131
6.2.1 Generation of Target-on-Queue with Limited Communication
Ranges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
6.2.2 Bobber-Agents . . . . . . . . . . . . . . . . . . . . . . . . . . 136
6.2.3 Convergence of Bobber-Agents towards Minimum Point in the
Cast-Zone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
6.2.4 Generation of Desired Trajectories for Robots . . . . . . . . . . 141
6.2.5 Overall Robot Target and Behavior Generation . . . . . . . . . 153
6.3 Simulation Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
6.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
7 Conclusions and Recommendations 161
7.1 Summary and Contributions . . . . . . . . . . . . . . . . . . . . . . . 161
7.2 Suggestions for Future Work . . . . . . . . . . . . . . . . . . . . . . . 164
Bibliography 167
A Author’s Publications 175
xi
List of Figures
1.1 The three layers of robot decision making according to information
processing requirements. The relationship of the Macro- and Micro-
decision making levels for multi-robot teams with these three layers is
also shown. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Different Levels of Robot Coordination. The coordination mechanisms
at each level may either be centralized or decentralized. . . . . . . . . . 5
2.1 Existence of subnetworks in workspace due to deep fading and signal
attenuation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2 Elements of each robot’s Internal Models. . . . . . . . . . . . . . . . . 21
2.3 Components and Phases of the COBOs. . . . . . . . . . . . . . . . . . 29
2.4 Reformulation of the SPP . . . . . . . . . . . . . . . . . . . . . . . . . 35

2.5 Closed room (10m ×10m) with a single communication network. . . . 46
2.6 Activity Charts of the robots in the presence of a connected network. . . 48
2.7 Evolution of Suitability of r
1
with the blue find attach(·) and red find attach(·)
basis tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.8 Robot allocation per task, and robot suitability for basis tasks at the end
of mission, for domain with only one network. . . . . . . . . . . . . . . 49
2.9 Activity Charts of the robots with one connected network and no un-
certain task specifications. . . . . . . . . . . . . . . . . . . . . . . . . 50
2.10 Activity Charts of the robots in the presence of multiple subnetworks.
The small kinks in the graphs, characterized by slightly raised lines,
indicate that the robot is in recruitment phase for the associated task. . . 52
2.11 Robot allocation per task, and robot suitability for basis tasks at the end
of mission, for domain with multiple subnetworks. . . . . . . . . . . . 53
3.1 Elements concerning an agent, in a Roam-Space with two agents a
jk
and a
j
1
k
1
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.2 The Dandelion Formation and connected graph for agents within and
between six Roam-Spaces. . . . . . . . . . . . . . . . . . . . . . . . . 59
3.3 Agent Clusters and Uncovered Spaces in a Roam-Space. . . . . . . . . 63
3.4 Gannt Chart and Convergence of Agents on Roam-Spaces when tasks
may end before the expected time or be prematurely removed from the
system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.5 Maximum completion date on each Roam Space . . . . . . . . . . . . . 68

xii
List of Figures
3.6 The number of tasks successfully completed. . . . . . . . . . . . . . . 69
4.1 Vectors q
i
, v
i
and κ
i
of r
i
in the world coordinate system . . . . . . . . 71
4.2 Examples of queues, and formation vertices (circles), where x
t
and y
t
are the axes of the coordinate frame of the target centered at V
1
. Open
queues are drawn with solid (and dashed) lines, indicating that they
extend indefinitely from the vertex. . . . . . . . . . . . . . . . . . . . . 75
4.3 Forces acting on a robot (r
i
) when it enters a queue. A robot is attracted
to the point on Q
j
(at q
(vj),nr
) that is nearest to it. . . . . . . . . . . . . 78
4.4 Instead of the original queue (that passes through the obstacle), the pres-

ence of the obstacle causes the robots (triangles) to be attracted to the
deformed queue that hugs the obstacle at a distance of ρ
adp
. . . . . . . . 81
4.5 3D view of the potential trench function of Q
4
in the (x,y)-coordinate
space of the vertex, V
3
. . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.6 Representation of the direction (n
q,i,a
) of a point in the coordinate frame
of r
i
by the pair (φ
a
,θ
a
). . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.7 Attractive & Repulsive forces on an robot joining an established queue . 89
4.8 Convergence of team to desired formation. Solid: Wedge, Dashed: Col-
umn, Dash-dot: Double Column, Dotted: Circle . . . . . . . . . . . . . 92
4.9 Scaling of formations. Solid: Wedge, Dashed: Column, Dash-dot:
Double Column, Dotted: Circle . . . . . . . . . . . . . . . . . . . . . . 92
4.10 Snapshots of the team of nine robots forming the wedge formation. . . . 93
4.11 Snapshot of corridor and waypoints . . . . . . . . . . . . . . . . . . . 94
4.12 Team Maneuver through a confined corridor Solid: δ, Dashed: δ
a
. . . . 95

4.13 Formation deformation during a turn . . . . . . . . . . . . . . . . . . . 95
4.14 Team Maneuver through a confined corridor. Solid: δ, Dashed: δ
a
. . . . 96
4.15 Snapshot of wedge deformed into a column in a narrow corridor . . . . 96
4.16 Snapshot of environment with Type I and II obstacles . . . . . . . . . . 98
4.17 Plot of δ vs time for team traversal through obstacle fields . . . . . . . . 98
4.18 Plot of δ vs time for disruption to a maximum of half the communica-
tions links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
4.19 Queue status of the robots for I
loss
=0.50, P
txloss
=0.05 . . . . . . . . 101
4.20 Effect of different degrees of communication breakdown on the forma-
tion. Circle (•): I
loss
=1.00, Cross (×): I
loss
=0.50, Triangle ():
I
loss
=0.25 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.1 Graphical Representation of Q-structures. Dotted circles represent vir-
tual vertices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
5.2 The triangular formation represented using connectivity graphs . . . . . 108
5.3 Robot convergence to formation with robot deactivation/removal at t =
10s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5.4 Robot Separation and Control Forces. . . . . . . . . . . . . . . . . . . 122
5.5 Distance of robots from related queue’s encapsulating area. . . . . . . . 123

5.6 Formation convergence with a moving target. . . . . . . . . . . . . . . 124
5.7 Robot Separation and Control Forces. . . . . . . . . . . . . . . . . . . 125
xiii
List of Figures
5.8 Robot convergence to formation with formation switching. Wedge: t =
[0s, 15s), Column: t = [15s, 30s), Line: t = [30s, 45s) . . . . . . . . . 125
5.9 Robot Separation and Control Forces. . . . . . . . . . . . . . . . . . . 126
6.1 Examples of queues, and formation vertices (V
1
to V
5
) in 3-D space. . . 131
6.2 Robots are attracted to their associated queues based on the potential
trenches in the 3-dimensional space. The potential trenches experienced
by each robot, given their positions, are also shown. . . . . . . . . . . . 132
6.3 Decision Flow within the system, as well as the different levels of com-
munications and decision making. . . . . . . . . . . . . . . . . . . . . 133
6.4 Cast-zone of a robot located at q
i
. . . . . . . . . . . . . . . . . . . . . 137
6.5 The attractive potential (U
tcast,i
) in the cast-zone. . . . . . . . . . . . . 141
6.6 Constellation agents around a robot . . . . . . . . . . . . . . . . . . . 144
6.7 Convergence of formation with 9 robots . . . . . . . . . . . . . . . . . 156
6.8 Inter-robot collision avoidance and robot directional changes . . . . . . 156
6.9 Convergence of formation with 6 robots changing between formations . 157
6.10 Inter-robot separation as the team changes between formations. . . . . . 157
6.11 Convergence of formation with 6 robots . . . . . . . . . . . . . . . . . 158
6.12 Inter-robot collision avoidance and robot directional changes . . . . . . 158

6.13 Convergence of formation with 6 robots through an obstacle field . . . . 159
6.14 Distance between different robots, and the distance between robots and
obstacles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
xiv
List of Tables
2.1 Comparison between different Task Allocation Architectures. For sim-
plicity, r = n
r
and n = n
ls
for this table. The figures pertaining to
ALLIANCE, BLE, M+, and Dynamic Role Assignment are based on
the work presented in [1]. . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.2 Robot Capabilities and suitability levels for basis tasks and Tasks. B:
Blue, R: Red. Tasks are prioritized using their subscript. (Note: Infor-
mation regarding Task 4 is presented here for convenience. It is, in fact,
introduced only at runtime as a new task.) . . . . . . . . . . . . . . . . 46
3.1 Robot Capabilities and Task Requirements/ Specifications. (R: Red, G:
Green, B: Blue) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.1 Parameter Values for simulations . . . . . . . . . . . . . . . . . . . . . 90
xv
Chapter 1
Introduction
This chapter presents a broad overview of the motivation and background for carrying
out the research work on collaborative mobile robot teams that is presented in this thesis.
The research objectives and scope of this research as well as the outline of this thesis is
also presented.
1.1 Motivation of Research: Multi-Agent Coordination
Multi-Agent Systems, consisting of huge numbers of interacting agents, linked to-
gether in complex networks, are becoming increasingly prevalent in the everyday con-

text. Such agents manifest themselves in the form of virtual robots, operating factory
processes, humans and embodied mobile robots. The advancement of technology in
robotics, control, computer science and communications, has made possible the deploy-
ment of large teams of mobile robots in real life scenarios. These systems have been
applied to a wide variety of areas, from manufacturing and warehouse automation, con-
struction and shipping industries, to autonomous robot humanitarian demining, surveil-
lance and urban search-and-rescue. A comprehensive overview of issues in multi-robot
cooperation may be found in [2].
In light of the growing autonomy of singular mobile robots and the great potential of
collaborative teams, this thesis focuses on the efficient coordination of embodied robots,
with the main aim of improving the autonomy of robot teams operating in dynamic
1
1.1. Motivation of Research: Multi-Agent Coordination
and unknown environments. Embodied robot systems are significantly different from
typical multi-agent technologies employed in Distributed Artificial Intelligence (DAI)
approaches [3] that considers software agents.
One major difference is that embodied robots respond and interact directly with
other entities in the environment. Virtual agents, on the other hand, perform tasks that
are mostly informational [4]. Furthermore, the constraints faced by software agents are
different from those faced by mobile robots, which also influences the type of solution
techniques available for each type of agents (virtual or embodied). For instance, the
range of capabilities each robot may be equipped with correspondingly affects the set
of feasible coordination mechanisms. In addition, disembodied agents are given the
opportunity to modify their intrinsic capabilities and even replicate themselves to cope
with the uncertain virtual environment in which they exist, while embodied robots have
to make the most out of whatever equipment each possesses and react in appropriate
ways to cope with malfunctions and highly uncertain sensor readings.
It is highly impractical, and even impossible, to require a single robot to be equipped
with a myriad of capabilities that can handle every possible scenario that may arise.
Moreover, such systems are extremely prone to malfunctions that cripple and jeopar-

dize the success of robot missions. The use of multi-robot teams removes the need for
every robot to be capable of performing complex missions single-handedly. Instead,
individuals cooperate with each other, volunteer their expertise for tasks, and utilize the
collective capabilities of the team to accomplish missions. This greatly improves the
robustness of robots against malfunctions.
However, the use of multiple robots for missions is not without problems. Espe-
cially for mobile robots operating in unknown and highly dynamic environments, it is
important for the team to be able to react suitably, given their limited capabilities, to
fully utilize expertise afforded by individuals, to ensure that the mission may be accom-
plished as much as possible.
The decision making process of each autonomous robot can generally be divided
into three interacting layers. According to the work in [5], the these are: (i) the Reactive,
(ii) the Routine, and (iii) the Reflective, layers shown in the right side of Fig. 1.1. As
2
1.1. Motivation of Research: Multi-Agent Coordination
the name implies, the reactive layer consists solely of behaviors that can be further
generalized into attractive and repulsive forces. This is the most primitive layer, where
the amount of information processing is the bare minimal. On a higher level, is the
routine layer, where case-based reasoning occurs. Classical conditioning also results in
the generation of behaviors in this layer, where responses are more “automated” (and
yet, not as primitive as the reactive layer behaviors). The largest amount of information
processing occurs on the reflective layer, where counter-factual reasoning and high level
deliberation can occur. The layer does not directly influence the robot’s actions, but
instead produces biases to the lower levels that in turn, affects the resultant actions.
Figure 1.1: The three layers of robot decision making according to information process-
ing requirements. The relationship of the Macro- and Micro-decision making levels for
multi-robot teams with these three layers is also shown.
Specifically, this thesis examines decision making within the routine and reactive
layers, and further sub-divides these layers into the macro- and micro-decision levels to
reflect the degree of coordination required for different processes within a multi-agent

team. The relationship of these two forms of multi-robot decision making with Ortony’s
three layers are also shown in Fig. 1.1. The two forms of multi-robot decision making
are further described as follows.
(i) Mission level planning and inter-task coordination. This involves planning the job
3
1.1. Motivation of Research: Multi-Agent Coordination
assignments to each robot (or robot sub-team), taking into account their capabili-
ties and the requirements of each job. Such mechanisms may be centralized and
performed by a single leader robot, or decentralized, in which robots reach a suit-
able arrangement through a series of observations and self-organizing inter-robot
negotiations. The allocation and planning mechanism acts as a top level man-
agement scheme that should manage the available resources to ensure mission
accomplishment. As mentioned earlier, it is important for such a mechanism to
be fault tolerant and able to cope suitably well with changes to team composition
and any uncertainties that may be present in the system.
(ii) Coordination for single task accomplishment. The resource management schemes
typically do not specify exact behavioral and interaction rules for individual robots.
These are lower level coordination mechanisms that explicitly plans the actions of
individual robots in relation to those of others. After going through mission level
planning and arriving at the actual task(s) to perform, a sub-team of robots need
to plan their motions and paths (thus coordinating amongst the members of the
sub-team) to ensure that the task can be completed. For instance, a transportation
task may require the sub-team to encircle the object and move in a tight formation
between two points in space. As with higher level planning, the issues of fault
tolerance and adaptability resurfaces, and must be considered.
Coordination mechanisms on both levels mentioned above are critical for the reli-
able operation of autonomous mobile robot teams. The relationships between them are
shown in Fig. 1.2. In this research, coordination mechanisms on both levels of planning
will be presented in detail, with the overarching theme of increasing the robustness of
robot teams in uncertain environments and allowing them to produce desirable results

even in the face of adversity, on both the macro and micro levels. The following sec-
tions will present the background literature and introduction to the main chapters of this
thesis.
4
1.2. Macro-Level Planning & Inter-Task Coordination
Macro Level Coordination
Mechanism

Task Dissemination and Allocation
Formation of Sub-Teams
Mission/ Task Specifications
Robot Team
Micro
Level
Coordination
Action Coordination
Robot Sub-Team
Micro
Level
Coordination
Action Coordination
Robot Sub-Team
Micro
Level
Coordination
Action Coordination
Robot Sub-Team
…
Individual Robot
Actions

Individual Robot
Actions
Individual Robot
Actions
Perceptions and
Observations
Figure 1.2: Different Levels of Robot Coordination. The coordination mechanisms at
each level may either be centralized or decentralized.
1.2 Macro-Level Planning & Inter-Task Coordination
The macro-level of the decision making process deals with the coordination of plans
within the agent team such that the overall mission objective may be met. This can
further be subdivided into short term, instantaneous allocation and longer term plan
generation. The difference, and when to use a particular plan coordination mechanism,
lies in the amount of information – for instance regarding the mission, tasks, other
members of the team – available to each agent. In highly dynamic environments, such
as where information regarding each task is incomplete or insufficient to derive accurate
estimates of quantities such as task durations, the team is only able to perform alloca-
tions instantaneously based on information it has at each time. On the other hand, when
the team has sufficient information regarding their mission – when future tasks are ex-
pected to occur, the duration of tasks, the lifespan of each team member – allocation
efficiency can be increased through planning over a extended time scale.
5
1.2. Macro-Level Planning & Inter-Task Coordination
1.2.1 Task Representation & Short Term Allocation
Much work has been done in recent years with regards to the coordination of robot
teams. Coordination architectures such as ALLIANCE [6], MURDOCH [4] and, more
recently, BOAs/COBOS [7, 8], with a focus on instantaneous allocations, have been
proposed for cooperating mobile robots in unstructured environments. The use of such
multi-robot teams removes the need for every robot to be capable of performing com-
plex missions single-handedly. Instead, individuals cooperate with each other, volunteer

their expertise for tasks, and utilize the collective capabilities of the team to accomplish
missions set in highly dynamic environments where information may be incomplete or
uncertain.
When robot teams operate in unknown environments (e.g. exploring/surveying an
unknown territory), it is difficult to predict all the challenges and tasks that robot teams
may face. A number of schemes have been proposed to deal with this problem. A well
known fault tolerant architecture is ALLIANCE [6], which integrates impatience and
acquiescence into each robot. The L-ALLIANCE [9] extends ALLIANCE by adding
parameter adaptivity into the architecture. Another approach is the Broadcast of Local
Eligibility (BLE) technique [10]. It uses cross inhibition of behaviors between robots in
a team, based on a calculated task eligibility measure that robots compute individually
and broadcast to the team. A decision theoretic technique based on Markov decision
processes is used for collaboration in persistent teams [11], of which embodied multi-
robot teams considered here are a part of. A task assignment architecture that uses
task templates and the prioritization of task instances in a task assignment planner had
also been proposed [12] for transportation applications in unknown, but static, environ-
ments. The M+ protocol [13] has a task allocation layer, and the negotiation process
used is based on the Contract Net Protocol. These schemes typically consider robot
operation in ST-SR-IA
1
domains. As opposed to SR-tasks, MR-tasks require more than
one robot to perform, and where the tasks cannot be decomposed into independent SR-
1
For convenience, we shall use the notations proposed in [1] (ST-MR-IA: Single-Task robots, Multi-
Robot tasks, Instantaneous Assignment systems; SR: Single-Robot tasks; TA: Time-extended Assign-
ment).
6
1.2. Macro-Level Planning & Inter-Task Coordination
tasks. Coalitions may be formed [14] for such situations. However, uncertain (and/or
incomplete) task specifications have not been considered although it greatly impacts al-

locations, especially in ST-MR-IA domains where the evaluation of team effectiveness
is impaired. This problem is especially true in environments where domain knowledge
is incomplete. Consider the task of clearing an obstructing rock detected during a mis-
sion. It is difficult for a remote user to determine, using only sensor information (e.g.
video images), the exact weight of the rock, and the number of robots required, which
may vary greatly across different teams.
Auctioning schemes, utilizing explicit communications have been used in various
forms in allocation schemes. Robust multi-robot cooperation may be achieved through
the use of market-based approaches [15]. Similarly, ‘Hoplites’ [16] have been proposed
for robots to execute tightly coupled tasks. Market-driven software agents [17] are also
used in electronic negotiations, and which is able to adapt their strategies depending on
the prevailing conditions. A dynamic role assignment scheme that represents roles as
control modes in hybrid automatons has also been proposed [18, 19]. The MURDOCH
scheme [4] also uses an auctioning approach with a publish/subscribe communication
model. A role assignment strategy based on multi-threaded computer programming was
used to resolve any risks and conflicts that may arise during dynamic role swapping
[20]. In cases where communication losses are considered (e.g. in MURDOCH [4]), it
is often assumed that persistent communication losses imply the failure of a robot, and
another robot will be activated to take over the task. This does not account for the fact
that the robot may still be performing the task adequately. In the absence of team-wide
communications, robots should be able to respond appropriately to achieve suitable
allocations. For instance, if a team of robot is deployed to survey separate regions, due
to security and practical issues, robots in one region would not be able to communicate
with those in other regions.
While a number of papers (such as [4, 21,22]) have considered the issue of adapting
to robot malfunctions, there is very little attempt to use a robot’s history of failures
in completing certain tasks to assess the operability of its onboard capabilities (and
to determine which capability is the most likely cause of failures), and to make use
7
1.2. Macro-Level Planning & Inter-Task Coordination

of these information for making future decisions. Although the work in [22] is able
to detect and adapt to partial robot malfunctions, it does not attempt to identify the
malfunctioning resource unless the malfunction can be directly assessed based on a
loss of access to the resource. This may not be applicable in certain scenarios, such as
when a resource only suffers from diminished capability (e.g. wear and tear of hardware
components on a gripper that reduces its ability to carry heavy loads) and the robot still
has continued access to the resource. Therefore, with regards to instantaneous planning
and task allocations, the work presented in this thesis will focus on:
(i) Improving the autonomy of robot teams by having robots decide independently
on a suitable allocation under uncertain operating conditions and with graded
variations in competencies between individuals.
(ii) Cooperation between robots in different spatial domains with disconnected broad-
cast networks, in addition to uncertain task specifications.
(iii) Improving the team’s robustness via the automatic identification of problematic
resources, and a review of an individual’s task achievement history, to adapt the
task allocation on-the-fly.
1.2.2 Plan Representation & Long Term Allocation
As mentioned in the previous section, a large amount of work has focused on instanta-
neous task allocation in embodied multi-robot teams used for missions such as recon-
naissance, surveillance, disaster search and rescue and toxic waste disposal. However,
in many cases, estimated values of task durations and earliest start times are available
as rough guides for allocation schemes. Thus, longer term scheduling of tasks can be
beneficial.
(Re)Scheduling problems have been typically examined in the domain of operations
research. In a more structured environment where conditions are less dynamic, by tak-
ing certain time constraints into consideration, scheduling of allocations and actions
may be performed. The availability of such information, however, does not diminish
8
1.2. Macro-Level Planning & Inter-Task Coordination
the need for systems to be fault tolerant, since unexpected events do occur on a regu-

lar basis. When this happens, scheduling systems must be able to react appropriately
and perform re-scheduling. The application of different scheduling approaches to the
scheduling problem in supply chains is investigated in [23]. In addition to obtaining a
schedule before commencement of an operation, schedule repairs and rescheduling may
be required for the system to cope with changes in operating conditions (e.g. demand
changes). This problem has been partly addressed in the work by Jenson [24], which
focuses on choosing a schedule based on a robustness measure. The GA has also been
used to evolve a population of agents based on an artificial immune system [25]. Sched-
ules may also be modified according to the algorithm proposed in [26] that is based on
an Activity-on-Node network flow model. The Polite Rescheduler (PRIAM) has been
proposed in [27] to minimize the propagation of disruptions through the network of
interconnected manufacturing cells. A rescheduling mechanism (applied to the prob-
lem of system maintenance) that takes into account the communication times between
different decision centers is proposed in [28]. The efficiency of rescheduling processes
is studied in [29], in which the Segment-Based Reactive Rescheduling (SBR) approach
was proposed, for structured environments where the user has a more or less accu-
rate description of processes and conditions. The Self-Adjusting Dynamic Scheduling
(SADS) technique is presented in [30] for the scheduling of parallel processors and
with one processor dedicated to scheduling. Similarly, a process that transits between
execution and reconciliation phases for dynamic rescheduling is proposed in [31]. A
Holonic architecture is proposed in [32] and a negotiation protocol, based on the con-
cepts in the Contract Net Protocol, is used between various components of the system
for the allocation tasks to resource holons. Viera et al. [33] provided a survey of several
existing approaches and issues involved in rescheduling, and proposed a general frame-
work within which rescheduling may be studied. These approaches, however, typically
involve large scale, long term (re)planning procedures which are computationally inten-
sive. These methods are therefore not directly applicable to autonomous mobile robot
teams which require fast reaction times when operating in uncertain environments, and
where the planning horizon is significantly shorter than that encountered in large scale
9

1.3. Micro-Level Coordination
logistics, manufacturing or supply chain scenarios. Nonetheless, the ability to utilize
task information can significantly improve the current allocation derived from instan-
taneous allocation mechanisms, by allowing robots to make better plans (e.g., motion
paths). In light of this, the thesis will consider:
(i) Single-Task Robots, Single-Robot Tasks, Time-Extended Assignment (ST-SR-
TA) [1], with hard earliest start time constraints and relaxed due date constraints.
For example, if a robot’s current task is nearing completion, it can perform a pre-
liminary evaluation of the required capabilities for the next task to detect possible
hardware failures
2
.
(ii) A flexible plan representation method and self-organizing framework that allows
fast and stable generation of feasible plans over time.
1.3 Micro-Level Coordination
Coordination within agent teams do not stop at the more abstract task and mission plan-
ning stage. For successful implementation of tasks that have been assigned to individu-
als or sub-teams, there is a need for additional, finer coordination, at the level of agent
actions. This often involves the explicit collaboration of movements and positioning of
individual robots for specific applications such as surveillance, scouting and coverage.
With respect to micro-level coordination, this thesis therefore focuses upon multi-robot
formation control as a representative domain within which lower level motion/behavior
coordination within an agent team can be examined.
1.3.1 Formation Representation & Control
Robust multi-robot formations in dynamic and uncertain environments has been inten-
sively studied in recent years. For instance, the approach adopted by Olfati-Saber et
2
Task scheduling can be achieved by using an Earliest Deadline (Start time) First heuristic, followed
by right-shifts to ensure that the earliest start time constraints are satisfied (e.g. the technique described
in [34]). However, this can create unnecessary idle times between tasks, which should be reduced with

additional mechanisms.
10