Robocup

1
1. INTRODUCTION
Robocup is short for Robots Soccer Cup is an international scientific initiative
invented in 1993 with the idea of robots playing soccer. After several years of
preparation, the first event was held in 1997 for both real and simulation robots with the
participation of over 40 teams. So far, there have been more than 70 teams coming from
more than 25 countries competing in this annually international cup. The aim of Robocup
is to promote robotics and artificial intelligence research under challenging and adverse
conditions.
FIGURE 1 ROBOCUP LOGO
1
Robocup provides a standard problem for testing, evaluating theories and
algorithms by offering challenging problem in which various of researching areas have
to deal with such as strategy acquisition, machine learning, real-time recognition,
planning, and reasoning, reasoning and action in dynamics environment, multi-agent
systems, context recognition, vision, strategic decision-making. One the most challenging
issues in Robocup is the integration those technologies into one system for achieving a
specific goal
1
. Variety of difficult, but interesting problems make Robocup attractive to
researchers. Many academic institutions have been taking Robocup a vehicle to advance
the state of the art of intelligent robots
All of our team members are interested in Robocup, we are on the way to build a
Robocup simulation team. The team is based on agent2D – the base code of HELIOS
team from Japan. Agent2D has been used for qualifying teams which want to compete in
annual Robocup competition event. From the base code, by improving both offensive and
defensive strategies we have made significant enhancement in team performance. For
1 www.robocup.org
2
offensive strategy, we did improve low both level skills such as through pass, scoring
skills and high level skills. For defensive strategy, we made change on the way fullback

players marking. We are now can defeat the base team. 7/11 teams in 2011 world final
event cannot win over base team for in qualification round.
This paper consists of six chapters. The first chapter is dedicated for the
introduction about Robocup and our work. The second chapter will talk about a Robocup
2D simulation, one of the oldest sub-leagues in Robocup. In chapter 3, we will review
several work and research on Robocup 2D Simulation. In the fourth chapter, the
improvement we have made on the base code will be discussed in detailed. The next
chapter is the evaluation of our work. And in the last chapter, we will discuss our work
and the future improvement for our team.
2. ROBOCUP 2D SIMULATION
In this chapter, we will introduce about Robocup Soccer 2D simulation
2
, its
architecture, the game environment as well as the players or the agents
3
.
2.1. Architecture
The architecture of Robocup 2D simulation consists of four main parts. The first
one is server which is in charge of controlling the game, updating information to players.
The second one is agents which are independent connect to server to receiver information
about the game. Agents are independent i.e. no share memory and players do not
permission to access to other players to get information. The third one is monitor that is
used to visualize the game for audience to see what is happening in the game and the last
part is coach. Coach is simulation of coach manager in real soccer game who can make
substitution, change strategy in game
2.2. Environment
In 2D simulation league, server simulates the real world soccer environment in 2
dimensions i.e. no highball, the ball always runs on ground. Agents play on a plane with
2D Cartesian coordinate which represents the soccer field. Each position in the soccer
field is characterized by two real number (x,y). x belong to the range [-54,54], y belongs

3
to the range [-32,32]. Timing system is different from the real world. A match kickoff
fulltime is 6000 cycles (approximate 6 minutes), each half lasts for 3000 cycle. The clock
stops when the ball is dead, therefore, no extra time in a match. Golden goal policy is
applied in case two teams draw in the 2 halves of the game.
The environment is noisy because actions are performed might be a little different
from intention. For example, a player wants to kick the ball to the position(x,y), the ball
may run to the position (x+dx, y+dy) with dx, dy added by server. All noises are
simulated in server. Another characteristic of the environment is partly-observed. An
agent can only see a specific region depends on the position on where he stands. Besides,
an agent sees objects that are far from less exact than the nearby objects.
2.3. Agents
Agents are computer software. They play roles as players on a match. Whenever
an agent wants to do an action (for example, move, kick, tackle…) he sends the command
to the server. The server updates the game environment and sends back to every agent the
current state of the match. Agents are characterized by many parameters such as running
speed, acceleration, dash rate, kick margin, stamina…There are total of 18 types of
agents, they are different in the value of those above mentioned parameters.
Every agent has the below primitive skills:
- Turn neck: Turn neck to a direction
- Turn body: Turn body to a direction
- Kick: Kick the ball to a direction
- Tackle: Try to get the ball from opponent
From those primitive skills, many more complex and advanced skills are build
based on those primitive skills such as: Dribble, direct pass, through pass, shoot.
The soccer server supports agents with some built-in models. Sensor model
enables agents to hear, to see and to get information about them. Movement model helps
them to move on the soccer field. Action Model which give agents abilities to catch the
ball (goalie only), to dash and to kick the ball.
.3.1 Sensor Model

A robocup agent has three different sensors, namely the aural, the visual and the body
sensor.
4
2.3.2.1. Aural Sensor Model
Aural sensor messages are sent when a client or a coach sends a say command. All
messages are received immediately.
The server parameters that affect the aural sensor are described in the following table:
Figure ABC

* Range of communication:
A message said by a player is transmitted only to players within audio_cut_dist meters
from that player. Messages from the referee can be heard by all players.
2.3.2.2. Visual Sensor Model
The visual sensor reports the object currently seen by the player. The information is
automatically sent to the player every sense_step, currently 150, milli-seconds.
Range of view
The server parameters sense_step and visible_angle determine the basic time step
between visual information and how many degrees the player’s normal view cone is.
The player can also influence the frequency and quality of the information by changing
ViewWidth and ViewQuality.
To calculate the view_frequency and view_angle of the agent use these equations:
view_frequency = sense_step * view_quality_factor * view_width_factor
where view_quality_factor is 1 iff ViewQuality is high and 0.5 iff ViewQuality is low;
view_width_factor is 2 iff ViewWidth is narrow, 1 iff ViewWidth is normal, and 0.5 iff
ViewWidth is wide.
view_angle = visible_angle * view_width_factor
5
where view_width_factor is 0.5 iff ViewWidth is narrow, 1 iff ViewWidth is normal, and
2 iff ViewWidth is wide.
2.3.2.3. Visual Sensor Noise Model

In order to introduce noise in the visual sensor data the values sent from the server is
quantized. This means that player can not know the exact positions of very far objects.
Body Sensor Model
The body sensor reports the current “physical” status of the player. The information is
automaticall sent to the player every sense_body_step, currently 100, milli-seconds.
.3.1 Movement Model
In each simulation step, movement of each object is calculated as following
manner:

where , and are respectively position and velocity of the object in
timestep t.
2.3.2.1. Movement Noise Model
In order to reflect unexpected movements of objects in real world, server adds
noise to the movements of objects and parameters of commands
2.3.2.2. Collision Model
If at the end of the simulation cycle, two objects overlap, then the objects are
moved back until they do not overlap. Then the velocities are multiplied by -0.1. Note
that it is possible for the ball to go through a player as long as the ball and the player
never overlap at the end of the cycle.
.3.1 Action Model
There are 7 types of action. They are:
6
Type of Action Description
Catch
The goalie is the only player with the ability to catch the ball.
The goalie can catch the ball if the ball is within the catchable
area and the goalie is inside the penalty area
Dash
This command is used to accelerate the player in direction of its
body. Dash take the acceleration power as the parameter.

Kick
Kick command takes 2 parameters, the kick power and the angle
the player kicks the ball to. Once the kick command arrived at
the server, the kick will be executed if the ball is kick-able for
the player and the player is not offside. The ball is kickable for
the player if the distance between the player and the ball is
between 0 and kickable_margin.
Move
Move command is used to place a player directly onto a desired
position on the field, it does not work during normal play and is
available at the beginning of each half and after the goal has been
scored. The second purpose of move command is to move the
goalie within the penalty area after the goalie caught the ball.
Say Used by players to broadcast messages to other players.
Turn
Used to change the players body direction. The argument for the
turn command is the moment. If the player does not move, the
moment is equal to the angle the player will turn.
TurnNeck
With turn_neck, a player can turn its neck somewhat
independently of its body. Turn_neck command can be executed
during the same cycle as turn, dash and kick commands.
3. RELATED WORK
3.1. Literature Review
Researching on Robocup 2D simulation is now a very active area in despite of its
challenges. More and more universities and institutions are conducting research on this
branch and strengthen their teams for compete in Robocup 2D. Japanese team HELIOS
the champion of 2010 competition is developed by Fukuoka University, National Institute
of Advanced Industrial Science and Technology (AIST) and Osaka Prefecture University.
The champion of 2011 – WriteEagle

4
team is owned by University of Science and
7
Technology of China. Other academic institutes that are worth mentioned for their high
ranks in the competitions and contribution for Robocup Community are University of
Freiburg from Germany, University of Oxford from England, and University of
Amsterdam from the Netherlands. Amongst the team participated in the annual event
competitions, there is some develop their own team from scratch, however, many of those
make improvement base on the released resources of others’ team which is also called
base team. There are many ways to improve a base team, in the following; we will list
some of them that are related to our improvement. They are passing and defensive
strategies.
Passing plays an important role in Robocup 2D simulation. Whatever offense or
defense, passing has great impact on the team performance. For attacking strategies,
passing is the main mean of deliver ball to the area near to the opponent goal. Many
affiliations attempted for improving the passing strategy for applying in their teams.
Guangdong University of Technology tried to combine the advantages of several
machine learning methods (Nash-Q
5
, CE-Q and WoLF-PHC
6
) into a new reinforcement
learning
7
. This method allows ball holder to better choosing teammate to pass ball.
HELIOS 2010 used game tree searching for a chain of actions including passing,
dribbling and shooting. However, they stated in Team Description Paper that its
performance is weak because the uncertainty and partly-observed environment of
Robocup 2D simulation makes the evaluation and generation chain of actions get low
accuracy. Agent2D – HELIOS base code using hand-coded passing generator mixing of

direct pass, lead pass and through pass.
Beside offense, defense also plays a very important role in a match. If the defense
is not strong enough then however strong the offense is, the team could never win.
However, defense is not easy to be improved. In fact, while a lot of papers on RoboCup’s
robotic 2D soccer simulation have focused on the players’ offensive behavior, there are
only a few papers that specifically address a team’s defensive problems. In the past, there
were few papers dealt with defensive problem such as a paper of Thomas Gabel, Martin
Riedmiller and Florian Trost from Germany discussing about “Defense behavior in
Soccer Simulation 2D: The NeuroHassle Approach”
8
. Their method requires multiple
trainings against different training opponents to get better result.
We have briefly described others related work that we have known. In the next section,
we will describe agent2d which we use as a framework for our team and
8
3.2. Helios base code
Creating a Robocup 2d team from scratch takes much time and effort since there
are many matters that need to be considered, for example: connection between server and
agents, agents design… Therefore, many team releases their code without high-level
decision making section as a framework in order to help other teams. Those frameworks
are often called base code. In our team, we use the Helios team’s base code.
Today, Helios is one of the best teams in Robocup 2d simulation league. They are
the champion in Robocup 2010 and runner-up in 2009 and 2011. Their base code is
called agent2d
9
. Agent2d also comes with a library package which supports the latest
soccer server and provides plenty of helpful functions.
Although the high-level decision making has been removed almost entirely, they
are still pretty strong and in recent year, they have been used as a benchmark for teams
who want to participate in Robocup. As in 2011, 7 teams among 20 qualified teams could

not defeat the base team and 2 teams lost all matches against agent2d. Beside, agent2d is
coded in C++and supports the latest GNU C++ compiler which makes it very convenient
to use. Moreover, we have found out that they have a very well-designed structure and
they also provides necessary basic skills like pass, dribble, shoot, etc.
3.2.1. Formation
In agent2d, they have implemented a very fantastic formation for the team. The
formation is defined by some specifics ball position in the field. Agent’s position will be
interpolated with Delaunay triangulation from those specific positions. Helios also
provides a tool for editing team’s formation
10
.
3.2.2. Low-level skill
In this project, we use the name low-level skills to call skills which composed
mostly of a series of basic actions of an agent without any hard decision making
problems. For example, currently, dribble is called a basic skill because it is as simple as
kick the ball toward a position then dash along it. But if we decided to make a more
complex dribbling skill which involved trick the opponent, we may classify it as high-
level skill. This section will describe some basic information about low-level skills in
agent2d.
9
3.2.2.1. Passing skill
We have decided to rewrite the passing skill since agent2d’s passing skill doesn’t
meet our expectation. Our passing skill will be described in later section.
3.2.2.2. Dribbling skill
Dribbling skill is the skill of moving with the ball to a specific position without
losing the ball to the opponent. As I have mentioned above, dribbling skill mostly
consists of simple kick and dash sequences. They also have a function to verify whether
the agent can catch up with the ball before opponents and a function to estimate the
number of dash commands will be used.
3.2.2.3. Shooting skill

In order to shoot, the Helios base does the following steps:
- Step 1: Create and evaluate possible shoots
- Step 2: Execute the one with highest score
- Step 3: Create and evaluate possible shoots
Figure <?>: Shoot model
Dividing the opponent goal line to m points. The first point is the goal left column
and the last point is the goal right column. The distance between any pair of adjacent
points are equal. Only shoot to one of these points will be considered. Using opponent
goal keeper body angle, speed and acceleration, check whether the ball can be caught or
not. The ball can be caught if it is inside goal keeper CATCHABLE_AREA which is
defined by the simulation server. Any shoot which can be caught will be discarded. The
rest will be evaluated based on the ball’s travelling distance, speed and distance to the
goal keeper.
10
3.2.2.4. Others low level skills
Agent2d also provide many helpful skills like:
Intercept ball: this is the skill used to receive the ball from a pass. The agent will
use a table to keep track of ball positions and use that table to choose the best ball
positions to receive.
Hold ball: this skill is basically staying in the same position while trying to
maintain ball possession.
Tackling skill: this is the skill used to take away opponent ball possession.
However, there will be chance for the agent to be booked (i.e. receive a card).
Since those skills are currently not our interest, we haven’t explored them yet.
3.1.1 High Level strategy
Although have been almost entirely removed, the remaining high-level strategy of
agent2d is still pretty strong. The remaining is part of their chain-action which as
described in their TDP [?] consists of:
- Generate a chain of actions from basic action like dribble, pass, shoot
- Estimate the final state of the action chain

Their performance can be described as simple wing play strategy. The ball is
intensively passed to the wing forward and he will try to dribble along the touch line until
reaching the corner area. From that position, he will keep dribbling along the goal line
and find a way to cross the ball in or score directly. That simple strategy works
surprisingly well against agent2d itself. However, after inspecting the movement of their
defenders, we found out that their defenders practically do nothing. In fact, they just
follow the team formation.
4. IMPROVEMENT
In this chapter, we will described in detailed what we have done to achieve better
performance in compared with the agent2D. The overall actions and strategies of
Agent2D team can be described as below.
11
Figure ABC: Overall actions and strategies of Agent2D
For better our team performance, we have made improvement on the passing and
shooting of basic skills and changed high level for both offensive and defensive
strategies.
4.1. Passing improvement
4.1.1.3.1. Helios base code through pass limitation
The pass is performed with lack of communication of the pass taker and the pass
receiver. The say communication in base code is very limited. First, agents cannot
determine whether or not the incoming message is from opponents or teammates. Second,
if opponents say ceaselessly, agents will be not able to receiving any message due to
threshold hearing defined by the Robocup 2D simulation.
Through pass is generated different from lead pass and direct pass, however, they
use the same evaluation method. This leads to the domination of direct pass action in the
match. As a result, through pass is rarely perform even agents is in good position for
through pass.
12
The through pass target in Helios base team is choosing from the points which is
very near the pass receiver. In addition, in real match, fullback players often stand near

the attackers. Therefore, the intended pass receiver often cannot beat the defender for the
ball from through pass because the difference in distance between the defender and pass
target is relatively small.
4.1.2.3.1. Improvement idea
From the limitation we have inspected from the source code of Helios base team,
we decided to improve it to better our team performance. Our approach is mainly focus
on the communication between pass taker and pass receiver. In the Robocup 2D server
version 8.0, there is a useful action that an agent can employ to collaborate with his
teammates. It is “point to” action, i.e. point agent’s arm to a desired position. In the task
of improving through pass, we use the point to action as a signal for pass taker to pass to
a good position where the pass taker can easily beat opponent’s
defenders for the ball. In the figure ##. Player number 11 is pointing to a good position to
signal the ball holder for making through pass.
Figure ABC: Point to action
We propose the flow of through pass actions as follow:
Step 1: The intended pass receiver try to find a position where he thinks it is good,
i.e. he can beat opponent defenders for the ball in that position and after receiving ball
from that position and he can make danger to the opponent goal. The intended pass
receiver point to that position
Generate all posible target
posions
Verify target posions Evaluate target posion Get best target posion
13
Step 2: The ball holder, after detecting the signal from his teammate, will get the
intended pass receiver point to direction. Then, he checks the area along and near the
direction to find a best through pass route. The purpose of checking area near the
direction is the noise added by server. The noise consists of incorrect seeing the position
of teammate, incorrect getting the teammate point to direction. After finding the best
pass, he will make a through pass and, again, point to action is used for communication
between pass maker and desired receiver. The ball holder after making through pass will

point to a pre-defined position. This position is
Step 3: After receiving the signal from the ball holder, the intended pass receiver
chases the ball and gets it.
We also separate through pass from direct pass and set highest priority to through
pass when attacking. This is because the threats make by through pass to the opponent
goal if it is performed successfully.
4.1.3.3.1. Implementation
In the following part, we will briefly describe how through pass is implemented with
collaboration between teammates using point to actions. There are several actions which
are necessarily implemented: Through pass, point to target, and move to strategic position
for getting through pass.
4.1.3.1. General model for calculating through pass, point to target
Through pass and point to actions share the same characteristic of choosing best target
before performing. The differences are the primitive actions after finding the best target.
Four passing, it the primitive action is kick the ball to the target. For point to actions,
agents will point arm to the target. Because of those similarity those actions are
implemented as the following model
Figure ABC
14
Step 1: In this step, depends on the type of actions, agents will create all possible
targets for the actions.
Step 2: Base on specific type of actions to verify whether they could be the
suitable target. This verifying process strictly ensures that the target can be used to
without any threats from losing the ball to opponent or offside
Step 3: Use some pre-defined criteria to evaluate the target positions which passed
the verifying process. After evaluating process, each target position is assigned a number
representing its score. The higher score is the better.
Step 4: The final target is choosing for performing action is the one has highest
score overall.
4.1.3.2. Through pass:

Step 1: Generate all possible through pass target position:
Figure ABC: Getting best through pass target
The ball holder gets the teammate point to direction( OX in the figure), Due to the
noise added by server, candidate through pass targets is choosing from those point inside
a cone with the vertex is the ball pass receiver position, bisector line is OX and the angle
is 30 degree.
Step 2. Verify target position
15
After getting all through pass candidate target, we discard one to where opponent
can dash before intended pass receiver can. Candidate through pass target that the
difference between time for ball and pass receiver get to target is large than T cycles is
removed. Finally, for every opponent in the field, if he can cut the ball before reaching
targets, we also discard those candidate positions.
Step 3. Evaluate
There score of a candidate through pass target depends on the tuple (X,Y,Z,T,K) where:
X: The distance between nearest opponent to the candidate pass target
Y: The confidence of ball holder to positions in soccer field
Z: The confidence of ball holder to the passing angle
T: The safe coefficient of the space in front of the pass target
K: The difference between the y coordinates of nearest defender and candidate target.
The more X,Y,Z,T,K are, the higher score candidate target is
4.1.3.3. Point to
This skills enable receiver points to a good strategic position where he can receive
the ball(alert the ball carrier that the pointto direction is a good position for making
through pass)
Step 1. Generate all possible target position:
The process of this step is similar to step 1 of through pass. The only difference is
OX in this case is the line starting from agent’s position and is perpendicular with the
goal line of opponents.
Step 2: Verify target position

This step is exactly the same as verify a through pass candidate
Step 3. Evaluate
The score function is in the form of A*L + B*D + P where:
L: The pass length
D: The minimum distance between the pass line and an opponent
P: x coordinate ball holder – x coordinate of target.
A, B: pre-defined constants and both is greater than 1.0
16
4.1.3.4. Move:
In this skill, agents without ball have to go to a strategic position in order to
collaborate with his teammate. We use the tools from Helios robocup team called fedit to
define the positions for which agents have to get to. There are only several main strategic
positions are pre-defined in the formation file. When the agents in the situation which is
not exactly the same as in formation file, linear interpolation is used to calculate the
strategic move for agents.
The formation file defines agents strategic positions solely based on the ball
position; therefore, they are not good in all situations. In order to deal with this problem,
we use a refine positioning method to re-calculate target positions for agent. The
modification in the target positions are relied on the situation in which agents and his
teammate are. For example: if an agent without ball sees that his teammate – the ball
holder can make a through pass for him, he will try move to the position near the offside
line to be ready for dashing to get the ball.
4.2. Scoring skill improvement
4.1.4.3.1. Motivation
4.1.5.3.1. Improvement in scoring
Scoring skill is a high-level skill, which means it composed from several basic
skills like shooting skill, dribbling skill and passing skill. Because the Helios’s base code
doesn’t have a specific solution when there is chance to score, we have implemented our
own scoring skill. There are two situations which we considered scoring chances:
- When our striker has passed all opponent defenders.

- When our strikers are near the opponent goal but there are still opponent
defenders.
17
Figure<?>: A situation in which the forward has passed the defenders and facing goal
keeper
We use two different approaches for each case.
Case 1: When our striker has passed all opponent defenders.
This is an easy case, hence, we use a simple approach:
find_best_position() : find the best position to dribble to, based on:
- Distance to opponent goal center, the closer the better.
- The angle LBR as seen in figure <?>, the wider the better.
- Distance to opponent goal keeper and defenders. We must not dribble to too close
to the goal keeper and keep a safe distance <m> with opponent defenders behind.
dribble(): Helios base’s dribbling skill (Section <?>)
shoot(): Helios base’s shooting skill (Section <?>)
Figure <?>: find_best_position() model
18
Action involved: dribble, shoot.
Case 2: When our strikers are near the opponent goal:
In this case, approaching the goal is very difficult since dribbling pass defenders is
a hard task in robocup 2d simulation. Currently, our approach can be described as a
greedy approach, but we intends to improve it in near future.
can_shoot(): Helios base’s function, return true if there are chances to score.
shoot(): Helios base’s shooting skill (Section <?>).
find_teammate_has_best_position(): find the teammate has greatest chance to score. This
function is based on Helios base’s shoot evaluation function.
try_approach_goal_or_hold_ball(): similar to previous case. Try to approach the goal
while keeping a safe distance to opponent defenders and goal keeper.
Action involved: hold ball, dribble, pass, shoot
4.3. Defensive strategy improvement

4.1.1.3.1. Motivation
Defense plays a very important role in a match. However, the defense behavior of the
Helios base is weak, so it’s necessary to improve it.
4.3.2 Defense behavior limitation
These strategic positions are not good enough. In fact, they are calculated based
only on the position of the ball but without positions of opponents. This results in poor
performance of defenders against different opponent teams. It cannot differentiate
different situations: a through pass and a normal pass are the same. This kind of
misunderstanding is very dangerous because different situations need different actions.
Because position of each defender is calculated based on position of the ball so there was
no cooperation between defenders in a team. This leads to poor performance in defense.
19
4.3.3 Improvement in defense
In order to improve the defense, we decided to add some new strategies to
remediate the weaknesses of the Helios base.
First of all, we want to improve the defense against the through pass of the
opponents. Through pass is a forward pass through a gap in a defensive line that
coincides with a timed run from the receiver in order to receive the ball behind the
defense going to goal. When this pass is made to a fast forward, it often creates a 1v1
situation between the attacker and the goalkeeper. In football, a through pass if succeeds
is often very dangerous with the high opportunity of scoring.
Our solution for this problem is to find out when the opponent is going to make a
through pass then take appropriate actions. This result is one example:
Figure ABC
20
Figure ABC
Figure ABC
Secondly, from our observation, defenders of the Helios base are weak against the
attack coming from two corners. As we mentioned earlier, there was no connection
between teammates in the Helios base team so its defensive strategy is not efficient and

creates many gaps for the opponents to score. To deal with this problem, we suggest
applying 1v1 defending. We use one nearest agent to the opponent ball leading player to
21
hassle the ball and try to take possession of the ball. At the same time, other defenders
find the nearest opponents in the penalty area to defend.
5. EVALUATION
5.1. Over all evaluation
Combining our improvement in a team, we can defeat the base code 14/16 matches
and lost 2 matches. There was no draw match because we applied the rule of extra time
and golden goal if the result of two main halves is a draw.
In which:
5.1.1. Through pass
The table shows the result of our through pass improvement in 5 games against
Agent2D. We win 3/5 matches. The total number of through passes performed is 33 in
compare with 2 of the Agent2D. In those through pass, 4 of them are the assist passes.
5.1.2. Defensive strategies
In the first strategy, we consider a successful case as when the defender can detect
the right through pass and running to the good position to stop the opponent attacker from
passing through the defense.
In the second strategy, a successful case is when our defenders can stop the pass
from the ball leading player from two corners to the opponent in the penalty area.
Result:
22
- Before improving the defense: the average number of goal conceded is
approximately 4 to 5 against Helios 2010.
- After improving: this losing figure reduced to 0 to 1 goal per match.
5.2. Individual skills evaluation
5.2.1. Through pass
For the purpose of testing through pass skill, we setup a small environment where
agents are forced to make through pass in small period of times.

Environment
The number of defenders N in this situation is set to 2 or 3 as similar to real game.
The size of field in the environment is W x H (meter). The noise of the real environment
is kept unchanged. In this task, the term episode is employed to represent game time. One
episode last much less than in full game time. One episode is 100 cycles in compared
with 6000 cycles of a game time. There are several terms and assumption in this
environment
Figure ABC small environment
Round(Episode):
- At the beginning of a round, 1 of 2 attackers possesses the ball. If one of these
conditions is satisfied, a round is considered end:
- Defenders successfully get the ball from attackers
- The ball runs out of the field
- Round time reaches 200 cycles.
Successful round requirement:
- Pass receiver is not offside and successfully get the ball
23
- When possessing ball, receiver stands below the last defender of opponent with at
least M (pre-defined constant) meter difference in x coordinate.
- Pass is performed in 200 cycles from beginning of round
Actions involved:
For both hand-coded and through pass strategy, there are several actions invoked in the
technique:
Direct pass: Pass the ball directly to the position of teammate
Through pass: Through pass: Pass through the space between opponents to a
good position. The teammate has to dash to the pass target to get the ball.
Point to: Point agent’s arm to a desired position
Holdball: Try to keep the ball from opponent without moving
Dribble: Dribble to a position
Move: Agents without ball do a strategic moving

Intercept: Try to take the ball from the opponent or receive the ball from a pass
High level strategy in small field
For attackers For defenders
IF kickable:
IF: Can make through pass?
Do make through pass
ELSE IF: Can dribble?
Dribble
ELSE IF: Can direct pass?
Do make direct pass
ELSE:
Hold ball
ELSE:
Move
Point to
IF kickable:
Kick the ball out of field
ELSE:
Intercept
Result
The result is based on the criterion the number of successful round/total round
executed. We did experiment on our through pass improvement and the original one
24
using the same high level strategies. For 100 rounds (episodes), our improvement has 27
successful rounds compared to 10 of the original one.
5.2.2. Scoring skills
In order to evaluate the scoring skill, we have planned to test it in 3 cases which are
different in the number of attackers and defenders.
1 attacker versus goal keeper
This is the case when the forward receives the ball from a through pass which left

all defenders behind. It is the easiest case since there is only the goal keeper left. We have
set up an environment where there are only 1 attacker and goal keeper. The attacker will
try to dribble toward the goal and shoot while the goal keeper will try to block the shoot.
Figure<?>: Versus goal keeper situation
In this case, our approach performance is very good with success rate is above
80%. However, the goal keeper performance is not very good. We will improve the goal
keeper and retest our approach later.
1 attacker versus 2 defenders and goal keeper
This is the case when our side forward attracting opponent side defender and the
ball is passed to our center forward.
25
Figure<?>: 1-versus-2 situation
In this case, we have tested it with Helios’ base code and currently, it performs
pretty well with success rate is about 50%.
3 attackers versus 4 defenders and goal keeper
A normal case when our attackers surround opponent penalty area. We haven’t
tested our approach in this case yet. Currently, our approach does not perform well in this
situation. We will try to improve in near future.
6. CONCLUSION
In this report, we have introduced about the Robocup and Robocup 2D simulation
league which we’re interested in. In our team, we use Helios’ base code named agent2d.
We have built our own strategies and also improved some basic skills in agent2d. Now,
our team performance is just acceptable but very potential.
Currently, we are exploring machine learning method like reinforcement learning and we
have planned to apply in our team. These methods are very promising in further
improving our team performance in order to take part in future Robocup competitions.
7. REFERENCES