Agent based evacuation modeling with multiple exits using neuroevolution of augmenting topologies

Advanced Engineering Informatics 35 (2018) 30–55

Contents lists available at ScienceDirect

Advanced Engineering Informatics
journal homepage: www.elsevier.com/locate/aei

Full length article

Agent-based evacuation modeling with multiple exits using NeuroEvolution
of Augmenting Topologies

T

Mehmet Erkan Yuksel
Department of Computer Engineering, Mehmet Akif Ersoy University, Burdur, Turkey

A R T I C L E I N F O

A B S T R A C T

Keywords:
Evolutionary computation
Neuroevolution
Genetic algorithms
Agent-based modeling and simulation
Evacuation modeling
Emergency evacuation

Evacuation modeling offers challenging research topics to solve problems related to the development of emergency planning strategies. In this paper, we built an agent-based evacuation simulation model to study the
pedestrian dynamics and learning process by applying the NeuroEvolution of Augmenting Topologies (NEAT)

which is a powerful method to evolve artificial neural networks (ANNs) through genetic algorithms (GAs). The
NEAT method strengthens the analogy between GAs and biological evolution by both optimizing and complexifying the solutions simultaneously. We set our main goal to develop a model by identifying the most appropriate fitness function for the agents that can learn how to change and improve their behaviors in a simulation environment such as moving towards the visible targets, producing efficient locomotion, communicating
with each other, and avoiding obstacles while reaching targets. The fitness function we chose captured the
learning process effectively and our NEAT-based implementation evolved suitable structures for the ANNs autonomously. According to our experiments and observations in the simulated environment, the agents accomplished their tasks successfully and found their ways to the exits.

1. Introduction
Evacuation modeling is an important part of pedestrian modeling
expertise. It is the process of guiding evacuees to the exits when an
emergency situation happens. It is developed to ensure the safest and
most efficient evacuation time of all expected residents of a building or
an area. It offers design capabilities that allow the simulation of complex scenarios in different types of environments [8]. Using evacuation
models to optimize the flow of people allows much more ergonomic,
effective and safe design at earlier stages of an evacuation plan in
crowded places. Some of the benefits of evacuation modeling are: designing an effective layout with respect to areas, reducing congestion
and queuing at key points, optimizing the position of objects and advertising signage in relation to population profile, facilitating successful
operational planning of access, safety and security with regards to
normal and emergency circumstances [18].
The simulation of a realistic human behavior is important for the
virtual world and it has practical values such as emergency planning for
evacuating the areas at risk. In real world, the safe evacuation of
thousands of people is a crucial operation. In such cases, obtaining
information on the nearest and fasted evacuation routes, the time to
evacuate people, the time between the happening and the arrival of the
emergency personnel, the procedures that the authorities should use to
avoid confusion, and loss of life are all very important. By observing

and understanding the possible scenarios in a simulated environment
beforehand, the authorities (i.e., decision-makers, evacuation managers, safety planners, researchers) can train emergency personnel so
that the response to the actual happening is successful [54,55]. Thus,
we aimed to provide an advantage/benefit for the authorities by developing an agent-based simulation model to have an ideal understanding of the human behaviors and environmental characteristics in a

specific zone, be able to estimate possible outcomes of different response and evacuation strategies under different conditions, and generate a series of evacuation plans accordingly.
In our study, we propose an agent-based evacuation modeling and
simulation using the NEAT method [12,41,77,78,76,88] for emergency
evacuation situation to help decision makers to determine the evacuation time for buildings or areas at risk. In our experiments, we observed
successful evacuation behaviors and obstacle avoidance using autonomously-evolved neural networks. We also showed that the learning
behavior is a crucial step in the direction of emergency planning and
how we achieved sophisticated behaviors through our NEAT-based
model.
2. Related work
Evacuation modeling contains iterative processes to determine the
best suitable egress routes/paths and calculate the estimated time

E-mail address:
/>Received 18 November 2016; Received in revised form 26 November 2017; Accepted 29 November 2017
1474-0346/ © 2017 Elsevier Ltd. All rights reserved.


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

movements of evacuees (i.e., physical abilities, local directional
changes, walking speeds, various physiological, psychological and social factors), the interactions and decisions among evacuees. They are
defined as a bottom-up approach in which evacuees are modeled as
individual entities that have unique characteristics such as age, gender,
disability, body size and walking speed. They describe the time-space
behaviors of the individual evacuees [38,58]. Formulas, expressions or
rules involving spatial transition probabilities are repeatedly applied to
temporal changes in situation or behavior. Microscopic models are
computationally intensive. Performing simulation of large-scale crowds

is difficult on traditional single-processor systems. However, parallel
computing techniques can be used to achieve this problem successfully.
The level of complexity in microscopic models can be overcome by
using analytical methods (i.e., to define the path choice using mathematical equations). Analytical methods are cheap to use. However, they
are limited to the complexity of evacuation problems. Because, as
analytical methods become more sophisticated and more detailed in an
evacuation model, they can become more complex to analyze the
evacuation process. In addition, peak or transient conditions are not
modeled, since analytical models assume that the system has reached a
steady state or equilibrium. Finally, it is hard to determine whether the
existing data characteristics can define the system under study without
measuring various design parameters [92].
Although much study has been done on microscopic models to
improve the behavioral realism, evacuation operations, natural locomotion and decision-making process, none of the existing models can
realistically analyze the high-density crowds. In order to develop an
efficient model, many choices and parameters (based on conditions,
cases, events, environments, and so on) have to be considered. Some
classifications can be helpful to facilitate these choices/parameters and
get an overview of the available options. Zheng et al. [93] indicate
seven classes for microscopic approaches which are: social force models,
cellular automata, agent-based models, lattice gas models, fluid-dynamic
models, approaches depending on the experiments with animals, and gametheoretic models. In the light of this information, we can present most
common microscopic models which can be used to get better results for
pedestrian simulation as follows: social forces models [29], rule-based
models [63], cellular automata [17], velocity-based models [83,84], and
the optimal steps model [71,70]. The difference between them is in the
discretization of space and time. In addition, we can specify the conceptual similarities and differences of these models according to the
perspective of scalar fields (also referred to as the superposition principle) for the simulation of pedestrian dynamics [70]. The perspective
of scalar fields is a powerful method and provides a common mathematical basis for many models, and the different use of it produces
distinct emergent effects. However, it has limitations such as the flexibility, superposition of binary interactions [50], calibration of model

parameters [28,34,49]. Furthermore, models based on scalar fields
concept that is used as potential, cost, utility, benefit, or probability can
be efficient for practical applications but lack a plausible representation
of the decision-making process and natural locomotion. Consequently,
identifying similarities and differences is important to choose an appropriate model or develop a new method if no other model meets the
requirements [69].
Social force models describe pedestrian behavior (i.e., route choice behavior, certain actions/motions) microscopically through social fields
triggered by the social behavior of the individuals. In these models; social
forces such as comfort zone, tension, stress, emotion, panic, anxiety,
pushing and fighting for space, herding, flocking, arching and clogging are
modeled, the interactions and decisions between individuals are tested and
validated. Social force models enable us to create simulations that look
more like particle animation than human movement [59]. They help researchers to find new relations and produce mathematical rules to understand crowd behaviors in normal and emergency conditions. However,
they are more complex decision-making processes and require much more
effort to achieve realistic results.

required to evacuate the areas at risk. Different models can be developed for evacuation applications depending on the scope, solution
methodology, and input parameters. In general, existing studies concentrated on modeling evacuation problems that emphasize the estimation of evacuation time are closely related to different research topics such as path planning, navigation in large virtual environments,
traffic assignment, operations research, process control, simulation,
optimization, network flow and many others [14,43,44,66,73,74,80].
Based on the scope, the-state-of-the-art approaches in evacuation
modeling research can be mainly classified as macroscopic and microscopic models, both of which can analyze the movements of evacuees
over time.
2.1. Macroscopic models
Macroscopic evacuation models take into account the movement of
evacuation as a homogeneous flow. They do not consider the individual
behaviors and movements of evacuees, the interactions and decisions
between evacuees for selecting the egress routes. They focus on the
system as a whole. They are defined as a top-down approach in which
collective evacuee movements are characterized by model parameters

through closed-form expressions. Crowds are represented in an aggregate manner using some distinguishing key features such as spatial
density, average velocity, and flow rate in relation to the location and
time. Since the time is a decisive factor for evacuation process, macroscopic models are mostly used to generate good lower bounds for the
evacuation time. These bounds can be used to analyze existing buildings or help planning new buildings [24,65]. Most of macroscopic
models use mathematical or analytical methods which depends on
static or dynamic network optimization formulations and some integer
programming models to solve evacuation problems. However, they
suffer from loss of accuracy as individual behaviors of evacuees can
affect evacuation time [5,26,54].
Common macroscopic models are: regression models [42,48], routechoice models [32], queuing models [45], and gas-kinetic models [30].
Regression models use statistically determined relations between flow
variables to predict pedestrian flow under specific conditions [60].
Pedestrian flow characteristics are related to infrastructures such as
corridors, lobbies, stairs, ramps, walkways, and so on. Route-choice
models define pedestrian wayfinding depending on the utility concepts.
Pedestrians choose the best appropriate destinations to maximize the
utility of their trips such as comfort, travel time, convenience, safety
and cost. Queuing models benefit from Markov chains to describe how
pedestrians move from one network node to another. In general, nodes
represent rooms and links represent doors. Markov chains are defined
by a set of states together using transition probabilities [60]. Only
transitions causing state change are taken into account. Gas-kinetics
models use an analogy with fluid or gas dynamics to determine how
crowd density and velocity parameters change over time using partial
differential equations [58,79].
Macroscopic evacuation models are mainly based on optimization
approaches. They are computationally efficient and suitable for largescale crowd simulations. While these models are good at producing the
general density-flow profiles observed in crowd evacuation, they are
unable to describe emergent crowd phenomena [26,56]. This restriction is understandable and acceptable given that macroscopic models
are expressions of deductive reasoning. In so doing, many simplifying

assumptions have to be made in order to keep such theorems tractable.
On the contrary, emergent phenomena arises spontaneously from
complex and dynamic interactions at lower levels that occur naturally
without influence from external signals or conventions [58].
2.2. Microscopic models
Microscopic evacuation models analyze the evacuation situation in
a detailed manner. They consider the individual behaviors and
31


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

The Optimal Steps model [71] is inspired by the rule based approach
of cellular automata. It overcomes the limitations of cellular automata
and provides an improvement towards a locomotion process. In contrast to cellular automata, agents do not move from cell to cell. The
movement is not restricted to a spatial grid of any kind but rather occurs in continuous space. The spatial discretization represents the
human step. The optimal steps model defines decision making through
utility optimization and locomotion as a discrete stepping process. It
captures the stepwise motion of a pedestrian and uses it as discretization scheme in the simulation. It finds the next position by maximizing
the utility or, equivalently, by minimizing the potential. It uses local
optimization on a circle around a pedestrian to determine the next
position. In optimal steps model, the target function is a navigational
field. Each individual’s stride length depends on his/her speed. This
introduces a delay in adaptation, because all speed measurements involve the past [71]. The optimal steps model has to investigate in a
series of controlled experiments where pedestrians had to walk around
obstacles that made them turn around to various degrees in real life.
Because, pedestrians are more likely to react instantaneously, and they
cannot change their direction of motion arbitrarily. They either have to

slow down or change the direction slowly. Therefore, new modeling
concepts can be developed to allow to simultaneously adapt the speed
and stride length [89,90]. In optimal steps model, agents use a greedy
algorithm to reach a target. This is based on the superposition of scalar
fields and a local optimization scheme. The scalar field is interpreted as
a utility function mapping each point in the plane to a utility value. The
local optimization can then be interpreted as utility optimization. Although this interpretation is questionable as a representation of human
decision-making processes [23], it is accessible to many disciplines (i.e.,
social sciences, physics). The area searched for the optimal next position coincides with the reach of the step length. Thus, one step of an
agent in the simulation represents the step of a pedestrian. This is intended to bring closer together the physical process of human walking
and pedestrian dynamic simulations [89,90].

Rule-based models are used to specify more believable pedestrian
behaviors for low and medium density crowds. They do not need to
calculate collision detection and response at all. They usually accept
conservative approaches through applying “wait rules” (i.e., to enforce
ordered crowd behavior for medium density crowds in a flocking style)
which work well for low densities in daily life crowd simulation, but
prevent realism for high-density conditions or panic situations [59].
The rule-based models do not consider the contacts (interactions)
among individuals and therefore they fail to simulate the pushing behaviors of individuals. These models can be combined with cognitive
models, hence different behavioral rules can be applied to crowds or
individuals to obtain more realistic pedestrian behaviors for crowd simulation methodologies [53,72].
Cellular Automata are discrete, abstract computational systems that
have demonstrated advantageous and useful both as models of complexity and as more specific representations of non-linear dynamics in
various research areas. They are “mathematical idealizations of physical systems in which space and time are discrete, and physical
quantities take on a finite set of discrete values” [91]. They are characterized by the discretization of space into cells. They evolve in discrete time steps (generation, iteration or cycle) and usually provide a
framework for discrete models with locally homogenous interactions.
Cellular automata are computationally efficient because of the inherent
discrete structure of space and time in the simulation, and the implicit

spatial data structure of cells. The main advantage of cellular automata
is their simplicity in use, which allows both for fast implementation and
computation. There are certain limitations to the cellular automata.
They limit the study of microscopic behaviors (i.e., stepping behavior,
continuous motion, positions in continuous space, inertia, contact
forces) in human crowds due to the fixed/coarse discretization of space.
They do not allow for interactions and contacts among individuals.
They provide realistic results for low-densities, however they offer
unrealistic results when individuals in high-density crowd situations are
forced into discrete cells. The size of cell usually specifies the physical
extension of agents and hence overly limits the maximum density.
Smaller cells can provide more detail and show improvements in some
phenomena. On the other hand, such cells may cause a loss in computational efficiency. More realistic behaviors can be achieved by
precomputing paths towards higher-level goals and storing them within
the grid [37,69]. Finally, cellular automata can be classified according
to some characteristic features for the simulation of pedestrian dynamics: they can be deterministic, probabilistic, rule-based approach,
floor field concept, and they can use different grid structures such as
rectangular, hexagonal and triangular cells.
Velocity-based models are usually formulated in continuous time and
space. However, they can also be evaluated at coarse time steps that
make them discrete in time and space. In order to simulate a model in a
computer, discretization is required for numerical computation.
Contrary to cellular automata, the next position is not chosen depending on the rules or transition probabilities. Instead of this, the
velocity (the speed and direction) is defined by a first-order ordinary
differential equation and then numerically integrated to obtain the
agents' positions at discrete simulation time steps. Velocity-based
models can be classified into three categories: the optimal-velocity models
which are also referred to as car-following models or car traffic simulations [84], obstacle-velocity models which were mainly studied in robotics, animation and computational science [87,9,11], the gradient
navigation model which is an ordinary differential equation based model
to simulate pedestrian dynamics [16,15]. In velocity-based models, the

decision-making process of an agent is performed by using the velocity
function. This causes some limitations to model advanced behavioral
features. It can also be questioned whether the decision-making of the
human movement can usually be represented with such equations. For
the simulation of the physical environment, the velocity function is also
limited because of the physical interactions that are commonly modeled
with forces which lead to a second-order differential equation.

2.3. Machine learning techniques
Emergency management and evacuation planning can be improved
significantly with the help of information technology solutions and
computer-based information systems such as decision support systems.
Previous studies were actually driven by defence applications [20] including enhanced reality simulators [21] and evacuation models that
incorporated models of human mobility and behavior [25]. Recent
studies focused on agent-based modeling techniques that offer some
level of realism by representing each individual evacuee as an agent
that follows specific goals [6,22,19,27,35,39]. In real world problem of
learning a task such as game playing [68] and controlling a robot [86],
the behavior must be learned by taking different actions and assigning
high values for good decisions and low values for bad decisions depending on the reinforcement feedback. For instance, obstacle avoidance is one of the most important aspects of controlling a robot [13].
The robot movement would be very restrictive and meaningless without
it. There are many methods for avoiding obstacles. The simplest forms
of them are based on image processing techniques. These techniques
exploit the color differences between the objects. They apply edge detection algorithms such as Roberts, Sobel, Prewitt, Canny, Marr-Hildreth to produce an image that contains only edges. However, they are
very limited and do not include any learning mechanisms.
Understanding the impact of Artificial Intelligence requires autonomous systems that learn to make better decisions. Reinforcement
learning is one powerful method for doing so, and it can be applied to a
wide range of sectors, including industrial control, robotics, manufacturing, logistics, finance, consumer modeling, automotive, telecommunication, healthcare, and game playing. It is one of the easy
paradigms to apply machine-learning approaches if the problem is
simple and cannot be solved by image processing techniques [36].

32


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

the most relevant software systems and tools for crowd simulation,
pedestrian dynamics and evacuation modeling that have been developed both from academia and industry are: Acumen [2,4], AnyLogic
(AnyLogic Company, 2000), Artificial Fishes [81], Autonomous Pedestrians [67], Crosses [85], Egress (AEA Technology, 2002), Exodus
[7], Legion (Legion International, 2003), MACES + HiDAC [57], Massive (Massive Software Inc., 2005), Menge [10], OpenSteer [64], Reactive Navigation [40], Simulex [82], Space Syntax [61], Steps (MottMacDonald, 2003), Vadere [71,70], ViCrowd [52,51]. Some of these
systems (i.e., Egress, Exodus, Steps) are based on cellular automata
approach, hence it is important to understand their movement simulations artifact. Using a cellular automata model in these systems creates several problems about grid size, alignment of grid to environment,
fatigue factor, speed, and route selection. For instance, fixed densities
and unrealistic flow rates at doors. In such a scenario, the grid size is a
critical parameter to calibrate for achieving the desired behavior [58].
In addition, some of the software systems mentioned (i.e., Acumen) use
particle-based approaches. They lack realism when they are applied to
three-dimensional virtual objects for animations. Finally, most of these
referred software systems (i.e., Artificial Fishes, Massive, OpenSteer,
Reactive Navigation, Crosses, ViCrowd) are rule-based crowd simulation models and commonly used in industry applications. They are
suitable for low and medium density crowds. However, they yield unnatural emergent behavior in high density crowds situations (i.e., pedestrians stop and wait for space to clear up). They also have several
limitations in terms of simulating realistic human behaviors depending
on the physiology, psychology, and social interactions [58].

Without a supervisor, a robot can evaluate its performance only in
terms of final outcomes. Whenever the robot collides with an obstacle, a
negative reinforcement signal can be generated. In the reinforcement
learning, the problem of assigning credit is solved by forming associations between sensory inputs and predictions of future actions. Several
authors proposed the use of reinforcement learning for robot control

[31,62]. However, for more complicated tasks such as avoiding an
obstacle, navigating in a simulated environment and moving the agents
towards an exit point using reinforcement learning is less straightforward. The main difficulty in these complicated systems is representing
the states and choosing the right approximation function. Currently,
these problems are handled by trial-and-error. In addition to this, there
are many other challenges in current reinforcement learning research
[3,46]. For example, storing the values of each state in the memory is
expensive since the problems can be very complex. As a solution, value
approximation approaches such as decision trees or neural networks
can be applied [1]. Introducing these value estimations causes problems
and it affects the quality of the solution. Furthermore, similar behaviors
usually reappear, and to avoid learning everything all over again,
modularity is introduced. Finally, it is generally not possible to completely determine the current state. Therefore, it will be a hard task to
apply traditional reinforcement learning methods to this domain.
Another common technique used to control a robot or avoid an
obstacle is supervised learning. This method requires training to control
the robot correctly. One way for controlling the robot is to randomly
move it and record the movements. One example of using the combination of reinforcement and supervised learning to avoid obstacles is to
start small and allow the evolution to be increasingly complex to help to
find the solutions faster, and find more complex solutions [47]. In this
study, the researchers did driving experiments to control cars remotely
at high speeds in outdoor environments that are unstructured. They first
calculated the estimate depths from single monocular images by applying supervised learning. After estimating the depths, the reinforcement learning was applied and a simulator rendered the synthetic
scenes. According to the output function of the vision system, the
system learned the control policy to select the steering direction. The
limitation of this approach is it is hard to generate training data for
complex tasks such as obstacle avoidance [47]. If the environment or
configuration of the robot changes, the robot must be trained again to
solve the problem. The obstacle avoidance problem can also be solved
by using path planning algorithms. Spong et al. [75] described an algorithm that uses attractive and repulsive fields to create a path to

avoid obstacles. James and Tucker [33] used A∗ search algorithm with a
path planner.
As a solution to accomplish complex tasks effectively, such as robot
control and obstacle avoidance, researchers also developed neural
network based models. Neural networks have several advantages. They
are fast, resistant to noise in comparison to traditional approaches. By
using neural network based controllers, a robot can successfully navigate in an environment with obstacles by considering the obstacle and
target information received from its sensors. If there are moving obstacles in the environment and the environment is more complex,
avoiding obstacles and moving around will be challenging. Therefore,
using neural networks will not be sufficient to solve the problems and
more effective methods need to be developed.

3. Our Multi-Agent based evacuation simulation model
Our model offers an open source cross-platform solution based on
NEAT, in which physical and analytical models are combined. It simulates and analyzes pedestrian dynamics and evacuation process in a
public indoor environment through combining the rules based on numerical calculations that consider agent behaviors and decisions affected by environmental conditions. The main idea is to create autonomous agents that use learned behaviors, develop their skills,
communicate with each other, avoid obstacles and reach targets strategically. Thus, our proposed model serves as a valuable software tool
for estimating evacuation time, observing pedestrian flow phenomena
and designing guidelines for emergency evacuation situations, in particular when the communications and interactions between individuals
play a significant role.
3.1. System architecture
The system architecture of our model is schematically shown in
Fig. 1. Each autonomous agent represents an individual pedestrian and
can act in a collaborative manner. An agent senses and evaluates its
surroundings, interacts with the environment and other agents, makes
optimal decisions depending on the agent behavior model (ABM) and
several rules on agent dynamics. Rules are derived at the levels of interactions between the agents. Consequently, our aim is to study on the
potential chaotic situations among the agents as they develop their
behaviors and achieve goals in the simulation environment.
The agents are trained and they develop highly complex ANNs

through NEAT. NEAT serves as a brain for an agent as illustrated in
Fig. 2. It controls the agent to select right behaviors based on the agent’s
sensory inputs. It gets information about the agent’s surroundings and
returns the appropriate action that the agent should take at that time
step. Therefore, the learning process of the agent is observed, the inputs
and fitness function are modified accordingly to help the agent avoid
obstacles and move to the targets.
In our model, there are N autonomous agents {A1 , ......, AN } , each
with a unique local position on a plane. Each agent develops its behavior and reaches a certain goal position depending on the

2.4. Software systems for pedestrian dynamics and crowd simulation
There are many commercially available computational tools and
software systems for the design, simulation and analysis of the evacuation process. Most of them focus on the modeling of spaces and
occupancies, and rarely consider human and social behaviors. These
systems fall into macroscopic (focus on groups of pedestrians rather
than individual characteristics, and analyze high-density, large-scale
system) and microscopic (study the characteristics of individual pedestrians and interactions with others) classes [58]. Some examples of
33


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

Fig. 1. System architecture.

each agent uses this capability to share its information (i.e., unique ID,
current position, desired velocity to neighbors, its fitness).
Since we demand advanced behaviors from the agents, we provided
them with three detailed sensor arrays for identifying the exits, obstacles, and other agents. Each sensor array divides an agent’s environment into eight pie slices and also three concentric zones that act

as radars. The inner zone detects the presence of objects that can cause
a collision with the agent, the middle zone detects the objects close to
the agent, and the outer zone detects any objects farther away (Fig. 3).
The three sensor arrays determine an agent’s situation in the environment. Each sensor array consists of three subarrays with 17-elements as illustrated in Fig. 4. The subarray with single-element defines
the objects colocated in a certain area that the agent occupies. The

environmental conditions. The agents move on the plane without colliding with each other and any static object and their motions are
planned in a decentralized manner. We modeled each agent Ai as a disc
with a radius r (Ai ) , an initial position P (Ai ) , a goal position GP (Ai )
and a preferred velocity PV (Ai ) that is limited to the range with uniformly distributed on [Vmin,Vmax ]. At time step t , the agent Ai has a new
position NP (Ai ) and moves with a new velocity NV (Ai ) in the environment. In the absence of any other agents and static objects on the
path to the agent Ai ’s goal position, the following equation is valid;
NV (Ai ) = PV (Ai ) ⩽ Vmax . In addition, an agent can sense the radius,
positions and velocities of its neighboring agents (a subset of agents
within a limited fixed sensing range) as shown in Fig. 3. Finally, the
agents are also capable of the limited one-way communication. Thus,

Fig. 2. NEAT-based agent behavior model.

34


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

Fig. 3. Agents with egocentric sensors.

movement is taken in that direction. If the movement is outside the
plane or onto static objects or another agent then the agent remains

stationary instead.
We take advantage of the homogeneous groups approach in which
all individuals within a group use the same controller ANN. The
homogeneous groups approach is suitable for crowd simulations as it
provides better search space scalability. From this point of view, our
basic method (Fig. 6) is to control the agents through neuroevolution,
coevolve them in separate subpopulations, and ensure that they communicate with each other in a common task. In addition, the fitness
function (shared knowledge) of the simulation is represented as a set of
goals (i.e., reaching a target, obstacle avoidance, collaboration), encoding the desired behaviors for a group of agents. In our method, each
autonomous agent performs specific tasks in every training cycle
(learning process) in order to develop its behavior. An agent perceives
its surrounding area through its sensors. It uses a database structure to
create, update, store, and share knowledge (perceptual data). Its controller ANN processes the knowledge as input values and then generates
output values to create a motivation pattern. Finally, the agent interprets this motivation pattern to determine what optimal actions should
be performed in its current location.

subarray with 8-elements defines the nearby objects, and the other
subarray with 8-elements defines any objects farther away. At the time
step of the agent’s action, all elements of the three sensor arrays are
loaded with the values directly related to the objects that are inside the
agent’s eight radial fields of view. Consequently, the three sensor arrays
are combined to provide a perceptual data (a 51-elements input data)
for our NEAT-based method that makes optimal decisions for an agent.
Each autonomous agent is controlled by an ANN as shown in Fig. 5.
In order to conform to the definition of a communication or cooperation, each agent uses an identical controller ANN. Hence, we basically
reutilize the same ANN for each agent. This approach indicates that
each agent has an identical control policy and any individual differences in the agents’ behaviors arise purely from the differences in their
sensor data. The controller ANN has ten motion units (wait, move and
the eight cardinal directions) in its output layer. Thus, the agent has
various choices that specify the agent’s desired direction and preferred

velocity at a given time step. The values obtained from the agent’s
sensor arrays are calculated and matched with the ANN’s inputs. The
inputs are then processed by the ANN to create a motivation/reward
pattern at the ANN’s output. The motivation/reward pattern determines
the agent’s motivation status and helps the agent to make better decisions on how to move and coordinate with the other agents. It is interpreted as an optimal choice of one of the actions available to the
agent. For instance, if the motivation level of the wait is higher than the
motivation level of the move then the agent remains stationary for the
current turn. Otherwise the eight outputs related to the directions are
analyzed to find which has the highest motivation level and a

3.2. The NEAT method
NEAT is a neuroevolution method (the evolution of ANNs using
GAs) that has a great potential to solve complex control problems (with
high-dimensional continuous state and action spaces) and perform

Fig. 4. Agent sensor array.

35


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

Fig. 5. Structure of the autonomous agent’s controller ANN.

sequential decision tasks. It finds solutions more efficiently than other
neuroevolution techniques. It consists of putting together the three key
features (tracking genes through historical markings, protecting innovation
through speciation, minimizing dimensionality through incremental growth

from minimal structure) into one system. These features cooperate to
create a system that is capable of evolving solutions of minimal

complexity and maintain a balance between performance and diversity
of solutions [12,41,77,88].
NEAT’s genetic encoding allows related genes to be easily lined up
when two genomes cross over during mating. As illustrated in Fig. 7, a
genome is the genetic material of the ANN and contains information
about network connectivity. It comprises two chromosomes: node

Fig. 6. Training the agents through NEAT.

36


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

Fig. 7. A genotype to phenotype mapping in NEAT [77]

Fig. 8. Structural mutation in NEAT [77]

37


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

Fig. 9. Our NEAT-based pathfinding algorithm.

diversity of complexifying network structures/topologies simultaneously [77,76]. Structural mutation occurs in two forms, as shown in
Fig. 8. Each mutation enlarges the size of the genome by adding connection gene or node gene. In add connection mutation, a single new
connection gene is created by connecting two previously unconnected
node genes, added to the end of the genome and given the next available historical marker. In add node mutation, an existing connection
gene is split and a new node gene is inserted between the two new
connection genes that are placed where the split connection gene located. The split connection gene is disabled and then the two new
connection genes are added to the end of the genome and given the next
available historical markers. This method of adding nodes is used in
order to minimize the initial effect of the mutation and incorporate new
nodes immediately into the ANN [77,76].

chromosome and connection chromosome. Node chromosome contains
a set of node genes that provides all available nodes in the network.
Each node gene has a unique identifier and a node type which represents input node, hidden node or output node. Connection chromosome contains a group of connection genes, each of them denotes
two node genes being connected. Each connection gene specifies the
input node, output node, connection weight, gene activation that tells
whether or not the connection gene is created, and an innovation
number which is assigned to each gene corresponding to its order of
appearance throughout evolution. Innovation number is a historical
marker which states the original historical ancestor of each gene in the
ANN system. It can also be referred to as a global counter that indicates
which mutation in the entire evolutionary history caused the creation of
that gene.
Structural mutation in NEAT alters both connection weights and
network topologies. NEAT starts evolution with a uniform population of
small, simple structures without hidden nodes and complexifies the
network topology into diverse species over the course of generations,
resulting in increasingly complex/advanced behaviors. In this way, the

network topology does not have to be known a priori and NEAT finds an
appropriate level of complexity for the task. In complexification, the
main characteristic that distinguishes NEAT from other machine
learning techniques is its unique approach to maintaining a healthy

3.3. System configuration and parameters
We built our model by using Java Programming Language, Eclipse
IDE, MongoDB, OpenGL API and the NEAT method. ANJI (Another
NEAT Java Implementation) [33] is used to implement our NEAT-based
model for the purpose of training the agents and observing their behaviors. The movement of each agent is controlled by our pathfinding
algorithm with NEAT, which is presented in Fig. 9. Algorithm 1 and

38


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

Table 1
System configuration and parameters in our NEAT-based simulation model.
Startup parameters

Settings

Definition

random.seed(int)

0

run.reset(boolean)

True (1)

It is the initial point to create random numbers through a random number
generator that allows the exact replication of runs with random elements.
It specifies whether or not a run is being continued or whether all persistent
data is removed and the run is started from scratch.

Evolution Parameters

Settings

Definition

numberof.generations(int)
population.size(int)

20
200

topology.mutation.classic(boolean)

False (0)

add.connection.mutation.rate(double)

0.02

remove.connection.mutation.rate(double)

0.01

remove.connection.max.weight(int)

100

remove.connection.strategy(int)

Skewed

It is the number of generations to perform for a run.
It determines the number of individuals in the initial population, and each
generation after reproduction is carried out.
It defines how topological mutations are handled. In our NEAT-based method,
the mutation rates of ANJI are the rates at which new topological mutations
arise between all possible locations where a valid, effective mutation occurs.
It is the probability of new connections being added. It depends on how the
“topology.mutation.classic” parameter is set. The initial weights of new
connections have a random value coming from the normal distribution.
It is the rate at which existing connections within the range of the
“remove.connection.max.weight” parameter are removed. All weights taking
place in that range are sorted in ascending order of the weight size. These
weights are then removed probabilistically in proportion to their size.
It is the size of weights to be removed by the “remove.connection” operator.
These weights are sorted in ascending order by size and removed depending
on the “remove.connection.mutation.rate” parameter.
It is a strategy for the “remove.connection” mutation. Possible values are the
following:

Skewed: The probability of a connection being removed is inversely
proportional to the weight size.
Small: Similar to skewed, however, it is not possible for the connections to be
removed with preference for those with smallest weight size.
All: All connections, regardless of weight, have an equal chance of being
removed.
It is the probability of new nodes being added. A node can only mutate at an
existing connection. (It depends on how the
“topology.mutation.classic” parameter is set).
As connections are removed, some nodes can be stranded. Therefore, this
parameter defines the rate at which stranded nodes are removed from the
chromosome.
It is the probability of existing connection weights being mutated by adding a
random value from the range of the “weight.mutation.std.dev” parameter.
It is the standard deviation for weight mutation values. The weights of
connection are mutated by adding a random double value generated from a
random normal distribution with currently defined standard deviation.
The maximum limit for the values of the connection weight.
The minimum limit for the values of the connection weight.
The population percentage which survives and reproduces each generation. It
is defined by the fitness values sorted in descending order.
It indicates that the fittest individuals from species with the size defined by the
“selector.elitism.min.specie.size” parameter are copied unchanged into the next
generation.
It indicates whether or not roulette selection is used.
The minimum number of individuals that a specie has to contain for its fittest
member to be copied unchanged into the next generation.

•
•

•

add.neuron.mutation.rate(double)

0.001

prune.mutation.rate(double)

1.0

weight.mutation.rate(double)

0.75

weight.mutation.std.dev(double)

1.5

weight.max(double)
weight.min(double)
survival.rate(double)

500.0
-500.0
0.2

selector.elitism(boolean)

True (1)

selector.roulette(boolean)
selector.elitism.min.specie.size(int)

False (0)
1

Speciation Parameters

Settings

Definition

chromosome.compat.excess.coefficient(double)

1.0

chromosome.compat.disjoint.coefficient(double)

1.0

chromosome.compat.common.coefficient(double)

0.4

speciation.threshold(double)

0.2

The coefficient used to specify the genetic difference between two
chromosomes. The compatibility value is adjusted depending on the number of

excess genes.
The coefficient used to specify the genetic difference between two
chromosomes. The compatibility value is adjusted depending on the number of
disjoint genes.
The coefficient used to specify the genetic difference between two
chromosomes. The compatibility value is adjusted according to the differences
in the values of common connection weights.
The compatibility threshold that indicates whether two individuals belong to
the same species.

Fitness Function Parameters

Settings

Definition

stimulus.size(int)
response.size(int)
fitness_function.class(Java class)

51
10
com.anji.pathfind.PathFindFitnessFunction

fitness.function.adjust.for.network.size.factor(double)

0.0

The number of input nodes for the initial neural network topology.
The number of output nodes for the initial neural network topology.

It is used to specify the fitness of the evolving population. This class depends
on the domain.
This parameter allows for an apparent fitness penalty to be applied to a
chromosome depending on its size.
(continued on next page)

39


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

Table 1 (continued)
Startup parameters

Settings

Definition

fitness.threshold(double)

0.99

fitness.target(double)

1.01

pathfind.maxtimesteps(int)
pathfind.numberoftrials(int)

2000
20

It indicates a chromosome is considered to have “solved” the problem presented
by the fitness function if the chromosome’s fitness exceeds threshold.
It determines when a chromosome’s fitness exceeds the target value, the
evolutionary run ends.
The maximum number of time steps to find the best route.
The number of trials to find the best route.

Sigmoid Activation Parameters

Settings

Definition

initial.topology.activation(string)

Sigmoid

initial.topology.activation.input(string)

Linear

initial.topology.activation.output(string)

Sigmoid

It specifies the activation function of the hidden nodes in the neural network.

The functions are: linear, sigmoid, tanh, evsail-sigmoid, tanh-cubic, step, signed.
It specifies the activation function of the input nodes in the neural network. The
functions are: linear, sigmoid, tanh, evsail-sigmoid, tanh-cubic, step, signed.
It specifies the activation function of the output nodes in the neural network. The
functions are: linear, sigmoid, tanh, evsail-sigmoid, tanh-cubic, step, signed,
clamped-linear, signed-clamped-linear.

Neural Network ArchitectureParameters

Settings

Definition

initial.topology.fully.connected(boolean)

True (1)

initial.topology.numberof.hidden.neuron(boolean)

False (0)

initial.topology.activation.input(string)

Linear

recurrent(string)

BestGuess

It defines whether or not the initial neural network topologies start with all input

nodes being fully connected to all output nodes.
The number of hidden nodes in the initial neural network topologies. If the value
is greater than 0, all input nodes are fully connected to hidden nodes. These
hidden nodes will then be fully connected to all output nodes.
It defines the activation function of the input nodes in the neural network. The
functions are: linear, sigmoid, tanh, evsail-sigmoid, tanh-cubic, step, signed.
It defines the policy for handling the recurrent connections. The supported
values are:
Disallowed: The mutations that cause a loop in the neural network are not
allowed.
BestGuess: Depending on a layer-based approximation of the neural network
topology, connections that potentially cause recurrent loops cache values
from the previous activation to avoid deadlock.
Lazy: All connections cache values from the previous activation.
The number of cycles a neural network with recurrency is activated after
receiving input before output values are read.
It indicates the open-source neural network framework that the developers use
to build networks from chromosomes.

•
•

recurrent.cycles(int)

1

ann.type(string)

ANJI

•

Algorithm 2. The fitness function to perform an agent’s action

Algorithm 2 show the pseudo-codes of our action selection method and
calculated fitness function, respectively.

while(maxTimeStep!=9999){
GenerateCrowd();
for(int i = 1; i < number_of_agents; i++){
PerformAction();
ShareStatus()
}
for(int j = 1;j < number_of_agents;j++){
CollisionAvoidance();
}
for(int k = 1;k < number_of_agents;k++){
make_decision();
correct_speed();
update_position();
if(agent reaches exit){
remove_agent();
}
}
for(int t = 1;j < number_of_agents;t++){
d1 = find_distance(agent.coordinates,
agentTargetCoordinates);
d2 = find_distance(0, 0, height, width);
d3 = find((1-d1)/d2);
numFit = increase_number_of_Fitness;

fitness/=nmFit;
findTimeFitness((maxTime-usedTime)/maxTimeSteps);
finalFitnessValue = TimeFitness∗0.3 + fitness∗0.3;
}
}

Algorithm 1. Action selection algorithm for an agent
Define a as an agent array
Define N as the number of neighbour agents
Define k as the number of agent actions within a fixed sensing range
for(int i = 1; i ≤ N; i++){
GetSensorInputs(a[i]);
while(didreachtarget(a[i])){
if(new action occured){
Compute(neighbours(a[i]));
Compute(constrained(a[i]));
for(int k = 0; k ≤ N; k++){
CalculatedReward[k]=EvaluateAction(k);
}
Motivation = FindBestReward(CalculatedReward);
PerformAction(a[i], Motivation);
ShareStatus(a[i], Motivation);
}
newReward = CollisionAvoidance(a[i], Motivation);
UpdateStatus(a[i], newReward);
}
Remove(a[i]);
}

40

Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

force approaches but in general, they are very expensive. We researched
into algorithms which are fast and inexpensive, easy to understand and
flexible. In our study, the agents have several sensors that sense collisions with other agents or obstacles. Each of these sensors detect the
number of collisions it is having with its surroundings. The number of
collisions from each sensor, along with a bias and an angle between the
agent’s target and the agent, these parameters act as inputs to the ANN.
The ANN then generates two outputs which are the speed correction
and the degree (or angle) correction. These outputs are used to update
the speed and the direction of each agent. As soon as the agent comes
near the target by a particular distance, then, it is considered to have
reached the target.
For the collision detection, we were inspired by ray tracing algorithm which is commonly used. A ray is represented using a vector
which has a start point and a vector which represents the direction in
which the ray travels. The ray starts from the start point and travels in
the direction of the direction vector. The equation for the ray is:

Table 2
Computer system specifications used for our simulation model.
Hardware

Specification

Display
Processor

Memory
Storage
Graphics

27 inch (diagonal) Retina 5 K
4.2 GHz quad-core Intel Core i7, Turbo Boost up to 4.5 GHz
32 GB 2400 MHz DDR4
1 TB SSD
AMD Radeon Pro 580 with 8 GB VRAM

Software

Specification

Operating System

macOS High Sierra (Version 10.13)

In our model, we applied an initial configuration with a set of evolutionary parameters that are presented in Table 1 to evolve a good ANN.
The agents choose right behaviors, communicate with each other and
make their movements depending on the corrections given by the ANN
that was evolved through our fitness function class. In addition, elitism
was selected as we wanted the genetic algorithm to converge more
quickly. With elitism, the fittest chromosomes in each generation are
guaranteed to be copied unchanged into new population. This means
that the fittest genome will never be lost to random chance.

PoR = RSP + (t ∗RD)

(1)

In Eq. (1); PoR is a point on the ray, RSP is the starting point of the ray,
and RD is the direction of the ray. t is a float which takes values from 0
to infinity. If t = 0 , we have the start point and substituting other values
we have the corresponding points along the ray. For the agent, obstacle
collision, we used Ray-Plane Intersection Detection technique. The
plane is represented using its vector representation as follows:

3.4. Obstacle detection and collision avoidance

Xn · X = d

Collision detection and response are difficult tasks and there are not
easy solutions for these problems. For every application, there is a
different way of finding and testing for collisions. There are also brute

(2)

In Eq. (2); Xn and X are vectors. Xn is its normal and X is a point on its
surface. d is a float representing the distance of the plane along the

Fig. 10. Status of the agents at the beginning of the simulation.

41


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

Fig. 11. Status of the agents while the simulation is running (after 60 ss).

20 m × 80 m × 5 m). The agents are randomly given a start position.
Each individual agent is trained to find a best suitable route and reach
the nearest target to itself. Each individual moves from a location to
another by a speed uniformly distributed on [0.2 m/s, 1.6 m/s]. In
addition, the diameter of each individual is uniformly distributed on
[0.3 m, 0.5 m]. The simulation is initiated when an emergency evacuation situation caused by an incident occurs. None of the agents are
aware of the emergency status at the beginning of the simulation. When
the incident takes place around the crowd in the environment, our
trained agents become aware of the incident, develop their behaviors
(i.e., choosing and applying right action, communicating, collaborative
learning, switching to emergency state, escaping), try to get away from
the region at a certain speed in reaction to the incident, and reach the
targets.
First, we carried out an experimental study of a scenario of 200
pedestrians for our model. We then simulated the scenario by increasing the number of pedestrians. Finally, we compared our simulation model with common pedestrian simulation software tools for the
scenarios of 250 and 500 pedestrians, respectively. Fig. 10 illustrates
our simulation for a scenario of 200 pedestrians. After starting the simulation and finding the right number of generations and the population size, the agents developed their skills and headed towards the
exists (Figs. 11, 12). Towards the end of the simulation as shown in
Fig. 13, the agents successfully reached their targets.
Table 3 indicates the number of evacuees from the exits at specific
time intervals. Fig. 14 illustrates the density at targets depending on the
time. Consequently, our simulation model is capable of evacuating 200
individuals in 208 s for the first scenario, as shown in Fig. 15.

normal, from the center of the coordinate system. If the ray intersects
the plane at a point, therefore there will be some points on the ray
which satisfies the plane equation as follows:

(Xn ·RSP ) + t ∗ (Xn ·RD) = d

(3)

In Eq. (3); t represents the distance from the start until the intersection
point along the direction of the ray. Substituting t into the ray equation,
we find our collision point. After we found where the collision takes
place, we find the intersection in the current time step which is the time
we move our agent from its current point according to its velocity. After
determining the time step, we move our agent, calculate its new position and find the distance between the start and end point. Additionally,
we find the distance from the start point to its collision point. If this
distance is less than the distance between start and end points, we
conclude that there is a collision and avoid it.
4. Results
In our simulation, the test platform is a personal computer which its
system specifications are summarized in Table 2. We simulated our
model for a public indoor environment with multiple exits. We can
change environmental features depending on our needs and add new
items to the test area. It is also possible to work with real floor plans and
3D models. The environment consists of agents that represent pedestrians, an indoor area of 100 m × 100 m, one arrival line where crowd
is generated, three targets (w/h: 1.5 m/2.1 m), obstacles which may be
walls, agents or any solid object in between the agents and exits (there
are three obstacles and their dimensions are: O1_w/l/h:
20 m x 30 m x 5 m,
O2_w/l/h:
30 m × 20 m × 5 m,
O3_w/l/h:
42

Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

Fig. 12. Status of the agents while the simulation is running (after 120 ss).

Our methodology and findings demonstrate that neuroevolution
techniques are powerful and play significant roles in areas where supervised learning is limited or not possible. During our tests, we saw
that when we ran the evolution for 20 generations, the fittest chromosomes were the champions of 16 previous generations. It would be
better if there was a way of stopping the evolution process once the
fitness reaches a plateau. This would also decrease the evolution time.
Figs. 16 and 17 show the fitness values for the generations of two different trials. Minimum and maximum fitness for each generation are
denoted with blue⁎ line. The average fitness for the population is indicated with black dot.
We found out that as the environment becomes more complex, the
evolution took more time. This is understandable as the fitness function
became more complex. In our simulation, we first started without any
obstacles and with less number of agents; the evolution was fast in
reaching the desired number of generations. We then added multiple
exits into the environment, and thus, the time spent for each generation
increased. Furthermore, we added obstacles and targets near the obstacles; at this stage, the evolution took around seven minutes per
generation (population size being 200). We observed that choosing the
right set of input values as the first step is important than exploring the
appropriate fitness function. As a result, the parameters and threshold
values can be adjusted. Additionally, we experimented with sigmoid
activation function and tanh function, and compared them with each
other. We saw that the sigmoid function performed better than the tanh

function. For 15 trials, the difference was not significant. However, the
results agree with the common belief that the sigmoid function performs better. We also experimented with the population size and observed how it affected the performance. The simulation was not very
successful with the population size 25. Furthermore, the agents were

not able to reach the targets and they did circular movements. When we
increased the population size to 50 without changing the number of
generations, the agents began to perform better and moved faster.
However, they were not able to reach the targets. We also tried 100 and
200 for the population size and saw that the performance was better
than the previous ones. The agents moved faster and reached their
targets successfully. Increasing the number of generations also affected
the performance of our solution. In our experiments, we increased the
number of generations from 25 to 50 and at the same time we increased
the population size from 50 to 100. We observed that the agents’ performances improved, their skills developed, and they found their ways
to the targets more effectively.
4.1. Performance comparison
In this section, we presented performance evaluations of our model
and some of the known software tools (AnyLogic, MassMotion,
Pedestrians Dynamics, PTV Viswalk, STEPS, and VADERE). These
evaluations emphasize the applicability and efficiency in the evacuation
domain to compare the contribution of our approach with others. We
implemented our method to analyze the learning process and verify the
quality of generated pedestrian dynamics. We first configured the elements (computational components that serve specific purposes) of our

⁎
For interpretation of color in Figs. 16 and 17, the reader is referred to the web version
of this article.

43


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

Fig. 13. Status of the agents towards the end of the simulation (after 180 ss).

comparison, we used similar configurations and settings for the mentioned software tools considering the specifications of our evacuation
domain. In all test cases, we observed that our model produced similar
results with others, however, agents trained with NEAT tend to adapt
their behaviors rather fast in order to achieve their goals. Furthermore,
our model provided smooth flows and realistic motions thanks to NEAT.
In low-density crowd, all simulation models worked very well. As soon
as the environment became more complex, all software tools produced
some unrealistic motions at first. To increase the realism of simulations,
we changed the characteristics of agents, varied the speeds among
agents as well as the waiting-time of each agent and the size of its
personal space. In both our model and other mentioned software tools,
we noted that the agents speed up, slow down or change their orientations to reach their goals. Although no significant differences were
found for average travel times and the speeds of individuals, our model
led to considerably short evacuation time. Besides the quality of generated behaviors, we were also greatly interested in the performance of
our proposed model. Since, on the average, CPU and memory usages
were not too busy for each time step of the simulation, we experimentally confirmed that our NEAT-based simulation model can be applied to pathfinding and evacuation modeling efficiently.
Figs. 18 and 19 show performance evaluations of AnyLogic multimethod simulation modeling software. In our tests, AnyLogic was
capable of evacuating 250 individuals in 238 s for the scenario of 250
pedestrians, and 500 individuals in 293 s for the scenario of 500 pedestrians.
Figs. 20 and 21 present performance evaluations of MassMotion
crowd simulation and pedestrian analysis software. In our tests,

Table 3
Number of evacuees at targets.
Time (s)

Target-1-

Target-2-

Target-3-

Total number of
evacuees

Average memory
usage (MB)

Startup
30
60
90
120
150
180
208

0
0
1
25
22
12
1
0

0

0
0
6
14
13
12
12

0
0
3
31
30
18
0
0

0
0
4
62
66
43
13
12

95
165
124
266

175
296
149
238

software framework, determined optimal settings, and calibrated NEAT
parameters of our model to carry out simulation scenarios. During tests,
we observed crucial situations in the scenarios and analyzed the behaviors of agents for each time step of the simulation. We devised a
number of quantitative quality metrics and test-cases. In particular, we
examined the density at targets, distribution of evacuation, average
evacuation times, crowd flow, cost-efficient (time-optimal) paths, and
average speeds. Total acceleration of an agent and degrees it turned
were also examined to get an idea of the amount of movement and
turning effort spent by the agent.
To demonstrate the usability of our model in evacuation modeling,
we performed some experiments through creating scenarios consisting
of 250 and 500 pedestrians, respectively. Each agent enters the environment through an arrival gate, communicates if necessary, avoids
obstacles, and moves towards randomly selected exits. For a fair
44


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

Fig. 14. Density at targets.

Fig. 15. General evacuation situation.

Fig. 16. Trial 1.

Fig. 17. Trial 2.

MassMotion was capable of evacuating 250 individuals in 229 s for the
scenario of 250 pedestrians, and 500 individuals in 257 s for the scenario of 500 pedestrians.
Figs. 22 and 23 illustrate performance evaluations of Pedestrian
Dynamics crowd simulation and modeling software. In our tests, Pedestrian Dynamics was capable of evacuating 250 individuals in 256 s
for the scenario of 250 pedestrians, and 500 individuals in 276 s for the
scenario of 500 pedestrians.

Figs. 24 and 25 show performance evaluations of PTV Viswalk pedestrian traffic simulation software. In our tests, PTV Viswalk was
capable of evacuating 250 individuals in 253 s for the scenario of 250
pedestrians, and 500 individuals in 281 s for the scenario of 500 pedestrians.
Fig. 26 and 27 present performance evaluations of STEPS pedestrian
movement software tool. In our tests, STEPS was capable of evacuating
45


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

a) Number of evacuees at targets

b) Density at targets

c) Distribution of evacuation

d) General evacuation situation

Fig. 18. Evacuation scenario of 250 pedestrians in AnyLogic Professional 8.2.

a) Number of evacuees at targets

c)

b) Density at targets

Distribution of evacuation

d) General evacuation situation

Fig. 19. Evacuation scenario of 500 pedestrians in AnyLogic Professional 8.2.

46


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

a) Number of evacuees at targets

b) Density at targets

c) Distribution of evacuation

d) General evacuation situation

Fig. 20. Evacuation scenario of 250 pedestrians in MassMotion 9.0.

a) Number of evacuees at targets

b) Density at targets

c) Distribution of evacuation

d) General evacuation situation

Fig. 21. Evacuation scenario of 500 pedestrians in MassMotion 9.0.

47


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

a) Number of evacuees at targets

b) Density at targets

c) Distribution of evacuation

d) General evacuation situation

Fig. 22. Evacuation scenario of 250 pedestrians in Pedestrian Dynamics 3.1.

a) Number of evacuees at targets

b) Density at targets

c) Distribution of evacuation

d) General evacuation situation

Fig. 23. Evacuation scenario of 500 pedestrians in Pedestrian Dynamics 3.1.

48


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

a) Number of evacuees at targets

b) Density at targets

c) Distribution of evacuation

d) General evacuation situation

Fig. 24. Evacuation scenario of 250 pedestrians in PTV Viswalk 10.

a) Number of evacuees at targets

b) Density at targets

c) Distribution of evacuation

d) General evacuation situation

Fig. 25. Evacuation scenario of 500 pedestrians in PTV Viswalk 10.

49


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

a) Number of evacuees at targets

b) Density at targets

c) Distribution of evacuation

d) General evacuation situation

Fig. 26. Evacuation scenario of 250 pedestrians in STEPS 5.4.

a) Number of evacuees at targets

b) Density at targets

c) Distribution of evacuation

d) General evacuation situation

Fig. 27. Evacuation scenario of 500 pedestrians in STEPS 5.4.

50


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

a) Number of evacuees at targets

b) Density at targets

c) Distribution of evacuation

d) General evacuation situation

Fig. 28. Evacuation scenario of 250 pedestrians in VADERE.

a) Number of evacuees at targets

b) Density at targets

c) Distribution of evacuation

d) General evacuation situation

Fig. 29. Evacuation scenario of 500 pedestrians in VADERE.

51

Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

a) Number of evacuees at targets

b) Density at targets

c) Distribution of evacuation

d) General evacuation situation

Fig. 30. Evacuation scenario of 250 pedestrians in our NEAT-based simulation model.

a) Number of evacuees at targets

b) Density at targets

c) Distribution of evacuation

d) General evacuation situation

Fig. 31. Evacuation scenario of 500 pedestrians in our NEAT-based simulation model.

52


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

b) For 500 pedestrians

a) For 250 pedestrians

Fig. 32. Comparison of evacuation time and general evacuation situation.

Table 4
Performance comparison of our model with other simulation software tools.

Our Model
AnyLogic
MassMotion
Pedestrian Dynamics
PTV Viswalk
STEPS
VADERE

Evacuation Time (s)
For 250 pedestrians

For 500 pedestrians

Average Memory Usage
For 250 pedestrians

For 500 pedestrians

Average CPU Usage (%)

For 250 pedestrians
For 500 pedestrians

210
238
229
256
253
247
220

254
293
257
276
281
265
298

238 MB
390 MB
395 MB
410 MB
454 MB
374 MB
1.25 GB

297 MB
475 MB
418 MB

456 MB
487 MB
430 MB
1.32 GB

57
85
45
53
61
92
58

62
89
48
59
64
96
65

Table 5
Comparison of different software systems for evacuation modeling and simulation.
Evacuation model

Behavior method

Collision response

Communication

Learning

Roles (Individuals)

Real time

Spatial structure

Acumen

Particle based

Cont.

AnyLogic

Rule based

Cont.

Artificial Fishes

Rule based

Cont.

Autonomous Pedestrians

Artificial life approach

Cont.

Cellular Automata

Cellular automata

2D Grid

Crosses

Rule based + FSM

Egress

Cellular automata

some

Hexagonal Grid

Exodus

Cellular automata

some

2D Grid

Legion

Least-effort

some

Cont.

Maces + Hidac

Social forces

Massive

Rule based + Fuzzy Logic

OpenSteer

Rule based

Reactive Navigation

Rule based

Rule-based

Rule based

Simulex

Distance maps

some

Cont.

Social Forces

Socail forces

some

Cont.

Space Syntax

Visibility graphs

Steps

Cellular automata

Vadere

Optimal Steps Model

2D Grid

ViCrowd

Rule based + FSM

Cont.

Our Model

NEAT

Cont.

Cont.

some

Cont.
Cont.
some

Cont.
Cont.
Cont.

Cont.
some

2D Grid

250 individuals in 220 s for the scenario of 250 pedestrians, and 500
individuals in 298 s for the scenario of 500 pedestrians.
Figs. 30 and 31 show performance evaluations of our NEAT-based
simulation model. Our model was capable of evacuating 250 individuals in 210 s for the scenario of 250 pedestrians, and 500

250 individuals in 247 s for the scenario of 250 pedestrians, and 500
individuals in 265 s for the scenario of 500 pedestrians.
Figs. 28 and 29 illustrate performance evaluations of VADERE (open
source microscopic pedestrian dynamics simulation framework) with
Optimal Steps Model. In our tests, VADERE was capable of evacuating
53


Advanced Engineering Informatics 35 (2018) 30–55

M.E. Yuksel

processes, and demonstrated that this neuroevolution method is feasible to apply. In our simulation scenarios, agents were able to accomplish their tasks and found their ways to exists, that means topology
evolution by using NEAT is efficient, provides advantage and can be
applied in the area of evacuation modeling. Consequently, our approach provides a good solution that helps decision-makers to meet the
evacuation modeling and tactics development goals. It is less complex
than analytical models. It provides a NEAT-based software tool to study
the behavior of evacuees without incurring the cost of performing and
analyzing real-time evacuation operations, and to develop alternative
methods to evacuation simulation models before a commitment is made
to one. Decision-makers, evacuation managers, safety planners, researchers can develop their systems through our open source software
platform to design and simulate various what-if scenarios, and generate
different evacuation plans under different conditions by examining
their results.

individuals in 254 s for the scenario of 500 pedestrians.
Figs. 32 represents the comparison of evacuation time and evacuation situation between our model and other mentioned simulation
software tools. Furthermore, Table 4 presents overall performance
evaluation of our model and these tools. Consequently, our experimental studies and results demonstrate significant improvement in

evacuation rates when using NEAT. This may shed light to the studies of
pedestrian dynamics and crowd simulation.
5. Conclusion
There are various approaches to analyze evacuation process and
minimize the evacuation time based on the level of abstraction and the
level of complexity. In addition, several models involve simulation
techniques to determine the flow conditions and traffic patterns depending on a set of rules. These models can estimate the evacuation
time as a function of flow. They act as a software framework in order to
evaluate private scenarios and generate solution recommendations.
Table 5 presents a comparison of some of the most significant software
tools in pedestrian dynamics and crowd simulation at a functional level.
This emphasizes the basic features in evacuation modeling to compare
the contribution of our model with others.
The main problem in evacuation simulation models is that they do
not have the ability to analyze pedestrian and crowd behaviors, estimate the evacuation time, and choose the best suitable routes that
provide safe access and egress. This observation, actually, sparked the
following state-of-the-art approach in our study; if we are able to build
an agent based model that integrates an optimization method and a
simulation routine for a specific scenario, then significant improvements to evacuation modeling can be achieved. Therefore, we developed a multi-agent based simulation model with NEAT to study the
pedestrian dynamics, analyze the crowd flow and group behaviors,
minimize the evacuation time in an emergency evacuation situation.
In our study, we use NEAT to train autonomous agents in the evacuation domain, discover highly sophisticated and successful strategies,
and solve challenging tasks such as navigation, communication, obstacle detection, and collision avoidance. NEAT supplies GAs to evolve
increasingly complex ANN architectures effectively. Evolution starts
with small and simple ANNs, and as evolution progresses, ANNs turn
into more complex structures over generations (characteristic features
of the ANN’s topology increase, either by adding a new neuron into a
connection path or by adding a new connection between neurons).
NEAT starts without any hidden layer and then adjusts weights while
adding hidden neurons and further connections. This process of starting

with small networks and allowing evolution to perform complexification helps to find complex solutions (behaviors) and solve the
problems faster. Hence, our approach is used to model the increased
complexity and detail effectively. It overcomes the limitations of analytical techniques by handling the varying levels of complexity through
NEAT (i.e., peak loads and transient behaviors can be examined). Since
it represents system more faithfully than macroscopic and microscopic
models, our model is well-defined, well-designed, well-established, and
can be more common in practice. It provides numerous options to design a system using NEAT. The realism degree incorporated into the
model directly reflects the level of detail it incorporates. Sources of
performance fluctuations can be identified by varying the design
parameters.
Unfortunately, it is usually not possible to evacuate people by carrying out collectively live practices and safety procedures. Because
conducting live exercises and safety procedures is a time-consuming
process and results in tremendous lost revenue. Therefore, developing a
feasible strategy for the evacuation of people is very crucial. In this
paper, we presented an agent-based simulation model based upon
NEAT that can be used to help the authorities or emergency management personnel to develop evacuation strategies. By employing the
NEAT, we analyzed the agents’ behaviors, studied the learning

Appendix A. Supplementary material
Supplementary data associated with this article can be found, in the
online version, at />References
[1] B. Abdulhai, L. Kattan, Reinforcement learning: introduction to theory and potential
for transport applications, Can. J. Civil Eng. 30 (6) (2003) 981–991.
[2] J. Allbeck, K. Kipper, C. Adams, W. Schuler, E. Zoubanova, N.I. Badler, M. Palmer,
A.K. Joshi, ACUMEN: Amplifying Control and Understanding of Multiple ENtities,
in: Proceedings of the First International Joint Conference on Autonomous Agents
and Multiagent Systems, Bologna, Italy, 2002, pp. 191–198.
[3] A.G. Barto, S. Mahadevan, Recent advances in hierarchical reinforcement learning,
Disc. Event Dynamic Syst.: Theory Applications 13 (4) (2003) 341–379.
[4] R. Bindiganavale, W. Schuler, J.M. Allbeck, N.I. Badler, A.K. Joshi, M. Palmer,

Dynamically altering agent behaviors using natural language instructions, in:
Proceedings of the Fourth International Conference on Autonomous Agents,
Barcelona, Spain, 2000, pp. 293–300.
[5] S. Bretschneider, Mathematical Models for Evacuation Planning in Urban Areas,
Springer-Verlag, Berlin Heidelberg, Germany, 2013.
[6] M. Busogi, N. Kim, D. Shin, H.B. Ryu, A. Yoo, D. Kim, Bayesian affordance-based
agent model for wayfinding behaviors in evacuation problems, in: Duffy V.G (Ed.),
Digital Human Modeling and Applications in Health, Safety, Ergonomics, and Risk
Management. Healthcare and Safety of the Environment and Transport, SpringerVerlag, Berlin Heidelberg, Germany, 2013, pp 297–306.
[7] N. Chooramum, P.J. Lawrence, E. Galea, Urban scale evacuation simulation using
buildingEXODUS, the 14th International Conference and Exhibition on Fire Science
and Engineering (Interflam 2016), London, UK, 2016, pp. 1645–1656.
[8] A. Cuesta, O. Abreu, D. Alvear (Eds.), Evacuation Modeling Trends, Springer, Cham,
Switzerland, 2016.
[9] S. Curtis, Pedestrian velocity obstacles: Pedestrian simulation through reasoning in
velocity space, PhD Thesis University of North Carolina at Chapel Hill, NC, USA,
2013.
[10] S. Curtis, A. Best, D. Manocha, Menge: a modular framework for simulating crowd
movement, Collect. Dynamics 1 (2016) 1–40.
[11] S. Curtis, D. Manocha, Pedestrian simulation using geometric reasoning in velocity
space, in: U. Weidmann, U. Kirsch, M. Schreckenberg (Eds.), Pedestrian and
Evacuation Dynamics 2012, Springer, London, UK, 2014, pp. 875–890.
[12] D.B. D'Ambrosio, K.O. Stanley, Scalable multiagent learning through indirect encoding of policy geometry, Evol. Intel. 6 (1) (2013) 1–26.
[13] T. D’Silva, R. Miikkulainen, Learning dynamic obstacle avoidance for a robot arm
using neuroevolution, Neural Process. Lett. 30 (1) (2009) 59–69.
[14] M.J. Dibley, H. Li, J.C. Miles, Y. Rezgui, Towards intelligent agent based software
for building related decision support, Adv. Eng. Inf. 25 (2) (2011) 311–329.
[15] F. Dietrich, S. Disselnkötter, G. Köster, How to get a model in pedestrian dynamics
to produce stop and go waves, in: V. Knoop, W. Daamen (Eds.), Traffic and Granular
Flow'15, Springer, Cham, 2016, pp. 161–168.

[16] F. Dietrich, G. Köster, Gradient navigation model for pedestrian dynamics, Phys.
Rev. E 89 (6) (2014) 062801.
[17] J. Dijkstra, H.J.P. Timmermans, A.J. Jessurun, A multi-agent cellular automata
system for visualizing simulated pedestrian activity, in: S. Bandini, T. Worsch
(Eds.), Theoretical and Practical Issues on Cellular Automata, Springer, London, UK,
2000, pp. 29–36.
[18] H.-S. Gana, K.-F. Richter, M. Shic, S. Winterd, Integration of simulation and optimization for evacuation planning, Simul. Model. Pract. Theory 67 (2016) 59–73.
[19] E. Gelenbe, H. Bi, Emergency navigation without an infrastructure, Sensors 14 (8)
(2014) 15142–15162.
[20] E. Gelenbe, Y. Cao, Autonomous search for mines, Eur. J. Oper. Res. 108 (2) (1998)
319–333.
[21] E. Gelenbe, K. Hussain, V. Kaptan, Simulating autonomous agents in augmented
reality, J. Syst. Softw. 74 (3) (2005) 255–268.
[22] E. Gelenbe, F. Wu, Large scale simulation for human evacuation and rescue,
Comput. Math. Appl. 64 (12) (2012) 3869–3880.

54