- ARTÍCULO PREVIAMENTE PUBLICADO - Abstract. Evolutionary Algorithms (EAs) are frequently used as a mechanism for the optimization of autonomous agents in games (bots), but knowing when to stop the evolution, when the bots are good enough, is not as easy as it would a priori seem. The first issue is that optimal bots are either unknown (and thus unusable as termination condition) or unreachable. In most EAs trying to find optimal bots fitness is evaluated through game playing. Many times it is found to be noisy, making its use as a termination condition also complicated. This paper summarizes our previous published work where we tested several termination conditions in order to find the one that yields optimal solutions within a restricted amount of time, to allow researchers to compare different EAs as fairly as possible. To achieve this, we examined several ways of finishing an EA who is finding an optimal bot design process for a particular game, Planet Wars in this case, with the characteristics described above, determining the capabilities of every one of them and, eventually, selecting one for future designs.

1. An overview on the
termination conditions in the
evolution of game bots
A. Fernández-Ares, P. García-Sánchez, A.M. Mora,
P.A. Castillo, J.J. Merelo, M.G. Arenas
Universidad de Granada (Spain)

2. A summary of previous works
Pablo García-Sánchez, Antonio Fernández-Ares, Antonio Miguel Mora,
Pedro A. Castillo Valdivieso, Jesús González, Juan Julián Merelo Guervós:
Tree Depth Influence in Genetic Programming for Generation of
Competitive Agents for RTS Games. EvoApplications 2014: 411-421
Antonio Fernández-Ares, Pablo García-Sánchez, Antonio Miguel Mora,
Pedro A. Castillo, Juan Julián Merelo Guervós: Designing competitive
bots for a real time strategy game using genetic programming.
CoSECivi 2014: 159-172
Antonio Fernández-Ares, Pablo García-Sánchez, Antonio M. Mora, Pedro
A. Castillo Valdivieso, Juan Julián Merelo Guervós, Maria I. García Arenas,
Gustavo Romero: It's Time to Stop: A Comparison of Termination
Conditions in the Evolution of Game Bots. EvoApplications 2015: 355-
368

3. Index
1. Introduction
a. RTS
b. Planet Wars
c. Genetic
Programming
2. GPBot
a. Conditions
b. Actions
3. SCORE - Fitness
4. Methodology
a. Termination Criteria
5. Experimental Setup
6. Results
a. Score
b. Generations
c. Tests
d. Benchmark
e. Comparative
7. Conclusions
8. Future Work

4. Introduction: RTS
Real-Time Strategy games (RTS-games)
● Resources
● Units
● Buildings
Victory: Get all resources, kill all enemy units or
destroy enemy buildings

5. Introduction: Planet Wars
Buildings:
Planets
Units:
Space ships
Resources:
Ships generated in
every planet

6. Introduction: Genetic Programming
Evolutionary Algorithm that evolves binary
decision tree
➔ Internal nodes: Conditions
➔ Leafs: Actions
Individual -> behavioural model (solution)
Evaluation -> Playing a game against a rival and getting
a score
Adapted operators

7. GPBot: Conditions
A logical expression composed by, at least, one
extracted standard game state variable and a
value between 0 and 1
● myShipsEnemyRatio
● myShipsLandedFlyingRatio
● myPlanetsEnemyRatio
● myPlanetsTotalRatio
● actualMyShipsRatio
● actualLandedFlyingRatio

8. GPBot: Actions
The possible actions just involve the movement
of an amount of ships from a source to a
destination planet
● Attack Nearest (Neutral|Enemy|NotMy)Planet
● Attack Weakest (Neutral|Enemy|NotMy) Planet
● Attack Wealthiest (Neutral|Enemy|NotMy)
Planet
● Attack Beneficial (Neutral|Enemy|NotMy)
Planet
● Attack Quickest (Neutral|Enemy|NotMy) Planet}
● Attack (Neutral|Enemy|NotMy) Base
● Attack Random Planet
● Reinforce Nearest Planet
● Reinforce Base
● Reinforce Wealthiest Planet

9. Score (Fitness)
SCORE
BASED IN
TURNS IN
VICTORIES
SCORE
BASED IN
TURNS IN
DEFEATS
SCORE
BASED IN
VICTORIES
> >

10. Methodology: Termination Criteria
[NG] Number of Generations
➔ 30, 50, 100, 200 Generations
[AO] Age of Outliers
➔ 1, 1.5, 2, 2.5 times the IQR
[RT] Replacement Rate
➔ n/2, n/4, n/8, n/16 with n = N/2
[FT] Fitness Threshold
➔ 22, 24, 26, 28
[FI] Fitness Improvement
➔ 3, 7, 10, 15

11. Experimental Setup
Parameter Name Value
Population Size (N) 32
Number of battles for scoring (NB) 30
Re-evaluation of individuals Yes
Crossover type Sub-tree crossover
Crossover rate 0.5
Mutation 1-node mutation
Mutation rate 0.25
Selection 2-tournament
Replacement Generational
Maximum Tree Depth 7
Runs 36

12. SCORE of the best individuals (of all runs) grouped by criterion

13. GENERATION attained (of all runs) grouped by criterion

14. Kruskal-Wallis Test of samples of each criterion by SCORE an
GENERATIONS.
Black means no statistically significant difference has been found.

15. Percentage of victories (of the 36 best individuals) in benchmark against five
different competitive bots available in the literature in 100 maps

16. Linear regression of the SCORE (fitness) with the results of the benchmark
(Percentage of victories)

17. Average score of the best individual
and average reached generations per
termination criteria
Average results of every criterion
(relative with respect to NG_30)

18. Conclusions I
[AO] Not good. As more restrictive
values don’t imply a significant score
improvement.
[FT] Best score. But the optimum fitness
might not be know. Needs the highest
amount of generations. It might not finish.
[FI] Useful to ‘detect’ local optima.
[RT] Provides the best results
considering all metrics.

19. Conclusions II
Fitness Threshold would be the most desirable
option
➔ It attains the best score.
➔ It is quite difficult to find an optimal fitness value to
use (normally it is unknown).
➔ Requires more computational budget, and it is
possible that it never ends (the criterion is not met).
Replacement Rate as stopping criterion, since it is a
compromise solution which relies in the population
improvement without an implicit use of the fitness.

20. Future work
● New problems (and algorithms) are being
addressed.
○ P. García-Sánchez, Alberto Tonda, Giovanni
Squillero and JJ. Merelo: Towards Automatic
StarCraft Strategy Generation Using Genetic
Programming. Accepted at CIG 2015.
● Mechanisms to improve the EA.
● Use larger and complex decision trees.

21. Questions
Thank you!
Antonio Fernández-Ares
antares@ugr.es - @antaress
Antonio M. Mora
amorag@ugr.es - @amoragar
Pablo García-Sánchez
pablogarcia@ugr.es - @fergunet