Presentation given to students from IES Montes Orientales about our work on computational intellgence for games by Geneura group of the University of Granada.

1. This was a triumph: Evolving
intelligent bots for videogames.
And for Science.
Pablo García Sánchez @fergunet
@geneura
University of Granada

2. Index
• Introduction
• Evolutionary
Algorithms
• FPS
• RTS
• Content
Generation
commons.wikipedia.org

3. Evolutionary
Algorithms
hideyourarms.com

4. What are the Evolutionary
Algorithms?
• Metaheuristics
• Solve Optimization problem
• Based on a fitness function
• Population based
• Natural selection
• Stochastic

5. Individuals
No Yes Yes No Yes No F=3
BlueVenom
Fire Fly
Fangs
Horns

6. Mating
No Yes Yes No Yes No
No Yes No No Yes Yes

7. Offspring
No Yes Yes
No Yes NoNo Yes No
No Yes Yes
F=2
F=4

8. Mutation
No Yes Yes No Yes Yes F=4Yes F=5

9. EA stages

10. Example: Super Mario Bot
• The actions of
Mario are
Evolved
• VIDEO TIME!

11. FPS
commons.wikipedia.org

12. Unreal Tournament 2004
• Edited with UnrealScript
• Java engine to access the game: POGAMUT

13. Unreal Bot
• Parameter optimization of a FSM to play Unreal
Tournament 2004
• Each chromosome is a vector of parameters

14. UnrealBot Team
• The same, but optimizing the parameters of a team.

15. Unreal ExpertBot
• Two-level FSM based in the experience of a expert player (Andalusia
champion) in 1vs1 matches.
• For example, it models which weapon to use. It wins “normal” human
players.

16. Unreal InteractiveBot
• From ExpertBot
• Experts perform actions
during the algorithm
• Expert player: answer
questions to block some
parts of the chromosome
• Expert researcher
• Second position
2KBotPrice 2014

17. RTS
commons.wikipedia.org

18. Real Time Strategy (RTS)
• Road traffic, finances or weahter forecasting:
• Complex and dynamic environments
• No completed nor perfect information
• RTS->simplification
• Uncertainty decisión taking
• Real time plannification

19. RTS features
• Simultaneous movement
• Durative actions (require time)
• Real time (quick decisions)
• Partially observable (fog of war)
• Undeterministic
• Complex

20. Challenges in RTS
• Planning
• Learning
• Uncertainty
• Spacial and temporal
reasoning
• Domain knowledge
exploitation
• Task decomposition
andreita42 en deviantart

21. Challenges: Planification
• Different abstract levels
• Long term strategy (match)
• Short term strategy (battles)
• Techniques: sampling or
hierarchical decomposition
pixabay.com

22. Challenges: Learning
• Existent learning (ex. matches
DB)
• In-game learning (ex.
reinforcement learning)
• Inter-game learning (ej. learn
from previous game to win
the next)
soft9000.com

23. Challenges: Uncertainty
• Spatial uncertainty (fog of
war)
• Enemy action
commons.wikipedia.org

24. Challenges: Reasoning
• Spatial reasoning: where to
build the base
• Temporal reasoning: when to
create technology
fc02.deviant.org

25. Challenges: task
decomposition
• Strategy
• Tactics
• Reactive control
• Terrain analisys
• Intelligent gathering
• Micro and macro tasks
insightly.com

26. Planet Wars
gaskin1982.deviantart.com

27. Planet Wars

28. Problem description
• Restrictions:
• 1 second to decide
• No memory can be used
• Current state: planets and fleets
• Actions: a text file read by the game
Current
state
Actions to
perform
Bot

29. AresBot
• Select base
• Select objective
• For each colony:
• Reinforce base
• Reinforce attack
(este es Antares)

30. FSM

31. Genebot Operation
• AresBot parameter
optimization
• Chromosome: parameter
array
• 2-Tournament
• BLX-alpha crossover
• Re-evaluation
• 40 seconds to evaluate an
individual
Propiedad de EA (no me demandéis!)

32. Exp-Genebot
• Detect the type of map to chose a previously
optimized set of parameters

33. GP-bot
• New approach: Genetic Programming
• The genome is a TREE
• Decisions: e.g. if distance to the base is lower than
0.25
• Actions: e.g. attack the base with the 20% of ships
• Java code compiled in each evaluation using CAssist

34. Example of a chromosome
(tree)
actualMyShipsRatio>0.711
attackWeakestNeutralPlanet(0.3)
attackNearestEnemyPlanet(0.2)
attackEnemyBase(0.91)
myShipsLandedFlyingRatio>0.2

35. Results (validation)
3G 7G UG 3E 7E UE
100200300400500
Configuration
Turnstobedefeated
3G 7G UG 3E 7E UE
10203040506070
Configuration
Percentageofvictories

36. Stop criteria?
• Number of generations
• Outlier age
• Replacement rate
• Fitness threshold
• Fitness improvement respect to average

37. When to stop?

38. What we learnt?
• Compare with previous version to IMPROVE
• Use hierarchical fitness
• Individuals reevaluation
• Validate always aginst bots/maps not used from
learning
• Replacement based stop criteria
• Use free software

39. StarCraft
Propiedad de Blizzard

40. Why StarCraft?
• Competitions: CIG, AIIDE, SCSC...
• Available frameworks and libraries
• Replays database (ej: http://bwreplays.com/)
• Unified Test-bed in research on RTS

41. BWAPI
• Create AIs (in .dll)
• Information about:
• Game: minerals, map, game type, latency...
• Units: distance, speed, detection, status...
• Player: race, status, resources, deaths...

42. ChaosLauncher

43. Tournament Manager

44. StarCraftGP
• Framework to automatically generate bots to play
Starcraft using Genetic Programming
• Basad on uGP3
• We use Linear-GP to generate strategies for bots using
the uGP3 framework (using OpprimoBot as base)
• In collaboration with INRA Politecnico di Torino

45. External Evaluator
Evolutionary Core
Constraints
.txt
.txt
TournamentManager
External Evaluator
Individual.DLL
results.txtParser
Fitness values
VM Server
VM Client
VM Client
VM Client
…
VStudio Compiler
Individual.cpp

46. Bot example

47. Results

48. Content
Generation
Story generation in massively
distributed environments: MADE
Propiedad de Bethesda

49. The problem

50. MADE
• Massive Artificial Drama Engine
• A GA optimizes a multi-agent system
• Archetype based
• Available in made.velonuboso.com

51. Archetypes
Laffent in deviantart.org

52. Methodology

53. Agent

54. Libraries

58. References (Planet Wars)
• A. Fernández-Ares, A. Miguel Mora, J. J. Merelo Guervós, P. García-Sánchez, C. M. Fernandes:
Optimizing player behavior in a real-time strategy game using evolutionary algorithms. IEEE
Congress on Evolutionary Computation 2011: 2017-2024 (2012)
• A. Miguel Mora, A. Fernández-Ares, J. J. Merelo Guervós, P. García-Sánchez, C. M. Fernandes:
Effect of Noisy Fitness in Real-Time Strategy Games Player Behaviour Optimisation Using
Evolutionary Algorithms. J. Comput. Sci. Technol. 27(5): 1007-1023 (2012)
• Antonio Fernández-Ares, Pablo García-Sánchez, Antonio Miguel Mora, Juan J. Merelo Guervós:
Adaptive bots for real-time strategy games via map characterization. CIG 2012: 417-721 (2012)
• Antonio Fernández-Ares, Pablo García-Sánchez, Antonio M. Mora, Pedro A. Castillo Valdivieso,
Juan Julián Merelo Guervós, Maria I. García Arenas, Gustavo Romero: It's Time to Stop: A
Comparison of Termination Conditions in the Evolution of Game Bots. EvoApplications 2015:
355-368
• Pablo García-Sánchez, Antonio Fernández-Ares, Antonio Miguel Mora, Pedro A. Castillo
Valdivieso, Jesús González, Juan Julián Merelo Guervós: Tree Depth Influence in Genetic
Programming for Generation of Competitive Agents for RTS Games. EvoApplications 2014:
411-421

59. References (Unreal)
• Antonio Miguel Mora, Ramón Montoya, Juan Julián Merelo Guervós,
Pablo García-Sánchez, Pedro A. Castillo, Juan Luís Jiménez Laredo,
Ana Isabel Martínez García, Anna Esparcia-Alcázar: Evolving Bot AI
in Unreal. EvoApplications (1) 2010: 171-180
• Antonio Miguel Mora, Francisco Aisa, Ricardo Caballero, Pablo
García-Sánchez, Juan Julián Merelo Guervós, Pedro A. Castillo,
Raúl Lara-Cabrera: Designing and Evolving an Unreal
TournamentTM 2004 Expert Bot. IWANN (2) 2013: 312-323
• Antonio Miguel Mora, M. A. Moreno, Juan Julián Merelo Guervós,
Pedro A. Castillo, Maribel García Arenas, Juan Luís Jiménez Laredo:
Evolving the cooperative behaviour in Unreal™ bots. CIG 2010:
241-248

60. References (MADE)
• Rubén Héctor García-Ortega, Pablo García-Sánchez, Juan Julián Merelo
Guervós, María Isabel García Arenas, Pedro A. Castillo Valdivieso, Antonio
M. Mora: How the World Was MADE: Parametrization of Evolved Agent-
Based Models for Backstory Generation. EvoApplications 2015: 443-454
• Garcia-Ortega, Ruben H., et al. My life as a sim: evolving unique and
engaging life stories using virtual worlds. ALIFE 14: The Fourteenth
Conference on the Synthesis and Simulation of Living Systems. Vol. 14.
Pages 580-587, 2014
• http://made.velonuboso.com
• @made_project
• @rubenhek

61. Free Software and Open
Science
• GitHub: fergunet, deantares, amorag, jjmerelo,
geneura, geneura-papers
• http://geneura.ugr.es
• Text in this presentation is CC-SA-BY (free license
images used, non commercial, except images from
Blizzard, Bethesda, EA and clippart.co )

62. Thanks!
• @fergunet
• @amoragar
• @geneura
• @antaress