This document discusses using video games for scientific purposes. It describes how video game controllers like the Wii remote and Kinect have been used for robot control, pattern recognition, and other scientific applications. It also discusses how video games incorporate scientific principles like physics and how artificial intelligence is an important area of research for game development. Some specific examples of research include evolving bot AI in Unreal Tournament using genetic algorithms and using games like Pac-Man and StarCraft for AI research challenges.

1. SCIENCE AND VIDEOGAMES
PLAYER 1 – ANTONIO M. MORA GARCÍA
- Press START -
© GAME-ON 2012

2. • Introduction
videogames market, players taxonomy,
current game systems. Videogames at the
University.
• Videogames applied to science
videogame system-based science, engineering
and technology.
• Science in videogames
scientific principles of videogames.
• Researching in videogames
main research fields in videogames.
• Examples: Our works

3. INTRODUCTION

4. • Very big growing of videogames market, due to
their movement to new groups of interest:
people older than 25 and younger than 10,
including parents and grandparents, in
addition to the feminine sector.
• This growing is mainly due to a change in the
videogames philosophy, offering more ‘adult’
contents, or the contrary, easier and child-
focused, in addition to direct and brief action
games.
There is a wide market for science!!!

5. • In the current market has arisen the so-called casual gamers:
sporadic players users of brief and direct action games (arcade, sports, mini-
games), or the so-called no-games (training games, art games, and so on).
• The usual players have auto renamed as hardcore gamers.
They (really) enjoy and profit the games, they are informed, like most types of
games, and play for long periods (if possible).
A friendly way to say “Virgin until the age of 37” More fun if they plug-in the console

6. • A sincere feeling:

7. • In addition to PC and mobile systems (iOS, Android, etc), there are some
extended systems:
• Home consoles
Wii Xbox 360 Playstation 3
• Portable consoles
Nintendo Playstation
3DS Vita

8. • And many more (open-philosphy consoles), not known for most of the people.
GP2X Wiz Caanoo
Dingoo Pandora

9. • Another (positive) consequence is the adaptation of the study plans
for videogames development.
• In Spain there are arising courses in Grades and Masters
• Anyway, we are still far from other countries in Europe:
– Example: Center for Computer Game Research (Copenhagen)

11. SCIENTIFIC
APPLICATIONS

12. Its (at the beginning) novel controller (Wiimote) has been very famous
mainly among the scientific and technical community:
• Robots control, reactive/touch-detecting screens or surfaces, or
pattern/subjects recognition, among others.
http://www.youtube.com/watch?v=v1AJ_OBJUpY http://www.youtube.com/watch?v=TkmxhVtvLoM http://www.youtube.com/watch?v=0awjPUkBXOU

13. There is even a project, WiiLab, which has created a Matlab toolbox for
interacting with Wiimote…
…and with Game Maker (an easy
game development framework)
http://code.google.com/p/giimote/
http://www.youtube.com/watch?v=EeBAYeoX7-8

14. • It was initially used for building
console-based clusters (super
computers), due to the powerful
chip Cell, and the cheap price it
had.
• It was possible to use an
additional Linux O.S. (Yellow
Dog), very flexible.
• But later, the console was
updated for not admitting the
installation of any additional
O.S., so Linux was lost forever.

15. Very famous in the community due to Kinect:
• Robots control by movement and/or voice, pattern and person
recognition, among others.
http://www.youtube.com/watch?v=Sw4RvwhQ73E http://www.youtube.com/watch?v=c6jZjpvIio4

16. SCIENCE IN
VIDEOGAMES

18. • Videogames have always respected physics rules, even a ‘simple’ one
(in appearance) such as Super Mario Bros. (jumps, trajectories,
inertia,…).
• Nowadays the tendency is to develop completely realistic games in
that sense, by implementing specific engines for physics modeling.
http://www.youtube.com/watch?v=B7_rPDwSKe8

19. The scientific principles of operation of the first main controller for
a console based in movement (Wiimote) are:
http://www.youtube.com/watch?v=ETAKfSkec6A

20. RESEARCH
IN VIDEOGAMES

21. • In addition to visual and physics realism, it is desired to model enemies and
partners , with an ‘intelligent’ (human) behavior.
• Thus a big amount of resources have been focused on artificial intelligence.
Realistic Game

22. • AI is the area of computer science devoted to implement
nonliving rational agents (at least in appearance).
• Inside a videogame, AI is focused on defining behavior
techniques for non-playable characters (NPCs), commonly
named bots, which simulate being rational. These characters
could be enemies or partners.
• It is not a matter of literally showing human behavior, since it
means the consideration of mistakes.

23. • In the very beginning, NPCs followed some
predefined behavior patterns, that the
programmer implemented at game
implementation and which were invariable.
• Reactive AIs proposed NPC’s actions as a
response to player’s actions.
• Dedicated AIs set different ‘personalities’
for NPCs.

24. • Later there were introduced the finite state machines, which
define a set of possible states for the NPC, and a set of
transitions between them. Transitions are based in
perceptions about the game or about the players.
By Fergu

25. • Other extended methods include rule-based systems and
decision trees. In both cases, there is a set of rules that the
NPC will follow, depending on the inputs or perceptions about
its environment.

26. • Nowadays it is usual to mix some of these techniques, thus in most games
NPCs follow predefined behavioral models (scripts), depending on player’s
actions.
• Their advantage is that it is easy to define them, considering programmer’s
experience and modeling player’s behavior.
• Their main disadvantage is the low flexibility they have in order to adapt to
new situations/events.
• Moreover, NPCs have additional advantages over the human player, such as
perfect aim (based in exact coordinates), or navigation points (waypoints in
maps modeling advantageous routes, shortest paths, etc).
• Just a few ‘scientific techniques’ have been used in commercial games…

27. • Traditionally in the scientific area it was called Game Theory, a
branch of applied mathematics in which there are some rewards
depending on the chosen decision. It involved simple, but difficult
to solve, games: Hanoi towers, prisoner’s dilemma, game of life.
• These games proposed problems to be solved by means of exact
methods, heuristics or metaheuristics: tree-based search, A*,
evolutionary algorithms, ant colony optimization,…
• Moreover, the resolution of traditional games (usually puzzles) has
also been studied from the ‘ancient times’ in science life: chess,
backgammon, mastermind, sudoku

28. • videogames provide a new environment for solving heterogeneous
problems.
• The most famous (and probably the first) problem addressed was AI
related issues. It still remains as the main (the most studied) problem
in the area.
• However, with the advances and improvement of technology,
videogames have increased their complexity, so new researching lines
have been arisen:
– Search in maps, combat prediction, or simulation, to cite a few
• Nowadays, there are a huge number of research fields inside
videogames scope, so research studies and publications have grown
exponentially.

29. • AI branch which applies metaheuristics and bioinspired
methods for the resolution of complex problems, usually by
means of adaptive systems.
• It is necessary to model the game (or a part of it) as an
optimization, search or learning problem, among others.
• Examples:
– Pathfinding
– Combat prediction
– Automated generation of behavioral rules
– Parameter tuning
– Objective decision

30. • The most used metaheuristics are: Genetic Algorithms (GA),
Ant Colony Optimization (ACO), Monte-Carlo Tree Search
(MCTS), A*, Genetic Programming (GP), Fuzzy Logic, Neural
Networks…
• Which are mostly applied over finite state machines (FSM),
scripts, rule-based systems (RS) or expert system (ES), among
others.

31. • NPC’s AI: try to model AI aspects for enemies
or partners. It is usual to apply GAs to
optimize parameters considered in behavioral
rules.
• Rule system generation: automated definition
of behavioral rules sets, which determine the
way the NPCs act in different situations. It is
usual to apply GP.
• Human-like behavior analysis and modeling: the
objective is to model NPCs which behave as
human players. Data mining and learning
techniques are usually employed.

32. • Cheating detection: trick detection
techniques, based on the study of statistics
about matches.
• Move and battle prediction: prediction
methods are trained (using neural networks)
analyzing data from recorded matches,
trying to anticipate future movements and
actions.
• Learning in games: adaptive agents can be
created by means of reinforcement learning.

33. • Game mechanics and features analysis: game
components are analyzed and parameterized in
order to get numeric valuations of the game
components.
• Exploration and search in games: search
algorithms are applied in order to find the best
paths to objectives in maps, or to explore some
areas maximizing the covering, for instance.
• Content, characters, levels and story generation:
is the so-called procedural content generation,
and is aimed to generate automatically contents.
They are valued by the players (interactive
methods) or by means of mathematical models.

34. EXAMPLES

35. i  initial population
f  evaluation
function (fitness)
?  stop condition
Se  selection
Cr  crossover
Mu  mutation
Re  replacement
by Johann Dréo

36. http://www.youtube.com/watch?v=ejxfTy4lI6I

37. Unreal is a first person shooter (FPS).
Famous due to the excelent AI of the enemies (bots), which makes it an
amazing multiplayer game. Unreal Tournament series is very well considered.
It offers an editor (UnrealEd) which lets us change almost anything in the game
even the behavior of the bots. It uses the language UnrealScript.

38. A java middleware for Unreal Tournament series games and
Defcon games.
The architecture is as follows:
It is possible to interact with the game from a java program, getting higher
independence (avoiding Unrealscript restrictions) and increasing the
Possibilities (java libraries).
On the contrary, the structures, classes, functions and workflows defined
in the Unreal engine, cannot be accessed, nor used.

39. • Analyze FSM
• Identify behavioral parameters
• Optimize them
Bot based in GA FITNESS EVALUATION
(GA-Bot)
population Std
AI Std Std
AI AI
Evolutionary
process
A.M. Mora et al.: Evolving bot AI in Unreal. EVO* 2010. LNCS 6024, Springer, pp. 170–179

40. • Analyze FSM
• Identify parameters devoted to
team performance
• Optimize them
Team of bots
based on GAs FITNESS EVALUATION
(GT-Bot) Std
Std
AI
AI
Std
AI
population
vs
Evolutionary
Process Or
A.M. Mora et al.: Evolving the cooperative behaviour in unreal bots. IEEE CIG 2010, pp. 241–248

41. • Define a FSM based in expert’s knowledge:
– Two state levels, Set of rules
• Optimize parameters by means of a GA

42. • Examples of NPCs/Bots/Agents:
http://www.youtube.com/watch?v=EiAWYGNpu9M http://www.youtube.com/watch?v=0Khtp2tEU1k

43. A good way to start working:
• 2K BotPrize: Unreal bots which should behave as human as possible.
• Starcraft: combats inside the famous RTS.
• Planet Wars: simpler RTS game. Google AI Challenge 2010.
• ANTS: RTS modeling ant’s fighting. Google AI Challenge 2011.
• Pac-Man: It can be implemented pac-man’s or ghosts intelligence.
• Simulated Car Racing: Car races, track generation, mechanical
optimization.
• Mario AI: Agent, learning, level generation.
http://geneura.ugr.es/cig2012/competitions.html

44. • Conferences:
– IEEE CIG
– CGAMES
– GAME-ON
– CGAT
– Special Sessions:
LION, IWANN, EVO*, GECCO, WCCI
• Journals:
– Transactions on Computational
Intelligence and AI in Games (IEEE)
– Entertainment Computing (Springer)
…
– Anyone which accept your paper