Evolutionary FSM-Based        Agents for Playing        Super Mario GameLION 2013            R.M. Hidalgo-Bermúdez, M.S. R...
INDEX1. INTRODUCTION2. PRELIMINARY CONCEPTS AND BACKGROUND3. MARIO AI: COMPETITION AND ENVIRONMENT4. EVOLUTIONARY FSM-BASE...
INTRODUCTION         Why did we choose Mario Bros?This game is worldwide known and remains verypopular nowadays.There is a...
INTRODUCTIONThe main goal of the game is to go across lateralplatforming levels, trying to avoid differentenemies and obst...
PRELIMINARY CONCEPTS AND          BACKGROUNDGAs are based on the model of darwinistic evolution. Astandard GA’s procedure ...
PRELIMINARY CONCEPTS AND           BACKGROUNDA Finite State Machine (FSM) is a computational model whichrepresents a set o...
MARIO AI: COMPETITION AND           ENVIRONMENTInfinite Mario Bros is an open source modification ofSuper Mario Bros. It i...
MARIO AI: COMPETITION AND          ENVIRONMENTThe possible actions of the Agent are:    Go Right            Go Left    ...
MARIO AI: COMPETITION AND          ENVIRONMENTGame data is presented in the form of two 19x19matrices. Each cell of each m...
MARIO AI: COMPETITION AND           ENVIRONMENTObstacles and useful items:
MARIO AI: COMPETITION AND          ENVIRONMENTEnemies:
EVOLUTIONARY FSM-BASED AGENTPossible states
EVOLUTIONARY FSM-BASED AGENTChromosome (modelling a FSM)
EVOLUTIONARY FSM-BASED AGENTTwo approaches:Mono-seed    All the individuals are tested in the same level    The length of ...
EVOLUTIONARY FSM-BASED AGENTThere are two types of fitness:    Generic fitness    Hierarchical fitnessThe selection mech...
EXPERIMENTS AND RESULTS    Mono-seed approach
EXPERIMENTS AND RESULTS    Mono-seed approach
EXPERIMENTS AND RESULTS
EXPERIMENTS AND RESULTS
EXPERIMENTS AND RESULTS    Multi-seed approach
SOME OF OUR APPROACHESDifficulty level 1 (completed):http://www.youtube.com/watch?v=6Pj6dZCE070Difficulty level 2 (complet...
Thanks!
FSM-Based Agents for Playing Super Mario Game
Upcoming SlideShare
Loading in...5
×

FSM-Based Agents for Playing Super Mario Game

3,151

Published on

Presentation of the work "Evolutionary FSM-Based Agents for Playing Super Mario Game" at LION 2013 (LION 7).

by Rosa Hidalgo, María Sandra Rodríguez, Antonio M. Mora,
Pablo García, Juan Julián Merelo y Antonio J. Fernández

Conference
http://www.intelligent-optimization.org/LION7/

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,151
On Slideshare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
12
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Esto se puede eliminar si lo consideras sabido por la audiencia
  • Esto se puede eliminar si lo consideras sabido por la audiencia
  • Decir que nos centramos en el track Gameplay. Buscamos hacer un agente para superar ese reto (según sus normas)
  • Las matrices son muy importantes para saber cómo actuar en cada instante (tick)
  • Supongo lo que son todos esos items. :D
  • Se codifican los posibles estados, incluyendo el 8 (no hacer nada) y el 11 (saltar y agacharse al mismo tiempo). Este último no está implementado en el simulador de Infinite Mario Bros, pero sí en los juegos de Mario.
  • Si no te queda claro, mírate esto en el artículo original (el de las 17 páginas).
  • Mono-seed se ejecuta siempre en la misma pantalla (con el mismo nivel de dificultad), y se va incrementando la longitud de la misma con las generaciones. Con el multi-seed se busca evitar el ‘ruido’ típico de la evaluación de individuos en juegos. Es decir, que un mismo individuo sea valorado como muy bueno en una partida y como muy malo en otra, dependiendo de factores pseudoaleatorios (comportamientos de enemigos, sucesos en la partida, etc).
  • Mírate los detalles de cada operador en el artículo. ;)
  • Parámetros de ejecución del algoritmo en mono-semilla. Obtenidos por experimentación sistemática.
  • Se ve el gran tamaño de población requerido para evolucionar un agente bueno.
  • El tiempo de ejecución es alto y los requisitos de memoria son un problema que hace que muchas veces ‘pete’ el programa (el algoritmo).
  • Se ve una tendencia positiva (el fitness mejora paulatinamente), aunque no es tan drástica como nos gustaría y hay oscilaciones debidas al ‘ruido’ al evaluar a los individuos.
  • Parámetros de multi-seed. Los resultados son similares y los problemas que ocurren (memoria y tiempo) se acrecentan al realizar muchas más evaluaciones y variaciones por individuo.
  • Mejor si bajas estos ejemplos a tu disco duro por si no hay conexión. Puedes usar el programa “aTube Catcher”, por ejemplo. El del nivel 12 petó, como en todos los casos, pero hasta ese momento su comportamiento era muy bueno.
  • Los agentes obtenidos se pudieron evolucionar hasta superar el nivel de dificultad 4, más allá ocurrían errores de memoria (más a medida que incrementaba el nivel de dificultad). Aún así, ese nivel es más que suficiente para lo requerido en el GamePlay track.
  • Las conclusiones, después de realizar este trabajo, son las siguientes:
  • Transcript of "FSM-Based Agents for Playing Super Mario Game"

    1. 1. Evolutionary FSM-Based Agents for Playing Super Mario GameLION 2013 R.M. Hidalgo-Bermúdez, M.S. Rodríguez-Domingo, A.M. Mora P. García-Sánchez, J.J. Merelo, Antonio J. Fernández-Leiva
    2. 2. INDEX1. INTRODUCTION2. PRELIMINARY CONCEPTS AND BACKGROUND3. MARIO AI: COMPETITION AND ENVIRONMENT4. EVOLUTIONARY FSM-BASED AGENT5. EXPERIMENTS AND RESULTS6. CONCLUSIONS AND FUTURE WORK
    3. 3. INTRODUCTION Why did we choose Mario Bros?This game is worldwide known and remains verypopular nowadays.There is an open source clone of the game namedInfinite Mario Bros.
    4. 4. INTRODUCTIONThe main goal of the game is to go across lateralplatforming levels, trying to avoid differentenemies and obstacles by using some usefulitems.We have developed an autonomous agent usingdifferent approaches based on Finite StateMachines (FSMs) combined with GeneticAlgorithms (GAs) techniques.
    5. 5. PRELIMINARY CONCEPTS AND BACKGROUNDGAs are based on the model of darwinistic evolution. Astandard GA’s procedure is:
    6. 6. PRELIMINARY CONCEPTS AND BACKGROUNDA Finite State Machine (FSM) is a computational model whichrepresents a set of states and connections between them.It is represented as a directed graph, where each nodecorresponds to a state.
    7. 7. MARIO AI: COMPETITION AND ENVIRONMENTInfinite Mario Bros is an open source modification ofSuper Mario Bros. It includes wide support forimplementing autonomous agents to control Mariocharacter using AI techniques.The Agent follows the rules of Mario AIChampionship. The tracks are:  Gameplay  Turing Test  Learning  Level Generation
    8. 8. MARIO AI: COMPETITION AND ENVIRONMENTThe possible actions of the Agent are:  Go Right  Go Left  Jump  Down  Shoot/RunMario can be in three different modes:  Fire  Big  Small
    9. 9. MARIO AI: COMPETITION AND ENVIRONMENTGame data is presented in the form of two 19x19matrices. Each cell of each matrix holds a value,which represents a type of obstacle or enemy ,respectively
    10. 10. MARIO AI: COMPETITION AND ENVIRONMENTObstacles and useful items:
    11. 11. MARIO AI: COMPETITION AND ENVIRONMENTEnemies:
    12. 12. EVOLUTIONARY FSM-BASED AGENTPossible states
    13. 13. EVOLUTIONARY FSM-BASED AGENTChromosome (modelling a FSM)
    14. 14. EVOLUTIONARY FSM-BASED AGENTTwo approaches:Mono-seed All the individuals are tested in the same level The length of the level and the time limit to complete it grow with each generationMulti-seed Every individual is tested in 30 random levels The fitness is computed considering the results of all the plays
    15. 15. EVOLUTIONARY FSM-BASED AGENTThere are two types of fitness:  Generic fitness  Hierarchical fitnessThe selection mechanism considers the best individual and apercentage of the best ones, selected by tournament according totheir fitness.Crossover is performed considering the best individual of thepresent generation as one of the parents, and one of theindividuals with positive fitness as the other parent.In the mutation operator, various genes in each of theseindividuals are randomly selected to be mutated
    16. 16. EXPERIMENTS AND RESULTS Mono-seed approach
    17. 17. EXPERIMENTS AND RESULTS Mono-seed approach
    18. 18. EXPERIMENTS AND RESULTS
    19. 19. EXPERIMENTS AND RESULTS
    20. 20. EXPERIMENTS AND RESULTS Multi-seed approach
    21. 21. SOME OF OUR APPROACHESDifficulty level 1 (completed):http://www.youtube.com/watch?v=6Pj6dZCE070Difficulty level 2 (completed):http://www.youtube.com/watch?v=gtfuY-L0WDADifficulty level 3 (completed):http://www.youtube.com/watch?v=qQVQ43sWwYYDifficulty level 12 (stacked):http://www.youtube.com/watch?v=zNGfBApX7sk
    22. 22. CONCLUSIONS AND FUTURE WORKAn agent has been implemented using Finite State Machine (FSM)models, improved by means of Genetic Algorithms (GA) andconsidering different evaluation schemes: mono-seed and a multi-seed evaluation approaches.Mono-seed can yield excellent agents for the level where they were‘trained’ (evolved), having a quite bad behaviour in a different level.Multi-seed takes much more computational time and has higherresource requirements, but the agents it yields are very goodplaying in any level of the considered difficulty (in the evolution).All these agents play much better than an expert human player andcan complete the levels in a time impossible to get for the human.
    23. 23. Thanks!

    ×