• Like
FSM-Based Agents for Playing Super Mario Game
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

FSM-Based Agents for Playing Super Mario Game

  • 2,860 views
Published

Presentation of the work "Evolutionary FSM-Based Agents for Playing Super Mario Game" at LION 2013 (LION 7). …

Presentation of the work "Evolutionary FSM-Based Agents for Playing Super Mario Game" at LION 2013 (LION 7).

by Rosa Hidalgo, María Sandra Rodríguez, Antonio M. Mora,
Pablo García, Juan Julián Merelo y Antonio J. Fernández

Conference
http://www.intelligent-optimization.org/LION7/

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
2,860
On SlideShare
0
From Embeds
0
Number of Embeds
7

Actions

Shares
Downloads
8
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Esto se puede eliminar si lo consideras sabido por la audiencia
  • Esto se puede eliminar si lo consideras sabido por la audiencia
  • Decir que nos centramos en el track Gameplay. Buscamos hacer un agente para superar ese reto (según sus normas)
  • Las matrices son muy importantes para saber cómo actuar en cada instante (tick)
  • Supongo lo que son todos esos items. :D
  • Se codifican los posibles estados, incluyendo el 8 (no hacer nada) y el 11 (saltar y agacharse al mismo tiempo). Este último no está implementado en el simulador de Infinite Mario Bros, pero sí en los juegos de Mario.
  • Si no te queda claro, mírate esto en el artículo original (el de las 17 páginas).
  • Mono-seed se ejecuta siempre en la misma pantalla (con el mismo nivel de dificultad), y se va incrementando la longitud de la misma con las generaciones. Con el multi-seed se busca evitar el ‘ruido’ típico de la evaluación de individuos en juegos. Es decir, que un mismo individuo sea valorado como muy bueno en una partida y como muy malo en otra, dependiendo de factores pseudoaleatorios (comportamientos de enemigos, sucesos en la partida, etc).
  • Mírate los detalles de cada operador en el artículo. ;)
  • Parámetros de ejecución del algoritmo en mono-semilla. Obtenidos por experimentación sistemática.
  • Se ve el gran tamaño de población requerido para evolucionar un agente bueno.
  • El tiempo de ejecución es alto y los requisitos de memoria son un problema que hace que muchas veces ‘pete’ el programa (el algoritmo).
  • Se ve una tendencia positiva (el fitness mejora paulatinamente), aunque no es tan drástica como nos gustaría y hay oscilaciones debidas al ‘ruido’ al evaluar a los individuos.
  • Parámetros de multi-seed. Los resultados son similares y los problemas que ocurren (memoria y tiempo) se acrecentan al realizar muchas más evaluaciones y variaciones por individuo.
  • Mejor si bajas estos ejemplos a tu disco duro por si no hay conexión. Puedes usar el programa “aTube Catcher”, por ejemplo. El del nivel 12 petó, como en todos los casos, pero hasta ese momento su comportamiento era muy bueno.
  • Los agentes obtenidos se pudieron evolucionar hasta superar el nivel de dificultad 4, más allá ocurrían errores de memoria (más a medida que incrementaba el nivel de dificultad). Aún así, ese nivel es más que suficiente para lo requerido en el GamePlay track.
  • Las conclusiones, después de realizar este trabajo, son las siguientes:

Transcript

  • 1. Evolutionary FSM-Based Agents for Playing Super Mario GameLION 2013 R.M. Hidalgo-Bermúdez, M.S. Rodríguez-Domingo, A.M. Mora P. García-Sánchez, J.J. Merelo, Antonio J. Fernández-Leiva
  • 2. INDEX1. INTRODUCTION2. PRELIMINARY CONCEPTS AND BACKGROUND3. MARIO AI: COMPETITION AND ENVIRONMENT4. EVOLUTIONARY FSM-BASED AGENT5. EXPERIMENTS AND RESULTS6. CONCLUSIONS AND FUTURE WORK
  • 3. INTRODUCTION Why did we choose Mario Bros?This game is worldwide known and remains verypopular nowadays.There is an open source clone of the game namedInfinite Mario Bros.
  • 4. INTRODUCTIONThe main goal of the game is to go across lateralplatforming levels, trying to avoid differentenemies and obstacles by using some usefulitems.We have developed an autonomous agent usingdifferent approaches based on Finite StateMachines (FSMs) combined with GeneticAlgorithms (GAs) techniques.
  • 5. PRELIMINARY CONCEPTS AND BACKGROUNDGAs are based on the model of darwinistic evolution. Astandard GA’s procedure is:
  • 6. PRELIMINARY CONCEPTS AND BACKGROUNDA Finite State Machine (FSM) is a computational model whichrepresents a set of states and connections between them.It is represented as a directed graph, where each nodecorresponds to a state.
  • 7. MARIO AI: COMPETITION AND ENVIRONMENTInfinite Mario Bros is an open source modification ofSuper Mario Bros. It includes wide support forimplementing autonomous agents to control Mariocharacter using AI techniques.The Agent follows the rules of Mario AIChampionship. The tracks are:  Gameplay  Turing Test  Learning  Level Generation
  • 8. MARIO AI: COMPETITION AND ENVIRONMENTThe possible actions of the Agent are:  Go Right  Go Left  Jump  Down  Shoot/RunMario can be in three different modes:  Fire  Big  Small
  • 9. MARIO AI: COMPETITION AND ENVIRONMENTGame data is presented in the form of two 19x19matrices. Each cell of each matrix holds a value,which represents a type of obstacle or enemy ,respectively
  • 10. MARIO AI: COMPETITION AND ENVIRONMENTObstacles and useful items:
  • 11. MARIO AI: COMPETITION AND ENVIRONMENTEnemies:
  • 12. EVOLUTIONARY FSM-BASED AGENTPossible states
  • 13. EVOLUTIONARY FSM-BASED AGENTChromosome (modelling a FSM)
  • 14. EVOLUTIONARY FSM-BASED AGENTTwo approaches:Mono-seed All the individuals are tested in the same level The length of the level and the time limit to complete it grow with each generationMulti-seed Every individual is tested in 30 random levels The fitness is computed considering the results of all the plays
  • 15. EVOLUTIONARY FSM-BASED AGENTThere are two types of fitness:  Generic fitness  Hierarchical fitnessThe selection mechanism considers the best individual and apercentage of the best ones, selected by tournament according totheir fitness.Crossover is performed considering the best individual of thepresent generation as one of the parents, and one of theindividuals with positive fitness as the other parent.In the mutation operator, various genes in each of theseindividuals are randomly selected to be mutated
  • 16. EXPERIMENTS AND RESULTS Mono-seed approach
  • 17. EXPERIMENTS AND RESULTS Mono-seed approach
  • 18. EXPERIMENTS AND RESULTS
  • 19. EXPERIMENTS AND RESULTS
  • 20. EXPERIMENTS AND RESULTS Multi-seed approach
  • 21. SOME OF OUR APPROACHESDifficulty level 1 (completed):http://www.youtube.com/watch?v=6Pj6dZCE070Difficulty level 2 (completed):http://www.youtube.com/watch?v=gtfuY-L0WDADifficulty level 3 (completed):http://www.youtube.com/watch?v=qQVQ43sWwYYDifficulty level 12 (stacked):http://www.youtube.com/watch?v=zNGfBApX7sk
  • 22. CONCLUSIONS AND FUTURE WORKAn agent has been implemented using Finite State Machine (FSM)models, improved by means of Genetic Algorithms (GA) andconsidering different evaluation schemes: mono-seed and a multi-seed evaluation approaches.Mono-seed can yield excellent agents for the level where they were‘trained’ (evolved), having a quite bad behaviour in a different level.Multi-seed takes much more computational time and has higherresource requirements, but the agents it yields are very goodplaying in any level of the considered difficulty (in the evolution).All these agents play much better than an expert human player andcan complete the levels in a time impossible to get for the human.
  • 23. Thanks!