This document describes the evolution of an expert bot for the game Unreal Tournament 2004 using genetic algorithms. The researchers created an initial expert bot (E-Bot) using a state machine approach that outperformed the game's standard bots. They then used genetic algorithms to evolve improved bots (GE-Bots) by optimizing parameters of the E-Bot's AI. Three approaches were tested: one with a 143-gene chromosome that performed poorly, one with a 26-gene chromosome, and one using a complex fitness function that considered additional factors like weapon usage. The 26-gene complex fitness approach produced GE-Bots that outperformed the original E-Bot in matches.
Designing and Evolving an Unreal Tournament 2004 Expert BotAntonio Mora
This presentation describes the design of a bot for the first person shooter Unreal Tournament 2004, which behaves as a human expert player in 1 vs. 1 death matches. This has been implemented modelling the actions (and tricks) of this player, using a state-based AI, and supplemented by a database for ‘learning’ the arena. The expert bot yields excellent results, beating the game default bots in the hardest difficulty, and even being a very hard opponent for the human players (including our expert). The AI of this bot is then improved by means of three different approaches of evolutionary algorithms, optimizing a wide set of parameters (weights and probabilities) which the expert bot considers when playing. The result of this process yields an even better rival; however the noisy nature of the fitness function (due to the pseudostochasticity of the battles) makes the evolution slower than usual.
Modelling Human Expert Behaviour in an Unreal Tournament 2004 BotAntonio Mora
This paper presents a deep description of the design of an autonomous agent (bot) for playing 1 vs. 1 dead match mode in the first person shooter
Unreal Tournament 2004 (UT2K4).
The bot models most of the behaviour (actions and tricks) of an expert human player in this mode, who has participated in international UT2K4 championships.
The Artificial Intelligence engine is based on two levels of states, and it relies on an auxiliary database for learning about the fighting arena. Thus, it will store weapons and items locations once the player has discovered them, as a human player could do.
This so-called expert bot yields excellent results, beating the game default bots in the hardest difficulty, and even being a very hard opponent for the human players (including the expert).
It has been presented at SEED 2013 (I Simposio de Entretenimiento Digital) in CEDI2013.
This was a triumph: Evolving intelligent bots for videogames. And for Science. Pablo García Sánchez
Presentation given to students from IES Montes Orientales about our work on computational intellgence for games by Geneura group of the University of Granada.
Combat game systems, animation, and flipbook making workshopEmmeline Dobson
Contains moving images and slides with various layers. Best downloaded and viewed locally. A workshop delivered for BA Games Cultures students at London South Bank University, 26/3/2014. Much of the content borrows from Street Fighter and Disney animators, bringing together The 12 Principles of Animation, silhouette, games systems, and considering how it feels for the player.
Abstract. This paper presents an interactive genetic algorithm for generating a human-like autonomous player (bot) for the game Unreal Tournament 2004. It is based on a bot modelled from the knowledge of an expert human player. The algorithm provides two types of interaction:
by an expert in the game and by an expert in the algorithm. Each one affects different aspects of the evolution, directing it towards better results regarding the agent’s humanness (objective of this work). It has been conducted an analysis of the experts’ influence on the performance, showing much better results after these interactions that the non-interactive version.
The best bot were submitted to the BotPrize 2014 competition (which seeks for the best human-like bot), getting the second position.
Designing and Evolving an Unreal Tournament 2004 Expert BotAntonio Mora
This presentation describes the design of a bot for the first person shooter Unreal Tournament 2004, which behaves as a human expert player in 1 vs. 1 death matches. This has been implemented modelling the actions (and tricks) of this player, using a state-based AI, and supplemented by a database for ‘learning’ the arena. The expert bot yields excellent results, beating the game default bots in the hardest difficulty, and even being a very hard opponent for the human players (including our expert). The AI of this bot is then improved by means of three different approaches of evolutionary algorithms, optimizing a wide set of parameters (weights and probabilities) which the expert bot considers when playing. The result of this process yields an even better rival; however the noisy nature of the fitness function (due to the pseudostochasticity of the battles) makes the evolution slower than usual.
Modelling Human Expert Behaviour in an Unreal Tournament 2004 BotAntonio Mora
This paper presents a deep description of the design of an autonomous agent (bot) for playing 1 vs. 1 dead match mode in the first person shooter
Unreal Tournament 2004 (UT2K4).
The bot models most of the behaviour (actions and tricks) of an expert human player in this mode, who has participated in international UT2K4 championships.
The Artificial Intelligence engine is based on two levels of states, and it relies on an auxiliary database for learning about the fighting arena. Thus, it will store weapons and items locations once the player has discovered them, as a human player could do.
This so-called expert bot yields excellent results, beating the game default bots in the hardest difficulty, and even being a very hard opponent for the human players (including the expert).
It has been presented at SEED 2013 (I Simposio de Entretenimiento Digital) in CEDI2013.
This was a triumph: Evolving intelligent bots for videogames. And for Science. Pablo García Sánchez
Presentation given to students from IES Montes Orientales about our work on computational intellgence for games by Geneura group of the University of Granada.
Combat game systems, animation, and flipbook making workshopEmmeline Dobson
Contains moving images and slides with various layers. Best downloaded and viewed locally. A workshop delivered for BA Games Cultures students at London South Bank University, 26/3/2014. Much of the content borrows from Street Fighter and Disney animators, bringing together The 12 Principles of Animation, silhouette, games systems, and considering how it feels for the player.
Abstract. This paper presents an interactive genetic algorithm for generating a human-like autonomous player (bot) for the game Unreal Tournament 2004. It is based on a bot modelled from the knowledge of an expert human player. The algorithm provides two types of interaction:
by an expert in the game and by an expert in the algorithm. Each one affects different aspects of the evolution, directing it towards better results regarding the agent’s humanness (objective of this work). It has been conducted an analysis of the experts’ influence on the performance, showing much better results after these interactions that the non-interactive version.
The best bot were submitted to the BotPrize 2014 competition (which seeks for the best human-like bot), getting the second position.
infoShare 2013: Adam Frańczak - Techniki optymalizacyjne w UDK/ UE3Infoshare
Adam Frańczak / Playsoft
Techniki optymalizacyjne w UDK/UE3.
Prezentacja z konferencji infoShare 2013 w Gdańsku.
Presented at infoShare 2013 conference in Gdańsk, Poland.
The main theme of our project is, it is a one kind of game. The application provides a endless runner environment where infinite randomized tiles are generated where player have to bypass obstacles to score points. The main advantage of this game is it helps to create brain activities, improves concentration power and increases hand to eye coordination.
How to Design Effective Learning Games: Sharon Boller and Karl KappSharon Boller
Slides used during September 2017 ATD Learn workshop facilitated by Sharon Boller & Karl Kapp: "Play to Learn: Effective Learning Game Design"
Includes numerous slides identifying DIY game creation resources, templates, tools for creating learning games.
Sarah Romoslawski and Marina Kobayashi presented this talk at the 1st Game Design Conference in San Francisco on September 17th, 2012. The talk defines Games User Research (GUR), and includes two perspectives on practicing GUR at different companies, using GUR with different platforms, genres, and with different target audiences.
These slides are about detection method for game bots, I presented the slides in NDSS 2016
You can find the original paper in https://www.internetsociety.org/sites/default/files/blogs-media/you-are-game-bot-uncovering-game-bots-mmorpgs-via-self-similarity-wild.pdf
Killzone Shadow Fall: Threading the Entity Update on PS4jrouwe
On the PS3 the SPUs were too limited for doing typical entity logic (e.g. humanoids walking around and shooting), so all of our entity logic was running on a single thread. For the PS4 we adopted the "entity as a job" approach and developed a set of rules that entities have to adhere to. We also developed a way to validate these rules at runtime so that no race conditions occur. This presentation will also show how we spread out entity updates across frames. The resulting system has proven itself on the PS4 -- and there have been surprisingly few race conditions that were not easily caught.
Ropossum is a framework that lets you play the beloved Cut The Rope game as much as you want and the levels will keep coming. You can design your own levels, check your designed levels for playability at real time, ask it to complete your unfinished designs according to your own preferences, or even suggest endless playable design variations according to your initial level design.
This talk was given at the 2015 AI Summit at the Game Developers Conference. Julian Togelius and I were asked to give a 40 minute overview of academic research in procedural content generation. It outlines several technical approaches to PCG with their implementation tradeoffs, identifies ways in which it can be used in design, and poses questions about how to evaluate it.
weekly AI tech talk #85 ml-agents Enabling Learned Behaviors with Reinforceme...Bill Liu
https://learn.xnextcon.com/event/eventdetails/W19061910
Behaviors in games---and in the real world---are often difficult to program explicitly. Reinforcement learning (RL) has shown success in learning behaviors based on a simple defined reward function that incentivises correct behavior.
Unity ML-Agents toolkit enables Unity developers to train reinforcement learning models to control behaviors within their games. Once these models are trained, they can be integrated across platforms into a game build via the Unity Inference Engine.
Furthermore, by enabling communication between a Unity build and Python code, ML-Agents enables RL researchers to use Unity games as training environments.
my planning docs for my mobile first space shooter. These slides outline my design approach, mechanics of the game needed, and necessary systems required
In the previous chapter, we showed how a game’s internal economy is one important aspect of its mechanics. We used diagrams to visualize economic structures
and their effects. In this chapter, we introduce the Machinations framework, or
visual language, to formalize this perspective on game mechanics. Machinations
was devised by Joris Dormans to help designers and students of game design create,
document, simulate, and test the internal economy of a game. At the core of this
framework are Machinations diagrams, a way of representing the internal economy
of a game visually. The advantage of Machinations diagrams is that they have a
clearly defined syntax. This lets you use Machinations diagrams to record and communicate designs in a clear and consistent way.
Based on book Game Mechanics - Advanced Game Design - E. Adams and J. Dormans. All credited to them
infoShare 2013: Adam Frańczak - Techniki optymalizacyjne w UDK/ UE3Infoshare
Adam Frańczak / Playsoft
Techniki optymalizacyjne w UDK/UE3.
Prezentacja z konferencji infoShare 2013 w Gdańsku.
Presented at infoShare 2013 conference in Gdańsk, Poland.
The main theme of our project is, it is a one kind of game. The application provides a endless runner environment where infinite randomized tiles are generated where player have to bypass obstacles to score points. The main advantage of this game is it helps to create brain activities, improves concentration power and increases hand to eye coordination.
How to Design Effective Learning Games: Sharon Boller and Karl KappSharon Boller
Slides used during September 2017 ATD Learn workshop facilitated by Sharon Boller & Karl Kapp: "Play to Learn: Effective Learning Game Design"
Includes numerous slides identifying DIY game creation resources, templates, tools for creating learning games.
Sarah Romoslawski and Marina Kobayashi presented this talk at the 1st Game Design Conference in San Francisco on September 17th, 2012. The talk defines Games User Research (GUR), and includes two perspectives on practicing GUR at different companies, using GUR with different platforms, genres, and with different target audiences.
These slides are about detection method for game bots, I presented the slides in NDSS 2016
You can find the original paper in https://www.internetsociety.org/sites/default/files/blogs-media/you-are-game-bot-uncovering-game-bots-mmorpgs-via-self-similarity-wild.pdf
Killzone Shadow Fall: Threading the Entity Update on PS4jrouwe
On the PS3 the SPUs were too limited for doing typical entity logic (e.g. humanoids walking around and shooting), so all of our entity logic was running on a single thread. For the PS4 we adopted the "entity as a job" approach and developed a set of rules that entities have to adhere to. We also developed a way to validate these rules at runtime so that no race conditions occur. This presentation will also show how we spread out entity updates across frames. The resulting system has proven itself on the PS4 -- and there have been surprisingly few race conditions that were not easily caught.
Ropossum is a framework that lets you play the beloved Cut The Rope game as much as you want and the levels will keep coming. You can design your own levels, check your designed levels for playability at real time, ask it to complete your unfinished designs according to your own preferences, or even suggest endless playable design variations according to your initial level design.
This talk was given at the 2015 AI Summit at the Game Developers Conference. Julian Togelius and I were asked to give a 40 minute overview of academic research in procedural content generation. It outlines several technical approaches to PCG with their implementation tradeoffs, identifies ways in which it can be used in design, and poses questions about how to evaluate it.
weekly AI tech talk #85 ml-agents Enabling Learned Behaviors with Reinforceme...Bill Liu
https://learn.xnextcon.com/event/eventdetails/W19061910
Behaviors in games---and in the real world---are often difficult to program explicitly. Reinforcement learning (RL) has shown success in learning behaviors based on a simple defined reward function that incentivises correct behavior.
Unity ML-Agents toolkit enables Unity developers to train reinforcement learning models to control behaviors within their games. Once these models are trained, they can be integrated across platforms into a game build via the Unity Inference Engine.
Furthermore, by enabling communication between a Unity build and Python code, ML-Agents enables RL researchers to use Unity games as training environments.
my planning docs for my mobile first space shooter. These slides outline my design approach, mechanics of the game needed, and necessary systems required
In the previous chapter, we showed how a game’s internal economy is one important aspect of its mechanics. We used diagrams to visualize economic structures
and their effects. In this chapter, we introduce the Machinations framework, or
visual language, to formalize this perspective on game mechanics. Machinations
was devised by Joris Dormans to help designers and students of game design create,
document, simulate, and test the internal economy of a game. At the core of this
framework are Machinations diagrams, a way of representing the internal economy
of a game visually. The advantage of Machinations diagrams is that they have a
clearly defined syntax. This lets you use Machinations diagrams to record and communicate designs in a clear and consistent way.
Based on book Game Mechanics - Advanced Game Design - E. Adams and J. Dormans. All credited to them
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIVladimir Iglovikov, Ph.D.
Presented by Vladimir Iglovikov:
- https://www.linkedin.com/in/iglovikov/
- https://x.com/viglovikov
- https://www.instagram.com/ternaus/
This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation.
Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners.
This case study covers various aspects, including:
People: The contributors and community that have supported Albumentations.
Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions.
Challenges: The hurdles in monetizing open-source projects and measuring user engagement.
Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration.
Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community.
Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations.
Mental Health: Maintaining balance and not feeling pressured by user demands.
Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth.
Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects.
Explore more about Albumentations and join the community at:
GitHub: https://github.com/albumentations-team/albumentations
Website: https://albumentations.ai/
LinkedIn: https://www.linkedin.com/company/100504475
Twitter: https://x.com/albumentations
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
Designing and Evolving an Unreal Tournament 2004 Expert Bot
1. Designing and Evolving an
Unreal Tournament 2004
Expert Bot
A.M. Mora, F. Aisa, R. Caballero, P. García-Sánchez, J.J. Merelo, P.A. Castillo, R. Lara-Cabrera
International Work-Conference on Artificial Neural Networks
3. Environment description
UNREAL
Unreal is a first person shooter (FPS)first person shooter (FPS).
Famous due to the excelent AI of the enemiesexcelent AI of the enemies (bots), which makes it an
amazing multiplayer game. Unreal Tournament series is widely extended.
It offers an editor (UnrealEd)editor (UnrealEd) which lets us change almost anything in
the game even the behaviour of the bots. It uses the language
UnrealScriptUnrealScript.
4. General description
POGAMUT
A java middlewarejava middleware for Unreal Tournament series games and
Defcon games.
The architecture is as follows:
It is possible to interact with the game from a java program,
getting higher independencehigher independence (avoiding Unrealscript restrictions)
and increasing the possibilitiesincreasing the possibilities (java libraries).
On the contrary, the structures, classes, functions andstructures, classes, functions and
workflows defined in the Unreal engine, cannot be accessedworkflows defined in the Unreal engine, cannot be accessed, nor
used.
5. Objectives
UNREAL EXPERT BOT
• Create an autonomous agent for playing Unreal
Tournament deathmatch championship.
• Considering the constraints of this competition:
- 1 vs 1 matches
- Small arenas
- Weapons are not respawned
- Some forbidden items (U-Damage, for instance)
- 15 minutes per match instead of a number of frags (kills)
• Human-like behaviour is desired.
• Modelling Expert player knowledge (and tricks).
- High control in timing (items respawn time)
- Deep knowledge about weapons and their advantages and
disadvantages
- Deep knowledge about items
6. Features
UNREAL EXPERT BOT
• Defined by means of a Finite State Machine based AIState Machine based AI with two
state levels.
• Translated into a set of rules which determine its behaviour.
• Database which models the bot’s memory, since it is uploaded
with data about locations of items and weapons in the map.
7. Bot performance
UNREAL EXPERT BOT
• Expert Bot (E-Bot) outperformed the standard bots in the game
(considering the number of frags), even in the maximum
difficulty level.
• This difficulty level is quite hard for a medium level
player.
• E-Bot is hard to beat for humans, even for the expert.
• Medium level players usually lose against it.
8. Evolutionary Algorithms
EXPERT BOT EVOLUTION
by Johann Dréo
i -> initial
population
f -> evaluation
function (fitness)
? -> stop condition
Se -> selection
Cr -> crossover
Mu -> mutation
Re -> replacement
9. Evolutionary Process in Unreal game
EXPERT BOT EVOLUTION
GE-BOT
Expert Bot based in a
Genetic Algorithm
Evolutionary
process
population
FITNESS EVALUATION
• Analyze Expert bot’s FSM
• Identify parameters
• Optimize them
Expert
Bot’s
AI
10. Approaches
EXPERT BOT EVOLUTION
• Generic Fitness
Just considers frags/deads
and damage produced/received
• Generational scheme
• 4-elitism
• Complex Fitness
- considers frags/deads
- damage produced/received
- time using the best or more
versatile weapons: Lightning
Gun and Shock Rifle
- getting the best items: Shield
and Super Shield
• Stationary scheme
• Chromosome 143
• Uniform Crossover
• Random mutation
• 4 Random individuals
• Chromosome 26
11. Approach 1: Chromosome 143 - Generic Fitness
EXPERT BOT EVOLUTION
• Generic Fitness
Just considers frags/deads
and damage produced/received
• Generational scheme
• 4-elitism
• Chromosome 143
• Uniform Crossover
• Random mutation
• 4 Random individuals
12. Approach 1. Results
EXPERT BOT EVOLUTION
• 30 generations
• 30 individuals
• 1 evaluation (left)
• 3 evaluations (right)
in order to avoid the
noisy nature of the
fitness function
• 15 minutes per
evaluation
• 10 days per run (left)
• One month (right)
• Lightly improvement
tendency
• Too many oscillations,
i.e. noise
• 143 genes are too much
13. EXPERT BOT EVOLUTION
• Generic Fitness
Just considers frags/deads
and damage produced/received
• Generational scheme
• 4-elitism
• Uniform Crossover
• Random mutation
• 4 Random individuals
• Chromosome 26
Approach 2: Chromosome 26 - Generic Fitness
14. Approach 2. Results
EXPERT BOT EVOLUTION
• 50 generations
• 30 individuals
• 5 minutes per
evaluation
• Results of 2 different
runs
• 5 days per run
• Again lightly
improvement tendency
• Too much noise
• Too much diversity
15. EXPERT BOT EVOLUTION
• Complex Fitness
- considers frags/deads
- damage produced/received
- time using the best or more
versatile weapons: Lightning
Gun and Shock Rifle
- getting the best items: Shield
and Super Shield
• Stationary scheme
• Uniform Crossover
• Random mutation
• 4 Random individuals
• Chromosome 26
Approach 3: Chromosome 26 - Complex Fitness
16. Approach 3. Results
EXPERT BOT EVOLUTION
• 40 generations
• 30 individuals
• 5 minutes per
evaluation
• Stationary scheme to
increase the
exploitation factor
• Results of 2 different
runs
• 5 days per run
• Quite good fitness
tendency
• Noise still remains,
but in a lower factor
17. Numerical results
E-BOT vs GE-BOT
• Expert Bot (E-Bot) and the best Genetic Expert Bots (GE-BOT)
have been fighting in four battles (in two maps).
• The average results of these matches are:
• The approach with 143 genes per chromosome is defeated
• GE-Bot with 26 genes outperforms E-Bot.
• The approach with the complex fitness function gets the best
results. Due to its lower noisy factor, and the higher
exploitation component.
19. CONCLUSIONS
• We have designed a human-like Expert Bot (E-Bot) which
outperforms the standard Unreal Tournament 2K4 bots in the
hardest difficulty.
• It is also a hard rival against human players.
• We have tested three different approaches for improving this
bot by means of Genetic Algorithms.
• Too long chromosomes population performs worse than small
length one.
• These algorithms are affected by a high noisy factor regarding
the generic (and easier) fitness function.
• We have defined a complex fitness function which performs
better, with a softer noisy effect.
• The bots obtained after evolution outperform the E-Bot.
Las imágenes de la izquierda corresponden a distintos juegos: Unreal (derecha), Unreal Tournament (izquierda) y Unreal Tournament 2004 (la que sobresale) Las de la derecha corresponden al editor (Unreal ED), que viene con cada juego y que deja abierto casi todo el motor (el Unreal Engine), para cambiar casi lo que quieras. Aunque hay muchas clases restringidas, claro. A la derecha se puede ver el árbol de clases, del que puede heredar al crear una nueva. ;) Sirve también para modificar personajes o escenarios (gráficamente). ;)
Para hacer los bots se usará Pogamut. Gamebots fue la primera librería que apareció para comunicarse con el motor de Unreal Tournament 2004. Está definida en el mismo entorno (como un plugin) Pogamut se basa en ella y utiliza Gavialib para la comunicación. Se programa en Java con la librería de Pogamut.
Al usar Pogamut, no se puede hacer uso del motor de IA de Unreal (basado también en estados), por lo que se ha creado una FSM o IA desde cero. Describir lo que es una FSM si quieres. El bot estará en un estado primario y cuando ocurran eventos se pasará a uno secundario (dentro de ese primario). Ejemplo: el bot está en Attack, pero recibe daño y en lugar de pasar a otro estado de retirada, busca salud, pero sin dejar de atacar. ;) Los datos se meten en la BD cuando el bot los ve en el mapa, no antes, como lo haría un humano. Aquí tienes una muestra sencilla de diagrama de flujo en el que se muestra cómo se elige el estado principal. ;)
Estos resultados se obtuvieron después de publicar el artículo. En él hay menos experimentos (2 partidas sólo). Se hicieron experimentos contra humanos, aunque un poco ‘informales’. :D
Esto te lo sabes. ;D Habría que decir que, una vez se tiene el bot experto funcionando, se va a evolucionar para mejorarlo.
Partiendo de los estados definidos en la FSM y las reglas que éstos modelan, se eligen los parámetros de los que dependen, se ‘extraen’ como cromosomas/individuos del genético, se hacen una población de individuos aleatorios (o pseudoaleatorios) y se evolucionan. Cada vez que se quiere evaluar un individuo, se usan esos parámetros como valores dentro de las reglas del bot experto y se pone a jugar contra otro experto (con los valores por defecto). Tras un rato, se valora lo que ha hecho con la función de fitness, que considera las muertes, así como el daño hecho y recibido.
Cr 143 considera decenas de valores relativos al uso de armas (ponderaciones de cual es mejor en cada posible situación) Cr 26 sustituye todo eso por un valor que pondera la elección de cada arma (se consideran 9 armas diferentes) Valores Health decidir cuándo está bien y cuándo mal Distance distancia al enemigo (cerca, media, lejos) Risk riesgo que está dispuesto a correr (salud a arriesgar en un estado agresivo antes de cambiar a uno defensivo, por ejemplo) Time tiempo que puede pasar buscando al enemigo en un estado Items ponderación del uso de cada item Ambos algoritmos son iguales, sólo cambia el tipo de cromosoma. Mira los detalles de cada operador en el artículo, si quieres.
Estos son los resultados de una de las ejecuciones. Las 3 evaluaciones se hacen para intentar ‘paliar’ el ruido de este tipo de problemas en los que una partida puede salir muy bien y otra muy mal para el mismo bot. ;) Cromosomas muy grandes y/o demasiada diversidad => malos resultados: demasiado ruido y muchas oscilaciones en la función de fitness
Estos son los resultados de dos ejecuciones. Más individuos, menos tiempo de evaluación. Todavía demasiada diversidad => malos resultados: demasiado ruido y muchas oscilaciones en la función de fitness
Stationary scheme para aumentar la presión selectiva => la explotación
Estos son los resultados de dos ejecuciones. Resultados aceptables en cuanto a la tendencia del fitness. Pero aún con mucho ruido. Habría que hacer varias evaluaciones por individuo.
Estos resultados se obtuvieron después de publicar el artículo. En él hay menos experimentos (2 partidas sólo). Se hicieron experimentos contra humanos, aunque un poco ‘informales’. :D