This document summarizes research on using genetic algorithms to learn strategies in repeated matrix games against other learning algorithms and humans. It introduces the problem of modeling multi-agent systems using matrix games and the motivation to incorporate human input. It describes prior work on adaptive learning in multi-agent systems and interactive artificial learning. The document outlines the evaluation environment and discusses genetic algorithm variations like propagating history, stabilization conditions, and dynamic mutation. It presents results and analysis on the performance of genetic algorithms in repeated matrix games with and without human input.

1. Genetic Algorithms in Repeated Matrix Games: The
Effects of Algorithmic Modifications and Human
Input with Various Associates
Yomna Hassan , Intel Middle East Mobile Innovation Center
(MEMIC), Egypt.
Jacob Crandall, Masdar Institute of Science and Technology,
UAE.
Session: Evolving Intelligent Agents
IEEE Symposium on Intelligent Agents (IEEE IA 2013)
IEEE Symposium Series on Computational Intelligence (SSCI 2013)

2. Outline
•
•
•
•
•
•
Introduction.
Prior Solutions.
Evaluation Environment.
Genetic Algorithms in Repeated Matrix Games
Results and Analysis.
Conclusions.

3. Introduction
• Problem Statement.
• Motivation.

4. Problem Statement
• Real-Life problems represented as threads of
multi-agent systems.
• Matrix games as a multi-agent system
simulator.
• Learning within repeated matrix games.
• Incorporating human input in the learning
process.

5. Motivation
• Multi-agent systems in real-world problems.
• Simulating Multi-agent systems through
Repeated Matrix Games.
• How human are involuntary involved in their
interactions with computational intelligencebased environments.

6. Prior Solutions
• Adaptive Learning in multi-agent systems and
dynamic environments.
• Interactive artificial Learning.
• Adaptive Learning using Genetic Algorithms
within RMGs.
• Incorporating human input in GAs learning
process.

7. Evaluation Environment

9. Genetic Algorithms in Repeated Matrix
Games
• Genetic algorithms Variations.
• Human Input Framework.

10. Genetic algorithms Variations
• Propagation of history.
• Stabilizing (stopping) condition.
• Dynamic mutation.

11. Human Input Framework

12. Results and Analysis

15. Conclusions
• GAs often learn successful strategies in RMGs played against other
learning algorithms, particular against learning algorithms with
strong convergence properties.
• Using a dynamic mutation rate serves to increase the performance
of the GA in RMGs.
• GAs do not always learn successful strategies in RMGs.
• Demonstrations of successful behavior by a human could
substantially improve the performance of GAs in RMGs played
against other learning algorithms in some circumstances.
• Uninformed human input on the other hand sometimes decreases
the performance of a GA, though the IGA we considered was often
able to mitigate these negative effects.

16. Future Work
1- Testing the effectiveness of interactive GAs in
more complex environments and against more
opponents.
2- Developing techniques to better overcome
the negative effects of uninformed human input.
3- Studying the use of actual human interactions
with a GA that is playing a RMG.
4- The potential of using human interactions
with other learning algorithms.

17. References
• B. Bouzy and M. Metivier, “Multi-agent learning experiments in
repeated matrix games”, in Proc. of the 27th Intl. Conf. on Machine
Learning, 2010.
• J. W. Crandall and M. A. Goodrich, “Learning to compete,
coordinate and cooperate in repeated general-sum games”,
Machine Learning, vol. 82(3), pp. 281–314, 2011.
• L. Kaelbling, M. Littman, and A. Moore, “Reinforcement learning: A
survey”, Journal of Artificial Intelligence Research, vol. 4, pp. 237–
285, 1996.
• Michael Bowling and Manuela Veloso, “Multiagent learning using a
variable learning rate”, Artificial Intelligence, vol. 136(2), pp. 215–
250, 2002.
• R. Axelrod, The Evolution of Cooperation: Revised Edition, Basic
Books, revised edition, 2006.

18. Open for Questions
Contacts
yomnax.hassan@intel.com
jcrandall@masdar.ac.ae

19. Chromosome Structure

20. Informed and Uninformed Strategies