SlideShare a Scribd company logo
Online Evolution for Multi-Action
Adversarial Games
Niels Justesen IT University of Copenhagen
Tobias Mahlmann Lund University
Julian Togelius New York University
Multi-action games
Sid Meier's Civilizations Heroes of Might and Magic Advance Wars
Hero Academy
https://github.com/njustesen/hero-aicademy
● 5 action points each turn
○ Movement ■
○ Healing ■
○ Attacking ■
○ Equipping ■
○ Swapping ■
● Branching factor:
○ One action: ~60
○ One turn: 605
= 7.78 × 108
= 778,000,000
Hero Academy
Hand-written heuristic
(b) Bonus added to units on special squares.(a) Bonus added to units with items.
Search algorithms in Hero Academy
● 1-ply search
○ Greedy on action-level
● 5-ply (1 turn) depth-first search
○ ~500,000 unique outcomes evaluated each turn (6 seconds)
○ Action pruning and sorting
○ Similar to MiniMax search depth-limited to 5 plies
○ Greedy on turn-level
Monte Carlo Tree Search
Chaslot, Guillaume, et al. "Monte-Carlo Tree Search: A New Framework for Game AI." AIIDE. 2008.
MCTS in Hero Academy
● No longer an anytime algorithm
● Rollouts have negative effects - use the evaluation function
● ~200 unique outcomes evaluated each turn (6 seconds)
● Side effect of best-first searches in multi-action games: avoids searching the
opponents turn
Comparison
Online Evolution
Rolling Horizon Evolution
Perez, Diego, et al. "Rolling horizon evolution versus tree search for navigation in single-player real-time games", 2013
Online Evolution
● Population size of 100
● 50% elitism
● Random selection of parents
● Uniform crossover
● 10% mutation rate
Online Evolution in Hero Academy
● ~10,000 unique outcomes evaluated each turn (6 seconds)
● ~3,500 generations each turn on average
Video
Future work
● Considering opponent actions
○ Rollouts
○ Competitive co-evolution
● Evolving heuristics
○ Parameter tuning of existing heuristic
○ 1-ply evolution
○ NEAT / Deep Learning
● MCTS variations for Hero Academy
● Online Evolution in multi-action games with more actions
Questions?
Email: njustesen@gmail.com
Hero AIcademy: https://github.com/njustesen/hero-aicademy

More Related Content

Viewers also liked

Eco-nomics, The hidden costs of consumption
Eco-nomics, The hidden costs of consumptionEco-nomics, The hidden costs of consumption
Eco-nomics, The hidden costs of consumption
Josh Beatty
 
6 Snapchat Hacks Too Easy To Ignore
6 Snapchat Hacks Too Easy To Ignore6 Snapchat Hacks Too Easy To Ignore
6 Snapchat Hacks Too Easy To IgnoreGary Vaynerchuk
 
Pixar's 22 Rules to Phenomenal Storytelling
Pixar's 22 Rules to Phenomenal StorytellingPixar's 22 Rules to Phenomenal Storytelling
Pixar's 22 Rules to Phenomenal Storytelling
Gavin McMahon
 
All About Beer
All About Beer All About Beer
All About Beer
Ethos3
 
SMOKE - The Convenient Truth [1st place Worlds Best Presentation Contest] by ...
SMOKE - The Convenient Truth [1st place Worlds Best Presentation Contest] by ...SMOKE - The Convenient Truth [1st place Worlds Best Presentation Contest] by ...
SMOKE - The Convenient Truth [1st place Worlds Best Presentation Contest] by ...
Empowered Presentations
 
Digital, Social & Mobile in 2015
Digital, Social & Mobile in 2015Digital, Social & Mobile in 2015
Digital, Social & Mobile in 2015
We Are Social Singapore
 
Healthcare Napkins All
Healthcare Napkins AllHealthcare Napkins All
Healthcare Napkins All
Dan Roam
 
You Suck At PowerPoint! by @jessedee
You Suck At PowerPoint! by @jessedeeYou Suck At PowerPoint! by @jessedee
You Suck At PowerPoint! by @jessedee
Jesse Desjardins - @jessedee
 
10 Powerful Body Language Tips for your next Presentation
10 Powerful Body Language Tips for your next Presentation10 Powerful Body Language Tips for your next Presentation
10 Powerful Body Language Tips for your next Presentation
SOAP Presentations
 
How Google Works
How Google WorksHow Google Works
How Google Works
Eric Schmidt
 
The Search for Meaning in B2B Marketing
The Search for Meaning in B2B MarketingThe Search for Meaning in B2B Marketing
The Search for Meaning in B2B Marketing
Velocity Partners
 

Viewers also liked (11)

Eco-nomics, The hidden costs of consumption
Eco-nomics, The hidden costs of consumptionEco-nomics, The hidden costs of consumption
Eco-nomics, The hidden costs of consumption
 
6 Snapchat Hacks Too Easy To Ignore
6 Snapchat Hacks Too Easy To Ignore6 Snapchat Hacks Too Easy To Ignore
6 Snapchat Hacks Too Easy To Ignore
 
Pixar's 22 Rules to Phenomenal Storytelling
Pixar's 22 Rules to Phenomenal StorytellingPixar's 22 Rules to Phenomenal Storytelling
Pixar's 22 Rules to Phenomenal Storytelling
 
All About Beer
All About Beer All About Beer
All About Beer
 
SMOKE - The Convenient Truth [1st place Worlds Best Presentation Contest] by ...
SMOKE - The Convenient Truth [1st place Worlds Best Presentation Contest] by ...SMOKE - The Convenient Truth [1st place Worlds Best Presentation Contest] by ...
SMOKE - The Convenient Truth [1st place Worlds Best Presentation Contest] by ...
 
Digital, Social & Mobile in 2015
Digital, Social & Mobile in 2015Digital, Social & Mobile in 2015
Digital, Social & Mobile in 2015
 
Healthcare Napkins All
Healthcare Napkins AllHealthcare Napkins All
Healthcare Napkins All
 
You Suck At PowerPoint! by @jessedee
You Suck At PowerPoint! by @jessedeeYou Suck At PowerPoint! by @jessedee
You Suck At PowerPoint! by @jessedee
 
10 Powerful Body Language Tips for your next Presentation
10 Powerful Body Language Tips for your next Presentation10 Powerful Body Language Tips for your next Presentation
10 Powerful Body Language Tips for your next Presentation
 
How Google Works
How Google WorksHow Google Works
How Google Works
 
The Search for Meaning in B2B Marketing
The Search for Meaning in B2B MarketingThe Search for Meaning in B2B Marketing
The Search for Meaning in B2B Marketing
 

Recently uploaded

Anemia_ different types_causes_ conditions
Anemia_ different types_causes_ conditionsAnemia_ different types_causes_ conditions
Anemia_ different types_causes_ conditions
muralinath2
 
Seminar of U.V. Spectroscopy by SAMIR PANDA
 Seminar of U.V. Spectroscopy by SAMIR PANDA Seminar of U.V. Spectroscopy by SAMIR PANDA
Seminar of U.V. Spectroscopy by SAMIR PANDA
SAMIR PANDA
 
Richard's entangled aventures in wonderland
Richard's entangled aventures in wonderlandRichard's entangled aventures in wonderland
Richard's entangled aventures in wonderland
Richard Gill
 
Body fluids_tonicity_dehydration_hypovolemia_hypervolemia.pptx
Body fluids_tonicity_dehydration_hypovolemia_hypervolemia.pptxBody fluids_tonicity_dehydration_hypovolemia_hypervolemia.pptx
Body fluids_tonicity_dehydration_hypovolemia_hypervolemia.pptx
muralinath2
 
Lab report on liquid viscosity of glycerin
Lab report on liquid viscosity of glycerinLab report on liquid viscosity of glycerin
Lab report on liquid viscosity of glycerin
ossaicprecious19
 
erythropoiesis-I_mechanism& clinical significance.pptx
erythropoiesis-I_mechanism& clinical significance.pptxerythropoiesis-I_mechanism& clinical significance.pptx
erythropoiesis-I_mechanism& clinical significance.pptx
muralinath2
 
insect morphology and physiology of insect
insect morphology and physiology of insectinsect morphology and physiology of insect
insect morphology and physiology of insect
anitaento25
 
NuGOweek 2024 Ghent - programme - final version
NuGOweek 2024 Ghent - programme - final versionNuGOweek 2024 Ghent - programme - final version
NuGOweek 2024 Ghent - programme - final version
pablovgd
 
Hemostasis_importance& clinical significance.pptx
Hemostasis_importance& clinical significance.pptxHemostasis_importance& clinical significance.pptx
Hemostasis_importance& clinical significance.pptx
muralinath2
 
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATIONPRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
ChetanK57
 
Circulatory system_ Laplace law. Ohms law.reynaults law,baro-chemo-receptors-...
Circulatory system_ Laplace law. Ohms law.reynaults law,baro-chemo-receptors-...Circulatory system_ Laplace law. Ohms law.reynaults law,baro-chemo-receptors-...
Circulatory system_ Laplace law. Ohms law.reynaults law,baro-chemo-receptors-...
muralinath2
 
Richard's aventures in two entangled wonderlands
Richard's aventures in two entangled wonderlandsRichard's aventures in two entangled wonderlands
Richard's aventures in two entangled wonderlands
Richard Gill
 
EY - Supply Chain Services 2018_template.pptx
EY - Supply Chain Services 2018_template.pptxEY - Supply Chain Services 2018_template.pptx
EY - Supply Chain Services 2018_template.pptx
AlguinaldoKong
 
Citrus Greening Disease and its Management
Citrus Greening Disease and its ManagementCitrus Greening Disease and its Management
Citrus Greening Disease and its Management
subedisuryaofficial
 
FAIR & AI Ready KGs for Explainable Predictions
FAIR & AI Ready KGs for Explainable PredictionsFAIR & AI Ready KGs for Explainable Predictions
FAIR & AI Ready KGs for Explainable Predictions
Michel Dumontier
 
Astronomy Update- Curiosity’s exploration of Mars _ Local Briefs _ leadertele...
Astronomy Update- Curiosity’s exploration of Mars _ Local Briefs _ leadertele...Astronomy Update- Curiosity’s exploration of Mars _ Local Briefs _ leadertele...
Astronomy Update- Curiosity’s exploration of Mars _ Local Briefs _ leadertele...
NathanBaughman3
 
platelets_clotting_biogenesis.clot retractionpptx
platelets_clotting_biogenesis.clot retractionpptxplatelets_clotting_biogenesis.clot retractionpptx
platelets_clotting_biogenesis.clot retractionpptx
muralinath2
 
Cancer cell metabolism: special Reference to Lactate Pathway
Cancer cell metabolism: special Reference to Lactate PathwayCancer cell metabolism: special Reference to Lactate Pathway
Cancer cell metabolism: special Reference to Lactate Pathway
AADYARAJPANDEY1
 
justice-and-fairness-ethics with example
justice-and-fairness-ethics with examplejustice-and-fairness-ethics with example
justice-and-fairness-ethics with example
azzyixes
 
Unveiling the Energy Potential of Marshmallow Deposits.pdf
Unveiling the Energy Potential of Marshmallow Deposits.pdfUnveiling the Energy Potential of Marshmallow Deposits.pdf
Unveiling the Energy Potential of Marshmallow Deposits.pdf
Erdal Coalmaker
 

Recently uploaded (20)

Anemia_ different types_causes_ conditions
Anemia_ different types_causes_ conditionsAnemia_ different types_causes_ conditions
Anemia_ different types_causes_ conditions
 
Seminar of U.V. Spectroscopy by SAMIR PANDA
 Seminar of U.V. Spectroscopy by SAMIR PANDA Seminar of U.V. Spectroscopy by SAMIR PANDA
Seminar of U.V. Spectroscopy by SAMIR PANDA
 
Richard's entangled aventures in wonderland
Richard's entangled aventures in wonderlandRichard's entangled aventures in wonderland
Richard's entangled aventures in wonderland
 
Body fluids_tonicity_dehydration_hypovolemia_hypervolemia.pptx
Body fluids_tonicity_dehydration_hypovolemia_hypervolemia.pptxBody fluids_tonicity_dehydration_hypovolemia_hypervolemia.pptx
Body fluids_tonicity_dehydration_hypovolemia_hypervolemia.pptx
 
Lab report on liquid viscosity of glycerin
Lab report on liquid viscosity of glycerinLab report on liquid viscosity of glycerin
Lab report on liquid viscosity of glycerin
 
erythropoiesis-I_mechanism& clinical significance.pptx
erythropoiesis-I_mechanism& clinical significance.pptxerythropoiesis-I_mechanism& clinical significance.pptx
erythropoiesis-I_mechanism& clinical significance.pptx
 
insect morphology and physiology of insect
insect morphology and physiology of insectinsect morphology and physiology of insect
insect morphology and physiology of insect
 
NuGOweek 2024 Ghent - programme - final version
NuGOweek 2024 Ghent - programme - final versionNuGOweek 2024 Ghent - programme - final version
NuGOweek 2024 Ghent - programme - final version
 
Hemostasis_importance& clinical significance.pptx
Hemostasis_importance& clinical significance.pptxHemostasis_importance& clinical significance.pptx
Hemostasis_importance& clinical significance.pptx
 
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATIONPRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
 
Circulatory system_ Laplace law. Ohms law.reynaults law,baro-chemo-receptors-...
Circulatory system_ Laplace law. Ohms law.reynaults law,baro-chemo-receptors-...Circulatory system_ Laplace law. Ohms law.reynaults law,baro-chemo-receptors-...
Circulatory system_ Laplace law. Ohms law.reynaults law,baro-chemo-receptors-...
 
Richard's aventures in two entangled wonderlands
Richard's aventures in two entangled wonderlandsRichard's aventures in two entangled wonderlands
Richard's aventures in two entangled wonderlands
 
EY - Supply Chain Services 2018_template.pptx
EY - Supply Chain Services 2018_template.pptxEY - Supply Chain Services 2018_template.pptx
EY - Supply Chain Services 2018_template.pptx
 
Citrus Greening Disease and its Management
Citrus Greening Disease and its ManagementCitrus Greening Disease and its Management
Citrus Greening Disease and its Management
 
FAIR & AI Ready KGs for Explainable Predictions
FAIR & AI Ready KGs for Explainable PredictionsFAIR & AI Ready KGs for Explainable Predictions
FAIR & AI Ready KGs for Explainable Predictions
 
Astronomy Update- Curiosity’s exploration of Mars _ Local Briefs _ leadertele...
Astronomy Update- Curiosity’s exploration of Mars _ Local Briefs _ leadertele...Astronomy Update- Curiosity’s exploration of Mars _ Local Briefs _ leadertele...
Astronomy Update- Curiosity’s exploration of Mars _ Local Briefs _ leadertele...
 
platelets_clotting_biogenesis.clot retractionpptx
platelets_clotting_biogenesis.clot retractionpptxplatelets_clotting_biogenesis.clot retractionpptx
platelets_clotting_biogenesis.clot retractionpptx
 
Cancer cell metabolism: special Reference to Lactate Pathway
Cancer cell metabolism: special Reference to Lactate PathwayCancer cell metabolism: special Reference to Lactate Pathway
Cancer cell metabolism: special Reference to Lactate Pathway
 
justice-and-fairness-ethics with example
justice-and-fairness-ethics with examplejustice-and-fairness-ethics with example
justice-and-fairness-ethics with example
 
Unveiling the Energy Potential of Marshmallow Deposits.pdf
Unveiling the Energy Potential of Marshmallow Deposits.pdfUnveiling the Energy Potential of Marshmallow Deposits.pdf
Unveiling the Energy Potential of Marshmallow Deposits.pdf
 

Online evolution for multi action adversarial games

  • 1. Online Evolution for Multi-Action Adversarial Games Niels Justesen IT University of Copenhagen Tobias Mahlmann Lund University Julian Togelius New York University
  • 2. Multi-action games Sid Meier's Civilizations Heroes of Might and Magic Advance Wars
  • 4. ● 5 action points each turn ○ Movement ■ ○ Healing ■ ○ Attacking ■ ○ Equipping ■ ○ Swapping ■ ● Branching factor: ○ One action: ~60 ○ One turn: 605 = 7.78 × 108 = 778,000,000 Hero Academy
  • 5. Hand-written heuristic (b) Bonus added to units on special squares.(a) Bonus added to units with items.
  • 6. Search algorithms in Hero Academy ● 1-ply search ○ Greedy on action-level ● 5-ply (1 turn) depth-first search ○ ~500,000 unique outcomes evaluated each turn (6 seconds) ○ Action pruning and sorting ○ Similar to MiniMax search depth-limited to 5 plies ○ Greedy on turn-level
  • 7. Monte Carlo Tree Search Chaslot, Guillaume, et al. "Monte-Carlo Tree Search: A New Framework for Game AI." AIIDE. 2008.
  • 8. MCTS in Hero Academy ● No longer an anytime algorithm ● Rollouts have negative effects - use the evaluation function ● ~200 unique outcomes evaluated each turn (6 seconds) ● Side effect of best-first searches in multi-action games: avoids searching the opponents turn
  • 10. Online Evolution Rolling Horizon Evolution Perez, Diego, et al. "Rolling horizon evolution versus tree search for navigation in single-player real-time games", 2013
  • 11. Online Evolution ● Population size of 100 ● 50% elitism ● Random selection of parents ● Uniform crossover ● 10% mutation rate
  • 12.
  • 13. Online Evolution in Hero Academy ● ~10,000 unique outcomes evaluated each turn (6 seconds) ● ~3,500 generations each turn on average
  • 14. Video
  • 15. Future work ● Considering opponent actions ○ Rollouts ○ Competitive co-evolution ● Evolving heuristics ○ Parameter tuning of existing heuristic ○ 1-ply evolution ○ NEAT / Deep Learning ● MCTS variations for Hero Academy ● Online Evolution in multi-action games with more actions
  • 16. Questions? Email: njustesen@gmail.com Hero AIcademy: https://github.com/njustesen/hero-aicademy