Successfully reported this slideshow.

×

Yes you can play Monopoly with a Genetic Algorithm - Niels Zeilemaker

Niels Zeilemaker is a Data Hacker at GoDataDriven, where he works for a wide range of companies where he spends his time doing feature engineering and building models. Before joining GDD, he finished his PhD thesis at the Technical University of Delft. In this talk he will explain how to 'solve' Monopoly with a genetic algorithm. He will first go into the official rules of Monopoly, as nobody seems to actually play according to the rules. Then into how to use a genetic algorithm to optimize a general buying rule, and finally how to optimize the number of houses to buy.

Niels Zeilemaker is a Data Hacker at GoDataDriven, where he works for a wide range of companies where he spends his time doing feature engineering and building models. Before joining GDD, he finished his PhD thesis at the Technical University of Delft. In this talk he will explain how to 'solve' Monopoly with a genetic algorithm. He will first go into the official rules of Monopoly, as nobody seems to actually play according to the rules. Then into how to use a genetic algorithm to optimize a general buying rule, and finally how to optimize the number of houses to buy.

More Related Content

Yes you can play Monopoly with a Genetic Algorithm - Niels Zeilemaker

1. 1. Yes, you can play Monopoly with a genetic algorithm Niels Zeilemaker
2. 2. About me • PhD in Parallel Computing @ TUDelft • Data hacker at GDD • Data scientist and/or data engineer • Basically, drink coffee a lot while waiting for
3. 3. Monopoly • Boardgame created in 1935 • For 2-6 players • Buy houses/hotels –> • Last man standing wins • Pronounced in dutch as moonoopooly
4. 4. Terminology • Players traverse the board by rolling two dice • When landing on an unowned property, a player can decide to buy it • If a player owns all properties of a single color, he can buy houses • Visiting a property owned by another player you will need to pay rent
5. 5. Rules • A player passing Go receives \$200 • All fines are paid into the free parking space • When a player does not buy an unowned property an auction is held • Rolling doubles three times in a row results in goto jail • If a player lands on Go he receives \$400 • Rolling doubles will release a player from jail • Paying \$50 will release a player from jail
6. 6. Rules • A player passing Go receives \$200 • All fines are paid into the free parking space • When a player does not buy an unowned property an auction is held • Rolling doubles three times in a row results in goto jail • If a player lands on Go he receives \$400 • Rolling doubles will release a player from jail • Paying \$50 will release a player from jail
7. 7. How much free time do you have? • Interesting game, • still doesn’t explain why you would attempt to “solve it” • GDD Friday + = “Monopoly Simulations” • He implemented a small python script which would roll dice + go to jail See: http://koaning.io/monopoly-simulations.html
8. 8. Probability distribution
9. 9. Datascientists writing code • Initial script, 46 lines • “Improved” script, 800 lines
10. 10. How much free time do you have? • Implemented • Board object • Player object • Deed object • BankruptException ;) • Unit-tested, coverage ~80%
11. 11. Player object class Player(object): def buy_position(self, deed, amount): # after landing on a deed def bid_position(self, deed, amount, current_bid): # while in an auction def anything_else(self): # at the end of the turn def raise_amount(self, amount): # when we need to pay for something
13. 13. Which deeds to buy? € 0.00 € 100,000.00 € 200,000.00 € 300,000.00 € 400,000.00 € 500,000.00 € 600,000.00 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 Money raised/invested raised invested
14. 14. But, which will help you to win? 0 50 100 150 200 250 300 350 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 Caused bankruptcy
15. 15. Next up, genetic algorithms • Manually selecting which deeds to buy/how many houses to build isn’t a lot of fun • Enter, the GAPlayer
16. 16. Genetic algoritms? • Search heuristic which mimics natural selection • Convert search space into chromosome • Eg 1,1,0,3,3, etc. • Evaluate performance of chromosome using fitness function • 1,1,0,3,3,… = 42 • 1,2,0,3,3,… = 42 I'm no expert on genetic algoritms, feel free to correct me
17. 17. Genetic algoritms? • After each round (or population) • Select top X chromosomes • Append some mutations • Generate offspring by combining those • Stop after X generations
18. 18. Genetic algoritms? Algorithm A: 1,0,0,0,0 Algorithm B: 0,0,2,2,2 Child C: 0,0,2,0,2 Child D: 1,0,0,2,2
19. 19. GAPlayer • Chromosome: • 0,6,0,5,0,1,2,0,3,2,0,5,1,5,4,1,1,0,0,1,0,0,0,0,0,1,1,0,1,0,0,0,1,0,0,1,0,4,0,4 • 0 = don’t buy • 1 = buy • 2..5 = build 1..4 houses • 6 = build hotel 65 1 232
20. 20. Fitness function • Implemented two approaches: • Play vs BuyAll • Use TrueSkill • Not sure if either is correct ;)
21. 21. Fitness function, vs BuyAll • Play 1000 matches against BuyAll • Score = #times won • Evaluate 10 generations • Population size = 56 • Select 20% best perfoming • Mutate 1% of the solutions • Single generation takes 1 minute to evaluate
22. 22. Scores vs Generations 400 450 500 550 600 650 1 2 3 4 5 6 7 8 9 10 vs BuyAll
23. 23. But wait • So, GaPlayer '[0, 6, 0, 5, 0, 1, 2, 0, 3, 2, 0, 5, 1, 5, 4, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 4, 0, 4]‘ is a better strategy than buying everything…. • But is it the best monopoly strategy? BuyAll GaPlayer….
24. 24. TrueSkill • A ranking system developed for Xbox player matching • In my understanding • Each player has a skill belief, eg a probability distribution • When you win/lose from another we update the probabilities accordingly • Let’s hope Vincent is in the room
25. 25. An example • Before vs After • Poke1 loses from Poke2
26. 26. Combining GA + Trueskill • Have a global leaderbord • Each new GA chromosome plays against 100 randomly selected others • Each game now is best of 10 • Leaderbord is sorted by value of curve descending
27. 27. Presenting 0,0,0,1,0,1,1,0,1,0,0,1,0,0,0,1,1,0,2,1,0,0,0,0,1,1,4,4,0,5,0,1,0,0,0,1,0,2,0,3 You will never beaten in monopoly again, maybe
28. 28. Runner ups • 0,0,0,1,0,1,0,0,0,0,0,0,0,0,1,1,6,0,6,6,0,0,0,0,0,1,0,1,0,0,0,5,4,0,4,1,0,2,0,3 • 0,0,0,1,0,1,0,0,0,0,0,1,0,0,1,1,1,0,2,1,0,0,0,0,0,1,4,4,0,5,0,0,0,0,0,1,0,2,0,3 • 0,1,0,1,0,1,0,0,0,0,0,1,0,0,1,1,1,0,0,1,0,0,0,0,0,1,4,4,0,5,0,0,0,0,0,1,0,2,0,3 • 0,0,0,1,0,1,0,0,0,0,0,1,1,0,0,1,1,0,1,0,0,1,0,0,0,1,1,1,0,0,0,0,0,0,0,1,0,3,0,4 • 0,1,0,2,0,1,1,0,0,1,0,1,1,1,0,1,1,0,2,2,0,0,0,0,0,1,4,3,1,4,0,0,0,0,0,1,0,2,0,3 • 0,2,0,1,0,1,1,0,0,1,0,0,1,0,1,1,6,0,6,6,0,0,0,0,0,1,0,1,0,0,0,5,4,0,4,1,0,3,0,3 • 0,2,0,2,0,1,1,0,0,1,0,0,1,0,0,1,6,0,5,6,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,1,0,3,0,2 • 0,0,0,1,0,1,1,0,0,0,0,1,0,0,1,1,1,0,1,1,0,0,0,0,0,1,4,5,0,5,0,0,0,0,0,1,0,2,0,3 • 0,2,0,1,0,1,1,0,2,1,0,0,1,0,1,1,6,0,6,6,0,0,0,0,0,1,0,1,0,0,0,5,4,0,4,1,0,4,0,3
29. 29. So • We get some probable “best” solutions • But is monopoly rock paper scissors? • We should be able to notice this by it not converging • And, TrueSkill has some ”magic” numbers • Blame vincent
30. 30. Future work/known issues • We implemented a single behavior to • Bid in an auction • Buy/Sell houses • When to start buying/selling houses • We did not implement • Trying to buy streets from other players • Opensource the stuffs
31. 31. We’re hiring / Questions / Thank you Niels Zeilemaker