AI3391 Artificial Intelligence Session 18 Monto carlo search tree.pptx

1. AI3391 ARTIFICAL INTELLIGENCE
(II YEAR (III Sem))
Department of Artificial Intelligence and Data Science
Session 18
by
Asst.Prof.M.Gokilavani
NIET
1/24/2024 Department of AI & DS 1

2. TEXTBOOK:
• Artificial Intelligence A modern Approach, Third Edition, Stuart
Russell and Peter Norvig, Pearson Education.
REFERENCES:
• Artificial Intelligence, 3rd Edn, E. Rich and K.Knight (TMH).
• Artificial Intelligence, 3rd Edn, Patrick Henny Winston, Pearson
Education.
• Artificial Intelligence, Shivani Goel, Pearson Education.
• Artificial Intelligence and Expert Systems- Patterson, Pearson
Education.
1/24/2024 Department of AI & DS 2

3. Topics covered in session 18
1/24/2024 Department of AI & DS 3
• Game theory
• optimal decision in games
• alpha beta Search
• Monte Carlo tree search
• stochastic games
• partially observed games
• Constraint satisfaction problem
• Constraint propagation
• Backtracking search for CSP
• Local search for CSP
• structure of CSP.

4. Introduction
• Monte Carlo tree search (MCTS) is a heuristic search algorithm for
decision processes, most notably those employed in software that
plays board games.
• It’s a probabilistic and search algorithm that combines classic tree
search implementations alongside machine learning principles
of reinforcement learning.
• Definition: Monte Carlo tree search is a heuristic search algorithm that
relies on intelligent tree search to make decisions. It’s most often used
to perform game simulations, but it can also be utilized in
cybersecurity, robotics and text generation.
1/24/2024 Department of AI & DS 4

5. What Is Monte Carlo Tree Search?
• Monte Carlo tree search only searches a few layers deep into the tree
and prioritizes which parts of the tree to explore.
• It then simulates the outcome rather than exhaustively expanding the
search space.
• In doing so, it limits how many evaluations it has to make.
• The individual evaluation relies on the playout/simulation in which the
algorithm effectively plays the game from a given starting point all the
way to the leaf state by making completely random decisions, and then
it records the results which is then used to update all the nodes in the
random path all the way to the root.
• When it completes the simulation, it then selects the state that has the
best rollout score.
1/24/2024 Department of AI & DS 5

6. What Is a Combinatorial Game?
• Monte Carlo tree search is used in combinatorial games.
• These are sequential games with perfect information.
• The preconditions for a combinatorial game include:
• It must be a two-player game
• It must be a sequential game in which players take turns to play
their move.
• It must have a finite set of well-defined moves.
Examples: chess, checkers, Go and tic-tac-toe.
1/24/2024 Department of AI & DS 6

7. What Is a Game Tree?
• A game tree is a graph representing all possible game states within a
combinatorial game.
• A game tree is a directed graph whose nodes represent a particular
state of the game, and the edges represent possible next states from the
given state.
• The leaf states of the tree represent either a win, lose or tie for the
game.
• This can be used to measure the complexity of a game, as it represents
all the possible ways a game can pan out.
1/24/2024 Department of AI & DS 7

8. Example: Game Tree : Tic Tac Toe
1/24/2024 Department of AI & DS 8

9. How Monte Carlo Tree Search Works
The Monte Carlo tree search algorithm has four phases. We assign state
values and number of interactions for each node.
1. Selection
2. Expansion
3. Simulation
4. Backpropagation
1/24/2024 Department of AI & DS 9

10. Selection
• In this phase, the algorithm uses the following formula to calculate
the state value of all the possible next moves and pick the one which
gives the maximum value.
1/24/2024 Department of AI & DS 10

11. 1/24/2024 Department of AI & DS 11

12. 1/24/2024 Department of AI & DS 12

13. 1/24/2024 Department of AI & DS 13

14. 1/24/2024 Department of AI & DS 14

15. Topics to be covered in next session 19
• Stochastics Games
1/24/2024 Department of AI & DS 15
Thank you!!!