This is an assignment done in the university of abertay dundee, for the Artificial Intelligence for Games course. My topics were N-Grams and Bayes Inference. The slides should give a basic introduction to N-Grams predictiors and Bayes Inference, as well as Bayes Networks.

1. AI for Games Seminar
by Andrea Tucci
N-GRAM PREDICTION
AND
BAYES INFERENCE
27/10/2014
Slides by Andrea Tucci - @andreatux

2. OUTLINE
➔ Context
➔ Action Prediction
◆ Techniques
◆ N-Grams
● Window Size
◆ Considerations
➔ Decision Learning
OUTLINE
◆ Techniques
◆ Naive Bayes Classifiers
◆ Bayes Networks
● Bayes Inference
● Probability Propagation
● Games Application
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

3. OUTLINE
OUTLINE - 2
➔ Conclusion
➔ Literature and Further Readings
➔ Questions
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

4. OUTLINE
● Learning in games
Context
○ more dynamism, basing on the player
○ online and offline
○ build a bot from user data/decision
○ difficult to debug
Uses
● Parameter Modification
● Action Prediction
● Decisions
● ...more
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

5. ACTION PREDICTION
Action Prediction
● Study the player’s behaviour, guess next move
● Adaptive AI: the player cannot use the same
technique
● Same challenge, different experience
● A little bit of randomness is needed…
○ otherwise the player will predict the bot!
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

6. ACTION PREDICTION
● Probability
Techniques
○ store data of the player’s choices
○ pick the most predictable
+ very easy to implement
- the player can soon learn the mechanism
● String Matching
○ a string is used to store the player’s decisions
○ to predict the next move, the string is parsed
○ search for identical situation in the string
○ prediction is based on history of choices
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

7. ACTION PREDICTION
Example:
Techniques - 2
● In a RPG context the player can perform a physical
attack and 3 spells: Fire, Blizzard, Thunder.
○ Encoding:
■ Physical Attack: “A”
■ Fire : “F”
■ Blizzard: “B”
■ Thunder “T”
● During a battle we store his/her decisions
○ “AAFBAFBAATTFBBAAFB”
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

8. ACTION PREDICTION
● Next move?
Techniques - 3
● Search in the string what the player did in the past,
after performing a Fire spell followed by a Blizzard
spell.
● AAFBAFBAATTFBBAAFB (?)
○ A physical attack is more predictable
● Window size : how many moves should be looked up
○ is 2 in the example above
○ important for the performance of the prediction
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

9. ACTION PREDICTION
N-Grams
● Record the probabilities for each move given all
combination of choices for the previous “N”
moves.[Millington]
● “N” = window size + 1
○ 3-Gram keeps track the probabilities for a window
size of 2
● Update the list when new actions are performed
● Predict the one with higher probability
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

10. ACTION PREDICTION
N-Grams - 2
Example: Left or Right?
LRRLLLRLLRLRL..(?)
3-Gram predictor
->R ->L
LL 2/3 1/3
LR 1/4 3/4
RL 1/3 2/3
RR 0/1 1/1
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

11. ACTION PREDICTION
N-Grams - 3
Easy to predict things like...
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

12. ACTION PREDICTION
N-Grams - Window Size
● Determine the accuracy of the prediction
● Increasing the window size means better prediction
○ ...until it get worse
● Optimal window size should not be too large
○ Long sequences tend to have too much randomness
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

13. ACTION PREDICTION
N-Grams - Window Size
Accuracy of N-Grams for the Left-Right game.[Millington]
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

14. ACTION PREDICTION
Considerations
● The more the window increase in size, the more memory
is required to store probabilities
● Best performance are achieved when sequences are
“known”
○ otherwise learning takes a while…
● Very suitable for combat games, computational linguistics,
computational biology and in general pattern-based
actions
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

15. DECISION LEARNING
Decision Learning
● Learn how to make decisions
● From a set of possible actions, AI learn which one apply
basing on observation data
● Weak and strong supervision
○ give feedback to the AI choices
● Huge amount of information in games
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

16. DECISION LEARNING
Techniques
● Decision Tree Learning
● Artificial Neural Networks
● Naive Bayes Classifiers
● Reinforcement Learning
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

17. DECISION LEARNING
Naive Bayes Classifiers
● Decide an action from previous observation
○ can be other players decisions, in the same context
● Basing on evidence, gives the probability of “how good” is
executing an action
● Bayes rule
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

18. DECISION LEARNING
Naive Bayes Classifiers - 2
● Core concept: express the conditional probability of A
given B, in terms of its inverse, B given A.
○ the probability that it rained can be expressed as the
observation that the ground is wet
Example: in a soccer game, should we shoot at goal?
● Base our decision on a training set
● Try to find decision patterns
● Data in the example is labeled
○ this help to generalise similar context
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

19. DECISION LEARNING
Naive Bayes Classifiers - Example
Shoot? Distance Direction Markers
Yes Medium Central Low
Yes Large Central None
No Short Central High
No Large Angular High
Yes Short Angular None
No Medium Angular High
Yes Medium Central Medium
No Short Central High
Yes Medium Angular Low
[...] [...] [...] [...]
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

20. DECISION LEARNING
Naive Bayes Classifiers - Example ctd.
Should we shoot if...
○ distance = medium
○ direction = angular
○ markers = none
P ( shoot? | dist, dir, mark) = P(shoot? | dist) * P(shoot? | dir) *
P(shoot? | mark)
● Applying the bayes rule, we have an indication based on
the data collected
○ Google Docs
● Player should shoot!
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

21. DECISION LEARNING
Bayes Networks
● Reason under uncertainty
○ AI reasons in a human like way, no cheats
● Arrange variables and their relationship in a “belief network”
○ acyclic graph
○ variables are encoded as node
○ relationships are arcs
○ variable probability distribution encoded in a table
● Inference using probabilistic reasoning
○ Causal Inference: given A → B, we know A
○ Diagnostic Inference: given A → B, we know B
○ Intercausal Inference: given A → C and B → C, we know C
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

22. DECISION LEARNING
Bayes Networks - 2
Away from home, our neighborhood tell us that he heard our
alarm rang.. What happened?
● P(A) = P(B) * P(E) * P(A|B,E)
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

23. DECISION LEARNING
Bayes Inference
P(B) P(E) P(A|B,E) P(A) N. P(A)
T=.001 T=.002 .95 .000002 .000795
T=.001 F=.998 .94 .000938 .372814
F=.999 T=.002 .29 .000579 .230127
F=.999 F=.998 0.001 .000997 .396264
.002516 1
● If the news inform us on an earthquake… the probability
that a burglary occurred as well, are low!
○ B → A, E → A. P(E) changed and P(B) changed as
well
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

24. DECISION LEARNING
Bayes Inference - 2
Determining the probability of a burglary after the news
P(B) P(E) P(A|B,E) P(A) N. P(A)
T=.001 T=1 .95 .00095 .003269
T=.001 F=0 .94 0 0
F=.999 T=1 .29 .28971 .996731
F=.999 F=0 .001 0 0
.29066 1
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

25. DECISION LEARNING
Bayes Inference - 3
● Find the probability distribution over a variable X given the evidence e
○ P(X | e) ?
● Inference can be exact or approximated
● Computing the probability of a node: summing the probabilities of the
parent nodes
○ chain rule
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

26. DECISION LEARNING
Probability Propagation
● The belief of each node is updated by prior or posterior
evidence updates
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

27. DECISION LEARNING
● Visibility
Bayes Networks in
○ “fog of war”
● Dependency graphs
games
○ RTS technology graph
■ infer existence of some technologies by the
presence of others
■ which dependencies a player is attaining
■ etc
● Detect Intruder
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

28. DECISION LEARNING
Conclusion
● Learning can be used for many reasons in game
development
● Main question.. do we need it?
○ balance effort of building a learning system with
outcomes
● Offline learning is often preferred
● Learning a whole new behaviour is not so easy
● Adaptive AI can be the future of gaming
○ ideally no repetitive challenges
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

29. DECISION LEARNING
Literature and Further Readings
● Artificial Intelligence for Games [Ian Millington, John Funge]
● AI Games Programming Wisdom [Steve Rabin]
● Teaching Bayesian behaviours to video game characters [Ronan Le
Hy, et al]
● MIT open courseware
● PR-OWL
● Bayesian Networks [Judea Pearl, Stuart Russell]
● Design and Development of a Compound DSS for Laboratory
Research [Tomáš Hujer]
● Bayesian Artificial Intelligence [Kevin B. Korb, Ann E. Nicholson]
● Player Prediction techniques for AI in video games [Michael
Webersdorfer]
● Adaptive AI for fighting games [Ricciardi, Thill
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

30. CONCLUSION
That’s it!
Thank you for your
attention
Questions?
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux