SlideShare a Scribd company logo
1 of 62
Download to read offline
expressiveintelligencestudio
Integrating Learning in a
Multi-Scale Agent
Ben Weber
Dissertation Defense
May 18, 2012
expressiveintelligencestudio UC Santa Cruz
Introduction
 AI has a long history of using games to
advance the state of the field
[Shannon 1950]
expressiveintelligencestudio UC Santa Cruz
Real-Time Strategy Games
 Building human-level AI for RTS games
remains an open research challenge
StarCraft II, Blizzard Entertainment
expressiveintelligencestudio UC Santa Cruz
Task Environment Properties
Chess StarCraft Taxi Driving
Fully vs. partially
observable
Fully Partially Partially
Deterministic vs.
stochastic
Deterministic Deterministic* Stochastic
Episodic vs.
sequential
Sequential Sequential Sequential
Static vs. dynamic Static Dynamic Dynamic
Discrete vs.
continuous
Discrete Continuous Continuous
Single vs. multiagent Multi Multi Multi
[Russell & Norvig 2009]
expressiveintelligencestudio UC Santa Cruz
Motivation
 RTS games present complex environments
and complex tasks
 Professional players demonstrate a broad
range of reasoning capabilities
 Human behavior can be observed, emulated,
and evaluated
[Langley 2011, Mateas 2002]
expressiveintelligencestudio UC Santa Cruz
Hypothesis
 Reproducing expert-level StarCraft
gameplay involves integrating
heterogeneous reasoning capabilities
expressiveintelligencestudio UC Santa Cruz
Research Questions
 What competencies are necessary for
expert StarCraft gameplay?
 Which competencies can be learned
from demonstrations?
 How can these competencies be
integrated in a real-time agent?
expressiveintelligencestudio UC Santa Cruz
Overview
 StarCraft
 Multi-Scale AI
 Learning from Demonstration
 Integrating Learning
 Evaluation
expressiveintelligencestudio UC Santa Cruz
StarCraft
 Expert gameplay
 300+ APM
 Evolving meta-game
 Exhibited capabilities
 Estimation
 Anticipation
 Adaptation
[Flash, Pro-gamer]
expressiveintelligencestudio UC Santa Cruz
StarCraft Gameplay
Expand Tech Tree
Manage Economy
Produce Units
Attack Opponent
expressiveintelligencestudio UC Santa Cruz
Gameplay Scales in StarCraft
 Individual
 Squad
 Global
Support
siege line
Worker
harassment
Aggressive mine
placement
expressiveintelligencestudio UC Santa Cruz
State Space
 The following number of states are possible,
considering only unit type and location:
(Type * X * Y)Units
 States on a 256x256 tile map:
(100*256*256)1700 > 1011,500
expressiveintelligencestudio UC Santa Cruz
Decision Complexity
 The set of possible actions that can be executed at a
particular moment:
O(2W(A * P) + 2T(D + S) + B(R + C))
 W – number of workers
 A – number of the type of worker assignments
 P – average number of workspaces
 T – number of troops
 D – number of movement directions
[Aha et al. 2005]
expressiveintelligencestudio UC Santa Cruz
Decision Complexity
 The set of possible actions that can be executed at a
particular moment:
O(W * A * P + T * D * S + B(R + C))
 Assumption
 Unit actions can be selected independently
 Resulting complexity:
 Assuming 50 worker units on a 256x256 tile map
results in more than 1,000,000 possible actions
expressiveintelligencestudio UC Santa Cruz
StarCraft
 Complex gameplay
 Real-world properties
 Highly-competitive
 Sources of expert gameplay
expressiveintelligencestudio UC Santa Cruz
Research Question #1
 What competencies are necessary for
expert StarCraft gameplay?
expressiveintelligencestudio UC Santa Cruz
Multi-Scale AI
 Multiple scales
 Actions are performed across multiple
levels of coordination
 Interrelated tasks
 Performance in each tasks impacts other tasks
 Real-time
 Actions are performed in real time
expressiveintelligencestudio UC Santa Cruz
Reactive Planning
 Provides useful mechanisms for building
multi-scale agents
 Advantages
 Efficient behavior selection
 Interleaved plan expansion and execution
 Disadvantages
 Lacks deliberative capabilities
[Loyall 1997, Mateas 2002]
expressiveintelligencestudio UC Santa Cruz
Agent Design
 Implemented in the ABL reactive planning
language
 Architecture
 Extension of McCoy & Mateas integrated agent
framework
 Partitions gameplay into distinct competencies
 Uses a blackboard for coordination
[McCoy & Mateas 2008]
expressiveintelligencestudio UC Santa Cruz
EISBot Managers
Strategy
Manager
Income
Manager
Production
Manager
Tactics
Manager
Recon
Manager
Gather
Resources
Construct
Buildings
Attack
Opponent
Scout
Opponent
expressiveintelligencestudio UC Santa Cruz
Multi-Scale Idioms
 Design patterns for authoring multi-scale AI
 Idioms
 Message passing
 Daemon behaviors
 Managers
 Unit subtasks
 Behavior locking
expressiveintelligencestudio UC Santa Cruz
Idioms in EISBot
Initial_tree
Tactics Manager Strategy Manager Income Manager
Form Squad
Squad Monitor
Squad Attack Squad Retreat
Attack Enemy Pump Probes
Legend
Subgoal
Daemon behavior
Message passingDragoon Dance
Timing Attack WME Probe Stop WME
expressiveintelligencestudio UC Santa Cruz
Multi-Scale AI
 StarCraft gameplay is multi-scale
 Reactive planning provides mechanisms for
multi-scale reasoning
 Idioms are applied in EISBot to support
StarCraft gameplay
expressiveintelligencestudio UC Santa Cruz
Research Question #2
 Which competencies can be learned
from demonstrations?
expressiveintelligencestudio UC Santa Cruz
Learning from Demonstration
 Objective
 Emulate capabilities exhibited by expert players
by harnessing gameplay demonstrations
 Methods
 Classification and regression model training
 Case-based goal formulation
 Parameter selection for model optimization
expressiveintelligencestudio UC Santa Cruz
Strategy Prediction
 Tasks
 Identify opponent build orders
 Predict when buildings will be constructed
0
100
200
300
400
0 4
Game Time (minutes)
Spawning Pool Timing
[Hsieh & Sun 2008]
expressiveintelligencestudio UC Santa Cruz
Approach
 Feature encoding
 Each player’s actions are encoded in a single vector
 Vectors are labeled using a build-order rule set
 Features describe the game cycle when a unit or
building type is first produced by a player
t, time when x is first produced by P
0, x was not (yet) produced by P
f(x) = {
expressiveintelligencestudio UC Santa Cruz
Strategy Prediction Results
0
0.2
0.4
0.6
0.8
1
0 1 2 3 4 5 6 7 8 9 10 11 12
RecallPrecision
Game Time (minutes)
NNge Boosting Rule Set State Lattice
expressiveintelligencestudio UC Santa Cruz
Strategy Learning
 Task
 Learn build-orders from demonstration
 Trace Algorithm
 Converts replays to a trace representation
 Formulates goals based on most similar situation
q = argminc ϵ L distance(s, c)
g = s + (q’ - q)
[Ontañón et al. 2010]
expressiveintelligencestudio UC Santa Cruz
Trace Retrieval: Example
 Consider a planning window of size 2
S =< 3, 0, 1, 1 >
T1 =< 2, 0, 0.5, 1 >
T2 =< 3, 0, 0.7, 1 >
T3 =< 4, 1, 0.9, 1 >
T4 =< 4, 1, 1.1, 2 >
expressiveintelligencestudio UC Santa Cruz
Trace Retrieval: Step 1
 The system retrieves the most similar case, q
S =< 3, 0, 1, 1 >
T1 =< 2, 0, 0.5, 1 >
T2 =< 3, 0, 0.7, 1 >
T3 =< 4, 1, 0.9, 1 >
T4 =< 4, 1, 1.1, 2 >
expressiveintelligencestudio UC Santa Cruz
Trace Retrieval : Step 2
 q’ is retrieved
S =< 3, 0, 1, 1 >
T1 =< 2, 0, 0.5, 1 >
T2 =< 3, 0, 0.7, 1 >
T3 =< 4, 1, 0.9, 1 >
T4 =< 4, 1, 1.1, 2 >
expressiveintelligencestudio UC Santa Cruz
Trace Retrieval : Step 3
 The difference is computed: T4 – T2 = <1,1,0.4,1>
S =< 3, 0, 1, 1 >
T1 =< 2, 0, 0.5, 1 >
T2 =< 3, 0, 0.7, 1 >
T3 =< 4, 1, 0.9, 1 >
T4 =< 4, 1, 1.1, 2 >
expressiveintelligencestudio UC Santa Cruz
Trace Retrieval : Step 4
 g is computed:
S =< 3, 0, 1, 1 >
T1 =< 2, 0, 0.5, 1 >
T2 =< 3, 0, 0.7, 1 >
T3 =< 4, 1, 0.9, 1 >
T4 =< 4, 1, 1.1, 2 >
g = s + (T4 – T2) = <4, 1, 1.4, 2>
expressiveintelligencestudio UC Santa Cruz
Strategy Learning Results
0
2
4
6
8
10
12
14
0 10 20 30 40 50 60 70 80 90 100
PredictionError(RMSE)
Actions performed by player
Opponent modeling with a window size of 20
Null
IB1
Trace
MultiTrace
expressiveintelligencestudio UC Santa Cruz
State Estimation
 Task
 Estimate enemy positions
given prior observations
 Particle Model
 Apply movement model
 Remove visible particles
 Reweight particles
[Thrun 2002, Bererton 2004]
expressiveintelligencestudio UC Santa Cruz
Parameter Selection
 Free parameters
 Trajectory weights
 Decay rates
 State estimation is represented as an
optimization problem
 Input: parameter weights
 Output: particle model error
 Replays are used to implement a particle model
error function
expressiveintelligencestudio UC Santa Cruz
State Estimation Results
0
20
40
60
80
100
120
140
160
0 2 4 6 8 10 12 14 16 18
ThreatPredictionError
Game Time (Minutes)
Null Model Perfect Tracker Default Model Optimized Model
expressiveintelligencestudio UC Santa Cruz
Learning from Demonstration
 Anticipation
 Classification and regression models
 Adaptation
 Case-based goal formulation
 Estimation
 Model optimization
expressiveintelligencestudio UC Santa Cruz
Research Question #3
 How can these competencies be
integrated in a real-time agent?
expressiveintelligencestudio UC Santa Cruz
Agent Architecture
expressiveintelligencestudio UC Santa Cruz
Integration Approaches
 Augmenting working memory
 External plan generation
 External goal formulation
Working
Memory
External
Components
expressiveintelligencestudio UC Santa Cruz
Augmenting Working Memory
 Supplementing working memory with
additional beliefs
expressiveintelligencestudio UC Santa Cruz
External Plan Generation
 Generating plans outside the scope of ABL
expressiveintelligencestudio UC Santa Cruz
External Goal Formulation
 Formulating goals outside the scope of ABL
expressiveintelligencestudio UC Santa Cruz
Goal-Driven Autonomy
 A framework for building self introspective
agents
 GDA agents monitor plan execution, detect
discrepancies, and explain failures
 Implementations
 Hand-authored rules
 Case-based reasoning
[Molineaux et al. 2010, Muñoz-Avila et al. 2010]
expressiveintelligencestudio UC Santa Cruz
GDA Subtasks
 Expectation generation
 Discrepancy detection
 Explanation generation
 Goal formulation
expressiveintelligencestudio UC Santa Cruz
Implementation
expressiveintelligencestudio UC Santa Cruz
Integrating Learning
 ABL agents can be interfaced with external
learning components
 Applying the GDA model enabled tighter
coordination across capabilities
 EISBot incorporates ABL behaviors, a particle
model, and a GDA implementation
expressiveintelligencestudio UC Santa Cruz
Evaluation
 Claim
 Reproducing expert-level StarCraft
gameplay involves integrating
heterogeneous reasoning capabilities
 Experiments
 Ablation studies
 User study
expressiveintelligencestudio UC Santa Cruz
GDA Ablation Study
 Agent configurations
 Base
 Formulator
 Predictor
 GDA
 Free parameters
 Planning window size
 Look-ahead window size
 Discrepancy period
Discrepancy
Detector
Explanation
Generator
Goal
Formulator
Goal
Manager
Discrepancies
Explanations
Goals
expressiveintelligencestudio UC Santa Cruz
GDA Results
 Overall results from the GDA experiments
Agent
Win
Ratio
Base 0.73
Formulator 0.77
Predictor 0.81
GDA 0.92
expressiveintelligencestudio UC Santa Cruz
User Study
 Experiment setup
 Matches hosted on ICCup
 3 trials
 Testing script
1. Launch StarCraft
2. Connect to server
3. Host match
4. Announce experiment [Dennis Fong, Pro-gamer]
expressiveintelligencestudio UC Santa Cruz
Performance on Tau Cross
0
500
1000
1500
2000
0 10 20 30 40 50
ICCupScore
Number of Games Played
Base
Formulator
Predictor
GDA
expressiveintelligencestudio UC Santa Cruz
ICCup Results
Agent Longinus Python Tau Cross Overall
Base 942 599 669 737
Formulator 980 718 1078 925
Predictor 1111 555 1145 937
GDA 952 860 1293 1035
expressiveintelligencestudio UC Santa Cruz
EISBot Ranking
 Rankings achieved by the complete GDA agent
Trial
Percentile
Ranking
Longinus 32nd
Python 8th
Tau Cross 66th
Average 48th
expressiveintelligencestudio UC Santa Cruz
Evaluation
 Ablation Studies
 Optimized particle model
 Complete GDA model
 Integrating additional capabilities into EISBot
improved performance
 EISBot performed at the level of a competitive
amateur StarCraft player
expressiveintelligencestudio UC Santa Cruz
Conclusion
 Objective
 Identify and realize capabilities necessary for
expert-level StarCraft gameplay in an agent
 Approach
 Decompose gameplay
 Learn capabilities from demonstrations
 Integrate learned gameplay models
 Evaluate versus humans and agents
expressiveintelligencestudio UC Santa Cruz
Contributions
 Idioms for authoring multi-scale agents
 Methods for learning from demonstration
 Integration approaches for ABL agents
expressiveintelligencestudio UC Santa Cruz
Integrating Learning in a Multi-Scale Agent
 Ben G. Weber
 Ph.D. Candidate
 Expressive Intelligence Studio
 UC Santa Cruz
 bweber@soe.ucsc.edu
 Funding
 NSF Grant IIS – 1018954
expressiveintelligencestudio UC Santa Cruz
References
 Aha, Molineaux, & Ponsen. 2005. “Learning to Win: Case-Based Plan
Selection in a Real-Time Strategy Game”, Proceedings of ICCBR.
 Bererton. 2004. “State Estimation for Game AI using Particle Filters”,
Proceedings of AAI Workshop on Challenges in Game AI.
 Hsieh & Sun. 2008. “Building a Player Strategy Model by Analyzing Replays
of Real-Time Strategy Games”, Proceedings of IJCNN.
 Langley. 2011. “Artificial Intelligence and Cognitive Systems”, AISB
Quarterly.
 Loyall. 1997. “Believable Agents: Building Interactive Personalities”, Ph.D.
thesis, CMU.
 Mateas. 2002. “Believable Agents: Building Interactive Personalities”,
Ph.D. thesis, CMU.
expressiveintelligencestudio UC Santa Cruz
References
 McCoy & Mateas. 2008. “An Integrated Agent for Playing Real-Time
Strategy Games”, Proceedings of AAAI.
 Molineaux, Klenk, Aha. 2010. “Goal-Driven Autonomy in a Navy Strategy
Simulation”, Proceedings of AAAI.
 Muñoz-Avila, Aha, Jaidee, Klenk, Molineaux. 2010. “Applying Goal Driven
Autonomy to a Team Shooter Game”, Proceedings of FLAIRS.
 Ontañón, Mishra, Sugandh, Ram. 2010. “On-line Case-Based Planning”,
Computational Intelligence.
 Russell & Norvig. 2009. Artificial Intelligence: A Modern Approach.
 Shannon. 1950. “Programming a Computer for Playing Chess”,
Philosophical magazine .
 Thrun. 2002. “Particle Filters in Robotics”, Proceedings of UAI.

More Related Content

Viewers also liked

Limites et perspectives du cinéma d'horreur français
Limites et perspectives du cinéma d'horreur françaisLimites et perspectives du cinéma d'horreur français
Limites et perspectives du cinéma d'horreur français
AmauryPascaud
 
Bigyan Tatha Prabidhi Niti 2061
Bigyan Tatha Prabidhi Niti 2061Bigyan Tatha Prabidhi Niti 2061
Bigyan Tatha Prabidhi Niti 2061
bishal406
 
Making Effective Presentations
Making Effective PresentationsMaking Effective Presentations
Making Effective Presentations
ssslawton
 
AXON ivy Pte Ltd Profile
AXON ivy Pte Ltd ProfileAXON ivy Pte Ltd Profile
AXON ivy Pte Ltd Profile
Aashish Mehra
 

Viewers also liked (20)

R&D Initial Presentation
R&D Initial PresentationR&D Initial Presentation
R&D Initial Presentation
 
Limites et perspectives du cinéma d'horreur français
Limites et perspectives du cinéma d'horreur françaisLimites et perspectives du cinéma d'horreur français
Limites et perspectives du cinéma d'horreur français
 
FBC, Belmont, MS Weekly Announcements 3 30-14
FBC, Belmont, MS Weekly Announcements 3 30-14FBC, Belmont, MS Weekly Announcements 3 30-14
FBC, Belmont, MS Weekly Announcements 3 30-14
 
Technology 101
Technology 101Technology 101
Technology 101
 
Study Strategies I
Study Strategies IStudy Strategies I
Study Strategies I
 
Memorial ISTEP/IREAD3 parent information 2014
Memorial ISTEP/IREAD3 parent information 2014 Memorial ISTEP/IREAD3 parent information 2014
Memorial ISTEP/IREAD3 parent information 2014
 
Hello angular
Hello angularHello angular
Hello angular
 
Intro to the Austin Developer Scene
Intro to the Austin Developer SceneIntro to the Austin Developer Scene
Intro to the Austin Developer Scene
 
Bigyan Tatha Prabidhi Niti 2061
Bigyan Tatha Prabidhi Niti 2061Bigyan Tatha Prabidhi Niti 2061
Bigyan Tatha Prabidhi Niti 2061
 
Bartkowiak balcerza prezentacja
Bartkowiak balcerza prezentacjaBartkowiak balcerza prezentacja
Bartkowiak balcerza prezentacja
 
Pixelnoid
PixelnoidPixelnoid
Pixelnoid
 
Factsheet
FactsheetFactsheet
Factsheet
 
Technical Analysis
Technical AnalysisTechnical Analysis
Technical Analysis
 
Welcome to First Baptist Church, Belmont, MS
Welcome to First Baptist Church, Belmont, MSWelcome to First Baptist Church, Belmont, MS
Welcome to First Baptist Church, Belmont, MS
 
FBC, Belmont Announcements 2-2-14
FBC, Belmont Announcements 2-2-14FBC, Belmont Announcements 2-2-14
FBC, Belmont Announcements 2-2-14
 
Making Effective Presentations
Making Effective PresentationsMaking Effective Presentations
Making Effective Presentations
 
AXON ivy Pte Ltd Profile
AXON ivy Pte Ltd ProfileAXON ivy Pte Ltd Profile
AXON ivy Pte Ltd Profile
 
the seven deadly sins of twitter marketing
the seven deadly sins of twitter marketingthe seven deadly sins of twitter marketing
the seven deadly sins of twitter marketing
 
Holding Effective Data Meetings
Holding Effective Data MeetingsHolding Effective Data Meetings
Holding Effective Data Meetings
 
Mermaid Telecommunication
Mermaid Telecommunication Mermaid Telecommunication
Mermaid Telecommunication
 

Similar to Dissertation defense

Tracking a soccer game with big data
Tracking a soccer game with big dataTracking a soccer game with big data
Tracking a soccer game with big data
WSO2
 
Transfer Learning for Performance Analysis of Highly-Configurable Software
Transfer Learning for Performance Analysis of Highly-Configurable SoftwareTransfer Learning for Performance Analysis of Highly-Configurable Software
Transfer Learning for Performance Analysis of Highly-Configurable Software
Pooyan Jamshidi
 
Big Data in the Real World. Real-time Football Analytics
Big Data in the Real World. Real-time Football AnalyticsBig Data in the Real World. Real-time Football Analytics
Big Data in the Real World. Real-time Football Analytics
WSO2
 
ODSC 2019: Sessionisation via stochastic periods for root event identification
ODSC 2019: Sessionisation via stochastic periods for root event identificationODSC 2019: Sessionisation via stochastic periods for root event identification
ODSC 2019: Sessionisation via stochastic periods for root event identification
Kuldeep Jiwani
 

Similar to Dissertation defense (20)

Machine Learning With R
Machine Learning With RMachine Learning With R
Machine Learning With R
 
Tracking a soccer game with Big Data
Tracking a soccer game with Big DataTracking a soccer game with Big Data
Tracking a soccer game with Big Data
 
Strata 2014 Talk:Tracking a Soccer Game with Big Data
Strata 2014 Talk:Tracking a Soccer Game with Big DataStrata 2014 Talk:Tracking a Soccer Game with Big Data
Strata 2014 Talk:Tracking a Soccer Game with Big Data
 
Software Testing:
 A Research Travelogue 
(2000–2014)
Software Testing:
 A Research Travelogue 
(2000–2014)Software Testing:
 A Research Travelogue 
(2000–2014)
Software Testing:
 A Research Travelogue 
(2000–2014)
 
Baseball Prediction Model on Tensorflow
Baseball Prediction Model on TensorflowBaseball Prediction Model on Tensorflow
Baseball Prediction Model on Tensorflow
 
Tracking a soccer game with big data
Tracking a soccer game with big dataTracking a soccer game with big data
Tracking a soccer game with big data
 
Transfer Learning for Performance Analysis of Highly-Configurable Software
Transfer Learning for Performance Analysis of Highly-Configurable SoftwareTransfer Learning for Performance Analysis of Highly-Configurable Software
Transfer Learning for Performance Analysis of Highly-Configurable Software
 
Machine Learning: Classification Concepts (Part 1)
Machine Learning: Classification Concepts (Part 1)Machine Learning: Classification Concepts (Part 1)
Machine Learning: Classification Concepts (Part 1)
 
C programs
C programsC programs
C programs
 
Introduction to computing Processing and performance.pdf
Introduction to computing Processing and performance.pdfIntroduction to computing Processing and performance.pdf
Introduction to computing Processing and performance.pdf
 
Big Data in the Real World. Real-time Football Analytics
Big Data in the Real World. Real-time Football AnalyticsBig Data in the Real World. Real-time Football Analytics
Big Data in the Real World. Real-time Football Analytics
 
Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad
Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris HaddadStructure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad
Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad
 
Deep Learning And Business Models (VNITC 2015-09-13)
Deep Learning And Business Models (VNITC 2015-09-13)Deep Learning And Business Models (VNITC 2015-09-13)
Deep Learning And Business Models (VNITC 2015-09-13)
 
Dynamic Programming and Reinforcement Learning applied to Tetris Game
Dynamic Programming and Reinforcement Learning applied to Tetris GameDynamic Programming and Reinforcement Learning applied to Tetris Game
Dynamic Programming and Reinforcement Learning applied to Tetris Game
 
ODSC 2019: Sessionisation via stochastic periods for root event identification
ODSC 2019: Sessionisation via stochastic periods for root event identificationODSC 2019: Sessionisation via stochastic periods for root event identification
ODSC 2019: Sessionisation via stochastic periods for root event identification
 
Entity-Relationship Queries over Wikipedia
Entity-Relationship Queries over WikipediaEntity-Relationship Queries over Wikipedia
Entity-Relationship Queries over Wikipedia
 
Machine Learning with Go
Machine Learning with GoMachine Learning with Go
Machine Learning with Go
 
Chapter 1 Basic Concepts
Chapter 1 Basic ConceptsChapter 1 Basic Concepts
Chapter 1 Basic Concepts
 
Analyzing a Soccer Game with WSO2 CEP
Analyzing a Soccer Game with WSO2 CEPAnalyzing a Soccer Game with WSO2 CEP
Analyzing a Soccer Game with WSO2 CEP
 
Yoyak ScalaDays 2015
Yoyak ScalaDays 2015Yoyak ScalaDays 2015
Yoyak ScalaDays 2015
 

Recently uploaded

obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
yulianti213969
 
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
acoha1
 
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
ppy8zfkfm
 
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
dq9vz1isj
 
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotecAbortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh +966572737505 get cytotec
 
obat aborsi Bontang wa 082135199655 jual obat aborsi cytotec asli di Bontang
obat aborsi Bontang wa 082135199655 jual obat aborsi cytotec asli di  Bontangobat aborsi Bontang wa 082135199655 jual obat aborsi cytotec asli di  Bontang
obat aborsi Bontang wa 082135199655 jual obat aborsi cytotec asli di Bontang
siskavia95
 
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
jk0tkvfv
 

Recently uploaded (20)

Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...
Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...
Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...
 
Seven tools of quality control.slideshare
Seven tools of quality control.slideshareSeven tools of quality control.slideshare
Seven tools of quality control.slideshare
 
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
 
The Significance of Transliteration Enhancing
The Significance of Transliteration EnhancingThe Significance of Transliteration Enhancing
The Significance of Transliteration Enhancing
 
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
 
NOAM AAUG Adobe Summit 2024: Summit Slam Dunks
NOAM AAUG Adobe Summit 2024: Summit Slam DunksNOAM AAUG Adobe Summit 2024: Summit Slam Dunks
NOAM AAUG Adobe Summit 2024: Summit Slam Dunks
 
Genuine love spell caster )! ,+27834335081) Ex lover back permanently in At...
Genuine love spell caster )! ,+27834335081)   Ex lover back permanently in At...Genuine love spell caster )! ,+27834335081)   Ex lover back permanently in At...
Genuine love spell caster )! ,+27834335081) Ex lover back permanently in At...
 
社内勉強会資料_Object Recognition as Next Token Prediction
社内勉強会資料_Object Recognition as Next Token Prediction社内勉強会資料_Object Recognition as Next Token Prediction
社内勉強会資料_Object Recognition as Next Token Prediction
 
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
 
MATERI MANAJEMEN OF PENYAKIT TETANUS.ppt
MATERI  MANAJEMEN OF PENYAKIT TETANUS.pptMATERI  MANAJEMEN OF PENYAKIT TETANUS.ppt
MATERI MANAJEMEN OF PENYAKIT TETANUS.ppt
 
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
 
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
 
Statistics Informed Decisions Using Data 5th edition by Michael Sullivan solu...
Statistics Informed Decisions Using Data 5th edition by Michael Sullivan solu...Statistics Informed Decisions Using Data 5th edition by Michael Sullivan solu...
Statistics Informed Decisions Using Data 5th edition by Michael Sullivan solu...
 
Bios of leading Astrologers & Researchers
Bios of leading Astrologers & ResearchersBios of leading Astrologers & Researchers
Bios of leading Astrologers & Researchers
 
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotecAbortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
 
Aggregations - The Elasticsearch "GROUP BY"
Aggregations - The Elasticsearch "GROUP BY"Aggregations - The Elasticsearch "GROUP BY"
Aggregations - The Elasticsearch "GROUP BY"
 
obat aborsi Bontang wa 082135199655 jual obat aborsi cytotec asli di Bontang
obat aborsi Bontang wa 082135199655 jual obat aborsi cytotec asli di  Bontangobat aborsi Bontang wa 082135199655 jual obat aborsi cytotec asli di  Bontang
obat aborsi Bontang wa 082135199655 jual obat aborsi cytotec asli di Bontang
 
Northern New England Tableau User Group (TUG) May 2024
Northern New England Tableau User Group (TUG) May 2024Northern New England Tableau User Group (TUG) May 2024
Northern New England Tableau User Group (TUG) May 2024
 
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
 
Credit Card Fraud Detection: Safeguarding Transactions in the Digital Age
Credit Card Fraud Detection: Safeguarding Transactions in the Digital AgeCredit Card Fraud Detection: Safeguarding Transactions in the Digital Age
Credit Card Fraud Detection: Safeguarding Transactions in the Digital Age
 

Dissertation defense

  • 1. expressiveintelligencestudio Integrating Learning in a Multi-Scale Agent Ben Weber Dissertation Defense May 18, 2012
  • 2. expressiveintelligencestudio UC Santa Cruz Introduction  AI has a long history of using games to advance the state of the field [Shannon 1950]
  • 3. expressiveintelligencestudio UC Santa Cruz Real-Time Strategy Games  Building human-level AI for RTS games remains an open research challenge StarCraft II, Blizzard Entertainment
  • 4. expressiveintelligencestudio UC Santa Cruz Task Environment Properties Chess StarCraft Taxi Driving Fully vs. partially observable Fully Partially Partially Deterministic vs. stochastic Deterministic Deterministic* Stochastic Episodic vs. sequential Sequential Sequential Sequential Static vs. dynamic Static Dynamic Dynamic Discrete vs. continuous Discrete Continuous Continuous Single vs. multiagent Multi Multi Multi [Russell & Norvig 2009]
  • 5. expressiveintelligencestudio UC Santa Cruz Motivation  RTS games present complex environments and complex tasks  Professional players demonstrate a broad range of reasoning capabilities  Human behavior can be observed, emulated, and evaluated [Langley 2011, Mateas 2002]
  • 6. expressiveintelligencestudio UC Santa Cruz Hypothesis  Reproducing expert-level StarCraft gameplay involves integrating heterogeneous reasoning capabilities
  • 7. expressiveintelligencestudio UC Santa Cruz Research Questions  What competencies are necessary for expert StarCraft gameplay?  Which competencies can be learned from demonstrations?  How can these competencies be integrated in a real-time agent?
  • 8. expressiveintelligencestudio UC Santa Cruz Overview  StarCraft  Multi-Scale AI  Learning from Demonstration  Integrating Learning  Evaluation
  • 9. expressiveintelligencestudio UC Santa Cruz StarCraft  Expert gameplay  300+ APM  Evolving meta-game  Exhibited capabilities  Estimation  Anticipation  Adaptation [Flash, Pro-gamer]
  • 10. expressiveintelligencestudio UC Santa Cruz StarCraft Gameplay Expand Tech Tree Manage Economy Produce Units Attack Opponent
  • 11. expressiveintelligencestudio UC Santa Cruz Gameplay Scales in StarCraft  Individual  Squad  Global Support siege line Worker harassment Aggressive mine placement
  • 12. expressiveintelligencestudio UC Santa Cruz State Space  The following number of states are possible, considering only unit type and location: (Type * X * Y)Units  States on a 256x256 tile map: (100*256*256)1700 > 1011,500
  • 13. expressiveintelligencestudio UC Santa Cruz Decision Complexity  The set of possible actions that can be executed at a particular moment: O(2W(A * P) + 2T(D + S) + B(R + C))  W – number of workers  A – number of the type of worker assignments  P – average number of workspaces  T – number of troops  D – number of movement directions [Aha et al. 2005]
  • 14. expressiveintelligencestudio UC Santa Cruz Decision Complexity  The set of possible actions that can be executed at a particular moment: O(W * A * P + T * D * S + B(R + C))  Assumption  Unit actions can be selected independently  Resulting complexity:  Assuming 50 worker units on a 256x256 tile map results in more than 1,000,000 possible actions
  • 15. expressiveintelligencestudio UC Santa Cruz StarCraft  Complex gameplay  Real-world properties  Highly-competitive  Sources of expert gameplay
  • 16. expressiveintelligencestudio UC Santa Cruz Research Question #1  What competencies are necessary for expert StarCraft gameplay?
  • 17. expressiveintelligencestudio UC Santa Cruz Multi-Scale AI  Multiple scales  Actions are performed across multiple levels of coordination  Interrelated tasks  Performance in each tasks impacts other tasks  Real-time  Actions are performed in real time
  • 18. expressiveintelligencestudio UC Santa Cruz Reactive Planning  Provides useful mechanisms for building multi-scale agents  Advantages  Efficient behavior selection  Interleaved plan expansion and execution  Disadvantages  Lacks deliberative capabilities [Loyall 1997, Mateas 2002]
  • 19. expressiveintelligencestudio UC Santa Cruz Agent Design  Implemented in the ABL reactive planning language  Architecture  Extension of McCoy & Mateas integrated agent framework  Partitions gameplay into distinct competencies  Uses a blackboard for coordination [McCoy & Mateas 2008]
  • 20. expressiveintelligencestudio UC Santa Cruz EISBot Managers Strategy Manager Income Manager Production Manager Tactics Manager Recon Manager Gather Resources Construct Buildings Attack Opponent Scout Opponent
  • 21. expressiveintelligencestudio UC Santa Cruz Multi-Scale Idioms  Design patterns for authoring multi-scale AI  Idioms  Message passing  Daemon behaviors  Managers  Unit subtasks  Behavior locking
  • 22. expressiveintelligencestudio UC Santa Cruz Idioms in EISBot Initial_tree Tactics Manager Strategy Manager Income Manager Form Squad Squad Monitor Squad Attack Squad Retreat Attack Enemy Pump Probes Legend Subgoal Daemon behavior Message passingDragoon Dance Timing Attack WME Probe Stop WME
  • 23. expressiveintelligencestudio UC Santa Cruz Multi-Scale AI  StarCraft gameplay is multi-scale  Reactive planning provides mechanisms for multi-scale reasoning  Idioms are applied in EISBot to support StarCraft gameplay
  • 24. expressiveintelligencestudio UC Santa Cruz Research Question #2  Which competencies can be learned from demonstrations?
  • 25. expressiveintelligencestudio UC Santa Cruz Learning from Demonstration  Objective  Emulate capabilities exhibited by expert players by harnessing gameplay demonstrations  Methods  Classification and regression model training  Case-based goal formulation  Parameter selection for model optimization
  • 26. expressiveintelligencestudio UC Santa Cruz Strategy Prediction  Tasks  Identify opponent build orders  Predict when buildings will be constructed 0 100 200 300 400 0 4 Game Time (minutes) Spawning Pool Timing [Hsieh & Sun 2008]
  • 27. expressiveintelligencestudio UC Santa Cruz Approach  Feature encoding  Each player’s actions are encoded in a single vector  Vectors are labeled using a build-order rule set  Features describe the game cycle when a unit or building type is first produced by a player t, time when x is first produced by P 0, x was not (yet) produced by P f(x) = {
  • 28. expressiveintelligencestudio UC Santa Cruz Strategy Prediction Results 0 0.2 0.4 0.6 0.8 1 0 1 2 3 4 5 6 7 8 9 10 11 12 RecallPrecision Game Time (minutes) NNge Boosting Rule Set State Lattice
  • 29. expressiveintelligencestudio UC Santa Cruz Strategy Learning  Task  Learn build-orders from demonstration  Trace Algorithm  Converts replays to a trace representation  Formulates goals based on most similar situation q = argminc ϵ L distance(s, c) g = s + (q’ - q) [Ontañón et al. 2010]
  • 30. expressiveintelligencestudio UC Santa Cruz Trace Retrieval: Example  Consider a planning window of size 2 S =< 3, 0, 1, 1 > T1 =< 2, 0, 0.5, 1 > T2 =< 3, 0, 0.7, 1 > T3 =< 4, 1, 0.9, 1 > T4 =< 4, 1, 1.1, 2 >
  • 31. expressiveintelligencestudio UC Santa Cruz Trace Retrieval: Step 1  The system retrieves the most similar case, q S =< 3, 0, 1, 1 > T1 =< 2, 0, 0.5, 1 > T2 =< 3, 0, 0.7, 1 > T3 =< 4, 1, 0.9, 1 > T4 =< 4, 1, 1.1, 2 >
  • 32. expressiveintelligencestudio UC Santa Cruz Trace Retrieval : Step 2  q’ is retrieved S =< 3, 0, 1, 1 > T1 =< 2, 0, 0.5, 1 > T2 =< 3, 0, 0.7, 1 > T3 =< 4, 1, 0.9, 1 > T4 =< 4, 1, 1.1, 2 >
  • 33. expressiveintelligencestudio UC Santa Cruz Trace Retrieval : Step 3  The difference is computed: T4 – T2 = <1,1,0.4,1> S =< 3, 0, 1, 1 > T1 =< 2, 0, 0.5, 1 > T2 =< 3, 0, 0.7, 1 > T3 =< 4, 1, 0.9, 1 > T4 =< 4, 1, 1.1, 2 >
  • 34. expressiveintelligencestudio UC Santa Cruz Trace Retrieval : Step 4  g is computed: S =< 3, 0, 1, 1 > T1 =< 2, 0, 0.5, 1 > T2 =< 3, 0, 0.7, 1 > T3 =< 4, 1, 0.9, 1 > T4 =< 4, 1, 1.1, 2 > g = s + (T4 – T2) = <4, 1, 1.4, 2>
  • 35. expressiveintelligencestudio UC Santa Cruz Strategy Learning Results 0 2 4 6 8 10 12 14 0 10 20 30 40 50 60 70 80 90 100 PredictionError(RMSE) Actions performed by player Opponent modeling with a window size of 20 Null IB1 Trace MultiTrace
  • 36. expressiveintelligencestudio UC Santa Cruz State Estimation  Task  Estimate enemy positions given prior observations  Particle Model  Apply movement model  Remove visible particles  Reweight particles [Thrun 2002, Bererton 2004]
  • 37. expressiveintelligencestudio UC Santa Cruz Parameter Selection  Free parameters  Trajectory weights  Decay rates  State estimation is represented as an optimization problem  Input: parameter weights  Output: particle model error  Replays are used to implement a particle model error function
  • 38. expressiveintelligencestudio UC Santa Cruz State Estimation Results 0 20 40 60 80 100 120 140 160 0 2 4 6 8 10 12 14 16 18 ThreatPredictionError Game Time (Minutes) Null Model Perfect Tracker Default Model Optimized Model
  • 39. expressiveintelligencestudio UC Santa Cruz Learning from Demonstration  Anticipation  Classification and regression models  Adaptation  Case-based goal formulation  Estimation  Model optimization
  • 40. expressiveintelligencestudio UC Santa Cruz Research Question #3  How can these competencies be integrated in a real-time agent?
  • 41. expressiveintelligencestudio UC Santa Cruz Agent Architecture
  • 42. expressiveintelligencestudio UC Santa Cruz Integration Approaches  Augmenting working memory  External plan generation  External goal formulation Working Memory External Components
  • 43. expressiveintelligencestudio UC Santa Cruz Augmenting Working Memory  Supplementing working memory with additional beliefs
  • 44. expressiveintelligencestudio UC Santa Cruz External Plan Generation  Generating plans outside the scope of ABL
  • 45. expressiveintelligencestudio UC Santa Cruz External Goal Formulation  Formulating goals outside the scope of ABL
  • 46. expressiveintelligencestudio UC Santa Cruz Goal-Driven Autonomy  A framework for building self introspective agents  GDA agents monitor plan execution, detect discrepancies, and explain failures  Implementations  Hand-authored rules  Case-based reasoning [Molineaux et al. 2010, Muñoz-Avila et al. 2010]
  • 47. expressiveintelligencestudio UC Santa Cruz GDA Subtasks  Expectation generation  Discrepancy detection  Explanation generation  Goal formulation
  • 49. expressiveintelligencestudio UC Santa Cruz Integrating Learning  ABL agents can be interfaced with external learning components  Applying the GDA model enabled tighter coordination across capabilities  EISBot incorporates ABL behaviors, a particle model, and a GDA implementation
  • 50. expressiveintelligencestudio UC Santa Cruz Evaluation  Claim  Reproducing expert-level StarCraft gameplay involves integrating heterogeneous reasoning capabilities  Experiments  Ablation studies  User study
  • 51. expressiveintelligencestudio UC Santa Cruz GDA Ablation Study  Agent configurations  Base  Formulator  Predictor  GDA  Free parameters  Planning window size  Look-ahead window size  Discrepancy period Discrepancy Detector Explanation Generator Goal Formulator Goal Manager Discrepancies Explanations Goals
  • 52. expressiveintelligencestudio UC Santa Cruz GDA Results  Overall results from the GDA experiments Agent Win Ratio Base 0.73 Formulator 0.77 Predictor 0.81 GDA 0.92
  • 53. expressiveintelligencestudio UC Santa Cruz User Study  Experiment setup  Matches hosted on ICCup  3 trials  Testing script 1. Launch StarCraft 2. Connect to server 3. Host match 4. Announce experiment [Dennis Fong, Pro-gamer]
  • 54. expressiveintelligencestudio UC Santa Cruz Performance on Tau Cross 0 500 1000 1500 2000 0 10 20 30 40 50 ICCupScore Number of Games Played Base Formulator Predictor GDA
  • 55. expressiveintelligencestudio UC Santa Cruz ICCup Results Agent Longinus Python Tau Cross Overall Base 942 599 669 737 Formulator 980 718 1078 925 Predictor 1111 555 1145 937 GDA 952 860 1293 1035
  • 56. expressiveintelligencestudio UC Santa Cruz EISBot Ranking  Rankings achieved by the complete GDA agent Trial Percentile Ranking Longinus 32nd Python 8th Tau Cross 66th Average 48th
  • 57. expressiveintelligencestudio UC Santa Cruz Evaluation  Ablation Studies  Optimized particle model  Complete GDA model  Integrating additional capabilities into EISBot improved performance  EISBot performed at the level of a competitive amateur StarCraft player
  • 58. expressiveintelligencestudio UC Santa Cruz Conclusion  Objective  Identify and realize capabilities necessary for expert-level StarCraft gameplay in an agent  Approach  Decompose gameplay  Learn capabilities from demonstrations  Integrate learned gameplay models  Evaluate versus humans and agents
  • 59. expressiveintelligencestudio UC Santa Cruz Contributions  Idioms for authoring multi-scale agents  Methods for learning from demonstration  Integration approaches for ABL agents
  • 60. expressiveintelligencestudio UC Santa Cruz Integrating Learning in a Multi-Scale Agent  Ben G. Weber  Ph.D. Candidate  Expressive Intelligence Studio  UC Santa Cruz  bweber@soe.ucsc.edu  Funding  NSF Grant IIS – 1018954
  • 61. expressiveintelligencestudio UC Santa Cruz References  Aha, Molineaux, & Ponsen. 2005. “Learning to Win: Case-Based Plan Selection in a Real-Time Strategy Game”, Proceedings of ICCBR.  Bererton. 2004. “State Estimation for Game AI using Particle Filters”, Proceedings of AAI Workshop on Challenges in Game AI.  Hsieh & Sun. 2008. “Building a Player Strategy Model by Analyzing Replays of Real-Time Strategy Games”, Proceedings of IJCNN.  Langley. 2011. “Artificial Intelligence and Cognitive Systems”, AISB Quarterly.  Loyall. 1997. “Believable Agents: Building Interactive Personalities”, Ph.D. thesis, CMU.  Mateas. 2002. “Believable Agents: Building Interactive Personalities”, Ph.D. thesis, CMU.
  • 62. expressiveintelligencestudio UC Santa Cruz References  McCoy & Mateas. 2008. “An Integrated Agent for Playing Real-Time Strategy Games”, Proceedings of AAAI.  Molineaux, Klenk, Aha. 2010. “Goal-Driven Autonomy in a Navy Strategy Simulation”, Proceedings of AAAI.  Muñoz-Avila, Aha, Jaidee, Klenk, Molineaux. 2010. “Applying Goal Driven Autonomy to a Team Shooter Game”, Proceedings of FLAIRS.  Ontañón, Mishra, Sugandh, Ram. 2010. “On-line Case-Based Planning”, Computational Intelligence.  Russell & Norvig. 2009. Artificial Intelligence: A Modern Approach.  Shannon. 1950. “Programming a Computer for Playing Chess”, Philosophical magazine .  Thrun. 2002. “Particle Filters in Robotics”, Proceedings of UAI.