Htn in videogames


Published on

Seminar in AI, with chosen topic Hierarchical Task Networks.
Here is provided an overview of HTNs, based on Erol(94) paper. An idea of HTNs application is given by presenting some example based on video-game domain and, in general, real-time application.

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Htn in videogames

  1. 1. Hierarchical Planning inVideo Gamesby Andrea Tucci
  2. 2. Outline
  3. 3. OutlineIntroductionHierarchical Task NetworksOverviewDefinitionExpressivityComplexityApplicationsVideo Games AI : A Hierarchical Task Networks approachPlanning in gamesHTNs in gamesExample of Study : KillzoneConclusionsReferencesOutlineIntroductionHTNsVideo Games Ai : HTNsConclusionsReferences
  4. 4. Introduction
  5. 5. IntroductionHierarchical Planning in a real-time environmentapplication : video gamesOverview of Hierarchical Task NetworksVideo Games AI, focusing on planningKillzone : a commercial game with HTNsShow issues and advantages of Hierarchical Planning indynamic worldsIndexIntroductionHTNsVideo Games Ai : HTNsConclusionsReferences
  6. 6. Hierarchical TaskNetworks
  7. 7. Overview - Classical planning vs HTNsClassical AI planning : how reach a goal? → actionsan action is represented by an operator < c, e>, c is a precondition, e is the effect ofthe actionstates : configurations of the world generated by applying actionsplanner aim is to find the (minimal) path from initial state to the goal state, the path iscomposed of actionsHierarchical Task Network (HTN) planning : actions to reach a (composite) tasktask : composite or primitivecomposite tasks have to be decomposed in subtasksprimitive tasks can be executed directly by applying an operatorplanner decomposes compound tasks of a given task network until only primitive tasksare foundOutlineIntroductionHTNs>>Overview>Definition>Expressivity>Complexity>ApplicationsVideo Games Ai : HTNsConclusionsReferences
  8. 8. DefinitionGiven a first order HTN language L , we have a vocabulary < V, C, P, F, T, N > whereV = { v1, v2, ... } is an infinite set of variablesC is a finite set of constant symbolsP is a finite set of predicate symbolsF is a finite set of primitive tasksT is a finite set of compound tasksN = {n1, n2 ... } is an infinite set of task labelsGoals replaced by tasks. A task can be:goal task, tasks to achieveprimitive task, obtained by executing the corresponding actioncompound task, require decompositione.g. “travelByPlane” may require subtasks like buyTicket, goToAirport, fly, goToDestinationOutlineIntroductionHTNs>Overview>>Definition>Expressivity>Complexity>ApplicationsVideo Games Ai : HTNsConclusionsReferences
  9. 9. DefinitionA Task Network is a collection of tasks and constraints : (( n1 : α1), ... , (nm : αm), Φ)Each αi is a task labeled niΦ is a boolean formula; can contain :variable constraints (e.g x = y, x,y ∈ V or x ∈ V, y ∈ C )temporal ordering (e.g n ≺ n’, n,n’ ∈ N)truth constraints (e.g (n,l) i.e.“n must be true immediately after n, n ∈ N and l is aliteral)Operator is of the form (f(v1, ..., vk),l1, ..., lk) where f ∈ F, v ∈ V and li is a literalin this representation there are no preconditions in operators : they are expressed as goal tasks intask networksA method is a pair (α, d) where α is not a primitive task and d is a task networkThis means : in order to achieve α, we use d (i.e. all its subtasks)Example : (on(v1,v2),d) where d is a task network.OutlineIntroductionHTNs>Overview>>Definition>Expressivity>Complexity>ApplicationsVideo Games Ai : HTNsConclusionsReferences
  10. 10. DefinitionA planning problem is composed of:an initial statea task network, that encode the problem to be solveda planning domain, that contains methods and operatorsOutlineIntroductionHTNs>Overview>>Definition>Expressivity>Complexity>ApplicationsVideo Games Ai : HTNsConclusionsReferences
  11. 11. DefinitionGiven a planning problem with task network d, a rough Algorithm is :1.Perform a compound task by decomposing it in subtask (use a method);replace the compound task with the subtasks provided by the used method2.Are there other compound tasks? If yes goto (1.);3.Find a totally-ordered ground instantiation of d that satisfies all theconstraints.Example (from blocks world domain) : put v1 on v2OutlineIntroductionHTNs>Overview>>Definition>Expressivity>Complexity>ApplicationsVideo Games Ai : HTNsConclusionsReferencesSyntax used :do precedes a primitive taskperform precedes a compound taskachieve precedes a goal task
  12. 12. ExpressivityHTNs provides all the STRIPS concepts (states, actions, goals)translation of a STRIPS encoded domain to HTNs domain done in polynomialtimeIs the opposite true?NO!HTNs can express situations impossible to express using STRIPS operatorsHTNs provide a constraint language to express many type ofinteractionsSTRIPS does not have the same level of abstractionThere exist HTN planning domains that can not be represented byany finite number of STRIPS-style operators1OutlineIntroductionHTNs>Overview>Definition>>Expressivity>Complexity>ApplicationsVideo Games Ai : HTNsConclusionsReferences
  13. 13. ComplexityA planning problem P can bepropositional : no variables are allowedtotally ordered : all tasks in any task network are totally orderedregular : all the task networks contain max one non-primitive task and is the first or the lasttaskprimitive : the task network we need to plan for contains only primitive tasksPlan existence isstrictly semi-decidable2;decidable if methods are restricted to be acyclic3is EXPSPACE-hard and DOUBLE-EXPTIME if the planning probem(P) is restricted to be totally ordered;PSPACE-hard and EXP-TIME if it is propositional4EXPSPACE-complete if P is regular5PSPACE-complete if P is regular and propositional6is NP-complete if P is primitive7OutlineIntroductionHTNs>Overview>Definition>Expressivity>>Complexity>ApplicationsVideo Games Ai : HTNsConclusionsReferences
  14. 14. ComplexityOutlineIntroductionHTNs>Overview>Definition>Expressivity>>Complexity>ApplicationsVideo Games Ai : HTNsConclusionsReferences
  15. 15. ApplicationsSchedulingVideo GamesAutomated Composition of Web ServicesEvaluating Terrorist ThreatsEvacuationMilitary TasksOthersOutlineIntroductionHTNs>Overview>Definition>Expressivity>Complexity>>ApplicationsVideo Games Ai : HTNsConclusionsReferences
  16. 16. Applications - SHOPSimple Hierarchical Ordered Planner 2 (SHOP 2) is ahierarchical planner developed in 2001, written in LISP.OutlineIntroductionHTNs>Overview>Definition>Expressivity>Complexity>>ApplicationsVideo Games Ai : HTNsConclusionsReferences(defdomain basic-example ((:operator (!pickup ?a) () () ((have ?a)))(:operator (!drop ?a) ((have ?a))((have ?a)) ())(:method (swap ?x ?y)((have ?x))((!drop ?x) (!pickup ?y))((have ?y))((!drop ?y) (!pickup ?x)))))(defproblem problem1 basic-example((have banjo)) ((swap banjo kiwi)))Plans:(((!DROP BANJO) (!PICKUP KIWI)))
  17. 17. Video Games AI :A Hierarchical Task NetworksApproach
  18. 18. Planning in GamesPlanning in games arises in the last yearsPlanning approach makes video games agents (bots,NPCs) more dynamic w.r.t. old approachesPlanning approach can split action planning from action executionPlanning increases autonomy of NPCs : they have alternate solutions to fulfill a goalPlanning can handle new situationsFirst Encounter Assault Recon (F.E.A.R.) developed by Monolith was the first commercial gamethat used planning in its AIin 2005 won “best AI award” from GameSpotstill considered as one of the best games for AIGOAP : Goal Oriented Action Planning,decide what to do AND how to do itHierarchical planning : more natural in some situations (reasoning on tasks rather thanactions)OutlineIntroductionHTNsVideo Games Ai : HTNs>>Planning in Games>HTNs in Games>Example of Study>KillzoneConclusionsReferences
  19. 19. Planning in GamesOutlineIntroductionHTNsVideo Games Ai : HTNs>>Planning in Games>HTNs in Games>Example of Study>KillzoneConclusionsReferences
  20. 20. Planning in GamesOutlineIntroductionHTNsVideo Games Ai : HTNs>>Planning in Games>HTNs in Games>Example of Study>KillzoneConclusionsReferences
  21. 21. Hierarchical Task Networks in GamesUsed in both individual and coordination decisionsIn video games AI, actions can be dictated also byOutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>>HTNs in Games>Example of Study>KillzoneConclusionsReferences- error prone +little CPU overhead- human effort- case specificScriptHTNs can be used as script generatorsoffline planning, then the output actions are converted to scriptsOther approach : real-time planning
  22. 22. Examples of StudyUnderstand how HTNs work in practiceReal-Time planningHow planners are integrated with the entire systemDesign of HTN based agentsDifferences with other implementationsOutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>>Example of Study>KillzoneConclusionsReferences
  23. 23. Killzone 2
  24. 24. Killzone 2First Person Shooter, developed by Guerrilla Games published in 2009Need for a new multiplayer bot system...why?In its predecessor, Killzone, coordination between bots of the same team weremissingGoals :Extensible bot architectureImprove Killzone AIInclude cooperative behaviorOutperform old AIEntertain and challenge playersOutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
  25. 25. Killzone 2Quick overview of AIOutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferencesStrategyBotSquad•Goal-based•Defines what to achieve in themission by using heuristics•Creates squads•Execution of mission tasks•Coordinate bots(e.g. targets)•Action synchronization•Execution ofSquad task• Inform their squad
  26. 26. Killzone 2 vs KillzoneKillzone 2KillzoneOutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
  27. 27. Killzone 2 vs Killzone (2)OutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferencesKillzone Killzone 2ApproachGoal selectionGoal satisfactionAtom partFront endGoal-based HTNsHeuristicsHigh-level HTNchoicesBehavior PlanAction Primitive taskBody goals Body goals
  28. 28. Killzone 2 - AIGoal behavior system of Killzone, replaced by task planning mechanismImprove flexibilityDynamic : decomposition of a compound task may lead to different plans (i.e.different behavior)HTN planner based on SHOP syntaxSpecialized primitive tasks : is possible to add facts in the HTN database for a certainamount of time and delete them in any moment(!remember ?period ?fact) (!forget ?fact)HTN planner is used for evaluate the relevance of goals and find how to reach themKnowledge domains provides how tasks are performed and also tasks priorityAgents use a combination of generalized and specialized knowledge rulesOutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
  29. 29. Killzone 2 - AI (2)What should AI do?Decided by high-level HTNsCompound task(s) as outputHow agent should perform decisions?Low-level HTNs decisionsEach agent has a set of daemons that provide info about the worldThe planner requires those facts before it startsHTNs used in squadThere is an invisible leader that uses planner for coordination purposesInformations sharedSquad plans are series of orders for team membersOutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
  30. 30. Killzone 2 - Squad AIOutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferencesStrategyTeam membermessagesFactsDaemonsPlanner PlanTaskexecutionOrder tomembers
  31. 31. Killzone 2 - AI (2)(:method (order_member_defend ?inp_member ?inp_id ?inp_level ?inp_marker ?inp_context_hint)…(branch_advance () ( (!forget member_status ?inp_member **) (!remember - member_status ?inp_member ordered_to_defend ?inp_id) (!start_command_sequence ?inp_member ?inp_level 1) (do_announce_destination_waypoint_to_member ?inp_member) (!order ?inp_member clear_area_filter) (!order ?inp_member set_area_restrictions (call find_areas_to_wp ?inp_member (call get_entity_wp ?inp_marker))) (!order_custom ?inp_member move_to_defend ?inp_marker ?inp_context_hint) (!order_custom ?inp_member send_member_message_custom completed_defend ?inp_id) (order_set_defend_area_restriction ?inp_member (call get_entity_area ?inp_marker)) (!order_custom ?inp_member defend_marker ?inp_marker ?inp_context_hint) (!end_command_sequence ?inp_member) )))OutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
  32. 32. Killzone 2 - AI (2)HTN used in individual bot’s decisionscombat behaviormission specific tasksother (heal, repair ... )OutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences(:method (select_weapon_and_attack_as_turret ?inp_threat)(branch_use_bullets // Only use bullets against humanoids and turrets.(and(or (threat ?inp_threat humanoid) (threat ?inp_threat turret) )(distance_to_threat ?inp_threat ?threat_distance)(call lt ?threat_distance @weapon_bullet_max_range) )((attack_as_turret_using_weapon_pref ?inp_threat wp_bullets)))( branch_use_rockets // Dont use rockets against humanoids and turrets.(and(not (threat ?inp_threat humanoid)) (not (threat ?inp_threat turret))(distance_to_threat ?inp_threat ?threat_distance)(call lt ?threat_distance @weapon_rocket_max_range) )((attack_as_turret_using_weapon_pref ?inp_threat wp_rockets))))
  33. 33. Killzone 2 - agent designOutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
  34. 34. Killzone 2 - real time environmentPlanning ahead is not so good, we are in a real-time environmentLong plans tend to invalidate☛ Local plans are madePlanning done 5 times per second, 2 times per second for squadInvalidation of a decided plan of actions : better plan or changing environmentProblem : re-planning so often isOutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferencesHEAVYHow to deal with this?
  35. 35. Killzone 2 - real time environment (2)While re-planning, if we come up with a current task, just update it.Only decompose the part of the plan which we are interested at the moment, delayingthe other.Re-plan only if a certain percentage of preconditions is not valid anymore (i.e. the worldchanged too much)Strategies for optimize re-planningtasks updatingsubtasks priority order changespartial decomposition of tasksdelayed decomposition of tasksOutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
  36. 36. Killzone 2 - conclusionsOld AI (bots from Domination game mode) vs new AI (Group Strategy) : 174 testsbattles in a multiplayer modalityOutlineIntroductionHTNsVideo Games Ai : HTNs>Planning in Games>HTNs in Games>Example of Study>>KillzoneConclusionsReferences
  37. 37. Conclusions
  38. 38. ConclusionsOverview of HTNsUse of HTNs in video gamesoffline planning (e.g. Script generation)online planningReal-Time (HTN) planningmuch more interestingdeal with re-planningOutlineIntroductionHTNsVideo Games Ai : HTNs>>ConclusionsReferences
  39. 39. Conclusions - HTN in real-time environmentMake planners for real-time environmentsbuild connection to the problem to be solved ... but is like “hard-code” the plannermake full logic representation, gaining expressive power and more generality ... but loosingefficiencyHow can we maintain a “generic” planner with efficiency?Some optimizations:Use critics to prevent backtracking in an unfeasible plancritics are used to remove conflicts within a plan as soon as possiblePartial re-planningwhy re-plan an entire plan? We can adapt it :(example) agent A needs an objectO, so a plan is made. A can’t find O, but he can replace it with another object thathas the same function, no need to re-plan for the whole task (go to the shop etc..)OutlineIntroductionHTNsVideo Games Ai : HTNs>>ConclusionsReferences
  40. 40. Conclusions - HTN in real-time environmentDomain compilationIntroduced for JSHOP plannerunique instance of the planner, generated compiling both the domain and the problem☛ Expert planner for that problemIn real-time environment we only can compile the domainTime slicingAllows to pause and restart the plannerSpread time of planning in more frame☛ Characters are able to perceive eventsOther optimizations based on the problemMake precondition checking light : heavy computations (e.g. pathfinding) are done by sensors of the agent andstored in a working memory☛ Sensors provide up-to-date data to the plannerCaching perceptions as working memory facts, containing important informationsGarbage collector : a sensor should delete a fact added that is no relevant anymoreOutlineIntroductionHTNsVideo Games Ai : HTNs>>ConclusionsReferences
  41. 41. Conclusions - HTN in real-time environmentSo...are HTNs good or not? Can be used well in a real timeenvironment?In general, planning in more than one layer is betterat any given level of the HTN, the number of applicableoperators is smaller than the total number (reduction of asearch space)invalid plan can be detected at a top level and never beconsidered for expansionmore suitable for dynamic world : support of re-planning isgoodOutlineIntroductionHTNsVideo Games Ai : HTNs>>ConclusionsReferences
  42. 42. ReferencesPresentation based onHTN Planning : Complexity and Expressivity (K. Erol, J. Hendler, D. Nau)Ms thesis : A hierarchically-layered multiplayer bot system for a first-person shooter (T. Verweij)Papers consultedSquadSmart Hierarchical Planning and Coordinated Plan Execution for Squads of Characters (P.Gorniak, I. Davis)Agent Architecture Considerations for Real-Time Planning in Games (J. Orkin)Hierarchical Plan Representations for Encoding Strategic Game AI (H. Hoang, S. Lee-Urban, H.Muñoz-AvilaPlanning with Hierarchical Task Networks in Video Games (J.Kelly, A. Botea, S. Koening)Other materialHierarchical Planning in Dynamic Worlds (N.Wallace, Ai Programming Wisdom vol. 2)Killzone 2 Multiplayer Bots slides (Slides by A.Champandard, T.Verweij, R.Straatman)Images of STRIPS-based and HTNs-based games, are taken from AiGameDev.comOutlineIntroductionHTNsVideo Games Ai : HTNsConclusions>>References
  43. 43. ReferencesNotes (from HTN Planning : Complexity and Expressivity)1. Theorem 72. Theorem 13. Theorem 24. Theorem 35. Theorem 46. Theorem 57. Theorem 6OutlineIntroductionHTNsVideo Games Ai : HTNsConclusions>>References
  44. 44. ThanksThank you for your attention!@andreatuxOutlineIntroductionHTNsVideo Games Ai : HTNsConclusionsReferences