Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Game Engines to Model MAS: A Research Roadmap

229 views

Published on

Game engines are gaining increasing popularity in various computational research areas, and in particular in the context of Multi-Agent Systems (MAS)—for instance, to render augmented reality environments, improve immersive simulation infrastructures, and so on. Existing examples of successful integration between game engines and MAS still focus on specific technology-level goals, rather than on shaping a general-purpose game-based agent-oriented infrastructure. In this roadmap talk, we point out the conceptual issues to be faced to exploit game engines as agent-oriented infrastructures, and outline a possible research roadmap to follow, backed up by some early experiments involving the Unity3D engine.

Published in: Science
  • Be the first to comment

  • Be the first to like this

Game Engines to Model MAS: A Research Roadmap

  1. 1. Game Engines to Model MAS A Research Roadmap Stefano Mariani Andrea Omicini {s.mariani, andrea.omicini}@unibo.it Dipartimento di Informatica – Scienza e Ingegneria (DISI) Alma Mater Studiorum – Universit`a di Bologna XVII Workshop “From Objects to Agents” (WOA 2016) Catania, Italy, 30 July 2016 Mariani & Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 1 / 34
  2. 2. Outline 1 Motivation & Goal 2 Research Landscape 3 GE & MAS 4 Discussion & Roadmap Mariani & Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 2 / 34
  3. 3. Motivation & Goal Outline 1 Motivation & Goal 2 Research Landscape 3 GE & MAS 4 Discussion & Roadmap Mariani & Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 3 / 34
  4. 4. Motivation & Goal Motivation & Goal I huge gap in technological advancement reached by Game Engines (GE) w.r.t. technological level of agent-oriented technologies—e.g. multi-agent systems (MAS) billionaire industry millions testers (gamers too are testers!) stability, performance, usability of software products pushed to unprecedented and incomparable quality ⇒ improve overall quality of MAS technologies huge gap in conceptual and design abstractions provided by GE w.r.t. agent-oriented software engineering poor agenthood notion poor notion of sociality ⇒ improve level of abstraction of GE Mariani & Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 4 / 34
  5. 5. Motivation & Goal Motivation & Goal II other reasons to devote research efforts provide new / improved solutions to, e.g., virtual / mixed / augmented reality systems widen the reach of state-of-art integration efforts, focussed on specific technology-level goals—no general-purpose game-based agent-oriented model Goal Outline a viable research roadmap to follow towards bridging the conceptual and technical gap between MAS and GE Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 5 / 34
  6. 6. Research Landscape Outline 1 Motivation Goal 2 Research Landscape 3 GE MAS 4 Discussion Roadmap Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 6 / 34
  7. 7. Research Landscape State-of-art: Achievements QuizMASter links MAS agents to game engines characters for educational learning [BL11] CIGA models both agents and environment for general-purpose virtual agents in virtual environments [vOVD12] GameBots provides a development framework and runtime for testing multi-agent systems in virtual environments [KVS+02] UTSAF focusses on environment modelling for distributed simulations in the military domain [PMHL04] Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 7 / 34
  8. 8. Research Landscape State-of-art: Shortcomings ! except for CIGA, other approaches consider the technological level solely while pursuing integration—no model, no architecture, etc. ! integration is domain- and goal-specific ! most approaches provide rich abstractions to deal with the environment very weak notions of agency basically nothing for sociality Remark Agent-oriented technology are quite strong on the agency side, mixed feelings for the social environment sides—maybe exception made for TuCSoN [OZ99], CArtAgO [RVO07] friends Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 8 / 34
  9. 9. Research Landscape GE: Analysis Rationale We examine some among most popular GE—Unity3D and Unreal Engine detect abstractions and mechanisms likely to have a counterpart in MAS—or providing some support in re-formulating MAS abstractions highlight opportunities for closing conceptual / technical gaps hindering integration Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 9 / 34
  10. 10. Research Landscape Looking at GE through MAS Eyes Unity3D I Main abstractions (sort of) game object only first-class citizen, everything a scene contains is a game object—a human player, a Non-Player Character (NPC), an environmental item, etc. script piece of code defining the behaviour to be attached to a game object executed by the unique control loop each script executed sequentially no concurrency, all scripts must be executed once within each frame rendering step co-routine sort of workaround to sequentiality imposed by scripts, enables partition and distribute a computation over multiple frame rendering steps Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 10 / 34
  11. 11. Research Landscape Looking at GE through MAS Eyes Unity3D II gap w.r.t. MAS no first-class agenthood abstraction provided societal abstraction missing environment modelling, control, and interaction far superior w.r.t. average MAS technology1 ⇒ e.g. many forms of agent-environment interaction such as shaping movement pathways, handling obstacle avoidance and collision detection, etc. opportunities combine a behavioural script with a co-routine for injecting (some degree of) autonomy into game objects—towards agenthood some forms of message passing among game objects may enable primitive message-based coordination 1 JADE [BCG07], Jason [BHW07], RETSINA [SPVVG03], TuCSoN [OZ99], CArtAgO [RPV11] Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 11 / 34
  12. 12. Research Landscape Looking at GE through MAS Eyes Unreal Engine I Main abstractions (sort of) game object similar to Unity3D game objects actor any component which can be rendered and whose behaviour is enacted by a controller—characters are actors with humanoid resemblance and capabilities (e.g. walking) blueprint similar to Unity3D script, the code specifying the behaviour of an actor direct blueprint communication enables blueprints to communicate one-to-one event dispatcher producer in a publish/subscribe architecture blueprints are consumers events are game-related happenings (e.g. level started, damage taken, shots fired, etc.) triggering execution of blueprints code Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 12 / 34
  13. 13. Research Landscape Looking at GE through MAS Eyes Unreal Engine II gap w.r.t. MAS ? agenthood through controllers? ? societies through event-groups? environment modelling, control, and interaction superior w.r.t. average MAS technology, similarly to Unity3D opportunities agenthood can be reconstructed based on actors and blueprints social interaction may be engineered on top of event dispatchers and direct blueprint communications Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 13 / 34
  14. 14. GE MAS Outline 1 Motivation Goal 2 Research Landscape 3 GE MAS 4 Discussion Roadmap Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 14 / 34
  15. 15. GE MAS The Mirror Worlds Approach mirror worlds (MW) are virtual representations of the “real” world [RCBV15]—which, as in our case, may be virtual too the MW model may help integrating GE and MAS ⇒ MAS agents may have GE avatars if they need a virtual representation—e.g. for interaction-related reasons ⇒ MAS agents rely on GE to represent the physical / virtual environment they are immersed in—thus, situatedness-related issues ⇒ the social dimension might be kept within the MAS world if direct communication between agents is needed . . . ⇒ . . . or expanded to the GE virtual world if communication mediated by the environment is needed ! nevertheless, the virtual world (GE) may influence social interaction in the MAS world—e.g. by either facilitating or hindering discovery Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 15 / 34
  16. 16. GE MAS Case Study: Dining Philosophers with Unity3D Situated Dining Philosophers: Scenario I A well-known coordination problem, tackled in a shared-space setting while extended towards situatedness 5 philosophers share 1 table with 1 spaghetti bowl, 5 smaller bowls, 5 chopsticks, and 5 chairs ! philosophers need 2 chopsticks each to eat a table object plays the role of the tuplespace, chopstick objects are shared tuples, and chair objects decouple and mediate interaction, while enabling situated coordination ! philosopher ⇐⇒ chopsticks assignment dynamically based on position (chair) at the table Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 16 / 34
  17. 17. GE MAS Case Study: Dining Philosophers with Unity3D Situated Dining Philosophers: Scenario II Figure: Gray philosophers are thinking; green chairs are free Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 17 / 34
  18. 18. GE MAS Case Study: Dining Philosophers with Unity3D Situated Dining Philosophers: GE-based Solution I tuple space =⇒ table game object decorated with a tupleSet (list of strings) representing the tuple space a inputQueue tracking requests for tuples (Unity3D messages) a pendingQueue tracking yet unsatisfied requests the script implementing tuple space API request types supported: out, in, rd operationally, at every frame rendering step 1 pendingQueue is considered first, checking request type if out, pendingQueue is checked looking for matching requests, which are served (if any), or tuple is put into tupleSet (if none) if in, when satisfiable, matching tuple is returned and removed from tupleSet—same for rd except removal 2 inputQueue is considered then 3 in case requests cannot be satisfied, they are either moved from inputQueue to pendingQueue, or kept in the latter Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 18 / 34
  19. 19. GE MAS Case Study: Dining Philosophers with Unity3D Situated Dining Philosophers: GE-based Solution II agents =⇒ humanoid game objects requesting coordination primitives synchronously through co-routines agents wait for reply by suspending execution check reply in next frame mediators (e.g., ACC [Omi02]) =⇒ chair game objects decoupling interaction while supporting (spatial) situatedness, by enabling philosophers to acquire chopsticks based on seat philosopher agents ask for chopsticks to the chair they are sit on, rather than directly to table ! philosophers do not explicitly refer chopsticks—not by name, neither by address, etc. ! chairs knows which chopsticks to consider through raytracing, enabling game objects to discover other objects chairs ask table for the right chopsticks on behalf of the philosopher sitting on them Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 19 / 34
  20. 20. GE MAS Case Study: Dining Philosophers with Unity3D Situated Dining Philosophers: Execution I 1 philosophers think until hungry 2 when hungry, they seek for a free chair—through raytracing 3 when found, philosopher goes there and sits, waiting to acquire chopsticks routing and collision avoidance—free lunch thanks to GE 4 when occupied, chairs understand which chopsticks to acquire based on position w.r.t. table—through raytracing spatial situatedness—free lunch thanks to GE 5 if chopsticks requests satisfied, the philosopher starts eating 6 if at least one request is not satisfied, the philosopher waits ! all interactions happen through Unity3D messaging facilities Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 20 / 34
  21. 21. GE MAS Case Study: Dining Philosophers with Unity3D Situated Dining Philosophers: Execution II Figure: Video available at https://youtu.be/tLFaDkcxsE8 Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 21 / 34
  22. 22. Discussion Roadmap Outline 1 Motivation Goal 2 Research Landscape 3 GE MAS 4 Discussion Roadmap Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 22 / 34
  23. 23. Discussion Roadmap Lessons Learnt I ! flow of control is in GE hands no matter what, because fluidity of rendering is the foremost concern clash with definition of autonomy as encapsulation of both control flow and criteria to handle it ⇒ co-routines in Unity3D and actors in Unreal Engine may help with agenthood ! environmental resources modelling and situated interaction are first-class citizens in any GE MAS likely to benefit of some free lunches—e.g. movement pathways computation, objects discovery, collision avoidance, etc. ! GE support to sociality depending on nature of interaction ? direct interaction =⇒ basic message-passing facilities—is this enough? (environment) mediated interaction =⇒ situatedness-related features such as discovery, routing, etc. Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 23 / 34
  24. 24. Discussion Roadmap Lessons Learnt II Viable path to follow Integration is likely to happen, at least at first, through environmental abstractions, since they are the best represented ones in GE world Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 24 / 34
  25. 25. Discussion Roadmap Now What? I Mirror worlds-like approaches seem appropriate ! leave autonomy to MAS, do not abuse GE to re-build agenthood ⇒ goal-directed / oriented behaviour stays within MAS. . . . . . but takes advantage of GE features for, e.g., situated planning to estimate effects of practical situated actions ! exploit GE environmental abstractions as a bridge between GE and MAS world ⇒ e.g. through the notion of artefact [RPV11] as defined in AA meta-model [ORV08]. . . . . . directly supporting situated (inter)action required by agenthood and sociality ! (re)shape sociality around environment of GE-based MAS ⇒ direct communication stays within MAS. . . . . . but accessory facilities might be delegated to GE—e.g. discovery of recipients based on spatial proximity Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 25 / 34
  26. 26. Discussion Roadmap Now What? II The goal to pursue A GE-based MAS infrastructure could then be designed by integrating mechanisms and paradigms around environmental abstractions avoiding to abuse either technology to take in charge responsibilities it is not meant to deal with Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 26 / 34
  27. 27. Acknowledgements Thanks Thanks to . . . Mattia Cerbara and Nicola Poli for the case study the anonymous referees for their challenging questions and thoughtful remarks which helped us improving the paper Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 27 / 34
  28. 28. Outline 1 Motivation Goal 2 Research Landscape Looking at GE through MAS Eyes 3 GE MAS Case Study: Dining Philosophers with Unity3D 4 Discussion Roadmap Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 28 / 34
  29. 29. References References I Fabio Luigi Bellifemine, Giovanni Caire, and Dominic Greenwood. Developing Multi-Agent Systems with JADE. Wiley, February 2007. Rafael H. Bordini, Jomi F. H¨ubner, and Michael J. Wooldridge. Programming Multi-Agent Systems in AgentSpeak using Jason. John Wiley Sons, Ltd, October 2007. Jeanne Blair and Fuhua Lin. An approach for integrating 3D virtual worlds with multiagent systems. In Advanced Information Networking and Applications (WAINA), 2011 IEEE Workshops of International Conference on, pages 580–585, March 2011. Gal A. Kaminka, Manuela M. Veloso, Steve Schaffer, Chris Sollitto, Rogelio Adobbati, Andrew N. Marshall, Andrew Scholer, and Sheila Tejada. GameBots: A flexible test bed for multiagent team research. Communications of the ACM, 45(1):43–45, January 2002. Andrea Omicini. Towards a notion of agent coordination context. In Dan C. Marinescu and Craig Lee, editors, Process Coordination and Ubiquitous Computing, chapter 12, pages 187–200. CRC Press, Boca Raton, FL, USA, October 2002. Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 29 / 34
  30. 30. References References II Andrea Omicini, Alessandro Ricci, and Mirko Viroli. Artifacts in the AA meta-model for multi-agent systems. Autonomous Agents and Multi-Agent Systems, 17(3):432–456, December 2008. Special Issue on Foundations, Advanced Topics and Industrial Perspectives of Multi-Agent Systems. Andrea Omicini and Franco Zambonelli. Coordination for Internet application development. Autonomous Agents and Multi-Agent Systems, 2(3):251–269, September 1999. Special Issue: Coordination Mechanisms for Web Agents. Phongsak Prasithsangaree, Joseph Manojlovich, Stephen Hughes, and Mike Lewis. UTSAF: A multi-agent-based software bridge for interoperability between distributed military and commercial gaming simulation. Simulation, 80(12):647–, 2004. Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 30 / 34
  31. 31. References References III Alessandro Ricci, Angelo Croatti, Pietro Brunetti, and Mirko Viroli. Programming mirror worlds: An agent-oriented programming perspective. In Matteo Baldoni, Luciano Baresi, and Mehdi Dastani, editors, Engineering Multi-Agent Systems, volume 9318 of Lecture Notes in Computer Science, pages 191–211. Springer, 2015. 3rd International Workshop, EMAS 2015, Istanbul, Turkey, May 5, 2015, Revised, Selected, and Invited Papers. Alessandro Ricci, Michele Piunti, and Mirko Viroli. Environment programming in multi-agent systems – an artifact-based perspective. Autonomous Agents and Multi-Agent Systems, 23(2):158–192, September 2011. Special Issue: Multi-Agent Programming. Alessandro Ricci, Mirko Viroli, and Andrea Omicini. CArtAgO: A framework for prototyping artifact-based environments in MAS. In Danny Weyns, H. Van Dyke Parunak, and Fabien Michel, editors, Environments for MultiAgent Systems III, volume 4389 of LNAI, pages 67–86. Springer, May 2007. Katia Sycara, Massimo Paolucci, Martin Van Velsen, and Joseph Giampapa. The RETSINA MAS infrastructure. Autonomous Agents and Multi-Agent Systems, 7(1):29–48, 2003. Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 31 / 34
  32. 32. References References IV Joost van Oijen, Lo¨ıs Vanh´ee, and Frank Dignum. CIGA: A middleware for intelligent agents in virtual environments. In Martin Beer, Cyril Brom, Frank Dignum, and Von-Wun Soo, editors, Agents for Educational Games and Simulations: International Workshop, AEGS 2011, Taipei, Taiwan, May 2, 2011. Revised Papers, pages 22–37. Springer, Berlin, Heidelberg, 2012. Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 32 / 34
  33. 33. Extras URLs Slides on APICe → http://apice.unibo.it/xwiki/bin/view/Talks/GamemasWoa2016 on SlideShare → http://www.slideshare.net/andreaomicini/ game-engines-to-model-mas Paper on APICe → http://apice.unibo.it/xwiki/bin/view/Publications/GamemasWoa2016 on CEUR → http://ceur-ws.org/Vol-????/paper ?.pdf Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 33 / 34
  34. 34. Game Engines to Model MAS A Research Roadmap Stefano Mariani Andrea Omicini {s.mariani, andrea.omicini}@unibo.it Dipartimento di Informatica – Scienza e Ingegneria (DISI) Alma Mater Studiorum – Universit`a di Bologna XVII Workshop “From Objects to Agents” (WOA 2016) Catania, Italy, 30 July 2016 Mariani Omicini (UniBo) Game Engines to Model MAS WOA 2016, 30/07/2016 34 / 34

×