Advertisement
Advertisement

More Related Content

Similar to Keynote at the 2023 Annual Meeting of the Society for the Neuroscience of Creativity(20)

Advertisement

Keynote at the 2023 Annual Meeting of the Society for the Neuroscience of Creativity

  1. LABORATORY FOR QUANTITATIVE EXPERIENCE DESIGN qed.cs.utah.edu Rogelio E. Cardona-Rivera, Ph.D. School of Computing | Entertainment Arts and Engineering University of Utah Society for the Neuroscience of Creativity 2023 Annual Meeting, Keynote 03 24 22 rogelio@cs.utah.edu http://rogel.io @recardona Computational Game Design A Frontier and Roadmap for AI-powered Artifactual Science
  2. Acknowledgements
  3. We have no clue what game design is, how to intentionally structure it, nor how to best support it.
  4. What should you learn to design games? The view from https://youtu.be/zQvWMdWhFCc
  5. Human Cost Telltale Games Rockstar Games
  6. Event Action • Content authoring increases exponentially with player choice ‣ Imagine: Choose-your-own-adventures • The Witcher 3: Wild Hunt (2015) had: ‣ ~450,000 lines of dialogue over 3.5 years ‣ At ~5 words per dialogue: ~2.25M words • The Game of Thrones Saga has: ~1.77M words A Sense of Scale Authorial Combinatorics Problem
  7. The Game Design State-of-the-Art Poorly Understood, Costly, and Effortful
  8. The Game Design State-of-the-Art Poorly Understood, Costly, and Effortful
  9. The Game Design State-of-the-Art Poorly Understood, Costly, and Effortful Surely generative AI can help offset cost and effort… …right? A Generative AI Frontier https://github.com/keijiro/AICommand • Artificial intelligence (AI) technology could help • Unfortunately, current AI is making it worse ‣ We do not understand it ‣ It shifts the design burden
  10. Or, artificial intelligence for creating (game) content A Generative AI Primer Input Output Behavior x f(x) f
  11. With up to 1.8 x 1019 planets A Generative AI Failure
  12. One key reason Generative AI is difficult to use The Kaleidoscope Effect • We can summarize the generative space quickly ‣ It ceases to be novel • Ground (AI) truth ≠ Perceptual truth Cardona-Rivera, Rogelio E.; Cognitively-grounded Procedural Content Generation. In the Workshop on What’s Next for Game AI at the 31st AAAI Conference on Artificial Intelligence. 2017.
  13. Second key reason Generative AI is difficult to use Generative AI and 2nd-Order Design • From “making design moves” to “prompt engineering” ‣ Coaxing computers to output what is needed changes how we design Turkle, Sherry. How Computers Change the Way We Think. Chronicle of Higher Education, vol. 50, iss. 21. January 30, 2004.
  14. It does not have to be this way. AI can help us better understand game design.
  15. The value in asking whether AI can design is… …it begs the question “how do people do it?” • Why it begs the question • Why building AI that designs can yield insight into how people do it • Why the future looks promising in the use of AI for design+creativity research
  16. Why it begs the question “How do people do it?”
  17. What happens when you do not ask “How do people do it?” • Not asking the question leads to context-less tools
  18. What happens when you do not ask “How do people do it?” • Not asking the question leads to context-less tools ‣ AI does not intrinsically help with design information or process
  19. What happens when you do not ask “How do people do it?” • Not asking the question leads to context-less tools ‣ AI does not intrinsically help with design information or process ‣ AI must be situated in context
  20. Game Design So, how do people do it?
  21. We have no clue what game design is, how to intentionally structure it, nor how to best support it. what is game design
  22. game design what is ?
  23. • Kuittinen and Holopainen: ‣ “…an activity similar to any other design field but that the form and the content are specific to the game design context” • So, what is design? game design what is ? Kuittinen, Jussi, and Jussi Holopainen. "Some Notes on the Nature of Game Design." In DiGRA Conference. 2009.
  24. No widely-held consensus on what design is I seek an answer by building AI that does it Output f(x) Given a desired Find to produce it Input Behavior x f
  25. • A set of activities… • …enacted by situated agents… • …who manipulate information… • …to invent an artifact. game design what is ? An operational definition (which?) (who?) (what kinds?) (for what?) Cardona-Rivera, Rogelio E.; Foundations of a Computational Science of Game Design: Abstractions and Tradeoffs. In Proceedings of the 16th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2020.
  26. situated Function—Behavior—Structure Or, one answer to open questions of our operational definition Cardona-Rivera, Rogelio E.; Foundations of a Computational Science of Game Design: Abstractions and Tradeoffs. In Proceedings of the 16th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2020.
  27. A language to talk about design activity and information Cardona-Rivera, Rogelio E.; Foundations of a Computational Science of Game Design: Abstractions and Tradeoffs. In Proceedings of the 16th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2020. Function—Behavior—Structure • 5 (artifact) information variables: ‣ F: What it accomplishes ‣ Be: What behavior achieves F ‣ S: What structure elicits Be ‣ Bs: What behavior S actually achieves ‣ D: How we manufacture S • 8 (creative) activities ‣ Each a function over the variables
  28. A modal language to talk about situated agents and artifacts Cardona-Rivera, Rogelio E.; Foundations of a Computational Science of Game Design: Abstractions and Tradeoffs. In Proceedings of the 16th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2020. situated(ness) • 3 worlds designers inhabit: ‣ External ‣ Interpreted (designer’s understanding) ‣ Expected (designer’s imagination) • Each FBS variable exists in all 3 worlds ‣ Worlds are modes that link the variables
  29. A modal language to talk about situated agents and artifacts Cardona-Rivera, Rogelio E.; Foundations of a Computational Science of Game Design: Abstractions and Tradeoffs. In Proceedings of the 16th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2020. situated(ness) • 3 worlds designers inhabit: ‣ External ‣ Interpreted (designer’s understanding) ‣ Expected (designer’s imagination) • Each FBS variable exists in all 3 worlds ‣ Worlds are modes that link the variables ‣ For example: say we’re designing a game F i F i e F e What our game should accomplish What you think our game should accomplish What might change what our game should accomplish
  30. situated Function—Behavior—Structure Or, one answer to open questions of our operational definition Cardona-Rivera, Rogelio E.; Foundations of a Computational Science of Game Design: Abstractions and Tradeoffs. In Proceedings of the 16th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2020. • FBS describes: ‣ design activities ‣ design information • s(ituatedness) describes: ‣ situated agents (i.e. designers) ‣ artifacts
  31. AI-powered Artifactual Science Roadmap Cardona-Rivera, Rogelio E.; Foundations of a Computational Science of Game Design: Abstractions and Tradeoffs. In Proceedings of the 16th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2020. situated Function—Behavior—Structure • AI offers a modeling language to label game design… ‣ …Inputs, ‣ …Behaviors, and ‣ …Outputs • Game design science efforts can be pinned on the map ! ! ! " " ! !
  32. The value in asking whether AI can design is… …it begs the question “how do people do it?” • Why it begs the question • Why building AI that designs can yield insight into how people do it • Why the future looks promising in the use of AI for design+creativity research
  33. Why building AI that designs can yield insight into how people do it The Case of Generative AI for Storygames
  34. Games that mediate action through a narrative framing • Generative AI-powered authorship requires knowing: ‣ How do people do it today? ‣ How can AI possibly do it? ‣ Toward what end is the authorship? - Baseline: comprehension ✦ How do people comprehend stories? • My work to date has targeted these Storygames Designer Authors with Story Director
  35. Be How do people author stories today? Clarifying narrative design practice • The Sequence Method ‣ Question-answering driven authorship - Keep audiences asking “why?” and “how?” ‣ No narrative design language to account for goals • We had to invent a formal language for game design goals ‣ The GFI Language Cardona-Rivera, Rogelio E.; José P. Zagal, and Michael S. Debus; GFI: A Formal Approach to Narrative Design and Game Research. In Proceedings of the 13th International Conference on Interactive Digital Storytelling, pages 133-148, 2020. # Runner-up for Best Paper S
  36. How do you accomplish that?
  37. How do you accomplish that? Finish Super Mario Bros.
  38. How do you accomplish that? • Ultimate Goals Finish Super Mario Bros. G
  39. Ultimate Goals In-game conditions players meet to succeed at a game Win Finish Prolong
  40. How do you accomplish that? • Ultimate Goals Finish Super Mario Bros. G
  41. How do you accomplish that? • Ultimate Goals • Imperative Goals Finish Super Mario Bros. Remove <B1> G
  42. Imperative Goals Closer-to-gameplay conditions players meet to accomplish the Ultimate Optimize Reach Remove Solve Synchronize Choose Configure Create Find Obtain
  43. How do you accomplish that? • Ultimate Goals • Imperative Goals Finish Super Mario Bros. Remove <B1> G
  44. How do you accomplish that? • Ultimate Goals • Imperative Goals Finish Super Mario Bros. Remove <B1> Reach <L1> G
  45. How do you accomplish that? The Ludological Goal Hierarchy • Ultimate Goals • Imperative Goals Finish Super Mario Bros. Remove <B1> Reach <L1> G
  46. What does that mean? The Ludological Goal Hierarchy • Ultimate Goals • Imperative Goals Finish Super Mario Bros. Remove <B1> Reach <L1> G
  47. What does that mean? The Ludological Goal Hierarchy Finish Super Mario Bros. Remove <B1> Reach <L1> Save the Princess Remove <B1> Reach <L1> G Narrative Goal
  48. What does that mean? The Ludological Goal Hierarchy Finish Super Mario Bros. Remove <B1> Reach <L1> Save the Princess Remove <B1> Reach <L1> G Narrative Goal Interpretation of Ludological Goal I
  49. What does that mean? The Ludological Goal Hierarchy Finish Super Mario Bros. Remove <B1> Reach <L1> Save the Princess Remove <B1> Reach <L1> G Narrative Goal Interpretation of Ludological Goal I How did we get there?
  50. What does that mean? The Ludological Goal Hierarchy Finish Super Mario Bros. Remove <B1> Reach <L1> Save the Princess Remove <B1> Reach <L1> G Narrative Goal Interpretation of Ludological Goal I How did we get there? Feedback F
  51. Feedback Multi-modal stimuli intended to convey perceptual information Phonological Lexical Grammatical Denotational
  52. What does that mean? The Ludological Goal Hierarchy Finish Super Mario Bros. Remove <B1> Reach <L1> Save the Princess Remove <B1> Reach <L1> G I F What does that mean?
  53. What does that mean? The Ludological Goal Hierarchy Finish Super Mario Bros. Remove <B1> Reach <L1> Save the Princess Remove <B1> Reach <L1> G I F What does that mean?
  54. What does that mean? The Ludological Goal Hierarchy Finish Super Mario Bros. Remove <B1> Reach <L1> Save the Princess Defeat Bowser Reach <L1> G I F What does that mean?
  55. What does that mean? The Ludological Goal Hierarchy Finish Super Mario Bros. Remove <B1> Reach <L1> Save the Princess Defeat Bowser Reach <L1> G I F What does that mean?
  56. What does that mean? The Ludological Goal Hierarchy Finish Super Mario Bros. Remove <B1> Reach <L1> Save the Princess Defeat Bowser Destroy Bridge G I F What does that mean?
  57. What does that mean? The Ludological Goal Hierarchy Finish Super Mario Bros. Remove <B1> Reach <L1> Save the Princess Defeat Bowser Destroy Bridge G I F What does that mean?
  58. What does that mean? The Ludological Goal Hierarchy Finish Super Mario Bros. Remove <B1> Reach <L1> Save the Princess Defeat Bowser Destroy Bridge G I F Why do you want to do this?
  59. What does that mean? The Ludological Goal Hierarchy Finish Super Mario Bros. Remove <B1> Reach <L1> Save the Princess Defeat Bowser Destroy Bridge G I F Why do you want to do this?
  60. What does that mean? The Ludological Goal Hierarchy Finish Super Mario Bros. Remove <B1> Reach <L1> Save the Princess Defeat Bowser Destroy Bridge G I F Why do you want to do this?
  61. What does that mean? The Ludological Goal Hierarchy Finish Super Mario Bros. Remove <B1> Reach <L1> Save the Princess Defeat Bowser Destroy Bridge G I F The Parallel Goal Hierarchies
  62. What does that mean? The Ludological Goal Hierarchy Finish Super Mario Bros. Remove <B1> Reach <L1> Save the Princess Defeat Bowser Destroy Bridge G I F The Parallel Goal Hierarchies Ludological Narrative
  63. Be How do people author stories today? Clarifying narrative design practice • The Sequence Method ‣ Question-answering driven authorship - Keep audiences asking “why?” and “how?” ‣ No narrative design language to account for goals • We had to invent a formal language for game design goals ‣ The GFI Language Cardona-Rivera, Rogelio E.; José P. Zagal, and Michael S. Debus; GFI: A Formal Approach to Narrative Design and Game Research. In Proceedings of the 13th International Conference on Interactive Digital Storytelling, pages 133-148, 2020. # Runner-up for Best Paper S
  64. Be How do people author stories today? Clarifying narrative design practice • The Sequence Method ‣ Question-answering driven authorship - Keep audiences asking “why?” and “how?” ‣ No narrative design language to account for goals • We had to invent a formal language for game design goals ‣ The GFI Language ✴ AI-driven Inquiry makes design practice precise Cardona-Rivera, Rogelio E.; José P. Zagal, and Michael S. Debus; GFI: A Formal Approach to Narrative Design and Game Research. In Proceedings of the 13th International Conference on Interactive Digital Storytelling, pages 133-148, 2020. # Runner-up for Best Paper S
  65. Games that mediate action through a narrative framing • Generative AI-powered authorship requires knowing: ‣ How do people do it today? ‣ How can AI possibly do it? ‣ Toward what end is the authorship? - Baseline: comprehension ✦ How do people comprehend stories? • My work to date has targeted these Storygames Designer Authors with Story Director
  66. • Event-based representation: narratives as plans ‣ Story generation as a classical planning problem - : initial state - : goal conditions - : set of (domain) actions, relations, and objects ‣ Given , find the sequence of actions to transform ⟶ How might AI author stories? Abstracting story structures as data structures S P = hsi, g, Di D g si g si P = hsi, g, Di Young, R. Michael, et al. "Plans and planning in narrative generation: a review of plan-based approaches to the generation of story, discourse and interactivity in narratives." Sprache und Datenverarbeitung, Special Issue on Formal and Computational Models of Narrative 37.1-2 (2013): 41-64.
  67. Narratives as Plans • Actions encoded as template operators (events) ‣ Planning Domain Definition Language • PDDL expanded with consenting agents ‣ (:action pick-up :parameters (?agent ?item ?location) :precondition (and (at ?item ?location) (at ?agent ?location)) :effect (and (not (at ?item ?location)) (has ?agent ?item)) :agents (?agent))
  68. Automated Planning AI paradigm that reasons about action and change • Solution to planning problem : a plan ‣ : steps ‣ : variable bindings ‣ : causal links - e.g. P = hsi, g, Di ⇡ = hS, B, Li S hs1, , s2i B L si g Pick-up Disenchant Pick-up (has ARTHUR SPELLBOOK)
  69. A Knight’s Tale Example AI-generated Storygame (define (domain KNIGHT) (:requirements :strips) (:predicates (at ?x ?y) (has ?x ?y) (path ?x ?y) (asleep ?x) (enchanted ?x)) (:action pick-up :parameters (?agent ?item ?location) …) (:action move :parameters (?agent ?from ?to) …) (:action disenchant :parameters (?agent ?obj ?location ?book) …) (:action wake-up :parameters (?agent ?sleeper ?location) …) (define (problem STORY) (:domain KNIGHT) (:objects ARTHUR MERLIN SPELLBOOK MERLINBOOK EXCALIBUR FOREST HOME) (:init (at ARTHUR FOREST) (at MERLIN FOREST) (has MERLIN MERLINBOOK) (asleep MERLIN) (at SPELLBOOK FOREST) (at EXCALIBUR FOREST) (enchanted EXCALIBUR) (path FOREST HOME)) (:goal (has ARTHUR EXCALIBUR)) Rogelio E. Cardona-Rivera and R. Michael Young; Symbolic Plan Recognition in Interactive Narrative Environments. Joint Workshop on Intelligent Narrative Technologies and Social Believability in Games (INT 8), pages 16-22, Santa Cruz, CA, USA, 2015
  70. A Knight’s Tale Example AI-generated Storygame (define (problem STORY) (:domain KNIGHT) (:objects ARTHUR MERLIN SPELLBOOK MERLINBOOK EXCALIBUR FOREST HOME) (:init (at ARTHUR FOREST) (at MERLIN FOREST) (has MERLIN MERLINBOOK) (asleep MERLIN) (at SPELLBOOK FOREST) (at EXCALIBUR FOREST) (enchanted EXCALIBUR) (path FOREST HOME)) (:goal (has ARTHUR EXCALIBUR)) Rogelio E. Cardona-Rivera and R. Michael Young; Symbolic Plan Recognition in Interactive Narrative Environments. Joint Workshop on Intelligent Narrative Technologies and Social Believability in Games (INT 8), pages 16-22, Santa Cruz, CA, USA, 2015
  71. (define (problem STORY) (:domain KNIGHT) (:objects ARTHUR MERLIN SPELLBOOK MERLINBOOK EXCALIBUR FOREST HOME) (:init (at ARTHUR FOREST) (at MERLIN FOREST) (has MERLIN MERLINBOOK) (asleep MERLIN) (at SPELLBOOK FOREST) (at EXCALIBUR FOREST) (enchanted EXCALIBUR) (path FOREST HOME)) (:goal (has ARTHUR EXCALIBUR)) A Knight’s Tale Example AI-generated Storygame Rogelio E. Cardona-Rivera and R. Michael Young; Symbolic Plan Recognition in Interactive Narrative Environments. Joint Workshop on Intelligent Narrative Technologies and Social Believability in Games (INT 8), pages 16-22, Santa Cruz, CA, USA, 2015
  72. (define (problem STORY) (:domain KNIGHT) (:objects ARTHUR MERLIN SPELLBOOK MERLINBOOK EXCALIBUR FOREST HOME) (:init (at ARTHUR FOREST) (at MERLIN FOREST) (has MERLIN MERLINBOOK) (asleep MERLIN) (at SPELLBOOK FOREST) (at EXCALIBUR FOREST) (enchanted EXCALIBUR) (path FOREST HOME)) (:goal (has ARTHUR EXCALIBUR)) A Knight’s Tale Example AI-generated Storygame Rogelio E. Cardona-Rivera and R. Michael Young; Symbolic Plan Recognition in Interactive Narrative Environments. Joint Workshop on Intelligent Narrative Technologies and Social Believability in Games (INT 8), pages 16-22, Santa Cruz, CA, USA, 2015
  73. (define (problem STORY) (:domain KNIGHT) (:objects ARTHUR MERLIN SPELLBOOK MERLINBOOK EXCALIBUR FOREST HOME) (:init (at ARTHUR FOREST) (at MERLIN FOREST) (has MERLIN MERLINBOOK) (asleep MERLIN) (at SPELLBOOK FOREST) (at EXCALIBUR FOREST) (enchanted EXCALIBUR) (path FOREST HOME)) (:goal (has ARTHUR EXCALIBUR)) A Knight’s Tale Example AI-generated Storygame Rogelio E. Cardona-Rivera and R. Michael Young; Symbolic Plan Recognition in Interactive Narrative Environments. Joint Workshop on Intelligent Narrative Technologies and Social Believability in Games (INT 8), pages 16-22, Santa Cruz, CA, USA, 2015
  74. (define (problem STORY) (:domain KNIGHT) (:objects ARTHUR MERLIN SPELLBOOK MERLINBOOK EXCALIBUR FOREST HOME) (:init (at ARTHUR FOREST) (at MERLIN FOREST) (has MERLIN MERLINBOOK) (asleep MERLIN) (at SPELLBOOK FOREST) (at EXCALIBUR FOREST) (enchanted EXCALIBUR) (path FOREST HOME)) (:goal (has ARTHUR EXCALIBUR)) A Knight’s Tale Example AI-generated Storygame Rogelio E. Cardona-Rivera and R. Michael Young; Symbolic Plan Recognition in Interactive Narrative Environments. Joint Workshop on Intelligent Narrative Technologies and Social Believability in Games (INT 8), pages 16-22, Santa Cruz, CA, USA, 2015
  75. (define (problem STORY) (:domain KNIGHT) (:objects ARTHUR MERLIN SPELLBOOK MERLINBOOK EXCALIBUR FOREST HOME) (:init (at ARTHUR FOREST) (at MERLIN FOREST) (has MERLIN MERLINBOOK) (asleep MERLIN) (at SPELLBOOK FOREST) (at EXCALIBUR FOREST) (enchanted EXCALIBUR) (path FOREST HOME)) (:goal (has ARTHUR EXCALIBUR)) A Knight’s Tale Example AI-generated Storygame Rogelio E. Cardona-Rivera and R. Michael Young; Symbolic Plan Recognition in Interactive Narrative Environments. Joint Workshop on Intelligent Narrative Technologies and Social Believability in Games (INT 8), pages 16-22, Santa Cruz, CA, USA, 2015
  76. (define (problem STORY) (:domain KNIGHT) (:objects ARTHUR MERLIN SPELLBOOK MERLINBOOK EXCALIBUR FOREST HOME) (:init (at ARTHUR FOREST) (at MERLIN FOREST) (has MERLIN MERLINBOOK) (asleep MERLIN) (at SPELLBOOK FOREST) (at EXCALIBUR FOREST) (enchanted EXCALIBUR) (path FOREST HOME)) (:goal (has ARTHUR EXCALIBUR)) A Knight’s Tale Example AI-generated Storygame Rogelio E. Cardona-Rivera and R. Michael Young; Symbolic Plan Recognition in Interactive Narrative Environments. Joint Workshop on Intelligent Narrative Technologies and Social Believability in Games (INT 8), pages 16-22, Santa Cruz, CA, USA, 2015
  77. (define (problem STORY) (:domain KNIGHT) (:objects ARTHUR MERLIN SPELLBOOK MERLINBOOK EXCALIBUR FOREST HOME) (:init (at ARTHUR FOREST) (at MERLIN FOREST) (has MERLIN MERLINBOOK) (asleep MERLIN) (at SPELLBOOK FOREST) (at EXCALIBUR FOREST) (enchanted EXCALIBUR) (path FOREST HOME)) (:goal (has ARTHUR EXCALIBUR)) A Knight’s Tale Example AI-generated Storygame Rogelio E. Cardona-Rivera and R. Michael Young; Symbolic Plan Recognition in Interactive Narrative Environments. Joint Workshop on Intelligent Narrative Technologies and Social Believability in Games (INT 8), pages 16-22, Santa Cruz, CA, USA, 2015
  78. A Knight’s Tale Example AI-generated Storygame Rogelio E. Cardona-Rivera and R. Michael Young; Symbolic Plan Recognition in Interactive Narrative Environments. Joint Workshop on Intelligent Narrative Technologies and Social Believability in Games (INT 8), pages 16-22, Santa Cruz, CA, USA, 2015 si g Pick-up Disenchant Pick-up The Player Script
  79. • Event-based representation: narratives as plans ‣ Story generation as a classical planning problem - : initial state - : goal conditions - : set of (domain) actions, relations, and objects ‣ Given , find the sequence of actions to transform ⟶ ✴ AI-driven Inquiry affords evaluating necessity and sufficiency criteria How might AI author stories? Abstracting story structures as data structures S P = hsi, g, Di D g si g si P = hsi, g, Di Young, R. Michael, et al. "Plans and planning in narrative generation: a review of plan-based approaches to the generation of story, discourse and interactivity in narratives." Sprache und Datenverarbeitung, Special Issue on Formal and Computational Models of Narrative 37.1-2 (2013): 41-64.
  80. Games that mediate action through a narrative framing • Generative AI-powered authorship requires knowing: ‣ How do people do it today? ‣ How can AI possibly do it? ‣ Toward what end is the authorship? - Baseline: comprehension ✦ How do people comprehend stories? • My work to date has targeted these Storygames Designer Authors with Story Director
  81. Toward what end is authorship? Baseline: that people comprehend AI-generated storygames So, how do people understand stories? S Bs
  82. Modeling Story(game) Understanding A Computational-Cognitive Model • Readers as Problem Solvers • Automated Planning is a model of Problem Solving • Idea: Use Narrative Plan as proxy for Mental State Gerrig, Richard J., and Allan BI Bernardo. "Readers as problem-solvers in the experience of suspense." Poetics 22.6 (1994): 459-472.
  83. Understanding as Planning • The QUEST Model (Graesser and Franklin, 1990) ‣ Comprehension operationalized as Q&A ‣ Predicts normative answers to Qs about events - Why | How | When did event X happen? What enabled | What was the consequence of event X? ‣ Predictions generated from The QUEST Graph Cardona-Rivera, R. E., et al.; Question Answering in the Context of Stories Generated by Computers. Advances in Cognitive Systems, 4, pages 227-246, 2016. Arthur disenchants Excalibur Excalibur disenchanted Arthur wants disenchanted Arthur wants Excalibur Consequence Outcome Reason Event State Goal
  84. Understanding as Planning • The QUEST Model (Graesser and Franklin, 1990) ‣ Comprehension operationalized as Q&A ‣ Predicts normative answers to Qs about events - Why | How | When did event X happen? What enabled | What was the consequence of event X? ‣ Predictions generated from The QUEST Graph Cardona-Rivera, R. E., et al.; Question Answering in the Context of Stories Generated by Computers. Advances in Cognitive Systems, 4, pages 227-246, 2016. Arthur disenchants Excalibur Excalibur disenchanted Arthur wants disenchanted Arthur wants Excalibur Consequence Outcome Reason Event State Goal Why did Arthur disenchant Excalibur?
  85. Understanding as Planning • The QUEST Model (Graesser and Franklin, 1990) ‣ Comprehension operationalized as Q&A ‣ Predicts normative answers to Qs about events - Why | How | When did event X happen? What enabled | What was the consequence of event X? ‣ Predictions generated from The QUEST Graph Cardona-Rivera, R. E., et al.; Question Answering in the Context of Stories Generated by Computers. Advances in Cognitive Systems, 4, pages 227-246, 2016. Arthur disenchants Excalibur Excalibur disenchanted Arthur wants disenchanted Arthur wants Excalibur Consequence Outcome Reason Event State Goal Why did Arthur disenchant Excalibur?
  86. Understanding as Planning • The QUEST Model (Graesser and Franklin, 1990) ‣ Comprehension operationalized as Q&A ‣ Predicts normative answers to Qs about events - Why | How | When did event X happen? What enabled | What was the consequence of event X? ‣ Predictions generated from The QUEST Graph Cardona-Rivera, R. E., et al.; Question Answering in the Context of Stories Generated by Computers. Advances in Cognitive Systems, 4, pages 227-246, 2016. Arthur disenchants Excalibur Excalibur disenchanted Arthur wants disenchanted Arthur wants Excalibur Consequence Outcome Reason Event State Goal Why did Arthur disenchant Excalibur?
  87. Understanding as Planning • The QUEST Model (Graesser and Franklin, 1990) ‣ Comprehension operationalized as Q&A ‣ Predicts normative answers to Qs about events - Why | How | When did event X happen? What enabled | What was the consequence of event X? ‣ Predictions generated from The QUEST Graph Cardona-Rivera, R. E., et al.; Question Answering in the Context of Stories Generated by Computers. Advances in Cognitive Systems, 4, pages 227-246, 2016. Arthur disenchants Excalibur Excalibur disenchanted Arthur wants disenchanted Arthur wants Excalibur Consequence Outcome Reason Event State Goal Why did Arthur disenchant Excalibur?
  88. Understanding as Planning • The QUEST Model (Graesser and Franklin, 1990) ‣ Comprehension operationalized as Q&A ‣ Predicts normative answers to Qs about events - Why | How | When did event X happen? What enabled | What was the consequence of event X? ‣ Predictions generated from The QUEST Graph Cardona-Rivera, R. E., et al.; Question Answering in the Context of Stories Generated by Computers. Advances in Cognitive Systems, 4, pages 227-246, 2016. Arthur wants disenchanted Arthur wants Excalibur Reason Goal Why did Arthur disenchant Excalibur? Candidate Answers
  89. Understanding as Planning Narrative Plan to QUEST Graph Mapping Algorithm Given a plan : 1. , generate event node ei with a. effects , generate state node ti with 2. Connect Consequence Arcs for all ti → ei , ei →ti+1 in 3. For all literals in , generate goal node li with 4. Connect Reason Arcs for all goal nodes, by ancestry 5. Connect Outcome Arcs for all li →ei in B L ⇡ = hS, B, Li B L L B 8s 2 S 8 e 2 S Cardona-Rivera, R. E., et al.; Question Answering in the Context of Stories Generated by Computers. Advances in Cognitive Systems, 4, pages 227-246, 2016. Mapping Data Structure Semantics to Cognitive Semantics
  90. Understanding as Planning Evaluating the Mapping • Replicated the QUEST Validation Experiment (Graesser, Lang, and Roberts, 1991) ‣ Original: manually-created QUEST graph ‣ Ours: generated QUEST graph • Participants gave goodness-of-answer Likert ratings for Q&A pairs ‣ We used our generated QUEST graph to predict ratings ‣ Strong support for our model (N=695) Cardona-Rivera, R. E., et al.; Question Answering in the Context of Stories Generated by Computers. Advances in Cognitive Systems, 4, pages 227-246, 2016.
  91. Toward what end is authorship? Baseline: that people comprehend AI-generated storygames So, how do people understand stories? ✴ AI-driven Inquiry affords imputing computational mechanisms to design-relevant cognition S Bs
  92. Games that mediate action through a narrative framing • Generative AI-powered authorship requires knowing: ‣ How do people do it today? ‣ How can AI possibly do it? ‣ Toward what end is the authorship? - Baseline: comprehension ✦ How do people comprehend stories? • My work to date has targeted these Storygames Designer Authors with Story Director
  93. Why building AI that designs can yield insight into how people do it The Case of Generative AI for Storygames
  94. Why building AI that designs can yield insight into how people do it The Case of Generative AI for Storygames Generation si g Pick-up Disenchant Pick-up Comprehension Authorship
  95. The value in asking whether AI can design is… …it begs the question “how do people do it?” • Why it begs the question • Why building AI that designs can yield insight into how people do it • Why the future looks promising in the use of AI for design+creativity research
  96. Why the future looks promising in using AI for design+creativity research The Search for Invariants
  97. On Invariants …in Natural Phenomena …in Artifactual Phenomena • Predictive relationship that relates quantities of interest • Predictive relationship that relates quantities of interest <latexit sha1_base64="Gc8Wk3hSnfLMddXi4CR5NKYHCkg=">AAACDHicbVDLSgNBEJyNrxhfUY9eBoPgKeyKohchIIjHKOYByRpmJ7PJkNmZZaZXCEt+wbNX/QZv4tV/8BP8C2eTPWhiQUNR1U13VxALbsB1v5zC0vLK6lpxvbSxubW9U97daxqVaMoaVAml2wExTHDJGsBBsHasGYkCwVrB6CrzW49MG67kPYxj5kdkIHnIKQErPVzjSxzhLvCIGUx65YpbdafAi8TLSQXlqPfK392+oknEJFBBjOl4bgx+SjRwKtik1E0MiwkdkQHrWCqJXeOn06sn+MgqfRwqbUsCnqq/J1ISGTOOAtsZERiaeS8T//UyBZQSZu4ACC/8lMs4ASbpbH+YCAwKZ8ngPteMghhbQqjm9gVMh0QTCja/ks3Gm09ikTRPqt5Z1b09rdTu8pSK6AAdomPkoXNUQzeojhqIIo2e0Qt6dZ6cN+fd+Zi1Fpx8Zh/9gfP5A5Ftm0U=</latexit> F = m ⇥ a <latexit sha1_base64="Rw4UFwukpYQGXy0YZJLiUajRZYc=">AAACDHicbVDLSgNBEJyNrxhfUY9eBoPgKeyKohchkIuejGIekKxhdjKbDJmdWWZ6hbDkFzx71W/wJl79Bz/Bv3A2yUETCxqKqm66u4JYcAOu++XklpZXVtfy64WNza3tneLuXsOoRFNWp0oo3QqIYYJLVgcOgrVizUgUCNYMhtXMbz4ybbiS9zCKmR+RvuQhpwSs9HCDL/E17gCPmMHVbrHklt0J8CLxZqSEZqh1i9+dnqJJxCRQQYxpe24Mfko0cCrYuNBJDIsJHZI+a1sqiV3jp5Orx/jIKj0cKm1LAp6ovydSEhkzigLbGREYmHkvE//1MgWUEmbuAAgv/JTLOAEm6XR/mAgMCmfJ4B7XjIIYWUKo5vYFTAdEEwo2v4LNxptPYpE0TsreWdm9PS1V7mYp5dEBOkTHyEPnqIKuUA3VEUUaPaMX9Oo8OW/Ou/Mxbc05s5l99AfO5w81G5sM</latexit> O = I ⇥ C
  98. On Invariants <latexit sha1_base64="Rw4UFwukpYQGXy0YZJLiUajRZYc=">AAACDHicbVDLSgNBEJyNrxhfUY9eBoPgKeyKohchkIuejGIekKxhdjKbDJmdWWZ6hbDkFzx71W/wJl79Bz/Bv3A2yUETCxqKqm66u4JYcAOu++XklpZXVtfy64WNza3tneLuXsOoRFNWp0oo3QqIYYJLVgcOgrVizUgUCNYMhtXMbz4ybbiS9zCKmR+RvuQhpwSs9HCDL/E17gCPmMHVbrHklt0J8CLxZqSEZqh1i9+dnqJJxCRQQYxpe24Mfko0cCrYuNBJDIsJHZI+a1sqiV3jp5Orx/jIKj0cKm1LAp6ovydSEhkzigLbGREYmHkvE//1MgWUEmbuAAgv/JTLOAEm6XR/mAgMCmfJ4B7XjIIYWUKo5vYFTAdEEwo2v4LNxptPYpE0TsreWdm9PS1V7mYp5dEBOkTHyEPnqIKuUA3VEUUaPaMX9Oo8OW/Ou/Mxbc05s5l99AfO5w81G5sM</latexit> O = I ⇥ C Outer environment (People) Inner environment (Artifact) Context …in Artifactual Phenomena • Predictive relationship that relates quantities of interest ‣ Design Science Equations ‣ Equations as AI systems • Why I am excited for the future: further precision demands your participation • Why I am excited for the future: further precision demands your participation
  99. However…
  100. AI stands poised to help you too via systematically organized neurocognitive computation Westlin, Christiana, et al. "Improving the study of brain-behavior relationships by revisiting basic assumptions." Trends in cognitive sciences (2023).
  101. LABORATORY FOR QUANTITATIVE EXPERIENCE DESIGN qed.cs.utah.edu • Why it begs the question ‣ If we don’t, we get context-less tools that muddle design • Why building AI that designs can yield insight into how people do it ‣ The Case of Generative AI for Storygames • Why the future looks promising in the use of AI for design+creativity research ‣ Reciprocal relationship between AI, Design, and Neuroscience …it begs the question “how do people do it?” rogelio@cs.utah.edu http://rogel.io @recardona The value in asking whether AI can design is…
Advertisement