Interac(ve Storytelling is a form of digital entertainment in which users create or where users can inﬂuence a drama(c storyline through ac(ons, either by issuing commands to the storys protagonist, or ac(ng as a general director of events in the narra(ve. Interac(ve storytelling is a medium where the narra(ve, and its evolu(on, can be inﬂuenced in real-‐(me by a user.
Redeﬁning meaning and context Plot Engagement Story Storytelling User interac3on
Drivers and inﬂuencers New technologies Design skills / approaches Social change Hybridiza(on and convergence Cogni(ve and the behavioral sciences User interfaces The user is the story
From gaming to interac(ve narra(ve spaces Non-‐linear – post-‐structuralist Abstract – nomadic Remix lifestyles Transmedia Mixed reality Collabora(ve interplay and improvisa(on Extension of how narratology and ludology intersect
From programmable movies to simulated reali(es Interference and guidance Alterna(ve plots and storylines Spa(al transforma(on Programmable environments Dialog management Cogni(ve feedback and explora(on The user designer
The IS designer Narra(ve genera(on engine/system (including inbuilt constraint selector and poten(al for diﬀerent POV) Authoring tools 3D Computer graphic soXware/ 3D modelers / anima(on camera visual rendering system interac(on mechanism between mul(ple agents, etc interac(ve authoring interface / hap(cs representa(on soXware/script language iden(ﬁca(on and “registra(on” system, The never-‐ending A.I.planner, etc. list
AI and machine learning Human computer interac(on: pro-‐ac(ve and responsive ac(ons Convergence of skills, performance, analysis and adjustment – improving the user experience Pathﬁnding and planning (incl. non player characters) Natural behavioral language for story-‐based believable agents Interac(ve actors Aﬀec(ve compu(ng Social engineering of characters Reinforcement learning Resource gathering
Character building / agents
LA Noir: adding emo(on • Facial emo(on • Real-‐world actors on the digital stage • Morality systems • Good conversa(ons
The Watchman _ synthe(c characters CG moves into the future Digitally-‐replicated actors wholl go on performing long aAer the actor is gone Rewri3ng the story Bringing back deceased actors
Avatar – super imposing actors Crea(ng an en(re world with computer graphics and then directly integra(ng ac(ng performances Japanese Telexistence surrogate Antropomorphic Head gear, gloves , vests with control gear
On-‐set mo(on capture
The hologram, the performer, the projec(on
Touchable holograms An array of computer-‐controlled ultrasonic wave emiaers with ﬁne points that emit pulses of air that feel like pressure on your hand. System knows where your hand is – behavioral feedback. The sensa(on of weight and texture.
Interac(ng with holograms
Next-‐Genera3on Synthe3c Performers Introducing levels of autonomy into the synthe(c performer (such as eye contact, facial and gestural movement, and lip sync), enabling more ﬂexible, engaging, and directable robot-‐ mediated performance on the set. Second Skin is a project to build a wearable fabric that supports millimeter-‐accurate loca(on and bio-‐parameter tracking at thousands of points on the body. Such a fabric can compute and predict 3-‐D representa(ons of human ac(vity and use the informa(on to augment human performance.
Remote interac(on with a robot Robo(c avatars will allow humans to fully experience the environment of other worlds. Through the eyes of robo(c avatars we will watch the sunrise over the rusty, red crater rims without having to "experience suﬀoca(on, the icy death of -‐200 degrees C on their skin or the s(ng of microscopic dust in their eyes.”
Learning the character’s soul Revisi(ng “Creatures” and Black and White” – the interac(ve storytelling learning versions. Game characters that can predict the behavior of players
The fantas3c ﬂying books of Mr. Morris Lessmore
Interac(ve for the ipad version
Interac(ve Storytelling in music videos Moonbot Studios and The Polyphonic Spree together have created an interac(ve, character-‐ based narra(ve music video. Features: -‐ Seamless world of interac(on -‐ Touch clouds, plants and animals to life -‐ Evolving visuals that change with the music -‐ Follow and interact with a creature named You-‐Me -‐ Tilt your device to sway You-‐Me as he falls -‐ Choose between the Studio or Acous(c song tracks
Context based interac(ve storytelling
Heavy Rain Interac(ve narra(ve if the choices we can make are merely altering speciﬁc events rather than the actual context. Surely the whole point of having the story play out diﬀerently depending on player choices is to add replay value.
Programmable movies Movies that change with context (observer, emo(ons, place, or (me). Long-‐ Distance Barcodes work to make both cameras and the world more intelligent, by allowing users to piece together and merge separate images using metadata encoded into the image. This work will allow for the merging of mul(ple viewpoints to create richer stories from varied storytellers.
Sensing Spaces § Interac(ve reconﬁgurable spaces § Integrated visual, virtual, real spaces § Use of robo(c and AI s(mula(on and simula(on § Sensor clusters and networks to drive performance cues and audio-‐visual interfaces
Sensing Spaces § Gravity to create performance eﬀects § Wearable compu(ng to create performance and eﬀects synchronicity § Narra(ve-‐changing sensors using audience feedback § Use of avatars as performer interfaces
Dynamic spatial interfacesInterac(ve installa(on uses dynamic spa3al interfaces and a human-‐scale display system to allow visitors to explore the photographic work of ar(st Ta(ana Parcero on mul(ple levels of their own choosing.
Interac(ve storytelling in annotated environments iRiS (Immediate Remote Interac(on System User can “paint”, create a game like a jigsaw puzzle, an interac(ve story with one or more par(cipants on a façade from a mobile device. ANW (ar(culated Naturality Web and computer vision
Interac(ve human body museum • Tac(le interac(ve museum experience that takes visitors through the spaces inside their own bodies. • At the museum you can experience hay fever from inside the nose, replete with a rollicking sneeze. • You can don 3D glasses and watch as cartoon sperm fer(lize an egg. You can bounce your way across a rubber tongue as you navigate taste buds and hear burps welling up in the deep.
Adding aﬀect and atmosphere Programmable surfaces and addi(onal displays New hap(cs, with gesture, smell, vibra(on and temperature ampliﬁca(on Immersive and responsive / Reac(ve video and portraits Socially aware systems, ambient and interac(on enhancers such as: causal mappings between past and future events or between an ac(on and its reac(on Programmed atmospheres and aesthe(c visual eﬀects
Situa(on awareness and Behavioral iden((es Ambient detec(on -‐ tools to share habits Annotated environments and Distributed Interac3ve Video Arrays for Event Capture and Enhanced Situa3onal and Interac3ve Awareness Human dynamics and emo(on modeling -‐ aﬀec(ve and cogni(ve percep(on, feedback and awareness -‐ modeling social interfacing and feedback -‐crea(ng behavioral iden((es
You are the performance-‐ SpherAleas device – mobile device with sensors that create sonorous shapes -‐ The Theater of the Senses -‐ "Echo of the Shadow,” Audience par(cipa(on in a labyrinth of performances -‐ Hidden Worlds of Noise and Voice (voice to vision)
Projec(ng yourself into the game Being There This projector-‐based approach provides a way to visualize re-‐crea(ons of real and imaginary sites that are both visually and spa(ally realis(c. Users have a strong sense of immersion and natural interac(on as they walk around a virtual site.
Cameras Computa(onal Cinematography and Display: -‐ future cameras for able to share visual content, and crea(ng 4-‐D and 6-‐D displays for richer collec(on and presenta(on of informa(on. Video of the future (video manipula(on, video coding, 3D, and perceptual studies), computa(onal cinematography, human and facial anima(on, and capture technologies. Addi(onal strengths include wireless networking and computa(onal materials
Sensor robots Capturing 3D models of environments, outdoor and indoor, and capturing ac(vity that is happening in environments, using cameras mounted on mobile robots. A new mode of deploying computer vision is now appearing as autonomous robots
Displays Interac(ve / HolographicTV No border screens Roll out portable OLED screens
Morphable movie studios This technology allows one physical space to represent a variety of things. Projects involve techniques to augment and programmatically change the appearance of physical objects. The physical object is illuminated with a data (or video or slide) projector. The images to be projected are computed with a 3-‐D graphics-‐rendering program. This allows you to change the appearance of real objects, adding special eﬀects to the world around you.
Mobile projec(on: Electric Poetry A web interface for construc(ng poems to be broadcast in real-‐(me on the walls of Londons underground sta(ons, for London Transports Plalorm for Art Similar to the popular f ridge game, people assemble words into poems which then appear in public or commercial spaces around the city
Storyteller Storyteller: Tools and technologies developed into toolkits for children to empower them craX their own compelling stories and characters to foster crea(vity and learning goals. Researchers will also leverage synthe(c performer technologies to create compelling robo(c or virtual characters that serve as learning companions for children
Interac(ve storytelling infancy Interac(ve Storytelling technologies in terms of performance and scalability To make the next genera(on of Interac(ve Storytelling technologies more accessible To develop a more integrated approach to interac(ve storytelling technologies Methodologies to evaluate interac(ve storytelling systems as well as the media experience of interac(ve narra(ve
Going forward Performance and scalability Adap(ng to new IS and user desires Adap(ng new hybrids of converged mix media and technologies Pumng the human in the landscape. There are roles of the integra(on of IS in so many diﬀerent areas – think beyond gaming and ﬁlm Past – Present -‐ Future