Designing Interactions at the Nexus of the Physical and Virtual World

280 views

Published on

Game Lecture Series
ITU Copenhagen
2011

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
280
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Designing Interactions at the Nexus of the Physical and Virtual World

  1. 1. Ars Electronica Futurelab Research & Innovation Group
  2. 2. Definition of what I am NOT talking about com·put·er game (plural com·put·er games) noun Definition: game played on computer: a game in the form of computer software, run on a personal computer or games machine and played by one or more people using a keyboard, mouse, control pad, or joystick. Microsoft Encarta Ars Electronica Futurelab Research & Innovation Group
  3. 3. Most Computer games nevertheless … … use a handful of standardized interfaces ... are build on a very limited number of concepts … ignore context and environment Ars Electronica Futurelab Research & Innovation Group
  4. 4. The reason is obvious This is the way how to sell things efficiently. Ars Electronica Futurelab Research & Innovation Group
  5. 5. But this is what you might miss … Ars Electronica Futurelab Research & Innovation Group
  6. 6. From Immersion … … to Transmersion? Transmersion? Not about making interfaces invisible, but about transforming the interface itself to the main element for the player’s connection to the physical and social environment Ars Electronica Futurelab Research & Innovation Group
  7. 7. An alternative, preliminary Definition of Computer Game … A computer game is structured, computer-based playing with goals rules challenge and interaction goals, rules, challenge, as key components. (work in progress) Ars Electronica Futurelab Research & Innovation Group
  8. 8. Shadowgram Interactive Workshop System 2010A person stands in front of an illuminated screen and isphotographed. The result is a shadow-image in which a humansilhouette is all that can be seen. This picture is then printed out as aminiature sticker and applied to a “map” whose topography has beenderived from a cluster of topics like education, environment, healthsystem and society, whereby the body language of the silhouette, aspeech balloon containing a brief statement, and the respectivethematic cluster coalesce into a message.Shadowgram Creative Catalyst for Social Brainstorming by Shadow Ars Electronica Futurelab Research & Innovation Group
  9. 9. For instance: the person in front of the illuminated screen covers his/her ears with his/her hands while beingphotographed. This silhouette is then allocated to the environment cluster and a speech balloon containing a remarkabout noise pollution is added to it. The upshot is a highly creative, graphic mode of “social brainstorming” and thus ameans of identifying problems and bringing out potential solutions."Shadowgram" is a work by the Ars Electronica Futurelab (AT). Ars Electronica Futurelab Research & Innovation Group
  10. 10. Ars Electronica Futurelab Research & Innovation Group
  11. 11. Ars Electronica Futurelab Research & Innovation Group
  12. 12. Ars Electronica Futurelab Research & Innovation Group
  13. 13. “Robotinity” is a portmanteau word that blends robotics and humanity and thus succinctly expresses the new RoboLab’s centraltheme: emblematic examples culled from fields of art, design and science illustrate just how closely human beings and machines arealready living and working together. These highly charged facts & circumstances will serve up a foretaste of the risks and rewards ourshared future just might have in store…ROBOTINITY Exhibition Formulation of Social Contextual Ars Electronica Futurelab Research & Innovation Group Robot Studio
  14. 14. myBOT workshop“myBOT” is an entertaining way to play with identities. The first step is to take portrait photos. Next, you use a simple program toprocess them into a short stop-motion animation sequence. Then you select just the right robot body from a gallery of options andspecify its behavior. The result is a customized myBOT that might be funny or serious, offbeat or nerdy. “myBOT” enables you todesign a robot in a way that’s fun and easy.MyBOT Make your bot! Ars Electronica Futurelab Research & Innovation Group
  15. 15. Outline of this talk (St)age of Futurelab Gulliver Fab Lab Partici- World pation Ars Research Electronica Questions Introduction Case Study Future Work Back- Q&A ground Deep Geocity Intention Space CADETt0 Ars Electronica Futurelab Research & Innovation Group
  16. 16. BackgroundArs Electronica Timeline ART – TECHNOLOGY – SOCIETY Ars Electronica assembles and configures the commentary, concepts and visions of our modern, information-based society, whereby currently prevailing facts & circumstances are always taken as points of departure Festival Ars Electronica (since 1979) for speculation about where we’re headed. Festival for Art, Technology and Society In short: future manifestations that are presently in the process of Prix Ars Electronica (since 1987) emerging are what Ars Electronica is interested in. Attention is never international Competition for Cyberarts focused singularly on art, technology or society, but rather on their complex metamorphoses and interrelationships. Ars Electronica Center (since 1996) Museum of the Future Ars Electronica Futurelab Laboratory for Future Innovations New Ars Electronica Center 2004 2005 2007 2009 Extension to 6500m², Opening Jan. 2009 1979 1987 1996 Ars Electronica Gallery in Vienna (Vienna Museumsquarter) Ars Electronica international Ludwig Boltzmann Research Institute for Media Art Research and Digital Archives Ars Electronica Futurelab Research & Innovation Group
  17. 17. Center Ars Electronica is a company owned by Linz City and an internationally unique platform for digital art an media culture, consisting of Futurelab four division: Festival PrixThe annual budget of Ars Electronica: Is this an institute of Linz City?balance sheet total 2009: 14,4 Mio € Ars Electronica Linz GmbH (=limited company) is 100% owned by the City ofFunding : 60 % subsidies Linz 22 % own income 18 % 3rd party projects Is AEC managed by tax? Founding consists of subsidies (mainly City of Linz and Region UpperNumber of employee: Austria), own income (entrance fee museum, festival tickets) and 3rd party76 full time employee projects developed by Ars Electronica Futurelab96 part time employeesum: 116 full time employee equivalent Is this an independent organization? Ars Electronica is owned by the city but independent within it’s missionPartner and past examples as a collaborative research statementSiemens CT, SAP AG , Vodafone Group R&D, KANSAITV, HONDA R&D, HAKUHODO, … Ars Electronica Futurelab Research & Innovation Group
  18. 18. Futurelab: A Visual Image for a Research System Business FL Production Team (Interface and Interaction Design, Information Design, Advanced Sensing Technologies, Network & Database Technologies etc…) Experience Design Media Architecture Media PerformanceProjects FL Research & Innovation GroupInnovative ResearchBasic Research FL Fundamental Research Futurelab is one department in Ars Electronica, for conducting research and development. In this collaborative project, FL Research & Innovation Group will be conducting the research, with working FL production team. Ars Electronica Futurelab Research & Innovation Group
  19. 19. Research Topics of Research & Innovation Group Persuasive Technology Interaction Creative Functional Ecology Catalyst Aesthetic ROBOTINITYThe Research & Innovation Group focuses on key issues for socialization of media and technologies.Right now, we have five research topics and all topics formulate international research hubs in each topic.These five topics are flexibly connecting and generating fusion models to coin new concepts, through actual working models. Ars Electronica Futurelab Research & Innovation Group
  20. 20. Interaction Ecology“How to Design “Interaction” in Embodied Media Environment”OverviewThis research topic aims for a holistic and analytic approach to Interaction Design.Traditional User Centered Design commonly focuses on single user interactions between the user and an interface. To dealwith today’s increasingly complex systems, where centralized control over information on a variety of arbitrary interfaces invarying detail is essential, this is not enough.It is inevitable to find new cognitive models and metaphors to get along in these ecologies. They constitute themselves as afunctional set of artifacts, people and the surrounding environment and the rich interaction in between. We identify these as"Interaction Ecologies".Research Records and PartnersVodafone, Siemens, SAP, Microsoft, HP, BMW, CADET (Austrian Governmental Fund) Ars Electronica Futurelab Research & Innovation Group
  21. 21. Functional Aesthetic“How to Design New Communication Integrating Functions and New Materials"OverviewThis research topic focuses on the new possibilities of functions produced by new materials. Now, media technologies areintergrated to our daily lives and it needs natural and organic user interfaces.In this research topic, we search for new forms of organic communication generated by new materials.Research Records and PartnersElekit, Johannes Kepler University Linz SWITCH Ars Electronica Futurelab Research & Innovation Group
  22. 22. Robotinity“How to Coexist with Robots in the Future”OverviewRobotinity is a study how people coexist with robot. Depending on cultural backgrounds, the meaning of robot and technologyis totally different. Whats the robot? What is our humanity? Both has to be considered when we design a robot.In this research topic, we aim to search for factors how people coexist with robot.Research Records and PartnersHonda R&D, Murata Manufacturing Company, ATR, Osaka University Meet ASIMO in Deep Space Ars Electronica Futurelab Research & Innovation Group
  23. 23. Creative Catalyst“How to Create a Medium and Place for Prompting Creativities”OverviewPeople are creative. However, many things are hidden in the world.How media technologies can prompt new creativities for people? In this research topic, we research methodologies and toolsto enhance creativities.Research Records and PartnersHakuhodo, Kansai TV, Smart Technologies, STUDIOLAB (EU-FP7) Shadowgram Ars Electronica Futurelab Research & Innovation Group
  24. 24. Persuasive Technology“How to Coexist with Technologies for Sustainable Society and CultureOverviewIt is dedicated to the application-oriented research and development of persuasive technologies and Ambient Devices, aimedat raising awareness of environmentally sustainable practices at individual and societal level. Persuasion in the psychologicalprocesses are studied believe that lead the addressee to setting changes and resulted in tangible changes in behavior. Withexpertise regarding this and may also need social change processes in a positive direction can be controlled.In this research, we search for social responsibilities how new media is developed for keeping sustainable society.analysis and personalized system adaptations.Research Records and PartnersJohannes Kepler University Linz Ars Electronica Futurelab Research & Innovation Group
  25. 25. Outline of this talk (St)age of Futurelab Gulliver Fab Lab Partici- World pation Ars Research Electronica Questions Introduction Case Study Future Work Back- Q&A ground Deep Geocity Intention Space CADET Ars Electronica Futurelab Research & Innovation Group
  26. 26. Case Study: Gullivers World Hidden Gullivers Worlds Box Gullivers World Geocity Fab Lab Deep Space
  27. 27. Case Study: Gullivers World Research Topics • Digital content creation • Scene creation and non-linear story telling • Playful and metaphoric Interaction Design • Interface and tracking technologies • Virtual / mixed reality installations • Collaborative Environment • Exhibition Design
  28. 28. Case Study: Gullivers World Constraints and Demands • Visitors time: 10 to 20 minutes • Workshops with 60+ minutes • Single user as well as • Groups of up to 15+ • 70.000+ visitors / year • Visitors age: 3 to 90+ • Visitors knowledge: novice to expert • Integrate topics of broader artistic, social and/or technological concern
  29. 29. Case Study: Gullivers WorldHidden Worlds of Soundand Voice (2002)An interactive audiovisual installation whose centraltheme is the magical relationship of speech to the etherealmedium which conveys it. Participants in this exhibit areable to "see" each others voices, made visible in the formof animated graphic figurations that appear to emergefrom the participants mouths.Golan Levin, Zachary Lieberman and Ars ElectronicaFuturelab
  30. 30. Case Study: Gullivers WorldGulliver’s Box (2003)This installation represents the effort to pursue newapproaches how to deal with Mixed Reality content. Thechallenge at the core of this project was to position aninnovative medium somewhere between theater, film andinstallation. The result is an infrastructure that offersartists new opportunities to convey audiovisualinformation, and one that ought to encourage creatives inevery discipline to work with these new approaches.Adrian Cheok, Hirokazu Kato and Ars Electronica Futurelab
  31. 31. Case Study: Gullivers World Gulliver’s Box: Interaction Design
  32. 32. Case Study: Gullivers World Gulliver’s Box: visitor generated content
  33. 33. Case Study: Gullivers WorldGulliver’s World (2004)Motivated by the success of Gulliver’s Box, Futurelabtook another long look at the concept and expanded itin several directions. Important new features designedinto Gulliver’s World are that users are no longerlimited to preset environments and characters;instead, they are called upon to design the artificialworld and its components themselves. This wasaccomplished by the development of intuitive editorswith which the environment can be totally revised andcustomized anew each time.Hirokazu Kato and Ars Electronica Futurelab
  34. 34. Case Study: Gullivers World Gulliver’s World: Character Design
  35. 35. Case Study: Gullivers World Gulliver’s World: Interface Design
  36. 36. Case Study: Gullivers World Gulliver’s World: Introduction Video
  37. 37. Case Study: Gullivers World Data Distribution and Synchronisation Gulliver’s World: complex interplay of different installations
  38. 38. Case Study: Gullivers World Gulliver’s World: “Beholder” as central world simulator
  39. 39. Case Study: Gullivers World Gulliver’s World: Clay workshop and 3d Scanner
  40. 40. Case Study: Gullivers World Way from the physical into the virtual world: 12 reference pictures
  41. 41. Case Study: Gullivers World Way from the physical into the virtual world: separation and calibration
  42. 42. Case Study: Gullivers World Way from the physical into the virtual world: Visual Hull reconstruction
  43. 43. Case Study: Gullivers World Way from the physical into the virtual world: Texture recreation
  44. 44. Case Study: Gullivers World Gulliver’s World: Rotation Editor
  45. 45. Case Study: Gullivers World Gulliver’s World: Rotation Editor
  46. 46. Case Study: Gullivers World Gulliver’s World: Object carriers as physical representation for digital content
  47. 47. Case Study: Gullivers World Gulliver’s World: Paint terrains and stamp objects
  48. 48. Case Study: Gullivers World Gulliver’s World: Jigsaw puzzle metaphor for the last missing piece of a world
  49. 49. Case Study: Gullivers World Gulliver’s World: Greenbox for video recordings of visitors
  50. 50. Case Study: Gullivers World Gulliver’s World: A transparent box to carry the recorded data inside
  51. 51. Case Study: Gullivers World Gulliver’s World: Character Editor
  52. 52. Case Study: Gullivers World Gulliver’s World: Character modification influences abilities, behavior and storyline
  53. 53. Case Study: Gullivers World Gulliver’s World: The World stage is the central element of the installation
  54. 54. Case Study: Gullivers World Gulliver’s World: Collaborative virtual environment
  55. 55. Case Study: Gullivers World Gulliver’s World: Motion analysis influences characters
  56. 56. Case Study: Gullivers World Gulliver’s World: Stereoscopic Tracking of Boxes
  57. 57. Case Study: Gullivers World Gulliver’s World: Infrared reflective markers
  58. 58. Case Study: Gullivers World Gulliver’s World: Exploration through virtual expedition
  59. 59. Case Study: Gullivers World Gulliver’s World: The shelf reveals the content of the box
  60. 60. Case Study: Gullivers World Gulliver’s World: Pictures from the virtual world can be sent home
  61. 61. Case Study: Gullivers WorldCity Puzzle (2006)An interactive simulation environment visualizestechnological approaches that city planners andarchitects of the future will be working with. Thisinstallation is an expanded spin-off of “Gulliver’sWorld,” the Ars Electronica Center’s mixed realityenvironment, and uses a simple urban planningmodel as an example illustrating concreteapplication possibilities. Visitors can manipulatethe various scenarios with haptic tools.Ars Electronica Futurelab
  62. 62. Case Study: Gullivers World City Puzzle: Characters become buildings
  63. 63. Case Study: Gullivers World City Puzzle: By placing the characters the visitor starts to build a growing city
  64. 64. Case Study: Gullivers World City Puzzle: The city comes to life
  65. 65. Case Study: Gullivers World Gulliver’s World: Additional and improved haptic interfaces
  66. 66. Case Study: Gullivers World City Puzzle: New interfaces allow the exploration of the virtual world
  67. 67. Hidden Gullivers Worlds Box Gullivers WorldGeocity FabLab Deep Space Ars Electronica Futurelab Research & Innovation Group
  68. 68. Imagine a future in which you no longer buy gym shoes in a store; instead, you print them out at home, use them, and then toss theminto the recycling bin when you’re done. What now seems like a weird futuristic vision could soon be reality. The idea behind FabLab(from fabrication) is the capability of downloading objects from the internet and printing them out at home. And not just on paper; asreal stuff! A 3D printer uses a computer model to generate an actual object, and a computer-controlled laser cutter does the high-precision shaping of any chosen material. Using the internet makes it possible to disseminate designs for articles of clothing orpieces of furniture just as easily as sharing music and pictures today. What is indeed unimaginable is this technology’s impact onestablished industries.FABLAB AEC Do it your self in the future Ars Electronica Futurelab Research & Innovation Group
  69. 69. Ars Electronica Futurelab Research & Innovation Group
  70. 70. The Gallery as central element Ars Electronica Futurelab Research & Innovation Group
  71. 71. Making digital objects physical Ars Electronica Futurelab Research & Innovation Group
  72. 72. Making physical objects digital Ars Electronica Futurelab Research & Innovation Group
  73. 73. Create Digital Content Ars Electronica Futurelab Research & Innovation Group
  74. 74. Workshop and Lectures with Artists Ars Electronica Futurelab Research & Innovation Group
  75. 75. Print your personal iPhone cover Ars Electronica Futurelab Research & Innovation Group
  76. 76. Using LASER cut silhouettes … Ars Electronica Futurelab Research & Innovation Group
  77. 77. … to create little phantasy worlds Ars Electronica Futurelab Research & Innovation Group
  78. 78. Create your plushy toys Ars Electronica Futurelab Research & Innovation Group
  79. 79. Hidden Gullivers Worlds Box Gullivers WorldFabLab Geocity Deep Space Ars Electronica Futurelab Research & Innovation Group
  80. 80. It’s estimated that planet Earth will have nearly 10 billion inhabitants in 2050, more than two-thirds of them living in cities. All over the world, metropolitan areas are exploding into newmegacities. Homo sapiens is evolving into homo urbanicus, a being whose hopes—and, all toooften, disappointments—are connected more closely than ever before to the course of “life in thebig city” … “GeoCity” examines global developments and their local impactGEO CITY Discursive and participatory urban Information Environment Ars Electronica Futurelab Research & Innovation Group
  81. 81. Linz, Upper Austria Ars Electronica Futurelab Research & Innovation Group
  82. 82. Three important Locations Ars Electronica Center New City Hall Old City Hall Ars Electronica Futurelab Research & Innovation Group
  83. 83. New City Hall Ars Electronica Futurelab Research & Innovation Group
  84. 84. 1:2000 scale model Ars Electronica Futurelab Research & Innovation Group
  85. 85. Old City Hall Ars Electronica Futurelab Research & Innovation Group
  86. 86. Changing Linz Ars Electronica Futurelab Research & Innovation Group
  87. 87. Interactive surface, graspable physical objects Ars Electronica Futurelab Research & Innovation Group
  88. 88. Designing the environment Ars Electronica Futurelab Research & Innovation Group
  89. 89. Ars Electronica Futurelab Research & Innovation Group
  90. 90. SimLinz Ars Electronica Futurelab Research & Innovation Group
  91. 91. Panorama Projection Ars Electronica Futurelab Research & Innovation Group
  92. 92. Digital Booklet & PenBooklet & pen Ars Electronica Futurelab Research & Innovation Group
  93. 93. Anoto Technology Ars Electronica Futurelab Research & Innovation Group
  94. 94. Information map Ars Electronica Futurelab Research & Innovation Group
  95. 95. Data Selection Ars Electronica Futurelab Research & Innovation Group
  96. 96. Interactive Table Surface Ars Electronica Futurelab Research & Innovation Group
  97. 97. Discursive Environment Ars Electronica Futurelab Research & Innovation Group
  98. 98. Ars Electronica Futurelab Research & Innovation Group
  99. 99. Ongoing DevelopmentSimLinz mobile, 2010 Ars Electronica Futurelab Research & Innovation Group
  100. 100. Urfahraner Markt Ars Electronica Futurelab Research & Innovation Group
  101. 101. Urfahraner Markt Ars Electronica Futurelab Research & Innovation Group
  102. 102. Urfahraner Markt Ars Electronica Futurelab Research & Innovation Group
  103. 103. Urfahraner Markt Ars Electronica Futurelab Research & Innovation Group
  104. 104. Life Signs: While your heart beats Ars Electronica Futurelab Research & Innovation Group
  105. 105. Sim Linz Ars Electronica Futurelab Research & Innovation Group
  106. 106. We are Linz Ars Electronica Futurelab Research & Innovation Group
  107. 107. Shadowgram Ars Electronica Futurelab Research & Innovation Group
  108. 108. Wall topography for Linz Ars Electronica Futurelab Research & Innovation Group
  109. 109. Hidden Gullivers Worlds Box Gullivers WorldGeocity Deep Space FabLab Ars Electronica Futurelab Research & Innovation Group
  110. 110. CAVE technology was yesterday. Deep Space is right now. Experience a newdimension of travel through space and time. Immerse yourself into breathtaking 3Dstereo universes featuring jumbo-format, high-definition images that willabsolutely blow you away!DEEP SPACE Huge Immersive Environment Ars Electronica Futurelab Research & Innovation Group
  111. 111. Ars Electronica Futurelab Research & Innovation Group
  112. 112. Playing Games Ars Electronica Futurelab Research & Innovation Group
  113. 113. Exploring the Universe Ars Electronica Futurelab Research & Innovation Group
  114. 114. Concerts, 3d visualized Ars Electronica Futurelab Research & Innovation Group
  115. 115. Ars Electronica Futurelab Research & Innovation Group
  116. 116. Ars Electronica Futurelab Research & Innovation Group
  117. 117. Give Lectures Ars Electronica Futurelab Research & Innovation Group
  118. 118. Conduct Research (August 31, 2010) Honda’s humanoid robot ASIMO will make its public debut in Austria at this years Ars Electronica Festival. September 2-8 in Linz, festivalgoers will have the opportunity to experience an impressive demonstration of ASIMO’s extraordinary capabilities at Deep Space in the Ars Electronica Center. Research & Innovation Group collaborates with Honda R&D for the future vision of humanoid robot. ASIMO Project Social Experiment in Deep Space Ars Electronica Futurelab Research & Innovation Group
  119. 119. Ars Electronica Futurelab Research & Innovation Group
  120. 120. Game based Interaction Experiments Ars Electronica Futurelab Research & Innovation Group
  121. 121. Outline of this talk (St)age of Futurelab Gulliver Fab Lab Partici- World pation Ars Research Electronica Questions Introduction Case Study Future Work Back- Q&A ground Deep Geocity Intention Space CADET Ars Electronica Futurelab Research & Innovation Group
  122. 122. Multiple Participants interacting with Asimo simultaniously Ars Electronica Futurelab Research & Innovation Group
  123. 123. Triggering “Soundballs” by stepping into Visualization Ars Electronica Futurelab Research & Innovation Group
  124. 124. Tracking systems upgrade Ars Electronica Futurelab Research & Innovation Group
  125. 125. Multicamera System Ars Electronica Futurelab Research & Innovation Group
  126. 126. Stereoscopic Depth Estimation Ars Electronica Futurelab Research & Innovation Group
  127. 127. Stereoscopic Depth Estimation Ars Electronica Futurelab Research & Innovation Group
  128. 128. Playful research that doesn’t just aim to entertain is what the Center for Advances in DigitalEntertainment Technologies is all about. The mission: take the latest breakthroughs in full-body & gesture tracking, bio-signal sensor systems and 3D technology and utilize them inprojects beyond the realm of the computer game sector.Ars Electronica Futurelab and the Salzburg University of Applied Sciences’ Department ofMultimedia Technology and Multimedia Art have launched this joint research effort, funded bythe Austrian FFG.CADET Center for Advances in Digital Entertainment Technologies Ars Electronica Futurelab Research & Innovation Group
  129. 129. The Ars Electronica Futurelab is the only non-university R&D facility to be granted fundingthis year by the FWF–Austrian Science Fund’s PEEK arts development program. Beginning in2011, the Futurelab will collaborate with renowned media artist, choreographer and composerKlaus Obermaier (AT) on a three-year project that aims to develop interactive dramaticperformances.PEEK (St)age of Participation Ars Electronica Futurelab Research & Innovation Group
  130. 130. Questions Ars Electronica Futurelab Research & Innovation Group
  131. 131. Contact AddressRoland HaringArs-Electronica-Straße 14040 Linz, Austriarolandha@aec.atresearch.aec.at Ars Electronica Futurelab Research & Innovation Group

×