IBM iForum - Augmented Reality Vtt by Charles Woodward

Slide 1: Augmented Reality Prof. Charles Woodward VTT Technical Research Centre of Finland

Slide 2: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Outline  Introduction  VTT  Virtual and Augmented Reality  Applications at VTT  Entertainment, marketing  Interior design, architecture  Mobile Augmented Reality  Production applications  Augmented Reality Future  Some new ideas  Enabling technologies  Trends and Drivers

Slide 3: VTT TECHNICAL RESEARCH CENTRE OF FINLAND INTRODUCTION

Slide 4: VTT TECHNICAL RESEARCH CENTRE OF FINLAND VTT Profile  Figures IES  2 700 employees Basic SIT  233 M€ research budget research R IVE  45 Knowledge Centres UN  7 Knowledge Clusters Applied VTT  Digital Information Systems research  Telecommunications IN  Microtechnologies and Sensors D US  Materials and Building TR  Industrial Systems Development Y  Biotechnology work  Energy and Pulp&Paper

Slide 5: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Virtual and Augmented Reality at VTT  VR/AR Roadmap report 2005  Fourteen VTT research teams contributing  Applications from entertainment to digital media, construction to production, ships, military, etc.  Technology, incl.  Core VR/AR solutions  Machine vision, feature tracking  Multimodality, 3D audio etc.  Motion tracking, avatars, sensors  GSM/GPS positioning, GIS systems  Ambient intelligence, context sensitivity, …  Special strenghts  Multi-disciplinary research  Strong industry relations and experience

Slide 6: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Virtual Reality (Introduction)  Devices, incl.  Head Mounted Displays (HMD), data gloves, motion trackers, ...  CAVE systems  “Lumeportti” - VTT’s proprietary CAVE for 3D data visualisation  Utilises video projectors, networked PCs and low-cost VR technology  Provides steroscopic walk-through functionality with gesture-based interaction  Applications e.g. space ship simulation for Alenia Spazio

Slide 7: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Augmented Reality  Augmented Reality (AR) - Mixes virtual objects with user’s view of real world - Properties: real-time, Reality Augmented Reality Augmented Virtuality Virtual Reality interactive, 3D Mixed Reality  Related technologies Augmented Virtuality - - Mediated Reality - Diminished Reality - Mixed Reality  Display devices - PC screen + web camera (”Magic mirror”) - Data glasses (video or optical see through HMD) - Hand held devices, camera phones (”Magic lense”)

Slide 8: VTT TECHNICAL RESEARCH CENTRE OF FINLAND AUGMENTED REALITY – APPLICATIONS AT VTT

Slide 9: VTT TECHNICAL RESEARCH CENTRE OF FINLAND AR in Marketing / Media  ARCatalogue  Enhanced ”Magic Book” application  In real marketing use by e.g. Lappset Group  ARCipher  Implementation on camera phones  Augmenting on matrix codes

Slide 10: VTT TECHNICAL RESEARCH CENTRE OF FINLAND AR in Games / Entertainment  CamBall  Augmented networked table tennis  SymBall  Virtual table tennis for camera phones  StereoGames  3D Stereo display enhancement

Slide 11: VTT TECHNICAL RESEARCH CENTRE OF FINLAND AR at Homes / Interiors  Basic solution  Virtual furniture object tied to each marker  ARInteriors  Augmented digital images  ARPhone  Virtual furniture on phone’s camera image  Downloading models by matrix codes

Slide 12: VTT TECHNICAL RESEARCH CENTRE OF FINLAND AR on the Table / Collaboration  ARScaleModel  Virtual scale model augmented on table  Stereo viewing using data glasses  Virtual cursor for user interaction  Including  ARCatalogue for selecting components  Virtual reality walk-through inside the model

Slide 13: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Mobile Augmented Reality  Outdoors visualisation of Digitalo  Commission by Senate Properties, spring 2003  Client/server implementation with PDA & WLAN  Using large 80 x 80 cm markers, Speed 2 frames / sec  Markerless implementation “AROnSite”  Feature based tracking  Interactive positioning of building to real scene  E.g. Helsinki Music Hall visualisation 2006

Slide 14: VTT TECHNICAL RESEARCH CENTRE OF FINLAND “Google Earth on Earth”  Google Earth + AROnSite  Building placed in Google Earth  Semi-automatic positioning using GPS  Applications, e.g.  Paper mill visualisations by Pöyry Plc 2007  Excavator visualisations for Metso Minerals 2007  Future work  Automatic positioning, e.g. matching map and cam view  Implementation on camera phones

Slide 15: VTT TECHNICAL RESEARCH CENTRE OF FINLAND AR in the Web  ARWebCam  Augmented reality webcamera  Application during Digitalo construction 2003 – 2005  VTT being the first in the world to implement  3D Terrain  Semiautomatic creation of 3D environment  Based on aerial photos and geographic information  Incl. trees, surrounding buildings, terrain shapes

Slide 16: VTT TECHNICAL RESEARCH CENTRE OF FINLAND PRODUCTION APPLICATIONS

Slide 17: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Industry Partners, e.g.

Slide 18: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Project SPECIAL  Further applications by Pöyry Plc  ARCatalogue for technical machinery documents  Helmet with integrated camera & data glasses  Emagin video glasses / Micro-optical PC viewer  Lightweight, battery mounted inside helmet

Slide 19: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Project PLAMOS  Objective  To provide ubiquitous access to plant information using intuitive mobile user interfaces / Augmented Reality  Results  Core-task analyses for operational and maintenance work to consider and define interfaces between actions and actors including outsourcing of services  An ontology based modeling environment for linking plant model information from several sources e.g. for augmented visualisation of 3D constructions  AR demonstrations on gasification simulation process; the user can e.g. change controls and study changes in process values, and retrienve service instructions of the identified equipment  Employing AR in the design phase to support information transfer between design, installation, implementation and maintenance

Slide 20: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Project NOSE  Objective: to study studies methods and tools for ubiquitous plant model usage  Continuation to PLAMOS, but stronger emphasis on locationing techniques & pilot applications  Location aware services and augmented reality techniques studied in process industry surroundings in order to provide plant model services to a mobile device  Technologies: model based tracking, pseudosatellites  Participating companies  Fortum Nuclear Services, Pöyry Forest Industry, Metso Paper, Wärtsilä, Nokia Research Center, Cadmatic, Space Systems Finland, R.A. TR-Consulting  Timetable  26.9.2007-31.7.2009, funding from Tekes and companies

Slide 21: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Project AugAsse  Augmented Assembly  Timetable: 1.8.2006-31.7.2008  Funding from Tekes and companies  Objective: Increasing efficiency in assembly work with Augmented Reality  Development of the manufacturing process using AR  Integration of AR to CAD/PLM/ERP systems  Content creation (authoring) for Augmented Assembly  Exploring AR display devices in assembly factory  Work tool sensor feedback into AR systems  Development of a AugAsse demonstration system  Development of pilot applications at customer companies  Advantages  Visual and smart information for manual working phases  Faster work and less faults  Uniform quality  Steep learning curve  Customer-driven design change management

Slide 22: VTT TECHNICAL RESEARCH CENTRE OF FINLAND SOME NEW PROJECT IDEAS

Slide 23: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Project Proposal ”TelePet” • Augmented Networked Games for Dogs • To entertain dogs over Internet during master’s absence • Game control based on gesture detection • Example game • Master throws virtual ball, dog has to “catch” • Dog sees silhouette on the screen, ball bounces back if “caught” • Further rewards, e.g. biscuit box operated over internet Internet

Slide 24: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Project Proposal ”AR4BC” • “Augmented Reality for Building and Construction” • To provide mobile AR (MR) tools linking real situation at construction site to Building Information Models (BIM) information, planning and follow-up • Augmented on-site visualisation • linking project plans with actual situation on site • Augmented site journaling • feedback to system: attach images, tag installed parts in 4D model • Office solutions • presentation techniques for augmented 4D models

Slide 25: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Project Proposal “MARACAT”  “Mobile AR for Architects, Communities, Ads and Things”  Technology – core solutions  ARPhone – products, things downloaded by matrix codes, augmented in camera phone view  ARMobile/AROnSite – products, things positioned in Google Earth maps, augmented in camera phone view  ARCipher – 3D content augmented to ads, products etc., based on visual tracking of matrix codes  Applications (“Things”)  Consumer products, furniture, decoration  Construction, houses, appartments  Augmented ads, packages, products  Entertainment, infotainment, games  Fashion, trends, digital arts  Personal digital items  Virtual brand products  Sharing by communities

Slide 26: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Connecting Physical and Digital Worlds • Mobile Augmented Reality as visual interface to location based (3D) Internet content  Internet content augmented in real world • “Google Earth on Earth” for displaying GE content on spot • E.g. virtual ads, tourist info, navigation, architecture  Mobile Internet content creation • Augmented and other media on camera phones, e.g. photos, video clips • Attaching the user generated (augmented) content to real world locations  Interfaces to virtual products • Digital products by virtual communities, with real monetary value • Current examples Habbo Hotel, IRC Gallery, Aapeli, Second Life, … • MAR interface to hyperdragging  Interfaces to virtual worlds • E.g. virtual world avatars augmented in real world • And, real people, things augmented in virtual worlds

Slide 27: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Mobile Augmented Reality one of the technologies “most likely to alter industries, fields of research, and the way we live”. - MIT’s annual review “10 Emerging Technologies 2007”

Slide 28: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Technology Enablers and Drivers • Smart mobile terminal • Technology convergence to smart phones • Incl. camera, 3D graphics, WLAN, WiMAX, 4G • Up to accelerometers, RFID, 3D displays • E.g. N95, iPhone, Samsung • Minituriarised data glasses • E.g. Lumus augmented reality glasses • MyVu video glasses for iPod • Game industry, e.g. • Sony Eye Toy • Nintendo Wii • Eye of Judgment • AR software technology • E.g., Markerless tracking (see video) • Commercial activities, companies Intersense (USA), Metaio (Germany), Total Immersion (France)

Slide 29: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Mixed Reality and 3D Web • Second Life etc. • Virtual communities and commerce • Avatar representations & access from real world by Mixed Reality • Google Earth • Incl. 3D buildings, photos, panorama, etc. • Content creation by virtual communities, “digital graffiti” • Microsoft • PhotoSynth, Massively multi-user content creation • 3D reconstruction of physical world • Nokia • Purchase of Navteq at 5 Billion € • MARA project & VCUI team work at Palo Alto 2007 • IBM • International Herald Tribune (April 25, 2007): IBM creates a mainframe platform for Virtual Worlds with game developer Hoplon, using the high-powered Cell processor, capable of supporting 100.000’s of users in realistic 3D games ...

Slide 30: VTT TECHNICAL RESEARCH CENTRE OF FINLAND Thank You! Prof. Charles Woodward Videos and Demos available at: http://www.vtt.fi/multimedia