Slide Pieter Jonker_Yolande Kolstee


Published on

Intervento tenuto dal Pieter Jonker e da Yolande Kolstee in occasione del Forum Multimedia per la Cultura, tenutosi a Vigevano l'11 e 12 giugno 2010

Published in: Technology, Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Slide Pieter Jonker_Yolande Kolstee

  1. 1. Augmented Reality Pieter Jonker Professor of Vision based Robotics Dept Bio-Mechanical Engineering, Faculty 3ME Delft University of Technology, Delft The Netherlands Yolande Kolstee Lector of Innovative Visualisation Techniques for Art & Design Royal Academy of Art, The Hague The Netherlands 01/07/2010 15:36 1
  2. 2. Augmented Reality vs Virtual Reality Het volgen van een enkele hypothese (toegepast in AR) 01/07/2010 15:36 2
  3. 3. Sorts of Augmented Reality Screen Based Head-Set Based 01/07/2010 15:36 3
  4. 4. Sorts of Augmented Reality Video see through AR Optical see through AR 2 cameras + 2 displays + 1 camera + 2 half translucent video in video mirrors + 2 displays 01/07/2010 15:36 4
  5. 5. History of our headsets 01/07/2010 15:36 5
  6. 6. The Measurement Problem How to hold the virtual objects in a perfect position and orientation in the physical world if I move my head? By searching and tracking (camera + image processing) one or more known landmarks! 01/07/2010 15:36 6
  7. 7. How to anchor onto a marker? Select closed Simple corner Grab an image Edge detection contours detection Intersect 4 lines Sub-pixel fit of Split contour Keep contours for corners the 4 lines into 4 lines with 4 corners Calculate pose of Determine the Calculate pose marker ID of camera 1 2 4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384 01/07/2010 15:36 7
  8. 8. Why not tracking at natural features? This costs too much time fort now, but soon… 01/07/2010 15:36 8
  9. 9. Is only a camera & image processing enough? No, we must know where the head will be in 30 msecs because rendering a virtual image costs time. We have to predict where the head will be bij measuring the motion of the head. That is why we have to fuse the data of the camera and image processing with the date from a so called inertia tracker to track fast head movements. This is comparable with the yes and the inner ear / vestibular organ of a human. An inertia tracker has 3 accelerometers to measure linear acceleration in 3 dimensions (x, y, z) and 3 gyroscopes to measure 3 rotational velocities (roll, pitch, yaw). This gives relative displacements at 100Hz that are combined with the 25Hz absolute position and orientation of the camera. 01/07/2010 15:36 9
  10. 10. Fusion of camera and inertia tracker data Inertia measurement 100 Hz Camera measurement Image acquisition Image processing 15 Hz Data Fusion Rendering Image 60 Hz time 01/07/2010 15:36 10
  11. 11. So how accurate can one place virtual objects in the physical world? Statical misalignment < 0.6 ° (< 0.03° = eye) Dynamical Misalignment < 2.3° (< 0.5° = eye) With a 20 x 20 cm marker at 5 m and a 1MPix camera Own and object motion will camouflage this 01/07/2010 15:36 11
  12. 12. Future elektronics to loose the backpack and headset 01/07/2010 15:36 12
  13. 13. Conclusions so far • Augmented Reality is not Virtual Reality • Augmented Reality is screen or headset based • Headset based AR is video or optical see-through • The AR problem is dominated by the the allignment problem of virtual and real world objects • This is solved by inertia and camera tracking • Due to the imperfect technology one can still see a bit alignment errors, but far better than GPS! • Happily the human visual system compensates a bit • Future hardware will make it as small as iPods. • AR is a new and complex medium to discover! 01/07/2010 15:36 13
  14. 14. The Vision of Yolande Kolstee Visualization in Art and Design 01/07/2010 15:36 14
  15. 15. Undergraduate / Bachelors Design: Textile, Fashion, Interior, Furniture, Graphic, Interactive Media Fine Arts & ArtScience Royal Academy of Photography Art (KABK) Graduate / Masters The Hague Type and Media The Netherlands ArtScience Interior Architecture Research in the Arts Industrial Design 01/07/2010 15:36 15
  16. 16. Undergraduate, Graduate & Doctorate Faculties: Aerospace Engineering Delft Applied Sciences University of Architecture Technology Civil Engineering and Geosciences Delft Electrical Engineering, Mathematics & Computer Sc. The Industrial Design Engineering Netherlands Mechanical, Maritime & Materials Engineering Technology, Policy and Management 01/07/2010 15:36 16
  17. 17. The fusion of perception of real and virtual images The AR Lab is for all students, also outside students, who need help in the AR domain. Institutes of cultural heritage come to us, like Museums: •Van Gogh •Boijmans van Beuningen •Stedelijk Museum •Kröller-Müller •Escher Manifestations of: Multi-media Modern-Art 01/07/2010 15:36 ICT 17
  18. 18. For example we used AR to show animated chairs and tables in the 2008 exhibition Salone del Mobile in Milan, Italy 01/07/2010 15:36 18
  19. 19. AR was used to demonstrate the construction process 01/07/2010 15:36 19
  20. 20. AR can be used to demonstrate the possibilities of the product Melanie Luchtenveld 01/07/2010 15:36 20
  21. 21. The virtual objects can be studied from all viewpoints 01/07/2010 15:36 21
  22. 22. Transportation and duplication of furniture is quite easy 01/07/2010 15:36 22
  23. 23. Connecting with the virtual world, experiments with interfaces Audience likes AR & Interaction Wii remote controller RFID Data glove: 5 flexor strips in fingers Markers 'stand-alone' or incorporated Fragments of pottery 01/07/2010 15:36 23
  24. 24. We use RFID technology to let (tagged) physical objects influence virtual objects 01/07/2010 15:36 24
  25. 25. We use a data-glove to manipulate virtual manipulated objects. Here to realize a virtual hand- puppet 01/07/2010 15:36 25
  26. 26. One can interact with virtual objects in several ways One can us real physical objects to change behavior of virtual puppets e.g. Bath-sponge towel and foam Pot-holder cook Fay-wand blow fish is crowned Cigarettes blow fish blows 01/07/2010 15:36 26
  27. 27. Interaction can also be performed by moving the markers 01/07/2010 15:36 27
  28. 28. Interaction with a WII mote Escher in the Palace: Wii remote controller: throw a visual to the wall 01/07/2010 15:36 28
  29. 29. Headset Design Students of the Royal Academy designed a new AR headset based on the VR/AR headset of the Dutch SME Cybermind who made a working version of the student’s mock-up 01/07/2010 15:36 29
  30. 30. Design requirements: More ease to wear Don't isolate the user Don't block the reality Contact with the environment Unique experience / attractive 01/07/2010 15:36 30
  31. 31. Headset 'George' camera at the right side inertia tracker left open view below wide screen Side view 01/07/2010 15:36 31
  32. 32. Various other designs have been made, but not yet prototyped Design Niels Mulder 01/07/2010 15:36 32
  33. 33. What to display? Man made: There is a variety of 3D design programs and skilled and talented people who master this: Fine Art Artists, Animators, 3D Designers, Architects Computer Generated: 3D imagery (industrial) machines especially from the bio-medical research CT scan generated 3D images, yield an inexhaustible amount of CT images Data visualization: There are various huge databases, e.g. geo 01/07/2010 15:36 information, annual reports 33
  34. 34. Art: Drops of White Marina de Haas 01/07/2010 15:36 34
  35. 35. Geo Info Data Visualization of Acid level in the first AR MAP on behalf of GSDI 11 World Conference, Rotterdam June 16-20 2009 Global Spatial Data Infrastructure association 01/07/2010 15:36 35
  36. 36. AR Art: first experiment for general public Marina de Haas 01/07/2010 15:36 36
  37. 37. Cultural Heritage: Displaying computer generated data: Using bio-medical imaging equipment: CT scans for visualizing sgraffito-pottery: Pre industrial original (1250-1450) Post industrial means Scanning- Digitalizing 3D printing physical copy 3D Archiving virtual copy 01/07/2010 15:36 37
  38. 38. 01/07/2010 15:36 38
  39. 39. Lessons learned Let the public view on a beamer what is seen by the HMD The “empty” third dimension is very useful for information display and interaction and detaches the application from gravity. One can have art or any info in the air ( e.g. drops of white) Motion of the virtual objects covers up for misalignment and jitter Design packages such as Cinema 4D make design with animated figures possible. For real 3D animated films with large plots, we are busy to incorporate game engines like Unity 01/07/2010 15:36 39
  40. 40. Headset based AR is at its best when a full immersive experience is required and people can walk around or within larger objects; buildings, DNA, airplanes to be refurbished For outdoor AR it is necessary that the ambient light intensity and the intensity of the LCD displays on the headset is in balance (automatic sunglasses) Augmented Reality can be fruitfully used to attract a broad public to displays of cultural heritage. Its narrative power is huge. The collaboration between researcher in the area of image processing with artists, designers and curators appeared to be very fruitful and has lead to many amazing productions and exhibitions 01/07/2010 15:36 40
  41. 41. ‘Why more realities?’ the old man in Milan said: ‘Live is already so complicated’. “It begins with a blessing and it ends with a curse; making life easy, by making it worse.” 01/07/2010 15:36 41