Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

You as a Puppet - UIST 2017

1,897 views

Published on

This slide was presented at UIST 2017 Physical interfaces session.
https://uist.acm.org/uist2017/

You as a Puppet: Evaluation of Telepresence User Interface for Puppetry
https://doi.org/10.1145/3126594.3126608

【Project page】
http://digitalnature.slis.tsukuba.ac.jp/2016/01/yadori/

【Project movie】
https://www.youtube.com/watch?v=MRy5pgXY8es
(ACE 2016 Creative Showcase Version)

【Presenter】
Mose Sakashita (坂下 申世)
University of Tsukuba, School of Informatics
College of Media Arts, Science and Technology
Digital Nature Group (Yoichi Ochiai)
http://www.u.tsukuba.ac.jp/~s1411453/index.html

【Abstract】
We propose an immersive telepresence system for puppetry that transmits a human performer's body and facial movements into a puppet with audiovisual feedback to the performer. The cameras carried in place of puppet's eyes stream live video to the HMD worn by the performer, so that performers can see the images from the puppet's eyes with their own eyes and have a visual understanding of the puppet's ambience. In conventional methods to manipulate a puppet (a hand-puppet, a string-puppet, and a rod-puppet), there is a need to practice manipulating puppets, and there is difficulty carrying out interactions with the audience. Moreover, puppeteers must be positioned exactly where the puppet is. The proposed system addresses these issues by enabling a human performer to manipulate the puppet remotely using his or her body and facial movements. We conducted several user studies with both beginners and professional puppeteers. The results show that, unlike the conventional method, the proposed system facilitates the manipulation of puppets especially for beginners. Moreover, this system allows performers to enjoy puppetry and fascinate audiences.

Published in: Engineering
  • DOWNLOAD FULL eBOOK INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookeBOOK Crime, eeBOOK Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

You as a Puppet - UIST 2017

  1. 1. 1 å You as a Puppet Evaluation of Telepresence User Interface for Puppetry Mose Sakashita Tatsuya Minagawa Amy Koike Ippei Suzuki Keisuke Kawahara Yoichi Ochiai
  2. 2. Introduction TV-shows Attractions 3 “Jim Henson's Legacy: A "Rainbow Connection" with UMD” Merrill College of Journalism Press Releases. (CC BY-NC 2.0) Communication Education Library teaches children the fun of reading | Stripes Guam http://guam.stripes.com/base-info/library-teaches-children-fun-reading
  3. 3. Introduction — Previous Puppetry Ways 4 Glove-puppet “German children performing a tale with selfmade puppets” Jürgen Heimann. (CC BY 3.0) “Marionette puppet show for kids in Asbury Park, NJ” Jackie, 2006. (CC BY 2.0) String-puppet
  4. 4. 5 Under the Desk… ¸¸¸¸¸¸¸¸¸¸¸
  5. 5. 6 Image by www.traveljunction.com Character Costume
  6. 6. 7 http://i843.photobucket.com/albums/zz352/loaloauk/Disney%20Planets/Mickey1.jpg Character Costume
  7. 7. 8 Performer’s location Physical limitations… Puppet’s size and shape “German children performing a tale with selfmade puppets” Jürgen Heimann. (CC BY 3.0) “Marionette puppet show for kids in Asbury Park, NJ” Jackie, 2006. (CC BY 2.0) Image by www.traveljunction.com
  8. 8. Related Work — Input System for Puppet 9 [Slyper et al. 2015] [Oshita et al. 2013][Slyper et al. 2011] Webcam Apple iSight Nicolas Esposito from France (CC BY 2.0) Web Camera Tongue Joystick Hand Gesture
  9. 9. Related Work — Computational Puppet 10 [Sekiguchi et al. 2001] [Bonanni et al. 2005] [Lee et al. 2008] PlayPals HuggableRobotPHONE
  10. 10. Introduction — Concept of Our System 11 Remote Place Objective Place Feedback Transmission
  11. 11. Implementation 13 Audiovisual Feedback Sensing & Transmission Performer to Puppet Puppet to Performer
  12. 12. Implementation — Sensing & Transmission 14 Head Pose
  13. 13. Implementation — Sensing & Transmission 15 Arms’ Movements
  14. 14. Implementation — Sensing & Transmission 16 Photoreflectors
  15. 15. Implementation — Sensing & Transmission 17 Mouth State
  16. 16. Implementation — Audiovisual Feedback 18 “Bones” of the Puppet Camera Microphone
  17. 17. Implementation — “Bones” of the Puppet 19
  18. 18. 20 How does the telepresence system affect the performer and the audience?
  19. 19. User Study 21 Performer Side Audience Side Enjoyable? Cute? Lifelike? Enjoyable? Fatigue? Useful?
  20. 20. User Study — Performer Side 22 Purpose To investigate how the participants feel when using the telepresence system and compare the results between the experts and beginners. Participants N = 14 (age 18-24) 5 professional puppeteers 9 beginners
  21. 21. User Study — Performer Side 23 1. We prepare a simple stage with two puppets for puppetry. 2. Each participant manipulates a puppet freely for approximately five minutes using our system, and only beginners take time to manipulate a puppet by hand. 3. We ask them to rate each question on a seven-point scale, and write free descriptions. Procedure
  22. 22. User Study — Puppet’s POV / Performer Side 24 Q. How useful was it to operate the puppet from the puppet’s point of view? Professional Beginner M SD 5.60 5.44 0.80 0.68 Not at all Very useful
  23. 23. User Study — Puppet’s POV / Performer Side 25 “Usually (by hand), I manipulate a puppet by imagining the puppet’s point of view, but it’s difficult to do that, and the puppet’s point of view sometimes goes in a different direction, contrary to my intention. So I feel it was useful that I could see things from the puppet’s point of view.” “I felt like I could became a character. It was similar to the feeling when I wear a costume.” — Professional — Beginner
  24. 24. Q. How useful was it to operate by moving your own body? Professional Beginner M SD 4.60 6.44 1.20 0.68 User Study — Moving body / Performer Side 26 Significant difference Not at all Very useful
  25. 25. User Study — Moving body / Performer Side 27 “I could experience it as if I had become the puppet itself… I think it has more potential if the performers practice, since it is easier to use than by hand.” “When I was asked to manipulate a puppet in a conventional manner, I attempted to move its hands using my thumb and little finger. However, I did not have a technique for moving its hands up and down.” — Professional — Beginner
  26. 26. Q. Overall, how useful was this system? Professional Beginner M SD 4.80 6.33 0.98 0.47 Significant difference User Study — Overall Usefullness/Performer Side 28 Not at all Very useful
  27. 27. “With the system, I could express my feelings in the puppet. When I manipulate a puppet directly by hand, it feels like I ‘manipulate’ a puppet, but with the system, I felt like I became a puppet.” — Beginner User Study — Overall Usefullness/Performer Side 29
  28. 28. Q. Overall, how enjoyable was it to operate a puppet with this system? Professional Beginner M SD 6.40 6.44 0.49 0.50 User Study — Overall Enjoyment / Performer Side 30 Not at all Very enjoyable
  29. 29. “I enjoyed that the mouth of the puppet moved in accordance with my voice.” “I enjoyed seeing the audience, because people come closer to the robot puppet, and they don’t normally do that. This made me laugh when our puppets communicated with each other.” — Professional — Beginner User Study — Overall Enjoyment / Performer Side 31
  30. 30. User Study — Audience Side 32 Purpose To investigate how the participants feel about the puppetry played by the telepresence system and compare the results between the hand-based and telepresence-based puppetry. Participants N = 22 (age 18-24)
  31. 31. 1. We record videos of the hand-based and telepresence-based puppetry acted by professional puppeteers. 2. Participants watch the two videos. 3. We ask them to rate each question about these puppet shows on a seven-point scale, and write free descriptions. Procedure User Study — Audience Side 33
  32. 32. 34 User Study — Audience Side Q. How enjoyable did you feel the puppetry was? Q. How cute did you feel the manipulated puppets were? Q. How lifelike did you feel the manipulated puppets were? Didn’t feel at all Felt very much Ours Hand Ours Hand Ours Hand 4.27 4.55 4.59 5.32 4.00 4.77 1.62 1.67 1.70 1.39 1.60 1.51 M SD
  33. 33. Limitations 35
  34. 34. Limitations — PositionofaPuppetonStageCannotbeChanged 36 “This robot cannot move its position, because it doesn’t have legs. I will get troubled when I use some props for detailed performance.” — Professional Possible Solution Gravity detection system like a NintendoWii Fit Other orientations detected by Kinect
  35. 35. Limitations — PerformerCannotConfirmHowaPuppetMoves 37 “…I couldn’t understand how much I could move a puppet when I moved my body. …” — Professional Possible Solution Display of the puppet’s state in the HMD Display of images from a fixed-point camera
  36. 36. Limitations — MovementsLookSomewhatMechanicalorArtificial 38 “I felt that the conventional puppets were more lifelike than the system-controlled ones…” “The hand-manipulated puppets’ movements were subtle and seemed more lifelike.” Possible Solution Extra servo motors / Background musics
  37. 37. Limitations — Stress & Fatigue on a Performer 39 “When I moved my body, the video that I saw was shaken a lot and I felt sick.” — Beginner Possible Solution Omnidirectional cameras
  38. 38. Limitations — Cost & Space 40 Possible Solution Pose estimation from RGB images / Google Cardboard
  39. 39. Future Work 41 Interactive Attractions Telecommunication Computational Design
  40. 40. Introduction — Summary of Contribution 42 1. We implemented an immersive telepresence system for puppetry. 2. We reported the effectiveness of the system for both the performer and audience side. 3. We discussed the limitations of the system and future work beyond the puppetry.
  41. 41. Research Group 43 Mose Sakashita Tatsuya Minagawa Amy Koike Ippei Suzuki Keisuke Kawahara Yoichi Ochiai Supported by
  42. 42. 44 å You as a Puppet Evaluation of Telepresence User Interface for Puppetry Mose Sakashita Tatsuya Minagawa Amy Koike Ippei Suzuki Keisuke Kawahara Yoichi Ochiai Demo Reception Please visit our booth in Today, 6pm - 9pm @Villeray, De Tourny

×