This document summarizes a seminar presentation about Puppetooner, a system for 3D animation using physical puppet models and Microsoft Kinect depth sensing. The system allows puppeteers to manipulate physical puppets in front of a Kinect to capture their motion in real-time. This captured motion is then applied to virtual 3D character models which are projected back onto the physical puppets through projection mapping, creating an augmented reality experience. The system provides a low-cost and accessible way to create 3D animations without requiring software expertise by allowing puppeteers to focus on physical performance.

2. SEMINAR PRESENTATION

3. Coordinated by,
Anupama A S
Asst Professor
Dept of ECE,
SIST
Presented by,
Arunima Unnikrishnan
S7 EC(13133414)
PUPPETOOONER
Guided by,
Renjith Sudarshanan
Asst Professor
Dept of ECE,
SIST

4.  Abstract
 Introduction
 Technologies & principles
 Kinect depth sensor
 Motion capture
 Capture module
 Adavantages
 Disadvantages
 Applications
 Future scope
 Conclusion
 References
 Thank you

5.  3D animation using miniature model.
 Puppets manipulated directly infront of kinect
depth sensor.
 Motion capturing and projection mapping used.
 Operates in real time.
 No prior animation experience needed.

6.  3D animations using physical objects.
 Usually 3D animation done by software.
 Software deals with complex tools
&commands.
 Increases processing time.
 Direct manipulation of puppets during
performance.
 Kinect depth sensor is used.
 Relation between real and virtual spaces.

7.  Image feature matching &3D shape matching.

8.  Puppets operated physically.
 Kinect depth sensors used.
 Tangible interfaces.
 Object based motion capture.
 Projection mapping.
 Video puppetry.

9.  Tracks human body and provides interaction
with virtual objects.
 Uses infrared laser light with a speckle pattern.
 First compute a depth
map, then infer body
position

11. Kinect depth sensor

12.  Consist of IR camera and laser based IR
projector.
 IR projector projects known pattern read by IR
camera.
 Physical puppet 3d model.
 Scanning and moving round the puppets.
 Stream of kinect cloud untextured 3d mesh.
 Single kinects and multi kinects.

13. Painting mode for colouring.
Physical puppet ReconstructMe
model
Colorized model

14.  Sampling and recording motion of objects as 3d data
 Object-based motion-capture systems.
 A single Kinect and a table laptop.
 Filter out points belonging to the puppeteer’s hands.
 Puppets with colors similar to puppeteer’s skin color
not used.

15.  Mainly 3 components.
 Puppet Identifier.
 Point-cloud segmenter.
 Pose Tracker.
 Inputs to Capture module-Puppet Database and Kinect
data.
 Puppet Identifier Point-cloud Segmenter Pose Tracker.

16.  PUPPET IDENTIFIER
 Identifies puppets within Kinect field.
 Estimates poses initialization for Pose Tracker.
 Find scale-invariant feature transform (SIFT).
 POINT CLOUD SEGMENTER
 Removes background points & hands from cloud.
 Splits remaining points to separate clouds.

17. Raw point cloud Background Removed Skin Removed Segmented by
Puppet

18.  POSE TRACKER
 Estimates pose of puppet accurately.
 Aligns segmented cloud points with 3d model database.
 Use iterative closest point (ICP) algorithm.
Initial Alignment 2 Iterations 4 Iterations 32 Iterations

19. Capture module

20.  PROJECTION MAPPING
 Video mapping and spatial augmented reality.
 Irregularly shaped objects display surface for
video projection.
 Texture,events&animations of 3d model projected to
miniature models.

21.  3D characters controlled physically.
 No complex tools or command set.
 Quick 3d animation.
 Operates in real time.
 Inexpensive capture hardware&simple setup.

22.  Familiar and approachable.
 Natural looking motion.
 Layered animation used.
 Flexibility and accessability.
 Enjoyable user experience.

23.  Deformable puppets not used.
 Mismatches can occur.
 Tracking errors.

24.  Gaming.
 Cartoons.

26.  Integrated Puppet Loading.
 Articulated Puppets.
 Deformable Puppets.
 Multiple Puppeteers.

27.  New interface for 3d animation with physical objects.
 Puppeteers focus on physical performance.
 User friendly and enjoyable technique.
 3d animation in hands.

29.  IEEE explorer official website-2013 paper.
 Besl, P., and McKay, N. A method for registration of 3-
D shapes. IEEE PAMI 14, 239–256.
 1. Autodesk. 123D Catch.
http://www.123dapp.com/catch.
 2. Autodesk. 3ds Max. http://usa.autodesk.com/3ds-
max/.
 3. Autodesk. Maya. http://usa.autodesk.com/maya/.
 4. Avrahami, D., Wobbrock, J. O., and Izadi, S.
Portico:tangible interaction on and around a tablet. In
Proc.UIST (2011), 347–356.

30. QUESTIONS
……???