3. Getting Started: Hardware
You will need:
• A Kinect, or generic equivalent such as the Asus Xtion
• USB/AC adapter cable, if you have a Kinect that was bundled
with the Xbox (don’t buy the Microsoft adapter, the cheap ones
work fine)
• Enough space to perform
5. Two competing SDKs
OpenNI
• Free and open source
• Works with all models of Kinect as well as other devices such as the Asus Xtion.
• Cross platform – even supports ARM processors and Android
Microsoft (official) SDK
• More tracking points
• Estimates occluded joints and hand/foot rotation
• Special case capture modes (seated figure etc.)
• Only works with Kinect For Windows – on Windows
9. “Working Backwards”
Traditional Animation Motion Capture Workflow
1. Start with scene already 1. Start with full color, full-motion
storyboarded and timed out characters
2. Create sketches of key poses for 2. Refine blocking and movement;
each character add facial animation
3. Add in-between drawings for 3. Choose shots and timing of
movement edit, add camera moves
4. Color and composite characters 4. (Optional) Re-record any
in final scene performances that aren’t working
13. Design Tips
• “Break” your model at every
joint.
• Build in overlap so joints have
room to rotate without
disconnecting.
• Parts that fall in front of and
behind another layer (e.g. the
hair of the model at left)
should be two layers.
• The hands, feet and head can
each have multiple elements.
14. Cutouts
Raster-based characters drawn or cut out of existing images or photographs.
• Work big if you want to be able to zoom in on your characters
• When cutting out layers, use “Refine Edge” to smooth your
edges and clean up any extra junk that got selected
• Clone/heal/paint in portions that will be missing or problematic
when the model changes pose
15. Vector-Based Characters
Drawn with math, not pixels. Easier to break into layers – but watch out.
• Can be built in Illustrator, Photoshop or with AE Shape Layers
• Vector-based characters are not infinitely scalable as you’d
expect: rigging with AE’s Puppet Tool generates a mesh at a set
resolution.
• Building puppets with vector layers adds an extra step: they
need to be precomposed before rigging.
18. Capturing
• Record 15-point 3D skeletal
tracking data
• Capture in sync to dialogue
• Remote control with a standard
presentation remote
• Open source and cross-platform:
runs on OpenNI, built in
Processing, also works with
generic sensors like the Asus
Xtion
19.
20. Acting for Kinect
• The Kinect can’t record what it can’t see.
• Keep your entire body in the camera’s range
• Face the camera when capturing
• Watch out for occluded joints (sitting down, putting your hands behind
your back etc. can cause it to lose the track)
• It’s surprisingly good at picking up subtle posture shifts, but not
smaller motions like laughter.
• Use a different performer for each character if possible
21. Data formats for export
KinectToPin can output quite a few data, but the ones we’ll be
using today are:
• XML (Native recording format, no export required)
• After Effects 3D Point Controls (CS5.5+)
• After Effects 2D Point Controls (CS4+)
Tip: Edit KinectToPin’s settings.txt file to choose your output
formats.
24. Meet the KinectToPin UI Panel.
• Create character
templates
• 2D and 3D setups
• Native XML import
(SLOW)
• Automatically add
expressions to smooth
tracking data
• Automate rigging
25. The Puppet Tool
THREE OR MORE PINS ON ONE LAYER: TWO PINS EACH, MULTIPLE LAYERS:
Elbows don’t bend this way. Ow. Rotation with stretch and squash!
26. AE Puppet best practices
• Work BIG. Start high-res, scale down in the puppet comp, then
scale back up in the project comp (enable Collapse
Transformations).
• Precompose if you’re using vectors or shape layers
• Keep multiple angles of the same character in sync
High-res
Start big. Character
Layers
Project Comp
End big.
Rigging
Template
31. Quick auto-lip sync
2. Alt-click the stopwatch to add an expression. Use the
pickwhip to connect the audio keyframes to the mouth animation.
32. Quick auto-lip sync
3. Size the audio data to fit. Use this expression:
audio = [your audio keyframes];
linear(audio,sourceMin,sourceMax,targetMin,targetMax)
Lowest value of source data
Highest value of source data
Closed mouth (probably 0)
Widest possible open mouth
34. Further non-K2P tools
FREE COMMERCIAL
Brekel Kinect iPi Soft
Open-source capture for 3D Uses two Kinects to record
animation (Windows-only) www.ipisoft.com
www.brekel.com
NI Mate
Duik
Kinect for OSC/MIDI/Blender
Inverse kinematics plugin for AE
www.ni-mate.com
www.duduf.net
Why DIY MoCap?Motion capture is now cheap and accessibleNo longer need a giant studioIt’s FAST.Ideal use cases: short-turnaround animation, rapid prototyping etc.
Microsoft SDK vs. OpenNI
Can add multiple motion tracks and duplicate nullsPin names need to match control nulls
2D vs. 3D: Avoiding the “Uncanny Valley”Awesome glitches
