The document describes a method for capturing human motion over 360 degrees using multi-view RGB-D video data from Kinect sensors. The method involves: 1) Acquiring RGB and depth frames from three Kinect cameras and merging the point clouds and skeleton data into a unified coordinate system. 2) Tracking SIFT features between frames to establish correspondences and estimate motion vectors for temporally coherent 3D animation reconstruction. 3) Evaluating the method by comparing it to direct RGB-D SLAM and feature-based RGB-D SLAM, and analyzing residual configuration, computation time, and failure modes.

1. Temporary Coherent
3D Animation
Akshat Singh
B.Tech IV year

2. Table of Content
» Defination
» Method
» Steps Involved
» Acquisition System
» Data Acquisition
» Unified Skeletal Animation Reconstruction
» Temporally Coherent 3d Animation Reconstruction
» SIFT features
» Merging of three cameras
» Bone-length Variation
» Evaluation

3. Defination
Method for capturing human motion
over 360 degrees by the fusion of
multi-view RGB-D video data from
Kinect sensors using feature point
sampling

4. Method
» Capture real-world objects and technique ranging
from shape matching to deformation toward
coherent animation
» Use RGB data,3D reconstruction for deapth
reconstruction
» Background segmentation dealing with RGB color
space

5. Steps Involved

6. Acquisition
System
» frame of input RGB and Depth
images
» separately resampled in a 3D
point cloud
» point clouds are merged in a
unified global coordinate system

7. » RGB frames from three cameras
» Frontal and profile faces are
detected in two cameras
» depth data with the overlaid
skeleton from Kinect
» unified skeleton from the two
cameras towards which the actor's
face is oriented
Data Acquisition

8. Merging of three cameras
» unified two point clouds black and red
» corresponding two skeletons after the extrinsic calibration
» Unified skeleton reconstructed

9. Unified Skeletal Animation
Reconstruction(Still)
» unified two point clouds black
and red
» corresponding two skeletons
after the extrinsic calibration
» Unified skeleton reconstructed

10. Unified Skeletal Animation
Reconstruction
Zoomed-out point cloud with feature
points shown in blue and green. Red
point at time-step t is to be matched,
and green points are the five nearest
feature points.
Shows the zoomed-in point cloud at t.
Motion vectors are calculated with
respect to the 5 nearest feature points.
These motion vectors are used to
calculate the matching point at t+1 ,not
centered on any point because the
matching is resolution independent

11. Temporally Coherent 3d Animation
Reconstruction
Estimating Optical
Feature Points
Estimating
Geometrical
Feature Points
Mapping
Unified
skeleton
reconstruct
from method
Alignment
using Motion
Vectors

12. SIFT features using a simple Euclidean
distance measure D
» Matching of optical feature
points between two RGB images
using SIFT.
» SIFT feature has a location
q(t) = (u, v, t)
» Optical feature points L(t)
» mapping between L(t) and
L(t + 1)

13. Bounding-box
and Skeleton
Overlap
Estimation

14. Bone-length
Variation

15. Evaluation
» Comparison against direct RGB-D SLAM
» Comparison against feature-based RGB-D
SLAM
» Evaluation of the residual configuration
» Depth vs inverse depth in the geometric
reprojection error
» Computational time
» Failure modes
» Qualitative results

16. Place your screenshot here
Code Link
» https://github.com/
alejocb/rgbdtam
» https://www.youtube.co
m/watch?v=sc-
hqtJtHD4

17. THANK you!
Any questions?
You can find me at
» Linkedin/akshat7497
» akshat7497@gmail.com