3. INTRODUCTION
Many computer graphics applications require realistic
3D models of human bodies.
Depth cameras such as Microsoft Kinects are able to
capture depth and image data at video rate.
Kinect is compact, low-price and as easy to use as a
video camera.
10/5/2013 3
5. RELATED WORK
Scanning devices based on structured light or laser
scan can capture human body with much high quality,
but is very expensive (about $240,000).
Two main approaches employed in depth camera
technology are:-
based on the time-of-flight principle, measuring
time delay between transmissions of a light pulse(about
$8,000).
based on light coding; projecting a known infrared
pattern onto the scene and determining depth based on
the pattern’s deformation.
10/5/2013 5
6. RELATED WORK(contd…)
A most popular one based on light coding is the Microsoft
Kinect Sensor which is at a price of only $150.
3 Main Types:-
Registration without a template.
This method requires high quality scan data & needs small
changes in temporal coherence.
Registration with a template.
This method needs a relative accurate template & then uses
the template to fit each scan.
10/5/2013 6
7. RELATED WORK(contd…)
Registration with a semi-template.
Rough template, such as the skeleton model of
articulated object, can be utilized.
The first type requires high quality input data & is
computationally expensive; the second one needs an
accurate template which is hard to fulfil in many
applications.
Here, this system uses the third type.
10/5/2013 7
8. RELATED WORK( contd…)
The idea of creating a graph of pairwise alignments
between scans.
First, pairwise rigid alignment is computed in the
geometric level.
Global error distribution then operates on an upper level,
where errors are measured in terms of the relative rotations
and translations of pairwise alignments.
The graph methods can simultaneously minimize the
errors of all views rapidly and do not need all scan in
memory.
10/5/2013 8
9. SYSTEM SETUP
Two kinects are used to capture the upper and the
lower part of a human body respectively, without
overlapping region, from one direction.
A third kinect is used to capture the middle part of the
human body from the opposite direction.
The distance between two sets of Kinects is about 2
meters.
A turntable is put in between them.
10/5/2013 9
11. RECONSTRUCTION APPROACH
Denote Di={Mi,Ii},i=1,….n as the captured data, n is the
number of captured frames, Mi is the merged mesh & Ii is
the corresponding image of the i-th frame respectively.
First, a rough template is constructed.
The template is used to deform the geometry of successive
frames pair wisely.
Global registration is performed to distribute errors in the
deformation space.
Finally, reconstructed model is generated using Poisson
reconstruction method.
10/5/2013 11
12. An accurate template is unavailable.
We construct an estimated body shape as the template
mesh T1 from the first frame.
It is impossible to use this template to register each
frame by geometry fitting but it can track the pairwise
deformation of successive frames.
T1={v1^k,k=1…K; K is the number of nodes of T1
(typically 50-60).
10/5/2013 12
13. Suppose Mi, i=1…n forming a cycle. fi,j denotes the
registration that can deform mesh Mi to register with
mesh Mj.
To find the pairwise registration f1,2,f2,3,…fn-1,n,fn,1.
Deformation Model:
Suppose we have two meshes Mi & Mj, and template
mesh at frame i is Ti, then
10/5/2013 13
14. Pairwise registration:
For successive frames Mi and Mi+1,corresponding feature
points are obtained by optical flow in the corresponding
images.
Projection to the first frame:
n-1 pair wise deformation is required to recover all the
relative position of all frames.(refer fig.6)
The desired pairwise deformation f̂1,2,f̂2,3,…f̂n-1,n,f̂n,1
should meet the following conditions:
1.It is cyclic consistent.
2.The original pairwise deformation is relatively correct, so
minimize the weighted square error of the new and old
deformation.
10/5/2013 14
18. RESULTS
Global non-rigid registration gives better result than
global rigid alignment.
Since the color image and depth information are
captured simultaneously and calibrated, the color
information of deformed mesh is generated
automatically.
Virtual try on.
Personalized avatar-video games,online shopping ,
human computer interaction etc.
10/5/2013 18
19. Realistic virtual try on experience based on the reconstructed
model.(Left)the try on results;(right)the corresponding meshes.
10/5/2013 19
20. Personalized avatar generated by our system.
The motion of the human body is driven by a given skeleton motion
sequence
10/5/2013 20
21. CONCLUSION
The proposed method can deal with non-rigid
alignment and complex occlusions.
The two stage registration algorithm is efficient and of
memory efficiency.
The system can generate convincing 3D human bodies
at a much low price.
It has good potential for home oriented VR
applications.
10/5/2013 21
22. REFERENCES
[1]Jing Tong; Jin Zhou; Ligang Liu; Zhigeng Pan; Hao
Yan, ”Scanning 3D Full Human Bodies Using
Kinects”,Visualization and Computer Graphics, IEEE
Transactions on, vol.18, no.4, April 2012.
[2]Srivishnu Satyavolu,Gerd Bruder,Pete
Willemsen,Frank Stenicke,”Analysis of IR-based
virtual reality tracking using multiple Kinects”,2012
IEEE Virtual Reality,2012.
10/5/2013 22
