This is the presentation slide of paper : Perspective-Aware Warping for Seamless Stereoscopic Image Cloning. It is made by Sheng-Jie Luo, National Taiwan University. Please refer to http://www.cmlab.csie.ntu.edu.tw/~forestking/research/SIGA12-StereoCloning/ for more detail.

1. Perspective-Aware Warping for
Seamless Stereoscopic Image Cloning
+ 
Sheng-Jie Luo1, I-Chao Shen2,
Bing-Yu Chen1, Wen-Huang Cheng2, Yung-Yu Chuang1
1 National Taiwan University, 2 Academia
Sinica

2. Stereoscopic in Daily Life
2

3. 3D Media: Depth Information
Left Right Depth
Depth
3

4. Stereoscopic Image Cloning
Source (3D Vis)
Target (3D Vis) Result
4

5. Challenges
The depth, shape, and color of the cloned object
should be locally adjusted. Global adjustment
Direct pasting
5

6. Previous Work
 Image cloning
 Poisson image editing [Perez et al. 2003]
 Drag and drop pasting [Jia et al. 2006]
 Coordinates for instant image cloning [Farbman et al.
2009]
 Natural and seamless image composition with color
control [Yang et al. 2009]
[Perez et al. 2003]
6

7. Previous Work
 Stereoscopic 3D copy & paste [Lo et al. 2010]
 Segmentation-based approach
[Lo et al. 2010]
7

8. Our Idea
 Segmentation-free
 vague boundary
 complex silhouettes
 Local adjustment
 depth
 shape
 color
Our method
8

9. Our Idea
Global shifting Our method
9

10. Our vs. Segmentation-based Method
Segmentation-based method
Our method
[Lo et al. 2010]
Objects that are difficult to be Objects that can be easily
segmented segmented out
Objects that attach on the Objects that stand on the
surroundings ground
10

11. Preprocessing
 Disparity map extraction
 Stereo matching [Smith et al. 2009]
 Feature extraction
 SIFT [Lowe 2004]
11

12. User Selection
Source
Target
Left image Right image
12

13. Contour Transfer
Inner mesh Contour Outer mesh
Left image Right image
Content-aware warping
13

14. Contour Transfer
Content-aware warping:
Left Right
1. Feature correspondence:
L R
Corresponding feature points
14

15. Contour Transfer
Content-aware warping:
Left Right
2. Vertical alignment:
L R
warping
15

16. Contour Transfer
Content-aware warping:
Left Right
3. Triangle shape distortion:
16

17. Contour Transfer
User specified contour Transferred contour Our method
using disparity values
17

18. Depth Adaptation
Cloned regions & meshes
Disparity map
Target images
18

19. Depth Adaptation
Step 1.
Compute the re-estimated disparity values.
(Gradient-domain disparity adaptation)
Step 2.
Adjust the shape using the expected depth values.
(Perspective-aware warping)
19

20. Gradient-domain Disparity Adaptation
 The depth is seamless along the boundary.
Weighting: Guidance field:
20

21. Perspective-Aware Warping
Relationship between scale and disparity:
A B Distance to
camera
21

22. Perspective-Aware Warping
1. Disparity-dependent scaling:
22

23. Perspective-Aware Warping
2. Disparity consistency:
Re-estimated
disparity map
23

24. Perspective-Aware Warping
3. Vertical alignment:
24

25. Perspective-Aware Warping
4. Position fixation:
25

26. Results
26

27. Results - Comparison
Source image
Target image
27

28. Results - Comparison
Direct pasting
Depth
y
28

29. Results - Comparison
Global adjustment
Depth
y
29

30. Results - Comparison
Our method
Depth
y
30

31. Results - Auto-resizing
Left image
Right image
Result
31

32. More Results
Source image
Target image
32

33. More Results
Result
33

34. More Results
Source image
Target image
34

35. More Results
Result
35

36. More Results
Source image
Target image
36

37. More Results
Result
37

38. Performance
Intel Core 2 Duo 3.2GHz CPU and 4GB RAM
Fig. 1 Fig. 8b Fig. 8c Fig. 9 Upper Fig. 10 Upper Fig. 10 Lower
38

39. Limitations
 Large perspective differences
 Accuracy of the disparity maps
39

40. Conclusion
 Stereoscopic image cloning is challenging
 depth, shape, size and color
 A gradient domain + perspective-aware
warping technique
 it doesn’t require precise segmentation of the objects
 it guarantees depth continuity across the boundary
and models perspective foreshortening
40

41. Future Work
 View interpolation or depth-image-based
rendering
 Depth control of the pasted objects
41

42. Acknowledgments
 SIGGRAPH Asia paper reviewers
 Photos from Flickr users
 Wayne Karberg (turbguy), -ytf-, Patrick McDonald
(clayspur), pinboke planet, and Dan Ridley-Ellis (Dan)
 NSC of Taiwan grants
 NSC100-2622-E-002-016-CC2
 NSC101-2628-E-002-031-MY3
 NSC101-2221-E-001-016
42

43. Perspective-Aware Warping for
Seamless Stereoscopic Image Cloning
Sheng-Jie Luo, I-Chao Shen, Bing-Yu Chen, Wen-Huang Cheng, Yung-Yu Chuang
Thank
you!
Additional Materials:
http://www.cmlab.csie.ntu.edu.tw/~forestking/research/SIGA12-StereoCloning/
43