Cinematic VR

1. Build Your Own VR Display
Panoramic Imaging and Cinematic VR
Robert Konrad
Stanford University
stanford.edu/class/ee267/

2. Summary
• overview
• panoramic imaging
• stereo / omnistereo panoramas
• camera rigs

3. Jaunt VR

5. Lytro

6. Google

7. Nokia

8. Facebook

9. Red

10. Samsung

11. Panorama v Stereo Movie v Stereo Panorama

12. Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation

13. Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation

14. Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation

15. Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation

16. Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
1 center of
projection!

17. Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
1 center of
projection!

18. Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
1 center of
projection!
center of
projection

19. Panoramas
SlidesfromMarcLevoy’sexcellentCS178course

20. Panoramas
SlidesfromMarcLevoy’sexcellentCS178course

21. Panoramas
SlidesfromMarcLevoy’sexcellentCS178course

22. Panoramas
SlidesfromMarcLevoy’sexcellentCS178course

23. Panoramas
SlidesfromMarcLevoy’sexcellentCS178course

24. Panoramas
SlidesfromMarcLevoy’sexcellentCS178course

25. Panoramas
SlidesfromMarcLevoy’sexcellentCS178course

26. Panoramas
SlidesfromMarcLevoy’sexcellentCS178course

27. Panoramas
SlidesfromMarcLevoy’sexcellentCS178course

28. Panoramas
SlidesfromMarcLevoy’sexcellentCS178course

29. Panoramas
SlidesfromMarcLevoy’sexcellentCS178course

30. Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
1 center of
projection!

31. Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
1 center of
projection!
2 centers of
projection!

32. Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
1 center of
projection!
2 centers of
projection!

33. Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
1 center of
projection!
2 centers of
projection!

34. Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
1 center of
projection!
2 centers of
projection!

35. Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
1 center of
projection!
2 centers of
projection!

36. Image courtesy of Google Jump

37. Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
1 center of
projection!
2 centers of
projection!

38. Head Rotation
could be IPD
Inter Pupillary Distance
sidebyHariLakshman(EE368)

39. Image courtesy of Google Jump

40. Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
1 center of
projection!
2 centers of
projection!
multiple centers of
projection

41. Image courtesy of Google Jump

42. Image courtesy of Google Jump
Store image pair for each direction  Problem: Too much data!!!

43. Image courtesy of Google Jump
Omnidirectional Stereo (ODS) Approximation
Peleg et. al 2001
Ishiguro et. al 1990

44. Image courtesy of Google Jump
Omnidirectional Stereo (ODS) Approximation
Peleg et. al 2001
Ishiguro et. al 1990

46. Comparison: Mono and Stereo Panoramas
Central,
a.k.a. Mono
Omnistereo,
Multiperspective
Peleg et al. 2001

47. Capture using Single Camera
Peleg et al. 2001
Figure 6. Toe in camera model for direct zero parallax generation. The panoramic images so created have
preset zero parallax distance.
Figure 7. Single camera capture of stereoscopic panoramic images, more common for real camera capture
sidebyHariLakshman(EE368)

49. Anderson et. al, SIGGRAPH Asia 2016

50. Limitations!
• Close objects
• Thin structures
• Semi-transparent
surfaces
• Flow mismatches
Anderson et. al, SIGGRAPH Asia 2016

51. Facebook’s Surround 360
RAW Data: 17 Gb/sec
Compute time: days to weeks on conventional computer,
minutes to hours on data center

52. Facebook’s Surround 360
RAW Data: 17 Gb/sec
Compute time: days to weeks on conventional computer,
minutes to hours on data center

54. Konrad et al.,
SIGGRAPH Asia 2017

55. Konrad et al.,
SIGGRAPH Asia 2017

56. Konrad et al.,
SIGGRAPH Asia 2017

57. Konrad et al.,
SIGGRAPH Asia 2017

58. Konrad et al.,
SIGGRAPH Asia 2017

60. Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
RicohTheta
horizontal-only
parallax

61. Additional Information
• M. Brown, D. Lowe “Automatic Panoramic Image Stitching using Invariant
Features”, IJCV 2007
• autostitch: http://matthewalunbrown.com/autostitch/autostitch.html
• S. Peleg, M. Ben-Ezra, Y. Pritch “Omnistereo: Panoramic Stereo Imaging” IEEE
PAMI 2001
https://stanford.edu/class/ee267/

62. Omnistereo Panoramas
Left panorama
Right panorama
Omnistereo Format
Peleg et al. 2001
sidebyHariLakshman(EE368)

63. Zero Disparity Distance
To control zero disparity distance: circularly shift left pano relative to right pano
sidebyHariLakshman(EE368)

64. Multiperspective Projection
JingyiYu

66. Left panorama
Right panorama
Disparity
Sphere-to-plane
distortions
Omnistereo examplesidebyHariLakshman(EE368)