Build Your Own VR Display
Panoramic Imaging and Cinematic VR
Robert Konrad
Stanford University
stanford.edu/class/ee267/
Summary
• overview
• panoramic imaging
• stereo / omnistereo panoramas
• camera rigs
Jaunt VR
Lytro
Google
Nokia
Facebook
Red
Samsung
Panorama v Stereo Movie v Stereo Panorama
Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
1 center of
projection!
Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
1 center of
projection!
Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
1 center of
projection!
center of
projection
Panoramas
SlidesfromMarcLevoy’sexcellentCS178course
Panoramas
SlidesfromMarcLevoy’sexcellentCS178course
Panoramas
SlidesfromMarcLevoy’sexcellentCS178course
Panoramas
SlidesfromMarcLevoy’sexcellentCS178course
Panoramas
SlidesfromMarcLevoy’sexcellentCS178course
Panoramas
SlidesfromMarcLevoy’sexcellentCS178course
Panoramas
SlidesfromMarcLevoy’sexcellentCS178course
Panoramas
SlidesfromMarcLevoy’sexcellentCS178course
Panoramas
SlidesfromMarcLevoy’sexcellentCS178course
Panoramas
SlidesfromMarcLevoy’sexcellentCS178course
Panoramas
SlidesfromMarcLevoy’sexcellentCS178course
Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
1 center of
projection!
Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
1 center of
projection!
2 centers of
projection!
Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
1 center of
projection!
2 centers of
projection!
Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
1 center of
projection!
2 centers of
projection!
Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
1 center of
projection!
2 centers of
projection!
Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
1 center of
projection!
2 centers of
projection!
Image courtesy of Google Jump
Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
1 center of
projection!
2 centers of
projection!
Head Rotation
could be IPD
Inter Pupillary Distance
sidebyHariLakshman(EE368)
Image courtesy of Google Jump
Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
1 center of
projection!
2 centers of
projection!
multiple centers of
projection
Image courtesy of Google Jump
Image courtesy of Google Jump
Store image pair for each direction  Problem: Too much data!!!
Image courtesy of Google Jump
Omnidirectional Stereo (ODS) Approximation
Peleg et. al 2001
Ishiguro et. al 1990
Image courtesy of Google Jump
Omnidirectional Stereo (ODS) Approximation
Peleg et. al 2001
Ishiguro et. al 1990
Comparison: Mono and Stereo Panoramas
Central,
a.k.a. Mono
Omnistereo,
Multiperspective
Peleg et al. 2001
Capture using Single Camera
Peleg et al. 2001
Figure 6. Toe in camera model for direct zero parallax generation. The panoramic images so created have
preset zero parallax distance.
Figure 7. Single camera capture of stereoscopic panoramic images, more common for real camera capture
sidebyHariLakshman(EE368)
Anderson et. al, SIGGRAPH Asia 2016
Limitations!
• Close objects
• Thin structures
• Semi-transparent
surfaces
• Flow mismatches
Anderson et. al, SIGGRAPH Asia 2016
Facebook’s Surround 360
RAW Data: 17 Gb/sec
Compute time: days to weeks on conventional computer,
minutes to hours on data center
Facebook’s Surround 360
RAW Data: 17 Gb/sec
Compute time: days to weeks on conventional computer,
minutes to hours on data center
Konrad et al.,
SIGGRAPH Asia 2017
Konrad et al.,
SIGGRAPH Asia 2017
Konrad et al.,
SIGGRAPH Asia 2017
Konrad et al.,
SIGGRAPH Asia 2017
Konrad et al.,
SIGGRAPH Asia 2017
Panorama v Stereo Movie v Stereo Panorama
Panorama
mono & head rotation
Stereo
stereo & no head rotation
Stereo Panorama
stereo & head rotation
RicohTheta
horizontal-only
parallax
Additional Information
• M. Brown, D. Lowe “Automatic Panoramic Image Stitching using Invariant
Features”, IJCV 2007
• autostitch: http://matthewalunbrown.com/autostitch/autostitch.html
• S. Peleg, M. Ben-Ezra, Y. Pritch “Omnistereo: Panoramic Stereo Imaging” IEEE
PAMI 2001
https://stanford.edu/class/ee267/
Omnistereo Panoramas
Left panorama
Right panorama
Omnistereo Format
Peleg et al. 2001
sidebyHariLakshman(EE368)
Zero Disparity Distance
To control zero disparity distance: circularly shift left pano relative to right pano
sidebyHariLakshman(EE368)
Multiperspective Projection
JingyiYu
Left panorama
Right panorama
Disparity
Sphere-to-plane
distortions
Omnistereo examplesidebyHariLakshman(EE368)

Build Your Own VR Display Course - SIGGRAPH 2017: Part 5