This document discusses building a VR video player from scratch. It covers topics like VR video formats, including 360 equirectangular images and stereoscopic 3D. It also discusses capturing equipment, social platforms, and audio formats like Ambisonics. The document provides information on visual and auditory immersion techniques using OpenGL shaders and binaural audio. It also covers optimization techniques for VR video players using debugging features, color space handling, and GPU processing. Issues with video size and simultaneous streaming are also mentioned.

1. Build a VR Video Player from scratch
Yu-Mei Chen
HTC Software Engineer

2. What is VR Video

3. 360 equirectangular images

4. 4

5. 5

6. Stereoscopic 3D
Left Eye View Right Eye View

7. Google IO 2016 talk: Spatial Audio and Immersion

8. Viewed in an HMD

9. What is VR Video
Left Eye View Right Eye View

10. Capturing Equipment

11. Social Platform

12. What is VR Video
Left Eye View Right Eye View

13. Yu-Mei Chen
Andre Chen
Open source date:
[ coming soon.]

15. Format Introduction
Immersive Perception
Optimization

16. Format Introduction

17. Angle Coverage

18. Stereo
Left Eye View
Right Eye View
TB -- Top Bottom
Left Eye View Right Eye View
SBS -- Side By Side

19. Stereo
TB -- Top Bottom
SBS -- Side By Side

20. Minimum resolution: 4K
• Display Resolution:
1080x1200 each eye
• Field of View:
110 degree
• Current 4K format: 3840, 4096,….

21. Immersive Perception

22. Visual

25. OpenGL fragment shader

27. 有Jack的圖再來合成

28. Auditory

29. Thrid Order Ambisonics
Ambisonics
Second Order Ambisonics
First Order Ambisonics

30. Audio Format
• First Order Ambisonics, a.k.a. FOA
 4 channels in a track
• TBE (Two Big Ears)  Specific Second Order Ambisonics
 4 channels in first track ( W, Y, X, Z+R )
 4 channels in second track ( U, V, T, S )
 2 channels in third track ( head lock )

31. Head Related Transfer Function (HRTF)
https://developers.google.com/vr/concepts/spatial-audio

32. Optimization

33. Vive Cinema Debugging feature

35. Or

38. Color Space
Rec.601 Rec.709
CPU
GPU
YUV

39. OpenGL fragment shader

40. Live Streaming

41. Known Issues
• Video size is too large
• Multiple Users watch the same movie simultaneously

42. Demo Video

43. 43
yumei_chen@htc.com
andre_chen@htc.com
Welcome to email: