The document presents a method called 3D Gaussian Splatting for high-quality real-time radiance field rendering, achieving over 100 frames per second at 1080p. It discusses various techniques related to scene reconstruction, neural rendering, and point-based rendering, citing multiple research papers that have contributed to these fields. Additionally, it includes information about datasets used for quantitative and qualitative evaluations of the proposed method.

1. 3D Gaussian Splatting for
Real-Time Radiance Field Rendering
(SIGGRAPH 2023)
이미지처리팀
김병현(발표자), 김준철, 최승준, 허정원
Sep 17th, 2023

2. Gaussian Splatting
 High-quality and real-Time radiance field rendering method (≥ 100
fps at 1080p) based on 3D Gaussians
2

3. Gaussian Splatting
 High-quality and real-Time radiance field rendering method (≥ 100
fps at 1080p) based on 3D Gaussians
3

4. Gaussian Splatting
 High-quality and real-Time radiance field rendering method (≥ 100
fps at 1080p) based on 3D Gaussians
4

5. RWs-1: Traditional Scene Reconstruction
 Lumigraph
5
Gortler, S. J., Grzeszczuk, R., Szeliski, R., & Cohen, M. F. (2023). The lumigraph.
In Seminal Graphics Papers: Pushing the Boundaries, Volume 2 (pp. 453-464).
The plenoptic function is a five dimensional
quantity describing the flow of light at
every 3D spatial position (x, y, z) for every 2D
direction (θ,Φ).

6. RWs-1: Traditional Scene Reconstruction
 Structure from Motion
6
Snavely, N., Seitz, S. M., & Szeliski, R. (2006). Photo tourism: exploring photo
collections in 3D. In ACM siggraph 2006 papers (pp. 835-846).

7. RWs-1: Traditional Scene Reconstruction
 Structure from Motion
7
Snavely, N., Seitz, S. M., & Szeliski, R. (2006). Photo tourism: exploring photo
collections in 3D. In ACM siggraph 2006 papers (pp. 835-846).

8. RWs-1: Traditional Scene Reconstruction
 Multi-View Stereo for Community Photo Collections
8
Goesele, M., Snavely, N., Curless, B., Hoppe, H., & Seitz, S. M. (2007, October). Multi-view stereo for community
photo collections. In 2007 IEEE 11th International Conference on Computer Vision (pp. 1-8). IEEE.

9. RWs-2: Neural Rendering & Radiance Fields
 View Synthesis by Appearance Flow
9
Zhou, T., Tulsiani, S., Sun, W., Malik, J., & Efros, A. A. (2016). View synthesis by appearance flow. In Computer Vision–ECCV 2016: 14th European
Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14 (pp. 286-301). Springer International Publishing.

10. RWs-2: Neural Rendering & Radiance Fields
 NeRF: Representing Scenes as Neural Radiance Fields for View
Synthesis
10
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2021). Nerf: Representing scenes
as neural radiance fields for view synthesis. Communications of the ACM, 65(1), 99-106.

11. RWs-2: Neural Rendering & Radiance Fields
 Mip-Nerf
11
Barron, J. T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., & Srinivasan, P. P. (2021). Mip-nerf: A multiscale representation
for anti-aliasing neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 5855-5864).

12. RWs-2: Neural Rendering & Radiance Fields
 Instant Neural Graphics Primitives with a Multiresolution Hash
Encoding
12
Müller, T., Evans, A., Schied, C., & Keller, A. (2022). Instant neural graphics primitives with a
multiresolution hash encoding. ACM Transactions on Graphics (ToG), 41(4), 1-15.

13. RWs-3: Point-Based Rendering
13
Sainz, M., & Pajarola, R. (2004). Point-based rendering
techniques. Computers & Graphics, 28(6), 869-879.

14. RWs-3: Point-Based Rendering
14
Yifan, W., Serena, F., Wu, S., Öztireli, C., & Sorkine-Hornung, O. (2019). Differentiable surface
splatting for point-based geometry processing. ACM Transactions on Graphics (TOG), 38(6), 1-14.

15. Q & A
Q & A
15

16. Method
16

17. 3D Gaussians
 A Gaussian distribution on 3D space with centered at point (mean) 𝜇
and a full 3D covariance matrix Σ
17
https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/
3D Gaussian Visualization:
Left side of each image is the final rendering result, and the right side is 3D Gaussians

18. 3D Gaussians
 A Gaussian distribution on 3D space with centered at point (mean) 𝜇
and a full 3D covariance matrix Σ
18
https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/

19. Adaptive Density Control
19

20. Fast Differentiable Rasterization
 Rasterization
20
https://www.scratchapixel.com/lessons/3d-basic-rendering/rasterization-
practical-implementation/overview-rasterization-algorithm.html
Concept of rasterization

21. Fast Differentiable Rasterization
21

22. Fast Differentiable Rasterization
22

23. Fast Differentiable Rasterization
23

24. Optimization
24

25. Q & A
Q & A
25

26. Datasets
26
Tank & Temple (150-300 Images)
(https://www.tanksandtemples.org/)
MIP-Nerf 360 (100-330 Images)
(https://jonbarron.info/mipnerf360/)
Deep Blending
(http://visual.cs.ucl.ac.uk/pubs/deepblending/)

27. Experiments
 Quantitative Comparison
27
Quantitative evaluation compared to previous work

28. Experiments
 Quantitative Comparison
28
Quantitative evaluation compared to previous work

29. Experiments
 Quantitative Comparison
29
Quantitative evaluation compared to previous work

30. Experiments
 Qualitative Comparison
30

31. Ablation Study
 Optimization Iteration
31

32. Ablation Study
 Initialization with SfM points
32

33. Ablation Study
 Initialization with SfM points
33

34. Shrunken Gaussians
34

35. Limitations
35

36. Q & A
Q & A
36

37. Thank you for your attention!
Thank you for your attention!
37