NeRF-based Real-Time Rendering Photo-Realistic Graphics Methods Review

Shunyu Yang

doi:10.62051/26c21r61

Authors

Shunyu Yang

DOI:

https://doi.org/10.62051/26c21r61

Keywords:

Neural Radiance Field, Computer Graphics, Neural Network Rendering, Real-Time.

Abstract

Recently, the publication of the Neural Radiance Field (NeRF) has sparked a surge in further research, unveiling a wealth of innovative solutions in related domains. With its potent predictive capabilities and operational efficiency, neural networks have addressed the traditional trade-off dilemma, enabling real-time rendering at state-of-the-art (SOTA) quality. This breakthrough has introduced a novel rendering approach in computer graphics (CG) and unlocked fresh opportunities for real-time applications such as video games, augmented reality (AR), and virtual reality (VR). This paper aims to offer a comprehensive overview of recent NeRF-like methodologies and explore potential pathways for enhancing NeRF to achieve both real-time performance and photorealistic standards. Our study includes a comparative analysis of these methodologies in terms of their efficacy and hardware requirements. Ultimately, this paper outlines potential future advancements in this field. The objective of this paper is to familiarize both newcomers and researchers with NeRF, catalyze in-depth investigations, propose enhanced methodologies.

Downloads

Download data is not yet available.

References

Mildenhall B, Srinivasan P P, Tancik M, et al. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 2021, 65(1): 99-106.

Gao K, Gao Y, He H, et al. Nerf: Neural radiance field in 3d vision, a comprehensive review. arXiv preprint arXiv:2210.00379, 2022.

Yu A, Li R, Tancik M, et al. Plenoctrees for real-time rendering of neural radiance fields, Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 5752-5761.

Zhang J, Huang J, Cai B, et al. Digging into radiance grid for real-time view synthesis with detail preservation, European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 724-740.

Hedman P, Srinivasan P P, Mildenhall B, et al. Baking neural radiance fields for real-time view synthesis, Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 5875-5884.

Kerbl B, Kopanas G, Leimkühler T, et al. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 2023, 42(4): 1-14.

Duckworth D, Hedman P, Reiser C, et al. SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Exploration. arXiv preprint arXiv:2312.07541, 2023.

Reiser C, Szeliski R, Verbin D, et al. Merf: Memory-efficient radiance fields for real-time view synthesis in unbounded scenes. ACM Transactions on Graphics, 2023, 42(4): 1-12.

Chen Z, Funkhouser T, Hedman P, et al. Mobilenerf: Exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 16569-16578.

Yariv L, Hedman P, Reiser C, et al. Bakedsdf: Meshing neural sdfs for real-time view synthesis, ACM SIGGRAPH 2023 Conference Proceedings. 2023: 1-9.

Tang J, Zhou H, Chen X, et al. Delicate textured mesh recovery from nerf via adaptive surface refinement, Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023: 17739-17749.

Guo Y C, Cao Y P, Wang C, et al. VMesh: Hybrid volume-mesh representation for efficient view synthesis, SIGGRAPH Asia 2023 Conference Papers. 2023: 1-11.

Reiser C, Garbin S, Srinivasan P P, et al. Binary Opacity Grids: Capturing Fine Geometric Detail for Mesh-Based View Synthesis. arXiv preprint arXiv:2402.12377, 2024.

Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman.Mip-NeRF 360: Unbounded anti-aliased neural radiance fields.CVPR, 2022.