Research on 3D Reconstruction Methods Based on Deep Learning

Abstract

Deep learning applications have been applied extensively and have made tremendous strides in the 3D reconstruction field in recent years. This paper offers a methodical review of deep learning-based techniques for single-view images, multi-view images, and video-based sequence approaches. For single-view methods, we focus on depth estimation using Convolutional Neural Networks (CNNs) and image-to-depth mapping using Generative Adversarial Networks (GANs). For multi-view methods, we explore 3D reconstruction based on multi-view stereo matching method, 3D points cloud reconstruction method and stereo flow estimation method. For video-based methods, we introduce depth estimation method based on optical flow and video sequence modeling using Recurrent Neural Networks (RNNs). Generally, the multi-view image method is more accurate and sophisticated than the single-view image method, while the method based on video sequences is more challenging and complex. Different 3D reconstruction methods depend on specific application scenarios and requirements. This review provides a considerable insight in research for 3D reconstruction and also make the conclusion as well as future prospect for this field.