A Review of Traffic Scene Reconstruction Based on Images and Point Clouds

Xiaoning Dong

doi:10.62051/ijcsit.v3n1.13

Authors

Xiaoning Dong

DOI:

https://doi.org/10.62051/ijcsit.v3n1.13

Keywords:

Traffic scenarios, 3D reconstruction, Image point cloud fusion, Multi element data, Semantic segmentation

Abstract

This paper elaborates on a three-dimensional scene reconstruction method based on point clouds, images, and the fusion of images and point clouds. Relevant evaluation indicators are used to evaluate the performance of traffic scene reconstruction technology. The problems in traffic reconstruction under image and point cloud elements are analyzed and summarized. Finally, the challenges and future research directions in the field of traffic scene reconstruction based on images and point clouds are pointed out.

Downloads

Download data is not yet available.

References

Li Y, Zhu C, Liu Y, et al. Geometric and semantic analysis of road image sequences for traffic scene construction [J]. Neurocomputing, 2021, 365: 336-339.

Tao F, Zhang H, Liu A, et al. Digital Twin in Industry: State-of-the-art [J]. IEEE Transactions on Industrial Informatics (S0956-5515), 2018, 15(3): 2305-2315.

Gomez-Ojeda R, Briales J, Fernandez-Moral E, et al. Extrinsic calibration of a 2d laser-rangefinder and a camera based on scene corners [C], in : IEEE International Conference on Robotics and Automation. IEEE, 2015, 3611-3616.

Babahajiani P, Fan L, Kamarainen J K, et al. Comprehensive automated 3D urban environment modelling using terrestrial laser scanning point cloud [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2016: 10-18.

Wang J, Ye L, Gao R X, et al. Digital Twin for rotating machinery fault diagnosis in smart manufacturing [J]. International Journal of Production Research, 2019, 57(12): 3920-3933.

Negri E, Fumagalli L, Macchi M. A review of the roles of digital twin in CPS-based production systems [J]. Procedia manufacturing, 2017, 11: 939-938.

Cui Y, Chen R, Chu W, et al. Deep learning for image and point cloud fusion in autonomous driving: A review [J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 23(2): 722-739.

Zhou L, Li Z, Kaess M. Automatic extrinsic calibration of a camera and a 3D lidar using line and plane correspondences [C], in : IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2018, 5562-5569.

Wang S, Zhang F, Qin T. Research on the construction of highway traffic digital twin system based on 3D GIS technology [C]//Journal of Physics: Conference Series. IOP Publishing, 2021, 1802(3): 032035.

Lim H, Hwang S, Myung H. ERASOR: egocentric ratio of pseudo occupancy-based dynamic object removal for static 3D Point cloud map building [J]. IEEE Robotics and Automation Letters, 2021, 6(2):2272-2279.

Lim H, Oh M, Myung H. PatchWork: concentric zone-based region-wiseground segmentation with ground likelihood estimation using a 3D LiDAR sensor [J]. IEEE Robotics and Automation Letters, 2021, 6(3):6358-6365.

Schmohl S, Sörgel U. Submanifold sparse convolutional networks for semantic segmentation of large-scale ALS point clouds [J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2019, 3: 77-83.

WEN C, YANG L, LI X, et al. Directionally Constrained Fully Convolutional Neural Network for Airborne LiDAR Point Cloud Classification [J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020(162):50-6.

Arief H A, Indahl U G, Strand G H, et al. Addressing Overfitting on Point Cloud Classification Using Atrous XCRF [J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019(155):90-101.

WANG Y J, JIANG T P, LIU J, et al. Hierarchical Instance Recognition of Individual Roadside Trees in Environmentally Complex Urban Areas from UAV Laser Scanning Point Clouds [J]. ISPRS International Journal of Geo-information, 2020(9):595.

Lou Z, Gevers T, Hu N. Extracting 3D layout from a single image using global image structures [J]. IEEE Transactions on Image Processing, 2015, 24(10): 3098-3108.

Yuan J, Li Y, Pan H, et al. 3D traffic scenes construction and simulation based on scene stages [C]//2018 Chinese Automation Congress (CAC). IEEE, 2018: 1334-1339.

Zhu C, Li Y, Liu Y, et al. Road scene layout reconstruction based on CNN and its application in traffic simulation [C]//2019 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2019: 480-485.

Wang H, Gould S, Roller D. Discriminative learning with latent variables for cluttered indoor scene understanding [J]. Communications of the ACM, 2013, 56(4): 92-99.

Lee S, Huh S, Yoo D, et al. Rich feature hierarchies from omni-directional RGB-DI information for pedestrian detection [C]. 2015 12th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), 2015: 362-367.

Byeon W, Breuel T M, Raue F, et al. Scene labeling with lstm recurrent neural networks [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 3547-3555.

Cai H, Pang W, Chen X, et al. A novel calibration board and experiments for 3D lidar and camera calibration. Sensors [J], 2020, 20(4): 1130.

Liao Q, Chen Z, Liu Y, et al. Extrinsic calibration of lidar and camera with polygon [C], in: IEEE International Conference on Robotics and Biomimetics. IEEE, 2018, 200-205.

Ye Q, Shu L, Zhang W. Extrinsic calibration of a monocular camera and a single line scanning lidar [C], in : IEEE International Conference on Mechatronics and Automation. IEEE, 2019, 1047-1054.

Huang J K, Grizzle J W. Improvements to target-based 3D lidar to camera calibration. IEEE Access [J], 2020, 8: 134101-134110.

Dong W, Isler V. A novel method for the extrinsic calibration of a 2-D laser-rangefinder & a camera [C], in: IEEE International Conference on Robotics and Automation. IEEE, 2017, 5104-5109.

Gupta S, Girshick R, Arbeláez P, et al. Learning rich features from RGB-D images for object detection and segmentation [C]. European conference on computer vision, 2014: 345-360.

Badrinarayanan V, Kendall A, Cipolla R. Segnet: A deep convolutional encoder-decoder architecture for image segmentation [J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(12): 2481-2495.

Noh H, Hong S, Han B. Learning deconvolution network for semantic segmentation [C]. Proceedings of the IEEE international conference on computer vision, 2015: 1520-1528.

Zhao H, Shi J, Qi X, et al. Pyramid scene parsing network [C]. Proceedings of the IEEE conference on computer vision and pattern recognition, 2017: 2881-2890.

Zhao H, Qi X, Shen X, et al. Icnet for real-time semantic segmentation on high-resolutionimages [C]. Proceedings of the European Conference on Computer Vision (ECCV), 2018: 405-420.

Chen L-C, Papandreou G, Kokkinos I, et al. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs [J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 40(4): 834-848.

Dai J, Qi H, Xiong Y, et al. Deformable convolutional networks [C]. Proceedings of the IEEE international conference on computer vision, 2017: 764-773.

Ghiasi G, Fowlkes C C. Laplacian pyramid reconstruction and refinement for semantic segmentation [C]. European Conference on Computer Vision, 2016: 519-534.

Lin G, Milan A, Shen C, et al. Refinenet: Multi-path refinement networks for high-resolution semantic segmentation [C]. Proceedings of the IEEE conference on computer vision and pattern recognition, 2017: 1925-1934.

Li X, Liu Z, Luo P, et al. Not all pixels are equal: Difficulty-aware semantic segmentation via deep layer cascade [C]. Proceedings of the IEEE conference on computer vision and pattern recognition, 2017: 3193-3202.

Pinheiro P H, Collobert R. Recurrent convolutional neural networks for scene labeling [C]. 31st International Conference on Machine Learning (ICML), 2014: 925-934.

Shuai B, Zuo Z, Wang B, et al. Dag-recurrent neural networks for scene labeling [C]. Proceedings of the IEEE conference on computer vision and pattern recognition, 2016: 3620-3629.