Validity of data estimation methods in large-scale insurance datasets - Zhao, Jinrong; Lan, Daibin; Meng, Tengfei; Kan, Wanting (Transactions on Computer Science and Intelligent Systems Research, 2024)
In the insurance industry, accurately processing and analysing large-scale datasets is critical for risk assessment and decision-making. In this study, a comprehensive database was built by collecting and integrating weather-related data, insurance industry data, and attribute-specific data to support in-depth analyses of the impact of extreme weather events. In the data preprocessing stage, we adopted mean-filling and plurality-filling methods to deal with missing data, while applying the 3σ rule to deal with outliers in the data to ensure the completeness and consistency of the dataset. In addition, we used multi-criteria decision analysis methods (VIKOR) and hierarchical clustering models (BRICH algorithm). These advanced analysis techniques not only optimise the data processing process, but also improve the reliability and accuracy of the analysis results. Through these methods, we are able to effectively identify and classify different risk levels, which in turn provides a scientific basis for the pricing of insurance products and risk management. This study shows that advanced data estimation methods can provide effective and accurate support in processing large-scale insurance datasets, which is of great practical significance to the development of the modern insurance industry.