The influence of network structure on different gradient descent optimization algorithms

Abstract

The gradient optimization algorithms and network architectures play pivotal roles in the field of artificial intelligence. However, there is limited research comparing multiple optimization algorithms across different network structures. In this paper, the effectiveness of several optimization techniques in image processing tasks is investigated, along with an investigation of their effects on different neural network architectures. LeNet, AlexNet, and the backpropagation neural network are the three popular neural network architectures included in the selection, along with three different optimization algorithms. An extensive assessment of training loss, test accuracy, convergence time, and other metrics was carried out to determine how well these algorithms worked in various network designs through rigorous experimentation on image datasets. The results show complex differences in the impact of optimization techniques across various neural network configurations, providing essential information for the choice of the best algorithms and network structures.