The Investigation on the Influence of Dimension Manipulation on Regret Performance Using Upper Confidence Bound Algorithm

Abstract

This paper explores the impact of dimensionality manipulation on recommendation algorithm performance amid the $5 trillion global e-commerce landscape, where traditional methods suffer from the curse of dimensionality-feature redundancy eroding efficiency and distorting recommendations. Focusing on the "exploration-exploitation" balance, we use Principal Component Analysis (PCA) for dimensionality reduction, combined with the Upper Confidence Bound (UCB) algorithm to quantify regret performance differences. Using an Amazon product dataset, PCA reduces 15 features to 7 principal components, retaining core variance while mitigating redundancy. Feature crossing generates interaction features (e.g., price-star rating products) to enrich the feature space. Experimental results show dimensionality reduction boosts computational efficiency but risks losing feature semantics, slightly degrading UCB’s exploitation accuracy. Dimensionality elevation, though increasing short-term exploration costs, enhances long-term performance by preserving critical feature correlations, outperforming reduced dimensions. This study highlights the trade-offs in dimensionality manipulation: reduction alleviates complexity but may distort information, while elevation enhances expressiveness at the cost of exploration. The findings offer a new approach to address dimensionality challenges in e-commerce recommendation systems, emphasizing the need to balance feature complexity and semantic integrity.