Fine - Tuning and Optimization of Live2D Facial Expression Recognition Based on Vision Transformer (ViT)

Abstract

With the development of fields such as virtual reality, the expressiveness of virtual character facial expressions has become increasingly crucial. Traditional CNN - based facial expression recognition methods have problems such as limited local feature extraction, sensitivity to pose changes, and high computational complexity. This research is based on the Vision Transformer (ViT) model, exploring its optimization and application in the facial expression recognition task, and combining with Live2D technology to achieve real - time and efficient expression conversion. ViT can capture global information through the self - attention mechanism, better model facial expression changes, has the potential for transfer learning, and is suitable for low - resource devices. The FER - 2013 dataset was used in this study. The ViT model was fine - tuned and quantized, and compared with traditional CNN models such as ResNet50. Experiments show that the optimized ViT model has higher accuracy and real - time performance in facial expression recognition, and can also operate efficiently in low - resource environments. The quantization technology also reduces the computational overhead, making it suitable for economical consumer software. This research enhances the application value of ViT in the field of affective computing, provides support for the development of Live2D technology, and promotes the wide application of virtual characters. In the future, the potential of ViT in multimodal emotion recognition will be explored, and its performance in complex scenarios will be optimized to provide a more comprehensive solution for intelligent interactions of virtual characters.