Deep Learning-Based Exploration of Text Meaning
DOI:
https://doi.org/10.62051/5ndjf655Keywords:
Recognizing of Textual Implications; Transformer-QRNN hybrid model; digital reasoning; multilingual scenarios; local dependency modeling.Abstract
In the field of Natural Language Processing (NLP), Recognizing of Textual Implications (RTE) belongs to one of the core tasks, aiming at recognizing semantic relations among texts. With the continuous improvement of related models, the application of Transformer has pushed the development of this field, but the existing models are still insufficient in numerical reasoning and multilingual scenarios. Therefore, this paper proposes and analyzes an improved hybrid transformer-QRNN model, exploring its potential to optimize the above shortcomings as well as its limitations in textual entailment tasks. Although this architecture outperforms traditional approaches on generalized datasets and significantly improves semantic analysis, it still faces shortcomings when dealing with scenarios involving numerical reasoning and multilingualism, and the structural complexity affects the efficiency when dealing with long texts. In the future, we need to optimize the numerical symbolization module, extend the multilingual support, and simplify the model structure. In addition, this hybrid architecture combining Transformer and QRNN proposed in this paper provides more efficient technical support for real-world scenarios such as legal contract contradiction detection and multilingual customer service systems.
Downloads
References
[1] S.R. Bowman, L. Vilnis, O. Vinyals, A.M. Dai, R. Jozefowicz, S. Bengio, Generating sentences from a continuous space, arXiv.org, 2016. https://arxiv.org/abs/1511.06349
[2] A.G. Gillioz, J. Casas, E.R. Mugellini, O.A. Khaled, Overview of the Transformer-based models for NLP tasks, IEEE Xplore, 2020. https://ieeexplore.ieee.org/abstract/document/9222960
[3] X. Zhu, Z. Yan, W. Wang, S. Hu, X. Wang, C. Liu, QRNN-Transformer: Recognizing textual entailment, 2024 IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2024, pp. 1–7. https://doi.org/10.1109/AVSS61716.2024.10672593
[4] A. Graves, Navdeep, A.R. Mohamed, Hybrid speech recognition with deep bidirectional LSTM, 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, 2013, pp. 273–278. https://doi.org/10.1109/ASRU.2013.6707742
[5] Y. Gong, H. Luo, J. Zhang, Natural language inference over interaction space, arXiv preprint, 2018. https://arxiv.org/abs/1709.04348
[6] M.E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, L. Zettlemoyer, Deep contextualized word representations, arXiv preprint, 2018. https://arxiv.org/abs/1802.05365
[7] A.K. Shahade, Multi-lingual opinion mining for social media discourses: An approach using deep learning-based hybrid fine-tuned Smith algorithm with Adam optimizer, Int. J. Inf. Manag. Data Insights 3(1) (2023) 100029. https://www.sciencedirect.com/science/article/pii/S2667096823000290
[8] N. Lan, E. Chemla, R. Katzir, Bridging the empirical-theoretical gap in neural network formal language learning using minimum description length, arXiv, 2024. https://arxiv.org/abs/2402.10013
[9] Research Gate, Unveiling the environmental implications of automatic text generation and the role of detection systems, n.d. https://www.researchgate.net/publication/373174176_Unveiling_the_Environmental_Implications_of_Automatic_Text_Generation_and_the_Role_of_Detection_Systems
[10] S. Raza, A. Omar, J. Alabi, J. Alistair, J.A. Name, T. Bolukbasi, Y. Cai, A. Caliskan, J.P. Chiu, H. Dawkins, S. Dev, L. Ding, T. Eftimov, … Y. Liu, Nbias: A natural language processing framework for bias identification in text, Expert Syst. Appl., 2023. https://www.sciencedirect.com/science/article/abs/pii/S0957417423020444?via%3Dihub
[11] Y. Zha, Y. Yang, R. Li, Z. Hu, Text alignment is an efficient unified model for massive NLP tasks, arXiv, 2023. https://arxiv.org/abs/2307.02729
[12] H. Ma, N. Afzal, H. Yang, G. Yu, S.A. Oliveira, R. Rajendran, Y.W. Chen, C.H. Lin, H. Wang, M. Xu, J. Liu, Z. Xu, S. Qu, Heart failure prognosis prediction: Let’s start with the MDL-HFP model, Inf. Syst., 2024. https://www.sciencedirect.com/science/article/abs/pii/S0306437924000668
[13] A. Paramasivam, A. Answering, A survey on textual entailment-based question answering, J. King Saud Univ. - Comput. Inf. Sci., 2021. https://www.sciencedirect.com/science/article/pii/S1319157821003311
[14] I.M.S. Putra, Recognizing textual entailment: A review of resources, approaches, applications, and challenges, ICT Express 9(3) (2023) 343–354. https://www.sciencedirect.com/science/article/pii/S2405959523001145
[15] V.S. Silva, A. Freitas, S. Handschuh, XTE: Explainable text entailment, arXiv, 2020. https://arxiv.org/abs/2009.12431
[16] A. Poliak, A survey on recognizing textual entailment as an NLP evaluation, arXiv, 2020. https://arxiv.org/abs/2010.03060
[17] M. Shajalal, Md. Atabuzzaman, M.B. Baby, Md.R. Karim, A. Boden, Textual entailment recognition with semantic features from empirical text representation, SpringerLink, n.d. https://link.springer.com/chapter/10.1007/978-3-031-33231-9_12
[18] I. Dagan, D. Roth, M. Sammons, Recognizing textual entailment: Models and applications, Springer, 2013.
[19] Research Square, A knowledge-based approach for recognizing textual entailments with a focus on causality and contradiction, 2024. https://www.researchsquare.com/article/rs-3826973/v1
Downloads
Published
Conference Proceedings Volume
Section
License
Copyright (c) 2025 Transactions on Computer Science and Intelligent Systems Research
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
How to Cite
