A Method for Ancient Book Named Entity Recognition Based on BERT-Global Pointer

Wen Jiang

doi:10.62051/ijcsit.v2n1.47

Authors

Wen Jiang

DOI:

https://doi.org/10.62051/ijcsit.v2n1.47

Keywords:

Ancient Texts of Twenty-Four Histories, Named Entity Recognition, Domain-Adaptive Pretraining, Model Fusion, Adversarial Training

Abstract

Correct identification of entities in ancient books and documents is the basic step of analyzing ancient Chinese texts, and provides an important prerequisite for in-depth mining of humanistic knowledge in ancient books and documents. In CCL2023 named entity recognition task of ancient books, according to the task definition and the re-quirements of the task organizer, this paper proposes the BERT Global Pointer named entity recognition model; Fine tune the field adaptation training based on the unlabeled 24 history ancient book text data; SWA, FGM, cross validation and post-processing are used to improve the recognition accuracy of the model. The experimental results show that the model and the strategy proposed in this paper have good recognition effect in the multi dynasties, cross domain ancient book entity recognition scene. F1 value on the final line reaches 95.083%.

Downloads

Download data is not yet available.

References

Zhou F, Wang C, Wang J. Named entity recognition of ancient poems based on Albert-BiLSTM-MHA-CRF model[J]. Wireless Communications and Mobile Computing, 2022, 2022. https://doi.org/10.1155/2022/6507719.

Song B, Bao Z, Wang Y Z, et al. Incorporating lexicon for named entity recognition of traditional Chinese medicine books[C]//Natural Language Processing and Chinese Computing: 9th CCF International Conference, NLPCC 2020, Zhengzhou, China, October 14–18, 2020, Proceedings, Part II 9. Springer International Publishing, 2020: 481-489. https://doi.org/10.1007/978-3-030-60457-8_39.

Yu P, Wang X. BERT-based named entity recognition in Chinese twenty-four histories[C]//International Conference on Web Information Systems and Applications. Cham: Springer International Publishing, 2020: 289-301. https://doi.org/10.1007/978-3-030-60029-7_27.

Su Q, Wang Y, Deng Z, et al. CCL23-Eval (GuNER2023)(Overview of CCL23-Eval Task 1: Named Entity Recognition in Ancient Chinese Books)[C]//Proceedings of the 22nd Chinese National Conference on Computational Linguistics (Volume 3: Evaluations). 2023: 34-40. https://aclanthology.org/2023.ccl-3.4.

Izmailov P, Podoprikhin D, Garipov T, et al. Averaging weights leads to wider optima and better generalization[J]. arXiv preprint arXiv:1803.05407, 2018.https://doi.org/10.48550/arXiv.1803.05407.

Goodfellow I J, Shlens J, Szegedy C. Explaining and harnessing adversarial examples[J]. arXiv preprint arXiv:1412.6572, 2014. https://doi.org/10.48550/arXiv.1412.6572.

Devlin J, Chang M W, Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018. https://doi.org/10.48550/arXiv.1810.04805.

Su J, Murtadha A, Pan S, et al. Global pointer: Novel efficient span-based approach for named entity recognition[J]. arXiv preprint arXiv:2208.03054, 2022. https://doi.org/10.48550/arXiv.2208.03054.

Madry A, Makelov A, Schmidt L, et al. Towards deep learning models resistant to adversarial attacks[J]. arXiv preprint arXiv:1706.06083, 2017. https://doi.org/10.48550/arXiv.1706.06083.

Wang P, Ren Z. The uncertainty-based retrieval framework for Ancient Chinese CWS and POS[J]. arXiv preprint arXiv:2310.08496, 2023. https://doi.org/10.48550/arXiv.2310.08496.

Wei J, Ren X, Li X, et al. Nezha: Neural contextualized representation for chinese language understanding[J]. arXiv preprint arXiv:1909.00204, 2019. https://doi.org/10.48550/arXiv.1909.00204.