A Cross-Language Information Retrieval Method Based on Multi-Task Learning
DOI:
https://doi.org/10.57152/malcom.v4i3.1384Keywords:
Cross-Language Information Retrieval, External Corpus, Information Retrieval, Multi-Task Learning, Neural Retrieval ModelAbstract
This study introduces a novel Cross-Language Information Retrieval (CLIR) method employing multi-task learning and soft parameter sharing to enhance neural retrieval models' feature extraction across languages. The approach integrates an interaction-based neural retrieval model with a semantic-based text classification model, exchanging hidden vectors for richer feature representation. Experimental results across four language pairs—English-Chinese, English-Arabic, English-French, and English-German—demonstrate significant performance improvements. The proposed method achieved the highest Mean Average Precision (MAP) scores: 0.419 for EN-ZH, 0.403 for EN-AR, 0.427 for EN-FR, and 0.441 for EN-DE, surpassing other models like BM25, BPNRM, KNRM, KNRM-Trans, and KNRM-Embed. This research underscores the potential of multi-task learning for CLIR, showcasing improved retrieval performance through semantic information and knowledge transfer.
References
Internet World Statistics. (n.d.). Internet World Stats. Retrieved from https://www.internetworld
stats.com/.
Wang, K. F., & Huang, K. (2019). A Study on the Influencing Factors of User Retrieval Failure in Digital Libraries. Library Information Work, 63(11), 25-34.
Kishida, K. (2005). Technical Issues of Cross-Language Information Retrieval: A Review. Information Processing & Management, 41(3), 433-455.
Zhou, D., Zhao, W. Y., Wu, X., et al. (2017). Research on Result Re-Ranking in Personalized Cross-Language Information Retrieval. Computer Engineering and Science, 39(10), 1923-1929.
Nie, J. Y., Simard, M., & Isabelle, P. (1999). Cross-language information retrieval based on parallel texts and automatic mining of parallel texts from the web. In Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 74-81). ACM.
Xiong, C., Dai, Z., Callan, J., et al. (2017). End-to-end neural ad-hoc ranking with kernel pooling. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 55-64). ACM.
Pang, L., Lan, Y., Guo, J., et al. (2016). Text matching as image recognition. In Proceedings of the 13th AAAI Conference on Artificial Intelligence (Vol. 30, No. 1). AAAI Press.
Guo, J., Fan, Y., Ai, Q., et al. (2016). A deep relevance matching model for ad-hoc retrieval. In Proceedings of the 25th ACM International Conference on Information and Knowledge Management (pp. 55-64). ACM.
Yu, P., & Allan, J. (2020). A study of neural matching models for cross-lingual IR. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 1637-1640). ACM.
Liu, Q. (2018). A Neural Approach to Cross-Lingual Information Retrieval [Doctoral dissertation, Carnegie Mellon University].
Guo, J., Fan, Y., Pang, L., et al. (2020). A deep look into neural ranking models for information retrieval. Information Processing & Management, 57(6), 102067.
Pang, L., Lan, Y., Guo, J., et al. (2017). Deeprank: A new deep architecture for relevance ranking in information retrieval. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (pp. 257-266). ACM.
Pang, L., Lan, Y., Guo, J., et al. (2016). Text matching as image recognition. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 30, No. 1). AAAI Press.
Huang, P. S., He, X., Gao, J., et al. (2013). Learning deep structured semantic models for web search using clickthrough data. In Proceedings of the 22nd ACM International Conference on Information & Knowledge Management (pp. 2333-2338). ACM.
Wan, S., Lan, Y., Guo, J., et al. (2016). A deep architecture for semantic matching with multiple positional sentence representations. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (Vol. 30, No. 1). AAAI Press.
Ensan, F., & Bagheri, E. (2017). Document retrieval model through semantic linking. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining (pp. 181-190). ACM.
Koehn, P. (2005). Europarl: A parallel corpus for statistical machine translation. In Proceedings of Machine Translation Summit X (pp. 79-86).
Liu, P., Qiu, X., & Huang, X. (2016). Recurrent neural network for text classification with multi-task learning. In Proceedings of the 25th International Joint Conference on Artificial Intelligence (pp. 2873-2879). AAAI Press.
Liu, X., Gao, J., He, X., et al. (2015). Representation learning using multi-task deep neural networks for semantic classification and information retrieval. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 912-921). Association for Computational Linguistics.
Sun, T., Shao, Y., Li, X., et al. (2020). Learning sparse sharing architectures for multiple tasks. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 34, No. 05, pp. 8936-8943). AAAI Press.
Nishida, K., Saito, I., Otsuka, A., et al. (2018). Retrieve-and-read: Multi-task learning of information retrieval and reading comprehension. In _Proceedings of the 27th ACM International Conference on Information and Knowledge Management_ (pp. 647-656). ACM.
Ahmad, W. U., Chang, K. W., & Wang, H. (2018). Multi-task learning for document ranking and query suggestion. In _Proceedings of the 6th International Conference on Learning Representations_.
Thakkar, G., Preradovic, N. M., & Tadic, M. (2021). Multi-task learning for cross-lingual sentiment analysis. In _Proceedings of CLEOPATRA@ The Web Conference 2021_ (pp. 76-84). ACM.
Bonab, H., Sarwar, S. M., Allan, J. (2020). Training effective neural CLIR by bridging the translation gap. In _Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval_ (pp. 9-18). ACM.
Huo, Z. L., Wu, J. F., Lu, Y., et al. (2018). A topic-based cross-language retrieval model with PLSA and TF-IDF. In _Proceedings of the 3rd International Conference on Big Data Analysis_ (pp. 340-344). IEEE.
Devlin, J., Chang, M. W., Lee, K., et al. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
Devlin, J., Chang, M. W., Lee, K., et al. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. _arXiv preprint arXiv:1810.04805_.
Liu, T. Y. (2009). Learning to rank for information retrieval. Foundations and Trends in Information Retrieval, 3(3), 225-331.
Chapelle, O., & Chang, Y. (2010). Yahoo! Learning to Rank Challenge Overview. In Proceedings of the 2010 International Conference on Yahoo! Learning to Rank Challenge-Volume 14. MLR Press.
Chien, L. F. (1997). PAT-tree-based keyword extraction for Chinese information retrieval. In Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 50-58). ACM.
Zhang, K., Xiong, C., Liu, Z., et al. (2020). Selective weak supervision for neural information retrieval. In Proceedings of The Web Conference 2020. ACM.
Hu, B., Lu, Z., Li, H., et al. (2014). Convolutional neural network architectures for matching natural language sentences. In Proceedings of the 27th International Conference on Neural Information Processing Systems (pp. 2042-2050). NIPS.
Wan, S., Lan, Y., Xu, J., et al. (2016). Match-SRNN: Modeling the recursive matching structure with spatial RNN. arXiv preprint arXiv:1604.04378.
Wang, B., Liu, K., & Zhao, J. (2016). Inner attention based recurrent neural networks for answer selection. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (pp. 1288-1297). ACM.
Mikolov, T., Sutskever, I., Chen, K., et al. (2013). Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems (pp. 3111-3119). NIPS.
Mitra, B., & Craswell, N. (2018). An introduction to neural information retrieval. Boston, MA: Now Foundations and Trends.
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323(6088), 533-536.
Vandenhende, S., Georgoulis, S., Van Gansbeke, W., et al. (2020). Multi-Task Learning for Dense Prediction Tasks: A Survey. arXiv preprint arXiv:2004.13379.
Ruder, S. (2017). An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098.
