A novel Data and Model Centric artificial intelligence based approach in developing high-performance Named Entity Recognition for Bengali Language.

Journal: PloS one
Published Date:

Abstract

Named Entity Recognition (NER) plays a significant role in enhancing the performance of all types of domain specific applications in Natural Language Processing (NLP). According to the type of application, the goal of NER is to identify target entities based on the context of other existing entities in a sentence. Numerous architectures have demonstrated good performance for high-resource languages such as English and Chinese NER. However, currently existing NER models for Bengali could not achieve reliable accuracy due to morphological richness of Bengali and limited availability of resources. This work integrates both Data and Model Centric AI concepts to achieve a state-of-the-art performance. A unique dataset was created for this study demonstrating the impact of a good quality dataset on accuracy. We proposed a method for developing a high quality NER dataset for any language. We have used our dataset to evaluate the performance of various Deep Learning models. A hybrid model performed with the exact match F1 score of 87.50%, partial match F1 score of 92.31%, and micro F1 score of 98.32%. Our proposed model reduces the need for feature engineering and utilizes minimal resources.

Authors

  • Khadija Akter Lima
    Department of Computer Science and Engineering, Daffodil International University, Dhaka, Bangladesh.
  • Khan Md Hasib
    Department of Computer Science and Engineering, Bangladesh University of Business and Technology, Dhaka, Bangladesh.
  • Sami Azam
    College of Engineering IT and Environment, Charles Darwin University, Casuarina, NT, Australia.
  • Asif Karim
    College of Engineering IT and Environment, Charles Darwin University, Casuarina, NT, Australia.
  • Sidratul Montaha
    Department of Computer Science and Engineering, Daffodil International University, Dhaka, Bangladesh.
  • Sheak Rashed Haider Noori
    Department of Computer Science and Engineering, Daffodil International University, Dhaka, Bangladesh.
  • Mirjam Jonkman
    College of Engineering, IT and Environment, Charles Darwin University, Casuarina, NT, Australia.