H.3. Artificial Intelligence
Mohammad Aalishahi; Mohammad Hadi Bokaei; Abolfazl Nadi
Abstract
Given the importance of Named Entity Recognition (NER), numerous studies have been conducted in this field. However, most research has focused on languages such as English, French, and Arabic. In contrast, studies on Persian remain limited, despite Persian being one of the most widely spoken languages ...
Read More
Given the importance of Named Entity Recognition (NER), numerous studies have been conducted in this field. However, most research has focused on languages such as English, French, and Arabic. In contrast, studies on Persian remain limited, despite Persian being one of the most widely spoken languages in West Asia, necessitating the development of NER methods for it. In this study, using Active Learning, a corpus of 1,351 advertisements from the Official Gazette was annotated. The GEMMA2b model was then fine-tuned on this data, achieving approximately 95% accuracy. This model was employed to extract around 13 types of named entities and their relationships within the advertisement texts. The primary advantage of this method is the model’s high accuracy compared to other approaches. Additionally, the use of Persian data—which, unlike languages such as English or Arabic, has fewer resources—is another notable feature of this research.
H.3.8. Natural Language Processing
Arash Keshtkar; Saeedeh Sadat Sadidpour; Hossien Shirazi
Abstract
Word Sense Disambiguation (WSD) is a longstanding challenge in natural language processing, particularly in morphologically rich and low-resource languages such as Persian. The inherent ambiguity of Persian named entities exacerbated by domain-specific contexts and limited labeled data complicates both ...
Read More
Word Sense Disambiguation (WSD) is a longstanding challenge in natural language processing, particularly in morphologically rich and low-resource languages such as Persian. The inherent ambiguity of Persian named entities exacerbated by domain-specific contexts and limited labeled data complicates both semantic interpretation and information extraction. In this study, we introduce the PWNC corpus, a large-scale, integrated dataset designed for both Named Entity Recognition (NER) and WSD in Persian. The corpus was automatically constructed through a semi-supervised framework, incorporating contextual similarity measures and clustering algorithms to annotate ambiguous entities across ten semantic categories. Utilizing a semi-supervised framework, the proposed homograph semantic categorization method achieved robust performance, with a precision of 83%, recall of 81%, and an F1-score of 82% across over 305K annotated paragraphs. Detailed error analysis revealed challenges in disambiguating closely related senses and weak entities, which were mitigated through contextual embedding strategies. This work provides the first publicly available dual-task corpus for Persian NER and WSD, offering a scalable solution for disambiguation in low-resource tasks and laying the baseline for future research in Persian semantic processing.
M. Asgari-Bidhendi; B. Janfada; O. R. Roshani Talab; B. Minaei-Bidgoli
Abstract
Named Entity Recognition (NER) is one of the essential prerequisites for many natural language processing tasks. All public corpora for Persian named entity recognition, such as ParsNERCorp and ArmanPersoNERCorpus, are based on the Bijankhan corpus, which is originated from the Hamshahri newspaper in ...
Read More
Named Entity Recognition (NER) is one of the essential prerequisites for many natural language processing tasks. All public corpora for Persian named entity recognition, such as ParsNERCorp and ArmanPersoNERCorpus, are based on the Bijankhan corpus, which is originated from the Hamshahri newspaper in 2004. Correspondingly, most of the published named entity recognition models in Persian are specially tuned for the news data and are not flexible enough to be applied in different text categories, such as social media texts. This study introduces ParsNER-Social, a corpus for training named entity recognition models in the Persian language built from social media sources. This corpus consists of 205,373 tokens and their NER tags, crawled from social media contents, including 10 Telegram channels in 10 different categories. Furthermore, three supervised methods are introduced and trained based on the ParsNER-Social corpus: Two conditional random field models as baseline models and one state-of-the-art deep learning model with six different configurations are evaluated on the proposed dataset. The experiments show that the Mono-Lingual Persian models based on Bidirectional Encoder Representations from Transformers (MLBERT) outperform the other approaches on the ParsNER-Social corpus. Among different Configurations of MLBERT models, the ParsBERT+BERT-TokenClass model obtained an F1-score of 89.65%.
