Categorizing Rules from the Expurgation Point of View using Large Language Models.

Deldar, Hassan; Homayounpour, Mohammad Mehdi

doi:10.22044/jadm.2026.16400.2763

Document Type : Applied Article

Authors

¹ Valiasr. Sharak ghods Qom

² amirkabir university

https://doi.org/10.22044/jadm.2026.16400.2763

Abstract

In most of the countries, the legislative process has a long history, which has led to increasing diversity and multiplicity of laws. This has made it difficult to access laws that are valid in both time and place. The focus of this article is on the application of artificial intelligence in the domain of legal statutes to assist in identifying the need for amendments to laws or specific provisions. The general framework of the proposed process consists of two key components.First, the texts of legal clauses or articles are enriched through the generation of enriched data using large language models, which involves producing embedding vectors, thematic classification,and extracting the provisions of each law. Second, a retrieval-augmented text generation (RAG) system is developed with the aid of large language models to determine conflicts or the need for expurgation in the output, utilizing the enriched data, predefined prompts, and the Chain of Thought (CoT) technique.The proposed method was evaluated on two benchmark datasets.On the COLIEE 2025 dataset, our approach outperformed the 2024 winners in legal implication tasks, achieving an F1 score of 0.6521 with minimal prompting. The second evaluation used over 1,000 legal clauses covering abrogation and neutral rules, yielding an impressive F1 score exceeding 73.41%.The findings of the proposed methodology demonstrate that, even with limited expertise in the legal domain, it is possible to identify conflicts and the necessity for refining legal texts to an acceptable degree within a reasonable timeframe for legal experts, leveraging the capabilities of large language models.

Keywords

Main Subjects

H.3.8. Natural Language Processing

References

[1] M. Alipour and A. Bahadori Jahromi, "Legal Feasibility Study of Crowdsourcing the Expurgation of Laws and Regulations, " In Persian p. https://www.sid.ir/paper/412080, 1401, [Online]. Available: https://www.sid.ir/paper/412080/fa.

[2] A. S. H. M. Mr. Mohammad Amin Keykha Farzaneh, "A Study of the Approach to the Expurgation of Laws and Regulations in the Iranian Legal System, " In Persian p . https://rc.majlis.ir/fa/report/show/1630975, 2010.

[3] M. M. Shora, "The Codification and Expurgation of Laws and Regulations of the Country, " In Persian p. https://rc.majlis.ir/fa/law/show/782398, 2010.

[4] S. Soltani, "Reasons for the Failure of the Codification and Expurgation of Laws in Iran, " In Persian 2014, [Online]. Available: https://civilica.com/doc/706648.

[5] H. Zhong, C. Xiao, C. Tu, T. Zhang, Z. Liu, and M. Sun, "How does NLP benefit legal system: A summary of legal artificial intelligence, " Proc. Annu. Meet. Assoc. Comput. Linguist., pp. 5218–5230, Apr. 2020, doi: 10.18653/v1/2020.acl-main.466.

[6] D. J. Langroodi, Essay on Legal Terminology. Ganj Danesh, In Persian, 1401.

[7] W. Hu et al., "BERT_LF: A Similar Case Retrieval Method Based on Legal Facts, " Wirel. Commun. Mob. Comput., vol. 2022, pp. 1–9, Apr. 2022, doi: 10.1155/2022/2511147.

[8] F. A. Naseri, A Collection of Criminal Laws and Regulations. Amir Kabir, In Persian, 1350.

[9] D. Chandrasekaran and V. Mago, "Evolution of Semantic Similarity—A Survey, " ACM Comput. Surv., vol. 54, no. 2, pp. 1–37, Mar. 2022, doi: 10.1145/3440755.

[10] A. Das and P. Rad, "Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey, " Jun. 2020, [Online]. Available: http://arxiv.org/abs/2006.11371.

[11] E. Taher, S. A. Hoseini, and M. Shamsfard, "Beheshti-NER: Persian Named Entity Recognition Using BERT, " Mar. 2020, [Online]. Available: http://arxiv.org/abs/2003.08875.

[12] I. Chalkidis, M. Fergadiotis, P. Malakasiotis, and I. Androutsopoulos, "Large-Scale Multi-Label Text Classification on EU Legislation, " arXiv Prepr. arXiv1906.02192, Jun. 2019, [Online]. Available: http://arxiv.org/abs/1906.02192.

[13] F. X. B. da Silva et al., "Named Entity Recognition Approaches Applied to Legal Document Segmentation, " in Anais do X Symposium on Knowledge Discovery, Mining and Learning (KDMiLe 2022), Sociedade Brasileira de Computação - SBC, Nov. 2022, pp. 210–217. doi: 10.5753/kdmile.2022.227949.

[14] C. Xiao, X. Hu, Z. Liu, C. Tu, and M. Sun, "Lawformer: A pre-trained language model for Chinese legal long documents, " AI Open, vol. 2, pp. 79–84, 2021, doi: 10.1016/j.aiopen.2021.06.003.

[15] S. Shaghaghian, Luna, Feng, B. Jafarpour, and N. Pogrebnyakov, "Customizing Contextualized Language Models forLegal Document Reviews, " Feb. 2021, [Online]. Available: http://arxiv.org/abs/2102.05757.

[16] E. Elwany, D. Moore, and G. Oberoi, "BERT Goes to Law School: Quantifying the Competitive Advantage of Access to Large Legal Corpora in Contract Understanding, " Nov. 2019, [Online]. Available: http://arxiv.org/abs/1911.00473.

[17] R. Z. Mahari, "AutoLAW: Augmented Legal Reasoning through Legal Precedent Prediction, " arXiv Prepr. arXiv2106.16034, Jun. 2021, doi: 10.48550/arXiv.2106.16034.

[18] I. Chalkidis, M. Fergadiotis, P. Malakasiotis, N. Aletras, and I. Androutsopoulos, "EGAL-BERT: The Muppets straight out of Law School, "” arXiv Prepr. arXiv2010.02559, Oct. 2020, [Online]. Available: http://arxiv.org/abs/2010.02559.

[19] Y. Shao et al., "BERT-PLI: Modeling Paragraph-Level Interactions for Legal Case Retrieval, " in Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, California: International Joint Conferences on Artificial Intelligence Organization, Jul. 2020, pp. 3501–3507. doi: 10.24963/ijcai.2020/484.

[20] M. Farahani, M. Gharachorloo, M. Farahani, and M. Manthouri, "ParsBERT: Transformer-based Model for Persian Language Understanding, " Neural Process. Lett., vol. 53, no. 6, pp. 3831–3847, Dec. 2021, doi: 10.1007/s11063-021-10528-4.

[21] M. S. Shahshahani, M. Mohseni, A. Shakery, and H. Faili, "PEYMA: A Tagged Corpus for Persian Named Entities, " Jan. 2018, [Online]. Available: http://arxiv.org/abs/1801.09936.

[22] H. Poostchi, E. Zare Borzeshi, and M. Piccardi, "BiLSTM-CRF for Persian Named-Entity Recognition ArmanPersoNERCorpus: the First Entity-Annotated Persian Dataset, " Eur. Lang. Resour. Assoc., 2018, [Online]. Available: https://aclanthology.org/L18-1701.

[23] V. Tran, M. Le Nguyen, and K. Satoh, "Automatic Catchphrase Extraction from Legal Case Documents via Scoring using Deep Neural Networks, " Sep. 2018, [Online]. Available: http://arxiv.org/abs/1809.05219.

[24] L. A. B. M. Gabriel M. C. Guimarães, Felipe X. B. da Silva, "Legal Document Segmentation and Labeling Through Named Entity Recognition Approaches, " J. Inf. Data Manag., 2024, doi: 10.5753/jidm.2024.3368.

[25] A. Jha, V. Rakesh, J. Chandrashekar, A. Samavedhi, and C. K. Reddy, "Supervised Contrastive Learning for Interpretable Long-Form Document Matching, " ACM Trans. Knowl. Discov. Data, vol. 17, no. 2, pp. 1–17, Apr. 2023, doi: 10.1145/3542822.

[26] R. Zhang et al., "Rapid Adaptation of BERT for Information Extraction on Domain-Specific Business Documents, " Feb. 2020, [Online]. Available: http://arxiv.org/abs/2002.01861.

[27] E. Mumcuoğlu, C. E. Öztürk, H. M. Ozaktas, and A. Koç, "Natural language processing in law: Prediction of outcomes in the higher courts of Turkey, " Inf. Process. Manag., vol. 58, no. 5, p. 102684, Sep. 2021, doi: 10.1016/j.ipm.2021.102684.

[28] V. K. Kommineni, B. König-Ries, and S. Samuel, "From human experts to machines: An LLM supported approach to ontology and knowledge graph construction, " Mar. 2024, [Online]. Available: http://arxiv.org/abs/2403.08345.

[29] H. B. Giglou, J. D’Souza, and S. Auer, "LLMs4OL: Large Language Models for Ontology Learning, " Jul. 2023, [Online]. Available: http://arxiv.org/abs/2307.16648.

[30] J. Wei et al., "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, " Jan. 2022, [Online]. Available: http://arxiv.org/abs/2201.11903.

[31] S. Yao et al., "Tree of Thoughts: Deliberate Problem Solving with Large Language Models, " May 2023, [Online]. Available: http://arxiv.org/abs/2305.10601.

[32] J.Long, "Large Language Model Guided Tree-of-Thought" May 2023, [Online]. Available: http://arxiv.org/abs/2305.08291.

[33] R.Goebel, Y. Kano, M.-Y. Kim, J. Rabelo, K. Satoh, ad M. Yoshioka, "Overview of Benchmark Datasets and Methods for the Legal Information Extraction/Entailment Competition (COLIEE) 2024, " 2024, pp. 109–124. doi: 10.1007/978-981-97-3076-6_8.

Journal of AI and Data Mining

Categorizing Rules from the Expurgation Point of View using Large Language Models.

References

References

Volume 14, Issue 3
July 2026
Pages 367-379

Categorizing Rules from the Expurgation Point of View using Large Language Models.

References

References

Volume 14, Issue 3July 2026Pages 367-379

Volume 14, Issue 3
July 2026
Pages 367-379