Categorization and Aspect Extraction of Online Shopping Comments Using Large Language Models

Ahmadlou, Mahdi; Daeichian, Abolghasem; Reihanian, Ali

doi:10.22044/jadm.2026.17595.2905

Document Type : Original/Review Paper

Authors

¹ Department of Computer Engineering, Sharif University, Tehran, Iran

² Department of Electrical Engineering, Faculty of Engineering, Arak University

³ Department of Computer Engineering, Faculty of Engineering, Arak University, Arak, 38156-8-8349, Iran

https://doi.org/10.22044/jadm.2026.17595.2905

Abstract

The expansion of e-commerce has changed customer purchasing habits, moving them from brick-and-mortar stores to online venues. In this shift, some fundamental customer behaviors had to change because online shoppers cannot physically feel the products and rely heavily on customer reviews for evaluations. However, the lack of structured textual data poses challenges in sifting through numerous, diverse, and sometimes contradictory comments to make an informed purchasing decision. This study proposes a multi-step system for autonomously and intelligently analyzing customer comments to organize comment sections, utilizing general-purpose large language models. First, the proposed system automatically separates comments that discuss online shop services from those specifically related to products by tagging them. Then, it extracts the various product aspects discussed across all comments. Finally, these comments are categorized based on the extracted aspects. Additionally, a new labeled non-English dataset has been created as a benchmark dataset featuring tagged online-shop-related comments. The experimental results showed that the best performing model was Qwen 2.5, achieving an accuracy of 91.7 %.

Keywords

Main Subjects

H.3. Artificial Intelligence

References

[1] M. Young, J. Soza‐Parra, and G. Circella, "The increase in online shopping during COVID‐19: Who is responsible, will it last, and what does it mean for cities?," Regional Science Policy & Practice, vol. 14, pp. 162–178, 2022.

[2] B. Rohrßen, "Digital Distribution: Online Sales and Online Platforms," in VBER 2022: EU Competition Law for Vertical Agreements: Digital, Dual, Exclusive and Selective Distribution plus Franchising: Springer, 2023, pp. 167–175.

[3] D. Holubinka, V. Vysotska, S. Vladov, Y. Ushenko, M. Talakh, and Y. Tomka, "Intelligent system for recognizing tone and categorizing text in media news at an electronic business based on sentiment and sarcasm analysis," International Journal of Information Engineering and Electronic Business (IJIEEB), vol. 17, no. 1, pp. 90–139, 2025.

[4] Bright-Local, "Local consumer review survey 2024," 2024. [Online]. Available: https://www.brightlocal.com/research/local-consumer-review-survey/

[5] N. Arizal, Nofrizal, W. Dwika Listihana, and Hadiyati, "Gen z customer loyalty in online shopping: an integrated model of trust, website design, and security," Journal of Internet Commerce, vol. 23, no. 2, pp. 121–143, 2024.

[6] Powerreviews, "Survey: The Ever-Growing Power of Reviews 2024," powerreviews, 2024. [Online]. Available: https://www.powerreviews.com/power-of-reviews-2023/

[7] B. Ganguly, P. Sengupta, and B. Biswas, "What are the significant determinants of helpfulness of online review? An exploration across product-types," Journal of Retailing and Consumer Services, vol. 78, p. 103748, 2024.

[8] Z. Wang, S.-j. Hu, S.-f. Niu, S.-y. Li, W.-d. Liu, and L.-y. Huang, "Research on product design improvement method based on online review and improvement importance performance competitor analysis," Expert Systems with Applications, vol. 279, p. 127400, 2025.

[9] J. Yi and Y. K. Oh, "The informational value of multi-attribute online consumer reviews: A text mining approach," Journal of Retailing and Consumer Services, vol. 65, p. 102519, 2022.

[10] D. Dalli, D. D'Acunto, and A. Tuan, "How online reviewers and actual customers evaluate their shopping experiences: evidence from an international retail chain," Mercati e competitività: 3, pp. 163–180, 2018.

[11] O. Coban, M. Yağanoğlu, and F. Bozkurt, "Domain effect investigation for BERT models fine-tuned on different text categorization tasks," Arabian Journal for Science and Engineering, vol. 49, no. 3, pp. 3685–3702, 2024.

[12] B. Pang, L. Lee, and S. Vaithyanathan, "Thumbs up? sentiment classification using machine learning techniques," in Proceedings of the 2002 conference on empirical methods in natural language processing (EMNLP 2002), pp. 79–86, 2002.

[13] M. Hu and B. Liu, "Mining and summarizing customer reviews," in the Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, Seattle, WA, USA, 2004.

[14] Y. Liu, J. Shi, F. Huang, J. Hou, and C. Zhang, "Unveiling consumer preferences in automotive reviews through aspect-based opinion generation," Journal of Retailing and Consumer Services, vol. 77, p. 103605, 2024.

[15] A. P. Richard Socher, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts, "Recursive deep models for semantic compositionality over a sentiment treebank," in the Proceedings of the 2013 conference on empirical methods in natural language processing, pp. 1631-1642, 2013.

[16] J. Feng, S. Cai, and X. Ma, "Enhanced sentiment labeling and implicit aspect identification by integration of deep convolution neural network and sequential algorithm," Cluster Computing, vol. 22, no. Suppl 3, pp. 5839–5857, 2019.

[17] D. Tang, B. Qin, and T. Liu, "Document modeling with gated recurrent neural network for sentiment classification," in Proceedings of the 2015 conference on empirical methods in natural language processing, 2015, pp. 1422–1432.

[18] S. Poria, E. Cambria, and A. Gelbukh, "Aspect extraction for opinion mining with a deep convolutional neural network," Knowledge-Based Systems, vol. 108, pp. 42–49, 2016.

[19] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding," in Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, vol. 1, pp. 4171–4186, 2019.

[20] D. Kirange, R. R. Deshmukh, and M. Kirange, "Aspect based sentiment analysis semeval-2014 task 4," Asian Journal of Computer Science and Information Technology (AJCSIT) Vol, vol. 4, 2014.

[21] L. Floridi and M. Chiriatti, "GPT-3: Its nature, scope, limits, and consequences," Minds and Machines, vol. 30, pp. 681–694, 2020.

[22] P. F. S. a. P. Huoviala, "Large language models for aspect-based sentiment analysis," arXiv preprint, arXiv:2310.18025, 2023.

[23] S. M. Al-Ghuribi, S. A. M. Noah, and S. Tiun, "Unsupervised semantic approach of aspect-based sentiment analysis for large-scale user reviews," IEEE Access, vol. 8, 2020.

[24] A. N. Karimvand, R. S. Chegeni, M. E. Basiri, and S. Nemati, "Sentiment analysis of persian instagram post: a multimodal deep learning approach," in 2021 7th International Conference on Web Research (ICWR), IEEE, pp. 137–141, 2021.

[25] S. Shumaly, M. Yazdinejad, and Y. Guo, "Persian sentiment analysis of an online store independent of pre-processing using convolutional neural network with fastText embeddings," PeerJ Computer Science, vol. 7, p. e422, 2021.

[26] K. Dashtipour, M. Gogate, A. Adeel, H. Larijani, and A. Hussain, "Sentiment analysis of persian movie reviews using deep learning," Entropy, vol. 23, no. 5, p. 596, 2021.

[27] M. Farahani, M. Gharachorloo, M. Farahani, and M. Manthouri, "Parsbert: Transformer-based model for persian language understanding," Neural Processing Letters, vol. 53, pp. 3831–3847, 2021.

[28] Ghasemi, Ali Reza, and Javad Salimi Sartakhti. "Multilingual Language Models in Persian NLP Tasks: A Performance‎ Comparison of Fine-Tuning Techniques." Journal of AI and Data Mining, vol. 13, no.1, pp. 107-117, 2025.

[29] H. Jafarian, A. H. Taghavi, A. Javaheri, and R. Rawassizadeh, "Exploiting BERT to improve aspect-based sentiment analysis performance on Persian language," in 2021 7th International Conference on Web Research (ICWR), IEEE, pp. 5–8, 2021.

[30] T. S. Ataei, K. Darvishi, S. Javdan, A. Pourdabiri, B. Minaei-Bidgoli, and M. T. Pilehvar, "Pars-off: a benchmark for offensive language detection on farsi social media," IEEE Transactions on Affective Computing, vol. 14, no. 4, pp. 2787–2795, 2022.

[31] G. Kontonatsios et al., "FABSA: An aspect-based sentiment analysis dataset of user reviews," Neurocomputing, vol. 562, p. 126867, 2023.

[32] A. Abaskohi et al., "Benchmarking large language models for Persian: A preliminary study focusing on ChatGPT," in Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), Torino, Italia, pp. 2189–2203, 2024.

Categorization and Aspect Extraction of Online Shopping Comments Using Large Language Models

References

References

Volume 14, Issue 3July 2026Pages 419-431

Volume 14, Issue 3
July 2026
Pages 419-431