DeGF Network -ABSA: Hybrid Approach with DeBERTa and Gated Fusion

Pir Mohammadiani, Rojiar; Hashemnia, Faezeh; Moradizadeh, Elham

doi:10.22044/jadm.2025.15593.2673

Articles in Press

Document Type : Original/Review Paper

Authors

¹ Computer Engineering Department, University of Sanandaj, Sanandaj, Iran.

² Cybersecurity Department, Tishk International University, Erbil, Iraq.

10.22044/jadm.2025.15593.2673

Abstract

Aspect-Based Sentiment Analysis (ABSA) has become a critical tool for extracting fine-grained insights from user opinions. This paper introduces DeGF-ABSA (DeBERTa-Gated Fusion for Aspect-Based Sentiment Analysis), a novel architecture that addresses key limitations in existing approaches by dynamically balancing global contextual features and aspect features. Unlike traditional methods that rigidly combine context and aspect representations—or transformer-based models lacking explicit mechanisms to disentangle aspect-specific signals—DeGF-ABSA leverages DeBERTa’s disentangled attention mechanism, which excels at modeling positional dependencies in technical texts, paired with a gated fusion layer. This layer adaptively weights the contributions of the context features that come from the [CLS] token, and the aspect-specific features come from the mean of aspect tokens. This helps in accurately determining the sentiment in complex sentences.

Experiments on SemEval 2014 datasets achieve state-of-the-art results: 86.68% accuracy (84.50% F1) for Laptops and 91.43% accuracy (86.83% F1) for Restaurants.Cross-domain generalization is critical for aspect-based sentiment analysis, as domain-specific aspects and vocabulary vary significantly. Sentiment expressions also differ across domains, such as 'delicious' for food reviews versus 'fast performance' for electronics, requiring adaptable models to capture contextual nuances. Evaluating the DeGF-ABSA model's performance on datasets from domains beyond laptops and restaurants would provide valuable insights into its ability to generalize and its potential for broader applicability.

Keywords

Main Subjects

H.8. Document and Text Processing

References

[1] D. Jayakody, A. V. A. Malkith, K. Isuranda, V. Thenuwara, N. de Silva, S. R. Ponnamperuma, G. G. N. Sandamali, and K. L. K. Sudheera, "Instruct-DeBERTa: A Hybrid Approach for Aspect-Based Sentiment Analysis on Textual Reviews," arXiv preprint arXiv:2408.13202, 2024.

[2] G. M. Shafiq, T. Hamza, M. F. Alrahmawy and R. El-Deeb, "Enhancing Arabic Aspect-Based Sentiment Analysis Using End-to-End Model," in IEEE Access, vol. 11, pp. 142062-142076, 2023.

[3] M. Pontiki, D. Galanis, J. Pavlopoulos, H. Papageorgiou, I. Androutsopoulos, and S. Manandhar, "SemEval-2014 Task 4: Aspect-Based Sentiment Analysis," in Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval, 2014), 2014.

[4] P. Chen, Z. Sun, L. Bing, and W. Yang, "Recurrent Attention Network on Memory for Aspect Sentiment Analysis," in Proc. EMNLP, 2017.

[5] Y. Song, J. Wang, T. Jiang, Z. Liu, and Y. Rao, "Attentional Encoder Network for Targeted Sentiment Classification," arXiv preprint arXiv:1902.09314, 2019.

[6] L. Li, Y. Liu, and A. Zhou, "Hierarchical Attention Based Position-Aware Network for Aspect-Level Sentiment Analysis," in Proc. CONLL, 2018.

[7] F. Fan, Y. Feng, and D. Zhao, "Multi-grained Attention Network for Aspect-Level Sentiment Classification," in Proc. EMNLP, 2018.

[8] J. Tang, Z. Lu, J. Su, Y. Ge, L. Song, L. Sun, and J. Luo, "Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis," in Proc. ACL, 2019.

[9] B. T. Do, "Aspect-Based Sentiment Analysis Using Bitmask Bidirectional Long Short Term Memory Networks," in Proc. FLAIRS-31, 2018.

[10] X. Li, L. Bing, W. Lam, and B. Shi, "Transformation Networks for Target-Oriented Sentiment Classification," in Proc. ACL, 2018.

[11] D. Tang, B. Qin, X. Feng, and T. Liu, "Effective LSTMs for Target-Dependent Sentiment Classification," in Proc. COLING, 2016.

[12] N. Majumder, S. Poria, A. Gelbukh, et al., "IARM: Inter-Aspect Relation Modeling with Memory Networks in Aspect-Based Sentiment Analysis," in Proc. EMNLP, 2018.

[13] P. Zhao, L. Hou, and O. Wu, "Modeling Sentiment Dependencies with Graph Convolutional Networks for Aspect-level Sentiment Classification," arXiv preprint, 2019.

[14] X. Bai, P. Liu, and Y. Zhang, "Investigating Typed Syntactic Dependencies for Targeted Sentiment Classification Using Graph Attention Neural Network," arXiv preprint, 2020.

[15] H. Xu, B. Liu, L. Shu, and P. S. Yu, "BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis," in Proc. NAACL, 2019.

[16] J. Dai, H. Yan, T. Sun, P. Liu, and X. Qiu, "Does Syntax Matter? A Strong Baseline for Aspect-based Sentiment Analysis with RoBERTa," in Proc. NAACL, 2021.

[17] A. Rietzler, S. Stabinger, P. Opitz, and S. Engl, "Adapt or Get Left Behind: Domain Adaptation through BERT Language Model Finetuning for Aspect-Target Sentiment Classification," in Proc. LREC, 2020.

[18] X. Cheng, W. Xu, T. Wang, and W. Chu, "Variational Semi-supervised Aspect-term Sentiment Analysis via Transformer," in Proc. CONLL, 2019.

[19] A. Karimi, L. Rossi, and A. Prati, "Improving BERT Performance for Aspect-Based Sentiment Analysis," in Proc. ICNLSP, 2021.

[20] R. He, W. S. Lee, H. T. Ng, and D. Dahlmeier, "An Interactive Multi-Task Learning Network for End-to-End Aspect-Based Sentiment Analysis," in Proc. ACL, 2019.

[21] H. Yang, B. Zeng, J. Yang, Y. Song, and R. Xu, "A Multi-task Learning Model for Chinese-oriented Aspect Polarity Classification and Aspect Term Extraction," arXiv preprint, 2019.

[22] Z. Li, Y. Wei, Y. Zhang, X. Zhang, X. Li, and Q. Yang, "Exploiting Coarse-to-Fine Task Transfer for Aspect-level Sentiment Classification," in Proc. AAAI, 2019.

[23] B. Xing and I. W. Tsang, "Understand me if you refer to Aspect Knowledge: Knowledge-aware Gated Recurrent Memory Network," arXiv preprint arXiv:2108.02105, 2021.

[24] S. Dehghan, R. Pir Mohammadiani, and S. Mohammadi,"The credibility assessment of Twitter/X users based on organization objectives by heterogeneous resources in big data life cycle", Computers in Human Behavior, Jan. 2025.

[25] S. Bastami, R. Pirmohamadiani, M. B. Dowlatshahi, and A. Abdollahpouri,"Enhanced high-dimensional data classification by combining fuzzy learning integration and graph transformers," Iranian Journal of Fuzzy Systems, vol. 22, no. 2, pp. 129-146, Apr. 2025.

[26] R. P. Mohammadiani and S. Mohammadi,"Criteria for evaluating influence value of social media users: A framework based on social media mining,"Iranian Communication and Information Technology, vol. 11, no. 3940, pp. 109-125, Jan. 2019.

[27] S. Dehghan, S. Mohammadi, and R. P. Mohammadiani,"The main components of evaluating the credibility of users according to organizational goals in the life cycle of big data,"Journal of Information and Communication Technology, vol. 55, no. 15, pp. 141-164, 2023.

[28] R. Pir Mohammadiani and Z. Malik,"The effects of interactivity of electronic word of mouth systems on value creation practices on social media,"in Proceedings of the AMA Winter Academic Conference, American Marketing Association, 2019.

Journal of AI and Data Mining

DeGF Network -ABSA: Hybrid Approach with DeBERTa and Gated Fusion

References

References

Articles in Press, Accepted Manuscript
Available Online from 12 November 2025

DeGF Network -ABSA: Hybrid Approach with DeBERTa and Gated Fusion

References

References

Articles in Press, Accepted Manuscript Available Online from 12 November 2025

Articles in Press, Accepted Manuscript
Available Online from 12 November 2025