Document Type : Original/Review Paper
Authors
Computer Engineering Department, University of Qom, Qom, Iran
Abstract
Text sentiment classification in aspect level is one of the hottest research topics in the field of natural language processing. The purpose of the aspect-level sentiment analysis is to determine the polarity of the text according to a particular aspect. Recently, various methods have been developed to determine sentiment polarity of the text at the aspect level, however, these studies have not yet been able to model well complementary effects of the context and aspect in the polarization detection process. Here, we present ACTSC, a method for determining the sentiment polarity of the text based on separate embedding of aspects and context. In the first step, ACTSC deals with separate modelling of the aspects and context to extract new representation vectors. Next, by combining generative representations of aspect and context, it determines the corresponding polarity to each particular aspect using a short-term memory network and a self-attention mechanism. Experimental results in the SemEval2014 dataset in both restaurant and laptop categories show that ACTSC has been able to improve the accuracy of aspect-based sentiment classification compared to the latest proposed methods.
Keywords
[1] A. Saxena, H. Reddy, and P. Saxena, "Introduction to Sentiment Analysis Covering Basics, Tools, Evaluation Metrics, Challenges, and Applications," in Principles of Social Networking: Springer, 2022, pp. 249-277.
[2] B. Torabian, "Sentiment classification with case-base approach," 2016.
[3] B. Liu, "Sentiment analysis and opinion mining," Synthesis lectures on human language technologies, vol. 5, no. 1, pp. 1-167, 2012.
[4] M. Dragoni and G. Petrucci, "A fuzzy-based strategy for multi-domain sentiment analysis," International Journal of Approximate Reasoning, vol. 93, pp. 59-73, 2018.
[5] M. Pontiki et al., "Semeval-2016 task 5: Aspect based sentiment analysis," in International workshop on semantic evaluation, 2016, pp. 19-30.
[6] B. Xing et al., "Earlier attention? aspect-aware LSTM for aspect-based sentiment analysis," arXiv preprint arXiv:1905.07719, 2019.
[7] D. Ma, S. Li, X. Zhang, and H. Wang, "Interactive attention networks for aspect-level sentiment classification," arXiv preprint arXiv:1709.00893, 2017.
[8] Y. Wang, M. Huang, X. Zhu, and L. Zhao, "Attention-based LSTM for aspect-level sentiment classification," in Proceedings of the 2016 conference on empirical methods in natural language processing, 2016, pp. 606-615.
[9] S. Gu, L. Zhang, Y. Hou, and Y. Song, "A position-aware bidirectional attention network for aspect-level sentiment analysis," in Proceedings of the 27th international conference on computational linguistics, 2018, pp. 774-784.
[10] Y. Song, J. Wang, T. Jiang, Z. Liu, and Y. Rao, "Attentional encoder network for targeted sentiment classification," arXiv preprint arXiv:1902.09314, 2019.
[11] A. Lakizadeh and Z. Zinaty, "A Novel Hierarchical Attention-based Method for Aspect-level Sentiment Classification," Journal of AI and data mining, vol. 9, no. 1, pp. 87-97, 2021.
[12] S. Vohra and J. Teraiya, "A comparative study of sentiment analysis techniques," Journal Jikrce, vol. 2, no. 2, pp. 313-317, 2013.
[13] B. Pang, L. Lee, and S. Vaithyanathan, "Thumbs up? Sentiment classification using machine learning techniques," arXiv preprint cs/0205070, 2002.
[14] Q. Ye, Z. Zhang, and R. Law, "Sentiment classification of online reviews to travel destinations by supervised machine learning approaches," Expert systems with applications, vol. 36, no. 3, pp. 6527-6535, 2009.
[15] Z. Zhang, Q. Ye, Z. Zhang, and Y. Li, "Sentiment classification of Internet restaurant reviews written in Cantonese," Expert Systems with Applications, vol. 38, no. 6, pp. 7674-7682, 2011.
[16] R. Prabowo and M. Thelwall, "Sentiment analysis: A combined approach," Journal of Informetrics, vol. 3, no. 2, pp. 143-157, 2009.
[17] E. Riloff, S. Patwardhan, and J. Wiebe, "Feature subsumption for opinion analysis," in Proceedings of the 2006 conference on empirical methods in natural language processing, 2006, pp. 440-448.
[18] C. Whitelaw, N. Garg, and S. Argamon, "Using appraisal groups for sentiment analysis," in Proceedings of the 14th ACM international conference on Information and knowledge management, 2005, pp. 625-631.
[19] W. Medhat, A. Hassan, and H. Korashy, "Sentiment analysis algorithms and applications: A survey," Ain Shams engineering journal, vol. 5, no. 4, pp. 1093-1113, 2014.
[20] P. D. Turney, "Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews," arXiv preprint cs/0212032, 2002.
[21] K. Church and P. Hanks, "Word association norms, mutual information and lexicography. I: ACL 27th Annual Meeting 76–83," Vancouver. Halvautomatisk ekserpering av anglisismer i norsk, vol. 85, 1989.
[22] M. Fernández-Gavilanes, T. Alvarez-López, J. Juncal-Martínez, E. Costa-Montenegro, and F. J. González-Castano, "Gti: An unsupervised approach for sentiment analysis in twitter," in Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), 2015, pp. 533-538.
[23] T. Zagibalov and J. A. Carroll, "Automatic seed word selection for unsupervised sentiment classification of Chinese text," in Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), 2008, pp. 1073-1080.
[24] D. W. Otter, J. R. Medina, and J. K. Kalita, "A survey of the usages of deep learning for natural language processing," IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 2, pp. 604-624, 2020.
[25] Y. Zhang, J. E. Meng, R. Venkatesan, N. Wang, and M. Pratama, "Sentiment classification using comprehensive attention recurrent models," in 2016 International joint conference on neural networks (IJCNN), 2016, pp. 1562-1569: IEEE.
[26] K. Vyas and D. Naik, "Language Model Fine-Tuning with Second-Order Optimizer," in Advances in Speech and Music Technology: Springer, 2021, pp. 355-366.
[27] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, "Distributed representations of words and phrases and their compositionality," in Advances in neural information processing systems, 2013, pp. 3111-3119.
[28] J. Pennington, R. Socher, and C. D. Manning, "Glove: Global vectors for word representation," in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 1532-1543.
[29] Y. LeCun, P. Haffner, L. Bottou, and Y. Bengio, "Object recognition with gradient-based learning," in Shape, contour and grouping in computer vision: Springer, 1999, pp. 319-345.
[30] Q. V. Le, W. Y. Zou, S. Y. Yeung, and A. Y. Ng, "Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis," in CVPR 2011, 2011, pp. 3361-3368: IEEE.
[31] J. J. Tompson, A. Jain, Y. LeCun, and C. Bregler, "Joint training of a convolutional network and a graphical model for human pose estimation," Advances in neural information processing systems, vol. 27, pp. 1799-1807, 2014.
[32] L. Deng, "Farewell editorial: Keeping up the momentum of innovations," vol. 22, ed: IEEE Press Piscataway, NJ, USA, 2014, pp. 1687-1687.
[33] Y. Kim, "Convolutional neural networks for sentence classification. arXiv 2014," arXiv preprint arXiv:1408.5882, 2019.
[34] M. D. Zeiler and R. Fergus, "Stochastic pooling for regularization of deep convolutional neural networks," arXiv preprint arXiv:1301.3557, 2013.
[35] K. He, X. Zhang, S. Ren, and J. Sun, "Spatial pyramid pooling in deep convolutional networks for visual recognition," IEEE transactions on pattern analysis and machine intelligence, vol. 37, no. 9, pp. 1904-1916, 2015.
[36] W. Ouyang et al., "Deepid-net: multi-stage and deformable deep convolutional neural networks for object detection," arXiv preprint arXiv:1409.3505, 2014.
[37] Y. Guo, Y. Liu, A. Oerlemans, S. Lao, S. Wu, and M. S. Lew, "Deep learning for visual understanding: A review," Neurocomputing, vol. 187, pp. 27-48, 2016.
[38] Z. C. Lipton, J. Berkowitz, and C. Elkan, "A critical review of recurrent neural networks for sequence learning," arXiv preprint arXiv:1506.00019, 2015.
[39] S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997.
[40] M. Dragoni and G. Petrucci, "A neural word embeddings approach for multi-domain sentiment analysis," IEEE Transactions on Affective Computing, vol. 8, no. 4, pp. 457-470, 2017.
[41] D. Bahdanau, K. Cho, and Y. Bengio, "Neural machine translation by jointly learning to align and translate," arXiv preprint arXiv:1409.0473, 2014.
[42] J. Li, M.-T. Luong, D. Jurafsky, and E. Hovy, "When are tree structures necessary for deep learning of representations?," arXiv preprint arXiv:1503.00185, 2015.
[43] L. Stappen, A. Baird, E. Cambria, and B. W. Schuller, "Sentiment analysis and topic recognition in video transcriptions," IEEE Intelligent Systems, vol. 36, no. 2, pp. 88-95, 2021.
[44] M. Rahman, Y. Watanobe, and K. Nakamura, "A Bidirectional LSTM Language Model for Code Evaluation and Repair," Symmetry, vol. 13, no. 2, p. 247, 2021.
[45] K. Dashtipour, M. Gogate, E. Cambria, and A. Hussain, "A novel context-aware multimodal framework for persian sentiment analysis," arXiv preprint arXiv:2103.02636, 2021.
[46] E. Cambria, Y. Li, F. Z. Xing, S. Poria, and K. Kwok, "SenticNet 6: Ensemble application of symbolic and subsymbolic AI for sentiment analysis," in Proceedings of the 29th ACM international conference on information & knowledge management, 2020, pp. 105-114.
[47] K. Dashtipour, M. Gogate, J. Li, F. Jiang, B. Kong, and A. Hussain, "A hybrid Persian sentiment analysis framework: Integrating dependency grammar based rules and deep neural networks," Neurocomputing, vol. 380, pp. 1-10, 2020.
[48] B. Huang, Y. Ou, and K. M. Carley, "Aspect level sentiment classification with attention-over-attention neural networks," in International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation, 2018, pp. 197-206: Springer.
[49] D. Tang, B. Qin, X. Feng, and T. Liu, "Effective LSTMs for target-dependent sentiment classification," arXiv preprint arXiv:1512.01100, 2015.
[50] K. Wang, W. Shen, Y. Yang, X. Quan, and R. Wang, "Relational graph attention network for aspect-based sentiment analysis," arXiv preprint arXiv:2004.12362, 2020.
[51] C. Yang, H. Zhang, B. Jiang, and K. Li, "Aspect-based sentiment analysis with alternating coattention networks," Information Processing & Management, vol. 56, no. 3, pp. 463-478, 2019.
[52] D. Meškelė and F. Frasincar, "ALDONAr: A hybrid solution for sentence-level aspect-based sentiment analysis using a lexicalized domain ontology and a regularized neural attention model," Information Processing & Management, vol. 57, no. 3, p. 102211, 2020.
[53] S. S. Sharma and G. Dutta, "SentiDraw: Using star ratings of reviews to develop domain specific sentiment lexicon for polarity determination," Information Processing & Management, vol. 58, no. 1, p. 102412, 2021.
[54] I. A. Farha and W. Magdy, "A comparative study of effective approaches for arabic sentiment analysis," Information Processing & Management, vol. 58, no. 2, p. 102438, 2021.
[55] Y. Tay, L. A. Tuan, and S. C. Hui, "Dyadic memory networks for aspect-based sentiment analysis," in Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, 2017, pp. 107-116.
[56] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding," arXiv preprint arXiv:1810.04805, 2018.
[57] C. Olah, "Understanding LSTM Networks.[(accessed on 25 February 2021)]," ed, 2015.
[58] M. Pontiki, D. Galanis, H. Papageorgiou, S. Manandhar, and I. Androutsopoulos, "Semeval-2015 task 12: Aspect based sentiment analysis," in Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), 2015, pp. 486-495.
[59] M. Al-Smadi, O. Qawasmeh, M. Al-Ayyoub, Y. Jararweh, and B. Gupta, "Deep Recurrent neural network vs. support vector machine for aspect-based sentiment analysis of Arabic hotels’ reviews," Journal of computational science, vol. 27, pp. 386-393, 2018.
[60] K. S. Tai, R. Socher, and C. D. Manning, "Improved semantic representations from tree-structured long short-term memory networks," arXiv preprint arXiv:1503.00075, 2015.
[61] Y. Tay, L. A. Tuan, and S. C. Hui, "Learning to attend via word-aspect associative fusion for aspect-based sentiment analysis," in Proceedings of the AAAI conference on artificial intelligence, 2018, vol. 32, no. 1.
[62] O. Habimana, Y. Li, R. Li, X. Gu, and G. Yu, "Sentiment analysis using deep learning approaches: an overview," Science China Information Sciences, vol. 63, no. 1, pp. 1-36, 2020.
[63] J. Zhou, J. X. Huang, Q. Chen, Q. V. Hu, T. Wang, and L. He, "Deep learning for aspect-level sentiment classification: survey, vision, and challenges," IEEE access, vol. 7, pp. 78454-78483, 2019.
[64] S. Zheng and R. Xia, "Left-center-right separated neural network for aspect-based sentiment analysis with rotatory attention," arXiv preprint arXiv:1802.00892, 2018.
[65] W. Xue and T. Li, "Aspect based sentiment analysis with gated convolutional networks," arXiv preprint arXiv:1805.07043, 2018.
[66] Q. Liu, H. Zhang, Y. Zeng, Z. Huang, and Z. Wu, "Content attention model for aspect based sentiment analysis," in Proceedings of the 2018 World Wide Web Conference, 2018, pp. 1023-1032.
[67] X. Wang, G. Xu, J. Zhang, X. Sun, L. Wang, and T. Huang, "Syntax-directed hybrid attention network for aspect-level sentiment analysis," IEEE Access, vol. 7, pp. 5014-5025, 2018.
[68] J. Liu and Y. Zhang, "Attention modeling for targeted sentiment," in Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, 2017, pp. 572-577.
[69] Z. Lin et al., "A structured self-attentive sentence embedding," arXiv preprint arXiv:1703.03130, 2017.
[70] Y. Song, J. Wang, Z. Liang, Z. Liu, and T. Jiang, "Utilizing BERT intermediate layers for aspect based sentiment analysis and natural language inference," arXiv preprint arXiv:2002.04815, 2020.
[71] F. Fan, Y. Feng, and D. Zhao, "Multi-grained attention network for aspect-level sentiment classification," in Proceedings of the 2018 conference on empirical methods in natural language processing, 2018, pp. 3433-3442.
[72] A. Kumar, V. T. Narapareddy, V. A. Srikanth, L. B. M. Neti, and A. Malapati, "Aspect-based sentiment classification using interactive gated convolutional network," IEEE Access, vol. 8, pp. 22445-22453, 2020.
[73] J. Liu, P. Liu, Z. Zhu, X. Li, and G. Xu, "Graph Convolutional Networks with Bidirectional Attention for Aspect-Based Sentiment Classification," Applied Sciences, vol. 11, no. 4, p. 1528, 2021.
[74] J. Cheng and H. Yi, "Aspect-Based Sentiment Analysis Based on Multi-Channel and Dynamic Weight," in IOP Conference Series: Earth and Environmental Science, 2021, vol. 693, no. 1, p. 012065: IOP Publishing.
[75] J. Wang et al., "Aspect Sentiment Classification with both Word-level and Clause-level Attention Networks," in IJCAI, 2018, vol. 2018, pp. 4439-4445.
[76] R. He, W. S. Lee, H. T. Ng, and D. Dahlmeier, "Effective attention modeling for aspect-level sentiment classification," in Proceedings of the 27th international conference on computational linguistics, 2018, pp. 1121-1131.
[77] M. Hu et al., "Can: Constrained attention networks for multi-aspect sentiment analysis," arXiv preprint arXiv:1812.10735, 2018.
[78] A. Essebbar, B. Kane, O. Guinaudeau, V. Chiesa, I. Quénel, and S. Chau, "Aspect Based Sentiment Analysis using French Pre-Trained Models," in ICAART (1), 2021, pp. 519-525.
[79] H. Le et al., "Flaubert: Unsupervised language model pre-training for french," arXiv preprint arXiv:1912.05372, 2019.
[80] C. Sindhu and G. Vadivu, "Fine grained sentiment polarity classification using augmented knowledge sequence-attention mechanism," Microprocessors and Microsystems, vol. 81, p. 103365, 2021.
[81] P. Chen, Z. Sun, L. Bing, and W. Yang, "Recurrent attention network on memory for aspect sentiment analysis," in Proceedings of the 2017 conference on empirical methods in natural language processing, 2017, pp. 452-461.
[82] Y. LeCun and Y. Bengio, "Convolutional networks for images, speech, and time series," The handbook of brain theory and neural networks, vol. 3361, no. 10, p. 1995, 1995.
)