The method proposed in this work is focused on analyzing sentiments efficiently, thus this segment comprises some recent studies categorized into three fields: Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), and Deep Learning-based SA methods. Towards the end of this section, the motivation and contribution of this research are also discussed.
Sentiment analysis (SA) techniques
This section focuses on the study of several recent works carried out in the sphere of sentiment analysis [6, 11, 15, 17, 22, 36, 38, 48, 51, 53].
The SentiDiff algorithm proposed by Wang et al. [51] combines textual information with sentiment diffusion patterns to enhance SA results on Twitter data. To analyze sentiment diffusion, the authors first investigated a phenomenon named sentiment reversal and found several exciting characteristics associated with sentiment reversals. Then they considered the associations among textual information in Twitter messages and diffusion patterns of sentiment for predicting sentiment polarities from Twitter messages. Extensive experiments on a real-world dataset show that the proposed approach improves the region beneath the Precision-Recall curve on the Twitter sentiment classification task when compared to state-of-the-art textual information-based SA techniques. Hao et al. [17] proposed a unique approach called CrossWord that handles cross-domain sentiment encoding problems using the stochastic word embedding technique. The proposed method provides an improved way of predicting probabilistic similarity associations between pivot words and the words in the source domain, in addition to labeled reviews in the source domain and unlabeled reviews in both domains.
SentiVec, a kernel optimization method for sentiment word embedding, is suggested by Zhu et al. [53]. The first phase of this study involves supervised learning, whereas the second phase involves unsupervised updating models like object-word-to-surrounding-word reward models (O2SR) and context-to-object-word reward models (C2OR). Experimental findings demonstrate that the optimum sentiment vectors effectively retrieve the features in terms of semantics and sentiment analysis, outperforming baseline approaches on tasks such as word analogy, similarity, and sentiment analysis. Bidirectional Encoder Representations from Transformers (BERT) model is employed to classify the public thoughts on Covid-19 by Singh et al. [48]. The authors performed sentiment analysis on two data sets in this paper: one data set contains tweets from individuals across the world, while the other data set comprises tweets from individuals in India. The validation accuracy of sentiment categorization is 94 percent, according to the experimental data.
A new method called the sentiment-based rating prediction method is suggested by Munuswamy et al. [36] to develop a recommendation system, which is proficient to mine useful knowledge from the user reviews posted on social media platforms to forecast the exact details adored by users based on their ratings. The opinions of users on an item are calculated using a sentiment dictionary in this model. Furthermore, item reputations are calculated using the three sentiments to anticipate and produce correct suggestions. For efficient classification of reviews posted on social media platforms, the n-gram methodology is included as a contemporary feature in semantic analysis and syntax, together with SVM, to boost the accuracy of the results.
A variety of classifiers and feature sets to perform sentiment quantification are explored by Ayyub et al. [6]. Based on the features set, an empirical performance assessment of traditional machine learning-based approaches, ensemble-based methods, and state-of-the-art deep learning-based methods is undertaken. The results reveal that different feature sets have an impact on classifier performance in sentiment quantification. The findings also demonstrate that deep learning methods outperform traditional machine learning algorithms. The analysis of 104 mental health apps on the App Store and Google Play with the help of five supervised machine learning techniques to perform sentiment classification on 88,125 user reviews is performed by Oyebode et al. [38] The top-performing classifier was then employed to forecast the sentiment polarity of reviews. Then, using the thematic assessment of negative and positive reviews, the authors found themes that represent numerous elements that influence the success of mental health apps in both positive and negative ways.
A unified framework that helps in bridging the gap between machine learning-based approaches and lexicon-based approaches is presented by Iqbal and Hashmi [15]. The authors developed a unique Genetic Algorithm-based feature reduction technique for addressing the scalability issue that occurs as the feature set develops. The authors successfully minimized the size of the feature set by 42% while maintaining accuracy by utilizing the proposed hybrid approach. Khan and Gul [22] used the Bag of Words (BoW) technique for obtaining features from movie reviews posted online & expressed those features as a vector. Then, the authors employed the Naive Bayes machine learning algorithm to categorize movie reviews expressed in the form of feature vectors into positive & negative classes. After that using the pairwise semantic similarities between categorized review sentences, an undirected weighted network was created, with the graph nodes representing review sentences and graph edges indicating semantic similarity weight. The absolute measure for all the review sentences in the graph was computed using the weighted graph-based ranking algorithm (WGRA). In the end, the extracted summary was created by selecting the top-rated sentences (graph nodes) built on the highest-ranking measures.
Chiong et al. [11] proposed 90 unique features that can be fed to a machine learning classifier to detect depression by analyzing users’ social media posts. The authors used a combination of sentiment lexicons and content-based approaches to extract these features. A comprehensive study of these features was conducted using two datasets of Twitter posts, four single classifiers, and four ensemble models. The authors found that all the proposed depression detection features performed best when used together, but the effectiveness of different feature groups ranged greatly. Additionally, the authors also found that ensemble models are more capable of overcoming data imbalance than single classifiers.
Aspect based sentiment analysis (ABSA) techniques
As a robust method for enhancing sentiment analysis accuracy, aspect terms extraction has attracted significant attention from researchers. This section discusses the numerous studies that have been recently suggested on ABSA [2, 3, 8, 21, 37, 41, 44, 46, 50].
Schouten et al. [44] presented a text-handling framework that can condense reviews. The subtask of this framework is to discover the general aspect types referred to in review sentences, for which the authors proposed two approaches in this paper. The first approach proposed is an unsupervised procedure that finds aspect categories by applying association rule mining to co-occurrence frequency data gathered from a corpus. With an F1-measure of 67 percent, the proposed unsupervised technique outperforms numerous simple baselines. The second technique proposed by the authors is a supervised variation that is found to perform better than the existing methods with an F1-measure of 84 percent. An ABSA hybrid approach to analyze the entities of smart app reviews integrating domain lexicons and rules is presented by Alqaryouti et al. [2]. In this method, language processing techniques, rules, and lexicons are utilized to address several challenges of sentiment analysis, and summarized results are produced. When implicit aspects are considered, the aspect extraction accuracy is found to improve dramatically.
Wang et al. [50] proposed a coarse alignment mechanism called Cross-Lingual Sentiment Classification to make an aspect-level fine-grained system from a group-to-group topic alignment. This unsupervised aspect, opinion, and sentiment unification model (AOS), trimodels aspects, opinions, and sentiments of reviews from various fields and uses a coarse alignment approach to help obtain more precise latent feature representation. To improve AOS even more, the authors offered ps-AOS, a partially supervised AOS model in which labeled source language data is used to reduce the difference in feature representations across two language domains via logistics regression. The results of massive experiments conducted on different multilingual product review datasets reveal that ps-AOS substantially outstrips several state-of-the-art methods. To optimize aspect-based sentiment analysis, one of the leading challenges of bipolar words in SA is addressed by Nandal et al. [37]. This research study examined the words changing polarity in the presence of context, its impact on the overall rating of the product, as well as the particular aspect, and came up with impressive results.
An automatic method is presented by Parthi et al. [41] to calculate sentiments of dynamic aspects from customer-generated reviews gathered through multiple sources using web scraping to deal with the cold start issue. By appending new stop words to the system, the authors improved the accuracy of the system. To address the issues of cost and time complexity, an unsupervised learning-centered approach for ABSA is recommended by Shams et al. [46]. The approach is based on three coarse-grained stages that are split into several fine-grained operations. The preliminary polarity lexicon and aspect word sets, as representations of aspects, are selected in the first stage to extract preceding domain knowledge from the dataset. As primitive information, these two resources are fed into an expectation–maximization algorithm, which determines the possibility of any word based on its aspect and emotion. To identify the polarity of any aspect in the last stage, the document is first split into its constituent aspects, and the possibility of each aspect/polarity is determined based on the document.
The end-to-end ABSA is investigated by Bie and Yang [8], and a unique multitask multiview network (MTMVN) model is proposed. The model prioritizes the unified ABSA as the primary job, with the two sub-chores: aspect term mining and aspect sentiment prediction as auxiliary chores. The authors considered the information gathered from the branch network of the central chore as the global view, and the information gathered from the two sub-chores as the two local views, each with diverse eminences. A multitasking approach enables crucial task completion through the incorporation of aspect boundary data and opinion polarity data. A Joint Aspect-level Sentiment Modification (JASM) model proposed by Jiang et al. [21] addresses the problem of reversing sentiment without affecting sentiments of other aspects within a sentence. With JASM, two coupled modules are jointly trained: aspect-specific sentiment word extraction and aspect-level sentiment transformation. In addition, the authors proposed a mechanism for learning aspect-aware sentiment representations and a method for dynamically selecting the next words based on aspect-aware sentiments or content information.
Aurangzeb et al. [3] presented a novel ABSA technique called Evolutionary Ensembler (EEn) to improve the diversity and accuracy of multi-label classifiers. The ABSA-based EEn highlighted multi-label-based models' accuracy and variety. To assess the accuracy and variety of learners using machine learning and to produce accurate and diverse forecasts, EEn dynamically optimizes two objective functions using an evolutionary multi-objective optimization technique.
Deep learning-based SA methods
Deep learning-based methods have also been applied to the sentiment analysis domain recently to further improve efficiency [1, 4, 5, 10, 12, 14, 20, 25, 26, 29, 30, 52].
The first end-to-end SEmi-supervised Multi-task Learning framework (SEML) for performing ABSA on user reviews is presented by Li et al. [26]. Both aspect mining and aspect sentiment classification in ABSA is learned together in a joint session. SEML also uses Moving-window Attentive Gated Recurrent Units (MAGRU) to create combined representations of reviews from three stacked and bidirectional neural layers. MAGRU expands GRU with the moving-window attention mechanism for obtaining substantial surrounding semantic contexts. The proposed model also uses Cross-View Training (CVT) for training auxiliary prediction modules on non-labeled reviews for enhancing representation learning.
The aspect-based sentiment analysis for demonetization tweets is performed using the improved deep learning method by Datta and Chakrabarti [12]. Pre-processing, extraction of aspects, polarity features, and sentiment categorization are all phases of the proposed model. To begin, the various demonetization tweets from the Kaggle data set are captured and pre-processed. Aspect extraction is used for the extraction of the sentiment words from the pre-processed data. With the support of polarity measure calculation and Word2vec, these retrieved aspect words are transformed into features. The authors optimized the polarity measures by integrating FireFly Algorithm (FF), and Multi-Verse Optimization (MVO), producing a novel algorithm called FireFly-oriented Multi-Verse Optimizer (FF-MVO). Recurrent Neural Network (RNN) is then applied to the combined features for the classification of sentiments as positive and negative.
Kumar et al. [25] suggested a method for sentiment analysis that combines three methods: developing ontologies for extracting semantic features, using Word2vec for transforming processed corpora, and creating convolutional neural networks (CNNs) for mining opinions. Particle swarm optimization is used for CNN parameter tuning to find non-dominant Pareto front optimal values. Using both ML and deep learning methods, an aspect-based sentiment analysis approach using polarity classification and sentiment extraction on reviews is recommended by Alamanda [1] to automatically extract the most interesting polarity aspects desired by customers. A search engine is being created to pull up tweets and reviews relating to a user-specified phrase and display the accompanying noteworthy aspects.
An aspect-gated graph convolutional network (AGGCN) is proposed by Lu et al. [30], which comprises a special aspect gate for guiding the encryption of aspect-specific information and employs a graph convolution network based on the sentence dependency tree for fully exploiting syntactical information and sentiment information. The model fails to address the issues related to noise and biases that get introduced during the encoding of aspect-specific information. Dragoni et al. [14] proposed an enhanced version of OntoSenticNet, a conceptual model for structuring emotions from multimodal resources, with SenticNet as a commonsense knowledge base for sentiment analysis. In addition to supporting the execution of semantic sentiment operations at reasoning time, the model also supports a conceptual model for sentiment dependencies and discovery paths.
Huang et al. [20] introduced a unique model for text sentiment recognition called Attention Emotion Enhanced (AEC)-LSTM, which seeks to improve the LSTM network by incorporating attention mechanism and emotional intelligence. First, an emotion-enhanced LSTM network is proposed, referred to as ELSTM, in which emotion modulation of learning is achieved using an emotion modulator and emotion estimator to enhance the feature learning ability of LSTM networks. The authors further integrated ELSTM with other operations, like convolution, pooling, and concatenation, to provide a better representation of various structure patterns in text sequences. Then, using the topic-level attention mechanism weight of hidden representations of text is adaptively adjusted. A Broad Multitask Transformer Network (BMT-Net) proposed by Zhang et al. [52] combines a feature-based approach with fine-tuning approach. The purpose of this system is to explore the high-level information of robust and contextual representations. The proposed structure enables global representations to be learned across tasks using multitask transformers. As a result of its ability to search for suitable features in deep and broad ways, BMT-Net can roundly learn the robust contextual representation that is used by the broad learning system.
Aygun et al. [5] employed ABSA technique to determine the attitude of Twitter users from the countries like Canada, France, Turkey, UK, USA, Spain, Italy, and Germany towards vaccination and vaccine types during the COVID-19 period. In this study, two datasets were prepared (English and Turkish language), containing 928,402 tweets related to vaccines. The authors used four different BERT models (mBERT-base, BioBERT, ClinicalBERT, and BERTurk) to classify tweets on four different aspects (policy, health, media, and others). Six different COVID-19 vaccines selected from the datasets were subjected to sentiment analysis using Twitter posts related to these vaccines. The authors obtained a total accuracy of 87% to present the views of the users on Covid-19 vaccination. The work proposed by Chandra et al. [10] compares selected translations of the Bhagavad Gita (mostly from Sanskrit to English) using semantic and sentiment analysis. The authors employed a hand-labeled sentiment dataset to fine-tune BERT model. The semantic analysis of selected chapters and verses across translations was also provided using novel sentence embedding models.
Based on tweets related to cryptocurrency, the work proposed by Aslam et al. [4] performs sentiment analysis as well as emotion detection for forecasting cryptocurrency market value. To increase the efficiency of the analysis, LSTM-GRU is proposed, a deep learning ensemble model that combines the features of two different RNN applications, including LSTM and Gated Recurrent Unit (GRU). Londhe et al. [29] suggested a novel method for ABSA utilizing the deep learning classifier LSTM-RNN. The hybrid LSTM-RNN is found to analyze and forecast the polarity of aspects with the highest degree of accuracy. Additionally, the major output is the correct extraction of multiple aspects from lengthy reviews with many sentences.
Motivation
Sentiment analysis has recently received substantial attention from researchers for a variety of applications, hence a framework to perform it efficiently while addressing scalability is needed. We investigated a variety of strategies employed to perform sentiment analysis, intended to improve the feature representation and classification using machine learning/deep learning-based techniques. The inclusion of fraudulent, sarcasm, spam, emoticons, negation and other types of content in a significant number of online reviews makes feature extraction challenging. The study carried out in this section shows that deep learning methods are more efficient than machine learning methods for sentiment classification. The accuracy of deep learning algorithms, we believe, is mostly determined by the feature set.
According to the findings of the studies mentioned above, sentiment analysis-based approaches are insufficient to handle the issues of accurately representing sentiments, opinions, and emotions, from online reviews. As a result, the ABSA approaches recently proposed are studied, however, their scope or application is also limited as they address the challenge of ambiguity and sarcasm to some extent, but don't consider emotions, negations, and long-term dependencies for accurate classification.
Aside from these issues, scalable review datasets have not been used to evaluate ABSA approaches. Some ABSA approaches relied primarily on unsupervised procedures, necessitating a significant number of manual data annotations. Because some SA/ABSA algorithms rely solely on symbolic feature extraction, their accuracy suffers. In comparison to machine learning and lexicon-based techniques, deep learning-based techniques provided superior scalability and efficiency.
Contribution
We suggest a novel technique of SA with special reference to consumer review summarization, based on the above-mentioned concerns of the state-of-the-art approaches. The suggested method comprises three stages: pre-processing, feature extraction, and sentiment classification. To denoise input reviews, the pre-processing step employs a standard procedure. The feature extraction step is ingeniously built as a hybrid step to overcome the difficulties of accurately representing the features of input reviews. In particular, the proposed model has the following contributions:
-
Based on our observations that hybrid feature sets accomplish effective results, we split the feature extraction phase into two parts: Review-Related Features (RRF) and Aspect-Related Features (ARF). After that, RRF and ARF numerical features are merged into a Hybrid Feature Vector (HFV).
-
As part of RRF, various methods like term frequency-inverse document frequency (TF-IDF), n-gram features & emoticon polarities, are employed for making emotional representations more accurate.
-
As part of ARF, we propose a novel way for extracting aspects terms employing co-occurrence frequencies and then assigning the polarities. The ARF method produces aspect terms with their polarities.
-
For the classification task, a deep learning classifier LSTM is used for the classification of input reviews either into negative, positive, or neutral classes.