Cross-domain aspect extraction for sentiment analysis
By : Flytxt R&D Team
In the backdrop of Industry 4.0 revolution, the digital channels have become a go-to place for customers for expressing opinions, sharing experiences and giving feedbacks. It becomes imperative for businesses to understand these customer opinions, adapt their offering to the market sentiments and enhance better experiences.
Sentiment analysis is a data mining process that measures the inclination of people’s opinions through Natural Language Processing (NLP) and machine learning. These opinions contain different aspects and their sentiments which belong to positive, negative or neutral categories. Thanks to the increase in the types and uses of social media (e.g., blogs, social posts, forums, and reviews), individuals are sharing their opinions across many domains. For example, some customers are sharing their reviews about a restaurant, while others are sharing their feedback about the laptop they bought. Now, to train the NLP models in a way that they can understand the sentiments belonging to different domains involves a huge computational cost and effort. This brings in the need to explore the possibility of domain agnostic machine learning approach for aspect based sentiment analysis.
What is Cross Domain ABSA?
Using sentiment analysis, we can decide whether a document, a sentence or an entity/phrase is having a tone that is positive, negative or neutral. The entities/phrases are called aspect terms and identifying the sentiment for each aspect term is called aspect based sentiment analysis (ABSA). The key technical challenge in sentiment analysis is that it is highly domain-dependent. In other words, a method that performs well in one domain might underperform in another. Cross domain aspect based sentiment analysis opens a new opportunity to link aspects and sentiments together across multiple domains. This results in a generic and domain-agnostic Natural Language Understanding (NLU) capability which in turn enables the models to analyze the sentiments of aspect terms belonging to different domains as well. This has been explained by an example in the next paragraph.
Aspect based sentiment analysis approaches are mainly divided into three categories, namely unsupervised, semi-supervised and supervised learning. In unsupervised learning, the labels of the training data are not available whereas in semi-supervised learning a minimal number of labeled training examples are required. In supervised learning, we need labeled training examples to build a model for aspect based sentiment analysis. This model does a good job in the same domain but might not work for other domains. It means that if we build a model using training data in one domain (e.g. Laptop domain) and test it on unseen data in other domain (e.g. Restaurant domain), the model may not perform well because the data distribution of two domains is not similar.
Cross domain aspect based sentiment analysis is achievable through domain adaptation. The general idea of domain adaptation is to use the knowledge of one domain where a sufficient labeled training data is available, known as the source domain, to transfer to other related new domain, known as target domain. This approach is very useful for aspect based sentiment analysis because the knowledge of one domain (e.g. Laptop domain) can be transferred to other new domain (e.g. Restaurant domain) without labeling the data in the new domain.
For example, let’s consider the review data of two domains, laptop and restaurant.
Laptop Data (source domain): it is of high quality with reasonable price, has a killer GUI, is extremely stable, is highly expandable, is bundled with lots of very good applications, and is easy to use.
Restaurant Data (target domain): Quality of food is excellent but the price is high.
Now, the source domain Laptop data is labeled with aspect terms like quality, price, GUI, etc. and their sentiments as positive, negative or neutral. The model built using the source domain labeled data will automatically identify the aspect terms and sentiments in the target domain like quality of food, price, etc. along with their sentiments.
This helps in saving a lot of time and effort since annotating data in the new domain from scratch is not required.
Flytxt’s approach for Cross Domain ABSA
We consider that domain adaptation is a classification task where x is input space and y = {0,1,2, …, L-1} is a set of ‘L’ possible labels. Here, the labels are positive, negative or neutral.
For aspect based sentiment analysis, we assume that the feature space x (input space) of source XS and target XT domains are same. In source domain XS, the labels are available but in target domain XT, labels are not available for all words. We represent each word with different features (x = x1, x2, …, xn where n is the number of features) which include semantic meaning, syntactic meaning, uppercase, word frequency, normalization, part-of-speech tags, special characters (#,@, etc.), stop words, etc. Then the feature input x is fed to a feed-forward neural network to represent joint features extractor F that maps an input sequence x to the shared feature space with multiple hidden layers. A sentiment label classifier C predicts the class label for x given the feature representation F(x). The sentiment label classifier output layer which is having a softmax function classifies aspect terms to different class labels (e.g. positive, negative or neutral). A discriminator D takes F(x) and predicts a scalar indicator whether x is coming from source XS or target XT. The discriminator D is having a binary classifier with a sigmoid layer on top which indicates input x is coming from source XS or target XT based on its hidden features F(x).
We consider that the domain adaptation is a process of learning representation (Ganin et al., 2015), so that the final classification decisions are made based on learning features that are very similar distributions in the source XS and target XT domains This is achieved by jointly optimizing the underlying features as well as a sentiment label classifier C and a discriminator D operating on the hidden features F(x): (i) the sentiment label classifier C that predicts class labels and (ii) the domain classifier D that discriminates between the source and the target domains during training. The sentiment label classifier C minimizes the training error whereas domain classifier D maximizes the domain classification loss for all samples in source and target domains.
Conclusion
With the explosive growth in the user-generated content, learning “what are people saying?” is now an extremely important piece of information. For businesses, identifying and analyzing customer reviews efficiently and precisely can help them to satisfy both current and potential customer needs. Our goal is to build a generic aspect based sentiment analysis model that works across domains. We are building an adversarial learning model which allows us to remove domain limitation and can be applied to any new domain for aspect based sentiment analysis. People can give any comments/reviews irrespective of the domain and our cross-domain aspect based sentiment analysis can identify aspect terms and their sentiments within the comments and reviews.
