Download Paper Title (use style: paper title)

International Journal of Innovations & Advancement in Computer Science
IJIACS
ISSN 2347 – 8616
Volume 3, Issue 9
November 2014
Study of Various Machine Learning Methods for Opinion
Mining and Sentiment Classification
Madhavi M. Kulkarni1, Mayuri Lingayat2
1. Student 2. Assistant Professor
Dept. Of CSE, G. H. Raisoni College of Engineering, Pune, Maharashtra, India
Abstract— Opinion mining is a latest study which is
the part of information retrieval that has been specific
by different conditions like subjectivity analysis, or
sentiment orientation, sentiment classification. It is the
process to determine the authors view about a specific
topic. There are several methods exists mostly employ
various machine learning technique and have varying
degree of effectiveness. Machine learning is the
technique which builds a system that can learn from
data somewhat follow any instructions. The goal of
Machine Learning is to grow an algorithm to enhance
the performance of the system using sample data or
previous experience. Based on desired outcome of the
algorithm, there are different types of machine
learning methods. In opinion mining it is used for
classifying the positive and negative reviews, topic
based classification. This paper mainly focuses on
various machine learning methods used for opinion
mining and sentiment classification.
Index Terms— Opinion mining, Machine learning, sentiment
classification
1.
INTRODUCTION
Opinion mining, or sentiment analysis, is the process of
determining the approach of a speaker with respect to a
topic. In the field of Computer Science, this generally
means automatically determining the attitude of given
input texts, often ranging in the order of thousands. It is the
field which is based on the reviews, feedbacks of the users
which are about the things based on their experience. Since
there is a rapid development in the web technology,
everyone is doing most of their tasks online. Online
registration, reservations, booking, banking, shopping are
some of the most famous tasks. So people can give their
reviews, feedbacks or opinions about such services.
Reviews could be positive, negative or neutral. These
22
Madhavi M. Kulkarni, Mayuri Lingayat
opinions are useful for the organizations or individual to
improve the performance of a service. So opinion mining
is the technique to extract the information about a
particular thing based on their reviews. Now a day it is
very popular area for doing research because web is the
emerging technology. So lots of information is needed to
be processed. Opinion mining is basically the area of
informational retrieval and text processing. Then
knowledge is needed to be extracted from such retrieved
information. Like here in opinion mining and sentiment
classification it is finding the polarity of words and then
doing classification, and finally putting the mined data all
together.
Machine learning is an effective way for classification of
an opinioned text. Machine learning is about developing an
algorithm that allows a computer to learn. Learning is the
process of finding the pattern or the consistencies in the
provided data. Thus goal of machine learning is to design
an algorithm based on the previous experience or the
provided sample data. Based on the desired outcome, there
are various types of machine learning algorithms. It is
explained in detail in section II. In sentiment classification
problem, it provides a solution which is supervised
machine learning approach where learning the model by
providing amount of training data and then based on the
trained model, the new data is going to classify. So
classification is the task where first data is preprocessed
and then pattern selection is done. Then in classification,
class label is assigned to each pattern. Here in opinion
mining the classification is done for finding the polarity of
words where class labels are positive, negative or neutral.
In literature there are several methods proposed which uses
different machine learning approaches and have changing
degree of usefulness.
In this paper various types of machine learning techniques
are explained in Section 2 and in Section 3 there is a brief
explanation about opinion mining and sentiment
International Journal of Innovations & Advancement in Computer Science
IJIACS
ISSN 2347 – 8616
Volume 3, Issue 9
November 2014
classification along with reference to previous work.
Proposed system is defined in Section 4.
2. MACHINE LEARNING TECHNIQUES
Machine learning is the study that generates an algorithm
to train the computer. Such system trains the machine by
available training data and then that trained system is used
to classify new data. So it is commonly used in data
mining for classification.
Based on the desired outcome of the system, machine
learning algorithms are divided into various types that
includes,





Supervised Learning: This is the types of
algorithm that creates a function which plots
given inputs to desired output. Classification
problem is the supervised learning task.
Unsupervised Learning: In this type labeled
examples are not available; it models a set of
inputs.
Semi-supervised Learning: This type of learning
combines both categorized and uncategorized
data to create an appropriate function.
Reinforcement learning: In this type of
algorithm, it learns from observation of the world
that how to act. Every act has some influence in
the environment, and the environment provides
reaction that directs the learning algorithm.
Transduction: It is like a supervised learning, but
rather than constructing a function explicitly, it
tries to predict new outputs based on provided
data.
Two general methods of building an ML-based classifier
exists in opinion mining. Supervised and unsupervised. So
in this paper, we mainly focused on supervised learning
and unsupervised learning types.
2.1 Supervised Learning
Supervised learning methods depend on the provided
training data. There are various types of supervised
classifiers in literature as probabilistic classifiers, linear
classifiers, decision tree classifiers and rule based
classifiers. Probabilistic classifiers are also called
generative classifiers. It assumes that there is a
combination and each class label is part of a combination.
It provides the probability of sampling a particular label
for that part. There are two mostly used probabilistic
classifiers are Naïve Bayes classifier (NB) and Maximum
Entropy classifier (ME) are discussed below. Linear
classifiers use separating hyperplane for classifications.
Linear classifier Support vector machine (SVM) and
Neural Network (NN) are discussed below. Decision tree
classifier and rule based classifier is also discussed below.
2.1.1
Naive Base classifier
It is proved to be simple and effective machine learning
method in previous classification studies. The main
assumption of the Naïve Base classifier is it computes
posterior probability of a class based on words
distribution in document. Bayes theorem is used for
predicting the posterior probability that a given feature set
belongs to a particular class label.
P (label/features) = P (label) * P (features/label)
P (features)
It is very easy to construct, not requiring any complicated
iterative parameter estimation schemes. So it can be freely
applied to enormous data sets. It is easy to understand, so
that users untrained in classifier method can realize the
process of classification. It always does unexpectedly
well.
Authors in [6] have used NBC because of its scalability.
They have used large data set as big data, so some
algorithms are not scale up. So in their proposed method
they evaluated scalability of NBC.
2.1.2
Fig 1: Machine learning techniques for sentiment
classification
23
Madhavi M. Kulkarni, Mayuri Lingayat
Maximum Entropy Classifier (ME)
The Maximum Entropy classifier is belongs to the class
exponential model of probabilistic classifier. Unlike the
Naive Bayes classifier, this classifier does not adopt that
features are conditionally independent of each other. It
follows to the Principle of Maximum Entropy that is it
chooses the model with maximum entropy among all the
models that are suitable for the training data. Various text
classification problems can be solved by this classifier. It
uses encoding that converts labeled feature sets into
vectors and then this encoded vector is used for
calculating weights of features. These features are then
International Journal of Innovations & Advancement in Computer Science
IJIACS
ISSN 2347 – 8616
Volume 3, Issue 9
November 2014
combined to find out most likely label for a feature set. In
general, the encoding maps each C{(featureset, label)}
pair to vector.
Where, in the training dataset p᷉(x) is the empirical
distribution of x. It often used when we don’t know
whatever about the prior distributions and when it’s
unsafe to put any assumptions. In text classification
problem, features are words which are dependent on each
other. That is there we cannot assume the conditional
independence of the different features. So Maximum
Entropy classifier is mostly used for text classification
problem.
2.1.3
Support Vector Machine (SVM)
Support vector machine is one of the most vigorous and
accurate method for classification among all well-known
machine algorithms. It has a complete theoretical base and
it needs only little samples for training. Also it is not
sensitive to the number of dimensions. There are various
effective methods developed for training SVM. The main
goal of SVM is to find the best function of classification
to distinguish between the two classes members in
training data. Both linear and nonlinear inseparable data
can be classified by SVM. Text data can also easily
classified by SVM. In linear classification problem, the
separating hyperplane is the classification function f(x,)
which passes between the middle of two classes. So once
the classification function is determined, new data can be
easily classified,
SVMs were used by Na Chen, Viktor K. Prasanna [8].
They used Linear Discriminant Analysis (LDA) as
learning to rank algorithm. Adaptive ranking module is
their core system. Akaichi, J. [7] have used SVM and
Naïve Bayes methods in their proposed system. They used
combination of these two methods because Naïve Bayes
algorithm for its simplicity and SVM for its effectiveness.
Authors in [3] used Support Vector Machine with Particle
Swarm Optimization (SVM-PSO) to classify opinions of
movie reviews from twitter data. They showed that
accuracy of their method is more than the method using
only SVM.
2.1.4
Neural network classifier
Neuron is the basic unit in neural network classifiers.
Neurons are arranged in layers which maps an input
vector into output vector. Each unit takes an input, applies
a function to it and then passes the output on to the next
layer. So it can make various classification tasks at once.
2.1.5
24
Decision tree classifiers
Madhavi M. Kulkarni, Mayuri Lingayat
Decision
tree
classifier
provides
hierarchical
decomposition of the training data. It applies some
condition on attribute value to divide the data. The
condition is to find the presence or absence of one or more
words. The process is done until the leaf node contain
some minimum number of values which are used for
classification.
Authors in [19] have used a new kind of decision tree as
opinion tree. They proposed hoe to build three level
flexible opinion trees. Authors in [15] have used tree
kernel approach for opinion mining of online products.
They presented that this method reduces the complexity in
features.
2.1.6
Rule based classifiers
Rule-based classifier makes use of set of IF-THEN rules
for classification. It can express the rule in the form as IF
condition THEN conclusion. To generate a rule there are
various criteria among which support and confidence are
most common. Complex decision trees can be problematic
to know sometimes. So rule set is the best alternative for
that which are of the form “if X and Y and Z and ... then
class A”, where rules for each class are gathered together.
A case is classified by finding the first rule whose
conditions are satisfied by the case; if no rule is satisfied,
the case is allotted to a default class.
2.2 Unsupervised learning
Unsupervised learning is the method where aim is to have
the machine that will do something without learning it the
task. In supervised learning, sometimes it is difficult to
create labeled documents. So it is easy to collect
unlabeled documents. Decision problems and clustering
are the most common types of problem in unsupervised
learning.
Wu, Xu and Li [14] used unsupervised learning algorithm
to find out low quality reviews. They used link analysis
algorithm for ranking. Authors in [9][15] used clustering
for grouping the customer’s reviews.
3.
OPINION MINING AND SENTIMENT
CLASSIFICATION
With the emergence of Web technology, customers can
freely write reviews about different entities, such as digital
products, services via various Web 2.0 platforms. Opinion
mining techniques extract, analyze and summarize the
opinions in a large number of reviews or feedbacks and
thus help users quickly summarize the opinions of interest.
Since people tend to be more interested in particular
features than the whole object, opinion mining techniques
International Journal of Innovations & Advancement in Computer Science
IJIACS
ISSN 2347 – 8616
Volume 3, Issue 9
November 2014
aiming at different features rather than the whole object are
especially attractive and have gained much attention in
recent years. Opinion mining is a latest study which is the
part of information retrieval and text mining that has been
specific by different conditions like subjectivity analysis,
or sentiment orientation, sentiment classification. It is the
process to determine the authors view about a specific
topic. Since there is a rapid development in web
technology; everyone is doing most of their tasks online.
Online registration, reservations, booking, banking,
shopping are some of the commonly used tasks. So people
can give their reviews, feedbacks or opinions about such
various online services. Reviews could be positive,
negative or neutral. Organizations and individuals are
mainly concerned to get opinion about the product,
services and event to improve the performance or for
making suitable choice. So in recent years opinion mining
is the important research field.
based on the social media data. In [5][7], authors used
Facebook data for sentiment classification. In [12], they
used twitter data stream for mining opinions. Authors in
[4][13] presented how the general social media and
opinion mining fields are connected.
4. PROPOSED SYSTEM
In the proposed work, it mainly focuses on online product
ranking based on the opinions about the product. In online
shopping, mostly people prefer reviews about that certain
product. That is it is useful for customers to take decision
about product based on the product reviews. But it is
difficult to see all the reviews about a particular product
and also the web data is unstructured, customer can get
various reviews which may not helpful to decide about
product. So it is necessary to design effective systems to
summarize the good and bad things of product
characteristics, so that customers can quickly find their
favorable products. For online product ranking using
opinion mining, there are various methods proposed in
literature which have their varying degree of effectiveness.
Still there are many challenges like improving accuracy of
algorithms for opinion detection, reduction of human
efforts needed to analyze contents, detection of spam and
fake reviews, cross platform opinion mining, multilingual
reference corpora etc. So in proposed solution, it is focused
on to overcome such challenges.
Fig 2: Machine learning in the context of opinion
mining
As shown in the above figure, it shows the sentiment
analysis process. After data collection, preprocessing step
is done where subjectivity and objectivity detection,
removal of negations and word sense disambiguation are
the operation performed. Then in feature selection is the
problem to extract and select features of text. Some
features are term presence and frequency, part of speech
(POS), opinion words and phrases, and negations. For
feature selection some frequently used methods are pointwise mutual information (PMI), Chi-square test and Latent
Semantic Indexing (LSI).
Authors in [3] proposed opinion mining method movie
reviews. Authors in [1][15][16][18][19][21][23] proposed
opinion mining method for web product ranking. Authors
in remaining references presented opinion mining methods
25
Madhavi M. Kulkarni, Mayuri Lingayat
Fig 3: Sentiment analysis process on product review
International Journal of Innovations & Advancement in Computer Science
IJIACS
ISSN 2347 – 8616
Volume 3, Issue 9
November 2014
The above figure shows the proposed method design. Data
collection is from reviews about a particular product
available on web. So all this data is unstructured and need
to be processed. So there are preprocessing steps like
filtering, data cleaning, and removing negations and like
that. There are features like term presence and frequency
which are individual words or n- grams, of words and its
frequency count, Part of Speech (POS) which is finding an
adjective, Opinion words and phrases which expresses an
opinions and feature negations which are the negative
words. For feature selection there are different methods
divided into lexicon based and statistical. Lexicon based
methods needs human interaction to create seed words
while statistical based methods are automatic. So in
proposed system it will follow later method. There are
again different methods in statistical approach some
frequently used methods are as Point-wise Mutual
Information (PMI), Chi-square and Latent Semantic
Indexing (LSI). In proposed system PMI will be the choice
of method for feature selection. It is a method to find the
mutual information between the feature and class.
In the above equation Mi(w) is the mutual information
between words w and class i. F(w). pi(w) is true cooccurrence between word and class while F(w). Pi is the
mutual independence. If
value of Mi(w) is greater than 0 then it is positively
correlated otherwise negatively correlated.
SVM machine learning technique is used as it is most
efficient method. On the other hand sensibleness of SVM
is affected due to the difficulty in selecting the SVM
parameters. So N-gram is the optimization technique and it
is very easy to apply. So the aim of this method is to
classify the opinions product review by using SVM with
N- gram.
5. CONCLUSION & FUTURE WORK
This survey paper presented an overview on machine
learning
methods used in opinion mining technique.
There are various methods used in literature are
summarized and categorized. After studying these articles,
it is clear that the enhancements of opinion mining and
sentiment analysis algorithms are still an open field for
research. Naive Bayes and Support Vector Machines are
the most commonly used machine learning algorithms for
solving sentiment classification problem. Information from
micro-blogs, blogs and forums as well as news source, is
widely used in sentiment analysis recently. This media
information plays a great role in expressing people’s
26
Madhavi M. Kulkarni, Mayuri Lingayat
feelings, or opinions about a certain topic or product. So in
this survey it is observed that using social network sites
and micro-blogging sites as a source of data still needs in
depth analysis. Proposed method is also presented that is
for opinion mining for online product reviews by using
SVM with N- gram. Future work could be committed to
this proposed method.
RFERENCES
[1] Yin-Fu Huang and Heng Lin, “Web Product
Ranking Using Opinion Mining.” IEEE
Conference on Computational Intelligence and
Data Mining (CIDM), 2013 IEEE Symposium on.
[2] Tanvir Ahmad and Mohammad NajmudDoja,
“Opinion Mining using Frequent Pattern Growth
Method
from
Unstructured
Text.”,
Computational and Business Intelligence
(ISCBI), 2013 International Symposium on IEEE
Conference Publications.
[3] Abd. Samad Hasan Basari, Burairah Hussin, I.
GedePramudyaAnanta and Junta Zeniarja,
“Opinion Mining of Movie Review using Hybrid
Method of Support Vector Machine and Particle
Swarm Optimization”, 1877-7058 © 2013 The
Authors. Published by Elsevier Ltd.
[4] Po-Wei Liang and Bi-Ru Dai, “Opinion Mining
on Social Media Data”, 2013 IEEE 14th
International Conference on Mobile Data
Management, 978-0-7695-4973-6/13 $26.00 ©
2013 IEEE.
[5] Jalel Akaichi, Zeineb Dhouioui and Maria José
López-Huertas Pérez, “Text Mining Facebook
Status Updates for Sentiment Classification”,
978-1-4799-2228-4/13/$31.00 ©2013 IEEE.
[6] Bingwei Liu, Erik Blasch, Yu Chen, Dan Shen
and Genshe Chen, “Scalable Sentiment
Classification for Big Data Analysis Using Naive
Bayes Classifier”, 2013 IEEE International
Conference on Big Data, 978-1-4799-12933/13/$31.00 ©2013 IEEE.
[7] Akaichi, J., “Social Networks' Facebook' Statutes
Updates Mining for Sentiment Classification”,
978-0-7695-5137-1/13
2013
IEEE
DOI
10.1109/SocialCom.2013.135.
[8] Na Chen and Viktor K. Prasanna, “Rankbox: An
Adaptive Ranking System for Mining Complex
Semantic Relationships Using User Feedback.”
IEEE IRI 2012, August 8-10, 2012, Las Vegas,
Nevada, USA.
[9] Darena, F., Zizka, J., Burda, K., “Grouping of
Customer Opinions Written in Natural Language
Using Unsupervised Machine Learning”,
Symbolic and Numeric Algorithms for Scientific
Computing (SYNASC), 2012 14th International
Symposium on DOI: 10.1109/SYNASC.2012.29
Publication Year: 2012.
International Journal of Innovations & Advancement in Computer Science
IJIACS
ISSN 2347 – 8616
Volume 3, Issue 9
November 2014
[10] Ningzhong, Yuefeng Li, and Sheng-Tang Wu,
“Effective Pattern Discovery for Text Mining”,
IEEE transactions on knowledge and data
engineering, vol. 24, no. 1, january 2012.
[11] Dominique Ziegelmayer and Rainer Schrader,
“Sentiment polarity classification using statistical
data compression models”, 2012 IEEE 12th
International Conference on Data Mining
Workshops, 978-0-7695-4925-5/12 $26.00 ©
2012 IEEE.
[12] Balakrishnangokulakrishnan,
Pavalanathan
Priyanthan
,
thiruchittampalamragavan
,
Nadarajah Prasath and ashehan Perera, “Opinion
Mining and Sentiment Analysis on a Twitter
Data Stream”, The International Conference on
Advances in ICT for Emerging Regions - icter
2012 : 182-188, 978-1-4673-5530-8/12/$31.00
©20 12 IEEE.
[13] Krzysztof Jędrzejewski, MikołajMorzy proposed
“Opinion Mining and Social Networks: a
Promising
Match”,
2011
International
Conference on Advances in Social Networks
Analysis and Mining IEEE.
[14] Hai-bing ma, Yi-Bing Geng and Jun-rui Qiu,
“Analysis of three methods for web-based
opinion mining”, 978-1-4577-0308-9/11/$26.00
©2011 IEEE.
[15] Jianwei Wu, Bing Xu and Sheng Li “An
Unsupervised Approach to Rank Product
Reviews”, 2011 Eighth International Conference
on Fuzzy Systems and Knowledge Discovery
(FSKD), 978-1-61284-181-6/11/$26.00 ©2011
IEEE.
[16] Peng Jiang, Chunxia Zhang, Hongping Fu,
ZhendongNiu, Qing Yang, “An Approach Based
on Tree Kernels for Opinion Mining of Online
Product Reviews”, 2010 IEEE International
Conference
on
Data
Mining.
27
Madhavi M. Kulkarni, Mayuri Lingayat
[17] Debnath Bhattacharyya, SusmitaBiswas, Taihoon Kim, “A Review on Natural Language
Processing in Opinion Mining”, International
Journal of Smart Home Vol.4, No.2, April, 2010.
[18] Weishu Hu, Zhiguo Gong and Jingzhi Guo, ”
Mining Product Features from Online Reviews”,
IEEE International Conference on E-Business
Engineering, 978-0-7695-4227-0/10 $26.00 ©
2010 IEEE.
[19] PeiliangTian, Yuanchao Liu, Ming Liu,
Shanzong Zhu, “Research Of Product Ranking
Technology Based On Opinion Mining”, 2009
Second International Conference on Intelligent
Computation Technology and Automation, IEEE.
[20] Juling Ding, Zhongjian Le, Ping Zhou, Gensheng
Wang, Wei Shu, “An Opinion-Tree based
Flexible Opinion Mining Model”, 2009
International Conference on Web Information
Systems and Mining IEEE.
[21] Jung-Yeon Yang, Jaeseok Myung and Sang-goo
Lee, “The Method for a Summarization of
Product Reviews Using the User’s Opinion”,
International Conference on Information,
Process, and Knowledge Management, 978-07695-3531-9/09 $25.00 © 2009 IEEE.
[22] Alexandra BALAHUR and Andrés MONTOYO,
“A Feature Dependent Method for Opinion
Mining and Classification”,978-1-4244-27802/08/$25.00 ©2008 IEEE.
[23] Jian Liu , Gengfeng Wu and Jianxin Yao,
“Opinion Searching in Multi-product Reviews”,
Proceedings of The Sixth IEEE International
Conference on Computer and Information
Technology (CIT'06) 0-7695-2687-X/06 $20.00
© 2006 IEEE.