Download DASA: Dissatisfaction-oriented Advertising based on

Expert Systems with Applications 37 (2010) 6182–6191
Contents lists available at ScienceDirect
Expert Systems with Applications
journal homepage: www.elsevier.com/locate/eswa
DASA: Dissatisfaction-oriented Advertising based on Sentiment Analysis q
Guang Qiu a,*, Xiaofei He b, Feng Zhang a, Yuan Shi a, Jiajun Bu a, Chun Chen a
a
b
Zhejiang Key Laboratory of Service Robot, College of Computer Science, Zhejiang University, Hangzhou 310027, China
College of Computer Science, Zhejiang University, Hangzhou 310027, China
a r t i c l e
i n f o
Keywords:
Contextual advertising
Advertising keyword extraction
Ad selection
Sentiment analysis
Syntactic parsing
a b s t r a c t
Online advertising has become one of the major revenue sources of today’s Internet ecosystem. The main
advertising channels used to distribute textual ads are sponsored search and contextual advertising. Here
we consider the problem of contextual advertising, i.e. associating ads with a Web page. Most of previous
work only focuses on topical relevance of ads whereas the consumer attitudes are ignored. In this paper,
we propose a novel advertising strategy, called Dissatisfaction-oriented Advertising based on Sentiment
Analysis (DASA), to simultaneously improve ad relevance and user experience. Specifically, by using syntactic parsing and sentiment dictionary, we propose a rule based approach to extract topic words of opinion sentences associated with negative sentiment, which are regarded as the advertising keywords. We
also design a prototype system for product information submission for the sake of ad selection. We take
into account the consumer attitudes and promote the competitors of those products with which the consumers are not satisfied. The experimental results on advertising keyword extraction and ad selection
have demonstrated the effectiveness of the proposed approach.
Ó 2010 Elsevier Ltd. All rights reserved.
1. Introduction
The growth of the Internet signified a dawn of a new age for
advertising. Compared with that on traditional media like newspaper and television, advertising on the Internet (online advertising)
boasts advantages including wide coverage, low cost and so on.
Driven by these unique advantages of online advertising and the
rapid development of World Wide Web, advertisers have paid
increasing attention to this novel and impressive advertising media. As mentioned in (Broder, Fontoura, Josifovski, & Riedel, 2007),
the total Internet advertiser cost in US alone in 2006 is estimated
at over 17 billion dollars with a growth rate of almost 20% year
over year which means more than 29 billion dollars in 2009.
The large investment in online advertising market provides
main funding for the majority of Internet companies. For some
Web sites, especially those for-profit non-transactional ones, revenue from advertising maintains not only current performance but
also future development. Therefore, it seems that these advertising-relied Web sites should get ads published as many as possible
to gain high revenue from advertisers. However, Bhargava et al.
have show in their work (Bhargava & Feng, 2002) that advertising
would create a disutility to Web users, consequently reducing sites’
market share and consumer-side revenue, further the profits from
advertisers. Also, Mccoy et al. have drew a similar conclusion in
q
This work is supported by China National Key Technology R&D Program
(2008BAH26B00).
* Corresponding author. Tel.: +86 571 87953955.
E-mail address: [email protected] (G. Qiu).
0957-4174/$ - see front matter Ó 2010 Elsevier Ltd. All rights reserved.
doi:10.1016/j.eswa.2010.02.109
their investigation (Mccoy, Everard, Polak, & Galletta, 2007). In
other words, from the perspective of consumers, advertising
should be constrained or even boycotted. The tradeoff between
financial revenue and market share triggers the emergence of relevant advertising to emphasize the relevance between ads and
Web pages for the sake of consumers.
Broder et al. categorize relevant advertising (textual ads)
according to the distribution channels into sponsored search (SS)
and contextual match (CM) (Broder et al., 2007). SS aims to place
originating query driven ads on the result pages from a Web search
engine. This kind of advertising is used by major Web search engines like Google, Yahoo! and Microsoft Live Search. CM aims to
place the commercial ads in the content of generic Web pages.
Some major Web search engines also provide such service and
there are also many smaller players. Fig. 1 gives illustrative examples for these two types of advertising. The primary focus of this
paper is on CM which is also referred to as contextual advertising.
The dominant strategy of most of current sponsored search and
contextual match is the keyword targeted marketing (RibeiroNeto, Cristo, Golgher, & Moura, 2005). Previous approaches to targeted/relevant advertising consider short-term and long-term
interests, however suffer from privacy issues (Wang et al., 2002).
In keyword targeted marketing, keywords are extracted from
search queries (in the case of SS) or Web pages (in the case of
CM) to match against those keywords associated with ads provided
by advertisers. It would be important to note that the privacy issues can be avoided in this kind of strategy.
The problem of placing relevant ads using the keyword targeted
marketing strategy (mostly for contextual advertising) has
G. Qiu et al. / Expert Systems with Applications 37 (2010) 6182–6191
6183
Fig. 1. Examples of sponsored search and contextual match. (a) Yahoo! search results for query ‘‘Honda automobiles”, in which we can see an ad of ‘‘Honda Autos” on the right.
(b) A snapshot of the home page of automotiveforums.com. There are ads provided by Google adsense on the left.
received a lot of attentions recently (Broder et al., 2007; Chakrabarti, Agarwal, & Josifovski, 2008; Lacerda et al., 2006; Ribeiro-Neto
et al., 2005; Wu & Bolivar, 2008; Yih, Goodman, & Carvalho, 2006).
These approaches only focus on topical relevance between ads and
Web pages, which may not be optimal in some situations. For
example, for a blog written by someone who is complaining about
the safety of Honda Accord, the ads related to Honda Accord are good
matches if we simply consider the topical relevance. However, the
consumer reading this blog may have a negative feeling about Honda Accord and therefore may not click the ads. Thus, it might be
desirable to place the ads of other automobiles with high safety
standard.
In this paper, we propose a novel ad targeting mechanism,
called Dissatisfaction-oriented Advertising based on Sentiment
Analysis (DASA), to address the above mentioned problem. Our basic idea is to combine traditional keyword relevance matching with
sentiment analysis (also known as opinion mining). In fact, sentiment analysis of texts is a kind of non-topical text analysis technique which is orthogonal to the topical based technique (Esuli &
Sebastiani, 2005). By sentiment analysis, we aim to discover the
author’s underlying sentiment when he/she is writing the text,
and eventually determine his/her attitude towards the mentioned
topics. This way, we are able to promote ads which not only relate
to the topics of the Web pages but also meet the consumers’ potential information needs. Naturally, in the example given above, given the author’s negative attitude towards the safety of Honda
Accord, we may consider promoting automobiles which have higher safety standard like Volvo.
The rest of the paper is organized as follows. Section 2 discusses
previous work on contextual advertising and sentiment analysis. In
Section 3, we provide an overview of our DASA advertising strategy, along with some discussions on application issues. In Sections
4 and 5, we describe our approaches for advertising keyword
extraction and ad placement in details, respectively. In Section 6,
we describe our experimental methodology and data sets, and
demonstrate experimental results with discussions. Finally, we
provide some concluding remarks in Section 7.
2. Related work
In this section, we provide a brief description of related work on
contextual online advertising and sentiment analysis which is the
key technique employed in our approach.
6184
G. Qiu et al. / Expert Systems with Applications 37 (2010) 6182–6191
2.1. Contextual online advertising
Contextual advertising is a major type of online advertising, in
which ads are placed on Web pages according to their content.
Published literature on contextual advertising is still limited. One
reason is that it is an emerging research area; in addition, commercial secrets also limit publications of research achievements. In
existing researches, some reduce the problem to keyword matching (Wu & Bolivar, 2008; Yih et al., 2006), while others inspect relevance of the entire text on the Web page (Broder et al., 2007;
Chakrabarti et al., 2008; Lacerda et al., 2006; Ribeiro-Neto et al.,
2005).
In the first line, Yih et al. (2006) emphasize the extraction of
appropriate advertising keywords in contextual advertising and
propose a classification based approach. That is, by using documents manually annotated with advertising keywords, they apply
logistic regression to train a classifier to classify a keyword as an
advertising keyword or not. The features used to represent keywords include linguistic features, html metadata, TFIDF, query
log, etc. A similar approach is recently proposed by Wu and Bolivar
(2008) which adopts linear regression to determine if a keyword is
an advertising one. The features used in their work are similar to
those in (Yih et al., 2006). It would be important to note that, in
our work, we develop a novel strategy to extract advertising keywords in which no training process is required.
In the second line, previous work on relevance measures between Web pages and ads can be roughly categorized into syntactic based and semantic based approaches. Ribeiro-Neto et al.
provide a syntactic solution to ad placement based on the cosine
similarity measuring between Web page and ad vectors (RibeiroNeto et al., 2005). They also propose a solution to solve the impedance problem between the vocabulary of ads and Web pages. In
their follow-up work (Lacerda et al., 2006), they design a ranking
strategy for displaying ads according to their relevance by effectively leveraging all individual features using genetic programming. They aim to optimize overall precision and minimize the
number of misplacements. The semantic approach proposed in
(Broder et al., 2007) stems from the linguistic phenomena of polysemy that a word has different meanings in different semantic contexts, which is contrary to the impedance problem in (Ribeiro-Neto
et al., 2005). They use the proximity of the ad and page classes as
the semantic score. Their experimental results show that semantic
score is significant as to improving the ad quality.
However, all previous approaches above focus only on the topical relevance and ignore the sentiment information between the
lines. In this work, we propose a novel strategy which considers
both the topic relevance and sentiment coherence between ads
and Web pages.
2.2. Sentiment analysis
Another work related to our approach is sentiment analysis.
Current researches on sentiment analysis mainly focus on topic related issues (Hu & Liu, 2004; Lu & Zhai, 2008; Mei, Cai, Zhang, &
Zhai, 2008; Mei & Ling, 2007) and sentiment classification (Ding
& Liu, 2007; Hatzivassiloglou & McKeown, 1997; Pang & Lee,
2004; Turney, 2002; Turney & Littman, 2003). Here, we only summarize some of previous approaches on topic modeling and word
sentiment classification, which are the most closely related to
our work.
In the work of (Mei & Ling, 2007), Mei et al. propose a novel
probabilistic model called Topic-Sentiment Mixture to capture
the mixture of topics and sentiment simultaneously. A topic model
and two sentiment models are defined in their work based on language models to model the probabilistic distribution of words in
different topics and sentiment polarities. The work by Lu and Zhai
(2008) studies the problem of how to integrate a well-written expert review about an arbitrary topic with many ordinary opinions
expressed in a text collection. They propose a semi-supervised topic modeling approach to cast the expert review as a prior in the
probabilistic topic model PLSA and fit the model to the text
collection.
Hatzivassiloglou et al. are the earliest to tackle the problem of
determining the semantic orientation of sentiment words
(Hatzivassiloglou & McKeown, 1997). Their proposed method tries
to predict the orientation of subjective adjectives by analyzing
pairs of adjectives extracted from a large unlabeled document
set. The underlying intuition is that the act of conjoining adjectives
subjects to linguistic constraints on the orientation of the adjective
involved. Turney et al. adopt a different methodology which requires little linguistic knowledge (Turney & Littman, 2003). They
first define two minimal sets of seed terms as descriptive of the
categories Positive Sp and Negative Sn. Then they compute the
point-wise mutual information (PMI) of the target term t with each
seed term ti as a measure of their semantic association.
Sentiment analysis of sentences is a major building block of our
proposed approach. The specific task here is to extract the topic
words of sentences that the contained sentiment words are about.
In our work, we propose a rule based approach to accomplish this
extraction task.
3. Overview of DASA advertising strategy
Our contextual advertising strategy DASA works as follows:
(1) Extract topic words as the candidate topics that ads may be
about from the Web pages (the traditional way to promote
ads);
(2) Determine consumers’ sentiment on the extracted topic
words and select those towards which consumers have negative sentiment as the final advertising keywords;
(3) Match the advertising keywords with appropriate ads.
In other words, an ad is considered to be relevant to a Web page
only if it is associated with the topic words towards which consumers have negative attitudes. The most crucial steps of our approach are topic word extraction and its sentiment classification,
which eventually lead to advertising keyword selection. Fig. 2
shows the architecture of DASA.
3.1. Application issues
There are several application issues we have to address before
the detailed elaboration of our approach.
Since our advertising approach depends on the analysis of consumers’ attitudes, the first issue is that whether there exists large
scale Web environment containing plenty of personal opinions
where ads can be placed. The answer to this problem is User Generated Content (UGC), which is also known as Consumer Generated
Media (CGM), referring to various kinds of media content produced
by consumers. It has been characterized as ‘‘conversational media”
which is a two-way process encouraging the publishing of users’
own content and commenting on others’ (http://www.en.wikipedia.org/wiki/User-generated content). Typical applications include
forums and blogs, which have already been prevailing Web applications on the Internet (http://www.businessweek.com/globalbiz/
content/jan2007/gb20070109559223.htm). Therefore, UGC is ideal
environment for our advertising strategy.
The second issue is the targeted audience of our advertising. In
UGC, there are two types of users participating in Web activities:
writers and readers of the content. When a writer is complaining
G. Qiu et al. / Expert Systems with Applications 37 (2010) 6182–6191
6185
Fig. 2. Illustration of the workflow of DASA. TWn means the topic word extracted from Web pages by the topic word extraction technique. AKk is the final advertising keyword
selected from the topic words by sentiment classification technique. These two steps form the advertising keyword extraction phase as described by the dashed rectangle.
about some product in his/her article, our advertising scheme
tends to place ads which are about rival products. This can potentially help the writer find a new product that satisfies his/her
needs. Besides, this scheme also provides alternatives to the readers who have similar opinions with the writer. When there are no
personal comments or only positive opinions contained in the article, traditional advertising strategy can be applied.
4. Advertising keyword extraction
As stated above, we reduce the task of advertising keyword
extraction to topic extraction and sentiment classification subtasks, i.e. to identify topics towards which consumers have negative attitudes. Often the consumers are more concerned about
the topics on which they have personal comments. Therefore, we
only consider the sentences which contain sentiment words. And
these sentences are referred to as opinion sentences throughout this
paper. Our extraction approach can be summarized in three steps:
mar, the relation between two words A and B can be described as
A(or B) depends on B(or A). There are also other complicated relations between A and B. For example, A depends on another word C
while C depends on B. We define two categories to summarize all
the relations, which are also illustrated in Fig. 3.
Definition 1. Direct relation (DR): A direct relation means one
word depends on the other one directly or they both depend on a
third word directly. In the context of sentiment word S and topic
word T, this relation means that S(or T) depends on T(or S), or S and
T both depend on another word H.
Definition 2. Indirect relation (IDR): An indirect relation means one
word depends on the other one through another word or they both
depend on a third word indirectly. In the context of sentiment
word S and topic word T, this relation means that S(or T) depends
on T(or S) through another word H, or S(or T) depends on a word H1
which depends on the same word H which T(or S) depends on.
(1) Identify the opinion sentences in the texts;
(2) Extract the topics in the opinion sentences given the labeled
sentiment words;
(3) Select advertising keywords from the extracted topic words.
4.1. Identify opinion sentences
In this step, as we define opinion sentences as sentences that
contain sentiment words, the identification task can be tackled
by discriminating if any of the contained words are sentiment
ones. In our work, we employ a dictionary based approach to judge
if a word is a sentiment word. The dictionary used in our work is
the General Inquirer (http://www.wjh.harvard.edu/inquirer/)
which consists of 1892 negative words such as ‘‘abandon”, ‘‘abuse”,
‘‘poor”, and 1563 positive ones such as ‘‘able”, ‘‘appreciate”, ‘‘good”.
4.2. Extract topic words of opinion sentences using rules
Given opinion sentences identified from the texts and the contained sentiment words as well, topic words associated with the
sentiment words can be extracted. Here we propose our solution
using pre-set rules based on syntactic parsing of sentences.
We take advantage of Minipar (http://www.cs.ualberta.ca/lindek/minipar.htm) as the syntactic parsing tool, which uses the
dependency grammar (Tesniere, 1959). After parsing, words in a
sentence are assigned with corresponding syntactic categories
and linked to each other by certain relations. In dependency gram-
Fig. 3. Different relations between sentiment word S and topic word T. (a–c) are the
three kinds of direct relations. (d–g) illustrate the four kinds of indirect relations.
6186
G. Qiu et al. / Expert Systems with Applications 37 (2010) 6182–6191
Note that in IDR, there may be more than one H or H1 in the
dependency path. In our work, we only consider the one word situation as compact dependency relations would guarantee the association between S and T with high certainty.
Given the seven relations between S and T, we design our rules
which are able to capture the relations as well as the word syntactic categories. The specific rules are listed in Table 1.
In our rule representations, S(T, H1, H)-Category and S(T, H1,
H)-Relation denote the syntactic category and relation of the word
S(T, H1, H), respectively. We also assign priority Pi (i = 1,...,7) to
each rule in case that more than one rule can be applied to the
extraction task. In the seven rules, R1 is assigned with the highest
priority P1 and R2 to R7 get lower priorities in descending order.
We set the priority for each rule by considering the certainty of
the association between S and T that the corresponding relation
can guarantee.
For the sake of simplicity, we only consider the topological
structures implied by these rules, ignoring most of the detailed
information of relation and syntactic category. One constraint is
that the category of topic word is limited to N (assigned by Minipar) which means the part-of-speech of the topic word should be
noun. Note that Minipar parses all pronouns and numbers also as
N. We do not consider the noun phrases in our current work, since
the state-of-the-art phrase identification is still not accurate enough. There are also some exceptions we need to exclude, such
as the conj relation in R1 and R2 which means the two related
words are of equal syntactic role in the sentence.
Using these rules and their priorities, we are now able to extract
the topic word in the opinion sentence. Take R1 for example, given
the sentiment word S parsed as S-Category, word with category TCategory and relation S-Relation with S is taken as the topic word T
of S with priority P1. The algorithm shown in Fig. 4 elaborates how
to extract topic word T using these rules.
4.3. Select advertising keywords
Clearly not all the above extracted topic words can be taken as
advertising keywords directly as explained in the introduction section. We should identify users’ attitudes towards the topics and
then select appropriate topic words as advertising keywords. From
the perspective of sentiment analysis, the implied sentiment of the
topic word is classified to be positive or negative based on the
polarity of the related sentiment word. As stated in the very beginning, advertising on topics towards which consumers have negative attitudes is more helpful and targeted to users. Therefore,
we take topic words which correspond to negative sentiment
words as the advertising keywords.
5. Appropriate ad placement
After the advertising keywords are obtained through topic
extraction and sentiment classification, we can select appropriate
Table 1
Rules defined in our work. S(T, H1, H)-Category and S(T, H1, H)-Relation mean the
syntactic category and relation of the word S(T, H1, H), respectively. We use these
rules to extract topic words in opinion sentences.
No. Rules
R1
R2
R3
R4
R5
R6
R7
S-Category < S-Relation > T-Category
T-Category < T-Relation > S-Category
S-Category < S-Relation > H-Category < T-Relation > T-Category
S-Category < S-Relation-T-Crelation > T-Category
T-Category < T-Relation-S-Crelation > S-Category
S-Category < S-Relation-H1-Relation > H-Category < T-Relation > T-Category
T-Category < T-Relation-H1-Relation > H-Category < S-Relation > S-Category
Fig. 4. Algorithm for extracting topic word.
ads that are relevant to these keywords and simultaneously meet
consumers’ needs. As stated above, our DASA ad placement strategy only concerns topics towards which consumers have negative
attitudes.
In real world applications, advertising keywords can be brand
names (such as BMW in the sentence ‘‘I do not like BMW”), as
well as composed features (such as safety in the sentence ‘‘I
am not satisfied with the safety of Honda Accord”). Therefore, different strategies should be adopted to select corresponding ads
for these two situations. In our work, we propose to promote
rivals’ products when the advertising keywords are brand names
and select products which have excellent performance in certain
features when the advertising keywords are feature names. In
case of multi-candidates, WTP (willingness to pay) (Feng, Bhargava, & Pennock, 2003) can be employed to determine which one
to place.
In order to select appropriate brands efficiently for the two situations, we need to know which two brands are rivals and which
features a brand is good at. In traditional advertising systems, an
ad is always represented by its bid phrases, self-explanatory title,
short description and URL of the target Web page. These pieces
of information can only provide a general description about the
product. However, we argue that ad description should be more
targeted and expressive in terms of its advantages over its rivals
so that the ad-network can be aware of products’ advantages and
consequently select appropriate ads to meet consumers’ needs. In
our work, we design a browser based system to construct a knowledge base for different products. Fig. 5 is a snapshot of the system.
Any manufactories or dealers can provide their product information through the system. Currently, we take automobile domain
as an example. Note that our approach can also be applied other
domains.
The product information is stored in XML files as shown in
Fig. 6. The element brand in the node AUTO is the brand name of
the auto, such as BMW, BENZ, etc. The element name in the node
MODEL is the model name, such as Z, X serials of BMW. The node
RIVAL stores brand names of rivals and node ADVANTAGE_FEATURE
contains the outstanding features of the model. An example is that
750li of BMW may be labeled as rival of S600 of BENZ. Another
example is that Volvo has excellent performance in safety. Thus,
we can promote ads of S600 when the consumer is complaining
about 750li and place ads of Volvo when he is unsatisfied with
the safety feature of Honda Accord.
6187
G. Qiu et al. / Expert Systems with Applications 37 (2010) 6182–6191
Fig. 5. Snapshot of our system for product information submission. Manufactories or dealers can provide advantageous feature information and rivals of their products
through this system.
D#2 for advertising keyword evaluation. Table 2 shows the detailed information of each data set.
In D#1, the average length of the 3783 opinion sentences is 21
words. All these sentences are labeled manually with the topic
words. We develop a tool for volunteers to do the labeling with
the instruction as follows:
Fig. 6. XML format for product information storage.
6. Experiments and results
Three experiments are conducted to evaluate the performance
of our approaches to topic word and advertising keyword extraction and ad placement. We begin with a description of the evaluation corpus.
6.1. Evaluation corpus
Since our advertising approach considers not only topical relevance but also consumers’ opinions, we use Web forums as the
evaluation corpus. Particularly, we choose the popular auto forum
automotiveforums.com as the source data in our experiments. In the
forums, the users’ posts are organized as one thread if they all respond to the same post. We crawl and store these posts in unit of
threads, i.e. store all the posts of one thread in a single file. We call
this file an article hereafter. Let D#0 denote the set of articles. Another two data sets are constructed from D#0 for the first two
experiments respectively, D#1 for topic word evaluation and
Suppose you are the writer of the sentence, given the sentiment
word, please figure out in the sentence what is the topic word that
the sentiment word related to. For example, in the sentence ”The z3
was awful and z4 is not much better”, the corresponding topic word
for ”awful” should be ”z3” and that for ”better” should be ”z4”.
We collect the articles which contain the 3783 opinion sentences and the final collection D#2 consists of 647 articles. For
advertising keyword labeling, we divide the 647 articles into 10
groups, and then request 10 annotators to label the keywords in
articles whose corresponding ads they consider appropriate
(”appropriate” means not only relevant but also not annoying).
These keywords are then taken as the correct advertising keywords
for the articles.
Table 2
Statistical information for the three data sets for D#0, D#1 and D#2.
D#0
D#1
D#2
128,425 articles
3783 opinion sentences
647 articles
6188
G. Qiu et al. / Expert Systems with Applications 37 (2010) 6182–6191
6.2. Experiments on topic word extraction
Topic word extraction is the key to the extraction of advertising
keywords, therefore we conduct separate experiments on it. We
begin with the evaluation strategy and criteria.
6.2.1. Evaluation strategy and criteria
To the best of our knowledge, there is no previous work on sentiment based topic word extraction in the research area of sentiment analysis. Therefore, in our experiments, we compare our
approach with a naive distance based approach that extracts the
nearest noun of the sentiment word as the topic word. We use
the accuracy as the evaluation criteria. The accuracy is defined to
be the percentage of the sentences whose automatically extracted
topic words (either by our approach or the distance based approach) are the same as the user pre-labeled ones.
6.2.2. Results and discussions
Table 3 shows the results of the 3783 opinion sentences in D#1
by using our rule based approach, the distance based approach and
two random approaches: Random-All and Random-Noun. The Random-All approach selects any one word from the sentence randomly as the topic word. Thus its corresponding accuracy value
is calculated as the inverse of the average sentence length (21
words as shown before). Similarly, the Random-Noun approach is
to select any noun in the sentence as the topic word and thus its
accuracy value is the inverse of the average number of words
parsed as N contained in the sentence (The number is 10.25. This
number is a little bit high because Minipar parses all pronouns
and numbers in addition to actual nouns as N and meanwhile some
parsing errors also exist.). We take the -Noun case into account as
both rule and distance based approaches impose the noun
constraint.
As can be seen, both our proposed approach and the distance
based approach significantly outperform random approaches. Particularly, our approach obtains 55% accuracy, while the distance
based approach obtains 42% accuracy. There is almost 31% performance improvement. This shows that our approach is effective in
extracting the topic word. Moreover, this result reveals the fact
that the relationship between sentiment word and topic word
Table 3
Performance results of our rule based approach compared with the distance based
approach and random approaches. The accuracies of random approaches are
calculated as the inverse of average sentence length (for Random-All) or the inverse
of average number of contained words parsed as N (for Random-Noun). The
CorrectSent column is the number of sentences whose topic words are extracted
correctly.
Approaches
CorrectSent
Accuracy
Rule
Distance
Random-Noun
Random-All
2063
1581
–
–
0.55
0.42
0.10
0.05
can be better characterized by syntactic structure rather than
distance.
Table 4 gives some examples of our rule based approach compared with other approaches.
6.3. Experiments on advertising keyword extraction
In this section, we first describe our evaluation criteria and
strategy, and then show the results and discussions.
6.3.1. Evaluation criteria and strategy
In order to evaluate the performance of different advertising
keyword extraction methods, we consider the quality of advertising keywords from two aspects: the commercial value (CV) and
traditional measurements (precision, recall and F1-measure)
which are calculated based on the manually labeled results. The
commercial value of extracted keywords in an article is defined
as the percentage of extracted keywords which are of real value
in some particular online advertising environment, like Google’s
AdWords and Yahoo’s Sponsored Search. It measures the quality
of keywords from the perspective of advertisers while traditional
measurements emphasize the consumers’ interests.
For comparison, we implement another keyword extraction approach based on the famous keyword weighting schema TFIDF,
which is regarded as the most useful feature for advertising keyword extraction (Yih et al., 2006). It would be important to note
that, however, our proposed approach can be easily incorporated
into previous approaches, since most of them only consider topical
relevance while our approach considers consumers’ attitudes. Our
aim in the experiments is to show the effectiveness of our approach instead of showing superior performance over sophisticated previous approaches. Therefore, a comparison to a typical
feature in advertising keyword extraction is enough. For our rule
based approach, the advertising keywords of an article are composed of those of the opinion sentences in the article. All these keywords are not weighted. For the TFIDF based approach, we remove
the stop words and select the top 10 keywords with the highest
TFIDF values as the advertising keywords. The average number of
the labeled keywords in an article is 2.76 and that of those extracted by our rule based approach is 2.26.
One important issue in our evaluation strategy is about the recognition of the commercial value of a word. We argue that a word
is commercially valuable if and only if there is at least one advertiser that bids on it. Particularly, in order to quantitatively evaluate
it, we submit a keyword to some search engine and check if there is
any ad shown in the result page. If there is ad shown, this keyword
is considered commercially valuable. In current experiments, we
make use of Google’s search engine and its AdWords service.
Fig. 7 gives two search examples (‘‘shoes” and ‘‘black”) on Google.
From the results, we can see that there are ads in the search result
page of ‘‘shoes” but no ads for ‘‘black”. So the keyword ‘‘shoes” is
considered commercially valuable while ‘‘black” is not. This conclusion is also consistent to common sense.
Table 4
Some examples for different approaches. The Sentiment column shows the sentiment word contained in the sentence. The Nouns column shows the words which are parsed as N
by Minipar in the sentence. The Labeled column shows manually labeled topic words. The Rule and Distance columns show the extracted topic words by rule and distance based
approaches, respectively.
Sentences
Sentiment
Nouns
Labeled
Rule
Distance
So the next thing is, does anyone know where I could get a cheap dealers
manual for the car?
The coil tower on these vehicles are notorious for corrosion.
But in the end, a nice turbo setup would cost approx.
Cheap
Thing, anyone, where, manual,
dealers, car
Tower, coil, vehicles, corrosion
End, setup, turbo, approx
Manual
Manual
Dealers
Tower
Setup
Tower
Setup
Vehicles
Approx
Notorious
Cost
G. Qiu et al. / Expert Systems with Applications 37 (2010) 6182–6191
6189
Fig. 7. Google search results for words ‘‘shoes” and ‘‘black”. As can be seen, there are ads in the result page of ‘‘shoes” but no ads for ‘‘black”. This means word ‘‘shoes” has
higher commercial value than ‘‘black”.
6.3.2. Results and discussions
In our experiments, we report CV, precision, recall and F1-measure of the top 10 selected keywords as the overall results for the
TFIDF based approach. Considering that the average number of
advertising keywords manually labeled and extracted by rule
based approach is close to three, we also evaluate the performance
of TFIDF based approach for the top three selected keywords. Table
5 lists all the results of different approaches.
From the table we can see that our approach outperforms the
TFIDF based approach (both TFIDF-Average and TFIDF-Top3) in
all the four measurements, except the precision in TFIDF-Top3.
The improvement of CV (17.3% over TFIDF-Average and 8.6% over
TFIDF-Top3) means that the advertising keywords extracted by
our approach are more valuable from the perspective of advertisers
and thus ensures these keywords would be related with ads instead of being abandoned because of being meaningless for
advertisers.
The superior performance of our approach in traditional measurements precision, recall and F1-measure shows that our approach can extract more advertising keywords with high
accuracy than other approaches. Note that, the F1-measure can
be essentially taken as the targeted-ness of the keywords from
the view of consumers as it is a combination of precision and recall.
Usually the more targeted the advertising keywords are, the higher
revenue the advertisers will obtain. The reason is that with the
improvement in the targeted-ness of keywords, ad relevance
6190
G. Qiu et al. / Expert Systems with Applications 37 (2010) 6182–6191
Table 5
Results of CV, precision, recall and F1-measure for our rule based approach and the
TFIDF based approach. For TFIDF, we consider two different variations TFIDF-Average
and TFIDF-Top3 which take the average value of top 10 and top 3 keywords,
respectively.
Approaches
CV
Precision
Recall
F1
Rule
TFIDF-Average
TFIDF-Top3
0.265
0.226
0.244
0.101
0.092
0.104
0.375
0.211
0.145
0.159
0.128
0.121
increases, therefore consumers will get less disturbed by ads and
thus will remain well or even better impression on advertising
Web sites. Finally, the market share would not be ill affected a
lot by ads and advertisers and Web sites will be likely to get more
profits. Therefore, F1-measure is more important than the other
two measurements. In the exception of the results, although our
approach is a little inferior in precision compared with TFIDFTop3, we gain 31.4% improvement in the F1-measure, which
means our approach still outperforms TFIDF-Top3.
Note that all the values in our measurements are relatively low.
However, in online advertising-like applications, an improvement
of X% in ad matching can lead to an improvement of X% in the
end result unlike many other tasks where the effect of performance
enhancement on the end result is not linear (Mishne & Rijke, 2006).
Therefore, regarding the magnitude of online advertising market, a
tiny improvement would lead to a considerable increase in the final revenue for advertisers.
6.4. Experiments on ad selection
The above experiments have examined the performance of our
approach for advertising keyword extraction, which is a key step
towards ad placement. We now conduct experiments on selecting
appropriate ads. Since we do not have the corpus of real world ads,
we represent our recommendations as descriptions such as ”autos
those perform excellent in safety”. As described before, there are two
kinds of ads that would be promoted to consumers as they may
complain on a certain brand or feature. For simplicity, we focus
only on features in our current experiments.
We compare DASA with the naive strategy that extracts advertising keywords in the same way but does not consider the atti-
tudes of consumers, i.e. take all the topic words as the
advertising keywords (TE-ALL). Volunteers are requested to make
the judgment on the recommended ads (the ad descriptions indeed). In our experiments, we construct a prototype system in
which volunteers provide texts commenting on some products
(only on features of autos in current work) and then get two lists
of ads returned by DASA and TE-ALL. Volunteers are unaware of
which list is the result of which approach. They are then asked to
determine which one they think is better to meet their needs with
less annoying.
Fig. 8 shows the comparison between DASA and TE-ALL. The Yaxis is the number of times that volunteers regard the results of an
approach are better than those of the other one. An obvious observation from the figure is that our approach (DASA) outperforms TEALL. For 35 times, volunteers judge that the results of our approach
are better than those of TE-ALL, while the contrast is 22 times. The
results show that it is reasonable to take consumers’ attitudes into
account.
7. Conclusions
Targeted advertising is of great importance for Internet companies to gain revenue from both advertisers and consumers. Previous
approaches focus only on the topical relevance while the consumers’ attitudes are ignored. These approaches fail to meet the actual
needs of consumers especially when they may have negative attitudes towards the mentioned topics. In this paper, we take the attitudes of consumers into account and propose a novel advertising
strategy DASA to promote ads according to what they are unsatisfied with. We use pre-set rule based sentiment analysis techniques
to tackle topic word extraction and consumers’ attitude identification in advertising keyword extraction. We also design a prototype
system for users to provide product information for the sake of ad
selection. For evaluation, we conduct separate experiments to measure the quality of topic words, advertising keywords and placed
ads. Specifically, we propose to use commercial value to measure
the quality of advertising keywords from the perspective of advertisers. Experiments show that our strategy outperforms the TFIDF
based approach in advertising keyword extraction and the naive approach which does not consider the attitudes of consumers in ad
selection. The results indicate that it is reasonable to take the attitudes of consumers into consideration.
References
Fig. 8. Comparison results of our approach DASA and TE-ALL which does not
consider the attitudes of consumers.
Bhargava, H. K., & Feng, J. (2002). Paid placement strategies for internet search
engine. In Proc. 2002 Int. WWW Conf. (WWW’02) (pp. 117–123).
Broder, A., Fontoura, M., Josifovski, V., & Riedel, L. (2007). A semantic approach to
contextual advertising. In Proc. 2007 Int. Conf. on research and development in
information retrieval (SIGIR’07), 2007 (pp. 559–566).
Chakrabarti, D., Agarwal, D., & Josifovski, V. (2008). Contextual advertising by
combining relevance with click feedback. In Proc. 2008 Int. WWW Conf.
(WWW’08) (pp. 417–426).
Ding, X., & Liu, B. (2007). The utility of linguistic rules in opinion mining. In Proc.
2007 Int. Conf. on research and development in information retrieval (SIGIR’07) (pp.
811–812).
Esuli, A., & Sebastiani, F. (2005). Determining the semantic orientation of terms
through gloss classification. In Proc. 14th ACM Int. Conf. on information and
knowledge management (CIKM’05) (pp. 617–624).
Feng, J., Bhargava, H. K., & Pennock, D. (2003). Comparison of allocation rules for
paid placement advertising in search engines. In Proc. 5th Int. Conf. on electronic
commerce (ICEC’03) (pp. 294–299).
Hatzivassiloglou, V., & McKeown, K. R. (1997). Predicting the semantic orientation
of adjectives. In Proc. 1997 ACL (ACL’04).
Hu, M., & Liu, B. (2004). Mining and summarizing customer reviews. In Proc. 2004
Int. Conf. knowledge discovery and data mining (KDD’04) (pp. 168–177).
Lacerda, A., Cristo, M., Goncalves, M., Fan, W., Ziviani, N., & Ribeiro-Neto, B. (2006).
Learning to advertise. In Proc. 2006 Int. Conf. on research and development in
information retrieval (SIGIR’06) (pp. 549–556).
Lu, Y., & Zhai, C. (2008). Opinion integration through semi-supervised topic
modeling. In Proc. 2008 int. WWW Conf. (WWW’08) (pp. 121–130).
G. Qiu et al. / Expert Systems with Applications 37 (2010) 6182–6191
Mccoy, S., Everard, A., Polak, P., & Galletta, D. (2007). The effects of online
advertising. Communications of the ACM, 50(3), 84–88.
Mei, Q., Cai, D., Zhang, D., & Zhai, C. (2008). Topic modeling with network
regularization. In Proc. 2008 Int. WWW Conf. (WWW’08) (pp. 101–110).
Mei, Q., & Ling, X. (2007). Topic sentiment mixture: Modeling facets and opinions in
weblogs. In Proc. 2007 Int. WWW Conf. (WWW’07) (pp. 161–175).
Mishne, G., & Rijke, M. (2006). Language model mixtures for contextual ad
placement in personal blogs. In Proc. 5th Int. Conf. NLP (FinTAL).
Pang, B., & Lee, L. (2004). A sentimental education: Sentiment analysis using
subjectivity summarization based on minimum cuts. In Proc. 42nd Ann. meeting
of the association for computational linguistics (ACL’04) (pp. 271–278).
Ribeiro-Neto, B., Cristo, M., Golgher, P., & Moura, E. (2005). Impedance coupling in
content-targeted advertising. In Proc. 2005 Int. Conf. on research and development
in information retrieval (SIGIR’05) (pp. 496–503).
6191
Tesniere, L. (1959). Elements de syntaxe structurale. Paris: Librairie C. Klincksieck.
Turney, P. (2002). Thumbs up or thumbs down? Semantic orientation applied to
unsupervised classification of reviews. In Proc. 40th Ann. meeting of the
association for computational linguistics (ACL’02) (pp. 417–424).
Turney, P. D., & Littman, M. L. (2003). Measuring praise and criticism: Inference of
semantic orientation from association. ACM Transactions on Information Systems,
21(4), 315–346.
Wang, C., Zhang, P., Choi, R., & D’Eredita, M. (2002). Understanding consumers
attitude toward advertising. In Proc. 8th Americas Conf. on information system
(pp. 1143–1148).
Wu X., & Bolivar, A. (2008). Keyword extraction for contextual advertisement. In
Proc. 2008 Int. WWW Conf. (WWW’08) (pp. 1195–1196).
Yih, W., Goodman, J., & Carvalho, V. R. (2006). Finding advertising keywords on web
pages. In Proc. 2006 Int. WWW Conf. (WWW’06) (pp. 213–222).