Download WAKAO-MASTERS-REPORT - The University of Texas at Austin

Copyright
by
Shinya Wakao
2012
The Report Committee for Shinya Wakao
Certifies that this is the approved version of the following report:
Wall Street News on Main Street
APPROVED BY
SUPERVISING COMMITTEE:
Robert C. Luskin, Supervisor
Scott Moser
Wall Street News on Main Street
by
Shinya Wakao, B.A.; M.A.
REPORT
Presented to the Faculty of the Graduate School of
The University of Texas at Austin
in Partial Fulfillment
of the Requirements
for the Degree of
MASTER OF ARTS
THE UNIVERSITY OF TEXAS AT AUSTIN
August 2012
Wall Street News on Main Street
Shinya Wakao, M.A.
The University of Texas at Austin, 2012
Supervisor: Robert C. Luskin
Over the past decades, people have had an increasing chance to receive economic information, especially news related to the stock market. This is because the
fraction of the U.S. population owning stocks has increased rapidly. However, it
does not mean that a majority of news sources have started to deal with financial
news more. We do not know how traditional media, such as newspapers, have dealt
with financial news during the same period, nor do we know the influence of this
environmental change on political attitudes.
In this report, I analyze the type of contexts in which the stock market has
been described in The New York Times from 1981 to 2011 by Wordfish and the
Latent Dirichlet Allocation (LDA) model. I find that a plunge in the stock market
and political events affect the amount of political topics in stock market news. In
particular, after the financial crisis of 2008–2009, stock market news consisted of
economic, political, and social topics.
iv
Table of Contents
Abstract
iv
List of Tables
vi
List of Figures
vii
Chapter 1.
Introduction
1
Chapter 2. Background
2.1 Analysis of News Coverage of the Stock Market . . . . . . . . . . . .
Chapter 3. Empirical Study
3.1 Data . . . . . . . . . . . . . . . . . .
3.2 Method: Wordfish and Topic Models
3.2.1 Assumption of Wordfish . . . .
3.2.2 Process . . . . . . . . . . . . .
3.3 Topic Models . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Chapter 4. Results
4.1 Term Frequencies . . . . . . . . . . . . . . . . . .
4.1.1 Wall Street, Stock Market, and Dow Jones
4.1.2 Presidential candidates . . . . . . . . . . .
4.1.3 Congress, Political Parties . . . . . . . . . .
4.1.4 Policy-related Words . . . . . . . . . . . . .
4.2 Wordfish Estimation . . . . . . . . . . . . . . . .
4.3 Topic Model Estimation . . . . . . . . . . . . . .
Chapter 5.
Conclusion
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
5
7
.
.
.
.
.
10
10
14
16
16
18
.
.
.
.
.
.
.
20
20
20
21
22
23
25
29
37
Bibliography
39
v
List of Tables
4.1
4.2
4.3
4.4
4.5
4.6
4.7
the Estimated
Five Topics in
Five Topics in
Five Topics in
Five Topics in
Five Topics in
Five Topics in
β and ψ from Select Terms in Figures 4.1 to 4.4
the Stock Market Paragraphs from 1981 to 1985
the Stock Market Paragraphs from 1986 to 1990
the Stock Market Paragraphs from 1991 to 1995
the Stock Market Paragraphs from 1996 to 2000
the Stock Market Paragraphs from 2001 to 2005
the Stock Market Paragraphs from 2006 to 2011
vi
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
26
31
32
33
34
35
36
List of Figures
3.1
4.1
4.2
4.3
4.4
4.5
4.6
(Top) Numbers of Stock Market Articles in The New York Times from
1981 to 2011. (Bottom) Percentage of “Wall Street” Articles in terms
of “Desk” from 1981 to 2011 . . . . . . . . . . . . . . . . . . . . . . .
Frequency of “Stock Market,” “Wall Street,” and “Dow Jones” in The
New York Times from 1981 to 2011 . . . . . . . . . . . . . . . . . . .
Frequency of Presidential Candidates in The New York Times from
1981 to 2011 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Frequency of “Congress” and Political Parties in The New York Times
from 1981 to 2011 . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Frequency of Policy Terms in The New York Times from 1981 to 2011
the Estimated Location of Terms in The New York Times from 1981
to 2011 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
the Estimated Location of Stock Market Paragraphs in The New York
Times from 1981 to 2011 . . . . . . . . . . . . . . . . . . . . . . . . .
vii
13
21
22
23
24
27
28
Chapter 1
Introduction
People receive economic information from many sources - the TV, Internet,
print, radio, and other news media outlets - using the information to evaluate the
health of the national economy. The media provide us with not only economic
indicators such as unemployment rate and GDP growth, but also related information
and analyses that help people better understand the health of the economy. In
addition, people viscerally feel the effects of inflation, such as when they put gas
into their cars. In other words, people consciously and unconsciously understand
how healthy the national economy is through media coverage as well as their daily
activities.
How the public evaluates the health of the national economy politically is a
major concern for politicians - both incumbents and challengers - during elections because economic conditions affect the evaluation of incumbents as well as the election
outcomes (Kinder and Kiewiet, 1979, 1981; Kiewiet, 1983; MacKuen, Erickson and
Stimson, 1992; Fiorina, 1981). Yet what kinds of economic indicators does the public examine when evaluating the economic conditions as well as incumbents political
performance?
As information via the media is the primary source of information for most
1
of the public, how the media treat the national economy politically is also an important matter. Government agencies report periodic economic indicators, such as the
unemployment rate, inflation rate, and GDP, which the public uses for political evaluations (e.g., Vavreck, 2009). Meanwhile, the media cover stock market conditions
more often than other indictors. Indeed, business and financial media such as CNBC
report on it every moment. When the stock market rises or falls dramatically, it becomes sensational news. People can also check past and current stock prices online.
However, the media also treat the stock market differently. Companies are evaluated
based on their stock conditions. When a company goes public, it sometimes becomes
a social phenomenon, such as Netscape in 1995, Google in 2004, and Facebook in
2012. In other words, news about the stock market is more dynamic and is reported
upon at a higher frequency compared to news related to other economic indicators.
I believe that this lack of study results from at least two reasons. First, the
stock market per se has not been as important for middle-class citizens in the U.S.
Although the media reported the stock markets ups and downs, such news did not
affect the majority of people, only those who manage their assets in the stock market
- or at least people did not care about this information. However, the relationship
between the stock market and American citizens has become closer over time. As
of 2012, more than fifty percent of American households invest their money in the
stock market directly and/or indirectly. One incentive for people to invest in the
stock market, even if they were initially not interested in doing so, is that fact
that many companies provide 401(k) benefits. The American taxation system also
provides people with an incentive to invest their money in the stock market as a
2
form of a retirement plan. For 2011 and 2012, people can contribute as much as
5,000 dollars a year, which is tax deductible, to their traditional or Roth individual
retirement accounts (IRAs). However, the recent financial crisis had a significant
impact on such investments. According to Bricker et al. (2011), more than sixty
percent of families lost their wealth from 2007 to 2009; stocks demonstrated the
sharpest declines among their assets. Needless to say, the financial crisis since 2008
and its aftermath became a main concern politically and economically among many
citizens.
The second reason for the lack of research into the relationship between the
stock market and media relates to methodological reasons. Traditionally, scholars
employ a human-coding method when they conduct a content analysis of text data.
Manually coding the data is difficult because the researcher must deal with the significant size of data in the form of text files (e.g., newspaper articles or transcripts of
audio tapes). For instance, most previous studies that have included a content analysis based on newspaper articles have focused on articles from specific newspapers
within a very limited period of time. As a result, we do not understand the change
in news coverage for certain topics over the longer term.
However, scholars have recently begun to employ the text mining method,
which conducts content analysis using computer programs to overcome the limitations of the traditional content analysis methodology. This new approach enables
scholars to deal with the significant amount of text data at lower costs of resources
in terms of time and funds. For example, Ho and Quinn (2008) analyzed the media
context of more than 1,500 editorials in 25 major American newspapers from 1994
3
to 2004, which would be practically impossible using the traditional content analysis
method.
The objective of the current report is to analyze whether the stock market has
only been mentioned in economic news, or it also appears with other topics such as
politics. If the stock market became politically important, politicians would discuss
it more in their speeches, and newspapers would include more information about
it. At the same time, editorials would blame the president for declines in the stock
market. I collected stock market news reported in The New York Times from 1981 to
2011 and examined term frequencies and estimated the latent trend by year as well
as the latent topics in each year using two text mining methods: Wordfish (Slapin
and Proksch, 2008) and topic models (Blei and Lafferty, 2009).
4
Chapter 2
Background
Compared to political information, people have more opportunities to receive economic information regardless of their ability, opportunity, and motivation.1
The by-product theory of information suggests that people receive economic information through daily activities; for example, people recognize increased inflation
rates in gasoline prices (Popkin, 1991). Other studies have found that people recognize increases in the unemployment rate not only from personal experience, but
also from conversations with others (known as the interpersonal context) in terms of
co-workers’ layoffs, for example (Mutz, 1992).
Meanwhile, communication scholars have emphasized the importance of media
among the public. The media-dependency theory argues that the more individuals
rely on media, the more important the media have become to those individuals (BallRokeach and DeFleur, 1976). In the context of politics and economy, the public relies
on television and newspapers for economic information as understanding the economy
requires a variety of knowledge and information (MacKuen, Erickson and Stimson,
1992).
1
Regarding the role of motivation, opportunity, and ability in political knowledge, see Luskin
(1990).
5
First, economic voting literature shows that the electorate’s voting choice depends on the health of the U.S. national economy (Kinder and Kiewiet, 1979, 1981;
Kiewiet, 1983; MacKuen, Erickson and Stimson, 1992; Fiorina, 1981). Since the
seminal work by Kinder and Kiewiet (1979), many scholars have analyzed the economic determinants of voting choice among the electorate, focusing on whether the
change of personal economic situation affects voting choice (pocketbook hypothesis)
or the health of state/national economy affects it (sociotropic hypothesis). According to Lewis-Beck (2006), more than 400 articles about economic voting behaviors
have been published. In the U.S., more evidence has substantiated the sociotropic
hypothesis than the pocketbook hypothesis.
Second, if the electorate understands the condition of the national economy
accurately and without bias, what scholars need to do is simply focus on the relationship between the macro-economy indicators and election results. However, in reality,
media coverage about the national economy is neither accurate nor unbiased; rather,
the media have demonstrated a tendency to report more negative news than positive
(Harrington, 1989; Hetherington, 1996; Zaller, 1992), report more incumbent-friendly
coverage (Hofstetter, 1978; Page and Shapiro, 1992; Brody, 1991), and lean toward
specific parties (Conover, Feldman and Knight, 1987; Mutz, 1992; Jackman, 1993).
Leaning toward a specific party or ideology is especially evident in talk radio (Barker,
1998) and on cable channels (Della Vigna and Kaplan, 2007).
In addition, the media per se influence citizen perception in two ways: priming
and framing. Iyengar (1991) defined two types of framing effects: episodic and thematic. Episodic framing provides more individual-oriented coverage and attributes
6
personal responsibilities. For example, news coverage about poverty on TV tends to
use episodic framing, which implies that the viewer perceives poverty not as a social
issue, but as a personal problem. On the other hand, thematic coverage includes
more general, background information about the issue. In addition, viewers and
readers interpret coverage using thematic framing as a social issue; thus, people recognize the issue as a social problem and blame decision makers. Iyengar (1991) also
demonstrated that unemployment coverage on TV news tends to frame the content
using a thematic structure. In other words, media coverage of unemployment tends
to provide more general statistical information and social background than personal
stories of individuals. Although Iyengar (1991) only dealt with the coverage of unemployment, I assume that we can apply the same characteristics of media coverage for
other issues commonly covered, including the inflation rate and the GDP, as regular
government statistics related to these issues are similarly reported.
2.1
Analysis of News Coverage of the Stock Market
When scholars analyze the relationship between national economy and polit-
ical attitudes, they employ unemployment rate, inflation, and GDP. For example,
Nadeau et al. (1999) analyzed economic news coverage in the AP and Washington
Post from January 1, 1977, to July 1, 1995, but they focused on the unemployment
rate and inflation. Harrington (1989), on the other hand, analyzed TV networks’
economic coverage, using the unemployment rate, inflation rate, and growth rate of
real GNP. I argue that such previous studies focus less on the effect of media coverage
of the stock market and its relationship with politics because, for the majority of
7
Americans throughout history, the stock market per se did not personally affect them
whereas other indicators did. Prior to the 1990s, primarily rich people owned stocks,
and the condition of the stock market had less of an impact on Americans’ personal
financial conditions. In addition, people did not consider stock market conditions
when they evaluated the overall health of the national economy.
However, several factors have changed for personal finances, making the stock
market-related news relevant not only for the rich, but also for ordinary citizens in
the U.S. First, many more people have come to use stocks and mutual funds as
retirement assets. Many companies provide 401(k) and stock options as employee
benefits, while many individuals invest in stocks and mutual funds through IRAs.
The percentage of American families owning stock has risen from approximately 25%
in 1978 to 32% in 1989 and 51% in 2007 (Avery et al., 1984; Bucks et al., 2009).
This rapid increase occurred mainly with the entrance of the middle class into the
stock market during the 1990s. Similarly, the percentage of middle-income families
(in the 40 to 59.9 income percentile) owning stocks has risen from 29% in 1989 to
about 50% in 2007 (Bucks et al., 2009).
Second, during this same time period, economic news became more popular
in the U.S. The media environment in the U.S. has changed dramatically over the
last several decades. The decline of traditional media sources and the rapid increase
of new media are conspicuous: Newspaper circulation has declined as more people
watch the news on cable networks or via the Internet compared to network news. In
terms of economic news, people can receive more specific information at a lower cost
than ever before. For example, CNBC, a satellite and cable business news channel,
8
was established in 1989; by October, 2010 about 98 million households watched this
channel.2
In addition, the government regularly reports official unemployment and inflation rates, but not every day. In contrast, stock market indicators such as the
Dow Jones industrial average and S&P 500 constantly change, and the media report
on what is happening in the market at least once a day. Moreover, stories about the
stock market are mentioned in a variety of types of newspaper articles. People use
“Wall Street” to describe not only the stock market per se, but also politicians (e.g.,
“He is from Wall Street, not Main Street”) or culture (e.g., Gordon Gekko in the
movie, “Wall Street”).
http://tvbythenumbers.zap2it.com/2010/11/12/cable-news-ratings-for-thursday-november-112010/71919
2
9
Chapter 3
Empirical Study
3.1
Data
Many previous studies have used newspapers to analyze how the media de-
scribe the national economy and affects people’s understanding of the economy, presidential approval, and voter choice. For instance, Goidel and Langley (1995) examined
The New York Times front pages from 1982 to 1992 and found that the articles followed negative economic conditions more often than positive economic conditions.
In this respect, news coverage is linked to the public’s evaluation of the economy.
Although traditional media have been in decline over the last decade as fewer
people read newspapers or watch network news than they used to, traditional media’s influence is still significant. In addition, when we analyze the changes of news
coverage over time, it is better to analyze the content of traditional media than new
media because no data on new media from the 1980s exist. In addition, it is not
appropriate to analyze business newspapers or business cable channels because ordinary people do not use these sources as major news sources. Rather, it is important
to know how the traditional media describe the stock market because these sources
still reach a very large portion of the middle-class; indeed, during most of this time
frame (1981-2011) newspapers were still one of the most important means of acquir-
10
ing news. I chose to analyze The New York Times because it remained the premier
paper for political, economic, and business news during this time.
Using LexisNexis Academic and focusing on The New York Times Section A1
from 1981 to 2011, I collected 11,035 articles using three keyword phrases: “Wall
Street,” “stock market,” and “Dow Jones.” The top of Figure 3.1 shows some important trends. When the stock market declined dramatically, the number of articles
increased. For instance, the average number of articles was fewer than 150 each year
from 1981 to 1986. However, after Black Monday in 1987, the number increased
rapidly in 1987 and 1988. A similar trend occurred in 2008 and 2009 after the
financial crisis.
The New York Times features eight main desks: Editorial, Foreign, National,
Business/Financial, Metropolitan, Cultural, Style, and Sports.2 In general, economic
news is issued from the Business/Financial Desk while political news is issued from
National Desk. Therefore, if the stock market is purely economic news, the ratio of
articles from the Financial Desk should be higher than those from other desks. On
the other hand, if the stock market becomes political news, the ratio of articles from
the National Desk should increase over time. In order to verify whether or not this
trend exists in The New York Times, I sorted each article in Section A based on
desks; the bottom of Figure 3.1 shows the ratio of stock market articles. According
The New York Times is organized by sections; most business and economic news is located in
the Business Section. However, I only analyzed Section A for two reasons. First, Section A is the
main section; it includes the front page and the main articles. Second, the purpose of this study is
to analyze how the news of the stock market described in political contexts and political news is
located in Section A.
2
There are twenty three desks in total.
1
11
to this figure, stock market news is not only featured at the Financial Desk; all desks
discuss the stock market and Wall Street. In the 1980s, the Business/Financial Desk
(Financial Desk hereafter) and Editorial Desk reported stock market news. However,
the ratio of Financial Desk declined in the early 1990s while that of the National Desk
peaked in 1992 and 1994, which were both election years. In particular, 1992 was
the election year in which Clinton won against George H. Bush with the quote, “It’s
the economy, stupid.” Another peak period for the National Desk was in 2000 and
2003. It is possible that these peaks stem from the debate about the privatization of
social security and the capital gain tax, which was cut by the Bush administration
in 2003.
Although Figure 3.1 is useful in understanding the structure of stock-related
news over time, it does not provide any information regarding the kinds of terms
The New York Times chose and reported on, or the topics behind the terms. To
determine this information, first I counted each term appearing in the paragraphs
that included three keywords. Second, using Wordfish, I estimated the latent trend of
articles over time. Finally, I estimated the latent topics of stock market paragraphs
using topic models.
12
13
Figure 3.1: (Top) Numbers of Stock Market Articles in The New York Times from 1981 to 2011. (Bottom)
Percentage of “Wall Street” Articles in terms of “Desk” from 1981 to 2011
3.2
Method: Wordfish and Topic Models
Content analysis has been widely used to study newspaper articles. Tradi-
tionally, researchers have employed a manual coding method - a methodology in
which researchers create rules for coding in advance and several researchers on the
team code each article manually. The advantage of coding by hand is that the researcher can design the original coding scheme as appropriate to the research goal.
However, a major drawback to this method is that the quality of the research relies
heavily on each coder’s objectivity and ability. In addition, it is difficult to deal
with a large dataset with this method. Therefore, in previous studies, researchers
only analyzed newspapers within short periods, such as analyzing newspaper articles
during campaign periods or a specific area.
Recent computer software improvements for content analysis have decreased
the cost involved in analyzing text-oriented data. According to Quinn et al. (2010),
content analysis includes five methods: reading, human coding, dictionaries, supervised learning, and topic models (the last three methods use computer software for
coding text data). As an example of the dictionary method, scholars have used InfoTrend (Fan, 1988) for political science content analyses since the 1990s (Shah et al.,
1999; Nadeau et al., 1999). Using InfoTrend, researchers prepare the lexicon that
includes the related terms and keywords as well as word relationship rules. In the
case of Nadeau et al. (1999), who analyzed presidential campaign articles for 1984,
1988, 1992, and 1996, the researcher created the rules that allowed the program to
sort each paragraph based on whether it is for or against the candidates. At the
same time, some coders randomly code-select paragraphs and compare them with
14
the results from the program.
Compared to other coding methods, topic models (Blei and Lafferty, 2009)
and Wordfish (Slapin and Proksch, 2008) allow scholars to analyze a large amount of
text data at a low cost (Ho and Quinn, 2008; Quinn et al., 2010; Lowe, 2008; Slapin
and Proksch, 2008). Using these methods, scholars do not need a dictionary or
code scheme. The significant difference between the previous method and the topic
model is that the computer counts a frequency of certain terms in each document
and locates them in one or more dimensions. For example, Slapin and Proksch
(2008) used the Wordfish package in R, which does not rely on human coding, but
counts the frequencies of the terms in articles within groups (e.g., as they manifest
in each political party), then estimates each document’s position. The advantage of
Wordfish is that the procedure is completely automated, thereby enabling researchers
to analyze large numbers of articles.
In this report, I initially employ Wordfish by Slapin and Proksch (2008) for
two reasons. First, the purpose of this study is to understand whether the stock market has been presented in the newspaper framed as a political issue. If so, I assume
that we might see large frequencies of political terms within the same paragraph
that include specific keywords (“Wall Street,” “stock market,” and “Dow Jones”).
The Wordfish methodology thus supports this study. Second, I use three decades of
newspaper articles in the same study. Therefore, a completely automated method
is necessary both to retain objectivity and to keep the time costs of research low
enough.
15
3.2.1
Assumption of Wordfish
Wordfish assumes that each word’s frequency has Poisson distribution. That
is:
yij ∼ P oisson(λij )
(3.1)
λij = exp(αi + ψj + βj ∗ ωi )
(3.2)
where yij is the count of word j in year i’s The New York Times articles, α is a set of
year fixed effects, ψ is a set of word fixed effects, β is the estimate of a word-specific
weight capturing the importance of word j in discrimination between years, and ω is
the estimate of year i’s position. In other words, if ω is the same between year i and
year i+1, it means that the stock market news is very similar between those years.
On the other hand, if ω changed over the years, it means that the stock market news
changed during that period. That is, ω and β allow us to understand which words
differentiate stock market news between years.
3.2.2
Process
To use the news articles in this computer software, we have to convert the
articles into computer-friendly text data.
Step 1: Extract only paragraphs that include keywords
In this method, the unit of analysis is each term. Thus, it is important to
eliminate the parts of the articles that are not important for the analysis. The goal
of this paper is to analyze the context through which the newspaper has described
16
the stock market. Although each article includes at least one of the three keywords
(“Wall Street,” “stock market,” or “Dow Jones”), the initial search does not reveal
which paragraph is related to the stock market. Even if the rest of the article is
not related to the stock market, the computer counts the frequency of each term in
each article, which would bias the results. To counteract this potential for bias, I
extracted each paragraph that includes the keyword(s) and created a single text file
for each year from 1981 to 2011.
Step 2: Convert the paragraphs to computer-friendly text data files
After making 31 text files, I converted them to a computer-friendly text data
file. In much of the text-mining software, the data file should be in a corpus format.
Using the package tm (Feinerer, 2010) in R, I removed numbers, punctuation, and
common words in English (e.g., is, you, me). Next, using Snowball (Hornik, 2009),
a stemmer package in R, all terms were converted to stems.3
I need to note that I did not use entire articles for my analyses. Rather, I
collected articles containing at least one of three keywords using LexisNexis, extracted
paragraphs that included the keywords, and created new documents by year. In many
cases, a single article included many topics. Suppose there are fifteen paragraphs in
one article and “stock market” is only mentioned in the last paragraph. If I want to
know the kinds of terms with which “stock market” is mentioned, it is appropriate
to analyze only the last paragraph; if I analyze the entire article, there will be many
terms that may not be used in relation to the keyword “stock market.” As a result,
3
For example, higher and highest become high.
17
I may not be able to find terms and topics related to the keyword.
3.3
Topic Models
Although the Wordfish method is simple and easy to employ via the software,
some limitations arose for the current study. First, we do not know what the Wordfish
score for ω means intuitively. If each article were categorized by topics in advance, ω
would show the change in location of topics over time. For example, Proksch, Slapin
and Thies (2011) estimated the party position in Japan using newspaper articles from
1960 to 1998. Before estimating the Wordfish score for ω or party location, Proksch,
Slapin and Thies (2011) sorted each document according to three topics: domestic
and social policy, economic policy, and foreign policy. However, in the current study,
I do not categorize each paragraph and document by topics. Rather, my goal is to
find the relationship between stock market terms and political terms without using
a pre-coding process. Because each document is a corpus of paragraphs in year i,
Wordfish estimates document i’s one-dimensional location over time. In other words,
we can see the “change” in documents over time but we do not know what it means.
On the other hand, topic models via Latent Dirichlet Allocation (LDA) (Blei
and Lafferty, 2009) estimate the probabilistic distribution of terms and find latent
topics behind terms used in documents. LDA assumes that we observe word wd
in document d, where there are latent topics k and the distribution of topic k is
described as βk . The proportion of topics for the dth document is expressed as θd .
Finally, topic assignments for the dth document are zd . That is:
18
p(β1:K , θ1:D , z1:D , w1:D ) =
K
�
p(βi )
i=1
�
N
�
n=1
19
D
�
d=1
p(θd )
�
p(zd,n |θd )p(wd,n |β1:K , zd,n )
Chapter 4
Results
4.1
Term Frequencies
This section discusses the frequency of important terms in my data. The data
provided more than 20,000 words in the paragraphs related to the stock market from
1981 to 2011. However, many words only appeared a few times in certain years,
making their calculation inefficient. For this reason, I removed all the words from
the files with a zero word count in 95% of the documents.
4.1.1
Wall Street, Stock Market, and Dow Jones
Figure 4.1 shows the number of the three keyword phrases (“Wall Street,”
“stock market,” and “Dow Jones”) from 1981 to 2011. “Wall Street” has the highest
frequency in The New York Times over this entire time period. Although “Dow
Jones” indicated no specific trend, two of the keyword phrases (“Wall Street” and
“stock market”) demonstrated the same trend: Their frequency rose in 1987, 2002,
and 2008. The reason for the increase in 1987 was Black Monday, which occurred in
October of that year. The frequency of these two phrases declined during the early
1990s, then increased again during the late 1990s, with two peaks in 1998 and 2002.
The numbers declined again after 2003 until the financial crisis of 2008.
20
Figure 4.1: Frequency of “Stock Market,” “Wall Street,” and “Dow Jones” in The
New York Times from 1981 to 2011
4.1.2
Presidential candidates
Figure 4.2 shows the frequencies of terms related to the presidential candi-
dates. Except in 1987, The New York Times did not use “president” in relation to
the stock market from 1981 to 1990. However, the frequency of “president” increased
gradually during the 1990s, peaking in 2002. It fell again in the mid-2000s and increased again in 2008. This trend could be related to the decline of the stock market.
Obviously, the peak in 1987 stems from Black Monday and the peak in 2008-09 is
related to the financial crisis. On the other hand, presidential candidates’ names
were used in election years in combination with stock market terms. Two spikes
occurred during the 2000s - in 2000 and 2002 - which might be related to the debate
about the privatization of social security during the elections. These data provide
21
strong evidence that stock news became political news in the 2000s. The financial
crisis and the bailout of the financial sector provide additional evidence as well.
Figure 4.2: Frequency of Presidential Candidates in The New York Times from 1981
to 2011
4.1.3
Congress, Political Parties
The trends related to Congress and the political parties are similar to those
related to presidents’ names. Four spikes occurred over the three decades: 1987,
2002, 2008, and 2010. In 1987, “Congress” appeared 56 times, while it appeared 47,
65, and 62 times in 2002, 2008, and 2010, respectively. On the other hand, specific
party names did not appear many times during the 1980s but they increased after
the 1990s. An interesting phenomenon is that, during the off-peak period, no gap of
usage occurred between Congress and party names. However, during the four spikes,
22
the gap increased significantly. Moreover, party names appeared more often than
usual in 2002, 2008, and 2010 while “Congress” was used more than party names in
1987. From this phenomenon, I find that stock market news has become partisan
since the 2000s.
Figure 4.3: Frequency of “Congress” and Political Parties in The New York Times
from 1981 to 2011
4.1.4
Policy-related Words
Public policies were also mentioned as related topics of the stock market in the
newspaper articles. Figure 4.4 shows the frequencies of five keywords (i.e., bailout,
mortgage, retire, social security, tax) within this period. “Tax” was mentioned more
than the other terms, especially when the stock market declined in 1987, 2002, and
2008. Obviously, “mortgage” spiked after the financial crisis in 2008 due to the sub23
prime problems. Although I expected the frequencies of “social security” and “retire”
to increase in 2000 due to the debate about the privatization of social security, no
significant trend related to these terms emerged. Based on these results, I find that
Figure 4.4: Frequency of Policy Terms in The New York Times from 1981 to 2011
the stock market has been used to describe the economy in The New York Times at
an increasing rate since the 1990s. More importantly, it was mentioned in an effort
to discuss not only economic issues, but also political issues. In particular, when
the stock market fell significantly, the media mentioned the stock market in terms
of social and political issues.
These results indicate that the stock market was described more often after
1990 than during the 1980s. More importantly, it was mentioned not only in terms
of the economy, but also in relationship to political issues. In particular, the stock
24
market is currently an important issue not only for stockowners, but also for nonstockowners as it is related to their retirement and mortgage.
4.2
Wordfish Estimation
Wordfish calculates latent positions of each term (Figure 4.5) and document
(Figure 4.6). Table 4.11 and Figure 4.5 show the estimated location of terms and
their plots, the Wordfish estimation of each word j’s location β and fixed effect ψ.
The high score of ψ (y axis) means that the word’s frequency is high. On the other
hand, if the word only occurred during certain years, then the word requires a larger
absolute number of β (x axis). Because all of the paragraph data include any of the
three keyword phrases, their ψ is high. On the other hand, terms related to Congress
and political parties appeared over time. In terms of policy-related terms, “bailout”
was used in certain years while “tax” and “pension” were commonly used from 1981
to 2011.
If many political terms do not distinguish the year, which terms differentiate
stock market news between years? To answer this question, first I checked terms with
large β and small ψ. This category includes many IT-related terms such as “Facebook” and “Twitter.” It also includes political terms like “occupy” and “debtceil.”
How about low β and low ψ terms? Many of these are business-related terms such
as “drysdale (Drysdale Government Securities)” and “conoco.” Although “regan” is
also in this category, no other political or social terms are included.
1
Table 4.1 shows only the β and ψ from terms in Figure 4.1 to 4.4
25
From these arguments, I find that, although presidential names distinguish
stock market news by year, common political terms such as “Congress” or “president”
do not. Rather, in the early 1980s, business and economic terms distinguish these
years from stock market news in other periods. Finally, news about “occupy (Wall
Street)” and “debt ceilings” are uniquely related to the stock market in the 2010s.
Table 4.1: the Estimated β and ψ from Select Terms in Figures 4.1 to 4.4
Term
β
ψ
wall
0.615 5.593
street
0.609 5.673
stock
0.119 5.432
dow
0.133 3.777
congress
0.459 2.497
republican 0.733 2.582
democrat
0.754 2.677
reagan
-2.494 0.778
clinton
0.146 2.185
bush
0.427 2.857
obama
2.399 -0.363
mccain
1.375 0.074
mondale
-2.633 -3.690
president
0.230 1.594
dukakis
-1.009 -3.347
kerry
0.643 -3.944
gore
-0.138 0.916
tax
0.330 3.425
retire
0.529 0.277
bailout
1.484 0.998
mortgage
0.704 -2.606
social
0.196 2.675
security
0.277 0.787
pension
0.485 1.981
26
Figure 4.5: the Estimated Location of Terms in The New York Times from 1981 to
2011
27
Figure 4.6: the Estimated Location of Stock Market Paragraphs in The New York
Times from 1981 to 2011
Wordfish also estimates the location of newspaper articles ωi by year (Figure
4.6). In this case, ω shows the relative locations of stock market articles in each
year. There was no significant movement from 1981 to around 2000, while the slope
became sharper after 2000. This means that the contents of stock market articles
are similar from 1981 to around 2000 but changes year-by-year after 2000.
28
4.3
Topic Model Estimation
In order to find latent topics in stock market paragraphs, I used topicmodels
package in R (Gruen and Hornik, 2011). Table 4.2-4.7 are the results of LDA with
Gibbs sampling. I selected five topics in each year and each topic is expressed by
five terms. Because all paragraphs include at least one of three key words (“Wall
Street,” “stock market,” “Dow Jones”), it is obvious that many topics are related to
these keywords. On the other hand, if there are any political terms in Table 4.2-4.7,
then the stock market is mentioned in political topics.
From these results, I find that there are some trends in topics, in terms of
the time period. First, from the late 1980s to the end of 1990s, most topics are
about the stock market per se or economy, while some are about the “president”
or “government.” Second, the stock market was described with the topic of Social
Security in 1998, 1999, and 2000. Interestingly, there are no political terms in other
topics within these three years. Topic 3 in 2000 consists of “social,” “security,”
“money,” “stock,” and “Bush.” It is clear that stock market news became Social
Security news because George W. Bush proposed a privatization of Social Security
in the 2000 presidential election. Third, after the financial crisis in 2008, stock market
news were included in a variety of topics, such as specific financial company’s name
(e.g., “Lehman Brothers,” “Goldman Sachs”), protest movements (e.g., “Occupy
Wall Street”), and topics related to the financial crisis in 2008 (e.g., “bailout”).
These results are consistent with the results shown in Figure 3.1. That is,
stock market news used to be mainly economic news during the 1980s but became
political, policy-related news in later years. In particular, the privatization of social
29
security and the financial crisis in 2008 had significant impacts toward these changes.
30
Table 4.2: Five Topics in the Stock Market Paragraphs from 1981 to 1985
Year
1981
1982
1983
1984
1985
Topic 1
street
company
investment
billion
share
street
prices
service
cities
reagan
oil
bank
million
investors
yen
page
months
financial
hong
price
market
time
industry
dow
close
Topic 2
president
time
republican
administration
house
wall
securities
million
government
chase
stock
market
percent
jones
dow
wall
street
firm
bank
people
street
wall
analysts
investment
president
Topic 3
wall
street
rates
streets
money
stock
market
rates
dow
average
street
wall
company
investment
economic
public
federal
service
billion
power
company
wall
government
chief
shares
31
Topic 4
stock
market
dow
jones
yesterday
economic
president
markets
company
budget
time
administration
inflation
president
week
economy
percent
reagan
news
president
wall
street
companies
yesterday
million
Topic 5
economic
reagan
financial
program
budget
street
wall
york
economy
tax
government
industrial
rates
tax
commercial
stock
market
rates
prices
dow
stock
chairman
regan
exchange
banks
Table 4.3: Five Topics in the Stock Market Paragraphs from 1986 to 1990
Year
1986
1987
1988
1989
1990
Topic 1
trading
boesky
exchange
insider
investors
markets
percent
dow
dollar
average
wall
street
firms
investment
streets
market
stock
trading
markets
investors
street
firm
company
executives
american
Topic 2
Topic 3
street
wall
wall
street
company
york
million
avenue
securities properties
wall
president
street
york
people
house
firms
reagan
securities
world
stock
october
market
crash
trading
collapse
york
firm
index
house
percent
crash
rates
yesterday
tax
october
average
economic
dow
bush
york
stock
people
market
million
percent
federal
markets
economy
rates
32
Topic 4
Topic 5
yesterday
stock
stocks
market
jones
markets
dow
financial
average
economic
stock
deficit
market
budget
trading
economy
investors
american
plunge
federal
company
markets
market
percent
billion
dow
securities
jones
companies
average
company
wall
street
street
investment
business
million
securities
economy
people
wall
financial
street
government
drexel
business
securities
money
firms
city
Table 4.4: Five Topics in the Stock Market Paragraphs from 1991 to 1995
Year
1991
1992
1993
1994
1995
Topic 1
Topic 2
street
market
wall
investors
firm
trading
salomon yesterday
company executives
wall
street
company
wall
financial
million
federal
firms
securities
city
street
wall
business
financial
company president
firms
economic
city
money
wall
street
million
wall
street
firms
real
company
firm
financial
percent
stock
economy
market
bond
stocks
investors
trading
billion
shares
Topic 3
Topic 4
Topic 5
wall
stock
city
president
dow
investment
people
percent
business
government
jones
bond
rating
market
financial
business
street
stock
economic
investment
market
clinton
billion
percent
government companies
economy
york
american
dow
street
market
stock
investment
percent
markets
firm
companies
york
public
million
day
house
jones
clinton
percent
investment
stock
bond
fund
market
markets
companies
trading
rates
money
prices
investors
billion
american
companies
wall
markets
dow
street
government
jones
company
federal
average
million
mexico
financial
analysts
mexican
33
Table 4.5: Five Topics in the Stock Market Paragraphs from 1996 to 2000
Year
1996
1997
1998
1999
2000
Topic 1
markets
economy
money
economic
federal
markets
financial
currency
american
economic
company
investment
business
banks
bank
percent
dow
investors
stocks
jones
percent
dow
stocks
jones
average
Topic 2
company
time
investment
million
mutual
wall
street
york
company
business
percent
market
dow
stocks
jones
stock
market
rates
economic
growth
market
stock
economy
prices
rates
Topic 3
Topic 4
Topic 5
wall
market
percent
street
stock
dow
financial
funds
jones
people
companies
stocks
city
government
average
stock
economy
percent
market
rates
dow
investors
federal
average
money
companies
stocks
government greenspan
jones
stock
stock
wall
market
markets
street
social
investors
billion
government
economy
analysts
security
financial
city
wall
market
companies
street
social
markets
york
investment
street
banks
money
company
time
security
day
security
wall
street
social
street
wall
money
companies
york
stock
financial
public
bush
company
million
34
Table 4.6: Five Topics in the Stock Market Paragraphs from 2001 to 2005
Year
2001
2002
2003
2004
2005
Topic 1
Topic 2
Topic 3
street
percent
wall
york
financial
street
city
dow
companies
center
day
investors
wall
index
analysts
market
percent
corporate
stock
company
bush
money
markets
economy
tax
dow
president
people
shares
economic
street
percent
tax
wall
companies
city
exchange
york
president
firms
average
health
million
grant
budget
stock
street
financial
market
million
economic
percent
wall
time
investors
firms
federal
dow
executives companies
wall
morgan
percent
street
company
financial
time
investment
stock
people
executive
markets
job
billion
average
35
Topic 4
Topic 5
stock
wall
market
president
economy
bush
markets
government
economic
money
wall
stock
street
investors
companies
enron
firms
financial
analysts
time
wall
stock
street
market
investors
markets
war
time
house
money
york
wall
company
street
photo
analysts
city
money
fund
bank
day
market
president
stock
security
investors
government
quarter
bush
price
Table 4.7: Five Topics in the Stock Market Paragraphs from 2006 to 2011
Year
2006
2007
2008
2009
2010
2011
Topic 1
street
wall
goldman
firms
time
wall
million
companies
billion
investment
crisis
economic
bailout
plan
mccain
obama
economy
economic
people
public
wall
street
money
people
industry
stock
market
percent
investors
markets
Topic 2
Topic 3
Topic 4
Topic 5
wall
stock
dow
investment
street
market
chief
funds
business
percent
average
capital
house
investors
executive
company
york
markets
executives
fund
york
street
stock
wall
people
analysts
market
street
business economy
percent
investors
page
firms
markets
mortgage
news
cut
dow
funds
york
economy
percent
banks
home
billion
markets
firms
industry
people
investors
credit
main
business
dow
government
brothers american
day
bank
york
stock
bonuses
financial
business
market
executives
banks
life
percent
goldman
money
company markets
bank
government
president investors
executive
firms
wall
market
street
york
street
stock
financial
business
banks
percent
obama
investment
financial
federal
democrats
president
goldman investors government
firm
wall
street
occupy
investment
street
wall
street
former
financial
people
wall
business
crisis
protests
protesters
fund
banks
obama
york
american
36
Chapter 5
Conclusion
Many historical documents have been recently converted to digital files. For
example, we are able to analyze the frequency of terms in many books from 1800
to the present via Google.1 With the increase in digital text data, the potential for
computer-driven content analysis has been extended.
Through the use of digital text data, this report aims to analyze how the
stock market has been described in traditional media over the last three decades. In
particular, I focus on the political keywords included in the stock market articles using three unique methods. First, I analyzed newspaper article contents from over the
last three decades (about 11,000 articles) in Section A of The New York Times. A
text mining computer program allowed me to analyze this large text-based dataset,
which would have been essentially impossible if manual coding were necessary. Second, I analyzed not only the articles that were categorized as business and economic
articles, but also all of the articles that mentioned the stock market in Section A.
In other words, I examined the context of the stock market not only as an economic
issue, but also as a political or a cultural issue. Finally, using Wordfish and LDA,
The result with “Wall Street” is: http://books.google.com/ngrams/graph?content=Wall+
Street&year_start=1800&year_end=2008&corpus=0&smoothing=0
1
37
I estimated the relative locations of each term and latent topics in stock market
paragraphs during the last three decades.
According to the two estimation methods, I find some characteristics of stock
market news during the last three decades. First, in terms of quantity of news,
there are two significant events that increase the quantity of news: a decline in
the stock market and policy proposals by election candidates. In particular, Black
Monday of 1987 and the financial crisis of 2008–2009 had strong impacts on increasing
discussion on the topic of the stock market and economy. Additionally, the debate
about privatization of Social Security made stock market news into a political topic.
The bailout of the financial industry after the financial crisis also extended a range
of topics. That is, stock market news consists of not only Wall Street topics, but also
topics about the president, political parties, policies, social movements, and ordinary
citizens—main street.
Finally, in order to extend this study, I have to overcome some issues. First, I
would apply this analysis to other media. One possibility is to analyze other national
and local newspapers. The other possibility is to analyze other media, such as radio
and TV. In that case, I should obtain transcripts of the media that I will analyze.
Second, even if I can obtain recent newspapers or transcripts as digital data, there is
no guarantee on how long we will be able to go back in history. For example, we can
obtain USA Today articles from 1989, while NBC news articles are available from
1997 via LexisNexis. If we want to examine the relationship between stock market
news and political news, we need data from a longer time period.
38
Bibliography
Avery, Robert B., Glenn B. Canner, Gregory E. Elliehausen and Thomas A.
Gustafson. 1984. “Survey of Consumer Finances, 1983.” Federal Reserve Bulletin
70:679–692.
Ball-Rokeach, Sandra and Melvin DeFleur. 1976. “A Dependency Model of MassMedia Effects.” Communication Research 3(1):3–21.
Barker, David C. 1998. “Rush to Action: Political Talk Radio and Health Care
(un)Reform.” Political Communication 15(1):83–97.
Blei, David M. and John. D. Lafferty. 2009. Topic Models. In Text Mining: Classification, Clustering, and Applications, ed. Ashok Srivastava and Mehran Sahami.
London, England: Taylor and Francis chapter 4, pp. 71–94.
Bricker, Jesse, Brian Bucks, Arthur Kennickell, Traci Mach and Kevin Moore. 2011.
“Surveying the Aftermath of the Storm: Changes in Family Finances from 2007
to 2009.”.
Brody, Richard A. 1991. Assessing teh President: The Media, Elite Opinion, and
Public Support. Palo Alto, CA: Stanford University Press.
Bucks, Brian K., Arthur B. Kennickell, Traci L. Mach and Kevin B. Moore. 2009.
“Changes in U.S. Family Finances from 2004 to 2007: Evidence from the Survey
of Consumer Finances.” Federal Reserve Bulletin 95:A1–A55.
39
Conover, Pamela J., Stanley Feldman and Kathleen Knight. 1987. “The personal and
Political Underpinnings of Economic Forecasts.” Journal of Politics 31(3):559–583.
Della Vigna, Stefano and Ethan Kaplan. 2007. “The Fox News Effect: Media Bias
and Voting.” Quarterly Journal of Economics 122(3):1187–1234.
Fan, David P. 1988. Predictions of Public Opinion from the Mass Media: Computer
Content Analysis and Mathematical Modeling. Westport, CT: Greenwood Press.
Feinerer, Ingo. 2010. Text Mining Package. R package version 0.5-4.1.
Fiorina, Morris P. 1981. Retrospective Voting in American National Elections. New
Heaven, CT: Yale University Press.
Goidel, Robert K. and Ronald E. Langley. 1995. “Media Coverage of the Economy
and Aggregate Economic Evaluations: Uncovering Evidence of Indirect Media
Effects.” Political Research Quarterly 48(2):313–328.
Gruen, Bettina and Kurt Hornik. 2011. “topicmodels: An R Package for Fitting
Topic Models.” Journal of Statistical Software 40(13):1–30.
Harrington, David E. 1989. “Economic News on Television: The Determinants of
Coverage.” Public Opinion Quarterly 53(1):17–40.
Hetherington, Marc J. 1996. “The Media’s Role in Forming Voters’ National Economic Evaluations in 1992.” American Journal of Political Science 40(2):372–395.
Ho, Daniel E. and Kevin M. Quinn. 2008. “Measuring Explicit Positions of Media.”
Quarterly Journal of Political Science 3(4):353–377.
40
Hofstetter, Richard C. 1978. “News bias in the 1972 campaign: A cross-media
comparison.” Journalism Monographs 58.
Hornik, Kurt. 2009. Snowball: Snowball Stemmers.
Iyengar, Shanto. 1991. Is Anyone Responsible?: How Television Frames Political
Issues. Chicago, IL: University Of Chicago Press.
Jackman, Simon. 1993. “Economic Expectations, Realizations, and Presidential Approval.” Presented at the annual meeting of the Midwest Political Science Association, Chicago, IL.
Kiewiet, Roderick. 1983. Macroeconomics and Micropolitics: Electoral Effects of
Economic Issues. Chicago, IL: University of Chicago Press.
Kinder, Donald R. and Roderick Kiewiet. 1979. “Economic Discontent and Political
Behavior: The Role of Personal Grievances and Collective Economic Judgments
in Congressional Voting.” American Journal of Political Science 23(3):495–527.
Kinder, Donald R. and Roderick Kiewiet. 1981. “Sociotropic Politics: The American
Case.” British Journal of Political Science 11:129–161.
Lewis-Beck, Michael S. 2006. “Does Economics Still Matter? Econometrics and the
Vote.” Journal of Politics 68(1):208–212.
Lowe, Will. 2008. “Unerstanding Wordscores.” Political Analysis 16(4):356–371.
Luskin, Robert C. 1990. “Explaining Political Sophistication.” Political Behavior
12(4):331–361.
41
MacKuen, Michael B., Robert S. Erickson and James A. Stimson. 1992. “Peasants
or Bankers? The American Electorate and the U.S. Economy.” American Political
Science Review 86(3):597–611.
Mutz, Diana C. 1992. “Mass Media and the Depoliticization of Personal Experience.”
American Journal of Political Science 36(2):483–508.
Nadeau, Richard, Richard G. Niemi, David P. Fan and Timothy Amato. 1999. “Elite
Economic Forcasts, Economic News, Mass Economic Judgments, and Presidential
Approval.” Journal of Politics 61(1):109–135.
Page, Benjamin I. and Robert Y. Shapiro. 1992. The Rational Public: Fifty Years of
Trends in Americans’ Policy Preferences. Chicago, IL: The University of Chicago
Press.
Popkin, Samuel L. 1991. The Reasoning Voter: Communication and Persuasion in
Presidential Campaigns. Chicago, IL: The University of Chicago Press.
Proksch, Sven-Oliver, Jonathan B. Slapin and Michael F. Thies. 2011. “Party system
dynamics in post-war Japan: A quantitative content analysis of electoral pledges.”
Electoral Studies 30(1):114–124.
Quinn, Kevin M., Burt L. Monroe, Michael Colaresi, Michael H. Crespin and
Dragomir R. Radev. 2010. “How to Analyze Political Attention with Minimal
Assumption and Costs.” American Journal of Political Science 54(1):209–228.
42
Shah, Dhavan V., Mark D. Watts, David Domke, David P. Fan and Michael Fibison.
1999. “News Coverage, Economic Cues, and the Public’s Presidential Preferences,
1984-1996.” Journal of Politics 61(4):914–943.
Slapin, Jonathan B. and Sven-Oliver Proksch. 2008. “A Scaling Model for Estimating
Time-Series Party Positions from Texts.” American Journal of Political Science
52(3):705–722.
Vavreck, Lynn. 2009. The Message Matters: The Economy and Presidential Campaigns. Princeton, New Jersey: Princeton University Press.
Zaller, John R. 1992. The Nature and Origins of Mass Opinion. New York, NY:
Cambridge University Press.
43