Download Data revisions and consequences 09May

DATA REVISIONS AND THEIR
CONSEQUENCES
Introduction
• Real Time Data sets
– Real-Time Data Set for Macroeconomists
• Philadelphia Fed + University of Richmond
– Need for good institutional support
– Club good: non-rival but excludable
Introduction
• Data sets
– Unrestricted access:
• U.S.: Philadelphia Fed, St. Louis Fed, BEA
• OECD
• Bank of England (recently updated)
– Restricted access:
• EABCN
– Fate unclear:
• Canada
– One-time research projects:
• Many, most not continuously updated
Introduction
• Analysis of data revisions is not criticism of
government statistical agencies!
– May help agencies improve data production
process
– Revisions reflect limited resources devoted to
data collection
– Revised data usually superior to unrevised
data (U.S. CPI vs. PCE price index)
Introduction
• Structure of data sets
– The data matrix
• Columns report vintages (dates on which data
series are observed)
• Rows report dates for which economic activity is
measured
• Moving across rows shows revisions
• Main diagonal shows initial releases
• Huge jumps in numbers indicate benchmark
revisions with base year changes
Vintage:
Date
47Q1
47Q2
47Q3
.
.
.
65Q3
65Q4
66Q1
.
.
.
07Q1
07Q2
07Q3
07Q4
11/65
REAL OUTPUT
2/66
5/66 . . .
11/07
2/08
306.4
309.0
309.6
.
.
306.4
309.0
309.6
.
.
306.4
309.0
309.6
.
.
.
.
1570.5
1568.7
1568.0
.
.
1570.5
1568.7
1568.0
.
.
.
.
.
.
.
613.0
621.7
NA
.
.
.
NA
NA
NA
NA
613.0
624.4
633.8
.
.
.
NA
NA
NA
NA
3214.1
3291.8
3372.3
.
.
.
11412.6
11520.1
11630.7
NA
3214.1
3291.8
3372.3
.
.
.
11412.6
11520.1
11658.9
11677.4
.
609.1
NA
NA
.
.
.
NA
NA
NA
NA
...
...
...
...
...
...
.
.
.
...
...
...
...
Data Revisions
Data Revisions
•
•
•
•
•
What Do Data Revisions Look Like?
Are They News or Noise?
Is the Government Using Information Efficiently?
Are Revisions Forecastable?
How Should We Model Data Revisions?
• Key issue: are data revisions large enough
economically to worry about?
Data Revisions
• What Do Data Revisions Look Like?
– Short Term (example)
– Long Term (example)
• What Do Different Types of Data
Revisions Look Like?
– Short run revisions based on additional
source data
– Benchmark revisions based on structural
changes or updating base year
Figure 1
Real Consumption Growth for 1973Q2
(as viewed from the perspective of 138 different vintages)
1.0
Percent
0.5
0.0
-0.5
-1.0
-1.5
1973
1976
1979
1982
1985
1988
1991
Vintage
1994
1997
2000
2003
2006
Table 2
Average Growth Rates of Real Consumption Over Five Years
Benchmark Vintages
Annualized percentage points
Vintage Year:
Period
49Q4 to 54Q4
54Q4 to 59Q4
59Q4 to 64Q4
64Q4 to 69Q4
69Q4 to 74Q4
74Q4 to 79Q4
79Q4 to 84Q4
84Q4 to 89Q4
89Q4 to 94Q4
94Q4 to 99Q4
‘75
‘80
‘85
‘91
‘95
’99
’03
‘07
3.6
3.4
4.1
4.5
2.3
NA
NA
NA
NA
NA
3.3
3.3
3.8
4.3
2.6
4.4
NA
NA
NA
NA
3.3
3.3
3.8
4.4
2.6
4.4
2.8
NA
NA
NA
3.7
3.3
3.7
4.4
2.5
3.9
2.5
3.2
NA
NA
3.9
3.4
3.8
4.5
2.6
3.9
2.5
3.1
2.3
NA
3.8
3.5
4.0
4.8
2.8
4.1
2.6
3.4
2.1
NA
3.8
3.5
4.1
4.8
2.8
4.2
2.8
3.7
2.4
4.0
3.8
3.5
4.1
4.8
2.9
4.1
2.9
3.7
2.6
4.1
Data Revisions
• Are Data Revisions News or Noise?
– Data Revisions Add News: Data are optimal
forecasts, so revisions are orthogonal to early
data; revisions are not forecastable
– Data Revisions Reduce Noise: Data are
measured with error, so revisions are
orthogonal to final data; revisions are
forecastable
Data Revisions
• Are Data Revisions News or Noise?
– Mankiw-Runkle-Shapiro (1984): Money data
revisions reduce noise
– Mankiw-Shapiro (1986): GDP data revisions
contain news
– Mork (1987): GMM results show “final” NIPA
data contain news; other vintages are
inefficient and neither noise nor noise
– UK: Patterson-Heravi (1991): revisions to
most components of GDP reduce noise
Data Revisions
• Is the Government Using Information
Efficiently?
• Theoretical Issue: Should the government
report its sample information or project an
unbiased estimate using extraneous
information?
Data Revisions
• Is the Government Using Information
Efficiently?
• Key Issue: What is the trade-off the
government faces between timeliness and
accuracy?
– Zarnowitz (1982): evaluates quality of
different series
– McNees (1989): found within-quarter
estimate of GDP to be as accurate as
estimate released 15 days after quarter end
Data Revisions
• Findings of bias and inefficiency of seasonally
revised data
– Kavajecz-Collins (1995)
– Swanson-Ghysels-Callan (1999)
• Revisions to seasonals may be larger than
revisions to NSA data: Fixler-Grimm-Lee (2003)
• Key question: Are seasonal revisions
predictable? Who cares if that is an artifact of
construction?
Data Revisions
• Key Issue: If early government data are
projections, then state of business cycle
may be related to later data revisions.
– Dynan-Elmendorf (2001): GDP is misleading
at turning points
– Swanson-van Dijk (2004): volatility of
revisions to industrial production and producer
prices increases in recessions
Data Revisions
• Are Revisions Forecastable?
– Conrad-Corrado (1979): use Kalman filter to
improve government’s monthly data on retail
sales
– Aruoba (2008): revisions to many U.S.
variables are forecastable
Data Revisions
• Are Revisions Forecastable?
– Key Issue: can revisions be forecast in realtime (or just ex-post)?
• Guerrero (1993): combines historical data with
preliminary data on Mexican industrial production
to get improved estimates of final data
• Faust-Rogers-Wright (2005): Examines G-7
countries’ output forecasts; find Japan & U.K.
output revisions forecastable in real time
Forecasting
Forecasting
• Forecasts are only as good as the data
behind them
• Literature focuses on model development:
trying to build a better forecasting model,
especially comparing forecasts from a new
model to other models or to forecasts
made in real time
• Details: Croushore (2006) Handbook of
Economic Forecasting
Forecasting
• Does the fact that data are revised matter
significantly (in an economic sense) for
forecasts?
Forecasting
• EXAMPLE: THE INDEX OF LEADING INDICATORS
• Leading indicators: seem to predict recessions quite
well.
• But did they do so in real time? The evidence suggests
skepticism.
• Diebold and Rudebusch (1991) investigated the issue,
using real-time data
• Their conclusion: The leading indicators do not lead and
they do not indicate!
• The use of revised data gives a misleading picture of the
forecasting ability of the leading indicators.
date
1974:08
1974:07
1974:06
1974:05
1974:04
1974:03
1974:02
1974:01
1973:12
1973:11
1973:10
1973:09
1973:08
1973:07
1973:06
1973:05
1973:04
1973:03
1973:02
1973:01
Value of Leading Index
Leading Indicators, vintage Sept 1974
185
180
175
170
165
160
155
150
145
140
Forecasting
• EXAMPLE: THE INDEX OF LEADING
INDICATORS
• Chart shows not much problem
• But recession started in November 1973
• Subsequently, leading indicators were
revised & ex-post they do much better
date
1974:08
1974:07
1974:06
1974:05
1974:04
1974:03
1974:02
1974:01
1973:12
1973:11
1973:10
1973:09
1973:08
1973:07
1973:06
160
1973:05
1973:04
1973:03
1973:02
1973:01
Leading Index
Leading Indicators, vintage Sept 1974 and Dec. 1989
185
100
180
98
Dec. 1989 vintage
175
96
170
94
165
92
Sept. 1974 vintage
90
155
88
150
86
145
84
140
82
Forecasting
• Why Are Forecasts Affected by Data
Revisions?
– Change in data input into model
– Change in estimated coefficients
– Change in model itself (number of lags)
– See experiments in Stark-Croushore (2002)
Forecasting
• What Do We Use as Actuals?
– Answer: Depends on purpose
– Best measures are probably latest-available
data for “truth” (though perhaps not in fixedweighting era)
– But forecasters would not anticipate
redefinitions and generally forecast to be
consistent with government data methods
(example: pre-chain-weighting period; 2013
capitalization of R&D)
Forecasting
• What Do We Use as Actuals?
– Real-Time Data Set: many choices
•
•
•
•
first release (or second, or third)
four quarters later (or eight or twelve)
Date of annual revision (July for U.S. data)
last benchmark (the last vintage before a
benchmark revision)
• latest available
Forecasting
• How Should Forecasts Be Made When
Data Are Revised?
– Key issue: temptation to cheat!
• Try method; it doesn’t work; but that’s because of
one outlier; dummy out that observation; the
method works!
• If data are not available, use a real-time proxy,
don’t peak at future data
• Cheating is inherent because you know the history
already
Forecasting
• Forecasting with Real-Time versus LatestAvailable Data
– Faust-Rogers-Wright (2003): research showing
forecastability of exchange rates depended on a
particular vintage of data; other vintages show no
forecastability
– Molodtsova (2007): combining real-time data with
Taylor rule allows predictability of exchange rate
– Moldtsova-Nikolsko-Rzhevskyy-Papell (2007):
dollar/mark exchange rate predictable only with realtime data
Forecasting
• Summary: for forecasting, sometimes
data vintage matters, other times it doesn’t
Forecasting
• Key Issue: What are the costs and
benefits of dealing with real-time data
issues versus other forecasting issues?
Monetary Policy
Monetary Policy: Data Revisions
• How Much Does It Matter for Monetary
Policy that Data Are Revised?
• How Misleading Is Monetary Policy
Analysis Based on Final Data Instead of
Real-Time Data?
• How Should Monetary Policymakers
Handle Data Uncertainty?
Monetary Policy: Data Revisions
• How Much Does It Matter for Monetary
Policy that Data Are Revised?
– Example: Fed’s favorite inflation measure is
the Personal Consumption Expenditures Price
Index Excluding Food & Energy Prices
(PCEPIXFE)
– But it has been revised substantially
Figure 1
Core PCE Inflation Rate from 1997Q1 to 2002Q1, Vintage May 2002
2.4
2.2
Inflation Rate
2.0
1.8
1.6
1.4
1.2
1.0
1997
1998
1999
2000
Date
2001
2002
Figure 3
Core PCE Inflation Rate from 1997Q1 to 2002Q1, Vintages May 2002, Dec. 2003, Aug. 2005
2.4
August 2005
2.2
Inflation Rate
2.0
Dec 2003
1.8
1.6
1.4
1.2
1.0
1997
May 2002
1998
1999
2000
Date
2001
2002
Monetary Policy: Data Revisions
• How Much Does It Matter for Monetary
Policy that Data Are Revised?
– Croushore (2008): PCE revisions could
mislead the Fed
– Maravall-Pierce (1986): The Fed optimally
signal extracts from the noise in money data,
so data revisions would not significantly affect
monetary policy
– Kugler et al. (2005): Monetary policy shojuld
be less aggressive because of data revisions
Monetary Policy: Data Revisions
• How Misleading Is Monetary Policy
Analysis Based on Final Data Instead of
Real-Time Data?
– Croushore-Evans (2006): Data revisions do
not significantly affect measures of monetary
policy shocks in recursive systems, but they
make identification of simultaneous systems
problematic
Monetary Policy: Data Revisions
• How Should Monetary Policymakers
Handle Data Uncertainty?
– Coenen-Levin-Wieland (2001): use money as
an indicator when GDP data are uncertain
– Bernanke-Boivin (2003): use factor model to
incorporate much data; results do not depend
on using real-time data instead of revised
data
Monetary Policy: Analytical
Revisions
• What Happens When Economists or
Policymakers Revise Conceptual
Variables?
– Output gap
– Natural rate of unemployment
– Equilibrium real interest rate
• Concepts are never observed, but are
centerpiece of macroeconomic theory
Monetary Policy: Analytical
Revisions
• Orphanides (2001): Fed overreacted to
perceived output gap in 1970s, causing
Great Inflation; but output gap was
mismeasured
Output Gap Revisions
• Most U.S. analysts look at CBO measure,
but it is revised extensively over time
• Problem is especially acute at the end of
the sample
Monetary Policy: Analytical
Revisions
• What Happens When Economists or
Policymakers Revise Conceptual
Variables?
– Key issue: end-of-sample inference for
forward-looking concepts (filters)
– Key issue: optimal model of evolution of
analytical concepts
• Most work is statistical; perhaps a theoretical
breakthrough is needed
Macroeconomic Research
Macroeconomic Research
• How Is Macroeconomic Research Affected
By Data Revisions?
– Croushore-Stark (2003): how results from
key macro studies are affected by alternative
vintages
– Boschen-Grossman (1982): testing neutrality
of money under rational expectations: support
for RE with revised data, but not with real-time
data
Macroeconomic Research
• How Is Macroeconomic Research Affected
By Data Revisions?
– Amato-Swanson (2001): the predictive
content of money for output is not clear in real
time; only in revised data
Macroeconomic Research
• Should Macroeconomic Models Incorporate
Data Revisions?
– Aruoba (2004): business-cycle dynamics are
captured better by a DSGE model that accounts for
data revisions than one that does not
– Edge, Laubach, Williams (2004): learning explains
long-run productivity growth forecasts; helps explain
cycles in employment, investment, long-term interest
rates
Macroeconomic Research
• Do Data Revisions Affect Economic Activity?
– Oh-Waldman (1990): false (positive) announcements
increase economic activity with leading indicators and
industrial production in real time
– Bomfim (2001): improving the signal in data would
exacerbate cyclical fluctuations if agents performed
optimal signal extraction; but if agents ignore data
revisions, then improving data quality would reduce
cyclical fluctuations
Macroeconomic Research
• Overall: literature in its infancy: more work
needed in all three areas (robustness of
research results, incorporating data
revisions into macro models, examining
how or whether data revisions affect
economic activity)
Current Analysis
• How Do Financial Markets React to Data
Revisions?
– Christoffersen-Ghysels-Swanson (2002):
need real-time data to properly determine
announcement effects in financial markets
Current Analysis
• How Is Business Cycle Dating Affected By
Data Revisions?
– Economists like to argue about the state of
the business cycle . . .
Current Analysis
• How Is Business Cycle Dating Affected By
Data Revisions?
– Chauvet-Piger (2003, 2005): test algorithms
to identify turning points in real time
– Chauvet-Hamilton (2006): develop alternative
recession indicators and forecasts in real time
– Nalewaik (2007): using real-time gross
domestic income helps forecast recessions
better than just using GDP
Current Analysis
• Overall: much additional research needed
in current analysis in real time
Summary
• Field of real-time data analysis offers
many opportunities for new research
• Most promising areas:
– Macroeconomic research: incorporating data
revisions into macro models
– Current analysis of business and financial
conditions
– Other areas are more mature & need more
sophisticated analysis