Download Service survey

Short-term forecasting of the GDP growth rate
using the BTS in industry and in services:
An out-of-sample analysis
by Hélène Erkel-Rousse
and Christelle Minodier
(INSEE, France)
Third Joint European Commission-OECD Workshop on International
Development of Business and Consumer Tendency Surveys,
Brussels, 12 - 13 November 2007
The two issues addressed in the paper:
› 1) BTS are widely used for the short-term forecasting
of economic activity.
However, to our knowledge, there has been no recent
attempt to establish the significance of their contribution
to the quality of forecasts (using modern econometric
techniques).
› 2) The service survey contains a specific piece of
information on GDP growth with respect to the industry
survey (Bouton and Erkel-Rousse, 2003).
However, it remains to be shown that this specific piece
of information permits one to significantly improve the
quality of short-term GDP forecasts with respect to
models involving variables from the industry survey
exclusively.
Page 2
The study consists in:
› - estimating several VAR models and univariate
calibration models of the GDP quarterly growth rate
encompassing miscellaneous variables derived from
the industry and service surveys as well as GDP lags
› - estimating competing models with no service variable
or, even, no BTS variable (simple AR models of GDP)
on several time periods (real-time analysis - RTA)
› - simulating each model up to a four-quarter forecast
horizon, and deriving series of forecasting errors (RTA)
› - comparing the predictive accuracy of the different
models using Clark-West or Harvey, Leybourne and
Newbold tests (depending on the models)
› Data used: quarterly survey data and, to a lesser extent,
monthly survey data.
Page 3
Structure of the Talk
› Data
› Methodology
› Results
› Conclusion
Page 4
Data (1) General characteristic features
› - 1962: creation of the INSEE industry survey
› - 1988: creation of the INSEE service survey (on a
quarterly basis)
› - June 2000: creation of the monthly service survey,
adding of a few questions
› - Since the 1990’s: progressive extension of the sector
coverage of the service survey (in the data used:
business services (2/3), household services (> 20%),
real estate activities (>10%))
› - January 2004: several changes in the wording of the
questionnaires of the two surveys and adding of a few
questions in the service survey for European
harmonisation purpose
› - January 2004: the two surveys become compulsory
Page 5
Data (2) The data used in the study
› Macro variable to be forecasted:
 the quarterly GDP growth rate from the French
Quarterly Accounts (hereafter “GDP”)
› Industry survey data:
 the 6 main monthly balances and 2 quarterly ones
(past and expected demand)
 2 static common factors in industry: a monthly one
(derived from the 6 main monthly balances – published
each month by INSEE) and a quarterly one
(encompassing the 6 monthly balances + the 2 quarterly
ones)
› Service survey data:
 the 3 main monthly balances and 3 main quarterly ones
 the dynamic common factor in services (introduced by
Cornec and Deperraz, 2007, derived from the 6 balances
and published each month by INSEE)
Page 6
Methodology (1)
› We define several forecasting models of GDP for each month in a
given quarter, so as to be able to up-date the short-term forecasts
of GDP each month on the basis of the most recent piece of
information given by the BTS.
Notation:
Ind_m1 = variable Ind derived from the industry survey relating to
month 1
Ser_m4 = variable Ser derived from the industry survey relating to
month 4…
with m1 (m2,m3, resp.) = 1st (2nd, 3rd, resp.) month in the current
quarter and m4 = 1st month in the following quarter.
Examples:
- In the second quarter of year 2000,
m1 = April, m2 = May, m3 = June, m4 = July, of year 2000.
- In the last quarter of year 2004, m1 = October, m2= November,
m3 = December 2004, m4 = January 2005.
Page 7
Methodology (2): VAR models
Comparison of the predictive accuracy of 3 VAR models of 2 kinds
(non restricted, restricted) per couple of survey variables used:
- VAR with 3 variables: GDP, Ind_mi, Ser_mi
- VAR with 2 variables: GDP, Ind_mi
- simple AR model of GDP (basic benchmark)
with Ind_mi (Ser_mi, resp.) = a survey variable from the industry
(services resp.) survey relating to month mi, i = 1 to 4.
Non restricted VAR = VAR with 2 lags estimated using OLS
Restricted VAR
= VAR with 4 lags with exclusion restrictions,
estimated using SURE
Tests of equal predictive accuracy in nested models
= Clark-West tests, with:
1) Scilab – Grocer (correcting autocorrelation within forecast error
series using Newey-West variances, for forecast horizons = 2 to 4
quarters)
2) SAS – procedure autoreg, options nlag=4 and backstep (testing
for autocorrelation up to 4 lags and correcting it
when necessary using Yule-Walker estimates) Page 8
Methodology (3): Univariate calibration models (1)
› Intuition: when the length of both estimation series and
forecast error series is short (months 2 and 3, especially 2 – no
survey in August), univariate calibration models might be better
adapted because more parsimonious than VAR models.
› Kinds of models estimated: one set per month mi (i = 1 to 4).
At a given month when BTS are available up to the 1st forecast
horizon:
Models used for the forecasting of GDP at a 1 quarter horizon:
GDP = function of the current and lagged values of
survey variables
Models used for the forecasting of GDP at a 2 quarter horizon:
GDP = function of the lagged values of survey variables
Models used for the forecasting of GDP at a 3 quarter horizon:
GDP = function of the lagged values of survey variables
to the exclusion of the first lags
Page 9
Methodology (4): Univariate calibration models (2)
Comparison of the predictive accuracies of several univariate
calibration models of GDP:
- some including explanatory variables from the two BTS
- some including explanatory variables from the industry
survey only
- AR model of GDP (basic benchmark)
- the “best” VAR3 models
with, again, different sets of models depending on the month in (or
just after) the quarter (mi, i = 1 to 4).
Depending on the models whose predictive accuracies are
compared, we performed:
- either Clark-West tests (for the comparison of nested models)
- or Harvey, Leybourne and Newbold tests (for the comparison of
non-nested models)
Software used: Scilab – Grocer.
Page 10
Results 1) VAR models (1)
› Choice of survey variables:
- Most leading balances:
Industry survey: expected production (monthly)
Service survey: expected operating profit (quarterly)
- Most correlated common factors (with GDP) at each
month mi:
Industry survey:
for m1 and m4: the quarterly common factor
for m2 and m3: the monthly common factor
Service survey: the dynamic common factor
Page 11
Results 1) VAR models (2)
› Contribution of BTS to the short-term forecasting of
GDP (with respect to benchmark AR models):
- Very significant at the 1 and 2 quarter horizons in
most cases (very small P-values)
- Significant at the 5% or 10% thresholds for several
models relating to quarterly months (m1 and m4) at
the 3 and, even, 4 quarter horizons (even though the
quality of their forecasts remains poor)
- Often more significant to forecast the first release
of GDP than the last available release (at the 3 and
4 quarter horizons and, to a lesser extent at the 2
quarter horizon)
Page 12
Results 1) VAR models (3)
› Contribution of the service survey to the short-term forecasting of
GDP (with respect to VAR models with 2 variables: GDP and Ind_mi):
1) In the case of “quarterly” months (m1 and m4), for which fairly
long time series in services are available:
- Significant, especially when the service variable is the dynamic
common factor
- Higher contribution: for the 2 and 3 quarter forecast horizons
(non significant at the 4 quarter horizon)
2) In the case of “non quarterly” months (m2 and m3), for which
only short time series for services are available
- Less high contribution of the service survey at this stage…
but some encouraging significant results for month m2
- Important remark: The methodology used creates a serious
bias against the service survey (either linear interpolations are
used for data before June 2000 or the last available quarterly
value of a balance – that in m1- is used for months m2 and m3
while more recent monthly industry data are used)
Page 13
Results 1) VAR models (4)
Example: Results from the non restricted VAR model with the quarterly
common factor in industry and the dynamic common factor in services (m4)
Forecast
Horizon
1
2
3
4
End
AR 1 vs VAR 2
1st result
AR 1 vs VAR 3
Last result
1st result
VAR 2 vs VAR3
Last result
1st result
Last result
2004Q4
0.0004
S
0.0028
S
0.0000
S
0.0003
S
0.0959
+
0.0327
5
2007Q2
0.0002
S
0.0010
S
0.0002
S
0.0003
S
0.4179
N
0.2261
N
2004Q4
0.0002
S
0.0000
S
0.0000
S
0.0000
L
0.0015
+
0.0326
5
2007Q2
0.0005
S
0.0000
S
0.0008
S
0.0002 *
0.0764
+
0.1110
+
2004Q4
0.0564
+
0.2248
N
0.0191
5
0.2288
N
0.0319
+
0.4788
N
2007Q2
0.2183
N
0.3817
N
0.1013
L
0.3639
N
0.0118
+
0.3369
N
2004Q4
0.0025
+
0.0732
A
0.0118
+
0.2168
A
0.8644
N
0.8864
N
2007Q2
0.0008 *
0.0508
L
0.0020
5
0.1210
N
0.5879
N
0.6410
N
Page 14
Results 2) Univariate calibration models (1)
We are currently working on these models.
Two kinds of models have been estimated, using the Scilab-Grocer
software:
- Models whose optimal specifications are automatically determined
by the software
- Models whose specifications are determined by the authors so that
every explanatory variable has an impact of the expected sign on
GDP.
Our first preliminary results suggest that these kinds of models
might enable one to lead to clearly positive results as concerns the
contribution of the service survey, notably in month m3 (especially
models whose specifications are automatically determined).
However (at least on the basis of our preliminary results), these
kinds of models do not appear to always lead to significantly better
GDP forecasts than the VAR models.
Page 15
Results 2) Univariate calibration models (2)
Example: Univariate models
relating to a three-quarter forecast horizon
Month in
the quarter
m1
m2
m3
m4
End
Industry vs
Industry + services
VAR3 vs
Industry + services
1st result Last result
1st result Last result
2004Q4
0.140
0.003
0.186
0.088
2007Q2
0.086
0.003
0.467
0.336
2004Q4
0.326
0.046
0.311
0.036
2007Q2
0.262
0.308
0.334
0.210
2004Q4
0.008
0.001
0.221
0.026
2007Q2
0.006
0.001
0.497
0.129
2004Q4
0.154
0.025
0.086
0.016
2007Q2
0.214
0.071
0.106
0.052
Page 16
Conclusion
› 1) The study clearly confirms the predictive power of BTS
for GDP growth at short-term horizons
› 2) It also shows that the quarterly service survey has
predictive power alongside with the manufacturing survey
› 3) As far as monthly service data are concerned, it is
definitely too early to have firm conclusions. Our results
derive from methodologies that generate negative biases
to the detriment of the service survey (use of either linear
interpolated data before June 2000 or less up-to-date data
than industry ones). Nonetheless, some of the results
suggest that monthly service data might also permit
one to improve the short-term forecasting of GDP.
To be confirmed in 6 or 7 years when long enough monthly
service series are available!
Page 17