Download Sharing best practices for the redesign of three business surveys 1466

Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada
Sharing best practices for the redesign of three business surveys
Charles Tardif
Statistics Canada, Ottawa, Ontario, Canada, K1A 0T6
how the changes introduced to these surveys make
them now more compatible with one another.
Abstract
Three monthly Statistics Canada business surveys, the
Monthly Wholesale and Retail Trade Survey
(MWRTS), the Monthly Survey of Manufactures
(MSM) and the Monthly Restaurants, Caterers and
Taverns Survey (MRCTS) have just undergone a
restratification or a complete redesign. These three
independent surveys are targeting different industries,
have different goals, and have data users with specific
expectations. On the other hand, they all share key
objectives.With their respective new restratification or
redesign, all three surveys are attempting to improve
the data quality (particularly in terms of timeliness,
accuracy, consistency and relevance) while reducing
costs and response burden.
2. Overview of a monthly business survey
In a simplified way, the objectives of a monthly
business survey are to produce monthly measures of
levels and trends on financial information such as
sales, revenues, expenses and characteristics variables,
by geography and by industrial level, with a given
level of quality or precision. In terms of the geographic
level in Canada, the estimates are usually produced by
provinces and territories, and sometimes at subprovincial levels such as Census Metropolitan Areas.
At the industrial level, the North American Industrial
Classification System (NAICS) is used to classify the
units. Classifications at the 3, 4, 5 or 6 digit NAICS are
commonly used to stratify the population and to
produce the estimates. The level of quality required,
usually defined in terms of coefficients of variation
(CV), can be measured at different levels, for example
at the provincial level, at the NAICS level or by crosstabulations of provinces and NAICS. Different target
CVs can be set based on the level and within each
level.
In this paper, we will review the issues faced by the
different surveys before their respective restratification
or redesign, and how these surveys have shared
common practices to achieve their objectives. More
specifically, we will see how these surveys have made
effective use of administrative data, such as tax data, to
replace collected data, how they have allocated and
selected their respective samples, how they have
integrated the Royce-Maranda algorithm for the takenone portion of the survey, and how the frame and
sample maintenance is done each month.
For a given monthly business survey, the desire is to
establish the best possible survey design to meet the
survey’s objectives. Among these, the six elements of
the Statistics Canada quality framework, namely
relevance,
accuracy,
timeliness,
accessibility,
interpretability and coherence are of prime importance.
Another objective of the business surveys, which has
become more and more important in the last few years,
is to maximise the use of administrative data, mainly
tax data, whenever possible in order to reduce the
response burden on the smaller businesses.
Keywords: Redesign, restratification, monthly surveys,
tax data
1. Introduction
Three monthly business surveys conducted by
Statistics Canada, the Monthly Wholesale and Retail
Trade Survey (MWRTS), the Monthly Survey of
Manufactures (MSM) and the Monthly Restaurants,
Caterers and Taverns Survey (MRCTS) have all
undertaken (or in some cases, completed) a redesign or
an important restratification in the last two years. Even
though these surveys are conducted independently,
each survey had an objective of harmonizing its
methodology with one another while undertaking their
restratification or redesign. In this paper, we will
present the different surveys, explain the methodology
before their last redesign or restratification, and see
With this general description of a business survey, let’s
see the specific characteristics of the three surveys we
are interested in, the MSM, the MRCTS and the
MWRTS.
2.1 MSM
The MSM produces monthly estimates of shipments,
inventories and orders. These values are used as
1466
Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada
indicators of the economic condition of manufacturing
industries, as inputs to Canada’s Gross Domestic
Product and as two components in the Statistics
Canada composite indicator. Data are used by both the
private and public sectors including finance
departments of the federal and provincial governments,
the Bank of Canada, Industry Canada, the System of
National Accounts, the manufacturing community,
consultants and research organizations in Canada, the
Unites States and abroad.
2.3 MWRTS (Retail)
For the retail component of the survey, MWRTS
collects monthly retail sales for all department stores
classified under the appropriate NAICS code. The
estimates are used by retailers, as well as by public and
private agencies across Canada.
2.4 MWRTS (Wholesale)
For the wholesale component of the survey, the
MWRTS provides monthly information on the
performance of the wholesale trade sector and is an
important indicator of the health of the Canadian
economy. In addition, the business community uses the
data to analyse market performance.
2.2 MRCTS
The MRCTS collects monthly sales and receipts data
from a sample of restaurants, caterers and taverns in
Canada. These data are used by federal and provincial
governments, private associations and food service
businesses for consulting, marketing and planning
purposes. The provincial and federal governments use
the information to estimate provincial taxation shares.
Table 1 below gives an overview of the key
characteristics of the three monthly business surveys.
Table 1: Characteristics of the three monthly business surveys
Variables of
interest
Geographic level
Industrial level:
Published NAICS
Population size
(establishments)
Collected sample
size
Number of
domains for
estimation
Yearly revenues
(in billion)
MSM
Manufacturers
Shipments,
inventories and
orders
Provinces,
territories
311 to 339, at the
3rd to 6th digit
100,000
MRCTS
Restaurants
Sales, number of
locations
MWRTS
Retail portion
Sales, number of
locations
MWRTS
Wholesale portion
Sales, inventories
Provinces,
territories
722, at the 5th digit
90,000
Provinces,
territories
44 and 45, for 19
trade groups
180,000
Provinces,
territories
41, for 15 trade
groups
100,000
11,000
2,100
12,000
8,000
Over 1,000
52
247 possible
domains
195 possible
domains
550
36
350
450
As we can see in Table 1, the three surveys meet the
standard characteristics described in the section above.
They also share other common characteristics such as
having a skewed population and an annual counterpart,
which in fact has been integrated into one survey, the
Unified Enterprise Survey (UES). If such an
integration of surveys exists for the annuals, would it
be possible to do the same for the monthly surveys? In
order to answer this question, we will look at different
elements of the three surveys before and after their last
restratification or redesign, and see what has been done
to make them more comparable with one another. The
following table summarizes the situation before the
changes were introduced to the different surveys, as
well as the situation after the changes were introduced.
1467
Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada
Table 2: Changes introduced to the three monthly business surveys
Frame
Before
Now
Sampling unit
Before
Now
Stratification
variables
Take-none stratum
Before
Now
Before
Now
Stratification
Use of tax data:
Goods and
Services Tax
(GST)
Before and now
Before
Now
Before and now
Sample selection
Edits and
imputation
Before
Now
Estimation and
variance
Before and now
MSM
Manufacturers
BR, no exclusions
MRCTS
MWRTS
Restaurants
Wholesale, Retail
Business Register (BR), with the exclusion
of the non-employing establishments
BR, no exclusions, using a survey specific universe frame
Establishment
Company level
level
Establishment or cluster of establishments
Province/territory, NAICS and a size measure: Gross Business
Income (GBI) from the BR
Province/territory, NAICS and a size measure:
Annualized monthly data, data from annual survey, tax data or
GBI
Bottom 2% by
No take-none
5% by geography and
province
stratum
Trade groups
Bottom 10% (MSM, MRCTS) or 5% (MWRTS) by province and
stratification
Lavallée-Hidiroglou algorithm used to stratify the population
(more details in the text below)
GST data were gradually introduced in the surveys
Micro approach,
Micro approach,
Micro approach,
selected units
all simple units
selected units
Random sample
within each
stratum
Random sample
within each
stratum and for
GST modelling
MSM in-house
MRCTS in-house
E&I program
E&I program
Currently under development, using
BANFF, a generalized system for E&I
Systematic sampling
within each stratum
MWRTS in-house
E&I program
MWRTS in-house
E&I program
Generalized Estimation System (GES) used to calculate the
estimates and the variance
Among the elements presented in Table 2, some
deserve more attention and will be described here. In
terms of stratification, as indicated in the table above,
the Lavallée-Hidiroglou algorithm is used to stratify
the population for all surveys. This algorithm performs
well when dealing with skewed populations, which is
the case for all three surveys. All surveys have take-all
and must-take strata, for the large establishments, as
well as one or a few take-some strata per geography
and NAICS combinations. Other constraints are also
used in the stratification such as a minimum sample
size per stratum and a maximum design weight. Also,
over-sampling is done to take into account the out-ofscope, dead and non-responding units.
A few words should also be said about the use of the
Goods and Services Tax (GST) data by the survey. All
surveys make use of the GST data for the simple
structure units. As seen in the table above, MSM and
MWRTS are making use of the GST in the same way,
while MRCTS differs slightly. Models are required
when using the GST data to take into account potential
differences in concepts between the GST data and the
collected data, as well as for timing issues related to
the fact the GST data for the reference month are not
available the first month the estimates are produced.
MSM and MWRTS use 50% of the sampled simples to
build the model but only replace a small number of
units using GST. MRCTS surveys 500 units to build
1468
Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada
the model and then models the value of all remaining
simples using GST.
Also worth mentioning is the fact that these surveys
maximise the use of generalized systems developed at
Statistics Canada for their regular production, namely
the Generalized Sampling (GSAM) system for the
selection of the sample, BANFF for the edit and
imputation and the Generalized Estimation System
(GES) for the production of the estimates. Using these
systems ensure more consistency between the surveys.
restratified or redesigned. There would also be some
operational issues. These surveys are currently
managed by three different subject matter divisions
within Statistics Canada. Integrating these surveys
would likely mean organisational changes, even
though the methodological support for these surveys is
already centralized. There would also be some impact
on the field work for the collection of these data, since
two of these surveys have their data collected by the
Statistics Canada regional offices while the other one
has its data collected directly from the Head Office.
3. A monthly unified enterprise survey
Conclusion
As seen in Table 2, these three surveys now use more
common methodologies, despite the different steps of
the surveys and the remaining differences for some
elements. Should we go one step further and integrate
these surveys into one unified monthly business
survey? There would be some advantages at doing so.
In conclusion, there used to be important differences
between these three Statistics Canada monthly surveys.
Significant steps have been made to share best
practices between these surveys to harmonize them. In
the near future, integrating these surveys into one
Monthly Unified Enterprise Survey should be
considered.
First of all, integrating all monthly surveys would be a
good opportunity to take advantage of the best
practices of each survey. It would also be easier to
ensure coherence between the different monthly
business programs. As mentioned earlier, the annual
business surveys are already integrated, so integrating
the monthly business surveys as well would make the
comparisons between the annual and the monthly
surveys easier. Thirdly, if there is a need to benchmark
the annual and the monthly estimates, this task would
be facilitated. Integrating all monthly surveys would
also facilitate the implementation of changes to the
surveys. For example, the introduction of the GST data
for the surveys was done at different times. If the
integration had been done simultaneously, comparing
the impact of the GST on the estimates would have
been easier. Lastly, if we want to make other changes
to the surveys in the future, such as the implementation
of other measures of the variability to take into account
the variance due to the imputation or the modelling,
the integration of these surveys would make the
implementation easier.
Acknowledgements
The author would like to express his sincere
appreciation to Martin Beaulieu, François Brisebois,
Gayle Keeley and Denis Malo, whose thoughtful
comments helped improve the quality of the article.
The author would also like to greatly thank the
methodologists working on these surveys, namely
Catalin Dochitoiu, Jean-François Dubois, Kathleen
MacEachern, Zachary Pritchard and Caroline Rondeau
whose support has been very appreciated.
References
Lavallée, P. and Hidiroglou, M., On the stratification
of skewed populations, Survey Methodology, June
1988 Vol.14, no. 1, pp 33-43, Statistics Canada
Thomas, S. and Cook K., The use of Goods and
Services Tax data in modeling survey data for the
monthly survey of manufacturing, Statistics Canada
working paper BSMD-2005-011-E
Of course, there would also be some drawbacks to the
integration of these surveys. From the methodological
point of view, there are some conceptual differences
between the surveys. For example, different data
elements are collected at different levels. However,
these differences could easily be factored in at the
sample design stage. As well, making changes to
surveys often translates into changes or breaks to the
survey’s time series and revisions of past series are
usually necessary to ensure consistency over time. This
is a situation that had in fact to be dealt with when the
three business surveys discussed in this paper were
Trépanier, Julie, The redesigned Canadian monthly
wholesale and retail trade survey: a postmortem of the
implementation, Statistics Canada working paper
1469