Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada Sharing best practices for the redesign of three business surveys Charles Tardif Statistics Canada, Ottawa, Ontario, Canada, K1A 0T6 how the changes introduced to these surveys make them now more compatible with one another. Abstract Three monthly Statistics Canada business surveys, the Monthly Wholesale and Retail Trade Survey (MWRTS), the Monthly Survey of Manufactures (MSM) and the Monthly Restaurants, Caterers and Taverns Survey (MRCTS) have just undergone a restratification or a complete redesign. These three independent surveys are targeting different industries, have different goals, and have data users with specific expectations. On the other hand, they all share key objectives.With their respective new restratification or redesign, all three surveys are attempting to improve the data quality (particularly in terms of timeliness, accuracy, consistency and relevance) while reducing costs and response burden. 2. Overview of a monthly business survey In a simplified way, the objectives of a monthly business survey are to produce monthly measures of levels and trends on financial information such as sales, revenues, expenses and characteristics variables, by geography and by industrial level, with a given level of quality or precision. In terms of the geographic level in Canada, the estimates are usually produced by provinces and territories, and sometimes at subprovincial levels such as Census Metropolitan Areas. At the industrial level, the North American Industrial Classification System (NAICS) is used to classify the units. Classifications at the 3, 4, 5 or 6 digit NAICS are commonly used to stratify the population and to produce the estimates. The level of quality required, usually defined in terms of coefficients of variation (CV), can be measured at different levels, for example at the provincial level, at the NAICS level or by crosstabulations of provinces and NAICS. Different target CVs can be set based on the level and within each level. In this paper, we will review the issues faced by the different surveys before their respective restratification or redesign, and how these surveys have shared common practices to achieve their objectives. More specifically, we will see how these surveys have made effective use of administrative data, such as tax data, to replace collected data, how they have allocated and selected their respective samples, how they have integrated the Royce-Maranda algorithm for the takenone portion of the survey, and how the frame and sample maintenance is done each month. For a given monthly business survey, the desire is to establish the best possible survey design to meet the survey’s objectives. Among these, the six elements of the Statistics Canada quality framework, namely relevance, accuracy, timeliness, accessibility, interpretability and coherence are of prime importance. Another objective of the business surveys, which has become more and more important in the last few years, is to maximise the use of administrative data, mainly tax data, whenever possible in order to reduce the response burden on the smaller businesses. Keywords: Redesign, restratification, monthly surveys, tax data 1. Introduction Three monthly business surveys conducted by Statistics Canada, the Monthly Wholesale and Retail Trade Survey (MWRTS), the Monthly Survey of Manufactures (MSM) and the Monthly Restaurants, Caterers and Taverns Survey (MRCTS) have all undertaken (or in some cases, completed) a redesign or an important restratification in the last two years. Even though these surveys are conducted independently, each survey had an objective of harmonizing its methodology with one another while undertaking their restratification or redesign. In this paper, we will present the different surveys, explain the methodology before their last redesign or restratification, and see With this general description of a business survey, let’s see the specific characteristics of the three surveys we are interested in, the MSM, the MRCTS and the MWRTS. 2.1 MSM The MSM produces monthly estimates of shipments, inventories and orders. These values are used as 1466 Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada indicators of the economic condition of manufacturing industries, as inputs to Canada’s Gross Domestic Product and as two components in the Statistics Canada composite indicator. Data are used by both the private and public sectors including finance departments of the federal and provincial governments, the Bank of Canada, Industry Canada, the System of National Accounts, the manufacturing community, consultants and research organizations in Canada, the Unites States and abroad. 2.3 MWRTS (Retail) For the retail component of the survey, MWRTS collects monthly retail sales for all department stores classified under the appropriate NAICS code. The estimates are used by retailers, as well as by public and private agencies across Canada. 2.4 MWRTS (Wholesale) For the wholesale component of the survey, the MWRTS provides monthly information on the performance of the wholesale trade sector and is an important indicator of the health of the Canadian economy. In addition, the business community uses the data to analyse market performance. 2.2 MRCTS The MRCTS collects monthly sales and receipts data from a sample of restaurants, caterers and taverns in Canada. These data are used by federal and provincial governments, private associations and food service businesses for consulting, marketing and planning purposes. The provincial and federal governments use the information to estimate provincial taxation shares. Table 1 below gives an overview of the key characteristics of the three monthly business surveys. Table 1: Characteristics of the three monthly business surveys Variables of interest Geographic level Industrial level: Published NAICS Population size (establishments) Collected sample size Number of domains for estimation Yearly revenues (in billion) MSM Manufacturers Shipments, inventories and orders Provinces, territories 311 to 339, at the 3rd to 6th digit 100,000 MRCTS Restaurants Sales, number of locations MWRTS Retail portion Sales, number of locations MWRTS Wholesale portion Sales, inventories Provinces, territories 722, at the 5th digit 90,000 Provinces, territories 44 and 45, for 19 trade groups 180,000 Provinces, territories 41, for 15 trade groups 100,000 11,000 2,100 12,000 8,000 Over 1,000 52 247 possible domains 195 possible domains 550 36 350 450 As we can see in Table 1, the three surveys meet the standard characteristics described in the section above. They also share other common characteristics such as having a skewed population and an annual counterpart, which in fact has been integrated into one survey, the Unified Enterprise Survey (UES). If such an integration of surveys exists for the annuals, would it be possible to do the same for the monthly surveys? In order to answer this question, we will look at different elements of the three surveys before and after their last restratification or redesign, and see what has been done to make them more comparable with one another. The following table summarizes the situation before the changes were introduced to the different surveys, as well as the situation after the changes were introduced. 1467 Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada Table 2: Changes introduced to the three monthly business surveys Frame Before Now Sampling unit Before Now Stratification variables Take-none stratum Before Now Before Now Stratification Use of tax data: Goods and Services Tax (GST) Before and now Before Now Before and now Sample selection Edits and imputation Before Now Estimation and variance Before and now MSM Manufacturers BR, no exclusions MRCTS MWRTS Restaurants Wholesale, Retail Business Register (BR), with the exclusion of the non-employing establishments BR, no exclusions, using a survey specific universe frame Establishment Company level level Establishment or cluster of establishments Province/territory, NAICS and a size measure: Gross Business Income (GBI) from the BR Province/territory, NAICS and a size measure: Annualized monthly data, data from annual survey, tax data or GBI Bottom 2% by No take-none 5% by geography and province stratum Trade groups Bottom 10% (MSM, MRCTS) or 5% (MWRTS) by province and stratification Lavallée-Hidiroglou algorithm used to stratify the population (more details in the text below) GST data were gradually introduced in the surveys Micro approach, Micro approach, Micro approach, selected units all simple units selected units Random sample within each stratum Random sample within each stratum and for GST modelling MSM in-house MRCTS in-house E&I program E&I program Currently under development, using BANFF, a generalized system for E&I Systematic sampling within each stratum MWRTS in-house E&I program MWRTS in-house E&I program Generalized Estimation System (GES) used to calculate the estimates and the variance Among the elements presented in Table 2, some deserve more attention and will be described here. In terms of stratification, as indicated in the table above, the Lavallée-Hidiroglou algorithm is used to stratify the population for all surveys. This algorithm performs well when dealing with skewed populations, which is the case for all three surveys. All surveys have take-all and must-take strata, for the large establishments, as well as one or a few take-some strata per geography and NAICS combinations. Other constraints are also used in the stratification such as a minimum sample size per stratum and a maximum design weight. Also, over-sampling is done to take into account the out-ofscope, dead and non-responding units. A few words should also be said about the use of the Goods and Services Tax (GST) data by the survey. All surveys make use of the GST data for the simple structure units. As seen in the table above, MSM and MWRTS are making use of the GST in the same way, while MRCTS differs slightly. Models are required when using the GST data to take into account potential differences in concepts between the GST data and the collected data, as well as for timing issues related to the fact the GST data for the reference month are not available the first month the estimates are produced. MSM and MWRTS use 50% of the sampled simples to build the model but only replace a small number of units using GST. MRCTS surveys 500 units to build 1468 Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada the model and then models the value of all remaining simples using GST. Also worth mentioning is the fact that these surveys maximise the use of generalized systems developed at Statistics Canada for their regular production, namely the Generalized Sampling (GSAM) system for the selection of the sample, BANFF for the edit and imputation and the Generalized Estimation System (GES) for the production of the estimates. Using these systems ensure more consistency between the surveys. restratified or redesigned. There would also be some operational issues. These surveys are currently managed by three different subject matter divisions within Statistics Canada. Integrating these surveys would likely mean organisational changes, even though the methodological support for these surveys is already centralized. There would also be some impact on the field work for the collection of these data, since two of these surveys have their data collected by the Statistics Canada regional offices while the other one has its data collected directly from the Head Office. 3. A monthly unified enterprise survey Conclusion As seen in Table 2, these three surveys now use more common methodologies, despite the different steps of the surveys and the remaining differences for some elements. Should we go one step further and integrate these surveys into one unified monthly business survey? There would be some advantages at doing so. In conclusion, there used to be important differences between these three Statistics Canada monthly surveys. Significant steps have been made to share best practices between these surveys to harmonize them. In the near future, integrating these surveys into one Monthly Unified Enterprise Survey should be considered. First of all, integrating all monthly surveys would be a good opportunity to take advantage of the best practices of each survey. It would also be easier to ensure coherence between the different monthly business programs. As mentioned earlier, the annual business surveys are already integrated, so integrating the monthly business surveys as well would make the comparisons between the annual and the monthly surveys easier. Thirdly, if there is a need to benchmark the annual and the monthly estimates, this task would be facilitated. Integrating all monthly surveys would also facilitate the implementation of changes to the surveys. For example, the introduction of the GST data for the surveys was done at different times. If the integration had been done simultaneously, comparing the impact of the GST on the estimates would have been easier. Lastly, if we want to make other changes to the surveys in the future, such as the implementation of other measures of the variability to take into account the variance due to the imputation or the modelling, the integration of these surveys would make the implementation easier. Acknowledgements The author would like to express his sincere appreciation to Martin Beaulieu, François Brisebois, Gayle Keeley and Denis Malo, whose thoughtful comments helped improve the quality of the article. The author would also like to greatly thank the methodologists working on these surveys, namely Catalin Dochitoiu, Jean-François Dubois, Kathleen MacEachern, Zachary Pritchard and Caroline Rondeau whose support has been very appreciated. References Lavallée, P. and Hidiroglou, M., On the stratification of skewed populations, Survey Methodology, June 1988 Vol.14, no. 1, pp 33-43, Statistics Canada Thomas, S. and Cook K., The use of Goods and Services Tax data in modeling survey data for the monthly survey of manufacturing, Statistics Canada working paper BSMD-2005-011-E Of course, there would also be some drawbacks to the integration of these surveys. From the methodological point of view, there are some conceptual differences between the surveys. For example, different data elements are collected at different levels. However, these differences could easily be factored in at the sample design stage. As well, making changes to surveys often translates into changes or breaks to the survey’s time series and revisions of past series are usually necessary to ensure consistency over time. This is a situation that had in fact to be dealt with when the three business surveys discussed in this paper were Trépanier, Julie, The redesigned Canadian monthly wholesale and retail trade survey: a postmortem of the implementation, Statistics Canada working paper 1469