Download Slide 1

The Role of Over-Sampling of
the Wealthy in the SCF
Arthur B. Kennickell
Federal Reserve Board
Opinions are those of the presenter alone and do not necessarily
reflect the views of the Federal Reserve Board or its staff.
Thanks!
• Organizers, esp. Andrea Brandolini
• FRB, SOI and NORC colleagues
• Interviewers
• Respondents
LWS Process
• Create conceptual categories
• Create “harmonized” (procrustean?)
variables for a number of countries
– Not everything is measured everywhere
– Some things are measured differently
Apples vs. Oranges
Kumkwats vs. Tangerines
Harmonize Samples?
• Samples are bedrock of a survey
– Determines what a survey represents
– Basis of scientific inference
– For surveys with a common population
definition, representational differences should
be ones of efficiency
• What if definition is different?
• Sample vs. participants
– Nonresponse probably not purely random
– Compensating adjustments to weights differ
Fruit Baskets
Understand and Document
• Differences may be due to structural
differences in economies
– Institutions
– Behavior
• Differences may be artifacts of different
types of measurement
– Questions in surveys (instruments)
– Survey samples and nonresponse
Theorem
• There is never an acceptable substitute for
thinking about what you are doing.
• Corollary: Don’t count on remembering the
details: document, document, document.
Survey of Consumer Finances
• Designed to measure wealth
–
–
–
–
Detail on the “Primary Economic Unit”
Summary information on other units in the household
Framed approach to measurement
Disaggregated information on assets and liabilities,
with supporting detail
– Strong emphasis on instrument development, training
and quality control
• Leave it to other exercises to evaluate the
equivalizing of variables in LWS
– Focus on sample
SCF Sample
• Dual-frame design
– Area-probability and List Samples
• Area-probability sample
–
–
–
–
–
Geographically based sample of addresses
Key elements of stratification to achieve balance
Multi-stage, with clusters (tracts) as penultimate stage
Postal address sequences usually used to order the final stage
All units selected with equal probability
– Robust coverage of the nation and good representation of
behaviors that are broadly distributed
List Sample
• List sample is a sample of named units
(individuals/couples) used to identify households
• Based on and selected from statistical records derived
from tax returns by Statistics of Income (IRS)
• Two-stages of selection
– Accepts high-level geographic selections of AP sample
• Otherwise, entirely independent
– Selected using “wealth index” derived from blend of two models
using income and associated data
• One, a simple grossing-up of capita income
• Other based on modeling of measured wealth in preceding survey
• Multiple years of income data to smooth fluctuations
• Over-samples wealthy units
• Robust coverage of the wealthy, but not whole
population
Role of Over-Sampling
• Increases efficiency of estimates affected
by upper tail of the wealth distribution
– Makes possible study of relationships that
would be too thinly represented in an AP
sample alone
• Means of detecting and correcting for
nonresponse bias
Evaluating Over-Sampling
• Compare AP sample alone with AP + list sample
• Use AP weights calibrated to key population
margins
• Use combined sample weights with full
calibrations in the separate and combined
samples
– LS determines the shape of the upper tail of the
wealth distribution
• Allows direct evaluation of the role of type of
over-sampling used in the SCF
LS as % Combined Samples, 2004
100
90
80
% of group
70
60
50
40
30
20
10
0
All
>=$100K
>=$500K
Wealth group
>=$1M
>=$5M
Narrowly Held Assets
• Of approximately 400 observations with
direct holdings of bonds, only ~10% were
AP cases
• Of approximately 1,500 observations with
direct holdings of publicly traded stocks,
only ~37% were AP cases
– ~25% of weighted total value attributable to
AP sample
Shares of Total Net Worth, 2004
50
45
40
AP only
AP+LS
Percent
35
30
25
20
15
10
5
0
0-50
50-90
90-95
Percentiles of net worth
95-99
99-100
Nonresponse
• Present in virtually all surveys
• If distribution of a variable for nonrespondents
differs from that for participants  bias
• Wealth surveys cover a particularly sensitive
topic: respondent resistance
• Systematic components to nonresponse
– E.g., NR increases with capital income and decreases
with age and charitable contributions
• Central area of research for SCF
– LS provides means of correction
(AP+LS)-AP, 2004
NET WORTH
$2.7M/74%
$13,600/4.3%
$3,500/3.9%
$250/1.9%
Effect on Wealth Distribution
• High-wealth list sample cases “displace”
top of the AP wealth distribution downward
• Similar pattern if add synthetic case to AP
sample with weight of 1% of population
and wealth at 99 percentile of combined
distribution
• Actual effects more subtle
Special Difficulties
• More complex interviews
– Requires more sophisticated instrument
– Training more important
• More difficult to contact and gain cooperation
– “Gatekeepers”
– Interviewers often have an incentive to avoid difficult
cases
– More expensive
• Same problems without an oversample, but
much less visible
Thanks