Download - UNDP-ALM

First Regional Training
Assessing Costs and Benefits of Adaptation: Methods and Data
SAMPLING AND NON-SAMPLING
ERRORS
ISSUES TO CONSIDER
BISHWA NATH TIWARI
UNDP-APRC
Email: [email protected]
BANGKOK
1
14 MARCH 2013
CONTENT OF PRESENTATION
Household Survey design and errors
• Steps for conducting survey
• Errors
Sampling and non-sampling errors
2
Suggestion to minimize those errors –
Issues to consider
NEED FOR DATA
• Data is critical for effective planning, design &
implementation of adaptation projects
• BUT there is lack of disaggregated data by:
• Region, districts, settlements (informal), villages
• Sex and age groups
• Caste & ethnic groups
• Economic groups
• Vulnerable/disadvantaged groups
• Lack of updated data, and regular availability of data
3
• Lack of quality data (with less errors)
SOURCES OF DATA
Sources of
Data
Secondary
National
international
Primary
Household
survey
Institutional
survey
Some international sources of data: Internal Energy Agency, Global Footprint
Network, Centre for International Earth Science Information Network, Centre for
Research on the Epidemiology of Disasters, World Bank World Development
Indicators
4
Generally,
secondary data
lack
disaggregation
CC is global
externality but its
impacts are local,
therefore need
for hh level data
STEPS FOR CONDUCTING A
HOUSEHOLD SURVEY
1. Define universe –households where the adaptation
project is to be implemented
2. Determine sample size
3. Decide sampling technique
4. Prepare sampling frame and sample households
5. Develop questionnaire taking into account the
variables affecting the adaptation capacity and
pretest the questionnaire
7. Check data in the field & minimize non-response
5
6. Organize field survey – selection and training of
enumerators, organization in groups with division
of responsibility including supervisory
responsibility
EXAMPLE OF HH SURVEY: DUMANGAS FARMERS
ARE WILLING TO PAY FOR ADAPTATION PROGRAM
 Gay (2005) conducted a survey in coastal town of Dumangas in
the Philippines to know if farmers are willing to pay to reduce
their vulnerability, and if so, how much do they value a planned
adaptation program to climate change.
 Using CVM, a WTP survey was conducted among 450 hhs; 391
(87%) were willing to pay for planned adaptation program, 59
were not.
 The mean and median WTP for planned adaptation program were
PHP34.37 and PHP23.96 per month, respectively.
 Factors influencing farmers’ WTP for planned adaptation
program were education, farm experience, farm size, knowledge
about climate change, land tenure, access to credit, and
household income.
 Relocation for people affected by sea level rise was the least
preferred option
Source: Defiesta, Gay (2010). Social Vulnerability and Willingness to Pay for Adaptation to Climate Change of Farmers in
6
 Provision of alternative livelihoods and training that are less
affected by climate change were the most important planned
adaptation projects for farmers
EXAMPLE OF HH SURVEY: SOUTH AMERICAN
FARMERS ADAPT TO CLIMATE CHANGE BY
CHANGING CROPS
Estimating a logit model across 949 farmers in 7 countries, Seo
and Mendelsohn (2008) found that both temperature and
precipitation affect the crops that South American farmers choose.
Farmers choose fruits and vegetables in warmer locations and
wheat and potatoes in cooler locations. …. Global warming will
cause South American farmers to switch away from maize, wheat,
and potatoes towards squash, fruits and vegetables.
Predictions of the impact of climate change on net revenue reflect
not only changes in yields per crop but also crop switching.
The paper use data collected in 7 South American Countries. The
Household surveys asked detailed questions on farming activities
during the one year period of July 2003 to June 2004.
7
Seo, S. Niggol and Robert Mendelsohn (2006). An analysis of crop choice: Adapting to climate change in South American farms. E C O
LOGICALECONOMICS67(2008)109–116
ERROR OR BIAS
Error due to
sampling
design and
sampling
technique
Nonsampling
error
Total
error
All other
error over
and above
the
sampling
error
8
Sampling
error
HOW TO MINIMIZE SAMPLING
ERRORS – POINTS TO CONSIDER
 Suggestions to minimize sampling errors
• Use of appropriate probability sampling technique
• Minimize stages of sampling – design effect
• Determine sample size taking into account the
variance of the main indicators/attributes of the
universe
• Prepare updated sampling frame so as to minimize
the non-response
10
 Sampling error depends on:
• Sample size
• Sampling technique
• Heterogeneity of universe
SOURCES OF NON-SAMPLING ERROR
Nonsampling
error
Respondent
enumerator
processor
• Strategic bias
• Under/over- reporting
• Non-response
• Lack of training
• Poor coding and editing
of questionnaire
• Mistake in data entry
• Programming errors
11
Instrument
• Definition/specification
• Defective
questionnaire
• Defective Measurement
tools
SOURCES OF NON- SAMPLING
ERRORS – EXAMPLES
Strategic Bias: While
piloting questionnaire of
NMIS cycle 3, it was found
that some respondents
were reporting false info
about sources of drinking
water with a strategy that
they will get tube well.
Error from enumerator due to lack of training: (i)
Use of lead question; (ii) Lack of probing
12
Specification error:
Unless all agricultural
crops are included in
a survey it is difficult
to enumerate income
of farmers from
agriculture
HOW NON-SAMPLING ERRORS CAN OCCUREXAMPLES
Because irrigated farms are less sensitive to
climate, where water is available, irrigation is a
practical adaptation to climate change in
Africa.
A specification error of
dry land or irrigated
crops can change the
result of the study!
Strategic bias (eg,
reporting less revenue)
from the farmers with
irrigated farms can
change the conclusion
of the study!
Source: Kurukulasuriya et. al (2006). Will African Agriculture Survive Climate Change? The
World Bank Economic Review.
13
Using data from a survey of more than 9,000
farmers across 11 African countries,
Kurukulasuriya et. al (2006) estimates how farm
net revenues are affected by climate change
compared with current mean temperature.
Revenues fall with warming for dry land
crops and livestock, whereas revenues rise
for irrigated crops. At first, warming has little
net aggregate effect as the gains for irrigated
crops offset the losses for dry land crops and
livestock. ….The final effects depend on
changes in precipitation, because revenues
from all farm types increase with precipitation.
HOW TO MINIMIZE NON-SAMPLING
ERRORS – POINTS TO CONSIDER
 Use of structured questionnaire and FGDs – skip pattern, codes
 Use of survey manual with standard definition and specification
 Pre-test of questionnaire
 Use of mix of instruments – structured and unstructured
questionnaire/FGD
 A thorough training of enumerators with background knowledge on
the subject
 Good rapport building with respondents
 No lead question; no guess
 Use of probing questions – who, what, where, why and how
 Request for time if the questionnaire is not completed in one sitting
 Interview place – calm and quiet environment
 Improved coding of questionnaire
 Data cleaning and editing
14
 Programming for data entry and double data entry to minimize
possible entry errors
15
THANK YOU