Download Shifting Cultivation Stabilization Pilot Project

Validation Report
Reference Number: PCV: LAO 2008-48
Project Number: 29210
Loan Number: 1688-LAO(SF)
December 2008
Lao People’s Democratic Republic:
Shifting Cultivation Stabilization Pilot Project
Operations Evaluation Department
ABBREVIATIONS
ADB
BTOR
EA
IMLNR
Lao PDR
MAF
OED
PCR
PIO
RRP
SAPE–ANR
–
–
–
–
–
–
–
–
–
–
–
UNODC
–
VDC
VDP
–
–
Asian Development Bank
back-to-office report
executing agency
integrated management of land and natural resources
Lao People’s Democratic Republic
Ministry of Agriculture and Forestry
Operations Evaluation Department
project completion report
project implementation office
report and recommendation of the President
sector assistance program evaluation on agriculture and
natural resources
United Nations Office of Drugs and Crime (formerly United
Nations Office of Drug Control)
village development committee
village development plan
NOTE
In this report, “$” refers to US dollars.
Key Words
adb, capacity building, environmental conservation, institutional strengthening, income
generation, irrigation, lao people’s democratic republic, lao pdr, operations evaluation, natural
resource management, participation, project completion report validation, roads, shifting
cultivation, sedentarization, social infrastructure
Director
R. B. Adhikari, Operations Evaluation Division 1, OED
Team Leader
Team Members
S. Shrestha, Evaluation Specialist, Operations Evaluation Division 1, OED
C. Infantado, Portfolio Evaluation Officer, Operations Evaluation Division 1,
OED
B. Q. Cafirma, Evaluation Assistant, Operations Evaluation Division 1, OED
PROJECTCOMPLETION REPORT VALIDATION
A. Basic Project Data
Project Number:
Loan Number:
Project Name:
Country:
Sectors:
ADB Financing
($ million):
PCR Validation
Date:
29210
1688-LAO[SF]
Shifting Cultivation
Stabilization Pilot Project
Lao People’s
Democratic Republic
Agriculture and Natural
Resources
ADF:
5.6
OCR:
Approval Date:
United Nations Office of
Drugs and Crime
11 May 1999
Signing Date:
Project Officers:
15 July 1999
Name:
Cofinanciers:
Appraisal
Implementation
PCR
Validator:
Quality Control
Reviewer/Peer
Reviewer:
Yoshiharu Kobayashi
Robin Erickson
Brette E. Coleman
Henry Tucker
J. H. Mir
Akmal Siddiq
J. H. Mir
Sisavanh Phanouvong
Sisavanh Phanouvong
David Moffatt,
Consultant
Total Project
Costs ($ million):
Loan/Grant
($ million):
(SDR equivalent)
Total Cofinancing
($ million):
Borrower ($ million):
Beneficiaries
($ million):
Others ($ million):
Effectiveness
Date:
Closing Date:
Location (HQ or
RM):
HQ
HQ
HQ
HQ
HQ
HQ
HQ
RM
RM
Director:
September 2008
Approved
8.8
Actual
9.2
5.6
5.6
4.1
1.3
2.1
1.8
0.1
1.4
0.1
13 October 1999
15 October 1999
31 December 2005
From (year)
04 May 2007
To (year)
7 July 1998
22 July 1998
May 1999
November 2001
December 2001
February 2002
March 2002
June 2003
July 2003
August 2004
September 2004
May 2005
June 2005
December 2005
January 2006
February 2007
16 February 2007 28 February 2007
R. B. Adhikari, OED1
Samjhana Shrestha,
Evaluation Specialist,
OED1
ADB = Asian Development Bank, ADF = Asian Development Fund, HQ = headquarters, OCR = ordinary capital resource,
OED1 = Operations Evaluation Division 1, PCR = project completion report, RM = resident mission, SDR = special drawing
rights, SF = special fund.
B. Project Description (Summarized from the RRP)
(i) Rationale: Most shifting cultivators live in poverty, their farming systems unable even to meet
household food consumption needs. Their villages lack reliable access and basic community facilities.
An already critical situation is being exacerbated by increasing population pressure, leading to shorter
rotations and consequent lower yields, and to accelerated deforestation with its attendant impact on
erosion and catchment water balance. Despite these problems, no tested and proven technologies
have been widely introduced to create sedentary farming systems that can generate adequate cash
incomes on land presently used for shifting cultivation.
(ii) Expected impacts: (a) reduced poverty among the targeted shifting cultivators; (b) increased food
production and incomes through on-farm and off-farm activities; (c) improved socioeconomic
conditions; (d) elimination of illegal opium production; and (e) conservation of natural resources.
2
(iii) Objectives or expected outcomes: (a) Improve the income of upland farmers; and (b) conserve
natural resources through (i) establishment of environmentally sustainable, diversified sedentary
farming systems as alternatives to shifting cultivation; and (ii) provision of basic rural infrastructure in
an environmentally sustainable manner.
The Project would also pilot test promising farming technologies and establish appropriate land
allocation procedures through extensive beneficiary participation.
(iv) Components and/or outputs:
(a) Component 1: Institutional strengthening and capacity building
(1) Capacity strengthening for land use planning and allocation. Technical assistance to
assist the Ministry of Agriculture and Forestry (MAF) in procedures for land allocation in upland
areas, support land allocation through the establishment of additional land use planning and
land allocation teams, and strengthen natural resource data management capacity of MAF
through training and the establishment of an integrated management of land and natural
resources (IMLNR) office.
(2) Strengthening agriculture and forestry services. Train provincial and district subject matter
specialists; domestic and international training for selected staff of MAF and the provincial
government of Houaphanh; provide training facilities at the provincial and district levels for
trainers and beneficiaries; strengthen agriculture, veterinary, and aquaculture services; and
establish two seed-multiplication nurseries.
(b) Component 2: Diversified sedentary farming systems development
(1) Consulting services.
(2) On-farm trials and demonstrations. Test and evaluate promising varieties and/or species,
technologies, and improved farming systems, and establish on-farm trials and demonstrations
in each village.
(c) Component 3: Village-based development
(1) Village and community development. Assist villagers in forming village development
committees (VDCs), establish a local development center in each subproject area, train 10
villagers per village as trainers, and provide funds for volunteers to support village planning
and development activities.
(2) Irrigation. Develop 160 hectares (ha) of irrigation in seven locations with permanent diversion
structures and irrigation canals.
(3) Water supply and sanitation. Construct new or renovate existing water supply systems.
(4) Rural tracks. Provide small-scale earthmoving equipment for road construction to upgrade
about 50 kilometers (km) of rural tracks using International Labour Organization's labor-based
road construction method.
(5) Village revolving funds for agriculture and income generation. Provide a seed fund of
$157,000 for short- and medium-term credit in a project revolving fund managed by the Lao
Women’s Union, provide training for the Lao Women’s Union, monitor needs for long-term
credit, and assist beneficiaries in gaining access to the Government's Agricultural Promotion
Bank.
Although not noted as a subcomponent in the project description, the activity schedule refers to the
reduction of opium poppy cultivation.
(d) Component 4: Rural infrastructure development
(1) Rural access roads. Upgrade or construct 55 km of access roads.
(2) Rural markets. Construct three rural markets and support market research in
assessing potential for new products, developing market and prices information, and identifying
opportunities for value-adding and processing of local products.
(e) Component 5: Project management
(1) Provide office space, office equipment, service vehicles, and motorcycles.
(2) Provide incremental staff for project management and implementation supervision at the field
3
level.
(3) Finance operational and travel costs, per diem expenses, and training in administrative and
accounting procedures.
Although not specified as a subcomponent, the report and recommendation of the President (RRP)
stated that an environmental awareness campaign focusing on forests, fisheries, and biodiversity
conservation generally would be designed and implemented.
C. Evaluation of Design and Implementation (PCR Assessment and Validation)
(i) Relevance of design and formulation: The project completion report (PCR) affirmed the relevance of
the Project at design stage. The Project attempted to address the key concerns of the national
Government as expressed in its national development plan for reducing poverty and improving access
to public services, while providing economic opportunities to remote upland communities. It was
consistent with the Government’s New Economic Mechanism, initiated in 1986 with the main goals of
economic stability and increased market orientation. This included stabilizing shifting cultivation, which
was given as one of the five priorities of the national agenda. Since 1976, the Government has
addressed the challenge of shifting cultivation, its initial solution leading to the resettlement of upland
villages in new, more accessible locations. More recently, its policy developed to incorporate in-situ
initiatives aimed at sedentarization, and the Project provided the opportunity to pilot a new approach
through natural resource management.
When the Project was designed, the country strategy and program of the Asian Development Bank
(ADB) for the Lao People’s Democratic Republic (Lao PDR) targeted sustainable economic growth
through infrastructure development, connecting feeder roads to the main road network, markets, and
other commercial opportunities. It specifically promoted the inclusion of ethnic groups in mainstream
economic activity. Consequently, ADB’s country strategy and program was directed at rural inhabitants
in an attempt to increase their participation in the market economy and to stabilize cultivation systems
that were then recognized as unsustainable.
The validator concurs with this and notes further the relevance of the Project to some of today’s global
concerns. The forests of the Lao PDR are decreasing at an alarming rate, with the irreplaceable loss of
valuable forest resources and the detrimental impact on climate. There is worldwide concern for the
control of illegal opium production, and the financial and practical involvement of the United Nations
Office of Drugs and Crime ([UNODC], formerly the United Nations Office of Drug Control) is evidence
of the relevance of the Project in this respect.
(ii) Project Outputs :
(a) Component 1: Institutional strengthening and capacity building
(1) Capacity strengthening for land use planning and allocation. The PCR makes no
reference to the development of procedures and policies for land allocation in upland areas
and, while a central database and facility for geographic information system training has been
established in the renamed National Agriculture and Forestry Research Institute, this does not
appear to be the integrated and consolidated facility with the broad policy and management
functions envisaged in the RRP for the IMLNR. Of the proposed additional 10 land use
planning and land allocation teams, only two were in place by mid-2002, although after this
number increased to nine. However, less than a quarter of the targeted temporary land use
certificates were issued.
(2) Strengthening of agriculture and forestry services. The PCR records 250 training courses
at provincial, district, and village level, and the establishment of a veterinary diagnostic
laboratory. There is no record of the two proposed seed-multiplication nurseries or of the
center for the multiplication of fingerlings.
(b) Component 2: Diversified sedentary farming systems development
(1) Consulting services. Recruitment of the implementation consultants ran 12 months behind
schedule, leading to major delays in the whole Project; there was some small increase and
4
some reallocation of the inputs of the implementation consultants. The PCR makes reference
to monitoring and evaluation consultants but gives no further detail about them. Although not
specified in the RRP, UNODC also provided consultants, including a chief technical adviser
together with a consulting team, which (it is assumed) was initially made up of the volunteers
described in the RRP. However, the RRP refers to four international and eight local volunteers,
while the PCR gives an input of 156 international and 258 national person-months of UNODCfunded inputs. Recruitment of the technical assistants and/or volunteers from the United
Nations ran nearly 2 years behind schedule, and three of the four volunteers were reported to
have left after 1 year. There is reference to their replacement by National Project Professional
Personnel employees in the back-to-office report (BTOR) of May 2003, but no details are
given. In order to get a clear picture of exactly what consultant, volunteer, and National Project
Professional Personnel support was provided to the Project there needs to be a consolidated
schedule of these. Without this it is impossible to get an accurate picture of the level of support.
(2) On-farm trials and demonstrations. The PCR reports trials and demonstrations that very
closely match the detailed targets given in the RRP. The final reports of the implementation
consultants subject matter specialists, as noted in the BTOR of May 2003, do express
concerns for the sustainability of new sedentary farming systems. Although actions were
agreed to address these concerns, as late as December 2005, the review mission was still
drawing attention to the need to expand production areas in order to achieve commercial
scales in accordance with provincial development plans and to establish reliable and
sustainable market links.
(c) Component 3: Village-based development
(1) Village and community development. The PCR records the establishment of two local
development centers and assistance with the establishment and training of village
development plans (VDPs). The activities ran substantially behind schedule largely because of
problems in fielding and retaining volunteers. It is noted that VDCs were being hastily
assembled, women’s issues were not being addressed, and village infrastructure was being
installed prior to VDCs being developed. While this situation significantly improved in later
stages of the Project, it would seem that a full participatory planning process in producing
VDPs that formed the basis of subsequent actions was not always followed.
(2) Irrigation. This appears to have exceeded targets, although it is not clear what is meant by
semipermanent schemes, which constitute nearly half the total area.
(3) Water supply and sanitation. The installation of gravity-fed water supply schemes and pit
latrines met or slightly exceeded targets.
(4) Rural tracks. Slightly exceeded targets
(5) Village revolving funds for agriculture and income generation. The PCR again indicates
that each village did set up savings and credit groups and the Project provided seed capital of
around $200,000. However, the PCR shows an expenditure on this component of only
$0.5 million against an estimated cost in the RRP of $2.3 million, and it is difficult to see how all
these targets were met at such a reduced cost. Some clarification of this is required, as the
only reference to this aspect is in the review mission’s BTOR of December 2002, at which
stage only 10 villages had savings and credit groups.
(d) Component 4: Rural infrastructure development
(1) Rural access roads. Construction of these was in accordance with the RRP; the PCR quotes
a cost per kilometer of $24,700.
(2) Rural markets. Three markets were established in accordance with the RRP. The PCR also
makes note of the construction or rehabilitation of 15 schools, which was not envisaged in the
RRP. However, the support to marketing and processing included in this component in the
RRP does not appear to have been provided. There is reference in review mission BTORs of
the possible provision of a consultant in these aspects, but there is no evidence that this ever
happened.
(e) Component 5: Project management
The PCR simply noted that a project implementation office (PIO) was established, equipped, and
staffed, and that the implementation consultant provided training. At validation, note was taken of
5
the considerable length of time required to develop a functional office and project management
system; the PIO was not fully staffed until mid-2002, and even at the end of that year it was being
reported that the PIO was having difficulty preparing annual budgets. The support of the provincial
government is fully acknowledged but it is important from the point of view of lessons learned to be
realistic in recognizing the difficulties and time scales in developing local management of projects
in remote locations and in involving people with no experience of this type of management.
(iii) Project Cost, Disbursements, Borrower Contribution, and Conformance to Schedule: The PCR
reports a total project cost of $9.2 million against an appraisal estimate of $8.8 million. There is an error
on the table of cost breakdown by project component in the PCR basic data (page iii, table 3). This
appears to have arisen because the individual cost lines give only the foreign exchange costs. Putting
in corrected estimated costs drawn directly from the RRP, there seem to be significant differences
between the estimated costs of individual components and the actual costs, and the basis of these
differences might warrant further elucidation.
Component
Capacity Strengthening
Farming Systems
Village Development
Rural infrastructure
Project Management
Total
Estimated Cost
$’1000
1,140
1,655
2,266
1,853
853
7,751
Actual Cost
$’1000
1,764
1,965
493
2,983
1,613
8,820
Disbursement was very slow in the early years of the Project. Taking figures given in the PCR, the
major catching up did not take place until the fifth year; by year 4, total disbursement was still only 68%
of the anticipated level, jumping to 92% of the anticipated level in year 5. These figures underline the
very slow start made by the Project.
The Borrower contribution decreased from an estimated $1.8 million at appraisal to an actual
contribution of $1.4 million. This was more than compensated for by an increase in UNODC
contribution from an estimated $1.3 million to $2.1 million; the ADB contribution remained as estimated
at $5.4 million.
The Project fell seriously behind schedule, which was to a large degree retrieved by a 12-month
extension. The PCR validly attributes the slow implementation to
(a) delay in recruiting implementation consultants,
(b) delay in recruiting UNODC consultants (this is also compounded by the problems related to the
original use of UN volunteers), and
(c) problems relating to road design.
To this list might perhaps be added the inexperience of the Executing Agency (EA), the Provincial
Government of Houaphan, and an inadequate allowance in the project design for the establishment of
a fully functional PIO.
(iv) Implementation arrangements, conditions and covenants, related technical assistance,
procurement, and consultant performance: The Project was something of a pioneer in its highly
decentralized implementation arrangements. The PCR noted the benefit of these in that money could
move quickly from the Ministry of Finance to the province, which was where all expenditure occurred.
It does, however, point to the downside to this in the minimal involvement of central technical agencies,
particularly MAF and its various departments. The PCR stated that “the communication linkages and
technical support from national institutions were tenuous at best.” Given that this was a pilot project
intended to have application more widely within the country, this was a weak point in the design and
implementation of the Project.
The PCR noted two failures by the Government and EA in meeting the loan conditions and covenants:
6
(a) lack of provision for the maintenance of the rural access roads; and (b) no benchmark survey was
undertaken at the beginning of the Project and, as far as the validator is able to ascertain, the baseline
data given for 2004 simply draws on district statistics.
Noting observations made by the Project Completion Review Mission, the deteriorating state of the
roads must be a matter of some concern.
The validator additionally notes that not all VDPs were properly prepared through a participatory
process, as reported by review missions. Also, although an office for IMLNR was indeed established
(but subsequently renamed), it does not appear that this actually resulted in the integration of activities
amongst the component agencies as was intended.
The PCR recorded three sets of consultants: implementation, monitoring and evaluation, and UNODC.
Recruitment of all consultants ran late. The situation with UNODC is particularly complicated and not
particularly clarified in the PCR or the review mission BTORs.
Overall it would appear that the performance of the implementation consultants was satisfactory,
although the PCR states that the team leader had to be replaced, and it is noted elsewhere that the
timing of some of the inputs did not necessarily accord with the Project’s needs. There is little reference
anywhere to what input was provided and exactly what was achieved by the monitoring and evaluation
consultants. UNODC was not originally a supplier of consultants but had a role in the deployment of
volunteers. At an early stage it is reported that three of four volunteers resigned after serving only 1
year. The blame for this is reportedly the attitude of the UNODC chief technical adviser who, in the
BTOR of the December review mission, is described as "creating a poisonous atmosphere in the PIO"
as well as being a major cause of the failure to undertake proper participatory village planning. UNODC
eventually agreed to replace him. It then appears that volunteers were replaced by local professional
staff, but under the supervision of UNODC.
The performance of the contractors is given as satisfactory, although it does appear that there was a
lengthy wrangle over the design, costs, and construction methods, such that it was eventually agreed
that the International Labour Organization should be brought in to support the road engineers (BTOR,
November 2001). The cost of road construction was still higher than original estimates, although
savings on bridge construction are said to have balanced this.
(v) Performance of the Borrower and Executing Agency: The PCR reported on difficulties in the
provision of counterpart funding, particularly where this involved actual cash as opposed to book
transactions related to duties and taxes. The PCR notes serious shortcomings in the EA’s ability to
fulfill its functions due to lack of familiarity with procedures, communication limitations caused by the
language barrier, and a general failure of the EA to comprehend some of the project components. The
validator also draws attention to the EA’s inability to field the required number of land use planning and
land allocation teams until an advanced stage of the Project. The PCR does state that performance of
the EA did improve and was satisfactory by the end, particularly after the greater involvement of the
resident mission. All of this would seem to have been entirely predictable and fault for any failures must
lie more with the design of the Project than with the EA. The validator concurs with the satisfactory
rating.
(vi) Performance of the Asian Development Bank: The PCR rated the performance of ADB partially
satisfactory, citing the frequent changes of project officer (seven changes in 6 years); poor
communication between ADB, the PIO, and the implementation consultants; and the erratic timing of
review missions. The validator concurs with this and notes the improvement in project performance
after the greater involvement of the resident mission. This should provide pointers for future similar
projects.
D. Evaluation of Performance (PCR Assessment and Validation)
(i) Relevance: The PCR rated the Project relevant, citing a large number of considerations in support of
this conclusion—focus on livelihoods, conformity with Government and ADB policies, sustainable
7
management of resources, poverty reduction in the project area, and improved land allocation
procedures. The validator fully concurs with these assessments, although some points may more
appropriately refer to effectiveness rather than relevance of the Project. In 2005, a review of the Project
as part of a sector assistance program evaluation on agriculture and natural resources (SAPE–ANR)
for Lao PDR also rated it highly relevant to the country contexts, government policies, and the need to
develop suitable and balanced approaches in order to stabilize shifting cultivation and reduce
cultivation and consumption of opium poppy. However, it pointed out that the different elements of the
Project can at times be conflicting in the short to medium term, noting in particular that too rapid a
reduction in opium poppy without immediate diversification into readily marketable crops can cause
serious hardship to farmers, who have relied on opium for cash incomes, and it therefore has a
detrimental effect on poverty. There was also a design problem in one of the major component of the
project (road).
It was entirely relevant in the context of this type of project that it should be highly decentralized, but in
so doing the Project lost some of the interaction with, and support from, the line agencies that were
critical to the activities of the Project; there was little evidence of any real ownership of the Project by
these line agencies.
(ii) Effectiveness in achieving outcome: The PCR rated the Project effective. Despite its slow start and
considerable early problems, the Project has been effective in reducing poverty in the project area,
increasing food production, reducing opium production, and protecting natural resources. The PCR
addressed each project component and confirms that each has been effective in achieving what it was
intended to achieve, as summarized below.
(a) Capacity building of government staff was constrained by late deployment of counterpart staff and
the lack of appropriate technical skills. However, at the community level, it enabled all villages to
form VDCs and to formulate their land use plans. (However, it is noted at validation that only
495 households of the target number of 2,259 received temporary land use certificates.)
(b) The introduction of diversified farming systems has significantly enhanced incomes and reduced
the area of land under shifting cultivation.
(c) Village infrastructure development, particularly irrigation, has been effective in stabilizing the
farming systems and giving farmers the confidence to adopt new approaches.
(d) The construction of roads and markets has made it possible to move into a market-orientated
economy by allowing physical access and reducing marketing costs.
(e) Finally, the Project was considered effective in introducing new approaches to land allocation, land
use planning, and long-term user rights. Furthermore, the Government has a greater appreciation
of the need to involve communities in the management of natural resources.
Taking the package of interventions as a whole, the validator concurs that within the project area, an
effective outcome has been achieved. Capacity building has provided the ability to plan and implement
changed farming systems and village infrastructure development, the introduction of alternative crops
and farming systems has provided alternative livelihood options, and the exploitation of these has been
made feasible by the improved access.
(iii) Efficiency in achieving outcome and outputs: The PCR rated the Project less efficient, the major
reasons for this low rating being the high cost of the access road and the high cost and sometimes
inefficient timing of consultant inputs. On the positive side, the PCR pointed to efficient capacity
building, efficiently implemented village development based on a highly participatory process, and a
more efficient approach to land certification. A meaningful assessment of efficiency is difficult in a
project which is developing methodologies and farming systems and is in a remote area with harsh
terrain. Despite the circumstances of the Project, an economic analysis gives an economic internal rate
of return for the project at 13.8%, which exceeds the opportunity cost of capital and is slightly better
than the projected rate at appraisal. Without the access road the benefits of the Project could not be
realized and so, despite its high cost per kilometer and per head of population served, it was an
essential component of the Project, but it could be argued that it should not be attributed as a specific
project cost but a cost that would have to be incurred anyway, serving purposes far beyond those of
the Project.
8
(iv) Preliminary assessment of sustainability: The PCR assessed the Project likely sustainable, however
this rating needs to be qualified on the basis of the following considerations. The PCR pointed to the
high cost of maintenance of the roads and, although it has been agreed that beneficiaries would
contribute around 20% of the costs, it will still be difficult for the province to meet its obligations.
Subsequent to the PCR, it has been reported that the Government has provided maintenance funds to
support routine maintenance of major infrastructure such as the two access roads, and provides
emergency support in case of landslide. While a more positive view is taken of the ability to maintain
the smaller structures, the PCR casts doubt on the viability of the Project’s solution to technical support
of training villagers as subject matter specialists, given the inability of government extension services to
support them. The November 2006 review mission BTORs comment on the post-project sustainability
plan that had been requested by ADB. This provided for a further 2 years of scaled-down project
support, but no commitment was received for the Government to meet the costs of this. While a lack of
commitment to this sustainability plan continues, the Government has apparently provided some
technical support by delegating its technical staff to be based in each of the local development centers
in order to support farmers to continue their income-generation activities. The continuation of such
support will be essential to the sustainability of project activities.
Alongside its local activities, Project sustainability must also be viewed in the context of the uptake and
scaling-up of piloted processes and technologies, and in this respect the validator notes significant
concern. There is no reference in the PCR to adoption of either the specific farming systems or to the
methodology of land use planning and allocation outside of the project villages. A pair of interlinked
projects was proposed in late 2004. In the preparation documents for these projects there is no
mention of the Project as a source of knowledge contributing to the design of these new projects, and
yet their aims are broadly in line with those of the Project. Quoting these and other ongoing projects,
the SAPE–ANR sector review of 2005 states that there is a need for clear interface and links among
these efforts, in order to assist them to benefit from each other’s experience and to meet their
objectives as pilot projects. The BTOR of the review mission of December 2005 suggested that a
workshop to exchange ideas amongst ADB and non-ADB projects include these projects, but this
appears to have amounted to no more than attendance by some of these projects at the final
workshop. A Project jointly funded by ADB and the International Fund for Agricultural Development is
scheduled for approval in 2009 The documentation available on this project at validation is not enough
to determine whether this may draw on the Project. It is understood that it has adopted some of the
approaches of the Project though the extent of this adoption is not specified.
The long-term sustainability of this pilot project depended on it being part of a planned continuing
process. This did not seem to be the case, and proposals for a second phase, although initially
entertained by ADB, were ultimately rejected.
The PCR has stated its concerns over sustainability of the locally implemented project, and the
validator concurs with these. However, some steps seem to have been taken to address these
concerns in terms of the Government making financial commitment to road maintenance and providing
some technical support to income-generation activities. It is clear that significant opportunities for wider
adoption of project technologies and processes within ADB and other projects have been missed. The
frequent changes of project officer have no doubt not helped in this regard. However, this is being
rectified at least on some new ADB projects and wider adoption may follow. The validator has
maintained the likely sustainable rating but qualifies this with the absolute necessity for the
Government to continue its funding of road maintenance and its provision of technical support to the
local development centers, and to concerted effort by ADB to adopt in its own projects the positive
processes and activities developed by the Project and to make every effort to ensure that the Project
findings are widely disseminated to other funding agencies and nongovernment organizations working
with people involved in shifting cultivation.
(v) Impact (both intended and unintended): The Project has had a high impact on the targeted
beneficiaries. Improved village infrastructure has freed up labor to be applied in more productive
activities, and has improved health with the incidence of malaria reducing from 13 per 1,000 to 9 per
1,000, and infant mortality decreasing from 34 per 1,000 to 26 per 1,000. Better access has opened up
markets and reduced the cost of transport of goods in and out of the project area. Irrigation has had a
9
major impact on food security and facilitated farmers to engage in cash cropping on settled areas. The
PCR recorded a 140% increase in household gross domestic product, a 58% reduction in poor
households, a 55% reduction in households with rice shortages, and a decrease in the area of swidden
cultivation from 1,200 ha shifting cultivation to 507 ha permanent cultivation (irrigated paddy areas
have increased from 54 ha to 319 ha). Income from weaving and handicrafts increased by 400%, and
the number of large ruminant livestock increased by 100%, poultry increased by 88%, pigs by 66%,
and goats by 188%. In addition, the area of opium cultivation has been reduced from 724 ha to zero,
and the number of addicts reduced from 230 to 36.
The only significant negative impact was the occurrence of landslides in some areas of steep terrain
where roads have been built or realigned. Note may also be made to references to logging in village
lands opened up by the roads.
E. Overall Assessment, Lessons, and Recommendations (Validation of PCR Assessment)
(i) Overall assessment: The PCR rated the Project successful noting that the approach to diversification
of incomes was successful, and that local communities took on a more involved and responsible role.
It accepts that circumstances inevitably mean that costs per beneficiary are high, which might have
been mitigated to an extent by a modified approach to the distribution of roads.
The validator concurs with this assessment but adds that, while the Project has succeeded in most of
its immediate objectives, the pilot interventions have only recently found application outside of the
project area and this would need to be maintained and extended. A continuing commitment by the
Government to maintain roads and provide technical support is also essential if the proposed rating of
the sustainability of the Project as likely sustainable is to hold.
(ii) Lessons: The PCR drew three lessons from the Project: (a) the need for more realistic
implementation schedules and recognition of constrained local capacity, (b) greater flexibility in project
design, and (c) more emphasis on training project implementation staff in project procedures. To
these lessons, the validator would add the recognition of problems caused by frequent changes of
project officer and the need to have systems in place that result in better transfer of knowledge
amongst projects at formulation and during implementation.
(iii) Recommendations: The PCR made four project-related recommendations:
(a) the wider application and adoption of piloted land allocation procedures,
(b) greater emphasis on infrastructure maintenance rather than on its extension,
(c) more cost-effective approach to access roads, and
(d) funding of the sustainability plan.
The PCR also made four general recommendations:
(a) realistic assessment of counterpart contributions,
(b) realistic scheduling of project activities,
(c) greater involvement of national agency staff in provincially implemented projects, and
(d) the need to grant finance capacity-building projects.
The validator has drawn particular attention to the broader issue of the greater involvement of the
resident mission in a project designed as a pilot project, and recommends that pilot projects should be
set within the context of a longer-term plan which includes specific proposals as to how the piloted
interventions might feed into the broader needs of the country. In particular, it is essential that
knowledge accumulated during implementation of the pilot project is fully reflected in the preparation of
potentially similar projects. A final recommendation stems from the apparent improvement in the
implementation of the Project once it was managed from the resident mission. For small projects in
remote locations and being locally implemented by staff with little experience of project procedures, the
support needed is far greater than in other projects, and in terms of providing greater management
support the resident mission is better placed in terms of both proximity and language skills.
10
F. Monitoring and Evaluation Design, Implementation, and Utilization (PCR Assessment and Validation)
It is very difficult to comment on the monitoring and evaluation of the Project. The loan covenants state that
“The Project Implementation Office (PIO) shall recruit an independent monitoring team to carry out benefit
monitoring and evaluation (BME) activities in coordination with the SUNV (a joint program of the
Government of the Netherlands and United Nations Volunteers) and the village development committees in
accordance with procedures and indicators agreed with ADB. Socioeconomic monitoring surveys shall be
carried out at the end of the second, fourth, and sixth year of project implementation. The Borrower shall
ensure that an initial benchmark survey shall be carried out by SUNV in collaboration with the PIO and
VDCs in the first year of the Project.” The PCR reports that monitoring activities started in 2003 and that the
baseline data as such was not captured, and says that alternative sources of baseline data were used,
making reference to Appendix 1. This appendix lists socioeconomic indicators and gives the source as
“project benefit monitoring and evaluation report and ADB PCR Mission, February 2007.” However, there is
no detail of the system of monitoring and evaluation and, although reference is made to monitoring and
evaluation consultants, there is no information on what this comprised nor any comment on the
methodology employed. Some clarification on this would be warranted.
The report of the SAPE–ANR sector review of 2005 is highly critical of the monitoring and evaluation of the
Project and states "As a pilot project, the relevance of the Shifting Cultivation Stabilization Pilot Project has
been compromised by the absence of a functioning monitoring and evaluation system that can define the
extent to which its pilot initiatives have yielded results."
G. Other (e.g., Safeguards, including Governance and Anticorruption; Fiduciary Aspects; Government
Assessment of the Project, as Applicable) (PCR Assessment and Validation):
It is noted that the EA lacked familiarity with, and acceptance of, proper ADB bidding procedures. Review
missions reported this as a particular problem in the contract for the veterinary diagnostic laboratory for
which procedures were not followed. However, ADB ultimately agreed that the contract could go ahead and
noted that had proper bidding procedures been followed the same contractor would have been selected.
H. Ratings
Relevance:
PCR
Relevant
Effectiveness in
Achieving Outcome:
Effective
Efficiency in Achieving
Outcome and Outputs:
Less
Efficient
Preliminary
Assessment of
Sustainability:
Likely
Sustainable
OED Review
Reason for Disagreement/Comments
Relevant
In respect of the immediate beneficiaries, and
contextually in terms of Government and ADB
policy and strategy, the Project was relevant.
Effective
The
project
interventions
provide
a
complementary set of actions which taken
together have had a significant impact on the
beneficiaries and in that respect the Project has
been effective.
Efficient
Although the PCR regards the Project as having
been less efficient, it may not be giving sufficient
weight to the fact that it was a pilot that the roads
served purposes other than the specific objectives
of the Project, and despite these considerations it
gave an economic internal rate of return of 13.8%
which exceeded the opportunity cost of capital.
The validator would see some justification in
according an "efficient" rating.
Likely
While the OED review has concurred with this
Sustainable
rating, it has done so with the absolute
qualification that recent Government commitment
to funding road maintenance and providing
technical support is continued, and that ADB
applies the successful activities and processes of
the Project in its future program and disseminates
these more widely. Ideally there would also be
11
H. Ratings
PCR
OED Review
Borrower and EA:
Satisfactory
Satisfactory
Performance of ADB:
Partially
Satisfactory
Partially
Satisfactory
Impact:
Overall Assessment:
Positive
Satisfactory
Positive
Satisfactory
Quality of PCR:
Reason for Disagreement/Comments
financial commitment to the Government’s postproject sustainability plan.
Despite early problems, performance was
ultimately satisfactory and it would probably be
more relevant to attribute those early problems to
a project design that did not give sufficient
cognizance to the circumstances of the Project.
Seven changes of project officer in 6 years had a
deleterious effect on the Project and it may be
noted that the Government did request more
stability in this, which was not forthcoming.
The Project faced substantial difficulties and
implementation fell well behind schedule. Given
the challenges of a project tackling complex
issues in a remote location, however, this is not
surprising. Following the midterm review, the
project proceeded with renewed impetus and by
the end of the extended implementation period it
had completed most of its planned outputs.
Satisfactory
ADB = Asian Development Bank, EA = executing agency, OED = Operations Evaluation Department, PCR = project
completion report.
I. Comments on PCR Quality
The validator considers the PCR to be largely satisfactory. It comprehensively covers all salient points and
presents cogent arguments for its conclusions. A number of points, however, are noted below.
(i) The approach taken to the Project is essentially in the context of a stand-alone project and there is little
comment on the apparent lack of wider application of processes, technologies, and systems that the
Project was intended to be piloting.
(ii) In the case of the sustainability of the Project, the weaknesses recognized in this by the PCR
essentially concur with the validator's view. The PCR does not carry these through to a more negative
overall assessment of the project's sustainability. There is some support for this view in terms of recent
Government commitments, and these would need to be maintained if the sustainability rating is to be
justified.
(iii) More attention might have been given to the way in which project monitoring and evaluation was
carried out and the related issues of the source and quality of the baseline data. Although there is
reference to monitoring and evaluation consultants, there is no comment on the methodology
employed or on the quality of the product. This makes any final conclusion on the reported impact of
the Project difficult to draw.
(iv) The PCR might have done more to clarify the quantity and nature of external support. The inputs of the
implementation consultants are spelled out but no details are given on the monitoring and evaluation
consultants. Even more importantly, there is a need to spell out exactly what support came from
UNODC; while the summary cost table (Appendix 5) gives a figure for consultants, there is a need for
some clarification of the mix of international consultants, national consultants, and volunteers
(international and national). A tabulation of all these support inputs would have been useful.
(v) There is an error in the basic data section of the PCR, (page iii, table 3), in which the costs given for
the appraisal estimates are only the foreign exchange costs (except for the total).
(vi) There are some large and surprising discrepancies between component costs at appraisal and at
completion, which would have merited comment.
(vii) The PCR would benefit from a revised project framework relating accomplishments to targets.
12
J. Recommendations for OED Follow-Up
A project performance evaluation may be undertaken in 3 years time, with particular focus on project
impact and sustainability.
K. Data Sources for Validation
(i) Project processing:
RRP for Loan to Lao PDR for the
Shifting Cultivation Stabilization Pilot project
Minutes of the Management Review Meeting
Minutes of the Staff Review Committee Meeting
Draft Minutes of the Loan's Board Meeting
Approval
April 1999
June 1998
March 1999
May 1999
(ii) Project administration:
The following BTORs:
Inception Mission
Review Mission No. 1
Review Mission No. 2
Review Mission No. 3
Review Mission No. 4
Review Mission No. 5
Review Mission No. 6
Midterm Review Mission
Review Mission No. 7
Review Mission No. 8
Review No. 9 - Handover
Review Mission No. 10
Review Mission No. 11
Project Completion Review
September 1999
February 2001
May 2001
October 2001
November 2001
February 2002
June 2002
December 2002
March 2003
April 2004
October 2005
March 2006
November 2006
March 2007
(iii) Project completion:
ADB's PCR
Sector Assistance Program Evaluation
November 2007
Agriculture and Natural Resources in Lao PDR
13
REGIONAL DEPARTMENT’S RESPONSE TO THE PROJECT COMPLETION REPORT
VALIDATION REPORT
On 7 November 2008, the Operations Evaluation Department (OED) received the following comments
from the Lao Resident Mission, Southeast Asia Department:
“We thank you for considering our comments of 16 October 2008.
Further, apart from applying in other projects the successful processes, technologies and
systems piloted under the Project (referred to in Section J, para. 1 of the Validation
Report), we would like to emphasize that they have been fully adopted in a JFPR funded:
Alternative Livelihood for Upland Ethnic Groups of Houaphanh province.”