Download Lessons learned (NEEP) - Evaluation Resource Center

LESSONS LEARNED: NEEF
With consideration of the evaluation and the conclusions drawn, the following recommendations are made for
future project planning, design, implementation, monitoring and evaluation of projects:
Issue
Recommendation
Policy and Regulatory change
Policy and regulatory change should not be targeted where it is not firmly
within the project’s span of control or influence unless there is a strong
commitment from a suitable stakeholder to champion this initiative
Design due diligence
The misalignment of scope and resources and the critical error of targeting the
revision of building codes that effectively did not exist, highlights the need to
conduct a stringent due diligence of the project design.
The inception workshop and inception report are critical for the project team
to take ownership of and internalize the scope. The projects should not
proceed without this critical step.
Project design structure (avoiding
policy and regulatory change)
GBCNA is an example of a clever project adaption that successfully leveraged
industry interest and commitment, achieved improved awareness and created
a platform for future energy efficiency in the country, that are not subject to
cumbersome government processes. Project designs that can incorporate
these characteristics may have greater success.
Definition of Outcomes
Output statements should be interrogated to check that they are relevant,
specific, attainable, measurable and realistically timed, i.e. that they are
SMART. The activities that support the outcome should also be aligned and
tangibly contributing towards the outcome and the emission reduction
goal. This recommendation is relevant to the design stage when the outputs
are being defined, but also at implementation stage, especially during the
inception workshop and as part of the ongoing M&E process.
This is a reiterating the GEF / UNDP guidelines for project development, but is
emphasized as critical to successful delivery.
Scope and resource alignment
Ensure there are adequate financial and human resources assigned or
committed to the project for the range of activities planned. The NEEP
probably needed three project managers with the support of a project
assistant/administrator to have delivered on the full design scope of the
project.
At least a high-level work breakdown structure and resource allocation should
be done to check the realistic delivery of the scope within budget and with the
available resources.
Positioning of the NEEP in the
REEEI and project management.
The REEEI faced challenges with implementation and did not leverage the
strength of its influential position and capacity as center of technical
excellence. Constraints related to capacity and skills are addressed separately
below, but perhaps the key to unlocking this inefficiency lies with a stronger,
structured project management approach focused on outcomes rather than
activities.
It is strongly recommended that, when it is not possible to find a technical
expert with suitable project management experience, the team structure for
similar projects have a full time project manager with part time technical
specialist support. It is further recommended that project management
capacity and structures be developed within the REEEI (NEI) to position it as a
stronger delivery partner.
Project management and reporting
culture
Project reporting is often handled as a report on the good progress only and
the tendency is to hide or downplay the challenges. A culture should be
instilled amongst project managers to use reporting as an opportunity for
raising concerns, communicate risks and appeal for assistance, inputs or
guidance as necessary.
Issue
Recommendation
Monitoring and Evaluation.
A comprehensive M&E plan and tracking during project implementation
against indicators and outputs are critical to demonstrate success and to
inform adaptive management. As this is already a requirement of GEF
funded/UNDP administrated projects, the necessity of this can only be
reemphasized.
A project that cannot demonstrate tangible progress in terms of the goals,
should be stopped.
Skill/competency and capacity
constraints.
It is critical to recruit and obtain the support of full time, suitably skilled
resources for a short-term, intensive project such as NEEP. It is strongly
suggested that a skills/capacity development strategy be incorporated where
this risk is identified. For example incorporating capacity development into
contracts, recruiting widely for specialist positions, but identifying a candidate
to shadow the specialist. In both these instances the contract conditions and
performance requirements can be structured to enable, track skills transfer
and to ensure retention of the ‘trainee(s)’. This addition may come at a
premium, but can be capped and included in the budget if properly planned.
The project team should furthermore ensure the requirements for studies are
clearly defined, that the terms of reference (technical evaluation criteria) for
work outputs form Consultants are clear and suitably specific, that the
evaluation and selection of service providers identify the required
competencies and that delivery is closely managed to the required outputs
Stakeholder participation.
It is essential to ensure the correct stakeholders are identified and involved
from the onset; that strong, jointly beneficial partnerships are actively
established towards a common goal; and that support is lobbied for
throughout the project.
Again, this is repetition of the existing guidelines, but proved a major barrier in
revising the building codes without the buy-in of the relevant ministry.
Co-funding contributions
In-kind co-funding contributions should be assessed in terms of the realistic
and specific contribution it will make towards the project goal and outcomes
and alignment with project scope.
While this project showed a co-funding contribution of >US$5 million, the
designed project implementation scope was limited to the GEF funding
component of US$859,000.