Download Essential - Participatory Impact Pathways Analysis (PIPA)

Limpopo Basin Impact Pathways
Workshop
15 – 17 November 2007
Birchwood Hotel, Johannesburg
Boru Douthwaite, CPWF Impact Project
CIAT, Cali, Colombia
Impact Pathways Matter
Why make Impact Pathways
explicit?
• People plan and implement projects (programs, countries
…) on the basis of their change models - their implicit
theories about how the world works, i.e., impact pathways
• If you can improve the impact pathways (IPs) you can
improve the practice, making impact more likely
• IPs show a project’s rationale and networks
– Help communicate what the project is doing
• More fundable
– Help with planning, including MTPs
– Provide a basis for evaluation
• Starting point for evaluation is a good model of what you think will
happen
• Provide information to support programmatic integration
• Provides impact hypotheses for ex-post impact assessment
PIPA makes Impact Pathways
explicit
It does so by developing two
perspectives ….
1. A problem tree that shows a linear logic
linking project outputs to project goal; and
2. Network maps that show the evolving
relationships necessary to achieve the goal
Problem Tree
>--------------Outcome-chain perspective------------------>
Impact pathways – a more complete
picture….
<-----the full picture---->
Network maps
>---------------Actor-orientated perspective---------
Foundations
• Adaptation of concepts from Program Evaluation
– Renger and Titcomb (2002) – problem trees
– Chen (2005) – program theory
– Mayne (2004) - performance stories
• Innovation histories
– Douthwaite and Ashby, 2005
• Social network analysis
– Cross and Parker, 2004
Identifying a linear logic linking
project outputs to project goal
Helps understand project rationale
and what needs to change
2. Outputs
What the project will produce
Identifying the evolving
network of actors needed
to achieve the vision
3. Vision
Integration of
both views
Products produced in the Workshop
Workshop
Road Map
1. Problem Tree
Where project is going - Goal
4. "Now"
network map
Necessary
relationships
in place
to produce
the OUTPUTS
6. Project
Scaling
Strategy
5. "Future"
network map
Necessary
relationships
to achieve
the VISION
7. Outcomes logic model
The outcomes the project will help achieve, how, and with whom
Outputs produced after the workshop
Impact
logic model
Impact
narrative
Outcome
targets and
milestones
Program-level
network maps
Technology
extrapolation
domain maps
Use of PIPA Outputs
Use of PIPA outputs
PIPA outputs used
Ex-ante impact
assessment
Essential:
Impact logic model, output targets, impact
narrative
Optional:
Extrapolation domain analysis, scenario analysis
Impact Pathways Analysis
Essential:
Outcomes logic model, output targets and
milestones, vision
Optional:
Impact logic model
Laying the foundation for
ex-post impact
assessment
Essential:
Outcomes and impact logic models
Optional:
Impact pathways analysis (that updates project
impact hypotheses)
Extrapolation domain analysis
Expectations
Causal analysis / Problem tree
PN 34 Improved fisheries productivity Problem Tree
4th LEVEL
Why?
Why?
Why?
3rd LEVEL
2nd LEVEL
Uncooperative
attitudes of fishers
with respect to
management of
fisheries resources
Lack of knowledge
of options of
enhancement
technologies
Lack of financial
resources capacity
to implement
scenarios for
improved fisheries
production
Determinants
Bad Harvesting
Strategies
Processing
limitations
Lack of
aquaculture
activities
Why is this problem happening?
1st LEVEL
Underused
Fisheries
Production
Capacity
Limited
Reservoir
Productivity
Problem
Depressed
Livelihoods
Start Here
Exercise
1
Refining and presenting
your problem tree
• Adapt or develop anew your project problem tree
for presentation (PowerPoint or cards)
• If cards, writing one problem per card
– Use one color for problems the project will address
– Use another for other problems
• Modify, and add as you see fit
– But don’t go into too much detail
• We’ll present the problem trees together with
project visions in plenary
Example of a Problem Tree
developed during an IP workshop
Why?
4th LEVEL
Why?
Why?
3rd LEVEL
Why is this problem happening?
2nd LEVEL
Uncooperative
attitudes of fishers
with respect to
management of
fisheries resources
Lack of knowledge
of options of
enhancement
technologies
Lack of financial
resources capacity
to implement
scenarios for
improved fisheries
production
Bad Harvesting
Strategies
Processing
limitations
1st LEVEL
Underused
Fisheries
Production
Capacity
Limited
Reservoir
Productivity
Problem
Depressed
Livelihoods
Lack of
aquaculture
activities
Start Here
Main
problem
to Goal
Determinants
Determinants
Problems to
34 (Improved Fisheries Production) Objectives Tree
toPNProducts
Outcomes
4th LEVEL
Fishers cooperate
in fisheries
management
Knowledge of
options in
enhancement
technologies
transferred
Financial resources
for improved
fisheries
productivity and
management
obtained
3rd LEVEL
Aquaculture
activities
implemented
Responsible
harvesting
strategies
Improved
processing
methods
2nd LEVEL
Optimized use of
fisheries
production
capacity
1st LEVEL
Goal
Increased
reservoir
productivity and
management
Enhanced
livelihoods
PN 34 Improved Fisheries
Management Problem and
Outcome Trees
Exercise
2
Deriving Products/
Outputs from the Problem
Tree
• The determinants are the problems the project is
directly addressing with its outputs
Hint: the use of the output solves the determinant
• Identify, write on cards and add to the problem
tree the outputs, showing which determinants
they correspond to
Example from Ground Water
Governance (PN42)
Some definitions

Activity – what we’re doing inside the project


Outputs – what we produce that other people make use of, that solve the
determinant






Hold IP Workshop
Improved rice variety; priority setting publication
Determinant – determinants are the problems the project is directly addressing
with its outputs
Next users – people and organizations who directly use the outputs
End users – the people and organizations that the next users work with. Often the
end users are the ultimate beneficiaries (e.g., resource-poor farmers), but not
always.
Politically-important actors– people and organizations whose support is needed
for project success
Outcomes – usually the results of the use of outputs by others (often come in
chains)





Promotion of rice variety by extension system
Adoption of rice variety by farmers
Higher rice yields
Higher income
More children sent to school
Level
project Levelofofinfluence
influenceofofaProject
change
High
C
O
N
T
R
O
L
Low
Research
activity
Output
target
Output
3 years
Outcome
Impact
10 - 30 years
Scaling Out and Scaling Up
• Scaling up - an
institutional expansion,
from adopters and their
grassroots organizations
to policy makers, donors,
development institutions
• Scaling out - spread of a
project outputs (i.e., a
new technology, a new
strategy, etc.) from farmer
to farmer, community to
community, within the
same stakeholder groups
Exercise
3
Develop a vision of project
success 2 years after the end of
the project
• Take 5 minutes to individually answer the question, then
develop common project vision by filling out Worksheet 1
Keep it realistic
– You wake up 2 years after your project has finished. Your project
has been a success and is well on its way to achieving its goal.
Describe what this success looks like to a journalist:
• What was the situation like before the project started (hint – look at the
problem tree)
– What were the unmet needs and requirements of next users and end users?
•
•
•
•
•
What are the next users now doing differently?
How are project outputs disseminating (scaling out)?
What political support is nurturing this spread (scaling up)?
What are the end users doing differently?
What are the benefits they are enjoying as a result of the project?
Causal Analysis
>--------------Outcome-chain perspective------------------>
Impact pathways – a more complete
picture….
<-----the full picture---->
Network maps
>---------------Actor-orientated perspective---------
Identifying a linear logic linking
project outputs to project goal
Helps understand project rationale
and what needs to change
2. Outputs
What the project will produce
Identifying the evolving
network of actors needed
to achieve the vision
3. Vision
Integration of
both views
Products produced in the Workshop
Workshop
Road Map
1. Problem Tree
Where project is going - Goal
4. "Now"
network map
Necessary
relationships
in place
to produce
the OUTPUTS
6. Project
Scaling
Strategy
5. "Future"
network map
Necessary
relationships
to achieve
the VISION
7. Outcomes logic model
The outcomes the project will help achieve, how, and with whom
What is a network?
A network is a collection of
people and / or things that
are connected to each other
by some kind of relationship



Many kinds of entities can be part of a network: people,
projects, organisations, documents, events, cities, countries,
etc.
Each of these entities can have different levels of influence in
the network, and
And there are many kinds of relationships that can link such
entities, involving transmission or exchange of information,
money, goods, affection, influence, infection, etc.
Advantages of network models

Actor-oriented descriptions:


Captures real-life complexity:



observable, understandable, verifiable
We are subject to multiple influences
We influence many others
Network models help us understand innovation
processes:


Innovation processes happen through different actors,
acting in networks
These interactions, relationships and influence are modeled
in network maps
Airline network
Road Network
Orlikowski and Hofman, 1997
How change happens
Improvements in poverty alleviation, food security
and the state of natural resources result from
dynamic, interactive, non-linear, and generally
uncertain processes of innovation.”
EIARD, 2003
A
network
diagram
(organisations
linked by a
project)
Influence pathway: actor + relationship + actor + relationship…
WRI
A plotted
network
diagram,
multiple
relations
FRANC
ENTERPRISE
CREPA
URBANET
MAs
ISSER
UDS
KNUST
SGs
CPWF
WRC
FGs
RVAU
UC
IWMI
MOFA
The human eye is an analytic tool of remarkable power, and eyeballing pictures of
networks is an excellent way to gain an understanding of their structure.
(The structure and function of complex networks, M. E. J. Newman)
A plotted network
diagram, one type
of relation
Network tasks…..
1. Identify relevant actors
2. Develop network diagrams of key relationships
(research, funding, scaling out and scaling up)
for


Your project now
Residual network 2 years after project has finished
3. Identify key levels of influence
4. Develop a scaling influence strategy (Worksheet 2)
Identify key actors
• Who are the actors involved in research, funding,
scaling out and scaling up in the area your
project is working in?
 Can be positions (eg. DDG-R) or organizations
 Remember actors at different scales: community/local, your
own organization, regional, national, international
Some types of actors
Government Organization
National Agricultural Research and Extension
Organization (NAREO)
CGIAR Centre
University
Ultimate Beneficiary
NGO
Research Organization
Private Sector
Donor
Other (please specify)
Exercise
Fill out an actor table
4a
ACRONYM
(Table 1)
FULL NAME
LOCATION
TYPE OF
ORGANIZATION
ROLE
IRRI
Int. Rice Research
Institute
Los Baños,
Philippines
CG Centre
Project
implementer
MOFA
Ministry of
Fisheries and Agric.
Accra,
Ghana
GO
Politicallyimportant
actor
FGs
Farmers Groups
Northern
Ghana
Ult. beneficiary
End user
PhilRice
Philippine Rice
Research Institute
Muños,
Philippines
NAREO
Next user
DfID
Dept. for Int.
Development
London,
England
Govnt. Org
Funding
agency
DDG-R
Deputy Director
CIAT, Cali
General of Research
CG Centre
Scaling-out
actor
Exercise
Develop a network diagram for
your project now
4b

Actors:


Use cards for nodes
Use different colour cards for different types of node


yellow = project implementers, blue = next user, green = end user,
red = politically-important actor, red with black dot = donor
Relationships


Use arrows to describe direction
Use colour to describe relationship type


Green = funding; brown = research / work; red = scaling out; black
= scaling up
Don’t use distance/length
Exercise
4c
Identify influence levels
and attitudes in the
networks
Actors:
 Construct
influence towers (0-3 chips) for key actors
 Indicate their attitude towards your project:
positive

neutral

negative

Exercise
5
Develop a future network
corresponding to the vision
Draw a second network showing how actors need to be
linked to achieve the vision
 Adjust the influence and attitude




Will the attitude of the actor remain the same or change?
Will the same actors still be equally influential?
Will there be new influential actors in the area?
Exercise
6a
Developing a Scaling Strategy
(Table 2)
Describe the most Why is the change
What are the
important
important to
project’s strategies
differences between achieve the vision? for achieving the
the two networks
change?
Exercise
Network Changes Achieved since
the beginning of the project
6b
Describe the most
important network and
influence changes since
the beginning of the
project
(Table 2)
Why was the change
important?
What were the project’s
strategies (e.g., codevelopment of
knowledge and
technology,
communication, political
lobbying, etc.) for
achieving these changes?
Identifying a linear logic linking
project outputs to project goal
Helps understand project rationale
and what needs to change
2. Outputs
What the project will produce
Identifying the evolving
network of actors needed
to achieve the vision
3. Vision
Integration of
both views
Products produced in the Workshop
Workshop
Road Map
1. Problem Tree
Where project is going - Goal
4. "Now"
network map
Necessary
relationships
in place
to produce
the OUTPUTS
6. Project
Scaling
Strategy
5. "Future"
network map
Necessary
relationships
to achieve
the VISION
7. Outcomes logic model
The outcomes the project will help achieve, how, and with whom
Developing the Outcomes Logic
Model (a description of the project’s impact
•
pathways)
Why (Dart, 2005)?
–
–
To evaluate or clarify the logic of the project
intervention
To provide a framework to evaluate the
performance of a project
•
•
Before, during and after
Evaluation can provide information to improve
decision making and enhance learning
Exercise
7
Outcomes Logic Model
(fill one table for all stakeholder groups)
Actor (or group
of actors who
are expected to
change in the
same way)
Change in
Practice
required to
achieve the
Project’s Vision
Change in KAS
required to
support this
change
Project
strategies to
bring about
these changes
in KAS and
Practice?
KAS = Knowledge, Attitudes and Skills
Exercise
8
Identify Outcome Targets
(prioritize changes from Outcomes Logic Model)
The key
outcomes the
project wishes
to monitor
Assumptions
SMART
outcome
target
Means of
verification?
By whom?
How?
SMART = Specific, Measurable, Attributable, Realistic, Timebound
Exercise
9
SMART Outcome
Target
Identifying Milestones
SMART Milestone
Means of
Who is
to be achieved in Verification? responsible
the next six
By whom? In for making
months
what form?
the
progress
described?
Where we are now
Future without
intervention
1
Time
5
4
Vision
3
2
Impact
Pathways
1
Impact Pathways
Workshop
Improvement
Impact Pathways Evaluation
2
Time
Future without
intervention
5
4
Adjusted
Impact Pathways
Adjusted
Vision
3
2
1
Reflection
Improvement
The process
3
Time
Future without
intervention
Actual
improvements
Visions
5
4
3
2
1
Impact pathways
Reflection
Workshops
Improvement
Identifying a linear logic linking
project outputs to project goal
Helps understand project rationale
and what needs to change
2. Outputs
What the project will produce
Identifying the evolving
network of actors needed
to achieve the vision
3. Vision
Integration of
both views
Products produced in the Workshop
Products produced
after the workshop
Workshop
Road Map
1. Problem Tree
Where project is going - Goal
4. "Now"
network map
Necessary
relationships
in place
to produce
the OUTPUTS
6. Project
Scaling
Strategy
5. "Future"
network map
Necessary
relationships
to achieve
the VISION
7. Outcomes logic model
The outcomes the project will help achieve, how, and with whom
Impact
logic model
Impact
narrative
Outcome
targets and
milestones
Program-level
network maps
Technology
extrapolation
domain maps
Use of PIPA Outputs
Use of PIPA outputs
PIPA outputs used
Ex-ante impact
assessment
Essential:
Impact logic model, output targets, impact
narrative
Optional:
Extrapolation domain analysis, scenario analysis
Impact Pathways Analysis
Essential:
Outcomes logic model, outcome targets and
milestones, vision
Optional:
Impact logic model
Laying the foundation for
ex-post impact
assessment
Essential:
Outcomes logic model and impact logic model
Optional:
Impact pathways analysis (that updates project
impact hypotheses)
Extrapolation domain analysis
Program network maps
Essential:
‘Now’ and ‘future’ network maps
Impact Logic Model for the Strategic
Innovations in Dryland Farming (SIDF) Project
Crop Related Outputs
Project Activities
carried out in Pilot
Sites with
stakeholders and
ultimate
beneficiaries
Crop Related Outcomes
3
Drought probability
map
Crop production
guides or manuals for
MoFA
1
Scaling
Out
Best-bet soil and water
conservation and
management options
manuals
Changes in
stakeholders
attitudes and
perceptions
Improved
knowledge of
stakeholders
at pilot sites
2
Improved cropping
systems in Northern
Ghana
Higher crop
yields
8
5
Farmers plant to
avoid crop loss due
to draught, majority
have intensified
cropping systems
Drought tolerant
varieties developed
Project Goals
Farmers using
appropriate
tillage methods
to conserve
soil moisture
Farmers using
drought probability
map and drought
tolerant varieties
Farmers routinely
generate organic
matter , e.g.
composting and
cover cropping
Soil and water
conservation improved
in farmlands in N.
Ghana
Scaling Up
Adoption of
National variety release
committee releases
varieties
Iterations of technologies
learning
and changes
cycle
in practice
Wider adoption of project outputs beyond
pilot sites
7
Adoption of project outputs by MoFA for
extension after project finishes
Stakeholders
modify and
innovate
Communities
trained on
efficient fish
production
techniques
Manuals on fish culture in
dugouts and dugout
maintenance
Manuals on
appropriate water
harvesting systems
11 up
Scaling
Methods developed to
institutionalize dialogue
about water use among
multiple users
Water Related Outputs
4
Scaling
Out
Dugouts
enhanced to
retain water
Communities
have knowledge
of low-cost
domestic waterharvesting
systems
Improved soil
fertility
More time for
income
generating
activities for
women
More water
available for
domestic needs
Adequate water
supply for dry
season
agriculture
Reduction in water
related diseases
Community
dugouts
efficiently
utilized for fish
production
Changes to
housing
structure to
meet water
harvesting
needs
Water Users
Associations formed
and strengthened
Improved utility of
community dugouts
10
Majority of
communities in
Northern Ghana have
constructed and are
using domestic water
harvesting systems
Effective management
of community water
resources
6
Water Related Outcomes
9
Improved
income for rural
households
High labour
productivity
Improved food
security and
rural
livelihoods
High land and
water
productivity
11
Says little about who does what …….an actor-orientated
perspective is missing
……provided by network mapping
….. as part of an impact narrative
Methodologies used for evaluating impact potential
Anticipated
Scenario analysis
Extrapolation
domain analysis
Exploring possibilities
Secured
Participatory
Impact Pathways
Analysis
Local,
close-toproject
Scope of
impact
Basinscale,
global
Link
between
regions
through biophysical
similarities
But socioeconomic
condition
also critical
EDA method
Project
Originorigin
Looking for
pixels with
similar socioeconomic (land
use factors)
Critical success
factors (soc)
selected
Representative
sites selected
WofE
Modelling
Homologue
P(soc)i
estimated
P(bio)i
estimated
Target
Looking for
pixels with
similar
climate (and
soil)
Extrapolation domain of aerobic rice in Asia
Areas that hold
promise for adoption
of aerobic rice
EDA: Summary
• Method in development
• Represents what what is believed, on basis
of
– Current experience
– Available global data
• Beware false prophets, BUT…
• …a rapid first step in exploratory dialogue
with new stakeholders
Exercise
10
Info needed for Extrapolation
Domain Analysis
What is the main change that your
project is contributing to that is likely
to scale-out and scale-up? Who will
adopt?
What are the factors likely to
influence scaling-out and scaling-up?
Where are your pilot sites in which
the change is starting to happen?
Volta – Research network
Next Steps
• Participants finish worksheets and send to
BFP-Impact Project (end of Nov)
• IP writes workshop report (end of Nov)
• IP send draft impact narratives to projects
(end of Jan)
• Participants respond to queries (end of Feb)
• IP write first draft of Limpopo Basin Impact
Pathways Working Paper (end of March)