Download Modelling where few have modelled before:

How can we know if EU
cohesion policy is successful?
Integrating micro and macro approaches to
the evaluation of Structural Funds
John Bradley
EMDA (Economic Modelling and Development Systems)
[email protected]
Copy of paper available on
www.gefra-muenster.de
National Development Plans
and Structural Funds
• Large-scale investment aid for physical
infrastructure, human resources, production
incentives
• EC and local (public & private) co-finance
• Targeted at lagging EU member states
• Massively expanded after 1989
• Implemented through multi-year National
Development Plans
Regional
GDP 2001
GDP per head
(PPS), 2001
Part I
Situation
and trends
< 50
50 - 75
75 - 90
90 - 100
100 - 125
>= 125
No data
Index EU 25
= 100
Source: Eurostat
Two aspects of cohesion policy
• Policy design and implementation: usually a
pragmatic process driven by local political
wishes with local and EC oversight
• Implementation success depends on
institutional capabilities and fiscal constraints
• Policy evaluation: (ex-ante, mid-term and expost)
• Logically the two aspects should be interlinked. In practice they tend not to be (but
ESRI “Investment Priorities” an exception)
Structural Fund
impact evaluation:
micro versus macro techniques
Micro (bottom-up)
Macro (top-down)
Level of
disaggregation
High (individual projects)
Low (sectoral aggregates,
whole economy)
Use of theory
Weak (judgemental,
CBA)
Strong (macroeconomics)
Model calibration
Judgemental/informal
Scientific(?)/econometric
Policy impacts
Treatment of
externalities
Informal/implicit/ranking Formal/explicit/quantified
/some quantification
Limited or ignored
Included/explicitly
modelled
How can we know if EU
cohesion policy is successful?
Integrating micro and macro approaches to the evaluation
of Structural Funds
(Bradley, Mitze, Morgenroth & Untiedt, March 2006)
Paper available on:
www.gefra-muenster.de
Implementing a micro-based
approach to evaluation
•
1.
2.
3.
4.
Welfare economics and the underlying rationale
for public expenditure
Public goods
Corrective pricing (due to presence of
externalities)
Targeted interventions (information
asymmetries)
Redistribution (agriculture, social housing: but
mainly through tax and social welfare system)
Public goods:
evaluation criteria
Corrective pricing: evaluation criteria
Targeted interventions:
evaluation criteria
Redistribution: evaluation criteria
Two strands to the macro debate
on cohesion policy effectiveness
• A political-economic literature that
stands back from technical analysis, but
argues in terms of theoretical paradigms
• An empirical literature that examines the
issues empirically, using a variety of
different analytical models.
Impact of recent research
in economics
• New Trade Theory (Helpman &
Krugman, 1985)
• New Growth Theory (Grossman and
Helpman, 1991)
• New Economic Geography (Fujitsa,
Krugman, Venables, 1999)
Empirical studies of cohesion:
Two methodological approaches
• [1] “Testing” methodologies:
Testing a null hypothesis (e.g., cohesion policy
has no effect)
• [2] “Impact evaluation” methodologies:
Tracing out complex causal chains of policy
consequences, and quantifying impacts
Macro evaluation:
[1] Hypothesis testing
• Ederveen et al, 2002:
“Funds and games: the economics of European cohesion
policy”
“Fertile soil for Structural Funds? A panel data analysis
of conditional effectiveness of European cohesion
policy”
• Midelfart-Knarvik & Overman, 2002
“Delocation and European integration: is structural
spending justified?”
Ederveen et al, 2002
• Used Barro-type regresions over the period
1960-65 to 1990-95 for 13 EU countries.
• Found no statistically significant “cohesion
policy” effect (except for Ireland!)
• Critique: Looks only for growth impacts (i.e.,
ignores “level” impacts); inappropriate data
sample; crude panel-regression model)
Mitelfart-Knarvik & Overman, 2002
• Focused on role of cohesion policy on industrial
location, as it affects the interplay between
agglomeration and dispersion forces
• Finds that cohesion policy influences
endowments, but endowments do not appear to
feed through to changes in production
structure
• Ireland also an outlier, due to pre-cohesion
policy investment in human capital.
Macro evaluation:
[2] Modelling causality and impacts
• Be aware of the “built-in” limitations of the
type of model selected: I-O, CGE, growth,
macro-sectoral
• Implement an appropriate level of sectoral
disaggregation on the production side
• Nest cohesion policy mechanisms within wider
domestic and global drivers of growth
• Address the difficult issue of model
“calibration”
The uses of the macro models
• Constructing internally consistent medium-term
baseline scenarios or forecasts
• Analysis of conventional policy shocks (external
environment, domestic policy, etc.)
• Analysis of complex policy shocks like a EU
Structural Funds)
Key issues arising from the
macromodel-based research
• Need to have a more explicit treatment of
FDI
• Need to evaluate carefully fiscal and
monetary crowding out mechanisms
• Need to incorporate migration
mechanisms, and treat labour inputs in
more detail.
Contexts for Structural Fund
impact analysis
• The model as a global framework for
economic analysis
• The model as an explanatory framework
for the study of growth and cohesion
• The model as an action framework for SF
impact analysis
Construction phase vs Use phase
in cohesion policy
• During construction phase, there will be large
demand-side impacts. These vanish after
completion (i.e., after 2013/15 for next NSRF)
• Increased stocks of infrastructure and human
capital can generate long-tailed supply-side
impacts
• The size of the supply-side impacts depend on
the appropriateness and effectiveness of the
NSRF
Physical infrastructure: PI
• Demand-side impacts (implementation):
PI  IG  I (total investment)
 (Keynesian multiplier)
 impact on GDP
• Supply-side impacts (mainly post-implementation):
PI  increased stock of infrastructure (KPI)
 boost to output/productivity
Human resources: HC
•
Demand-side impacts (implementation):
HC  Income & Public expenditure
 Keynesian multiplier
 GDP
•
Supply-side impacts (mainly post-implementation):
HC  stock of human capital (KHC)
 boost to output/productivity
A serious methodological challenge
• Ex-ante impact analysis of “yet-to-beimplemented” NDPs
• Is the NDP appropriate? How effective
will be the implementation?
• Strict monitoring and evaluation can
help, but do not guarantee success
Infrastructure and human capital
interaction effects
• The links between infrastructure and
human capital are difficult to measure.
• A parallel improvement in both is
probably necessary
• But we cannot say much about the
optimum balance between them within
an NDP
What macromodel?
QUEST versus HERMIN
• QUEST: quarterly; one-sector; modelconsistent expectations; no CEE models
of new EU member states
• HERMIN: annual; four-sector (+); autoregressive expectations; applied to “old”
EU and new EU member states
HERMIN versus QUEST
• The issue of “crowding out”
CEE economies operating below capacity
Public goods and externalities
Modest domestic co-finance requirement
Quasi-euro zone, so no monetary impacts
Presenting model-based cohesion
policy impacts
• Difficult to define an appropriate
counterfactual baseline scenario.
• Difficult to assign values to the spill-over
(or externality) elasticities to different
countries in he absence of empirical
research.
• Macro impacts are complex, and GDP is
an imperfect indicator
Long –run impact of cohesion policy
• Policy impacts build up gradually over time, so use
accumulated change in GDP relative to the no-policy
baseline
• Big SF injection implies big shock, so normalise SF
expenditure as a percentage of GDP
• Define the SF cumulative multiplier as the
accumulated percentage change in GDP compared to
the no-poicy baseline caused by a one-percent of GDP
SFshock
Ordinary policy multiplier
Change in GDP
---------------------------------Change in public investment
Cumulative policy multiplier
Cumulative percentage change in GDP
------------------------------------------------------Cumulative percentage share of SFs in GDP
Evolution of accumulated SF injection (as % of GDP) and the
accumulated percentage increase in the level of GDP:
Czech Republic: NDP 2007-2013
100
90
80
70
60
50
40
30
20
10
0
06
0
2
7
8
9
10
11
12
13
Cum SF as % GDP
14
15
16
17
Cum % incr GDP
18
19
20
0
2
Evolution of the Czech cumulative SF multiplier
Cumulative Multiplier
3
2.5
2
1.5
1
0.5
20
20
19
18
17
16
15
14
13
12
11
10
9
8
7
20
06
0
Classifying performance:
NDP 2007-2013
• Star performers: Czech Republic,
Slovenia, Estonia, Poland, Portugal
• Average performers: Latvia, Romania,
Spain, Hungary
• Under performers: East Germany, the
Italian Mezzogiorno, Greece
What explains differences in outcomes?
• A common set of “implementation” and
“effectiveness” parameters
• Nimble Small Open Economies?
Estonia, Slovenia, Czech Republic
• Structures oriented towards growth
(Polish “eagle”, Portugal)
• Need for a “bottom-up” analysis
(measure => operational programme => CSF)
• Mix of measures vital; also institutional &
organizational abilities
Some conclusions
1. Structural Funds, on their own, will never produce
cohesion (for example, of the dramatic Irish
variety)
2. However, returns to well-designed and effectively
implemented NDPs are probably high
3. Micro-evaluation studies have not been systematic
4. The macro “testing” literature conclusions are
probably overly negative and pessimistic
5. The HERMIN/QUEST macro-modelling studies
and mechanisms may understate the potential for
accelerated convergence
Towards a more constructive debate
• The Commission’s Cohesion Reports need to
draw on available analytical research (micro
and macro), even when critical
• Empirical approaches (“testing” and “impact
evaluation”) can always be improved, but only
examine limited aspects of cohesion
• Analysis needs to be broadened to include
insights from industrial strategy and other
policy frameworks (Vernon, Porter, Best),