Download Conducting effective research and analysis to support policy

Conducting effective research and analysis to
support policy delivery.
The Green Book: Appraisal and Evaluation in
Central Government (HM Treasury)
Tony Munton
Assistant Director
Research, Development and Statistics
Home Office
Evidence-based policies and indicator systems 2006
3 stages at which research and analysis should feed into the
policy process:
At the start… analysis to support a government decision to
adopt a new policy, or to initiate, renew, expand or re-orientate
programmes or projects, which would result in measurable
benefits and/ or costs to the public. This is the appraisal part
of the process.
In the middle…. Establish appropriate data collection so that
as the chosen option(s) is implemented, useful data can be
fed into the evaluation. This is the monitoring part of the
process
And at the finish ... to undertake a retrospective analysis of a
policy, programme or project at its completion, conclusion or
revision. This is the evaluation part of the process.
Evidence-based policies and indicator systems 2006
• Is there a clearly •identified
need?
What is to
be achieved by the intervention?
The ROAMEF model
• What would success look like?
• What would happen if we don’t intervene?
• Ensure that the lessons learnt from the ROAMEF cycle are
• What inputs, outputs and outcomes are
was actually implemented.
fed back into subsequent proposals. • Evaluate the option(s)• that
related to the intervention (develop a theory
Is there a market failure?
• Follows a similar process to the appraisalofbut
uses historic rather than forecast data.
change)?
What would
in
themeasure
absence of intervention?
• As with appraisal, an •evaluation
willhappen
identify
and
additional
costs
and
• What
are the fullthe
range
of options
that
might
benefits of implemented option(s) in comparison
to
the
“do
nothing”
option.
achieve
the
desired
outcomes?
• What will happen if we do intervene?
• What might different options achieve above
the current situation?
• Establish appropriate data collection so that as the
chosen option(s) is implemented useful data can be
fed into the evaluation stage of the cycle.
• Monitor delivery of milestones identified in the
• Identification of the “do nothing” option to be used as a baseline to compare the other options;
theory of change.
• Identifying the additional costs and benefits of each option in comparison to the “do nothing” option;
• Valuing the additional costs and benefits of each option in comparison to the “do nothing” option;
• Adjusting for the different timing and incidence of costs and benefits by discounting them to present values;
• Consideration of any risks and uncertainties;
• Initial consideration of what information may be needed at the monitoring and evaluation stages (i.e. to make sure the right
Evidence-based policies and indicator systems 2006
information is collected).
Practical steps for social research: Some
suggestions from the knowledge brokers
• Think about potential for forming multidisciplinary teams
• Use good theory (and theory of change)
• Go for effective evaluation strategies (RCTs)
• Think about effective communication of results
Evidence-based policies and indicator systems 2006