Download MARVEL Policy Evaluation

Defence
TNO | Knowledge for business
Bringing Causal Loop Diagrams to life
MARVEL – Policy Evaluation
You want to investigate a policy intervention but you do not have the data for
a quantified Stock and Flow Diagram? Then MARVEL may help.
MARVEL adds information to a Causal Loop Diagram that is partly
expressed in qualitative terms. The results give an initial insight into the
expected effects of a policy and act as a guideline for finding the best policy.
Available budget
Need to change
+
Cost
effectiveness
+
Management
quality
+
Available
(external)
resources
+
+
+
Number of
+ employees
-
Software
management
organisation capacity
Costs
-
Resistance
against software
Management
knowledge level
Production
efficiency
+
-
Clear mandate
assignment
+
+
Decision quality
Decision speed
Number of
Number of ad hoc
parties involved
design decisions
-
Software usage
-
+
Software quality
+
+
+
-
+
Production
+ quality
+
Software
acceptance
+
Design quality
-
Outcomes
MARVEL offers insight into:
1 General problem structure
2 Dynamic responses to policy interventions
3 “Optimal” policy intervention
4 Most powerful mechanisms
1. Problem structure
As with a normal Causal Loop Diagram, a
MARVEL diagram clarifies the problem
structure but also identifies potential policy
intervention points and goals. In addition, it
shows the speeds and strengths of causal
relations.
Figure 1. Example of a MARVEL diagram.
Method in between CLD and SFD
MARVEL is a new analysis method. It is
designed as a tool for conducting a first
policy intervention analysis for problems
with a limited availability of quantitative
data.
MARVEL offers more analysis opportunities
than a Causal Loop Diagram and requires
fewer data than a quantified Stock and Flow
Diagram.
Focus on changes
MARVEL interprets a policy intervention as a
change in selected model variables. MARVEL
then determines the propagation of these
interventions through the model in a special
way, providing an initial insight into the
effects that may be expected from an
intervention.
Speeds and strengths
MARVEL adds speed and strength to each
causal relation, using qualitative terms such
as “fast” or “weak”.
Policy interventions
MARVEL allows you to specify the variables
in which to intervene. You can also specify a
qualitative term for the ease with which that
intervention point can be used.
Performance measures
MARVEL integrates performance measures to
evaluate the effects of potential
interventions. These measures allow you to
evaluate the intervention effects from an
aggregate to a detailed level.
2. System behavior
MARVEL analyzes the effects of policy
interventions over time. It determines how
changes imposed on policy intervention
points cause other aspects of a problem to
change and interact.
3. “Optimal” policy
MARVEL is implemented in a software tool
called TIM (see www.tno.nl/tim). This tool
enables you to find the interventions or
combinations of interventions that have the
best effects and delivers rapid insights into
the best policy options.
4. Most powerful mechanisms
MARVEL allows you to find the most
powerful mechanisms. These are the loops
with the greatest product of total loop speed
and total loop strength.
This static analysis option supplements the
dynamic system behavior analysis.
Defence
MARVEL – Policy Evaluation
TNO Defence, Security and Safety
Illustrative example
A company is looking for ways to speed up
the introduction of a business-critical
software package. Possible policy
interventions include a budget increase or a
clear mandate assignment.
A group of stakeholders is asked to
investigate the problem. They produce the
MARVEL diagram as shown in Figure 1.
Figure 1 displays the essential problem
aspects they identified, plus the causal
relations. Each causal relation has a speed,
shown by the delay stripes, and a strength,
shown by the type of line and its thickness.
Four potential policy intervention points are
recognized and marked with green control
buttons. In addition, three goals for
evaluating the policy intervention effects are
identified and marked with bullets.
The dynamic system response to each
potential policy option is determined. The
dynamic effects of a change in budget are
shown in Figure 2.
The performances of the various policy
intervention options are subsequently
compared, in this case for each relevant
moment in time as shown in Figure 3.
Based on these results, the company decides
to improve management knowledge, as this
option produces the best long term effects.
TNO Defence, Security and Safety provides
innovative contributions to the advance of
comprehensive security and is a strategic
partner of the Dutch Ministry of Defence to
build up the defence knowledge-base. We
employ our acquired knowledge for and
together with contractors.
Benefits of MARVEL
• Limited data needed for first policy
evaluation
• Easily understood concepts:
• causal effects
• speeds and strengths of causal relations
• intervention points and goals
• Qualitative terms for speeds, strengths,
goal relevance and more
Drs. ing. C.W. (Kees) d' Huy.
Oude Waalsdorperweg 63
P.O. Box 96864
2509 JG The Hague
The Netherlands
P +31 70 374 00 21
F +31 70 374 06 42
[email protected]
www.tno.nl
Figure 2. Response to an intervention on budget.
DenV S080127
Figure 3. Performance of four intervention options.