Download Evaluation of Behavioural Additionality

Evaluation of Behavioural Additionality
Concept Paper1
1 Introduction
This note sets out some thoughts on the conceptual framework and survey design that
could be used to evaluate the behavioural additionality exhibited by firms affected by
science, technology and innovation policy measures. After a discussion of the
definition of behavioural additionality, in the context of broader additionality
considerations, the paper considers some aspects of firm strategy, and hence the
dimensions in which firm behaviour may be affected. A third section considers the
range of innovation policy instruments that may achieve such effects, with a particular
emphasis upon R&D grants. The discussion moves on to discuss operationalisation of
the concept – how it may be measured in the context of evaluation surveys with
sample questions proposed. Finally, issues which could form the basis of a future
research agenda are raised.
2 Definition of Behavioural Additionality
The concept of behavioural additionality emerged as an observed phenomenon in
early evaluations of collaborative R&D programmes when it was found that
traditional formulations of additionality did not capture well the effects of
programmes on firms (Buisseret et al, 1995). In simple terms the range of
additionality perspectives is:

Input additionality: a concern with whether resources provided to a firm are
additional, that is to say whether for every Euro provided in subsidy or other
assistance, the firm spends at least an additional Euro on the target activity.
Hence this approach raises issues of displacement (see David et al, 2000, and
Usher, 1994). The implication is that grants have to be targeted to activities
that would not have taken place. This perspective dominates State Aids
regulations and in particular the EU requirement that subsidy should not be
directed at a firm’s “core” activities. This argument shows a lack of
understanding of the role of R&D in relation to firm strategy (implying that
uncertainty is lower of it is within the core business – not necessarily true for
technological uncertainty at least) but demonstrates the importance
additionality frameworks can have on the real world. Bach and Matt (2003)
critique this approach by arguing that it rests on three assumptions all of which
would be challenged within a structuralist-evolutionary perspective:
o That there is a clear link between input and output if innovation
activities;
o Divisibility and constant returns to scale of the innovative activity; and
o No difference in the nature of the output generated by public and
private funding.
1
Based on a paper produced for IWT and OECD TIP. Thanks are due to Jan Larosse, Patries Boekholt
and Wolfgang Polt for their comments on this draft.
1

Output additionality: the proportion of outputs which would not have been
achieved without public support. This of course begs the question of what is
an output – are we dealing with the linear expectations of papers and patents,
or with effects such as sales of new products or applications of processes and
services. While this counterfactual scenario is simple in concept it requires
major assumptions about the connection between intervention and what is
measured.

Behavioural additionality: the difference in firm behaviour resulting from
the intervention. The assumption is that the behaviour is changed in a
desirable direction, though an evaluation should also be sensitive to perverse
effects, for example encouraging firms to take risks that they cannot afford.
Behavioural additionality has generally been ignored by econometric studies of the
effects of R&D support which focus on input additionality, where estimates are made
of additional R&D expenditure or output additionality, whereby firm performance is
compared between recipients and non-recipients of public support. These are both
interesting questions but in neither case is causality examined, nor is there an explicit
or implicit model of how the firm uses public support. Such a model is integral to the
concept of behavioural additionality.
The behavioural perspective is multi-layered. At its simplest it is confined to the
funded project and is manifested through questions about whether the support caused
the firm to increase the scale of its activity in the chosen area, its scope in
technological or other terms, and whether timing was affected (did the resources
allow acceleration of development?). Questions on these issues have become fairly
standard in evaluations in Europe at least.
However, underpinning such questions are broader issues which raise the question of
how support interacts with and affects the strategies and capabilities of firms. These
link closely to the systems failure rationale for public intervention. In strategic terms
typical questions might involve:
 Whether the support helps to overcome a lock-in failure by introducing a firm
to a new or extended technology or market area;
 Whether the support is building new networks or coordinating systemic
innovations such as those requiring establishment of standards, either between
firms or between firms and the science base;
 Whether the support has incentivised the firm to acquire new competences,
ranging from project management skills, through various acquired
technological and market routines and capabilities, and possibly encompassing
innovation and commercialisation capabilities (for example securing
intellectual property or raising venture capital investment).
Some further discussion of the topic has appeared in the literature. Davenport and
Grimes in assessing the effects of company support in New Zealand found that the
behavioural additionality concept provided an explanation for their findings.
Managers and policy administrators, they argue, can exploit the occurrence of
behavioural additionality to maximize the impact of a research policy, on the basis
that modified behaviour is likely to strengthen a policy's latent ability to influence the
2
creation of output additionality. They conclude that managers and policy-makers
should be identifying those interventions that lead to sustained improvements in
managerial practice, and in firm competitiveness. The aim then should be to manage
their diffusion within firms and throughout industries.
Luukkonen has criticised the additionality concept on the grounds that it is
insufficient to reveal the usefulness of public support. She cites empirical evidence to
show that projects deemed as trivial by firms at the time of support may in the long
run turn out to have been highly significant in their impacts, for example because they
may build capacity in areas where firms have suffered from what Salmenkaita and
Salo (2001) have subsequently labelled “anticipatory myopia” (a concept that related
well to “technological lock-in). Against this criticism it may be argued that whether a
project is additional or not is a separate question from that of the success of a project.
Indeed high additionality may easily be associated with an increased risk of failure
because the intervention has tempted a firm to move beyond its competences or to
undertake a project which was more risky than usual. Both of these may be positive
effects overall. There is also the case of high additionality where the policymaker
incentivised the firm to move in the wrong direction because the policymaker
misjudged the direction of technology or the market.
Empirical evidence from Hervik (1997) in a study of successive policies in Norway
found a clear trade-off between additionality and economic impact probably for the
first reason given above.
How can the different types or manifestations of additionality be reconciled with
current thinking on rationales for innovation policy? The market failure rationale
needs little explanation here. Following Arrow (1962) the argument follows the
general line of positive spillovers, non-appropriability and uncertainty creating a
situation in which there is under-investment in research (and by implication in other
knowledge-based innovative activities) in comparison with the socially desirable
level. As argued previously (Metcalfe and Georghiou, 1998) the market failure
perspective has been highly successful in providing a general rationale for policy
intervention but it is inherently unable to provide specific guidance on policy
prescriptions.
Lipsey and Carlaw (1998) in a study aiming to show that neo-classical and
structuralist evolutionary policies lead one to different conclusions in a technology
policy evaluation, engage in a discussion of how additionality (or in Canadian
terminology, incrementality) would be assessed under each perspective. They argue
that a neo-classical approach would insist at least on what they term a “narrow test of
incrementality” being that “some technology is developed or installed that would not
have been produced in the absence of the policy or programme under consideration”.
This corresponds to output additionality as discussed above. They argue that the neoclassical approach could also go further to demand a test of “ideal incrementality” in
which the policy is demonstrated to be an optimal use of government expenditure.
This invokes a series of tests attributed to the Canadian economist Dan Usher:
 The project must be the least costly way to undertake the desired level of R&D
investment;
 Social benefits must exceed the subsidy (including transaction costs,
deadweight and other leakages); and
3
 Discounted benefits must exceed discounted costs of intervention.
It is clear that the information requirements of these tests far exceed what is likely to
be available in any practical situation and may in themselves place undue transaction
costs upon the subsidy. The crucial criticism which Lipsey and Carlaw make is that
the structuralist/evolutionary perspective would apply only a “weak test of
incrementality”, defined as “something the policy makers are trying to do has
happened as a result of their expenditure of funds”. The difference from the neoclassical perspective is that, with no attempt at optimality, the desired effects are less
clearly specified (to allow for inherent variability between firms) and the effects
looked for include structural changes and enhancements of firms’ capabilities. For
innovation policies such as R&D subsidies where the main aim is to provide resources
to the firm it seems reasonable to expect both kinds of effect to be evident (the
targeted product and the longer term enhancements). However, when we come to
consider innovation policies which do not involve the provision of finance this
distinction becomes crucial.
Effects may or may not be intentional on the part of either the policymaker or the
recipient of funding. It is also of interest to assess the persistence of such effects.
While input and output additionality operate at a point in time, behavioural
additionality effects may be expected to endure beyond the period of R&D and to be
integrated into the general capabilities of the firm (Georghiou, 2002). Bach and Matt
distinguish these dimensions of behavioural additionality by labelling them “cognitive
capacity additionality” but the keyword here is capacity.
3 Firm Strategic Behaviour
3.1 Hierarchy of Decisions and Effects
There is an extensive literature on firm strategies and their relation to technology
and/or innovation strategies (see for example Tidd et al, 2001 for a review). From this
some key elements of strategy emerge:
 Building competences
 Sourcing technology
 Leadership or follower strategies
 Managing assets complementary to innovation
 Protection of intellectual property
For large firms there are issues of R&D strategy, location of R&D geographically,
relations between corporate and business R&D, and increasingly managing external
relationships with universities, start-up companies etc as part of the “new industrial
ecology” (Coombs and Georghiou, 2002). While the majority of small firms innovate
through their supply chain relationships, the behaviour of new technology-based firms
may be different. Implications for strategy are discussed below.
4
Table 1 Levels and Sustainability of Behavioural Effects
Strategy
Short-term effect
Project in new business area
for firm
New market alliance
Prioritisation
Project in new technology
area for firm
New technology alliance eg
with user
Operationalisation New project reporting
procedures to comply with
monitoring requirements
5
Sustainable Effect
Developing capabilities in
new business/ market
Joint venture or supply
chain arrangement
SME shift from contract
research to manufacturing
Acquired technological
competences
Sustained technology
alliance
Acquisition of
management capability for
collaborative projects
Table 1 uses a categorisation of levels of effect proposed by Jari Romanainen of
TEKES, Finland. The idea is that the effect on behaviour could vary according to the
level. Examples are shown. Operationalisation is interpreted as referring to
management capabilities, prioritisation to project or technology choice and strategy to
the overall direction of the firm. The distinction is made here between short-term
effects, normally manifested during the life of the project, and sustainable or
persistent effects which are acquired competences.
In terms of identifying effects on firm strategies many possible dimensions could be
identified, including:






Knowledge acquisition
Human resources
Capital investment
Market position
Manufacturing or service provision
Corporate responsibility and sustainability
Considering each briefly:
Knowledge acquisition includes issues of how R&D is organised within the firm, for
example corporate versus business level R&D and linkages between them. In some
cases corporate R&D is only sustained because of the cumulative effects of public
funding. Locational decisions about R&D, including international ones, may be
influenced by technology policies (see also human resources below).
Increasingly knowledge acquisition has become a matter of managing external
networks. With the growth of collaborative R&D, outsourcing to specialist suppliers
and universities and the planned acquisition of start-up firms either on the market or
through corporate venturing, we are seeing the emergence of an “new ecology of
industry” (Coombs and Georghiou, 2002). Innovation policies founded in the systems
perspective place a heavy focus upon the formation and promotion of the resulting
networks and hence this is a fertile area in which to look for behavioural additionality
effects. However, it is important to get an assessment of the values of the linkages as
at one extreme they could be a cost rather than a benefit.
Human Resources can be a direct aim of technology policy, as with schemes that
subsidise the hiring of researchers, or an indirect result as in the case of a company’s
researchers upgrading their skills or qualifications within the context of a funded
project. Management skills can also be acquired as a result of taking part in a project.
Examples from past evaluations include SMEs learning about control procedures
through compliance with planning and monitoring requirements demanded by a
funding agency, or large firms using international collaborative projects as a means of
training managers in internationalisation skills. These acquired competences can be
significant for future firm performance. A recent example from a PREST evaluation
in Japan showed that a team which had taken part in a project did not develop
anything of great value within the project but subsequently went on to apply their
acquired knowledge in other more successful developments.
6
Capital Investment strategy is not at first sight a behavioural issue but it is possible
that R&D support may influence the location of a company’s facilities or even an
entire laboratory, with long term consequences for the region concerned and for the
company’s future networking. It is also possible that support may induce a firm to
acquire equipment that it would not otherwise have, and as a result move in a different
direction or in some cases the same direction more quickly.
Market Position is another area of possible influence. R&D may transform a
follower to a leader, on the basis of new processes for example. The innovative
project may also introduce firms to new customers or to new markets. These may
extend to products and services other than those initially supported.
Manufacturing strategy or strategy for service provision may also evolve in the
context of public support. This could be directly as result of a process-oriented project
or arise indirectly because the advance in a firm’s knowledge enables it to change its
production or service delivery methods. An example could be increasing use of ecommerce to reduce inventories.
Corporate responsibility and sustainability can be an explicit aim of a project or
form a further type of externality. For example innovative activity may result in
reduced use of material or energy inputs and in turn may stimulate a reorganisation
within the firm to take advantage of this.
An approach to measurement of behavioural additionality should consider the effect
on these dimensions of corporate behaviour.
4 Innovation Policy Instruments
4.1 Introduction
There is a wide range of policy instruments available for the promotion of innovation.
A recent Report by an independent expert working group reporting to the European
Commission proposed the taxonomy shown in Figure 1. From this it may be seen that
subsidies or grants to firms form only one means to support firms.
7
The classification system developed for this study divides Direct Measures as follows:
Direct Measures
Demand side
Supply side
Finance
Support
for
public
sector
research
Support
for
training
&
mobility
Services
Grants
for
industry
R&D
Information
&
brokerage
support
Networking
measures
Systemic
policies
Procurement
Regulation
Framework Conditions: Human resources, science base, Regulatory framework (including State Aid, Competition &
IPR, General fiscal environment
In principle behavioural additionality could be expected to result from any of the
measures but for the purposes of this study it has been agreed that during this phase of
the work the focus will be on grants to firms. However, in several countries the same
firms are likely to have benefited from innovation-support services provided by public
agencies and possibly from demand side policies and targeted changes in framework
conditions. For this reason it will be necessary to set the effects of grants in the
context of other measures experienced by the firms (and of other influences that act
on the aspects of behaviour that are of interest).
A useful framework is to classify the policy types by the deficiencies they seek to
remedy2. These may be summarised as:

Resources: Where there is insufficient resource, usually money, to undertake
the work, without public funds. This is generally the case for academic
research and is accepted to be so for certain areas of business R&D which are
highly uncertain and/or where social returns justify an investment which does
not meet private criteria.

Incentives: Where the scientific structures or the market do not provide
sufficient incentives for socially desirable behaviour, for example academicindustrial collaboration.
2
For an earlier classification of European policies by this framework see Metcalfe JS and Georghiou L,
Equilibrium and Evolutionary Foundations of Technology Policy, June 1998, STI Review No.22,
Special issue on “New Rationale and Approaches in Technology and Innovation Policy”,
8

Capabilities: Where organisations lack key capabilities needed for the
innovation process, for example the ability to write business plans or raise
venture capital.

Opportunities: This refers to the generation of opportunities for innovation
and provides one of the main justifications of public science.
Table 2 lists the main categories of measures available to policymakers, though it
does not capture the variety which can be achieved through differences in application
process and eligibility for participation, sectoral, technological or innovation phase
specificity, financial conditions and intellectual property frameworks to name but a
few characteristics. It also shows the deficiencies which particular types of measure
principally address.
Table 2 Policy Measures
Measure
Deficiency
addressed
Comment on Application
Support for basic
research
Support for public
research directed
to industry
Opportunities
Resources
Resources
Incentives
Capabilities
Opportunities
Directed to universities and public laboratories
Support for
training &
mobility
Resources
Capabilities
Grants for
industrial R&D
Resources
Incentives
Opportunities
Fiscal support for
R&D
Resources
Incentives
Offer non-discriminatory finance for R&D either by volume
or for incremental spend with no selection process.
Equity support for
venture capital
Co-location
measures
Resources
Incentives
Opportunities
Incentives
Compensates for deficiencies in VC market. Particularly
important in pre-seed and seed capital phases.
Increase innovation through proximity of industry and
science and critical mass effects. Include provision of
facilities for company labs on campuses, and establishment
of incubators, science parks and technology parks. Total
amount of R&D taking place in such environments is
Includes support for public sector scientific institutions with
conditions attached to increase the benefit to industry eg
prioritisation of areas of interest to industry, grants
conditional upon collaboration with firms, arrangements for
use of equipment belonging to either party, and incentives
and awards for collaboration. Public laboratories carry out
increasing proportions of contract research for industry,
extending the range of industrial R&D and potentially
bringing R&D to firms without the capability to do it
themselves.
As well as the basic production of graduates this covers
tailored courses or graduate schools for firms, training in
entrepreneurship and innovation skills, promotion of
secondments from science to industry and vice versa, and
employment subsidies for recruitment of researchers by
firms.
Gradual evolution away from support of near-to-market
research, large firms and single company support in favour
of support for SMEs and for collaborative, “precompetitive” R&D. Conditional loans are a variation on
grants. Principal value in providing finely tuned incentives,
for instance encouraging firms to do higher risk R&D or to
perform it in different ways eg collaboratively.
9
relatively small but is important in terms of generating new
firms that may subsequently grow large.
Information and
brokerage support
Capabilities
Opportunities
Include support for databases of contacts relevant to
innovation, advisory services, provision of information on
technological developments in other countries, technology
transfer offices, organisation of brokerage events, funding
for demonstrators and for use of patent databases. Almost
exclusively directed towards SMEs.
Networking
measures
Opportunities
Capabilities
Procurement
Incentives
Opportunities
Systemic policies
Incentives
Opportunities
Include support for clubs which exchange information and
for activities such as foresight programmes which aim to
develop common visions around which future oriented
R&D networks can be formed
The situation when a public agency places an order to
another organisation for a product or service that does not
yet exist. This means that R&D and innovation need to take
place before delivery. The procurer specifies the functions
of a product or system but not the product as such. This
measure is normally appropriate for large scale systems and
hence large as well as small firms. Measures are also
possible to stimulate innovative procurement between
private organisations, as in a supply chain. Can attract new
public resources into R&D and present firms with a
guaranteed market, thus lowering the risks attached to their
own R&D investments
These policies, for example cluster policies, aim to
stimulate interactions between strong concentrations of
industries supporting each other. Enhancement of private
investment in R&D through clusters comes through
increased awareness and confidence among firms, lowering
risks associated with innovation and providing linkage
between global players and their actual or potential subcontractors, including those further down the supply chai n.
4.2 Grants
Even within grant-based measures it is quite possible that the same firm will have
befitted from multiple grants, perhaps from different schemes. Behavioural effects
may have arisen from a specific instance or may be cumulative.
A further distinction that will have to be made is which aspect of a grant influenced
the firm and to what degree. A grant is more than a straightforward subsidy. It
involves an awareness and application stage which may influence firms by calling
to their attention the existence of opportunities or stimulating them to form a research
project proposal in a particular way that may be influential even if they do not apply
or do not succeed. This is particularly the case where close negotiations with experts
representing the funding agency take place.
The next step will be the award of a contract and the conditions imposed here also
affect firm behaviour. For example it is likely to embody rules about collaboration
and intellectual property rights.
10
Once the project is under way the influence upon the firm may result from
monitoring and reporting procedures. However, the main impact is likely to arise
from the content of the research that the firm is doing and the linkages that it forms
with any external organisations either in project, or more rarely, in a programme
context.
Finally, the influence of the public intervention may arise from any post-project
support measures that are available, including requirements to form exploitation
plans, and the provision of assistance with commercialisation through, for example,
advisory services which provide links to venture capital.
A question that policy makers may wish to pursue is that of the relationship between
the proportion or amount of public support and the influence on firm behaviour.
4.3 Other Policy Instruments
The evaluation will need to be aware of concurrent policies that may have influenced
the firm in the domain of the project. A recent evaluation of the innovation support
system in Finland found that firms which use innovation support systems generally
use the full range of support systems simultaneously and do not follow any linear or
sequential model.
Hence the grant application may have arisen from participation in a foresight exercise
or as a result of advice from a business support agency. Training and competence
acquisition measures may be taking place in parallel with the research and a firm in
receipt of grants may also have benefited from help with access to capital. In more
recent policies, the influence of support for clusters or technology platforms may also
affect the type of behaviours and linkages of interest here.
4.4 Spillover of Behavioural Effects
In principle, behavioural effects are liable to result in spillovers as good practice
diffuses to other companies. This will be particularly difficult to measure but may be
worth investigating further.
5 Operationalisation of the Concept
This section gives a first consideration as to how the concept of behavioural
additionality could be investigated.
5.1 Survey Objectives
Questions on behavioural additionality are likely in most cases to be set in a context
of a broader survey investigating impacts and effects, programme management issues
etc. Some of these questions will be of shared relevance with the main issue here.
However, the discussion below will assume that there is a “module” or separate
questionnaire on the behavioural additionality issue.
The overall aim of a survey is to identify the main elements of a company’s
technology strategy and its linkages to business strategy and then, with this reference
point, to assess the changes induced by research and innovation support measures.
11
5.2 Target Population
The target population is firms in receipt of assistance, with the possibility that firms
not receiving assistance could be used as a comparison group. However, it is unlikely
that the same questions could usefully be asked of all types of firms. Hence, it is
proposed that the firm population is divided into three hypothetical groups:
1. Large and relatively R&D intensive firms: In this case the supported projects
will probably be small in relation to the firm’s overall R&D spending and the issue
will be to examine strategic fit and the degree of influence. Technology and business
strategy will be connected but can to some extent be investigated separately. A
publicly-funded project is likely to be based upon motives of knowledge acquisition
rather than direct exploitability of results. Case-studies are the likely implementation
route.
2. Traditional SMES and medium-sized firms: In this case the project is likely to
be significant in relation to the firms overall R&D or even innovative activities as a
whole but possibly be peripheral in terms of current business strategy. It may be
expected that projects here are more development than research-oriented. Projects are
likely to involve an external provider of R&D or technical advice. Nonetheless there
is considerable scope for behavioural effects, including the discovery of the potential
of an innovative strategy.
3. Technology-based start-up firms: This group, if it in receipt of grants, is almost
certainly using them to support its central business strategy. There is no separation
between technology and business strategy and probably there will be only one R&D
project (though funding may come from more than one source). One issue of
separation will be between public equity inputs and grant funding, as both may be
applied for the same purpose. The firm will probably have minimal routines and
hence be highly likely to be influenced by external inputs. On the other hand
overcoming systems failures such as lock-ins will be less significant as the firms are
less likely to be “locked-in”.
There is also an issue of which part of a firm or person in a firm should be surveyed.
Some questions are likely to go beyond the capacity of a research manager to judge
and would require a response from a business manager.
6 Future Research Agenda
The preliminary thinking embodied in this paper raises many more issues than it
resolves. In consequence an agenda for further research, beyond a current round of
surveys and case studies remains which includes the following elements:
Ethnographic Approach
A more ambitious form of case study would be to track a grant through its life cycle
with the aim of cataloguing effects as they occur by observation rather than relying on
ex post hindsight.
Extension to Other Innovation Policies and To Policy Mix Issues
12
When firms are in actual or potential receipt of multiple incentives and forms of
support for innovation it is artificial to separate only one – the research grant. A future
phase of the study should extend the behavioural additionality concept to other
policies and to their combined impact. Certain types of policy (eg advisory and
consultancy services) are explicitly targeted at behaviour of firms. On the other hand
behavioural effects may interact positively or negatively between policies.
Consideration of Negative Behavioural Effects
This paper has mainly assumed that behavioural effects of grants are positive and
intentional. There is also the situation where they are negative and unintentional - or
even in some case negative and intentional. An example of negative and unintentional
would be to lead a firm into an alliance which turned out to be unproductive and
costly. An example of negative and intentional would be to persuade an SME to
perform high risk R&D when it cannot really afford to do so and should be devoting
resources to consolidation.
Long-term Learning and Persistence of Effects
Assuming that positive behavioural changes have been achieved, there is an
interesting question on how these are maintained – what is the capacity of a company
to retain learning in its routines. Does learning need reinforcement policies?
Econometric Analyses
Currently, econometric approaches are better suited to measurement of input
additionality. However, when a clearer understanding of behavioural additionality is
developed the possibility of statistical linkages to firm performance may be explored.
Linkage to Work on Corporate Technology Strategy
Understanding of changes in corporate technology strategy and the attendant
processes is itself proceeding at a rapid pace, not least under OECD auspices. Closer
linkage of this work with emerging findings from that activity could produce a better
understanding of the interaction between public support and private strategy-making.
.
13
References
Bach L and Matt M (2002) “Rationale for Science and Technology Policy” in
Georghiou L and Rigby J (eds) Assessing the Socio-Economic Impacts of the
Framework Programme, Report to DG Research (in press)
Buisseret TJ, Cameron H and Georghiou L (1995) “What difference does it make?
Additionality in the public support of R&D in large firms”, International Journal of
Technology Management, Vol.10, Nos 4/5/6 pp587-600
Coombs R and Georghiou L (2002) A New Industrial Ecology, Science April 2002
Davenport S, Grimes C, Davies J (1998) Research collaboration and behavioural
additionality: A New Zealand case study Technol Anal Strateg 10 (1): 55-67 Mar
1998
Georghiou L (2002) Impact and Additionality of Innovation Policy in Boekholt P (ed)
Innovation Policy and Sustainable Development: Can Innovation Incentives Make a
Difference? , IWT-Observatory:Brussels
Hervik A (1997) Evaluation of user-oriented research in Norway: the estimation of
long-run economic impacts in Papaconstantinou G and Polt W (eds) Policy
Evaluation in Innovation and Technology- Towards Best Practices, OECD
Lipsey RG and Carlaw K, (1998) Technology Policies in Neo-Classical and
Structuralist-Evolutionary Models, STI Review No. 22 Special Issue on “New
Rationale and Approaches in Technology and Innovation Policy”, OECD Paris
Luukkonen T (2000) Additionality of EU Framework Programmes Research Policy
29 711-724
Metcalfe JS and Georghiou L (1998) Equilibrium and Evolutionary Foundations of
Technology Policy, STI Review No. 22 Special Issue on “New Rationale and
Approaches in Technology and Innovation Policy”, OECD Paris
Salmenkaita J-P and Salo A (2001) Rationales for Government Intervention in the
Commercialisation of new Technologies, Helsinki University of Technology, Systems
Analysis Laboratory Research Reports, September 2001
Sheehan J (2001) Changing Business Strategies for R&D and their Implications for
Science and Technology Policy: OECD Background and Issues paper, OECD
document DTSP/STP(2001)29, OECD Paris
Smith K (2000) Innovation as a Systemic Phenomenon: Rethinking the Role of Policy
Enterprise and Innovation Management Studies, Vol.1, No.1, 73-102
Tidd J, Bessant J and Pavitt K (2001) Managing Innovation – Integrating
Technological, Market and Organisational Change, Second Edition, Wiley:
Chichester
14