Download Management control system design and effectiveness

Management control system design and effectiveness
NYENRODE BUSINESS UNIVERSITEIT
Management control system design and effectiveness
Proefschrift
ter verkrijging van het doctoraat
aan de Nyenrode Business Universiteit
op gezag van de Rector Magnificus, prof. dr. E.A. de Groot
en volgens besluit van het College voor Promoties.
De openbare verdediging zal plaatsvinden op
woensdag 16 april 2008
des namiddags om vier uur precies
door
Anne-Marie Kruis
geboren op 10 februari 1977
te Leidschendam
Leescommissie
Promotoren:
Prof. dr. R.F. Speklé
Prof. dr. G.W.J. Hendrikse (Erasmus Universiteit Rotterdam)
Overige leden:
Prof. dr. J. Bouwens (Universiteit van Tilburg)
Prof. dr. E.A.G. Groenland
Prof. dr. F.G.H. Hartmann (Erasmus Universiteit Rotterdam)
Nyenrode Research Group (NRG)
Internet: www.nyenrode.nl/nrg
ISBN 978-90-8980-001-5
Cover lay out by Bingo Graphic Design, Rotterdam
Printed by HAVEKA, Alblasserdam
© 2008, Anne-Marie Kruis
All rights reserved. No part of this publication may be reproduced or transmitted in any form
or by any means electronically or mechanically, including photocopying, recording, or by any
information storage and retrieval system, without prior permission in writing from the author.
Contents
Preface.............................................................................................................................................. i
Summary.........................................................................................................................................ii
Chapter one – Introduction ......................................................................................................... 1
1.1 Point of departure...................................................................................................................... 1
1.2 The main themes ....................................................................................................................... 4
1.3 The nature of this study ............................................................................................................ 6
1.4 Outline of this thesis ................................................................................................................. 8
Chapter two – Configurations of control ................................................................................ 11
2.1 Management Control Systems ............................................................................................... 11
2.1.1 Purposes and dimensions of control............................................................................... 11
2.1.2 Complementarities........................................................................................................... 13
2.2 The transaction cost approach to management control......................................................... 15
2.2.1 The general argument...................................................................................................... 16
2.2.2 Control problems: activities and contingency variables............................................... 18
2.2.3 Control solutions: archetypes ......................................................................................... 20
2.2.4 Matching problems and solutions: effectiveness ........................................................... 22
2.3 Reflections on the theoretical ideas ....................................................................................... 25
2.3.1 The configurational approach ........................................................................................ 25
2.3.2 Occurrence of misfits....................................................................................................... 27
2.4 Concluding remarks ................................................................................................................ 29
Chapter three – Methodology ................................................................................................... 31
3.1 The methodological approach and the choice for a survey .................................................. 31
3.1.1 The methodological framework ...................................................................................... 31
3.1.2 Survey research: method and critique............................................................................ 33
3.2 Design of the questionnaire .................................................................................................... 35
3.2.1 The questionnaire’s set-up – matching the theory......................................................... 35
3.2.2 Focus on activities - matching the level of analysis ...................................................... 37
3.3 Validity-feedback interviews ................................................................................................. 39
3.4 The pre-test of the questionnaire............................................................................................ 41
3.4.1 Purposes of pre-testing.................................................................................................... 42
3.4.2 The Three-Step Test-Interview........................................................................................ 42
3.4.3 Pre-test execution and results......................................................................................... 44
3.5 Sampling and survey administration...................................................................................... 45
3.6 Description of the dataset ....................................................................................................... 47
3.7 Concluding remarks ................................................................................................................ 50
Chapter four - Operationalizing the framework.................................................................... 53
4.1 Mapping management control systems ................................................................................. 54
4.2 MCS Effectiveness.................................................................................................................. 61
4.3 Characteristics of the activities .............................................................................................. 63
4.3.1 Uncertainty....................................................................................................................... 64
4.3.2 Asset specificity................................................................................................................ 69
4.3.3 Ex-post information asymmetry ...................................................................................... 73
4.4 Measuring misfit ..................................................................................................................... 73
4.4.1 Defining the archetypes................................................................................................... 74
4.4.2 Calculating misfit............................................................................................................. 77
4.4.3 The misfit variables ......................................................................................................... 80
4.5 Control variables ..................................................................................................................... 81
4.6 Closing remarks ...................................................................................................................... 82
Chapter five - Analyses and results .......................................................................................... 83
5.1 The relevant archetype............................................................................................................ 84
5.2 A comparison of the subsamples ........................................................................................... 89
5.3 The occurrence of fit and misfit ............................................................................................. 93
5.4 Testing the MCtce..................................................................................................................... 96
5.4.1 The general model ........................................................................................................... 97
5.4.2 Arm’s length control........................................................................................................ 99
5.4.3 Result Oriented Machine Control.................................................................................104
5.4.4 Action Oriented Machine Control ................................................................................107
5.4.5 Exploratory control ....................................................................................................... 111
5.4.6 Boundary control ...........................................................................................................113
5.5 Misfit and the contingency variables ...................................................................................114
5.6 Effectiveness of control and organizational effectiveness .................................................118
5.7 Summary of findings ............................................................................................................120
Chapter six - Conclusion and discussion ...............................................................................123
6.1 The effectiveness of control systems ...................................................................................123
6.1.1 Summary of findings ...................................................................................................... 124
6.1.2 Limitations......................................................................................................................129
6.2 Results related to methodology............................................................................................132
6.3 Future research - another point of departure ....................................................................... 133
6.3.1 Configurational fit and complementarities .................................................................. 134
6.3.2 Uncertainty and the perception of control ................................................................... 135
6.3.3 Effectiveness of control and effectiveness of the organization ................................... 137
6.3.4 MCS design and effectiveness .......................................................................................138
References................................................................................................................................... 139
Appendix A - Description of a management control system ...................................................I
Appendix B - Overview of survey questions ......................................................................... VII
Samenvatting........................................................................................................................... XVII
Curriculum Vitae .....................................................................................................................XIX
Preface
People who dedicate their lives to scientific research are curious. However, it was more than
sheer curiosity that led me to decide on becoming a PhD student. After finishing my studies I
felt there was still much more to learn and discover; what better place to do that than at
university. I have very much enjoyed my time as a PhD student. I have learned many things,
visited great places, and met wonderful people. At this point I would like to thank everyone
who has contributed to this research project, each in his or her own way.
First of all, many thanks to the students of group 1 MAC3.2/3.3 (2006 and 2007) for
collecting the data. I very much appreciate your efforts. Also those people who participated in
the pre-test of the questionnaire deserve a warm ‘thank you’. Tony Hak, thank you for
discussing the TSTI method with me. I guess we share a passion for methodological issues.
Furthermore, I would like to thank my colleagues and fellow PhDs from the Netherlands as
well as abroad, for showing me that life in academia is both challenging and fun. Both at
Erasmus University Rotterdam and at Nyenrode Business University, I was very lucky to be
part of a great team and I thank my colleagues for making me feel at home at university. To
Hilco, my officemate, I would like to say thanks for making the office a good place to be.
Some people deserve more than a general acknowledgement. I thank Marcel Bonnet for his
valuable advice on many issues and for anticipating a perfect fit between PhD student,
project, and supervisor. I thank George Hendrikse, my thesis supervisor, for always revealing
different lines of thought and showing me alternative perspectives. Special thanks to Roland
Speklé, my thesis supervisor, for teaching me so many things about academia and about doing
research. Thank you for giving me room to explore and grow, while being there for me
providing support and giving advice.
I also thank all my friends for believing in me, being there, and distracting me with lots of fun
stuff. My parents, my brother, and my sister-in-law I would like to thank for their support,
trust, and enthusiasm and for listening to all my stories.
Finally, to those of you who wonder what will be next: I intend to keep on doing those things
I enjoy and that I am reasonably good at. I am a researcher, I am a teacher, and I am still
curious. Therefore, I will stay in academia.
Anne-Marie Kruis, Feb. 2008
Summary1
This doctoral thesis describes the set-up and findings of an empirical study of management
control system design and effectiveness. Management control systems are packages of control
instruments that are used to coordinate the work and to provide incentives through rewarding
and punishment. Examples of control instruments are rules and regulations, budgets,
performance targets, and bonuses. Organizations choose from a variety of instruments to set
up their control systems. However, not every possible combination of instruments will be
effective. The workings of one control instrument depend on its combination with the others.
They are interrelated.
Numerous studies focus on individual control instruments, but there is little empirical work
that researches control systems in their entirety. Therefore a lot remains to be learned about
manifestations of these systems as packages of interrelated control instruments. Even less is
known about the effectiveness of control. With this study I contribute to the existing
knowledge base by specifically concentrating on management control systems in their entirety
and assessing their relative effectiveness. My central problem is to find out which control
systems are effective given the circumstances.
The Transaction Cost Theory of Management Control (Speklé, 2001a, 2001b, 2004) provides
insights into these matters and is taken as the theoretical basis of this study. The theory
submits that five effective forms of control systems (so-called archetypes) exist: arm’s length
control, result oriented machine control, action oriented machine control, exploratory control,
and boundary control. Each archetype is effective for the control of specific activities, but not
for others. Examples of activities are the production of a good or service, human resources, or
research and development. These activities are characterised by their level of uncertainty,
asset specificity, and ex-post information asymmetry. The theory specifies, for instance, that
the archetype arm’s length control is the most effective control system for activities with low
levels of uncertainty and low levels of asset specificity. The main claim of the theory is that if
the management control system of an organizational unit closely resembles its relevant
archetype, it will be more effective in controlling its activities than other management control
systems would. The relevance of the archetype depends on the activity of the organizational
unit. Using survey data on 258 organizational units, I test these claims empirically.
1
Een Nederlandse samenvatting vindt u achterin dit proefschrift.
To enable hypotheses testing, I first need to determine the relevant archetype for each case.
Therefore each observation is allocated to one of five subsamples (one for each archetype)
according to the characteristics of its activities. The results from a regression analysis show
that resemblance of a management control system with the archetype arm’s length control has
a positive impact on the perceived effectiveness of the control system. As predicted by the
theory, this effect is confined to the group of cases that experience low levels of both
uncertainty and asset specificity. For those cases that have result oriented machine control as
their relevant archetype, regression analysis shows a null-effect: resemblance of a
management control system with the archetype does not affect the effectiveness of the control
system. For action oriented machine control, exploratory control, and boundary control the
respective subsamples are too small to allow regression analysis. The bivariate analyses used
instead provide tentative results that do not support the hypotheses.
Other results relate to the research topic more broadly. First of all, I find a positive correlation
between the effectiveness of the management control system and the effectiveness of the
organizational unit. The strength of this relationship differs across the subsamples that
represent the archetypes. Secondly, evidence suggests that uncertainty has a direct negative
impact on the perceived effectiveness of the control system. Because uncertainty also
influences the design of these systems, its overall role is complex.
A final contribution of my study is a methodological one. I explicate all necessary steps to
operationalize the theory and to map management control systems in their entirety. Therefore,
this thesis exemplifies one approach that enables the study of management control system
design and effectiveness as such. I also introduce ‘validity-feedback’ interviews as a powerful
method to pre-test the quality of survey instruments. It is generally applicable and provides
feedback on the quality of questionnaires beyond that retrieved by other pre-test methods.
Finally, to enhance the measurement of uncertainty I set up a general measurement model
with formative indicators (instead of reflective ones). This improves the construct validity.
The approach can also benefit the future measurement of asset specificity.
Chapter one – Introduction
1.1 Point of departure
In this doctoral thesis I build directly on the work by Speklé in which he applies Transaction
Cost Economics (TCE; Williamson, 1975, 1985, 1991) to study management control
problems. He has created the Transaction Cost Theory of Management Control (Speklé,
2001a, 2001b, 2004) and claims that control structures are coherent combinations of control
instruments that must be studied simultaneously. Not only does he show what these systems
might look like, by articulating a comprehensive framework with several distinct control
archetypes, but -using TCE logic- he also explicates the circumstances under which each
archetype exhibits the largest relative effectiveness. His theory can be characterised as a
contingent configuration theory (Speklé, 2004).
Speklé is not the only one who takes on a configurational approach to study control, nor is he
the first to apply TCE in this light2. Hofstede (1981), for instance, implicitly adopts the idea of
control packages and presents a typology of six different control types: routine control, expert
control, trial and error control, intuitive control, judgemental control, and political control. He
relates the applicability of each to characteristics of organizational activities. Ouchi (1979,
1980) relies partly on TCE to study three mechanisms of control within organizations:
markets, hierarchies, and clans. He explains that the mechanisms themselves overlap in
organizations and he identifies several contingency variables that determine which one should
be emphasized. Snell (1992) builds from these works and studies use of several human
resource management practices by executives. He differentiates between behaviour control
systems, input control systems, and output control systems. He explicitly describes these
types as combinations of practices that, as a set, establish control.
The common thread running through these studies is the identification of several generic
control forms. The authors choose similar contingency variables that determine the
applicability of these control forms: (1) knowledge about desirable actions/transformation
processes, (2) measurability of outputs/results, and (3) ambiguity of goals. These contingency
variables are also represented in Speklé’s theory, be it under different names. The generic
2
The aim here is merely to illustrate some main themes of interest by referring to several studies. My
approach is eclectic and focuses on management control literature. For extensive reviews of research
within the MC literature see, for instance, Otley, Broadbent, and Berry, 1995; Fisher, 1995; Luft and
Shields, 2003; Chenhall, 2003.
1
Chapter one
control forms represent combinations of multiple control instruments. What remains unclear
from the studies, however, is how to balance the different control forms/instruments in a
given situation in order to establish an effective control system. It is unknown, for instance,
which combination of hierarchical and clan controls (Ouchi, 1980) is effective for the control
of a certain organization. Speklé elucidates these matters by specifying five archetypes that
represent these balances. He provides rich descriptions of the characteristics of each
archetype. This enables the comparison of observed management control systems with the
archetypes. Moreover, he makes explicit claims about their relative effectiveness. Thus,
building from and combining prevalent knowledge, he adds to our understanding of control
the explicit propositions on effective combinations of control instruments.
With this study, I test the theoretical claims of the Transaction Cost Theory of Management
Control (henceforth: MCtce). This will provide insights into the empirical power of the theory
and involves theory refinement. Since its development in 2001, parts of the MCtce have been
used in several studies (see for instance, Nicholson, Jones, and Espenlaub, 2006; Widener,
2004; Langfield-Smith and Smith, 2003; Van den Bogaard and Speklé, 2003). Some helpfully
apply the theory to explain their findings of case studies (Nicholson et al., 2006; Van den
Bogaard and Speklé, 2003). Those authors find their evidence consistent with patterns and
control forms as predicted by the theory. Others want to extend the framework either by
adding additional contingency variables related to the strategy of the organization (Widener,
2004), or by taking in trust to represent the social context of control (Langfield-Smith and
Smith, 2003). As of yet, however, little is known about the empirical power of the MCtce. This
doctoral thesis encompasses the first large-scale quantitative study of the theory. Therefore
challenges lie ahead in operationalizing the ideas.
The goal of my research project is to study control system design and effectiveness. The
central problem is to find out which management control systems are effective. Specifically
concentrating on management control systems in their entirety, I study about 20 different
aspects of control simultaneously. Moreover, I assess the effectiveness of these systems using
a newly developed measurement instrument. Studying management control systems, as
opposed to studying single control instruments, receives ever more attention. The
conceptualization of management control systems as packages of interrelated elements (see
for instance, Otley, 1980; Flamholtz, 1983; Abernethy and Chua, 1996) is generally accepted.
Moreover, authors stress the importance of assessing these elements of control jointly (Otley,
1980; Holmstrom and Milgrom, 1994) and warn that studying specific instruments in
isolation likely leaves models underspecified (Chenhall, 2003). Despite these regular calls for
a more holistic approach in studying control systems, little empirical work emerged that
2
Introduction
focuses specifically on multiple control instruments and their interrelationships (Otley,
Broadbent, and Berry, 1995; Chenhall, 2003). In his review of contingency-based research on
management control systems, Fisher (1995) concludes that most of the empirical work
focuses on a single contingency variable and a single control variable.
Two recent studies form notable exceptions: both Widener (in press) and Bedford (2005)
specifically take on a more holistic approach towards control systems. Both studies will be
introduced shortly; a detailed discussion follows in later chapters. Using survey data, Widener
(in press) studies Simons’ levers of control framework (Simons, 1995) at the company level.
She takes in both antecedents and outcomes of control in her models to explain organizational
effectiveness. She also explores complementarities among the beliefs, boundary, diagnostic,
and interactive systems of control. Bedford (2005) uses cluster analysis to research which
combinations of cybernetic controls, administrative controls, and socio-ideological controls
are present at the level of the strategic business unit. In this study, I also focus on control
systems in their entirety and add to the empirical evidence on control system design. Building
from the MCtce, my study starts with explicit theoretical expectations on what combinations of
different control elements will be effective. Therefore, I can study the balance of control
elements more rigorously than the before-mentioned authors could. Moreover, I focus on
effectiveness of control directly as the dependent variable.
A lot remains to be learned about manifestations of management control systems, but even
less is known about the effectiveness of control. First of all, researchers typically relate
aspects of the management control system to one or more contingency variables, assuming
effectiveness rather than testing for it (Chenhall, 2003). Secondly, researchers who aim to test
control effectiveness as a separate variable take the effectiveness of the organization as a
proxy and do not actually measure the effectiveness of the management control system. In so
doing, however, they assume linkages between the workings of the management control
system and the organizational outcomes, for which there is no compelling evidence (Chenhall,
2003). The only exception is Ferreira and Otley (2005) who develop a measure of the
perceived effectiveness of a control system at the company level. Inspired by their work, I set
up a new measure of control effectiveness that covers more aspects of control and that is more
generally applicable. I also measure effectiveness of organizational units and can thus shed
some light on the linkages between the performance of the management control system and
unit effectiveness.
This study contributes to research within the field in several ways. First of all, we gain
knowledge about the empirical power of the Transaction Cost Theory of Management Control
3
Chapter one
and engage in theory refinement. Secondly, this large-scale empirical study on management
control systems and their effectiveness contributes to our understanding of the workings of
control. Moreover, I develop the measurement instruments needed to actually conduct this
type of study, working with generally applicable measures and mapping control systems in
their entirety. Therefore my study also contributes by exemplifying one approach to conduct
this type of study. Finally, along the way, I address a number of methodological issues of
broader importance. These issues relate, for instance, to use of the survey method and to scale
construction. The following paragraphs describe the main themes of this research project and
the nature of my work. This chapter ends with an outline of the thesis.
1.2 The main themes
This study is about design and effectiveness of management control systems (MCSs). My
main concern is to learn about manifestations of control systems, their resemblance (or lack
thereof) with the theoretical archetypes as specified by Speklé, and their contingent
effectiveness. The general research problem is to find out which control systems are effective
given the circumstances. An introduction to management control systems as such, the theory,
and effectiveness of control follows.
All managers experience the workings of management control systems and face choices
inherent in the design of these systems. A variety of control instruments is at their disposal,
for instance financial rewards, budgets, and rules or behavioural guidelines. These
instruments should further the accomplishment of organizational goals through their influence
on employee behaviour within the organization. We refer to their combination as a
management control system3. The systems are used to motivate employees, to communicate
organizational goals, and to stimulate or prevent certain behaviour. Because of their influence
on employee behaviour, MCSs affect organizational performance. An MCS that does not
function well can negatively impact the organization. At best, a dysfunctional control system
keeps the organization from obtaining results. The worst case scenario, however,
encompasses theft or even bankruptcy when managers exploit resources for their own sake.
The application of control instruments serves several purposes. Speklé (2004) indicates as the
main purposes (1) knowledge capturing, (2) coordination, and (3) providing incentives
through rewarding and punishment (see also Jensen and Meckling, 1992, and Zimmerman,
3
The word ‘system’ here is just a synonym for ‘structure’. This use of terms is consistent with research
within the field of management accounting and control.
4
Introduction
2000). The first purpose concerns making knowledge that rests in several parts of the
organization available for decision making. The control instruments used in this light are
related to the allocation of decision rights. Coordination can be established, for instance, by
communicating goals through performance targets. Providing incentives concerns both
furtherance of positive behaviour and prevention of behaviour that could harm the company.
This can be done, for instance, by using financial rewards and setting boundaries. The control
purposes directly relate to the raison d’être of the organization: they further the
accomplishment of organizational goals.
Management control systems are packages of control instruments. The effects of single
control instruments depend on how these are being used as well as on their combination with
other instruments. The packages are thought to inhibit synergies (Milgrom and Roberts, 1990,
1995). More importantly, choices already made for certain control instruments within a
management control system restrict the available alternatives for choosing additional ones
(Speklé, 2004, p11). The control instruments cannot be used in all possible combinations.
Therefore, researchers within the field adhere more and more to the study of packages of
control instruments or control systems, rather than concentrating on single control
instruments. But, what do we know about effective design of control systems? Are some of
these systems effective, whereas others are not? Or does it depend on the specifics of the
situation which systems best mitigate control problems?
A management control system is expected to be effective when it can manage the specific
control problems an organization or organizational unit faces. This is the central tenet of the
Transaction Cost Theory of Management Control. The theory submits that specific groups of
control problems can be predicted to prevail by examining the characteristics of the activities
of an organization. As a consequence, the relevant level of analysis is the activity and I study
the management control systems that surround these. Looking at organizational units, for
instance a business unit or a production department, activities can be a multiplicity of things
ranging from the production of a good or service, to research and development activities or
human resources. Therefore, to study activities identification of organizational units as such
will not suffice.
The theory identifies three dimensions over which activities differ: the associated level of
uncertainty, asset specificity, and ex-post information asymmetry. Once an activity is
characterized over these dimensions control problems associated with the activity can be
foreseen. It makes sense to match expected control problems with possible control solutions,
which management control systems provide. Not every MCS will be an effective match
5
Chapter one
though. The MCtce shows that five theoretical constructs of MCSs (archetypes) differ in their
problem solving capabilities and, consequently, that each can handle best a different group of
control problems. Thus, the archetypes represent effective control structures, each effective
for the control of certain activities, but not for others. This implies that if an MCS of an
organizational unit closely resembles its relevant archetype, it will be more effective in
controlling its activities than other forms of MCSs would. Using survey data I test these
theoretical claims empirically by mapping the MCSs of a large number of organizational units
and assessing their contingent effectiveness.
1.3 The nature of this study
To learn about the effectiveness of management control systems I draw hypotheses from the
Transaction Cost Theory of Management Control. As already mentioned, this is the first
large-scale quantitative study of this theory, and overall there is little empirical work on
management control systems in their entirety and the effectiveness thereof. Thus, although
specific hypotheses drawn from theory will be tested in the chapters to come, for two reasons
this study is (at least partly) exploratory in nature.
First of all, decisions have to be made regarding the operationalization of the theory. The
archetypes, for instance, are described qualitatively and empirical profiles have to be set up.
All decisions regarding operationalization still have to be made as part of this research
project. Secondly, the theory introduces five archetypes, each effective for the control of
specific activities. Therefore, to enable hypotheses testing for all archetypes a variety of
activities must be studied. I will compare, for instance, the MCS of a production unit with that
of a research and development unit, or the MCS of a sales unit with that of a recruitment unit.
I extend prior research by focussing explicitly on control systems in their entirety and deviate
from it by studying a range of different activities. A lack of examples from prior research
necessitates the development of new measures. This has consequences for the methodological
approach underlying my work. In the remaining part of this paragraph I will introduce and
illustrate some of the main issues.
Regarding data collection, three things are important: first of all, it is necessary to map
management control systems (1). Detail is important to pick up a control system in its entirety
without loosing track of the nuances of control. At the same time though, only a large number
of observations can enable hypotheses testing (2). Moreover, to test hypotheses for all five
archetypes I need data on various activities (3). Such variety in activities can be found
throughout different industries and companies, and within companies at different hierarchical
6
Introduction
levels. This type of data collection necessitates the use of a survey, which can reach all kinds
of organizational units each with its own activities, and meet the requirements of providing a
large quantitative dataset.
To compare such a diverse set of activities I abstain from studying the specifics of the control
system and focus on generic forms of control. For instance, I do not document the specific
rules or procedures used within an organizational unit, like a rule on handling toxic fluids or
rules on private use of the internet. Rather I will ask about whether rules or regulations apply
as such and about the consequences of non-compliance. Taking on this abstract view towards
control enables the study of generic control structures and hence the comparison of control
systems from distinct organizational units. Of course, as already shown above, studying
generic control instruments as such is not new. However, the possibilities for building from
prior research are limited, because there are hardly any studies that focus on MCSs in their
entirety.
Similar things go for studying the characteristics of the activities of organizational units.
These must also be measured in a way that enables comparison among a large variety of
them. Activities are characterised by their level of uncertainty, asset specificity, and ex-post
information asymmetry. These variables are difficult to measure, especially in a way that
enables comparison across industries (Shelanski and Klein, 1995). Consequently, researchers
applying TCE typically study a selected industry or one specific type of organizational unit,
such as sales departments (e.g. Anderson, 1985) or internal audit departments (e.g. Speklé,
Van Elten, and Kruis, 2007). Again, the possibilities for building from prior research are
limited.
Opting for a survey necessitates the development of new measurement instruments able to
pick up MCSs and to assess their quality in a generally applicable way that enables comparing
a variety of organizational units. Verification of the content validity will be of utmost
importance. Several questions have to be answered to assess this validity: can we actually
grasp general strands of MCSs as wished-for using solely the information obtained by the
questionnaire? And, is this picture more or less complete, i.e. will it suffice for our purposes
of comparing configurations of control? As a means to address these problems, pre-testing the
quality of the questionnaire receives ample attention. I introduce a new technique, referred to
as ‘validity-feedback interviews’, as a powerful method to obtain valuable feedback on the
content validity of survey instruments. Measurement issues are also dealt with explicitly
throughout this work, especially regarding formative versus reflective indicator models.
Researchers from marketing and organization recently expressed their concerns about the lack
7
Chapter one
of consistent scale construction and the serious consequences thereof (see for instance
MacKenzie, Podsakoff, and Jarvis, 2005; Diamantopoulos and Siguaw, 2006). Bisbe, BatistaFoguet, and Chenhall (in press) conclude that in empirical research into management
accounting and control systems the issue is rarely addressed and that it is common practice to
rely uncritically on reflective models. All in all, two chapters deal with methodological and
statistical issues.
1.4 Outline of this thesis
The problem of management control system design and effectiveness and the empirical test of
the Transaction Cost Theory of Management Control form the core of this thesis. Chapter two
starts by elucidating the theoretical ideas. It introduces the Transaction Cost approach to
studying control problems and discusses configurational theories. Management control
systems are described and their substance is defined. Moreover, in this chapter I set out the
research question and hypotheses, and address the underlying assumptions.
Chapter three explains the methodology and introduces a methodological framework. The
framework highlights a step by step approach to test the hypotheses. Each step involves
choices regarding methods. This chapter discusses the general methodology. The
development of the questionnaire is the central topic and testing the instrument’s quality
receives considerable attention. Chapter three also covers the data collection process and
holds a description of the dataset. Next, chapter four provides details on the operationalization
of the methodological framework. Separate paragraphs focus on the measurement of every
variable and provide descriptive statistics.
Chapter five contains the analyses and results of this study. Firstly, some final preparatory
steps are taken. Then, I present the results of hypotheses testing, as well as results on
additional analyses regarding the resemblance (or ‘fit’) of observed control systems and the
archetypes, and the relationship between fit and effectiveness. Moreover, the role of
uncertainty and relationships between the contingency variables and MCS design are objects
of study. Chapter six presents the overall conclusions and discusses the results, limitations,
and future research avenues.
Two appendices are added to this thesis. Appendix A holds a description of the management
control system of an organizational unit to illustrate the type of information as well as the
amount of detail the questionnaire generates. Such descriptions were used during interviews
as part of the pre-test of the questionnaire to assess the validity of the instrument.
8
Introduction
Appendix B gives an overview of all questions posed. Along with details on each question,
like answer categories and sources, I provide notes and guidance on use of the questionnaire.
Reading this appendix should enable other researchers to work with (parts of) the instrument
themselves should they choose to do so.
9
Chapter two – Configurations of control
In this thesis, I adhere to the idea of control configurations and study management control
systems in their entirety rather than single control instruments. Explaining contingent
effectiveness of control systems is my central concern. This chapter discusses the theoretical
basis. First of all, paragraph 2.1 provides a working definition of a management control
system. Claiming to study control in its entirety requires a broad view of the MCS as a
package of interrelated control instruments. The Transaction Cost Theory of Management
Control plays a central role in this research project and will be introduced next in paragraph
2.2. I will discuss the general arguments of the theory as well as the claims it makes regarding
MCS effectiveness. Moreover, hypotheses are drawn from this theory. Paragraph 2.3 contains
reflections on the theoretical ideas and discusses the configurational approach. It deals with
the notion of fit, the occurrence of misfits, and the assumptions about effectiveness. Finally,
paragraph 2.4 ends this chapter with concluding remarks and some thoughts on the testing of
the theoretical ideas.
2.1 Management Control Systems
In the introduction, a management control system was simply referred to as a package of
control instruments. Examples of control instruments are the budget, work instructions,
performance targets, and bonuses. However, there is ambiguity about what precisely
constitutes a control system (Fisher, 1995). In this paragraph, therefore, I introduce the
working definition of an MCS that underlies this study. This working definition stems from
Speklé (2004) and integrates ideas from the accounting literature and findings from prior
research. Firstly, starting from the purposes of the management control system, four
dimensions of control are described (2.1.1). Then focus shifts to the interrelationships
between them (2.1.2).
2.1.1 Purposes and dimensions of control
Within organizations the management control system serves several purposes. Jensen and
Meckling (1992) state that the control system provides incentives through performance
measurement and evaluation, and through a system for rewarding and punishment. The
authors further submit that together with a system that partitions decision rights, the
organization’s control system compensates for the lacking ‘invisible hand’ (Smith, 1776) that
coordinates activities in the market place. Speklé (2004) adheres to a similarly broad view of
11
Chapter two
management control systems. He summarizes the main purposes of the MCS as knowledge
capturing, coordination, and providing incentives through rewarding and punishment.
To achieve these control purposes combinations of control instruments are used. For instance,
making knowledge that rests within the organization available for decision making, can be
achieved by delegation of tasks and responsibilities to those who possess the knowledge.
Also, use of financial rewards provides incentives and could further certain behaviour. Speklé
(2004) refers to four dimensions of control to describe comprehensively the control structure:
(1) the allocation of decision rights, (2) use of standards, rules, and regulations, (3)
performance evaluation, and (4) rewards and incentives. This is consistent with the prior
literature (for instance Zimmerman, 2000; Jensen and Meckling, 1992; Flamholz, 1983), but
underlines the role of standards and rules by making that a separate dimension. A
specification of each control dimension follows illustrating the control instruments that
belong to the dimension. Throughout, I will discuss control from the perspective of a
supervisor who exerts control on a manager.
The first dimension ‘allocation of decision rights’ incorporates the delegation of tasks and
responsibilities by the supervisor to the manager. Choices have to be made regarding how
many and which responsibilities to involve. Key control instruments are the level of
decentralization and the amount of work autonomy a manager has.
Use of standards, rules, and regulations make up a broad and diverse second dimension. For
instance, a supervisor can rely on standard operating procedures and work instructions to
coordinate the work. This is one type of standardization, commonly referred to as action
control (see Merchant, 1982). Other types of rules and standards that focus on employee
behaviour, specify what not to do and act as boundary systems4. Rules and regulations thus
play different roles within the organization. Another means of coordinating the work is
provided by use of performance targets (compare with result control; Merchant, 1982).
Different types of these targets belong to this control dimension: financial targets, nonfinancial targets, emergent targets, and (market) benchmarks. A final control instrument that
fits this dimension is the budget. Like rules and regulations, budgets can play different roles
within organizations. They function, for instance, as a performance target, or serve primarily
as a coordination device.
4
The term boundary systems stems from the work by Simons (1995). He explains that boundary
systems “delineate the acceptable domain of activity for organizational participants” p39.
12
Configurations of control
Regarding performance evaluation, the third dimension, a supervisor’s central concern is the
choice of a basis for evaluation. Evaluation can strictly be based on compliance with rules and
standards, or hinge on target achievement. For purposes of performance evaluation, a
supervisor can also use subjective judgement, or choose to benchmark achievements to long
term organizational performance. Naturally, different evaluation bases are present.
The final dimension of control represents the reward system and incentives. Both the reward
itself and the conditions for being granted one are important. Again, a whole range of
possibilities exists: organizations work with a fixed reward, a variable reward, or both.
Moreover, circumstances under which a reward is being granted can differ widely. Apart from
focusing on the short term benefits related to a bonus, career perspectives play an important
role when it comes to rewards and incentives in the long run.
The four dimensions of control form the basis of a working definition to study MCSs.
Together they represent a broad view of control that is a prerequisite for the study of control
in its entirety. Considering these four dimensions each with numerous control instruments, the
potential number of distinct MCSs seems endless. However, because of interrelationships
between the control elements, only few combinations are expected to be effective. This will
be discussed next.
2.1.2 Complementarities
Above a management control system is presented as a package of multiple control
instruments that together cover four dimensions. However, the effects of control instruments
depend on how these are being used as well as on their combination with the other
instruments. This idea is consistent with the conceptualization of management control systems
as packages of interrelated elements (Otley, 1980; Abernethy and Chua, 1996) that as a set
establish control.
The following example illustrates how the dimensions connect5. Imagine a situation in which
a supervisor delegates tasks and responsibilities to a manager (the first dimension). The
manager can only act upon this when he/she is not at the same time required to comply with
specific work instructions or regulations (the second dimension). Rather, to guide the
manager’s behaviour performance targets may be agreed upon. If the manager’s performance
5
This example only illustrates how the dimensions could interrelate and complement one another. How
this works exactly and why, will be explained when discussing the theory in the next paragraphs.
Among other things, the explanation hinges on the assumptions made about human behaviour.
13
Chapter two
evaluation is based on target achievement, the manager will likely take the targets into
account when making decisions (the third dimension). However, if the achievements are not
linked to rewards (the fourth dimension), the manager will not face the consequences of
his/her actions and might make decisions that do not fit the organization’s goals.
The example shows that one dimension of control is not enough to establish effective control
within an organization and that control instruments must be balanced somehow. Not every
potential combination of control instruments will be effective. As Zimmerman (2000) points
out, the systems that partition decision rights, evaluate performance, and reward and punish,
compose a three-legged stool. The dimensions must be balanced. The reason is that
complementarities among the elements of control exist. Holmstrom and Milgrom (1994)
stress the importance of complementarities among elements of an organization’s structure. In
their analytical study, they show that elements of the incentive system complement each other
and influence each other’s performance.
Hence interrelationships are believed to exist between the different elements of control
systems. However, due to a lack of studies that address the workings of packages of control
instruments (Otley et al., 1995), there is little empirical evidence. Two recent studies
(Bedford, 2005, and Widener, in press) provide us with some insights. Both focus explicitly
on management control systems, instead of on separate elements of control. Bedford studies
management control configurations of strategic business units. He incorporates cybernetic
controls, administrative controls, and socio-ideological controls in his study. Using cluster
analysis he can distinguish and interpret meaningfully four distinct groups of control systems.
He cannot relate the groups systematically to the strategy, environment, or size of the strategic
business units. However, the evidence suggests that configurations of control exist, i.e. not
every possible combination of control instruments is present in his sample.
In her study of Simons’ levers of control framework, Widener (in press) finds evidence that
suggests the levers are complementary. Using survey data obtained from chief financial
officers, she finds a positive association between the beliefs system and each of the other
levers of control (the boundary, the interactive, and the diagnostic systems; Simons, 1995).
More emphasis on the beliefs system implies more emphasis on the other levers as well.
Moreover, she finds that the joint effect of the four levers on firm performance is stronger
than the sum of their separate effects. This suggests that the systems are complementary and
should be studied simultaneously to understand performance effects. However, whereas
Widener’s study suggests that the joint effect on performance might be stronger, Abernethy
and Chua (1996) stress in their longitudinal case study on control system design, the risk of
14
Configurations of control
misinterpreting performance effects of single controls when several instruments act as
substitutes.
The studies mentioned above together underline the importance of studying control elements
in combination. If we expect from the large number of control instruments that a variety of
MCSs might be used by organizations, but that because of complementarities within control
systems, only few will be effective, the key question is which combinations are effective. The
theoretical answer to this question is provided by the Transaction Cost Theory of
Management Control (Speklé, 2001a, 2001b, 2004). The theory submits that only a few
combinations of control instruments are effective. The actual number of distinct MCSs will be
limited, because choices already made for certain control instruments within one of the
dimensions restrict the available alternatives for choosing additional ones from the other
dimensions (Speklé, 2004, p11). From the theory it becomes clear how the elements of
control must be balanced to establish effective control. The next paragraph introduces the
theory.
2.2 The transaction cost approach to management control
The Transaction Cost Theory of Management Control (Speklé, 2001a, 2001b, 2004) provides
insights into which control systems are effective and under what circumstances. In his theory
Speklé applies Transaction Cost Economics logic (TCE; Williamson, 1975, 1985, 1991) to
management control problems. The general approach is the same as in regular TCE and we
come across familiar concepts like uncertainty and asset specificity. The two perspectives
share a concern for the relative effectiveness of alternative governance structures. As
Williamson (1985) states, effectiveness is established by assigning transactions (which differ
in their attributes) to governance structures (which differ in their adaptive capacities and the
associated costs) in a discriminating way. In terms of management control this translates into
the discriminating alignment of activities (i.e. transactions) and management control systems
(i.e. governance structures).
An introduction to the Transaction Cost Theory of Management Control (henceforth: MCtce)
follows, starting with the general argument. I explain the contents of the theory in three steps.
Firstly, I focus on control problems as they exist in activities executed within organizational
units. Secondly, I discuss management control systems as devices that provide control
solutions. Then, the last paragraph brings it all together matching problems with solutions. I
will confine myself to those parts of the theory relevant to the study of MCSs at the level of
the organizational unit. The MCtce, for instance, also deals with hybrids (i.e. outsourcing of
15
Chapter two
activities) which fall outside the scope of this study (for a detailed overview of the complete
theory see Speklé 2001a).
2.2.1 The general argument
The MCtce revolves around the idea of matching activities and their associated control
problems with management control systems that provide control solutions in an economizing
way. The theory submits that control problems are inherent in the carrying out of the activities
of an organization or organizational unit (Speklé, 2004). Specific activities will induce
specific control problems and by studying activities it can therefore be predicted which
control problems will occur. Moreover, the theory introduces five control archetypes as
effective forms of control. Each archetype is relatively the most effective in controlling
certain groups of activities. Establishing effective control comes down to matching control
problems and control solutions.
Thus, control problems are related to the activities of an organizational unit. For the purposes
of this study, I define an organizational unit as consisting of a manager and all employees
reporting (in)directly to him/her. Organizational units exist at different hierarchical levels
throughout the organization. Activities of these units can be a multiplicity of things ranging
from the production of a good or service, to marketing or Human Resources, to finance or
ICT.
Control problems originate from the interaction of the specifics of an activity and human
traits. Two characteristics of human beings come into play: bounded rationality and
opportunism. Bounded rationality refers to people’s inability to take in and process all
available information (Simon, 1957). It prevents people from taking the most efficient actions
possible and from making optimal decisions. Opportunism is defined as “self interest seeking
with guile” (Williamson, 1975, p255). People acting opportunistically can cause loss of
resources or damage the organization’s reputation. The role of these characteristics and the
severity of their consequences depend on the situation, i.e. the activity of the unit. Different
activities incorporate different information needs and bring about diverse opportunities for
opportunistic behaviour. Hence different activities cause various control problems. On the one
hand, these control problems involve a lack of knowledge dissemination throughout the
organization, so that information cannot be made available for decision making or
performance evaluation. On the other hand, they are associated with waste of resources when
employees act in their own interest instead of applying themselves to organizational goal
attainment. Taking into account the two human traits, the MCtce predicts which control
16
Configurations of control
problems will arise given the specifics of the unit’s activity. In sum, under the assumptions of
bounded rationality and opportunism, variation in activities gives rise to variation in control
problems.
Control solutions are provided by management control systems. MCSs mitigate control
problems, for instance, by providing means of knowledge capturing, and by curbing
opportunistic behaviour through incentives. However, specific control systems provide
solutions only to certain control problems, but not to all. Their relative capabilities to confront
control problems differ. Key to effective control, then, is to create as close a match as possible
between activities and management control systems.
The theory introduces five ideal types or archetypes as effective forms of control and relates
those to specific activities. By characterizing the activity of an organizational unit, the
relevant archetype for that unit can be determined. Because the archetypes represent effective
forms of control, MCSs that resemble their relevant archetype will be more effective than
those that do not. Figure 2.1 gives an overview of the theoretical ideas.
Figure 2.1 Matching control problems with control solutions
bounded rationality
opportunism
activity of
organizational unit
(UNC, AS, IA)
control
problems
control
solutions
MCS of
organizational
unit
archetypes represent an
effective match
Note: UNC = uncertainty, AS = asset specificity, and IA = ex-post information asymmetry
Source: Speklé 2001b, modified
The MCtce can be characterised as a contingent configuration theory. Doty and Glick (1994,
p232) define configurational theories as follows: “… (configurational theories) identify
multiple ideal types, each of which represents a unique combination of the organizational
attributes that are believed to determine the relevant outcome(s).” The MCtce introduces five
archetypes, and explicates the circumstances under which each is effective thus taking on a
contingent configurational approach. In the next paragraphs I will elaborate on both control
problems associated with activities and their solutions as provided by the archetypes.
Moreover, I will explain the main argument regarding MCS effectiveness.
17
Chapter two
2.2.2 Control problems: activities and contingency variables
From the general argument it follows that control problems are inherent in the activities of an
organizational unit and occur because human beings characterised by bounded rationality and
opportunism carry out the activities. The MCtce compares activities over three characteristics:
their level of (1) uncertainty, (2) asset specificity, and (3) ex-post information asymmetry.
Scores on these three variables together characterize the activity and give rise to specific
control problems, whereas each aspect has its own unique influence.
Uncertainty or complexity surrounding an activity creates a lack of ex-ante knowledge about
the activity. This knowledge concerns both processes (how best to carry out an activity) and
outcomes (what results can we expect). Within TCE-based research, we encounter multiple
guises of uncertainty. Generally, researchers distinguish environmental uncertainty and
behavioural uncertainty (see Rindfleisch and Heide, 1997; Boerner and Macher, 2001).
Regardless of the type, the consequences are always a diminished capability to program
activities (Speklé, 2004). When uncertainty is low, programming of activities is possible.
Hence, supervisors are able to identify the tasks to be performed and can set up specific
instructions. In case of high uncertainty, predefining tasks appears difficult or even impossible
and control has to be exercised in alternative ways. Control problems related to dissemination
of information and (hence) goal alignment will be present. Note that high uncertainty is
problematic because of bounded rationality. In the presence of uncertainty, the limits of
rationality could be reached, so that it becomes too costly or impossible to obtain the
information required (cf. Williamson, 1975). If it were possible, for instance, to document all
potential scenarios and establish detailed procedures for each, control problems surrounding
uncertainty would vanish.
Uncertainty becomes problematic only in the presence of the second characteristic of an
activity, asset specificity. In case of low asset specificity the market provides all required
knowledge about transactions through the price mechanism, regardless of the level of
uncertainty. However, if asset specificity is present to a nontrivial degree (Williamson, 1985,
p60), the market cannot provide the required knowledge and the effects of uncertainty cannot
be averted. Studying the hierarchy, only moderate and high levels of asset specificity are
considered here.
Asset specificity concerns opportunity costs of investments made to support the activity
(Speklé, 2004, p12). Central concern is whether investments made in specific assets can be
redeployed (Williamson, 1985). If the assets cannot be redeployed, the investments will be
18
Configurations of control
considered a loss in case of determination of those activities that exploit the assets. Imagine,
for example, machinery that can only produce one component. If it cannot be used for the
production of anything else, nor be of any use in a similar organizational unit of another
company, opportunity costs of investing in this machinery are significant and asset specificity
is high. This situation gives rise to control problems. The organizational unit that produces the
component cannot rely on multiple potential buyers. Moreover, the unit that needs the
component cannot obtain it elsewhere. The investments, thus, create dependencies that in turn
leave room for opportunistic behaviour, which consequently has implications for control
needs.
Six kinds of asset specificity have been distinguished (Williamson, 1991): site specificity,
physical asset specificity, human asset specificity, brand name capital, dedicated assets, and
temporal specificity. One or more of these forms could be relevant for activities of an
organizational unit, determining asset specificity as such. The presence of asset specificity
creates control problems related to opportunistic behaviour. In case of moderate levels of
asset specificity, market benchmarks are available to assess performance (Speklé, 2004).
Despite some tailoring to specific company needs, supervisors can more or less compare the
activity of the organizational unit to alternatives in the market and curtail opportunism to
some extent. Controlling activities with high asset specificity calls for a different approach.
These activities simply do not have a market alternative that could function as a disciplining
device benchmarking performance. Supervisors have to turn to administrative performance
targets or rules and regulations to confront opportunism.
Finally, activities inhibit different amounts of ex-post information asymmetry6. In
organizational and management control literature alike, information differences between
actors are a familiar concept. The MCtce, however, assigns a distinctive role to information
asymmetries that exist at the end of a period (i.e. when performance evaluation takes place)7.
Once an activity has been carried out, a supervisor might not be able to evaluate actually
delivered performance due to substantive information differences. What exactly has been
achieved? And, did the manager achieve the best results possible or not? Like before, this
situation offers possibilities for opportunistic behaviour and calls for specific control
approaches.
6
Speklé uses both the terms information impactedness and information asymmetry. The former implies
that it is either too costly or impossible to transfer information (Williamson, 1975). Thus, information
asymmetry is a direct consequence. I will only focus on information asymmetry because of the more
regular use of this terminology within MC and organizational research.
7
Ex-ante information asymmetries are part of uncertainty and not treated separately.
19
Chapter two
Uncertainty, asset specificity, and ex-post information asymmetry jointly define the activity.
Each characteristic has its own effect on control needs, but its relevance depends on the
combination of all three. For instance, by definition information asymmetries are irrelevant in
case of low uncertainty. These variables are treated as contingency variables in this study.
They are three different aspects of activities that interact and therefore need to be dealt with
simultaneously.
Like standard TCE, the MCtce relies on two traits of human beings, -bounded rationality and
opportunism-, to explain why control problems occur. This might seem very strict. Not all
human beings will act opportunistically at every opportunity. However, it is unnecessary to
assume that this is the case. It suffices to claim that at least some people are opportunistic
sometimes and therefore control problems exist.
Note that the MCtce deviates somewhat from standard TCE based research in its choice for the
contingency variables. Instead of focusing on frequency as the third variable, the theory
stresses the role of ex-post information asymmetry. Frequency cannot play a distinguishing
role in explaining different forms of control within the hierarchy (Speklé, 2004). Studying
management control design choices within the hierarchy, frequency will always be relatively
high because activities with a low frequency will be left to the market (Williamson, 1985).
Therefore, to explain differences in the effectiveness of management control structures, the
MCtce gives ex-post information asymmetries a more prominent role. This variable also stems
directly from TCE (see for instance Williamson, 1975), and is characterised as a derivative
condition that arises from uncertainty and opportunism.
2.2.3 Control solutions: archetypes
Generally speaking, management control systems provide solutions to control problems.
However, not all control systems are effective under all circumstances. The Transaction Cost
Theory of Management Control articulates a framework of five archetypes: arm’s length
control, result oriented machine control, action oriented machine control, exploratory control,
and boundary control. The archetypes exist of a combination of different control instruments
that as a set provide effective control solutions. The archetypes derive their effectiveness from
their ability to deal with specific control problems in a coherent way. The archetypes are not
necessarily equally effective (also see 2.3.1), but each archetype is the relatively most
effective for the control of specific activities. The theory gives rich descriptions of all
archetypes covering the four dimensions of control. An introduction to the archetypes follows.
Table 2.1 gives an overview of their main characteristics.
20
Configurations of control
Table 2.1 Characteristics of the control archetypes
Arm’s Length
Control
Machine Control
(Result Oriented)
Machine Control
(Action Oriented)
Structure
Relative
autonomy;
involvement
higher level
management
limited as
long as
performance is
satisfactory
Decentralized with
clearly defined
areas of
responsibility and
accountability
Standardization
Market-related
outcome
requirements;
external
performance
benchmarks
Performance
assessment
relative to ‘the
market’
Monitoring and
performance
evaluation
Reward &
incentive
structure
Performance
dependent
bonuses
Exploratory
Control
Boundary
Control
Well-defined
tasks; strict
hierarchy; limited
room for
discretionary
behaviour
Relatively flat
hierarchy; fluid
and permeable
matrix-like
project
structures;
vague
responsibilities
Relative
autonomy
within defined
boundaries
Predefined
performance
targets of
administrative
origins
Standardization of
behaviour;
detailed rules,
norms, and
instructions
No ex ante
standards and
targets; ‘do your
best’; emerging
standards
Proscriptive
codes of
conduct;
boundary
systems;
emphasis on
behaviour to
be avoided
Monitoring
focused on target
achievement;
performance
assessment
relative to targets
Monitoring and
supervision to
ensure
compliance to
norms and
standards
Based on
emerging
standards;
subjectively
assessed
contributions to
long term
organizational
performance
Focused on
compliance;
observance
of
interdictions;
external
audits
Performance
dependent
bonuses
No direct link
between
performance and
rewards
Career
prospects
dependent on
long term past
performance;
peer pressure
Emphasis on
threat of
punishment
of rulebreaking
behaviour;
tie-in through
‘hostages’
Source: Speklé, 2004
Discussing control again from the perspective of a supervisor who exerts control on a
manager, with arm’s length control the supervisor keeps at a distance. The manager has a lot
of autonomy and is not subjected to all kinds of rules and regulations specifying what to do.
Performance evaluation takes place relative to the market. Thus market benchmarks act as
performance targets. In case of satisfactory performance the manager receives a bonus.
The second and third archetypes are two variants of machine control: action oriented machine
control and result oriented machine control. Standardization of behaviour and detailed work
instructions epitomize the action oriented variant. The manager has little autonomy and
his/her compliance with rules and standards will be monitored and provides the basis for
performance evaluation. There is no direct link between performance and rewards. In stark
contrast to this is machine control of the result oriented kind. With this archetype, the
manager also has clearly defined responsibilities, but focus is on achievement of performance
targets instead of on compliance with rules and regulations. However, performance targets as
part of machine control are administrative targets, internally determined. Within this result
oriented variant the manager will be rewarded a bonus when he/she achieves the targets.
21
Chapter two
The fourth archetype is exploratory control. A main characteristic of this archetype is a lack
of ex-ante work instructions and rules. The supervisor defines general goals and gives the
manager considerable leeway to act upon them. Moreover, preset performance targets are
absent and emergent targets take over. During the carrying out of an activity, information
about what can be achieved becomes available for evaluation purposes. The supervisor uses
subjective judgements to assess performance and looks at performance in the long run. The
manager awaits good career prospects instead of short term bonuses as a reward for his or her
job.
Boundary control, the fifth archetype, provides the manager with a very high autonomy,
restricted only by prohibitive guidelines or boundary systems. There are no performance
targets or work related instructions to coordinate the work. The supervisor keeps at a distance
and the only thing monitored is compliance with the proscriptive rules. Non-compliance has
serious consequences for the manager’s career.
2.2.4 Matching problems and solutions: effectiveness
To summarize the argument: uncertainty, asset specificity, and ex-post information
asymmetry jointly characterize activities of organizational units. The carrying out of these
activities by human beings characterised by both bounded rationality and opportunism creates
control problems. Different MCSs can handle different control problems. Therefore some
systems control certain activities more effectively than others. Five archetypes represent
effective control forms. Each archetype is the relatively most effective for the control of
specific activities.
From these theoretical insights it follows that a management control system of an
organizational unit should resemble its relevant archetype in order to be effective. Which
archetype is relevant depends on the activity of the organizational unit. The main claim is: the
closer an observed MCS of an organizational unit resembles its relevant archetype, the more
effective it will be. Figure 2.2 gives an overview of the archetypes and the activities they
control best.
22
Configurations of control
BOUNDARY CONTROL
Low
ARM’S LENGTH
CONTROL
MACHINE CONTROL
action oriented
and result oriented
Moderate
High
Low
High
Uncertainty
EXPLORATORY CONTROL
Ex-Post Information
Asymmetry
High
Figure 2.2 Archetypes of control and their habitat
Asset Specificity
Source: Speklé, 2001b, p76
In case of low uncertainty and moderate asset specificity, arm’s length control provides an
effective solution. Activities are programmable and market-based standards are available. Use
of market benchmarks in guiding behaviour and evaluating performance is thus both feasible
and natural. From this the first hypothesis follows (cf. Speklé, 2004):
H1:
In case of low uncertainty and moderate asset specificity, resemblance of an MCS
with the archetype arm’s length control is positively associated with MCS
effectiveness.
Combined with a high level of asset specificity, low uncertainty calls for machine control.
Whether the result oriented or the action oriented type prevails, depends on “the ability to
define meaningful and sufficiently restrictive output targets” (Speklé, 2004, p 22). It is a
forced choice because the available information will enable the one or the other form. To
distinguish between the two types of machine control, I take on the measurability of outputs
as an extra contingency variable. The variable functions as a proxy representing the definition
of output targets. Researchers generally use this variable to indicate the appropriateness of
action versus result controls (see for instance Merchant, 1982 and Ouchi, 1980). When asset
specificity is high, market-based standards are no longer available to supervisors. However, if
the outputs can be quantified, output control can still provide an effective solution using
administrative targets instead of market benchmarks. Result oriented machine control is the
relevant archetype. Therefore, the second hypothesis claims:
23
Chapter two
H2:
In case of low uncertainty, high asset specificity, and high measurability of outputs,
resemblance of an MCS with the archetype result oriented machine control is
positively associated with MCS effectiveness.
When asset specificity is high, but it is impossible to quantify the outputs, result control is no
longer feasible. Due to the programmability of activities though, rules and behavioural
guidelines provide control solutions. Thus, action oriented machine control reigns. From this
hypothesis three follows:
H3:
In case of low uncertainty, high asset specificity, and low measurability of outputs,
resemblance of an MCS with the archetype action oriented machine control is
positively associated with MCS effectiveness.
High levels of uncertainty make programming of activities difficult or impossible. Standard
operating procedures or specific performance targets cannot be used. Other forms of control
will be effective depending on the level of ex-post information asymmetry. Sometimes it is
possible to evaluate performance afterwards. This is the case when standards that can be used
for evaluation purposes, emerge during the process of activity execution. Accordingly, expost information asymmetry is low and exploratory control is the best option. However,
standards do not always emerge during the process and ex-post information asymmetry
reaches a high level. Consequently, control is hardly possible at all and supervisors can only
rely on prohibitive rules specifying what not to do rather than what to do. The archetype
boundary control will be effective in this situation. From this line of reasoning hypotheses
four and five follow:
H4:
In case of high uncertainty and low ex-post information asymmetry, resemblance of
an MCS with the archetype exploratory control is positively associated with MCS
effectiveness.
H5:
In case of high uncertainty and high ex-post information asymmetry, resemblance of
an MCS with the archetype boundary control is positively associated with MCS
effectiveness.
24
Configurations of control
2.3 Reflections on the theoretical ideas
This paragraph highlights some aspects of the theory that deserve more attention, and firstly,
elaborates on configurational thinking (2.3.1). I focus on the notion of fit, which is a central
concept in configurational as well as contingency thinking. Next, subparagraph 2.3.2 gives
explanations on why misfits would occur. The assumptions about effectiveness and its link to
misfit also deserve our attention and will be discussed as well.
2.3.1 The configurational approach
Previously, the Transaction Cost Theory of Management Control has been characterised as a
contingent configuration theory. Configurations or archetypes represent internally consistent
packages of organizational elements (Doty and Glick, 1994). Only a few configurations are
effective, because of the interrelationships between the elements. These theoretical
assumptions make the approach well suited for the study of management control systems as
interrelated packages of control instruments. In this subparagraph I discuss the main features
of the configurational approach and the implications for testing the hypotheses.
The configurational approach incorporates multiple levels of theorizing (Doty and Glick,
1994). At the level of the grand theory, the link between contingency variables, archetypes,
and effectiveness is relevant. The hypotheses stated above are direct tests of the theory at this
level. Configurational approaches also incorporate middle-level theories that relate to the
internal consistency of the archetypes. Doty and Glick (1994) explain that the configuration of
organizational elements has a synergistic rather than an additive effect. These middle-level
theoretical ideas will also be addressed in this study, be it in an exploratory way.
The configurational approach incorporates a specific notion of fit. Generally speaking, ‘fit’ is
the state that leads to effectiveness: there is a match between the organizational structure and
the situation in which the organization operates (Donaldson, 2001). In this study, the level of
fit depends on the resemblance of an observed management control system with its relevant
archetype. Which archetype is relevant depends on the combination of the contingency
variables. For instance, for the control of activities with low levels of uncertainty and asset
specificity, arm’s length control is the relevant archetype. If the observed MCS in this
situation does not resemble the archetype, there will be a misfit and a negative effect on
performance. Different concepts of effectiveness exist and different notions of fit. The
perception of fit central to this research project can be characterised as configuration
contingency fit (in terms of Gerdin and Greve, 2004), as (contingent) ideal types fit (cf. Doty,
25
Chapter two
Glick, and Huber, 1993), or as systems fit (in terms of Drazin and Van de Ven, 1985).
Distinguishing features of this type of fit are (1) that it concerns multiple structure variables
(combined in the archetype) as well as multiple contingency variables, and (2) only a discrete
number of fit states exist. Misfit is always expected to influence performance negatively.
When dealing with configurations one should compare the effectiveness among members that
belong to the same archetype but differ in level of resemblance (Doty et al., 1993). It is
predicted that members that marginally resemble the configuration are less effective than
those that closely resemble it. Thus, to test configuration theories one should not assign cases
to groups according to their relevant archetype and then compare effectiveness among the
archetypes.
The question arises whether one should compare effectiveness among the archetypes at all. Is
the archetype arm’s length control as effective as boundary control if matched to the
activities? Doty and colleagues state that configurational theories incorporate the equifinality8
assumption: “multiple equally effective organizational forms exist” (1993, p1199). They
assume each ideal type is equally effective: there are more means to an end. Regarding the
Transaction Cost Theory of Management Control, the equality or difference in effectiveness
among the archetypes is not addressed explicitly, but the equifinality assumption seems
inappropriate. Speklé (2001a) submits that the archetypes are each effective given the
circumstances, but they represent also the best we can do, and boundary control, for instance,
is characterized as “a control structure of last resort”.
Assuming equifinality has consequences for the way configurational theories can be tested.
For instance, Doty et al. (1993) test relationships between misfit and effectiveness for
Mintzberg’s (1983) organizational types using the complete sample. For each case they firstly
determine the relevant archetype and thus the relevant misfit score, but in their test they use
all cases simultaneously. Under the assumption of equifinality this is perfectly sound.
However, when the archetypes are not thought to have the same level of effectiveness, one
can only test the theoretical claims for subsamples and compare only those cases that share
the same relevant archetype, or alternatively, somehow correct for the differences in the levels
of effectiveness each archetype can attain. In chapter five that sets out the analyses, I will
explain how I have dealt with these issues in this study.
8
The concept of equifinality can be interpreted in different ways. For an overview and discussion, see
Donaldson, 2001.
26
Configurations of control
2.3.2 Occurrence of misfits
To test the hypotheses as formulated above, variation in the level of misfit as well as variation
in the level of effectiveness must be observed. The underlying assumption is that the way in
which organizational units establish control is not by definition effective. Some MCSs are
more effective than others depending on the level of fit, in this case their resemblance with
the relevant archetype. This implies that misfits will occur in practice.
In his Transaction Cost Theory of Management Control, Speklé (2004) mentions a natural
tendency towards matching MCSs to activities (because that is effective), yet suggests that
mismatches occur routinely. There are several reasons to expect the occurrence of deviations
from the archetypes. Regarding the alignment of transactions and governance structures,
Williamson (1996) refers to the complexities of organization as well as men’s limited
cognitive abilities to explain the occurrence of failures in alignment. It is also acknowledged
that effectiveness is the most important, but not the only determinant of control structure.
Besides, mismatches can occur during times of change. Contingency literature assumes that
states of misfit occur and can last for some time, when one moves from one state of fit to
another (Donaldson, 2001). In sum, there are multiple reasons to presume that misfits occur
regularly. Thus from theory, I expect that there will be a general tendency towards matching
the archetypes, but misfits will occur as well.
This triggers another question: can misfits exist for single control elements? Or, put
differently, can an organization gradually move from one archetype to another? Paragraph 2.1
described a management control system as a package of interrelated control elements,
stressing complementarities among the elements. The MCtce submits that only specific
combinations (represented by the archetypes) are effective. The theory does not explicate to
what extent deviations from the archetypes are possible, or whether adjustments to the control
system accord with changes in the contingency variables. However, in their review of
contingency forms of fit in management accounting research, Gerdin and Greve (2004) state
that researchers who advocate the configurational approach, assume change to occur by
‘quantum jumps’, rather than via gradual adjustments. Organizations are expected not to
adjust their systems until the performance effects are severe enough to justify the costs
associated with substantial change. This view differentiates the configurational approach from
the contingency approach as illustrated in figure 2.3.
27
Chapter two
Figure 2.3 Contingency versus configurational fit
Panel A
Panel B
S
T
R
U
C
T
U
R
E
S
T
R
U
C
T
U
R
E
CONTINGENCY
CONTINGENCY
Source: Gerdin and Greve, 2004, p306; modified
Panel A shows the contingency view to fit. Multiple states of fit exist that are all on the fitline. A deviation from the line implies a state of misfit, but organizations can move along the
line and gradual adjustments of the organizational structure are possible (see also Donaldson,
2001). Panel B represents the configurational approach to fit. Only a discrete number of fit
states are possible (indicated by the three circles). To move from one state of fit to another a
quantum jump has to occur. In all cases, organizations that are in a state of fit show superior
performance over those that are in misfit.
Change of control systems as such is not under scrutiny in this research project, but
assumptions about the occurrence of misfit and the relationship between misfit and
effectiveness are relevant. These assumptions have implications for modelling the
relationships (for a debate on these issues see Gerdin, 2005a/2005b, and Hartmann, 2005).
The theoretical form of the relationship between misfit and effectiveness of control remains
unclear. Does a single misfit immediately impact effectiveness? When the complementarities
among the control elements are strong this might be the case. Or, is there a minimum level of
misfit to be reached before performance effects occur? Burton, Lauridson, and Obel (2002)
provide some empirical insights. They study a model with multiple misfits for small- and
medium-sized enterprises. They distinguish between the situational context (environmental
conditions and technology, for instance) and characteristics of the organization
(decentralization and formalisation, for instance) to identify multiple states of fit. The authors
show that firms with no misfits perform significantly better than firms with misfits. Moreover,
they conclude that neither the type nor the number of misfits matter. A single misfit can lead
to a loss in performance. The issues raised in this paragraph have implications for the tests of
28
Configurations of control
the theoretical ideas and for the models used in the analyses. The next paragraph looks ahead
and renders thoughts on the theory’s empirical test.
2.4 Concluding remarks
To research the central question of this project – which MCSs are effective given the
circumstances – I build from the Transaction Cost Theory of Management Control (Speklé,
2001a, 2001b, 2004). This theory provides a theoretical answer to the research question. It
submits that there are five effective forms of management control systems referred to as
archetypes of control: arm’s length control, result oriented machine control, action oriented
machine control, exploratory control, and boundary control. Each is effective only for the
control of certain activities, but not for others.
In this project, I study organizational units with various activities. Examples are production,
human resources, research and development, and sales. These activities are characterised over
three dimensions: the level of uncertainty, asset specificity, and ex-post information
asymmetry. The effectiveness of the archetypes is contingent on these characteristics. For
instance, arm’s length control is the most effective control structure for activities with low
uncertainty and low asset specificity. From this the main claim of the theory follows: the
closer an observed MCS resembles its relevant archetype, the more effective it will be. Five
hypotheses are formulated to test this main claim, one for each of the archetypes.
The MCtce is a complex theory that studies multiple contingency variables and multiple
archetypes and combines these to explain effectiveness of control. It is also a far-reaching
theory, which pertains to all kinds of activities. Looking into different activities ultimately
implies comparing organizational units as distinct as production and research and
development. Therefore, to test the theory and learn about management control system
design, I take on a generic approach towards control instead of trying to study all the specifics
of control systems. To study control in its entirety I will work from the broad working
definition of management control systems and map all four control dimensions.
Apart from measurement issues, testing the theory involves the operationalization of the
archetypes and the hypothesised relationships. The theory describes the archetypes and the
situations in which each is effective in a qualitative manner. Choices have to be made
regarding the definition of the archetypes, the boundaries to be set that distinguish high and
low levels of the contingency variables, and the appropriate link between misfit and
effectiveness. Establishing a match between the theoretical ideas and the empirical constructs
29
Chapter two
will be a central concern throughout. The next two chapters deal with the methodology of this
study (chapter three) and the operationalization of the concepts (chapter four).
30
Chapter three – Methodology
Data form the basis of every empirical study and their quality is a necessary (but not
sufficient) condition for making valid inferences about theoretical claims. Thus, to be able to
conclude anything about management control system design and effectiveness, firstly reliable
measurement of these variables must be ascertained. As I will argue below, the appropriate
method for this study is the survey method. Survey research has received a lot of criticism as
to the quality of the data obtained. Central concerns are matching the theory with the
empirical level of analysis, applying the survey method correctly, and assuring content
validity. This chapter shows how I have confronted this critique and assured the quality of my
data. I introduce a new technique, referred to as ‘validity-feedback interviews’, as a powerful
method to obtain valuable feedback on the content validity of survey instruments.
This chapter provides firstly an overview of the general methodological approach. A
methodological framework illustrates all necessary steps that enable hypotheses testing.
Moreover, the choice for survey based research will be justified and the critique on survey
studies discussed. Then, paragraph 3.2 describes the development of the questionnaire which
led to a basic set-up. Next, the validity-feedback interviews are introduced (3.3) and the
extensive pre-test the questionnaire was put through is described (3.4). In paragraph 3.5 I
discuss sampling and the data gathering process. This chapter ends with a description of the
dataset (3.6), and concluding remarks on the validity and reliability established through the
methodological approach (3.7). Chapter four will present details on exact questions asked and
deals with the measurement of individual variables.
3.1 The methodological approach and the choice for a survey
This paragraph explicates the methodological steps needed to test the theoretical claims. From
that the choice for the survey method follows. Next, I discuss the critique survey research has
recently received.
3.1.1 The methodological framework
The last chapter set out the theoretical ideas underlying this research project. To learn about
MCS design and effectiveness I work with the Transaction Cost Theory of Management
Control that explicates several archetypes as well as the circumstances under which each is
effective. The main theoretical claim is that the effectiveness of a management control system
31
Chapter three
increases with the resemblance of the control system to its relevant archetype. I focus on
measuring misfit as the lack of resemblance. Testing hypotheses that relate misfit to
effectiveness of control will provide insights into the empirical power of the theory. To enable
such hypotheses testing I set up a framework that shows all necessary methodological steps
(figure 3.1). Each step involves choices regarding methods, with central concerns the validity
and reliability of data collection, scale construction, and measurement of variables.
Figure 3.1 Methodological framework
define
define activity
activity
determine
determine relevant
relevant
archetype
architype
compare
compare MCS
MCS with
with
relevant
relevant archetype
archetype
map MCS
link misfi
misfitt to
to
link
effectiveness
effectiveness
measure
measure MCS
MCS
effectiveness
effectiveness
First of all, the framework indicates that data collection should return information on
management control systems in their entirety and on their effectiveness. This will be done for
various activities (for instance, production or human resources), since activities form the level
of analysis and I study MCSs surrounding those. Therefore, as a separate step, it is necessary
to identify the activities and capture the characteristics thereof. Scores on the three
contingency variables, uncertainty, asset specificity, and ex-post information asymmetry,
jointly define the activity. The theory indicates the relevant archetype for different groups of
activities. In the next step I use the scores on the contingency variables to determine the
relevant archetype for each case. For instance, for an activity, which scores high on
uncertainty, moderate on asset specificity, and high on ex-post information asymmetry, the
relevant archetype is boundary control. According to the theory, resemblance of an MCS with
its relevant archetype leads to effectiveness. Therefore, once the relevant archetype for each
case is determined, the next step comprises calculation of a misfit score expressing the lack of
resemblance of the MCS to its relevant archetype. This step necessitates translation of the
archetypes’ qualitative descriptions into quantitative scores for every single element. Finally,
misfit and effectiveness of the management control system have to be brought together in a
testable model.
In sum regarding data collection, three things are important: first of all, it is necessary to map
management control systems (1). Detail is important to pick up a control system in its entirety
without loosing track of the nuances of control. At the same time, though, only a large
32
Methodology
number of observations can enable hypotheses testing (2). Moreover, to test hypotheses for all
five archetypes I need data on a vast variety in activities (3). Such variety in activities can be
found throughout different industries and companies, and within companies at different
hierarchical levels. A survey can reach all kinds of organizational units each with its own
activities, and meet the requirements of providing a large quantitative dataset. Moreover,
applying the survey method can return detailed information on numerous organizational
aspects simultaneously. Archival data are generally unavailable for studies on management
control systems (cf. Ittner and Larcker, 2001), especially at lower levels within the hierarchy
(think, for instance, about the amount of work autonomy). Moreover, to be able to compare
multiple activities and test the theoretical claims, I need to abstain from specifics of the
control system and focus on generic forms of control. This generic approach limits the use of
available archival data, which are tailored to the organization and difficult to compare over
distinct organizational units. Case studies would easily provide enough detail, but lack
volume. Thus, survey research can best serve the purposes of this study and is the logical
choice.
However, survey research requires the operationalization of the constructs, which brings some
challenges. First of all, precise definition of an MCS ex-ante is necessary. The challenge is to
balance the amount of detail desired against the number and complexity of questions feasible
within the survey method. Moreover, taking on a generic approach towards control and
focussing on variety in activities implies comparing, for instance, a production unit and a
research and development unit in terms of their control structure and its effectiveness as well
as the characteristics of their activities. This requires the use of generally applicable
measurement instruments. As will become clear below, a lack of examples from prior
research limits their use. Apart from having to deal with the critique survey research receives,
this study thus faces the invention of new measurement instruments able to pick up MCSs in
their entirety and to assess their quality in a generally applicable way that enables comparing
a variety of organizational units. Therefore, verification of their content validity will be of
utmost importance.
3.1.2 Survey research: method and critique
The survey method provides researchers with a powerful tool for conducting empirical
research. However, use of the method within accounting research receives much criticism.
This critique relates to two main concerns. The first concern is the quality of the match
between theory and empirical operationalization, specifically between the theoretically
implied level of analysis and the measured level of analysis (Luft and Shields, 2003). A
33
Chapter three
mismatch has serious consequences, because it leaves us unsure of whether indeed the theory
has been tested or something else. Authors recommend to define precisely the constructs and
to be specific about the relationships between them.
The second concern with current survey research involves preventing both measurement error
and sampling bias. Critique in this regard stresses the lack of effort put into using available
techniques to prevent such biases (Ittner and Larcker, 2001; Zimmerman, 2001) leading to
data of poor quality. Bias introduced through, for instance, non-random sampling impedes
generalization of findings. Moreover, low response rates can distort neatly designed samples.
Several techniques are available to curb these problems (for suggestions see Dillman, 2000).
Furthermore, measurement error leaves us unsure of content validity: have we actually
measured what we intended? Pre-testing a questionnaire proves an effective tool in addressing
this problem. Multiple methods are available, yet only few researchers deploy them. Van der
Stede, Young, and Chen (2005, p670) find only 23% of all studies part of their assessment on
the quality of survey research indicate a pre-test of their instruments. Moreover, researchers
report only sparingly on their pre-tests, leaving others not only in the dark about the quality
and benefits of these tests, but also without examples of good practice.
All in all, Van der Stede et al. hit the nail on the head in concluding that “the key issue with
the survey method, (…), centres more on how it is deployed, rather than with the method
itself” (2005, p656). The indicated problems with the survey method can thus be overcome by
taking the design and execution of the method seriously.
The rest of this chapter shows how I have confronted this criticism to obtain data of high
quality. Assuring the link between theory and empirics through the operationalization of
constructs is our first concern. To establish validity, it is important to be informed by the
theory (1) and to put the questionnaire up to pre-testing (2). Both issues will be dealt with
extensively in paragraphs 3.2, 3.3, and 3.4. Reliability can be anticipated by using multiple
items/questions to measure a single construct, but is being assessed post hoc using statistics
(for instance, Cronbach’s Alpha)9. The second issue in survey research concerns preventing
selection bias and assuring a reasonable response rate. Both are part of the sampling strategy
and the survey administration. I explain the strategy chosen and the results in terms of the
dataset obtained in paragraphs 3.5 and 3.6.
9
The next chapter presents details on the measurement of all variables, together with an assessment of
their construct validity and reliability.
34
Methodology
3.2 Design of the questionnaire
This paragraph gives an overview of how the questionnaire came into existence and thereby
addresses the first critical comment on survey research: assuring a match between theory,
measurement of constructs, and empirical level of analysis. It starts with a description of the
questionnaire’s set-up and explains how it was obtained. Next, attention shifts to creating a
focus within the questionnaire to match the theoretical and empirical levels of analysis. The
questionnaire’s design consisted of several stages. Starting from the theory, I set up working
definitions for all the constructs. Firstly, careful translation of variables’ working definitions
into measures led to a basic set-up, which experts reviewed. Two rounds of pre-testing the
quality of the questionnaire subsequently resulted in a final version. Figure 3.2 summarizes
the design process of the questionnaire.
Figure 3.2 Questionnaire design phases
translation of
working
definitions into
measures and
questions
expert review;
finish basic
set-up
(Tailored Design
Method)
pre-test with two
phases
(TSTI plus validityfeedback interviews;
16 interviews)
finish
questionnaire:
start data
collection
first pre-test
(TSTI; 6 interviews)
revision, feed
forward loop to
theory
second pre-test
(TSTI plus validityfeedback interview;
10 interviews)
Note: TSTI = Three-Step Test-Interview (explained in the next paragraph)
3.2.1 The questionnaire’s set-up – matching the theory
The purpose of the survey is to map management control systems in their entirety, measure
the contingency variables, and assess control system performance in order to test the
theoretical claims. The information retrieved should enable comparison between activities as
distinct as production and Research and Development. This calls for both a high
correspondence to the Transaction Cost Theory of Management Control and generally
applicable measures that focus on generic control forms. To assure a match with the theory, I
build directly from the definitions given in chapter two and turn to prior research as much as
possible. The next chapter provides details on individual questions and variables. The
questionnaire’s basic set-up was subjected to an expert review in which correspondence
between questions and theory was judged.
35
Chapter three
Starting of with the rich qualitative descriptions of the archetypes (see table 2.1) I set up a
working definition of a management control system covering all four dimensions of control.
Next, I turned to both accounting and organization literature in search of measurement
instruments. Books by Van de Ven and Ferry (1980) and Price (1997) were especially helpful
in finding instruments for parts of the MCS. However, the possibilities for building from prior
research are limited, not because researchers do not study general types of control
instruments, but because there are hardly any studies that focus on MCSs in their entirety.
This triggered the invention of new measures, especially to pick up on the interrelationships
between the separate dimensions of control. Instead of focusing on every detail of an MCS,
general strands of control form the object of study. For instance, I do not document the
specific rules or procedures that are used within organizational units, but ask about whether
rules or regulations apply and about the consequences of non-compliance. Taking on this
abstract view towards control enables the study of generic control structures and hence makes
comparison of various organizational units and their MCSs possible.
The theory makes explicit claims about the effectiveness of control. Therefore, this variable
should be picked up by the questionnaire as well. However, for MCS effectiveness there is no
ready-made measurement instrument available either. Not only does the search for generally
applicable measures complicate matters, but also a lack of studies that assess control
effectiveness as such. Researchers do not actually measure effectiveness of management
control systems, but use the effectiveness of the organization or organizational unit as a proxy
(cf. Chenhall, 2003). In so doing, however, they assume linkages between the working of the
management control system and the organizational outcomes, for which there is no
compelling evidence (Chenhall, 2003). The only exception is Ferreira and Otley (2005) who
develop a measure of the perceived effectiveness of a control system at the company level.
Inspired by their work, I have developed a new measure of MCS effectiveness that covers
more aspects of control effectiveness and is generally applicable.
Apart from mapping the management control system and assessing its effectiveness, the
questionnaire must be able to identify the activities. Generally applicable measures should
pick up the characteristics of these activities in terms of uncertainty, asset specificity, and expost information asymmetry. Considering the enormous amount of empirical TCE-based
studies (for recent overviews see David and Han, 2004 and Boerner and Macher, 2001), one
would expect to find ready-made measures for these key variables. However, both asset
specificity and uncertainty have been measured in very diverse ways (David and Han, 2004;
Boerner and Macher, 2001) generally tailored to specific industries or types of organizational
units. The variables are difficult to measure, especially in a way that enables comparison
36
Methodology
across industries (Shelanski and Klein, 1995). Since I need generally applicable measures
these examples are of limited use. Moreover, both asset specificity and uncertainty are broad
constructs that can take on multiple guises. Most researchers focus on single aspects, typically
human asset specificity and unpredictability (Rindfleisch and Heide, 1997; Boerner and
Macher, 2001). If one can reasonably assume ex-ante that one particular type of, for instance,
asset specificity will be most important, this approach seems justifiable. However, my study
compares very distinct activities for which such assumptions cannot be made. To the contrary,
the most important type of both asset specificity and uncertainty likely differs between
activities. To establish a close resemblance between theory and operationalization, I measure
multiple guises of both constructs. Although I build from prior research whenever possible,
my approach deviates substantially and multiple new measurement instruments were
developed.
Overall this approach resulted in a basic set-up of the questionnaire that contained 81
questions, half of which are newly developed. The design was made in accordance with the
Tailored Design Method (Dillman, 2000). The practical recommendations Dillman provides
in his book were taken on for general design (how to pose a question), lay-out (navigation,
back cover), and writing a cover letter.
3.2.2 Focus on activities - matching the level of analysis
The relevant level of analysis is the activity and management control systems surrounding
these should be mapped by means of the questionnaire. Examples of activities are sales,
accounting, and the production of a certain good or service. To test the theoretical claims I
study a variety of activities found at different hierarchical levels throughout the organization.
This complicates the ex-ante identification of one activity to ask managers about, because it is
entirely possible that managers head an organizational unit with multiple activities, where the
unit is their initial point of reference, not the separate activities10.
Consider what happens without maintaining an explicit focus on one activity. Managers could
probably still answer the questions on management control instruments in use, but would take
average scores for all the activities of their unit. Think for instance about the level of work
autonomy of unit employees. This might differ between groups of employees working on
different activities. In fact, theory predicts that it would. At the outset, it is difficult to assess
the full scale and the impact of the problem. If there is only one activity in an organizational
10
Indeed in my sample, 88 percent of the respondents indicated that within their organizational unit
multiple activities are carried out (see 3.6 for a description of the dataset).
37
Chapter three
unit, there is no problem. If multiple activities are carried out in a unit, but do not differ in
terms of the contingency variables, there is no problem either. It would still be possible to
relate MCSs to certain types of activities. However, for cases in which dissimilar activities are
combined, the theory cannot be tested. The empirical level of analysis would not fit the
theoretical one and render tests of the theory invalid.
Both the expert review and the pre-test returned anecdotal evidence showing that simply
asking managers about control of their unit’s main activities without specifying them, cannot
solve the problem. Not only did managers regularly identify several ‘main’ activities within
their organizational units, they also indicated differences in control of these activities. This
became most apparent during the first pre-test phase when a manager started answering
questions on the coordination of work twice: one answer pertained to people working on
administrative tasks (half his employees) and another to controllers working as internal
consultants (the other half)11. The case is all the more interesting because the manager also
indicated differences between these groups in programmability of tasks (an aspect of
uncertainty) and in weeks of training needed to perform the job (an aspect of asset
specificity). Clearly, when studying management control system design, taking the
organizational unit as a level of analysis is different from taking the activity as the focal point.
Moreover it is problematic to ask about aspects of control without specifying activities first. It
is thus necessary firstly to identify for each organizational unit its main activity. After that it
is possible to ask about the characteristics of that activity and the MCS surrounding the
activity.
To assure a focus on activities as the relevant level of analysis, the questionnaire starts by
illustrating what is meant by an organizational unit. Later on managers are asked to list up to
five activities of their unit and choose one to focus on throughout when answering questions
on the MCS and the contingency variables. Several reminders help to maintain this focus.
Figure 3.3 visualizes organizational units (in the questionnaire referred to as subunits). Recall
that a manager and all employees reporting directly or indirectly to him or her comprise the
unit. The same picture appears on the first page of the questionnaire.
11
The question posed was: “how do you coordinate the work within your unit?” with items: planning,
standard operating procedures/work instructions, detailed assignments, assignments stated in general
terms with little detail, proposals made by employees followed by consultation with you. Scale rang: 1
(not at all) – 5 (always). The question has been modified since and has number 54 in the questionnaire.
38
Methodology
Figure 3.3 Organizational units and focus of research
CEO
manager
manager
subunit
this is your
subunit
your
supervisor
manager
(this is you)
manager
employee
employee
manager
subunit
manager
employee
employee
3.3 Validity-feedback interviews
Central to this research is the study of management control systems in their entirety instead of
the study of single control instruments. The questionnaire should thus be able to map these
control systems. Moreover, it should do so for a whole range of different organizational units.
Because I extend prior research by explicitly studying control systems in their entirety and
deviate from it by focussing on a whole range of activities, two questions must be addressed:
can we actually grasp general strands of MCSs as wished-for using solely the information
obtained by the questionnaire? And, is this picture more or less complete, i.e. will it suffice
for our purposes of comparing configurations?
Such questions relate to the content validity of the survey instrument and can be partly
assessed using existing methods for pre-testing like discussing the contents with peers or
conducting interviews with respondents on clarity of questions. A more sophisticated method,
like the Three-Step Test-Interview (Hak, Van der Veer, and Jansen, 2004; Jansen and Hak,
2005), provides more detailed information and feedback. However, these methods generally
provide feedback on single questions, not on the content validity of the overall design. This is
especially true when the overall survey is extensive and complex, which complicates
maintaining an overview of all information.
39
Chapter three
I want to obtain information beyond that retrieved by existing methods and apply a new
powerful technique to assess the content validity of the survey as a whole: validity-feedback
interviews. The basic idea of this technique is simply to describe the object of study (in casu
the management control system) using solely the answers filled out on the questionnaire, and
to check the validity and the completeness by comparing the description to the theory, and
discussing it with the respondent.
In the complete process, firstly, I ask a potential respondent to fill out the questionnaire. Then,
purely based on his/her answers, I make a description of the MCS of his/her organizational
unit and the effectiveness thereof as well as the characteristics of the unit’s activities.
Numerical scores on the items are translated back into sentences. Consider, as an example, the
following question:
14) How much does your supervisor use subjective judgements to evaluate your
performance?
scale: not at all (1), very little (2), somewhat (3), quite a bit (4), very much (5)
The respondent circles 4. In the description this answer becomes: ”The unit supervisor uses
subjective judgements quite a bit to assess the manager’s performance”. This provides an
overview of information obtained when one can solely rely on the survey. During a follow-up
meeting with the manager, I pose three general questions to discuss the description:
1. Do you recognize your unit from the description?
2. Do you feel important aspects/characteristics are missing that (also) have an influence on
motivating employees and directing their behaviour?
3. Does the description hold any incorrect or false statements?
The interviews return two types of feedback: first of all the descriptions provide information
on whether we pick up on all aspects of interest. Moreover, the discussion with respondents
ascertains we can actually make valid claims about practice.
The validity-feedback interviews were held as part of the overall pre-test to be discussed next.
Appendix A holds one of the descriptions made of a management control system to illustrate
the type of information as well as the amount of detail the questionnaire generates. It is based
on the final version of the questionnaire as used for data collection.
40
Methodology
3.4 The pre-test of the questionnaire
As a next step in the development of the questionnaire, the basic set-up was put up to pretesting. As mentioned before, pre-tests help assess the quality of a questionnaire and are a
prerequisite for solid survey research.
One of the critical observations on survey research within accounting is the lack of pre-testing
(Van der Stede et al., 2005). Researchers tend to rely on measures simply because they have
been used before. However, often there is little known about measures except their use as
such. When used in a new study measures are usually modified in numerous ways. General
matters are different order of questions, their combination with other questions, and
adjustments in wording (however small). Even in the rare occasion that one can use the exact
same question or part of a questionnaire, different settings intrinsically change the situation.
Researchers cannot predict the impact of these changes without testing. Therefore, all
measures should be included in the pre-test, even when they have been used and tested before
(cf. Van der Stede et al., 2005, p671, footnote 12).
To illustrate this point, consider a result from my pre-test that well illustrates the type of
problems addressed and the kind of information obtained through pre-testing. Starting from a
study by Anderson (1988), I measure environmental unpredictability as one of the guises of
uncertainty. She uses four items that relate to instability associated with environmental
turbulences12. Respondents indicate on a seven-point scale whether the market for their
product line was simple or complex (1), stable or volatile (2), easy or difficult to monitor (3),
and certain or uncertain (4) (see Anderson, 1988, p616). In my questionnaire, I asked to
describe the business environment of the organizational unit over the same four dimensions.
Two managers of the same company who are faced with the same business judged their
business environment completely different. One indicated it was simple, stable, and certain,
while the other claimed the exact opposite. To explain this afterwards they compared their
environment with the situation a hospital patient on intensive care is in. One can interpret this
as uncertain and unstable, but when the patient’s situation is diagnosed ‘stable’ one can look
at it from a different perspective as well. Because of this result, the question was deleted from
the questionnaire for returning unreliable answers.
12
The original measure holds nine items. The other items are related to another type of uncertainty;
hazards of entering new markets. These items are not generally applicable to the variety of activities I
study.
41
Chapter three
The pre-test conducted in this research project existed of two phases. Both phases and the
results of the test will be described, but first subparagraph 3.4.1 examines the purposes of pretesting, and the subsequent paragraph discusses the pre-test method applied apart from the
validity-feedback interviews.
3.4.1 Purposes of pre-testing
Pre-testing a questionnaire encompasses a host of actions ranging from establishing face
validity when discussing the contents with peers, to conducting interviews with respondents
on clarity of questions. Purposes and methods of pre-tests vary accordingly. Here pre-testing
serves several purposes. Firstly, the quality of individual questions needs to be assessed.
Respondents must understand the questions. Wording might be ambiguous and answer
categories might be missing. Apart from concentrating on single items, the quality of the
overall design is assessed. Is there enough space for comments? Is it clear where to start and
when to skip a question? How long does it take respondents to complete the questionnaire?
These are general goals when pre-testing any survey. Addressing these issues will benefit
response rates and data validity because it enhances the likelihood that questions are
understood the way they are intended.
On a different level, but equally important, is assessment of content validity. In pursuit of this
pre-test goal the relevance of the questionnaire as a whole comes under scrutiny. I use two
methods to test the quality of my questionnaire. Apart from the validity-feedback interviews
already discussed, I use the Three-Step Test-Interview (Hak et al., 2004), which will be
introduced next.
3.4.2 The Three-Step Test-Interview
This study relies on the Three-Step Test-Interview (TSTI), a method especially designed for
pre-testing self-completion questionnaires (Hak et al., 2004)13. Aim of the TSTI is to observe
what happens when a respondent encounters a questionnaire as he would had he received it on
his own desk. Characteristic to the method is concurrent think-aloud, a form of cognitive
interviewing, which entails the process in which verbal information about responding to a
draft version of a questionnaire is being collected (cf. Beatty, 2004, p8).
13
I want to thank Tony Hak for discussing my first experiences with the TSTI with me and providing
helpful feedback and stimulating advice.
42
Methodology
The TSTI consists of three steps. The first involves the concurrent think-aloud. During this
step the interviewee provides insights into his thought process by verbalizing his thoughts
when reading and responding to the questionnaire’s items. The researcher merely acts as an
observer. This step returns detailed data on clarity of questions, on associations certain words
elicit, on whether the interviewee can provide an answer (at all), at what point problems occur
and many more. The second step involves completing this think-aloud data by filling gaps on
information still missing. An interviewee might stop to express what he thinks at some point
or might skip a question without giving the reason. Probing will return the required data at
this stage. The type of question asked is: “You shortly paused at question 21. What were you
thinking of?”14 The third and last step consists of a regular retrospective interview in which a
respondent can be asked to give his overall opinion on the questionnaire, or discuss its
contents and the relevance of single questions.
In their 2004 article Hak et al. provide a manual with practical guidelines for use of the TSTI.
This makes the method generally available for use. I closely followed the manual and opted
for field interviews instead of a laboratory setting. At the start of a meeting with each
participant of the pre-test, the think-aloud technique was explained and practiced. All
interviews were recorded.
There is an ongoing debate on both theoretical and empirical validity of cognitive interview
techniques. Can thought processes really be revealed and under which circumstances? And,
can the effectiveness of these interviews be proved? Willis (2005, Ch13) discusses these
issues in his evaluation of cognitive interview techniques. His overall judgement is positive
and he advocates their use. Beatty (2004) also reflects on the use of cognitive interviewing
and concludes that there is no consensus on the best way to conduct it. He also points out a
lack of practical guidelines, for example on how many interviews to conduct and whom to
talk to. Consequently a lot is left to the researcher.
Hak et al. (2004) provide a manual for use of the TSTI, but cannot solve these practical issues
either. How many rounds of testing to conduct, which people to interview, and when to stop
asking questions remain judgement calls. During its development, Hak and several other
researchers tested the TSTI in three pilot studies (see Hak et al. 2004). In a separate study the
productivity of the method is compared to an expert review in which the authors test a
questionnaire on alcohol consumption (Jansen and Hak, 2005). Results show that the TSTI
14
Notice the difference with this question: “why did you give this answer?” The difference lies in
asking about a respondent’s thoughts a moment ago when answering the question and asking what he is
thinking now, in retrospect (cf. Hak et al., 2004, p7).
43
Chapter three
produces data beyond that obtained using other methods like an expert review or interviews
based on probing.
3.4.3 Pre-test execution and results
The pre-test consisted of two consecutive interview phases. The first was held during October
and November of 2005 and the second during January and February of 2006. A total of
sixteen interviews have been conducted with managers of organizational units who are
potential respondents. Both men and women from profit and not-for-profit organizations
working at different hierarchical levels participated. Their units were production units,
Research and Development, Human Resources, project units, Finance and Accounting,
consultancy, recruitment, and Information Technology. While selecting pre-test participants, I
specifically tried to cover those organizational units/activities for which each of the
archetypes might be relevant. Each interview lasted one and a half hours. Throughout the
TSTI was used as an instrument for pre-testing. During the second pre-test phase this
technique was complemented with the validity-feedback interviews in a follow-up meeting
with each manager. During the pre-test trajectory attention shifted deliberately from a focus
on more general testing of clarity and completeness in the first phase to specific focus on
validity of information obtained in the last.
The first phase consisted of six interviews. Its purpose was a general one with central issues:
do respondents experience any difficulty in answering questions (as such) and can they
translate generic ideas on what management control is about to their own situation? After the
first pre-test some key changes have been made to the questionnaire. The most important one
is the addition of questions on activities of the unit. In the revised version managers are asked
to specify up to five activities of their unit first, and then choose one to focus on throughout.
Several reminders help managers to maintain this focus. This improvement enhances validity
because respondents will give information about controls surrounding one activity as opposed
to providing an average score for several activities. This, in turn, establishes a fit between the
theoretical level of analysis and the empirical one.
Moreover, returning to the working definition of an MCS, a structural problem became
apparent. The current version of the questionnaire was unable to grasp the interrelationships
between the four dimensions of control as defined in the theory. Gaining knowledge on these
relations is important for learning about control systems in their entirety as opposed to
studying separate instruments. Revision resulted in the deletion of about ten questions and at
the same time the introduction of about ten new ones. Other changes involved choosing
44
Methodology
different wording, adding an answer category, shortening questions, and adding more detail to
the specification of the hierarchical level.
The second pre-test stage involved meetings with five managers. Quality regarding clarity and
completeness of questions is no longer of primary interest, but assessment of content validity
of the questionnaire as a whole becomes the main concern. Will the questionnaire return
information about the MCS as intended? Therefore this stage consisted of two interviews with
each manager. During the first meeting I applied the TSTI again to assess the overall quality
of the questionnaire and particularly the quality of new and modified questions. During a
follow-up meeting, the validity-feedback interviews took place, discussing the description of
the management control system solely based on the answers filled out on the questionnaire.
Results of the validity-feedback interviews were quite encouraging. All managers recognized
their units even though described in generic terms. Only one manager indicated a motivational
aspect missing that in his opinion strongly guided employees in their daily behaviour. He
referred to an explicit focus within the company on corporate responsibility. In the
questionnaire this is only picked up in general when asking about the importance of
behavioural guidelines in guiding employee behaviour. Few aspects indicated as being
incorrect could be dealt with through minor revisions in wording.
Changes to the questionnaire were made after each interview. However, two managers last in
line responded to the same and final version. These last few interviews were held purely as a
confirmatory step.
3.5 Sampling and survey administration
In its final form the questionnaire contains 89 questions15. The pre-test revealed that managers
need about 50 minutes to complete it by themselves, up to one and a half hours when being
interviewed. Intended respondents are managers who head an organizational unit with at least
ten employees. Recall that a manager and all employees reporting (in)directly to him/her
comprise an organizational unit. This paragraph will firstly discuss the sampling strategy, and
then the survey administration.
15
Only part of the questionnaire is used for this study. The complete questionnaire contains questions
on control at the level of the organizational unit (control experienced by the manager) as well as on
control within each unit (control exercised by the manager). In this study I focus on the former type,
control of the organizational unit.
45
Chapter three
In setting up a sampling strategy, three requirements for the sample emerged. Overall the
sample should be reasonably large (at least 200 observations), it should represent enough
variety in activities, and it should hold enough observations of those situations for which
theory predicts the occurrence of each of the archetypes. A random sample is unlikely to
entail enough information to enable analysis of each archetype because some activities will be
carried out more frequently than others. Thus ideally, sampling is based on selection of
activities. As different organizational units are likely to engage in different activities, I have
defined sampling groups for different types of organizational units (see table 3.1). Dividing
observations over these groups should return enough variety in activities. Note that this
approach will not introduce selection bias because the activity’s characteristics are
explanatory variables in this study (King, Keohane, and Verba, 1994, p137).
Table 3.1 Sampling groups
I
divisions
II
finance & accounting
III
research & development
business units
production units
marketing & sales
human resources
internal auditing
shared service centres
treasury
project units
internal consultancy
ICT (maintenance)
ICT (system development)
As two basic ways to administer a survey, Brownell (1995, Ch3) indicates the written
questionnaire and the personal interview. However, with such an extensive questionnaire as
the one used in this study it is hard to get reasonable response rates. Moreover, in case of high
non-response chances of finding enough potential observations for the analysis of each
archetype are diminished. Therefore I am extremely grateful that students16 helped contact
respondents and collect data, which made it possible to administer the survey by interview. In
a meeting on site, the student interviewed the manager posing all questions directly from the
questionnaire. The manager has his own questionnaire, can read along, and fill in the answers
himself. Only after having completed the entire questionnaire, there was some time left for
discussion. The data from these discussions are not available for analysis, but served as an
additional learning opportunity for the students only. Meetings took approximately one and a
half hours.
There are several advantages of this approach: it provides the opportunity to correct
misunderstandings, forces respondents to give thought to each question before answering, and
16
Students took part in the course Management Accounting and Control Research, which is taught in
the part-time MSc programme in Controlling at Nyenrode Business University. The survey was
integrated into the course material. The students already work as controllers or hold a management
position.
46
Methodology
minimizes chances of skipping a question. Moreover, this approach did not only assure
minimum sample size, it also facilitated selection of respondents based on types of
organizational units thereby enhancing chances of finding variety in activities.
As the major disadvantage of the personal interview, Brownell (1995) justly states the risk of
interviewer bias. Although present to some extent in every interview, this caveat can be
mitigated – at least partly – by introducing clear protocols/instructions. Moreover, because 60
different students were involved in the interviews, chances of systematic bias are small. I
explicitly addressed these issues when briefing students. They were instructed to read aloud
each question exactly as formulated, and then pause to let the manager respond. Students were
also instructed on how to respond to requests for clarification of questions and words.
Managers could ask them to translate a word (from English to Dutch). This might introduce
bias if students choose slightly different wordings. Therefore, students and I went through all
questions together to clarify the meaning of each.
3.6 Description of the dataset
Data were collected two years in a row (April 2006 and April 2007)17. The first round of data
collection returned 134 questionnaires. The second round delivered another 124 cases. Data
collection in two shifts created the possibility to fine-tune the variety in activities.
Consequently, samples intentionally differ in the division of observations over types of
organizational units. Both samples combine into a single dataset. There is no indication of any
unintentional difference between them that might disturb the analyses.
Most of the questionnaire’s items contain semantic differential scales with five categories (see
Lattin, Carroll, and Green, 2003, p174). For instance, respondents choose from five possible
scores: very easy, quite easy, neither easy nor difficult, quite difficult, and very difficult. An
advantage of this type of scale is that respondents always get a reference point instead of just
a number. This is especially helpful when a questionnaire contains lots of questions with
differing answer categories. The questions related to the variables that measure effectiveness
of both the MCS and the unit have a seven point scale, which increases the likelihood of
finding enough variance for the statistical analyses. The data obtained are respondents’
perceptions measured on both nominal and ordinal scales. In the analyses, I treat ordinal data
17
I needed two shifts because this research project was integrated in a yearly course with
approximately 30 students participating each year.
47
Chapter three
as if they were interval data, since “the assumption of an interval scale is reasonable” (Lattin
et al., 2003, p8) for my measures.
Table 3.2 reports the descriptive statistics for the dataset, including hierarchical level, the
number of unit employees, and the number of employees who work on the main activity18.
The hierarchical level indicates how many steps away from the CEO the manager’s unit is,
and how many steps are left to the bottom level within the hierarchy.
Table 3.2 Descriptive Statistics
Variable
nr of steps to CEO
N
258
minimum
0
maximum
10
mean
2.22
std. deviation
1.39
nr of steps down the hierarchy
258
nr of unit employees
257
1
7
2.36
0.99
4
10800
188
nr of main activity employees
802
258
2
3400
88
ratio main activity empl/ unit empl
325
257
0.27
100
63.24
28.41
The dataset shows a vast variety in hierarchical level (from CEO to ten levels down the
hierarchy) and in the number of unit employees (from 4 to 10,800). Frequency tables reveal
that about 80% of the companies are profit companies, 20% are not-for-profit. Regarding
industry I find all sorts of industries represented. Financial services, government, and
banking/insurance make up the largest part. None of these sectors are dominant, however,
because their individual shares never exceed 11% of the total19.
Managers also indicated the type of unit they work in, for instance, a production unit or a
treasury department. Recall the goals from my sampling strategy (see 3.5): the dataset should
be large, represent a variety in activities, and hold enough observations to represent each of
the archetypes. The last goal will be assessed during the analyses, but descriptive statistics of
the sample give clues on the first two. Table 3.3 shows the variety in types of units within the
sample. Business units, production units, and units finance and accounting respectively form
the largest groups. Overall, a multiplicity of various activities is represented in the database.
18
In some cases the number of employees is very small. I have run the analyses (chapter five) both
with and without the smallest cases. Results are robust to these changes.
19
Note that there are multiple observations per company. 252 observations stem from 132 different
companies (6 were anonymous). This will not introduce bias because the level of analysis is the activity
of an organizational unit. Within companies differences between units exist, and within units multiple
activities. Moreover, if for instance a company-wide reward system would be installed, I should be able
to pick up performance differences for those units for which this is inappropriate and thus I should be
able to assess MCS design and effectiveness as intended. Also, in none of my (sub)samples a single
company dominates. The number of observations per company is simply too small.
48
Methodology
Table 3.3 Description of the dataset
type of unit
frequency
percentage
Business unit
48
18.6
Production
32
12.4
Finance and Accounting
31
12.0
Division
19
7.4
ICT
19
7.4
Marketing or sales
17
6.6
Project unit
17
6.6
Human Resources
14
5.4
Research and Development
11
4.3
Company as a whole
8
3.1
Facility management
5
1.9
Treasury
4
1.5
Legal department
3
1.2
Other*
20
11.6
Total
258
100.0
*This group is quite diverse. Examples of units within the group are purchasing, internal consulting,
quality management, and back office.
In the questionnaire, managers specify the activities of their unit and choose one as the main
activity to focus on throughout. Examples of activities specified by respondents are product
development, provision of technical support, project management, reporting, and policy
making20. The pre-test revealed the importance of asking to identify the activities, which
makes it possible to link an MCS to a specific activity and establish a fit between the
theoretical and empirical level of analysis. Only 32 out of 258 managers (12%) identified one
activity. This underlines the importance of creating the focus in the questionnaire, because
without it 226 managers would have provided average scores for several unknown activities.
Asking people to choose the main or most characteristic activity for their unit does not
automatically imply they choose the activity the largest number of employees work on.
Therefore I asked about the number of unit employees as well as about the number of
employees working on the main activity as specified. The ratio between employees working
on the main activity and total unit employees ranges from 0.27% to 100% with a mean of
63%. Of all managers, 67% choose an activity on which at least half their employees work.
20% indicate all employees (also) work on the main activity. Table 3.4 below gives an
overview.
20
There is a large variety in the qualitative descriptions of the activities. Therefore, translating these
into keywords and summarising those, is little informative when looking at the complete sample. There
are either too many categories or there is too little extra information over that already given here by the
division of cases over unit types. In chapter five I will again address the issue, compare subsamples,
and illustrate a number of differences in activities for individual unit types.
49
Chapter three
Table 3.4 Percentage of employees who work on the main activity
percentage employees
frequency
percent
cumulative percentage
0 – 10 %
10 – 50 %
9
76
3.5
29.6
3.5
33.1
50 – 95 %
120
46.7
79.8
95 – 100 %
52
20.2
100
Total
257
100
3.7 Concluding remarks
Because I extend prior research by explicitly studying control systems in their entirety and
deviate from it by focussing on a whole range of activities, confidence as to the quality of the
survey data must be given before running the analyses.
Survey research is widely applied yet frequently criticised. Some critique relates to problems
inherent to the method, which cannot be resolved: once the questionnaire leaves your desk it’s
out of your hands. Other methods also suffer from their own inherent limitations though.
However, most of the criticism on survey research within accounting relates to improper use
of the method and the resulting poor quality data. This critique can be confronted successfully
as I have shown in this chapter.
Attention has been paid to assuring a match between theory and operationalization in terms of
constructs and level of analysis. Two questions were central: can we actually grasp general
strands of MCSs as wished-for using solely the information obtained by the questionnaire?
And, is this picture more or less complete, i.e. will it suffice for our purposes of comparing
configurations?
The extensive pre-test helped identify lots of small and big problems, which have been
addressed to the benefit of the questionnaire’s quality. It seems safe to conclude that
respondents will understand the questions and can answer these. The questions, moreover,
inform us about critical aspects of the theory and can therefore potentially help answer the
research problem. From the validity-feedback interviews, we know that it is possible to pick
up control practices as experienced by managers and to be quite complete in doing so. The
description of a unit’s MCS illustrates this. This new technique proves a powerful method to
effectively obtain feedback on the content validity of questionnaires.
In sum, the questionnaire is able to map different aspects of an MCS and obtain a detailed yet
broad overview of how control takes form at the level of the organizational unit. Moreover, it
50
Methodology
makes possible identification and characterisation of activities of the unit as well as
assessment of perceived effectiveness of both the unit and the management control system.
As for the prevention of selection bias and non-response, survey administration by interview
enabled data collection for 258 organizational units with a vast variety in activities. Nonresponse therefore was not an issue.
51
Chapter four - Operationalizing the framework
The previous chapter presented an overview of the methodology, introducing the
questionnaire, mode of data collection, and a description of the dataset, without providing
details on the measurement of the variables. This chapter fills that void and describes the
operationalization of the methodological framework (figure 4.1). Separate paragraphs deal
with each of the highlighted steps. As in the prior chapter, of central concern is the
establishment of a match between the theoretical constructs and their measures.
The dataset under study belongs to a larger dataset, which holds information on both control
of organizational units (control experienced by the manager), and control within these
organizational units (control exercised by the manager)21. In this thesis the former is under
research. Firstly, in separate paragraphs I shed light on the mapping of management control
systems (4.1) as well as their effectiveness (4.2). Next, paragraph 4.3 describes the
measurement of the contingency variables and discusses formative measurement models.
Paragraph 4.4 deals with both the operationalization of the archetypes and misfit calculation.
Translating the theoretically described archetypes into measurable profiles on which the
measurement of misfit depends is an important step in testing configurational theories (see
Doty et al. 1993; Doty and Glick, 1994). This step carries a potential risk, because if the
profiles do not correspond to the theoretical description, the test of the theory might not be
valid (Doty and Glick, 1994). Describing exactly how this step was taken, I show how I have
minimized this risk. Finally, paragraph 4.5 presents several control variables and paragraph
4.6 closes this chapter with concluding remarks. The two methodological steps left
(determining the relevant archetype and linking misfit to effectiveness) are part of the
analyses and will be dealt with in the next chapter.
Figure 4.1 Methodological framework
define
define activity
determine
determine relevant
relevant
archetype
architype
compare MCS with
relevant
relevant archetype
archetype
map MCS
link misfi
misfitt to
to
link
effectiveness
effectiveness
measure
measure MCS
MCS
effectiveness
effectiveness
21
Data has been collected all at once by means of the questionnaire introduced in chapter three.
53
Chapter four
Appendix B gives an overview of the exact questions posed, along with notes and guidance
on their use. Apart from describing the measurement of the variables and introducing new
measurement instruments with a general applicability, some issues of broader importance will
be addressed in the next paragraphs. First of all, this chapter exemplifies one approach to
operationalize a configurational theory and to study control systems in their entirety as well as
the effectiveness thereof. Moreover, I suggest ways to improve general measurement of two
main TCE variables: uncertainty and asset specificity. To better grasp the essence of these
constructs, I build a formative indicator model (see 4.3).
4.1 Mapping management control systems
This paragraph presents the first step of the methodological framework: mapping
management control systems. Referring both to theory and prior research, relevant measures
and variables will be identified and described, and new measures will be introduced. Most
questions contain semantic differential scales with five categories (see Lattin et al., 2003,
p174). For instance, respondents choose from five possible scores: very easy, quite easy,
neither easy nor difficult, quite difficult, and very difficult. Several questions stem from the
work by Van de Ven and Ferry (1980). In their book ‘measuring and assessing organizations’
they develop an extensive survey instrument which enables them to map a large number of
organizational aspects at different hierarchical levels. Also noteworthy is the ‘handbook of
organizational measurement’ by Price (1997). His overview of frequently studied variables
within the organization literature and their measures along with an assessment of their quality
was extremely helpful.
Chapter two presented four control dimensions that together describe management control
systems: allocation of decision rights (1), use of rules, standards, and regulations (2),
performance evaluation (3), and the reward structure (4). To grasp control systems in their
entirety and enable comparison to the archetypes, all dimensions must be represented in the
dataset. I use a total of 20 variables to cover all four. Special attention has been paid to the
identification of interrelationships between the dimensions. For instance, I do not only ask
about whether performance targets are in use as such, but also about the importance of these
targets for performance evaluation and rewarding. The clarification of each variable follows.
Like before, I discuss control from the perspective of a supervisor who exerts control on a
manager. Table 4.1 gives an overview of how the 20 variables cover the different dimensions
of control and how each variable relates to the definition of the archetypes as provided
previously in chapter two (2.2.3).
54
55
Source: Speklé, 2004; extended
Performance dependent
bonuses
Reward &
incentive
structure
Standardization
Performance dependent
bonuses
Monitoring focused on
target achievement;
performance
assessment relative to
targets
Predefined performance
targets of administrative
origins
Market-related outcome
requirements; external
performance
benchmarks
Structure
Performance
assessment relative to
‘the market’
Decentralized with
clearly defined areas of
responsibility and
accountability
Relative autonomy;
involvement higher level
management limited as
long as performance is
satisfactory
Monitoring and
performance
evaluation
Machine Control
(Result Oriented)
Arm’s Length Control
No direct link between
performance and
rewards
Focused on compliance;
observance of
interdictions; external
audits
Emphasis on threat of
punishment of rulebreaking behaviour; tiein through ‘hostages’
Career prospects
dependent on long term
past performance; peer
pressure
Proscriptive codes of
conduct; boundary
systems; emphasis on
behaviour to be avoided
Relative autonomy within
defined boundaries
Boundary Control
Based on emerging
standards; subjectively
assessed contributions
to long term
organizational
performance
No ex ante standards
and targets; ‘do your
best’; emerging
standards
Standardization of
behaviour; detailed
rules, norms, and
instructions
Monitoring and
supervision to ensure
compliance to norms
and standards
Relatively flat hierarchy;
fluid and permeable
matrix-like project
structures; vague
responsibilities
Exploratory Control
Well-defined tasks; strict
hierarchy; limited room
for discretionary
behaviour
Machine Control
(Action Oriented)
Table 4.1 Variables covering the dimensions of control related to the control archetypes
percentage variable reward;
importance in rewarding and
for career of:
target achievement
compliance with rules
long term achievements
professional skills
-
importance in performance
evaluation of:
target achievement
compliance with rules
long term achievements
subjective judgements
use of financial targets;
use of non-financial targets;
benchmarking;
boundaries;
role of the budget;
emergent targets
work autonomy;
amount of work related
discussion
variables that map the MCS
Operationalizing the framework
Chapter four
The first dimension of control (allocation of decision rights) is picked up by asking about the
coordination of work and assessing the relative autonomy of the manager. Two variables
represent this dimension: the amount of work related discussion, and the level of work
autonomy. The amount of discussion indicates whether or not the manager works a lot on his
own and whether the work is organized such that it requires mutual coordination. This
variable is measured with a single question developed by Van de Ven and Ferry (1980). I ask
about the frequency of discussions related to the coordination of work, between the manager
and other managers or supervisors. To measure work autonomy, managers indicate their
influence in several decisions (see table 4.2). The first four items stem from Van de Ven and
Ferry (1980). I extended their measure with two items to learn also about work instructions
and behavioural guidelines. The average of the six items forms the score on work autonomy.
Factor analysis22 supports their combination into a single scale, and the reliability of the scale
is satisfactory (Cronbach’s alpha = 0.736).
Table 4.2 Items for work autonomy (Q42)
Cronbach’s = 0.736
component loadings
a
deciding what work or tasks are to be performed in your unit
b
setting standards/targets for subunit performance
0.743
c
deciding upon criteria for performance appraisal of employees
0.694
d
designing the reward system
0.614
e
deciding upon standard operating procedures/work instructions
0.703
f
deciding upon other behavioural guidelines
Source: items a-d; Van de Ven and Ferry, 1980; modified
0.542
0.672
A number of variables cover the second dimension of control and indicate the presence of
standards, rules, and regulations. To cover those aspects of control that differ between the
archetypes, the variables should identify use of performance targets, distinguish market
related targets from administrative ones, and pick up use of emergent targets and boundary
systems. The amount of rules and regulations, like work instructions and behavioural
guidelines, is not measured as such. I study their importance only in light of performance
evaluation and rewarding.
First of all, managers indicate the extent to which several performance targets are in use for
their jobs (see table 4.3). The distinction between the different types stems from Bouwens and
Van Lent (2004, p15). Mapping management control systems at different hierarchical levels,
different target types might be relevant. I am primarily interested in the intensity of target use.
22
Throughout I use principal components factor analysis (PCA) to assess whether items load on a
single underlying construct. For my sample size, a minimum component loading of 0.5 is considered
significant (Hair, Anderson, Tatham, and Black, 1998, p111/112).
56
Operationalizing the framework
Therefore, as an indicator for the intensity of financial target use, I take the highest score out
of items a-e. Item f asks directly about non-financial quantitative targets and will be used as
such23. Moreover, to learn about benchmarks, managers indicate to what extent two types of
performance benchmarks are in use (see table 4.3). Again, the highest of both forms the score
on benchmarking.
Table 4.3 Performance targets (Q7) and benchmarks (Q8)
types of performance targets
a
stock price related targets
d
b
return targets
e
revenue targets
cost targets
c
profit targets
f
non-financial quantitative targets
types of benchmarks
a
market benchmarks or benchmarks based on performance of external peer groups
b benchmarks based on performance of internal peer groups or the performance of other subunits
Source: types of performance targets based on Bouwens and Van Lent (2004, p15)
I capture the presence of boundary systems by asking about the consequences of noncompliance with prohibitive behavioural guidelines. Managers indicate their agreement with
the following statement: “Violating subunit-specific prohibitive guidelines will always have
serious consequences for my career.” (Q18). It takes two introductory questions to create a
focus on boundary systems. Firstly, one question helps to distinguish prohibitive guidelines
from general behavioural guidelines (Q16). The distinction is important because only those
guidelines that specify what not to do or what to avoid act as boundary systems. A second
question is asked to distinguish general prohibitive guidelines that hold for all company
employees from subunit-specific guidelines only relevant to the employees and the manager
of the unit (Q17). This is important to separate those guidelines that relate to the
organizational unit and its activities, from those that are unspecific. I want to learn about, for
instance, authorisation levels and procedures instead of about general ethical norms or rules
about private use of the internet. This way I conform to the level of analysis and map the
MCS surrounding the activity of an organizational unit. Recoding the scores of question Q18
returns an ordinal scale with four values. Each value indicates an increasing importance of
boundary controls. The lowest value implies a boundary system is absent.
Another control instrument that I relate to ‘standards, rules, and regulations’ is the budget. A
budget can have different roles. Respondents indicate the order of importance of three such
roles (see table 4.4). Frequencies show how often a particular role was deemed the most
23
Question 7 has one more item, which is not used in the analysis. That item measures use of
qualitative targets and represents a rest category.
57
Chapter four
important one. Finally, a single question records the use of emergent targets. Managers
choose from two statements the one that best describes how their performance targets come
about (see table 4.4).
Table 4.4 Roles of the budget (Q11) and emergent targets (Q9)
a
role of the budget
the budget serves as a performance target to be met
frequency
118
percentage
47
b
the budget gives guidance about the way to go
74
29.5
c
the budget sets limits to what can be done
59
23.5
251
100
total
emergent targets
performance targets are set at the beginning of a period and
187
72.8
will not change much under normal circumstances
b
performance targets come about steadily over time/evolve
70
27.2
during a period
total
257
100
Note: both are nominal variables, N = 258, there are 6 missing values, in two cases there was no budget
a
All in all, eight variables cover two dimensions of control: allocation of decision rights and
use of standards, rules, and regulations. Next, I describe the variables that cover performance
evaluation, and the reward and incentive structure, as well as the interrelationships between
the dimensions. I use groups of similar questions.
Regarding performance evaluation, the questionnaire focuses on several aspects: skills,
performance targets (including the budget), and compliance with rules/behavioural guidelines
(see table 4.5). For each aspect managers indicate its importance to both positive (Q12) and
negative (Q13) appraisal. In my sample, managers generally do not distinguish between the
two and therefore I can combine their scores. Both questions are new, but the ideas about the
set-up stem from studies by Bouwens and Van Lent (2004), Verbeeten (2005), and Van de
Ven and Ferry (1980).
Table 4.5 Items for performance evaluation (Q12 and Q13)
a
social skills
b
professional skills
c
achievement of performance targets
d
budget versus actuals
= 0.820
e
compliance with standard operating procedures/work instructions
= 0.823
f
compliance with other behavioural guidelines
Source: inspired by Bouwens and Van Lent (2004), Verbeeten (2005), and Van de Ven and Ferry (1980)
58
Operationalizing the framework
I form two variables: the importance of target achievement including the budget (the average
of items c and d), and the importance of compliance with work instructions/behavioural
guidelines (the average of items e and f). Factor analysis and reliability analysis confirm the
appropriateness of combining these items into averaged summated scales (Cronbach’s alphas
are 0.820 and 0.823 respectively). Note that these questions do not only provide information
on performance evaluation as such, but that they enable the study of interrelationships
between control dimensions. I have already measured use of performance targets as such, and
now their role in performance evaluation becomes clear as well.
Regarding performance evaluation I do not use the items related to skills. From a theoretical
point of view professional skills are only relevant for performance evaluation in the long run
or for career perspectives. Social skills do not play any role in distinguishing between the
archetypes. During the pre-test, managers indicated that skills might be (or should be)
important for long run performance evaluation. Thus, for completeness and consistency I ask
about the role of social and professional skills in all questions related to performance
evaluation, rewarding, and career perspectives. However, I only focus on items that are
closely related to the theory, in casu achievement of performance targets/the budget, and
compliance.
Two more variables complete the information about performance evaluation. Managers
indicate how much their supervisor takes into account long term performance of the unit
(Q15a) and/or the company (Q15b). The scores on items a and b are averaged (Cronbach’s
alpha = 0.772). Moreover, one question (Q14) picks up the use of the supervisor’s subjective
judgements in performance appraisal.
Regarding the reward structure, firstly, managers indicate the size of their variable financial
reward (Q21). The original five answer categories are recoded (see table 4.6).
Table 4.6 Financial variable reward (Q21)
frequency
percentage
a
no variable financial reward
size of variable reward
58
22.6
b
1-15%
103
40
c
more than 15%
96
37.4
total
257
100
Note: other answer categories referred to larger rewards, but these are rare in my sample. 12 managers
indicated that 46-60% of their reward is variable, 7 people indicated that this is more than 60%. There is
one missing value.
59
Chapter four
Using the same aspects that might play a role in performance evaluation, one question (Q22)
picks up the importance of several aspects for obtaining a variable financial reward (see table
4.7). Finally, the same question occurs once more (Q23), but now focus is on career
prospects. Managers indicate the importance of each aspect for their future career. Again,
factor analysis and reliability analysis indicate the appropriateness of combining compliance
with work instructions and behavioural guidelines into a single variable for both questions
(Cronbach’s alphas are 0.861 and 0.803 respectively). The achievement of performance
targets and compliance with the budget, however, cannot be combined and form separate
variables. Moreover, in light of career perspectives, the importance of professional skills is
taken in as a separate variable as well (Q23b).
Table 4.7 Items for obtaining a variable financial reward and career perspectives (Q22
and Q23)
a
social skills
b
professional skills
c
achievement of performance targets
d
budget versus actuals
e
compliance with standard operating procedures/work instructions
items used
separately
f
compliance with other behavioural guidelines
Source: idem questions 12 and 13 (table 4.5)
= 0.861 (Q22)
= 0.803 (Q23)
Together these 20 variables map management control systems over four dimensions
considering their interrelationships. They provide a comprehensive yet detailed picture of
control. Moreover, the overall measurement instrument is generally applicable to map MCSs
of all kinds of organizational units. Similarly important, mapped in this way, the MCSs can be
compared to the archetypes.
The individual variables relate directly to the descriptions of the archetypes, but they also
represent a combination of elements of control that are frequently studied. Traditional
controls, like standard operating procedures, financial targets, and budgets, are represented.
Moreover, I pick up the contrast between reliance on output controls versus action controls
(Merchant, 1982) by assessing the importance of target achievement as well as compliance
with rules in performance evaluation and rewarding. Finally, a number of variables represent
the more interpersonal and subjective aspects of control, for instance the amount of
discussion, use of non-financial targets, use of subjective judgements in performance
evaluation, and the importance of professional skills for the career. Table 4.8 gives an
overview of all variables that map an MCS and provides descriptive statistics. Appendix B
provides the details on each question.
60
Operationalizing the framework
Table 4.8 Descriptive statistics for all MCS variables
1
work autonomy
2
amount of discussion
3
use of financial targets
4
use of non-financial
targets
questions
Q42
N
257
min
1.67
max
5.00
mean
3.80
std.
deviation
0.66
Cronbach’s
alpha
0.736
Q56c
Q7a-e
(highest)
254
1
5
2.93
1.10
-
258
1
5
4.29
0.90
-
Q7f
258
1
5
3.06
1.35
-
257
1
5
3.31
1.27
-
258
0
3
0.81
1.11
-
5
benchmarking
6
boundaries
Q8a,b
(highest)
Q18
7
role of budget
Q11
253
0
3
nominal variable
8
9
emergent targets
percentage variable
reward
Q9
Q21
257
257
0
1
1
3
nominal variable
nominal variable
255
1.50
5.00
3.80
0.77
0.820
255
1.00
5.00
3.36
0.87
0.823
importance in
performance evaluation:
10
target achievement
11
compliance with rules
12
long term achievements
Q12c,d;
Q13c,d
Q12e,f;
Q13e,f
Q15a,b
255
1.00
5.00
3.48
0.97
0.772
13
subjective judgements
Q14
255
1
5
3.39
0.84
-
-
importance in rewarding:
14
target achievement
Q22c
257
1
5
3.63
1.65
15
budget versus actuals
Q22d
257
1
5
3.16
1.64
-
16
compliance with rules
Q22e,f
257
1.00
5.00
2.16
1.25
0.861
-
importance for career:
17
target achievement
Q23c
256
1
5
3.99
0.82
18
budget versus actuals
Q23d
256
1
5
3.38
1.06
-
19
compliance with rules
Q23e,f
256
1.00
5.00
3.19
0.95
0.803
20
professional skills
Q23b
256
1
5
4.29
0.70
-
4.2 MCS Effectiveness
To measure the effectiveness of a management control system, I have developed a new
measurement instrument. Reviewing literature, only one study emerged that measures
effectiveness of control directly: Ferreira and Otley (2005) develop a measure of the
perceived effectiveness of the MCS at the company level. They focus on the benefits an MCS
potentially offers and divide those over three dimensions: quality of generated information
(accuracy, for instance) and cost-effectiveness of the system (1), contribution to overall firm
performance (2), and satisfaction with meeting information requirements (3). The role of
information generation by the control system dominates, whereas other purposes of control,
61
Chapter four
like providing incentives, are lacking. I build from their ideas, but assess more aspects of
control effectiveness.
Organizational objectives form the accepted frame of reference in effectiveness literature
(Hitt and Middlemist, 1979). Starting from this basic idea, I capture effectiveness of the
control system directly by relating it to the achievement of control goals. In chapter two,
several goals of control have been distinguished (2.1.1). To summarise, the control system
enables knowledge capturing, supports coordination, and provides incentives through
rewarding and punishment (cf. Speklé, 2004, p7). To measure perceived effectiveness,
managers rate the performance of their MCSs on 11 aspects, each representing an underlying
goal of the MCS (see table 4.9). A seven point scale is used that runs from ‘very poor’ to
‘very good’. The item ‘supporting decision making’ represents knowledge capturing. The
items guiding behaviour, coordination, and monitoring belong to the goal of coordination.
Items concerning motivation, perceived fairness, performance evaluation, and rewards relate
to the goal of providing incentives. Finally, following Ferreira and Otley (2005), one item that
captures the contribution of the control system to the performance of the organizational unit
was added. This is considered to be an ultimate goal of the control system.
The scale originally contained four more items. These items related to the amount of work
autonomy managers and employees have, to the use of financial rewards (as opposed to nonfinancial rewards), and to the use of behavioural guidelines. Factor analysis revealed that
these do not fit the same underlying construct as the other items and therefore they are no
longer part of the scale. However, enough items are left to assess the effectiveness of the
management control system.
Table 4.9 Items for effectiveness of the MCS (Q85 and Q86)
Cronbach’s = 0.891
component loadings
a
Supporting managers in decision making
0.729
b
Guiding employees’ behaviour
0.648
0.690
c
Guiding managers’ behaviour
d
Coordination of work
0.622
e
Monitoring
0.624
f
Motivating employees
0.717
g
Motivating managers
0.769
h
Perceived fairness
0.679
i
Performance evaluation
0.681
j
Use of non-financial rewards
0.629
k
Contribution to overall subunit performance
Source: item k: Ferreira and Otley, 2005
62
0.751
Operationalizing the framework
The 11 items listed in the table load on a single factor and taking their average score returns a
summated scale (‘MCS effectiveness’). Cronbach’s alpha for this construct is high (0.891). In
cases where either a manager replied ‘not applicable’, or when a value is missing, the average
of the items left is taken as the score for MCS effectiveness. Because the sample entails a
diverse set of organizational units, items are not expected to be all relevant for each case.
Therefore only those items that are relevant should stay on. Following this procedure, the
smallest number of items averaged is six24. People indicated poor as well as good
performance with a mean score of 4.5 and sufficient variance (table 4.10).
Table 4.10 Descriptive statistics for MCS effectiveness
scale 1-7
MCS effectiveness
questions
Q85/86
min
2.18
max
6.55
N
258
mean
4.50
std. deviation
0.79
Cronbach’s
alpha
0.891
The focus on attainment of control goals to assess control effectiveness has two advantages:
(1) a broad approach to assessing management control systems is taken on covering different
purposes of control, and (2) the approach is suitable for all kinds of organizational units
(production, R&D, profit, not-for-profit, et cetera). An apparent disadvantage of this approach
is the inevitability of ending up measuring perceived effectiveness. Some argue that selfassessment of performance is inherently poor. Ittner and Larcker (2001), for instance, are
sceptical about the use of perceived measures and suggest backing them up with ‘hard’ data.
However, views on use of subjective measures differ. Van der Stede et al. (2005, p675)
conclude that “subjective indicators should not be viewed as poor indicators of performance
by virtue of being subjective”. Both have their merits and their use depends on the specific
objectives and settings of the study. In my case, it is impossible to work with hard data, since
I am comparing such a large variety of organizational units throughout the hierarchy. The
only way to compare effectiveness of control among them is to ask the manager directly for
an assessment. This approach fits my research objectives well, since I study the relative
effectiveness of various MCSs.
4.3 Characteristics of the activities
The previous paragraphs accounted for the first two steps of the methodological framework:
mapping the MCS and assessing its effectiveness. The next step involves characterizing the
activities. Activities obtain scores on the three contingency variables uncertainty, asset
specificity, and ex-post information asymmetry. These scores subsequently help determine
24
One case only was left with six items. Its score on MCS effectiveness is not particularly high or low.
63
Chapter four
which archetype is relevant for each case according to the theory. To distinguish between the
action oriented and result oriented variants of machine control, measurability of outputs is
taken on as a fourth contingency variable.
Two issues deserve our attention. In the first place, TCE-based research has generated lots of
measures for both uncertainty and asset specificity, yet their general applicability is limited.
Most TCE research studies homogenous samples and tailors measurement of variables to the
specific situation (see 3.2.1). Therefore there are no universally applicable measures
available. Moreover, researchers tend to focus on single aspects of uncertainty and asset
specificity, typically human asset specificity and unpredictability (Rindfleisch and Heide,
1997; Boerner and Macher, 2001). However, both asset specificity and uncertainty are broad
constructs that can take on multiple guises. The diversity in organizational units studied here
necessitates measurement of these different guises, because their relevance likely differs
among the units. In some units, human asset specificity might dominate, whereas in other
units physical asset specificity determines governance choice. Thus, measures need to be
broad and generally applicable. This approach also ascertains a better fit with the theory.
To combine different guises of uncertainty and asset specificity in a single measure, I set up
measurement models with formative indicators. Regular averaged summated scales are
inadequate for this approach (more on this below). In recent studies several authors have
underlined the importance of distinguishing reflective from formative indicator constructs and
they showed the implications of making the wrong choice (e.g. Diamantopoulos and
Winklhofer, 2001; Jarvis, MacKenzie, and Podsakoff, 2003; Diamantopoulos and Siguaw,
2006). Together those studies illustrate how misspecification of the relationship between a
construct and its indicators can have serious consequences both in terms of construct validity
(we are basically measuring and thus studying different things) and in parameter estimation
relating the constructs to other variables, thus emphasizing the dangers of drawing false
conclusions. Bisbe et al. (in press) conclude that in empirical research into management
accounting and control systems the issue is rarely addressed and that it is common practice to
rely uncritically on reflective models. The next paragraphs provide details on the
measurement of the contingency variables.
4.3.1 Uncertainty
In chapter two, uncertainty was defined as a lack of ex-ante knowledge of the activity, which
results in a diminished capability to program activities (2.2.2). This knowledge concerns both
processes (how best to carry out an activity) and outcomes (what results to expect).
64
Operationalizing the framework
Uncertainty arises from two main sources: unpredictability of circumstances surrounding the
activity (environmental uncertainty) and lack of possibilities to assess performance afterwards
(behavioural uncertainty) (Williamson, 1985). From several review articles (for instance,
David and Han, 2004 and Rindfleisch and Heide, 1997), it becomes clear that researchers
usually focus on one of these or take them on as separate variables. To measure uncertainty as
one broad construct, I focus on three variables. To capture environmental uncertainty, or more
generally the lack of ex-ante knowledge about processes, I measure unpredictability of the
amount and kind of work. Apart from this first variable, I assess the inability to measure
outputs, and goal ambiguity. Those two variables relate to behavioural uncertainty, or more
generally, they indicate the lack of knowledge about the outcomes. These three are formative
indicators, which together define in broad terms what uncertainty is.
Formative indicator constructs are inherently different from reflective ones and thus require
different treatment (Bollen and Lennox, 1991). Theory specifies which formative indicators
relate to a construct. However, the items are not effects of the construct, but rather cause or
define it. For instance, an increase in the level of uncertainty does not imply more goal
ambiguity or less measurability of outputs. To the contrary, goal ambiguity leads to or creates
uncertainty. Items used for formative constructs together define the construct, but they do not
have to correlate (Bollen and Lennox, 1991). Consequently, if one would decide to leave one
of the items out, that would alter the construct and thus the definition of uncertainty.
Calculating a summated scale is inappropriate for this type of constructs, as are standard
reliability and validity analyses based on Classical Test Theory (Bisbe et al., in press).
Measurement of a construct with formative indicators involves the creation of an index
(Diamantopoulos and Winklhofer, 2001). Basically, the index is a linear combination of the
indicators. However, running an ordinary regression analysis is impossible because the
construct is a latent construct. One way to solve this is to estimate a MIMIC model (Multiple
Indicators of MultIple Causes; Jöreskog and Goldberger, 1975). Diamantopoulos and
Winklhofer (2001) mention use of this model as one of the solutions in dealing with formative
indicators. Jarvis, Mackenzie, and Podsakoff (2003) also advocate the use of this model as the
best way to handle formative scale construction25. Figure 4.2 shows the general model.
25
See Diamantopoulos and Winklhofer (2001) for other possible strategies in handling formative
indicator constructs.
65
Chapter four
Figure 4.2 General form of a MIMIC model
reflective
indicator
reflective
indicator
construct of
interest
1
formative
indicator
3
2
formative
indicator
formative
indicator
Source: Diamantopoulos and Winklhofer, 2001, p272
A MIMIC model incorporates both formative and reflective indicators. The formative
indicators are combined through a multiple regression. The gamma’s, , indicate the
regression weights of each on the construct. Their linear combination enables the calculation
of the index. However, only incorporating formative indicators and the latent construct into
the model leaves it underidentified: more than one possible solution exists. To solve for this
problem, two reflective indicators are added to the model. Reflective indicators are
manifestations of the construct and thus caused by it (Bollen and Lennox, 1991). Thinking
about uncertainty as the construct of interest, this construct complicates the assessment of
what has been achieved as well as the assessment of the output quality. Both are reflective
indicators of uncertainty. Reflective indicators are expected to correlate.
How to interpret the overall model? “If all (indicators) are conceptually appropriate measures
of a single construct” (Jarvis et al., 2003, p214), one interpretation of the model is the
measurement of a single construct with both formative and reflective indicators.
To measure uncertainty I estimate a MIMIC model with three formative indicators
(unpredictability, the inability to measure outputs, and goal ambiguity), and two reflective
ones (difficulty to assess the work, and the difficulty to assess the output quality). Figure 4.3
shows the general structure. The outcome will be an index that combines all indicators to
obtain a score on uncertainty. Before calculating the index, however, I will firstly introduce
the indicators.
66
Operationalizing the framework
Figure 4.3 Indicators for uncertainty: general model form
unpredictability
diff assess work
not measure outputs
uncertainty
assess output quality
goal ambiguity
To measure unpredictability I combine three items (see table 4.11). The last two stem from
Van de Ven and Ferry (1980). Unpredictability is the sum of these three variables. Thus, I
treat this variable also as a formative construct. The combination of items defines
unpredictability, the items measure different aspects of the construct, and they do not have to
correlate. Therefore, Cronbach’s alpha cannot inform us about the reliability of the construct
and is not calculated.
Table 4.11 Items for unpredictability (Q37 and Q38)
a
unpredictable variation in the amount of work
b
number of exceptions that arise in the unit
c
differences in day-to-day situations
Source: items b and c: Van de Ven and Ferry, 1980, Q6 (p433), Q3 (p432)
items are
summed
Next, I measure goal ambiguity by combining several items that stem from a study by Rainey
(1983) (see table 4.12). I extend the original measure with one more item asking about the
specificity of goals.
Table 4.12 Items for goal ambiguity (Q31-Q34)
a
clarity of goals
b
specificity of goals
c
clarity of goals to outsiders
= 0.654
d
goals known to insiders (reverse score)
Source: items a,c,d: Rainey, 1983
The items are reflective indicators of goal ambiguity, which I thus measure with their average
score. The items load on a single factor and their Cronbach’s alpha is acceptable (0.654).
Finally, one question assesses how much of the outputs can be measured objectively and
expressed in a number (Q82). Its reverse score forms the third formative indicator of
uncertainty: the inability to measure outputs.
67
Chapter four
Unpredictability, goal ambiguity, and the inability to measure outputs form the formative
indicators of the MIMIC model, together defining uncertainty. Uncertainty in turn causes
difficulty in assessing the outputs. Two questions measure this aspect. I pick up the difficulty
the supervisor experiences in assessing the work of the manager and his/her employees (Q44).
Moreover, I ask about the difficulty in reaching agreement with superiors when appraising the
quality of unit performance (Q83). Both variables are direct consequences of uncertainty and
thus the reflective indicators in my MIMIC model. Table 4.13 gives descriptive statistics on
all variables related to uncertainty.
Table 4.13 Descriptive statistics for all uncertainty indicators
1
unpredictability
questions
Q37a,b, RQ38*
N
256
min
5
max
14
2
3
mean
9.65
goal ambiguity
RQ31-34**
258
1.00 4.00
2.00
inability to measure
RQ82
258
1
5
2.28
outputs
4 difficulty to assess
Q44
256
1
5
2.59
work
5 agreement on output Q83
257
1
5
2.49
quality
Note: the abbreviation RQ implies the reverse score for this question is used.
*the sum of these items is taken
**the average of these items is taken
std.
deviation
1.92
indicator
type
formative
0.62
1.09
formative
formative
0.95
reflective
0.84
reflective
Figure 4.4 shows how the variables that measure uncertainty combine into the MIMIC
model 26. The formative indicators give rise to the latent construct uncertainty, which in turn
causes the reflective indicators.
Figure 4.4 MIMIC model for uncertainty
unpredictability
.10
.19
not measure outputs
.59
.50
uncertainty
.35
.39
.25
2
R = .71
.17
e1
diff assess work
.12
.35
assess output quality
e2
goal ambiguity
e3
In AMOS 7, maximum likelihood estimation (MLE) renders a significant model (N = 254,
Chi-square = 3.241, p = 0.198) with a good model fit (RMR = 0.017, RMSEA = 0.05,
26
When I incorporate the separate questions directly into the model instead of the constructs (like goal
ambiguity), the model is distorted and overall model fit insignificant. Taking in the constructs is less
precise measurement, but prevents these problems.
68
Operationalizing the framework
NFIDelta1 = 0.975, CFI = 0.99). All coefficients are significant at the 10% level.
Unpredictability gets the highest p-value of 0.086. Together the three formative indicators
account for approximately 71% of the variance in uncertainty. Violations of multivariate
normality are minor. There are no indications for multicollinearity and outliers do not pose
any problems either. As suggested by Jarvis and colleagues (2003), I interpret this MIMIC
model as a single construct measured with both reflective and formative indicators. Therefore
it makes sense to combine the indicators and form a single measure for uncertainty. AMOS 7
returns factor loadings for every variable. I combine these in equation 4.1 to obtain a score for
uncertainty. As a final step these uncertainty scores are rescaled to a range of 1 to 5. This
simplifies the interpretation of scores on different variables throughout this study. Table 4.14
shows the descriptive statistics for uncertainty.
Equation 4.1 Calculating uncertainty
uncertainty = 0.038 * unpredictability + 0.234 * goal ambiguity + 0.227 * not measure outputs + 0.059 *
assess output quality + 0.087 * diff assess work
Table 4.14 Descriptive statistics for uncertainty
scale 1-5
questions
N
min
max
mean
std. deviation
uncertainty
Q37a,b; RQ38; RQ31-34;
RQ82; Q44; Q83
254
1.28
4.05
2.39
0.57
4.3.2 Asset specificity
Everything that I have mentioned while discussing uncertainty goes for asset specificity as
well. This too is a construct that I measure with formative indicators and once more use of a
MIMIC model to create an index is preferred. However, measurement problems for the
reflective indicators distort the MIMIC model and I am left with an inferior solution: the
construction of a summated scale.
In chapter two, asset specificity was defined as the opportunity costs of investments made to
support the activity (2.2.2). Multiple types of asset specificity were distinguished, but I focus
on those three that seem able to grasp the essence of the construct in a general way and might
be relevant for all kinds of organizational units: physical asset specificity, human asset
specificity, and interdependencies. These variables are formative indicators that cause or
create asset specificity and together define it. Causality runs from the items to the construct.
An increase in human asset specificity creates more asset specificity. The opposite is not
necessarily true: an increase in asset specificity does not increase human asset specificity.
Figure 4.5 gives an overview of all variables related to asset specificity. Since I intended to
69
Chapter four
combine them into a MIMIC model, both formative and reflective indicators are identified.
An introduction to the variables follows.
Figure 4.5 Indicators for asset specificity: general model form
human asset specificity
obtain elsewhere
physical asset specificity
asset
specificity
insourcing
interdependencies
Regarding physical asset specificity, I distinguish two types (see table 4.15). I use a modified
measure developed by Coles and Hesterly (1998) to assess the uniqueness of equipment. I
have added a question on uniqueness of systems. Managers specifically compare their unit to
other companies’ units that handle similar activities. The two items are summed to measure
physical asset specificity. Thus, again, I treat this variable as a formative construct because
either one of both types might be relevant for an organizational unit and determine the amount
of physical asset specificity. They do not have to correlate.
Table 4.15 Physical asset specificity (Q29 and Q30)
a
uniqueness of equipment (e.g. machinery, tools, warehousing)
b
uniqueness of systems (e.g. software, communication systems)
Source: item a: Coles and Hesterly, 1998; modified
items are
summed
Three questions grasp human asset specificity (see table 4.16). The first two items stem from
a study by Anderson and Schmittlein (1984). I have added a question on the uniqueness of
employees’ skills. Managers compare their employees to employees of other companies who
work on similar activities. All three are reflective indicators of human asset specificity, which
I measure with their average score. The items load on a single factor and their Cronbach’s
alpha is marginally acceptable (0.597).
Table 4.16 Human asset specificity (Q49/50/52)
a
difficulty a new employee experiences in learning the ins and outs
number of weeks of training a new employee needs who has experience in
b
a similar profession.
c
the uniqueness of unit employees’ skills
Source: items a and b: Anderson and Schmittlein, 1984; modified
Note: for item b the natural log is used throughout
70
= 0.597
Operationalizing the framework
Thirdly, I measure interdependencies between organizational units as a formative indicator of
asset specificity. I use a measure by Keating (1997) that consists of two parts. First of all, the
impact a respondent’s unit has on work carried out in other organizational units is assessed
(Q35). The next item relates to the impact the respondent’s unit experiences from other units’
activities (Q36). Summing the items returns a score on interdependencies.
Together physical and human asset specificity, and interdependencies constitute the formative
indicators of asset specificity, defining the construct.
To complete the MIMIC model a pair of reflective indicators is needed that represent direct
consequences of the construct under study. The existence of asset specificity makes it difficult
to compare activities to the market. Two questions pick up on this (Q39 and Q40). The first
asks about the possibilities to obtain the organizational unit’s products/services from another
company without compromising price or quality (‘obtain elsewhere’). A high score implies
difficulty as a consequence of asset specificity. Question 40 indicates whether outsourcing the
unit’s activities would imply a major strategic change. In presence of asset specificity
outsourcing would indeed imply such a change. A high score on this question thus underlines
the importance of insourcing, hence the name of this variable ‘insourcing’. Table 4.17 gives
descriptive statistics on all variables related to asset specificity.
Table 4.17 Descriptive statistics for all indicators of asset specificity
1 physical asset specificity
questions
Q29, Q30**
N
251
min
2
max
10
mean
4.86
std.
deviation
1.90
indicator
type
formative
2 human asset specificity
Q49, Q50,Q52*
257
0.44
4.42
2.68
0.73
formative
3 interdependencies
Q35, Q36**
253
2
10
7.37
1.63
formative
4 obtain elsewhere
RQ39
254
1
4
3.10
0.74
reflective
5 insourcing
Q40
253
1
5
4.04
Note: the abbreviation RQ implies the reverse score for this question is taken.
*the average is taken; the natural logarithm of Q50 is used
**the sum is taken
1.40
reflective
Unfortunately, the items cannot be combined into a MIMIC model. Trying to estimate the
model
returns
insignificant
coefficients
for
both
physical
asset
specificity
and
interdependencies (two out of three formative indicators). Removing insignificant variables
from the model does not provide a solution because leaving out formative indicators alters the
definition of the construct; in my case the definition of asset specificity. Therefore,
Diamantopoulos and Winklhofer (2001) warn us not just to remove indicators from the
model. At the same time, the estimated MIMIC model defeats its own object because it does
not enable me to create an index. Literature does not provide a clear-cut solution to this
71
Chapter four
dilemma and Diamantopoulos and Winklhofer conclude: ”how to balance these
considerations is a question that has not yet been fully resolved” (2001, p272).
A likely explanation for the problems experienced with the MIMIC model for asset
specificity, is the particular distribution of the reflective indicators. Especially Q40 on the
importance of insourcing the activity of the organizational unit is problematic. 64% of the
respondents indicate that insourcing is vital. This variable does not vary enough and
underscores the difficulties in measuring asset specificity with reflective indicators. These
variables, however, play a significant role in index construction and the model depends
strongly on them. This is a limitation of the MIMIC model because one likely chooses to
build a formative indicator construct in cases in which it is difficult to come up with overall
reflective measures of the construct.
To obtain a measure for asset specificity after all, I decide simply to add up the scores on the
formative indicators as shown by equation 4.2. Note that both physical asset specificity and
interdependencies are already combinations of two variables. Thus their scores range from 2
to 10. Scores on human asset specificity range from 1 to 5. To give all three indicators the
same weight, human asset specificity gets a weight of two in the formula. Together these
items define asset specificity. This approach fits the idea of scale construction with formative
indicators in that I stick to the definition and do not take in items that highly correlate only.
However, it is an inferior solution, because it is no different from an ordinary summated
scale. As a consequence the measurement of this variable is less precise. This might have an
impact on the results of the analyses.
As a final step the scores on asset specificity are rescaled to a range of 1 to 5 to simplify
comparison with the other (contingency) variables. Table 4.18 shows descriptive statistics for
asset specificity.
Equation 4.2 Calculating asset specificity
asset specificity = physical asset specificity + 2* human asset specificity + interdependencies
Table 4.18 Descriptive statistics for asset specificity
scale 1-5
questions
N
min
max
mean
std. deviation
asset
specificity
Q29; Q30, Q49; Q50; Q52;
Q35; Q36
245
1.58
4.37
2.93
0.53
72
Operationalizing the framework
4.3.3 Ex-post information asymmetry
To measure ex-post information asymmetry, I modify a measurement instrument created by
Dunk (1993) and used regularly in other studies (see for instance, Abernethy, Bouwens, and
Van Lent (2004), and Bouwens and Van Lent, 2004). The manager indicates his information
advantage over his supervisor on several items (table 4.19).
Table 4.19 Items for ex-post information asymmetry
Cronbach’s = 0.842
component loadings
a
the type of activities undertaken in the subunit
0.717
b
the type of input output relations inherent in the internal operations
0.751
0.735
c
realization of the performance potential of the subunit
d
the technical aspects of the work
0.686
e
the impact of internal factors on the managers activities
0.679
f
the impact of external factors on the managers activities
0.689
g
understanding the achievements of the subunit
0.789
Source: all items except item f: based on Dunk, 1993; modified
Dunk’s original instrument measures information asymmetries in general and I am
particularly interested in ex-post information asymmetry; differences in information between
the supervisor and the manager that persist to exist at the end of a period, i.e. when evaluation
takes place. Adding words to stress the ‘end of a period’ and using different answer categories
creates the appropriate focus. Factor analysis indicates that all seven items load on a single
construct and I combine these into a summated scale (Cronbach’s alpha = 0.842). One case
has a missing value on one of the items and as before I use the remaining items to calculate
information asymmetry for this case. Table 4.20 shows descriptive statistics for all
contingency variables.
Table 4.20 Descriptive statistics for all contingency variables
uncertainty
N
254
minimum
1.28
maximum
4.05
mean
2.39
std. deviation
0.57
measurability of outputs
258
asset specificity
245
1
5
3.72
1.09
1.58
4.37
2.93
ex-post information asymmetry
255
1
0.53
5
3.37
0.83
4.4 Measuring misfit
Ultimately, the main explanatory variable in my analyses is misfit between an MCS and its
relevant archetype. I hypothesize that misfit is negatively related to MCS effectiveness. A
total of 20 variables together map the MCS. The qualitative definition of each archetype given
73
Chapter four
by the theory (see table 4.1) can be translated into operational terms using these 20 variables
(thereby extending the definition and obtaining quantitative scores). A deviation from such a
score for an MCS implies deviation from the archetype and thus adds to misfit. To take this
methodological step, firstly working definitions of the archetypes have to be set up, and then
misfit has to be calculated.
4.4.1 Defining the archetypes
When testing configurational theories, descriptions of the ideal types must be transformed
into ideal profiles containing measurable items. The archetypes and their empirical profiles
should be matched. This is an important step because the outcomes of the research project
directly depend on the measurement of misfit, which in turn depends on the operationalization
of the archetypes.
Researchers choose from two main approaches to operationalize archetypes (cf. Doty et al.,
1993; Doty and Glick, 1994): empirical specification or theoretical specification. Using
empirics, the researcher should select those cases from the sample whose MCSs most closely
resemble the ideal types (Doty and Glick, 1994). Their scores then define the archetypes and
function as the benchmark for the other cases. However, taking this approach, the definition
of the archetypes becomes sample-specific. Another sample might have shown cases with an
even closer resemblance to the theoretical descriptions of the archetypes, and, –if the theory is
correct-, these cases inhibit higher levels of effectiveness. Moreover, archetypes are
theoretical constructs and they represent organizational forms that might exist (Doty and
Glick, 1994) rather than forms that will actually occur. Starting from empirics could thus
narrow down the definition of the constructs. Alternatively, for empirical specification one
might think of selecting those cases with the highest effectiveness and use their MCSs as
benchmarks. This, however, would render tests of the theory invalid: those cases that
resemble the archetypes would be effective by definition, which makes it impossible to test
the theoretical ideas.
The theoretical approach has advantages over the empirical approach: there are no restrictions
because of sample and a closer fit with theory can be established. Purely based on the
interpretation of the theory, expert raters or the theorist himself, set values for all variables
that describe the archetypes. Also in my study, during a range of discussions with the theorist
himself every archetype obtained scores on all 20 variables that map the MCSs. Thus, taking
on this approach, the theoretical and the empirical profiles are matched.
74
Operationalizing the framework
Table 4.21 shows the values assigned to each archetype: arm’s length control (ALC), result
oriented machine control (MR), action oriented machine control (MA), exploratory control
(EC), and boundary control (BC) respectively.
Table 4.21 Operationalizing the archetypes – defining fit
1
work autonomy
ALC
4
MR
3
MA
3
EC
3
BC
4
2
amount of discussion
<3
2 or 3
2 or 3
>2
<3
3
>3
>3
<4
<4
<4
irrelevant
irrelevant
irrelevant
<4
<4
5
use of financial targets
use of non-financial
targets
benchmarking
>3
irrelevant
irrelevant
irrelevant
irrelevant
6
boundaries
irrelevant
irrelevant
irrelevant
irrelevant
3
7
role of budget
irrelevant
1
1
1
0 or 3
8
irrelevant
0
0
1
irrelevant
1
1
1
1
1
10
emergent targets
percentage variable
reward
importance in
performance evaluation:
target achievement
4
4
3
3
3
11
compliance with rules
3
irrelevant
4
3
3
12
long term achievements
4
4
3
4
3
13
subjective judgements
<4
<4
<4
>3
>2
4
9
importance in rewarding:
14
target achievement
>3
>3
<4
<4
<4
15
budget versus actuals
redundant
>3
irrelevant
<4
irrelevant
16
compliance with rules
3
3
4
3
3
importance for career:
17
target achievement
>3
>3
<4
<4
<4
18
budget versus actuals
redundant
>2
<4
<4
<4
19
compliance with rules
3
irrelevant
4
3
3
20 professional skills
irrelevant
irrelevant
irrelevant
>3
>3
Note: the grey boxes indicate extreme misfits: a deviation regarding the particular variable implies a
substantial deviation from the archetype. Therefore, if a misfit occurs for this value, its score will be
multiplied by three when calculating misfit. Note 2: ‘equal to or larger/smaller than’ signs are used for
summated variables.
The definition of the archetypes is based directly on an interpretation of the theory, but the
choices made are largely consistent as well with findings from contingency studies within the
field. On the one hand, this link is established through the MCtce itself that builds from prior
research and integrates existing knowledge. On the other hand, the choices made in the
operationalization reinforce this link. The variables that together map an MCS were not
chosen arbitrarily, but relate for a large part to previous studies (see paragraph 4.1). The same
holds true for many of the underlying hypothesised relationships between the individual
elements of control and the contingency variables. Consider for instance, a summary of
findings from contingency studies on management control system design by Chenhall (2003).
75
Chapter four
Regarding the role of task uncertainty, he writes that high task uncertainty is associated with
less reliance on standard operating procedures, behaviour controls, and accounting
performance measures. Task uncertainty is also associated with more personal controls,
higher participation in budgeting, and usefulness of broad scope MCS (Chenhall, 2003,
p141). The operationalization of the archetypes is consistent with these findings. As opposed
to the other archetypes, exploratory control and boundary control are both related to activities
with high uncertainty27. From the definitions it become clear that these archetypes rely less on
target achievement for performance evaluation and more on subjective judgements.
Moreover, the primary role of the budget is not to act as a performance target, but to give
guidance or set limits.
The insights from contingency studies on MCS design stem mainly from empirical work that
focuses on a single contingency variable and a single control variable (Fisher, 1995; Chenhall,
2003). Therefore it is impossible to justify all values set for individual elements of the
definitions by referring directly to prior work. The discussions were a necessary tool to
operationalize the archetypes and both integrate existing knowledge and expand it to a setting
with multiple contingency variables studying packages of interrelated control instruments.
For most variables, we did not define a single round number, but a range of values (see table
4.21). For instance, the archetype arm’s length control has a value of four or five on the
variable work autonomy. Recall that semantic differential scales were used for measurement
of most variables. The meaning attached to each value of these scales provided guidance for
the decisions on the exact values. For instance, to measure work autonomy managers indicate
the amount of influence they have in several decisions. In the scaling the value four implies
‘quite a bit influence’, whereas the value three relates to ‘some influence’. The archetypes
arm’s length control and boundary control are both characterised by high levels of work
autonomy and thus obtain scores of at least four in the definition.
Another aspect of the operationalization is the assignment of weights to some of the variables
(in table 4.21 this is indicated by the grey boxes). There are two reasons for assigning
weights. First of all, some variables are extremely important in characterising the archetype.
A misfit on one of these variables is considered to be an extreme misfit, a true deviation from
the essence of the archetype. Every archetype has its own extreme misfits: not using market
27
It is difficult to compare results for uncertainty directly across studies. Different definitions of the
construct are used and its measurement is as diverse. Here the comparison is also just a general one
because my measure of uncertainty encompasses much more than task uncertainty. In terms of their
effects both can be compared however.
76
Operationalizing the framework
related benchmarks in an arm’s length control structure is an extreme misfit because it truly
differs from the essence of the archetype. Similarly, a budget should be used primarily as a
target to be achieved when thinking about result oriented machine control. For action oriented
machine control pre-set targets are extremely important (as opposed to emergent targets),
whereas emergent targets characterise exploratory control. The essence of the latter archetype
is also grasped by a focus on long term achievements, work related discussions, and use of
subjective judgements. Finally, boundaries are extremely important for boundary control, but
do not play a distinguishing role in the other archetypes.
The second reason for applying weights is that some variables represent one of the
dimensions of control by themselves, whereas other dimensions are represented by multiple
variables. Without taking those differences into account, an imbalance would be introduced
leading to the domination of some aspects of the management control system in misfit
calculation. This is the case for work autonomy, which is measured by one composite variable
and therefore gets extra weight in the misfit calculations. The next paragraph exactly
describes the calculation of misfit.
4.4.2 Calculating misfit
The observed scores for the MCSs and the theoretical scores for the archetypes can now be
compared one on one. Since I am interested in overall dissimilarity between both profiles,
though, I want to measure the overall distances. For this purpose, I calculate Euclidian
distances, a generally accepted measure28 both within contingency and configurational
research (Doty et al., 1993; Doty and Glick, 1994; Donaldson, 2001). The concept is easy to
understand: distance between two points is measured as a direct line connecting them.
Moreover, measurement of Euclidian distances can easily be extended to a multi-dimensional
space (Hair, Anderson, Tatham, and Black, 1998) and hence enables the comparison of
profiles.
To obtain Euclidian distances, firstly I calculate the deviation between the archetype and an
MCS’s score for each of the 20 variables 29. Thus, a simple subtraction suffices. Consider the
following example: a case obtains a score of 4 on the variable ‘amount of discussion’. Fit with
the archetype arm’s length control is defined as a score of either 1 or 2. The deviation
28
Apart from Euclidian distances, researchers work with Squared Euclidian distances. This saves one
step in the calculations. Note that this has implications for the hypothesized relationship with
effectiveness, however.
29
Definitions of the archetypes do not cover all 20 variables, because some of these are irrelevant or
redundant. This differs per archetype (see table 4.21).
77
Chapter four
between the case and the archetype arm’s length control is then given by 4-2 = 2. Next, the
deviations just calculated are squared. These values are summed. Finally, misfit can be
calculated by taking the square root of the sum. Table 4.22 below illustrates the process for
some of the variables.
Table 4.22 Calculating misfit
archetype:
arm’s length control
observed MCS:
case 21
(Xalc – Xmcs)
work autonomy
4
3
1
amount of discussion
<3
2
0
4
variable
use of financial targets
>3
2
irrelevant
4
-
benchmarking
>3
3
1
(… more variables…)
…
…
…
use of non-financial targets
2
1/2
MISFIT = (3*1 + 0 + 4 + 3*1 +...)
Note: grey boxes indicate extreme misfits. The squared deviation of these values is multiplied by three
before entering the formula. Details follow below.
This general approach, however, needs a few additional steps to handle nominal variables (1),
routed variables (2), and weighting of variables (3). Next, I will explain all three issues and
their solutions.
Nominal variables
In my study I use both ordinal and nominal variables. For nominal variables the distance
between scores 1 and 2 should be the same as between scores 1 and 3, because these only
indicate membership to a certain group and no degree of a variable. This can be easily taken
into account when calculating the distances for each variable separately. However, nominal
variables generate a maximum deviation of 1 point, whereas ordinal variables can generate 2
or 3 points. To make them comparable and give them equal weights, I rescaled the absolute
deviations to a range of 0-3 (where zero implies a fit and three the largest possible misfit).
Routing
Due to routing in the questionnaire, scores on three variables related to bonuses depend on
whether there is a bonus or not. When scores on variables depend on one another and as a
group provide information about the MCS, their misfit scores should also be treated in
combination. This is the case for the variables in my study that assess whether managers
obtain a variable financial reward and when they receive this reward30. Firstly, the variable
30
The relevant questions have numbers 19, 21, and 22 in the questionnaire. The exact questions are to
be found in appendix B.
78
Operationalizing the framework
‘percentage variable reward’ measures how large the variable financial reward is (ranging
from zero to more than 15%). The next three variables indicate the importance of targets (1),
attaining the budget (2), and compliance with behavioural guidelines (3) for being granted
this financial reward. Simply combining the deviation scores for each of these variables,
could be misleading. Therefore I take them as a group and combine their scores as follows.
If not having a bonus implies fit, cases that indeed do not have a bonus automatically score a
perfect fit on all three variables. Cases that do have a bonus, obtain regular deviation scores.
This way it all adds up: a large bonus adds more to misfit, than a small bonus. A bonus based
on target achievement instead of compliance also brings an extra increase to misfit. Basically,
a small bonus based on compliance with behavioural guidelines indicates a smaller misfit,
than a large bonus based on performance targets. Correspondence to the definition of the
archetypes is thus established.
If having a bonus implies fit, not having one leads to misfit. However, cases that do have a
bonus can still obtain misfit scores if the bonus is based on compliance instead of on
achievement of performance targets. This way, having a bonus based on the ‘wrong’ things is
worse than not having a bonus at all.
Setting weights
In the misfit calculation some deviations are multiplied by three. As already mentioned, there
are two reasons for applying weights when calculating misfit: the first is the existence of
extreme misfits, the second is imbalance in the number of variables that represent each of the
dimensions of control. Misfit scores that obtain extra weight in the misfit calculation are
multiplied by three before being summed. This weight of three is decided upon because in
cases where multiple variables represent one dimension of control, together these could create
an approximately three times larger misfit than other variables by themselves.
Taking all the above into account, I calculate misfit between each case and all four
archetypes, not just the relevant archetype. Standard procedures for calculating Euclidian
distances (in SPSS for instance) cannot help out and I decided to do all calculations ‘by hand’
building syntaxes in SPSS. To make misfit scores on different archetypes comparable, I
rescale them to a range of 1-5. Note that several combinations of misfit on single items can
lead to the same amount of overall misfit. The next paragraph provides details on the misfit
variables.
79
Chapter four
4.4.3 The misfit variables
Figure 4.6 shows the misfit scores on the Y-axes grouped per archetype: arm’s length control,
result oriented machine control, action oriented machine control, exploratory control, and
boundary control respectively.
Figure 4.6 Misfit scores compared
5
4
3
2
1
arm’s length
machine result
machine action
exploratory
boundary
A closer look at the misfit variables reveals that none of the cases in my sample closely
resembles the archetypes exploratory control or boundary control. A number of cases
resemble arm’s length control; one case even shows a perfect fit. The same holds true for
result oriented machine control. For this archetype four cases in the sample show a perfect fit.
A quick scan of type of organization, number of employees, and some other descriptive
variables for these perfect fit cases, does not disclose any unique characteristics. Table 4.23
shows the descriptive statistics for the misfit variables.
Table 4.23 Descriptive statistics for the misfit variables
scale 1-5
misfit arm’s length control
N
248
min
1.00
max
3.79
mean
2.30
std. deviation
0.49
misfit result oriented machine control
243
1.00
3.62
2.49
0.65
misfit action oriented machine control
243
1.51
4.28
3.14
0.49
misfit exploratory control
243
1.70
4.07
3.01
0.47
misfit boundary control
244
1.88
4.07
3.19
0.42
80
Operationalizing the framework
4.5 Control variables
Ultimately, the effectiveness of the management control system is the dependent variable in
this study. I am primarily interested in the effect of misfit on the effectiveness, but other
variables might have an impact as well or influence the relationship. Therefore a number of
control variables are taken in. Expectations about the role or influence of these control
variables will be discussed in chapter five. This paragraph describes their measurement.
First of all, I incorporate the effectiveness of the organizational unit into the analyses. To
build a measure of the effectiveness of an organizational unit, I combine ideas from different
studies. Hitt and Middlemist (1979) and Hitt, Ireland, Keats, and Vianna (1983) developed a
method to establish the criteria managers use for the assessment of their unit’s effectiveness.
Their method is extensive, and cannot be replicated in this study, but part of their results is
useful. During the first stage of their research, the authors conduct interviews with unit
managers in which they identify a range of criteria used to assess the effectiveness of
organizational units. These criteria are all related to achievement of organizational goals.
From this list with criteria, I select those that seem generally applicable. I also use some items
from a measure by Van de Ven and Ferry (1980). They measure perceived unit performance. I
combine the items from the before-mentioned studies and add a few additional ones. Table
4.24 shows all items I use to measure effectiveness of the organizational unit.
Table 4.24 Items for the effectiveness of the organizational unit (Q84)
Cronbach’s = 0.845
component loadings
a
achievement of subunit goals
b
cooperation with other subunits
0.726
0.540
c
satisfaction of internal customers of your subunit
0.692
d
satisfaction of external customers of your subunit
0.723
e
employee satisfaction
0.616
f
compliance with standards and behavioural guidelines
0.526
g
problem-solving ability of the subunit
0.540
h
the quantity or amount of work produced
0.599
i
the quality or accuracy of work produced
0.693
j
overall subunit performance
0.859
Source: items a, h, i: Van de Ven and Ferry, 1980, Q93 (p455); items a, b, e, f, g: based on Hitt and
Middlemist, 1979 and Hitt et al. 1983
Factor analysis shows all items load on a single construct. Cronbach’s alpha reveals the
reliability of the scale. Its value is high (0.845). Based on these results, the ten items are
summed and averaged to obtain a summated scale. In cases where managers replied either
‘not applicable’ or when a value is missing, the average of the items left is taken as a score for
unit effectiveness. The smallest remaining number of items is seven (two cases). Managers
81
Chapter four
indicated both good and bad performance with a mean score of 5.06 and sufficient variance
(see table 4.25).
Table 4.25 Descriptive statistics for unit effectiveness
scale 1-7
unit effectiveness
questions
Q84
N
258
min
2.80
max
6.50
mean
5.02
std. deviation
0.69
Cronbach’s
alpha
0.845
Apart from unit effectiveness, other aspects of the unit should be considered. The following
variables act as control variables as well in the analyses: the number of employees who work
on the main activity (a proxy for size), the number of steps up the hierarchy, and a dummy
variable indicating profit versus not-for-profit companies. In the overall sample the share of
not-for-profit companies is 20%. Descriptive statistics on these variables are to be found in
chapter three (3.6), and repeated here for ease of reference.
Table 4.26 Descriptive Statistics
variable
nr of steps to CEO
N
258
minimum
0
maximum
10
mean
2.22
std. deviation
1.39
nr of main activity employees
258
2
3400
88
323
4.6 Closing remarks
In this chapter I have operationalized most steps of the methodological framework. Of central
concern was establishing a match between the theoretical constructs and their measures, and
assessing the validity and reliability. Apart from describing the measurement of the variables
and introducing new measurement instruments with a general applicability, some issues of
broader importance were addressed. As discussed in previous chapters, empirical work that
studies management control systems in their entirety and measures the effectiveness of
control are extremely scarce, yet needed to learn about the workings of control. Therefore, my
study contributes to research within the field of management control by exemplifying one
approach to conduct this type of study. This chapter in particular presented methodological
steps that enable the study of MCS design and effectiveness. Another topic that extends
beyond the scope of this particular project, concerns scale construction. I have suggested
ways to improve general measurement of two main TCE variables: uncertainty and asset
specificity. To better grasp the essence of these constructs, I suggest working with formative
indicator models and could built one myself for uncertainty. The next chapter deals with the
final steps of the methodological framework: determining the relevant archetype and relating
misfit to effectiveness. It reports on and discusses the analyses and results of this study.
82
Chapter five - Analyses and results
The goal of this research project is to gain insights into management control system design
and effectiveness. Therefore, five hypotheses were formulated based on the claims of the
Transaction Cost Theory of Management Control (MCtce). The previous chapters dealt with
several methodological steps that enable the testing of these hypotheses. Below, the
methodological framework reappears highlighting the final steps (figure 5.1). This chapter
reports on those steps and describes hypotheses testing. Apart from these direct tests of the
theoretical ideas, other subjects are addressed. The first paragraph discusses the identification
of the relevant archetype. This is the final preparatory step before we can relate misfit to
effectiveness. To determine the relevant archetype, several alternatives are available and a
discussion of the choices involved follows. The determination of the relevant archetype
results in the allocation of each case to one of the archetypes. In this way five subsamples are
created and paragraph 5.2 will compare these on general characteristics. Next, before testing
the hypotheses, in paragraph 5.3 I explore the occurrence of fit and misfit and the predictive
power of the theory. Then, paragraph 5.4 reports on hypotheses testing. This paragraph firstly
describes the general model used for the analyses. Then, it deals with each of the hypotheses
separately and reveals whether misfit does influence effectiveness or not. As will become
clear though, I cannot test all five hypotheses. In the last part of this chapter, two more topics
are covered. Paragraph 5.5 studies the relationship between the contingency variables and
misfit, and focuses particularly on the complex role of uncertainty. Finally, additional tests
provide insights in the relation between MCS effectiveness and the effectiveness of the
organizational unit. This chapter ends with a summary of the findings.
Figure 5.1 Methodological framework
define
define activity
determine relevant
archetype
architype
compare MCS with
relevant
relevant archetype
archetype
map MCS
misfitt to
link misfi
effectiveness
effectiveness
measure
measure MCS
MCS
effectiveness
effectiveness
83
Chapter five
5.1 The relevant archetype
Recall that the theory submits that the more a management control system resembles its
relevant archetype, the more effective it will be. The relevance of an archetype depends on the
activity of the organizational unit. Three variables (uncertainty, asset specificity, and ex-post
information asymmetry) characterize the activities, and identify those situations each
archetype can handle most effectively. For instance, machine control excels when low
uncertainty and high asset specificity combine. However, two types of machine control exist:
the action oriented type and the result oriented type. Previously (see 2.2.4), I explained my
choice for an additional contingency variable to distinguish between the two: the
measurability of outputs. Taking in this extra variable leads to a refinement of the original
figure from Speklé (2001b) that illustrates the effective matches between activities and
archetypes, as shown below (figure 5.2). Distinguishing groups of cases that share the same
relevant archetype is a prerequisite for the testing of the theoretical claims. The claims on
effectiveness concern contingent effectiveness. The circumstances determine which
management control system will perform relatively the best, whereas the archetypes represent
these effective control systems. The theory specifies the effects of misfit regarding the
relevant archetype only.
RESULT
machine control
Moderate
Low
High
Low
ACTION
machine control
ARM’S LENGTH
CONTROL
Low
High
Uncertainty
EXPLORATORY CONTROL
Ex-Post Information
Asymmetry
BOUNDARY CONTROL
Measurability
of outputs
High
Figure 5.2 Archetypes of control and their habitat
High
Asset Specificity
Note: in case of high measurability of outputs, uncertainty is lower than in case of low measurability of
outputs. Source: Speklé, 2001b, p76, modified
The MCtce indicates the levels of the contingency variables in qualitative terms of high,
moderate, and low. Therefore, in the operationalization of the theory, I have to set the
boundaries: what is high and what is low? For this purpose several alternatives are available,
however, each of them involves making arbitrary choices. Starting from empirics, the mean
84
Analyses and results
and median seem to provide natural boundaries and indeed samples are usually split in half
(see for instance, Gresov, 1989). However, this choice assumes that about half the sample
experiences low levels of the contingency variables, whereas the other half faces high levels.
Although the dataset contains much variety in terms of activities and therefore covers a broad
range of values for the contingency variables, claims as to why a division into high and low
would be equal within the sample or the population, cannot be justified. Also from a
theoretical point of view, there is no reason to expect situations of, for instance, high
information asymmetry to be as common as situations of low information asymmetry.
Several alternatives exist to set the boundaries though. One of them is to determine the
boundary based on the meaning attached to values of a semantic scale. This works fine for
information asymmetry: any case with values four or higher bears the mark ‘high’. In the
scaling that runs from 1 to 5 this value implies a ‘significant advantage’. Respondents who
thought they have a significant information advantage on a certain item circled the number
four. Measurability of outputs receives the same treatment. Its boundary is set at the score of
three, which implies that outputs are ‘somewhat’ measurable. Any case with values three or
lower bears the mark ‘low’.
The alternative chosen for information asymmetry and measurability of outputs is not feasible
for uncertainty and asset specificity, because the semantic meaning of different values in these
scales was lost during their composition. For instance, I cannot claim that the value four in
these scales represents ‘quite a bit uncertainty’. However, it is possible to determine the
highest and lowest scores respondents could have obtained given the data and the
measurement scales ensued. Even though not all of these scores actually show up in the
sample, they identify the range of all possible scores. Following this line of reasoning, I
decide to use the midpoints of the scales for uncertainty and asset specificity as boundaries.
Note that in case of asset specificity, the MCtce only pertains to moderate and high levels,
whereas I distinguish between low and high levels in the operationalization. Like standard
TCE, the theory assigns activities with low asset specificity to the domain of the market
(Speklé, 2001a). Thus, in case of effective alignment only those activities that inhibit at least
moderate levels of asset specificity are present within the hierarchy. However, as discussed in
chapter two (2.3), within the hierarchy sometimes misalignments (misfits) will occur between
the activities and mode of governance. The same holds true for alignment of activities with
either markets or hierarchies. Misalignments will occur at this level as well. Assuming that
only cases with moderate and higher levels of asset specificity show up in my dataset is thus
85
Chapter five
inappropriate. Therefore, I stick to the simple division into low and high, without trying to
restrict the scale of asset specificity to moderate levels.
Table 5.1 shows descriptive statistics for the contingency variables and compares the
boundaries chosen to the empirical mean and median. A t-test shows that for all four variables
the boundary significantly differs from both the mean and the median. For uncertainty, asset
specificity, and ex-post information asymmetry the boundary exceeds both mean and median.
The boundary of measurability of outputs is significantly lower. The implications are the
same: working with these boundaries instead of with the means or medians, assumes that less
than half the organizational units in my sample face circumstances more difficult to control,
with high levels of uncertainty, asset specificity, and information asymmetry, and less
possibilities to measure the outputs.
Table 5.1 Contingency variables and their boundaries
N
scale
min
max
mean
median
boundary
uncertainty
254
1-5
1.28
4.05
2.39
2.33
3.00
measurability of outputs
258
1-5
1.00
5.00
3.72
4.00
3.00
asset specificity
ex-post information
asymmetry
245
1-5
1.58
4.37
2.93
2.90
3.00
255
1-5
1.00
5.00
3.37
3.43
4.00
Together these four contingency variables and their boundaries identify the relevant archetype
for each case. In determining the relevant archetype, firstly, cases are split up into groups with
low ( 3.00) and high (> 3.00) uncertainty and then, cases with low uncertainty are further
separated into groups with low ( 3.00) and high (> 3.00) asset specificity. This second split
identifies the groups arm’s length control and machine control respectively. As a further step,
the distinction between low ( 3.00) and high (> 3.00) measurability of outputs is made
within the group of low uncertainty and high asset specificity. This distinguishes the groups
for action oriented and result oriented machine control. Note that (the reverse of)
measurability of outputs is a form of uncertainty and as such already a part of that construct.
As a consequence, activities that belong to action oriented machine control inhibit more
uncertainty than activities that belong to the result oriented type. Overall though, uncertainty
is still low for both groups. Finally, the cases with high uncertainty fall into separate groups
characterized by low (< 4.00) and high ( 4.00) ex-post information asymmetry, which
returns the groups exploratory and boundary control.
This process leads to the allocation of all cases over the archetypes as shown in table 5.2. The
table reveals that most activities in the sample combine low uncertainty and low asset
86
Analyses and results
specificity. Thus, cases that have arm’s length control as their relevant archetype form the
largest subgroup.
Table 5.2 Relevant archetypes
archetype
frequency
percentage
arm’s length control
120
49.2
result machine control
66
27.0
action machine control
19
7.8
exploratory control
24
9.8
boundary control
15
6.1
total
244
100.0
missing values
14
To check whether the allocation indeed returns groups that significantly differ in mean levels
of uncertainty, asset specificity, and ex-post information asymmetry, I run a number of tests.
If variation in the contingency variables is limited or lots of cases are on or close to the
boundary, the groups might not actually differ. To compare the groups nonparametric tests are
called for because sample size differs largely among the groups, and some are small.
Nonparametric tests are valid when working with small samples (Newbold, 1995). They are
insensitive to outliers and distribution free: no normality assumptions are required.
Firstly, a Kruskal-Wallis test for equality of means 31 (this is the nonparametric alternative to a
one-way ANOVA) indicates that the groups differ significantly. For pair wise comparisons I
use Mann-Whitney tests (the nonparametric alternative for a t-test). Significant differences
between groups show up as desired. The mean score on uncertainty of both exploratory and
boundary control, for instance, significantly exceeds that of arm’s length control and machine
control. Moreover, action oriented and result oriented machine control differ significantly on
measurability of outputs.
Figure 5.3 shows how the cases spread out when compared on scores for the contingency
variables. Panel A compares the scores on uncertainty and asset specificity, whereas panel B
confronts uncertainty and information asymmetry. The scatters show how the boundaries
determine the allocation of cases over the archetypes. Although a number of cases lie very
close to or on the boundaries, most cases are spread out. The choices made to determine the
relevant archetype apparently return distinct groups that can be used for hypotheses testing.
31
These nonparametric tests do not actually test for differences in means, but rather test for differences
in central location using rank orders. In this case, both are pretty much the same and I use the word
means because it is easy to interpret.
87
Chapter five
Figure 5.3
Boundaries that determine the relevant archetypes
Panel A
uncertainty
4,00
3,00
2,00
1,00
1,50
2,00
2,50
3,00
3,50
4,00
4,50
asset specificity
Panel B
uncertainty
4,00
3,00
2,00
1,00
1,00
2,00
3,00
information asymmetry
88
4,00
5,00
Analyses and results
The choices I make to set the boundaries are motivated, but arbitrary. Alternative choices
would have rendered different allocations of the cases over the archetypes and that might
impact the results of this study. When testing the hypotheses, therefore, I perform robustness
checks by shifting the boundaries and assessing the impact. Before testing the claims of the
MCtce directly, however, I compare the groups just created and explore the differences
between them.
5.2 A comparison of the subsamples
In the previous paragraph determination of the relevant archetype resulted in the allocation of
all cases over five groups or subsamples. These groups differ in terms of their activities’
scores on the contingency variables. Because the sample holds a large variety in
organizational units it is interesting to compare the groups on general characteristics as well.
In this paragraph, I compare the groups on types of organizational units represented, share of
profit versus not-for-profit organizations, hierarchical level, and the number of unit
employees. Like before, nonparametric tests are appropriate32.
First of all, table 5.3 compares the types of organizational units represented in each of the
subsamples and the overall sample. The table shows the absolute number of units. The
category ‘other’ is not further explored because of the large diversity in unit types within this
group.
32
Since the goal is basically profile analysis, use of discriminant analysis might spring to mind.
However, large difference in sample size among the groups likely renders unreliable results (Hair, et
al., 1998). Therefore, I confine myself to pair wise comparisons and nonparametric tests.
89
Chapter five
Table 5.3 Division of cases of types of units within each subsample and overall
Arm's
Length
26
Result
Machine
13
Action
Machine
3
Exploratory
Control
3
Boundary
Control
1
total
sample
48
Production
Finance and
Accounting
Division
15
13
1
2
0
32
16
6
3
5
1
31
13
4
0
0
1
19
ICT
5
3
3
5
2
19
Marketing or sales
9
7
0
0
1
17
Project unit
12
4
0
0
0
17
Human resources
Research and
Development
Company as a whole
7
2
2
1
2
14
3
1
2
2
2
11
2
1
0
1
0
8
Facility management
1
2
1
0
1
5
Treasury
3
1
0
0
0
4
Legal department
0
0
1
1
1
3
Other
8
9
3
4
3
20
Total
120
66
19
24
15
258
type of unit
Business unit
The division over types of organizational units within the group arm’s length control differs
slightly from the overall sample: there are relatively more business units, divisions, and
project units, whereas units ICT and research and development are represented less. Business
units form the largest group.
The result oriented type of machine control shows a different picture. Compared to the overall
sample, production departments and units marketing and sales are relatively more
represented, whereas units finance and accounting as well as units research and development
only have small shares within the subsample. Business units and production units are the
dominant types. The share of production units within the subsample is larger than in the other
groups.
Within the group action oriented machine control, I find that divisions, units marketing and
sales, and project units are not represented at all, and the percentage of production units is
much smaller compared to the overall sample. There is no dominant type, but business units,
units finance and accounting, and ICT units form the largest groups and have equal shares.
The subsample exploratory control shows largely the same picture as action oriented machine
control. Within the subsample, divisions, units marketing and sales, and project units are not
represented at all, whereas the percentage of units ICT and finance and accounting are the
largest within the subsample and much larger than in the overall sample.
90
Analyses and results
Finally, boundary control cases show a relatively large number of human resources units and
units research and development. Production units and project units are not present within this
subsample.
The subsamples can also be compared over other general characteristics. Table 5.4 shows the
percentages of profit and not-for-profit organizations for each group and for the overall
sample.
Table 5.4 Profit versus not-for-profit organizations within each subsample
percentage
Arm's Length
Result
Machine
86
Action
Machine
79
Exploratory
Control
62
Boundary
Control
60
total
sample
80
profit
83
not-for-profit
17
14
21
38
40
20
total (%)
100
100
100
100
100
100
The share of not-for-profit companies within the groups arm’s length control and action
oriented machine control does not differ much from the overall sample (about 20%). Within
the groups exploratory and boundary control, however, the relative share of not-for-profit
organizations is much larger. Result oriented machine control has the smallest share (14%).
Next, I use Mann-Whitney tests for pair wise comparisons between the subsamples to explore
differences in terms of hierarchical level and the number of unit employees. No clear profiles
for the groups appear but some differences occur. Significant results (p-values of 0.10 or less)
are summarized next. Regarding hierarchical level, I find that units from the group result
oriented machine control are present at lower levels within the hierarchy than units arm’s
length control and action oriented machine control. Regarding the size of the unit, I test for
differences in number of unit employees. Nonparametric tests assess differences in the
median rather than the mean. Therefore these tests are robust to the influence of outliers.
When studying the variable unit employees this is particularly relevant because within the
overall sample this number varies from 4 to 10,800. Some differences among the subsamples
are significant: organizational units within the group result oriented machine control are
significantly bigger than all other units. The smallest units belong to the subsample boundary
control. The other groups do not differ significantly from one another.
The subsamples are compared on a number of characteristics and table 5.5 summarizes the
main findings.
91
Chapter five
Table 5.5 Distinguishing features of the subsamples related to the archetypes
Arm's Length
Result Machine
Action Machine
Exploratory Control
Boundary
Control
business units
are largest
group; units at
higher
hierarchical
levels
relatively large
shares production
units and business
units; lower
hierarchical levels;
relatively large
number of
employees
relatively large
shares for finance
and accounting
units and ICT; no
divisions, marketing
or sales, or project
units
relatively large
shares of finance
and accounting
units and ICT; no
divisions,
marketing or sales,
or project units
relatively large
share human
resources and
R&D; relatively
small
In sum, regarding unit types some differences among the subsamples exist. The cases are
allocated according to the characteristics of their activities and therefore some of these
differences might be expected to exist. Generally speaking, it is not surprising to find a
relatively large number of production units within the sample of result oriented machine
control, which are found at lower hierarchical levels and employ a large number of people.
The activities conducted within production departments will usually be more programmable
and thus exhibit lower levels of uncertainty. In the same vane, from the outset I would expect
units research and development to be present within the groups of exploratory or boundary
control. These activities are difficult to program and might be characterised by high levels of
information asymmetry. However, the divisions of organizational unit types are not definite.
Units human resources, for instance, are represented in all subgroups and the groups action
oriented machine control and exploratory control largely exhibit similar characteristics. In
chapter three I stressed the importance of distinguishing activities from organizational units
because units generally have multiple activities that might differ in terms of their levels of
uncertainty, asset specificity, and ex-post information asymmetry. Indeed, only 12% of the
managers in the overall sample indicated that their unit has only one activity. Therefore it is
not surprising to see human resources units turn up in each subsample. There are two
explanations: human resource units have different functions within different organizations, or
within these units multiple activities take place, which differ in terms of uncertainty, asset
specificity, and information asymmetry.
Going back to the raw data, I find some evidence to support this view. In the questionnaire
managers were asked to specify up to five activities of their units and choose one to focus on
throughout. Comparing these descriptions for the units human resources, at least two types of
activities can be distinguished: within the groups exploratory and boundary control activities
are related to HR policy making and internal HR consulting. For arm’s length control and
machine control managers chose activities like recruitment and administrative tasks. Thinking
again in terms of programmability of tasks and measurability of outputs, -or about uncertainty
92
Analyses and results
more general-, these differences fit the theoretical ideas. Activities that inhibit lower
uncertainty end up in the group arm’s length control.
I also compare the activities between the ICT units from action oriented machine control and
exploratory control. For the group exploratory control activities chosen are mostly related to
IT projects, whereas the activities within the other group relate to policy making and strategic
decisions. I find no clear differences in activities among these groups for the finance and
accounting units.
From the analysis above, it appears that testing for differences between the subsamples does
not lead to big surprises. Since the samples are generated based on the characteristics of their
activities and compared on other characteristics, this provides some evidence for the quality
of the data. Apparently, the underlying characteristics of the activities are picked up well. It
also stresses the importance of my approach to specifically focus on activities instead of on
organizational units as such. This assures the best correspondence between the theoretical and
empirical levels of analysis.
In this paragraph I have compared the subsamples on their general characteristics. The next
step will be to compare them in terms of their misfit scores. This way I will explore the
predictive power of the theory.
5.3 The occurrence of fit and misfit
In this paragraph, the subsamples will be compared once more, but focus will be on the misfit
variables. Recall that for each case a misfit score regarding each archetype has been
calculated (misfit regarding arm’s length control, misfit regarding result oriented machine
control, et cetera). In chapter two (2.3) expectations as to the occurrence of fit as well as
misfit were discussed. To summarize the arguments: on the one hand, cases are expected to
resemble their relevant archetype because that is effective. On the other hand, states of misfit
will occur routinely because of complexities and during times of change. To research the
issue I can compare misfit scores on all archetypes within each subsample, and I can compare
misfit scores for each archetype between the subsamples.
Firstly, I will compare the average misfit regarding each of the archetypes among the
subsamples. The cases within each subsample are expected on average to resemble their
relevant archetype more closely than all other cases resemble the same archetype. If
organizations strive for effectiveness, and thus for resemblance with the relevant archetype,
93
Chapter five
but mistakes are made, then average resemblance might be larger within the sample. As
before, Mann-Whitney tests are the appropriate tests to make these comparisons. Table 5.6
summarized the results.
Table 5.6 Differences in the average misfit: subsamples against all other cases
misfit variable
misfit arm’s length control
sample sizes of groups
compared
118
120
average misfit score
sig.
lower
0.093
misfit result machine control
63
170
no difference
0.554
misfit action machine control
18
215
(lower)
0.199
misfit exploratory control
22
211
lower
0.002
misfit boundary control
14
220
(lower)
0.178
Note: results are based on the Mann-Whitney U statistic for pair wise comparisons. This nonparametric
test is based on rank orders rather than actual means.
When I compare the cases within the subsample arm’s length control to all other cases, I find
that within the subgroup, units score on average significantly lower on misfit regarding arm’s
length control. For misfit regarding exploratory control this result also appears. There is no
difference in average misfit regarding result oriented machine control between the subsample
and all other cases. Thus, the cases in this subsample do not distinguish themselves from the
other cases in their average level of misfit regarding result oriented machine control. Finally,
for the groups action oriented machine control and boundary control, the outcome is less
convincing: the average misfit scores within the groups seems to be lower, but this result is
only significant at the 20% level. Considering the small sample sizes in these tests, however,
only strong effects are likely to be picked up at all, therefore the result is worth mentioning.
Moreover, one-directional testing is justified because of theoretical expectations (Newbold,
1995).
Above I have compared the average misfit regarding each of the archetypes among the
subsamples. From the outset I expected the average misfit regarding the archetypes to be
lower within their subsample than within the rest of the sample. This is indeed the case for
two of the subsamples. For two other samples the effect is also present but weak, and in only
one subsample there is no effect. I find no results contrary to expectations, however.
Additional insights might be gained by comparing misfit scores within each archetype group.
Table 5.7 shows how many cases resemble their predicted archetype more closely than they
resemble any of the other archetypes. Recall that for each case a misfit score with each
archetype has been calculated. These misfit scores are compared and the lowest determines a
new group ‘minimal misfit’. This comparison sheds light on the predictive power of the
94
Analyses and results
theory. If the theory can help predict which MCS will occur under specific circumstances this
is directly relevant to the study of control structure variety.
Table 5.7 Fit between an MCS and its relevant archetype
minimal misfit score
Action
Exploratory
Machine
Control
9
9
Arm's
Length
44
Result
Machine
50
Result Machine Control
25
27
3
8
0
63
Action Machine Control
6
6
2
2
1
17
Exploratory Control
5
6
4
6
1
22
Boundary Control
3
4
1
5
1
14
Total
83
93
19
30
7
232
relevant archetype
Arm's Length
Boundary
Control
4
total
116
For 44 cases, the MCtce would predict the archetype arm’s length control to prevail and the
case’s MCS indeed resembles that archetype more closely than any of the others. Overall, a
total of 80 cases are predicted correctly, making up 34.5%33. Although there is no significant
linear association between the rows and the columns, the distribution of cases is not uniform
either.
The biggest mistakes in predicting minimum misfit occur between arm’s length control and
result oriented machine control. A large number of cases that mostly resemble arm’s length
control fall into the group result oriented machine control (25 cases). The opposite also occurs
(50 cases). This triggers the question whether I am able to distinguish the two groups well
enough. The two misfit variables are significantly positively correlated. However, the
correlation is moderate (r = 0.391; p = 0.000). Cases do obtain different misfit scores. The
minimum of these misfit scores determines the minimal misfit group. Differences between the
scores might be very small or even zero. From the 75 cases, however, only 8 have very small
difference scores (less than 0.10 on a five point scale). Taking these possible disturbances into
account will not alter the findings in terms of correctly predicted cases.
It is important to realize that this minimal misfit can be relatively small, but still substantial in
absolute terms. Previously, chapter four showed that none of the cases in my sample closely
resembles the archetypes action oriented machine control, exploratory control or boundary
control. A number of cases closely resemble arm’s length control and result oriented machine
33
Studying relationships within a cross-tabulation, one could run a Likelihood ratio test to compare the
predictive power to a random prediction. However, this is not helpful here, because for the random
prediction all possible cells are taken into account and I am only interested in the diagonal.
95
Chapter five
control. The 7 cases in table 5.7 that mostly resemble boundary control thus have large misfits
whereas the 83 cases for arm’s length control can have both large and small misfit scores.
It is also possible to compare the maximum misfit scores within each archetype group. Table
5.8 shows this comparison. 16 cases resemble their relevant archetype least of all (7%).
Again, there is no clear association, but the distribution is not uniform.
Table 5.8 Maximum misfit between an MCS and its relevant archetype
relevant archetype
Arm's Length
Arm's
Length
3
maximum misfit score
Result
Action
Exploratory
Machine
Machine
Control
2
7
36
Boundary
Control
68
total
116
Result Machine Control
0
3
6
18
36
63
Action Machine Control
1
3
0
5
8
17
Exploratory Control
0
7
0
5
10
22
Boundary Control
0
2
1
6
5
14
Total
4
17
14
70
127
232
All in all, the analyses in this paragraph return the following tentative results: within four
subsamples cases on average resemble the relevant archetype more under the predicted
circumstances than under other circumstances. Moreover, almost 35% of cases can be
correctly predicted by the theory in terms of the archetype they resemble most. Only 7% of
cases show less resemblance to their relevant archetype than to any of the other archetypes.
However, my approach here is highly exploratory and provides tentative results. It is
impossible to draw conclusions about the theory without making reference to the
effectiveness of the control systems. I might not find that the MCSs match their predicted
archetype more than any of the others, but without an assessment of the effectiveness this is
little informative. To make claims about the theory therefore, it is necessary to incorporate
effectiveness of control in the analyses. This will be done in the next paragraph.
5.4 Testing the MCtce
In this paragraph I focus on testing the five hypotheses that stem directly from the MCtce. This
involves the final step in the methodological framework: relating misfit to effectiveness. In
chapter four, all cases were compared to each of the archetypes and related misfit scores were
calculated. Each case thus received a score on five different misfit variables (misfit regarding
arm’s length control, misfit regarding action oriented machine control, et cetera). Only one
misfit score is relevant for each case and this has been determined in paragraph 5.1. However,
I cannot combine these scores and test the link between misfit and effectiveness for the
96
Analyses and results
complete sample. As explained in chapter two (2.3), this is inappropriate because of possible
differences in maximum achievable effectiveness among the archetypes. Equifinality among
the archetypes is not assumed. Therefore, one can only test the theoretical claims for
subsamples and compare those cases that share the same relevant archetype34. I estimate
linear regression models to test the hypotheses. Subparagraph 5.4.1 introduces the general
model. Next, for each archetype the analyses and results are presented. Unfortunately, only
the groups for arm’s length control and result oriented machine control are large enough to
accommodate regression analysis. However, bivariate analysis and descriptive statistics shed
some light on the relationships of interest within the other groups.
5.4.1 The general model
The MCtce claims that resemblance of a management control system with its relevant
archetype leads to effectiveness of that system. To study the relationships, I focus on lack of
resemblance and measure misfit. I use a linear regression model to test the hypotheses for
each of the archetypes. Equation 5.1 shows the general form.
Equation 5.1 The general model
effectiveness MCS = + 1*misfit + 2*effectiveness unit + 3*uncertainty + i*control variables + (i = 4, 5, ..., n)
The effectiveness of the management control system is the dependent variable. Misfit
regarding the relevant archetype is the primary variable of interest and expected to be
negatively associated with MCS effectiveness. Apart from misfit, I take on effectiveness of
the organizational unit as well as uncertainty.
Effectiveness of the organizational unit functions as a control variable. I expect the two
measures of effectiveness to be positively correlated. Little is known about the relationship
between the two measures of effectiveness, mainly because researchers generally use
organizational effectiveness as a proxy for effectiveness of control. They do not measure
effectiveness of control directly, nor provide evidence for the relationship between the two
variables (cf. Chenhall, 2003). The assumption that the two are positively correlated will not
meet much opposition. However, making causal claims is problematic because causality
34
An alternative would be to correct for differences in effectiveness when testing the complete sample.
However, differences in the relationships between misfit and effectiveness might also exist among the
groups. The simplest solution therefore is to work with subsamples. It is also the most theory-consistent
solution because apart from assumptions that stem directly from the theory no additional ones are
made.
97
Chapter five
seems to run both ways. Since the MCS is supposed to influence employee behaviour in order
to accomplish organizational goals, a failing control system will likely have a negative
influence on the organizational unit. At the same time, a highly effective organizational unit
can invest more in its MCS to improve it. Moreover, there might be a psychological effect in
rating the perceived performance of the MCS. A manager of a successful organizational unit
might show overall optimism in rating the performance of both the unit and the management
control system. Similarly, when the organizational unit underperforms, a bad feel about
overall performance might impact the assessment of the MCS negatively. Thus, halo error
might be present to some degree, which is the tendency of raters to make global assessments
when they are unable to discriminate between different constructs (see Brownell, 1995, p45).
Note that both effectiveness measures are influenced by many other factors, so that the
relations are probabilistic rather than deterministic. For now, the effectiveness of the unit is
taken in as a control variable. Paragraph 5.6 examines the relationship between the two
effectiveness variables separately.
Uncertainty also enters the equation. Although I have allocated cases according to their levels
of the contingency variables, within each group variation still occurs. Only for uncertainty I
expect a direct effect on MCS effectiveness, apart from its indirect effect through the
determination of the relevant archetype. I argue that a psychological effect of uncertainty
might influence the manager’s assessment of control effectiveness. This is relevant because I
measure perceived effectiveness. Consider a manager of an organizational unit with activities
that are highly certain, he/she will have an overview of what should be done within the unit,
and can assess the outcomes of the work properly. The manager likely experiences a sense of
being in control of the situation. If the MCS functions well, the manager will also be positive
in his/her assessment. To the contrary, in case of high uncertainty, it might be difficult or
impossible to get an overview of the work and to assess the outcomes. Therefore, even if the
control system would be the best we can do (a perfect match according to the theory), the
manager might not experience it to be the most effective system. He/she is always more
comfortable with a control system that enables precise planning and measurement. The
system in place might be the best in absolute terms, but relatively poor. Following this line of
reasoning, I expect the perception of control effectiveness to be negatively influenced by
uncertainty. The role of uncertainty is studied in more detail in paragraph 5.5.
Finally, a number of other control variables appear in the analyses. Which ones and how
many depends on the archetype being tested and will be explicated in each subparagraph.
Before and during the analyses, assumptions underlying ordinary least squares regression
98
Analyses and results
estimation (e.g. normality, multicollinearity, heteroskedasticity, influential observations) are
checked and violations are dealt with accordingly.
5.4.2 Arm’s length control
Firstly I will test the general model for those cases that have arm’s length control as their
relevant archetype. Recall the first hypothesis:
H1:
In case of low uncertainty and moderate asset specificity, resemblance of an MCS
with the archetype arm’s length control is positively associated with MCS
effectiveness.
The largest number of cases falls into this group: 120 organizational units have activities
combining low uncertainty and low asset specificity. As a consequence several control
variables can enter the analysis. Firstly, since my sample includes organizational units at
different hierarchical levels, I take on the number of steps to the CEO to indicate this level. It
might be that managers at low levels within the hierarchy perceive the effectiveness of the
MCS differently from managers higher up the hierarchy. Moreover, differences in the
relationships studied might exist because of the number and diversity of activities within
units, which is probably larger at higher hierarchical levels. The number of steps up the
hierarchy rather than the number of steps down is chosen because the latter only informs us
about structure within the unit.
Secondly, the number of employees working on the main activity enters the equation. In my
model this variable represents size, which is known to influence organizational structures
(Chenhall, 2003; Donaldson, 2001) and might therefore impact misfit or MCS effectiveness. I
choose this variable rather than the number of unit employees for consistency. The level of
analysis is the activity rather than the unit, and both the MCS and the effectiveness thereof are
measured surrounding the activity. The relevant indicator of size therefore, is also the number
of employees working on the activity35.
Finally, I take on a dummy indicating whether the unit is part of a profit rather than a not-forprofit organization. Not-for-profit organizations make up 17% of the cases in this subsample.
The theory does not pertain specifically to profit companies or not-for-profit companies, but
differences in the composition of misfit might exist that could influence the relationships
35
For some cases the number of employees who work on the main activity is very small. Rerunning the
analyses without these cases does not alter the results.
99
Chapter five
studied. Moreover, differences in the perception of effectiveness might exist. Table 5.9 shows
the descriptive statistics for the subsample.
Table 5.9 Descriptive statistics: subsample arm’s length control
1
MCS effectiveness
N
120
min
2.18
max
6.55
mean
4.62
std deviation
0.71
2
misfit arm’s length control
118
1.00
3.39
2.24
0.49
3
unit effectiveness
120
2.80
6.22
4.99
0.67
4
uncertainty
120
1.28
3.00
2.24
0.43
5
nr of main activity employees
120
2
3400
87
337
6
nr of steps to CEO
120
0
10
2.21
1.47
An examination of the variables leads to the following conclusions and adjustments.
Uncertainty shows enough variation to be incorporated into the model, even though the
sample is restricted to cases with low uncertainty. The distribution of two variables deviates
substantially from the normal distribution (high skewness and kurtosis). These variables are
transformed before the regression analysis is performed. I take the natural logarithm of the
number of employees. Moreover, I turn the variable number of steps to the CEO into a
variable with scores zero to five. Each implies the exact number of steps, except the highest
score, which pertains to all cases with five or more steps to the CEO. After these changes, the
assumption of normal distribution for all variables is no longer violated. When assessing
univariate outliers, one case shows up with an extremely low score on MCS effectiveness. An
examination of the original questionnaire and the raw data reveals neither mistakes, nor
curiosities. To assess its influence, the regression analysis is run both with and without this
outlier.
Table 5.10 shows the correlations among the variables as they will be entered in the
regression model. Uncertainty is negatively associated with both effectiveness of the unit and
the number of employees who work on the main activity. The strength of these relationships
is moderate. Overall the correlations between the independent variables are not strong enough
to prompt any concerns for serious multicollinearity. During the regression analysis, however,
I will specifically test for multicollinearity.
100
Analyses and results
Table 5.10 Bivariate correlations within the group arm’s length control
1
Pearson correlations
MCS effectiveness
1
1
2
3
2
misfit arm’s length control
3
unit effectiveness
0.424(**)
0.036
4
uncertainty
-0.248(**)
-0.055
-0.209(*)
5
nr of main activity employees
0.002
-0.147
-0.017
4
5
-0.096
-0.217(*)
6 nr of steps to CEO
-0.128
0.016
0.020
-0.022
0.520
Note: N ranges from 118 to 120, * = significant at the 5% level (2-tailed), ** = significant at the 1% level
(2-tailed).
The correlation table also reveals significant associations between MCS effectiveness and
effectiveness of the unit, and between MCS effectiveness and uncertainty, both showing the
expected sign. In this bivariate analysis, misfit is unrelated to MCS effectiveness.
The next step involves multivariate analysis. Regression analysis is used to test the general
model. Ordinary least squares estimation returns the results as shown in table 5.11, panel A.
The initial model for arm’s length control is significant and explains 25.3% of the variance in
MCS effectiveness (adjusted R2 = 21.1%, N = 115). There are no indications of
multicollinearity or heteroskedasticity in this model. Two cases with extremely large residual
values are identified as outliers.
Table 5.11 Regression models for arm’s length control
Panel A – initial model
B
std. error
T
sig.
5.019
.000
-.135
-1.578
.118
1.055
.406
4.746
.000
1.057
.145
-.173
-1.990
.049
1.097
-.046
.053
-.075
-.870
.386
1.077
nr of steps to CEO*
-.092
.049
-.159
-1.881
.063
1.031
dummy profit companies
.146
.163
.076
.895
.373
1.051
(intercept)
3.781
.753
misfit with arm’s length control
-.196
.124
effectiveness of the unit
.430
.091
uncertainty
-.288
nr of main activity employees**
Beta
2
VIF
2
R = 25.3%, adjusted R = 21.1%, N = 115
Panel B – base model
B
std. error
T
sig.
6.769
.000
-.146
-1.824
.071
1.005
.384
4.625
.000
1.078
.125
-.270
-3.254
.002
1.078
.042
-.099
-1.229
.222
1.003
(intercept)
4.180
.618
misfit with arm’s length control
-.193
.106
.389
.084
uncertainty
-.407
nr of steps to CEO*
-.052
effectiveness of the unit
Beta
2
2
VIF
R = 29.5%, adjusted R = 27%, N = 115
Note: the number of cases is equal for both models. However, these are not all the same cases.
* this variable has been transformed. Its scaling runs from zero to five and indicates the number of steps
to the CEO, except for the highest score obtained by cases with 5 steps or more.
** the natural log of this number is used
101
Chapter five
As expected, the effectiveness of the organizational unit positively correlates with the
effectiveness of the MCS. This variable accounts for most of the variance explained.
Moreover, misfit shows a negative effect on MCS effectiveness, which is significant at the
12% level (p = 0.118)36. Furthermore, uncertainty shows a significant negative coefficient.
Regarding the control variables only the number of steps to the CEO seems to be relevant. A
negative effect of this variable on MCS effectiveness emerges from the analysis, which
implies that managers at lower hierarchical levels indicate a poorer perceived performance of
their MCSs. An explanation might be that they have less influence in the design of the MCS
than managers in positions higher up the hierarchy. Higher involvement might lead to a
higher appreciation. The other control variables do not have an impact on MCS effectiveness.
As a second step, I run a trimmed model leaving out insignificant control variables.
Moreover, the two cases that are identified as outliers (because of their extreme residual
values) are removed from the analysis as well. One of them is the same case already identified
as an outlier in the univariate analysis. This case turned out to be extremely influential in the
regression model, especially in the relationship between misfit and MCS effectiveness.
Combining trimming and outlier deletion results in the base model as shown in table 5.11,
panel B. The model shows an overall improvement to the initial model. The model better fits
the data and the variance explained in MCS effectiveness increases by about 4% (R2 =
29.5%). Again there are no indications of multicollinearity or heteroskedasticity. For
effectiveness of the unit and for uncertainty, results remain the same and show lower levels of
significance. The result for the misfit variable improves and becomes significant at the 7%
level of confidence (p = 0.071). The direction and the strength of the effect remain the same.
The effect of the number of steps to the CEO disappears and becomes insignificant in the base
model. Together the findings from the initial and the base model support the first hypothesis
and also return the expected results for the variables effectiveness of the unit and uncertainty.
As discussed in the previous paragraph, when allocating cases according to their relevant
archetype, I use arbitrary boundaries. Some cases are close to the boundary and would fall
into different categories when the boundary changed only a little. Therefore, it is relevant to
test whether choosing different boundaries would render different findings. I perform
robustness checks on the results for arm’s length control adjusting the boundaries of
respectively asset specificity and uncertainty. After each adjustment, cases are again allocated
36
It would have been appropriate here to perform one-tailed tests. The p-value for misfit then drops to
0.06. Testing one-tailed is appropriate when strong indications about the direction of the effect exist
(Newbold, 1995).
102
Analyses and results
according to their relevant archetype. Next, the analyses are run again for the new samples of
cases that belong to arm’s length control.
For the robustness checks I use the base model without outliers and insignificant control
variables37. Firstly, two models with respectively a higher (3.2) and a lower (2.8) boundary
for asset specificity are estimated. These values are chosen based on visual inspection of the
scatter plot shown in figure 5.3 panel A. An adjustment should change the sample to
incorporate cases that are close to the boundary, but should not involve too large a shift of that
boundary. I do not want to test a different strategy in boundary setting, but merely test
whether the choices I have made are sensitive to small changes. The number of cases for
samples with the higher and lower boundaries of asset specificity is 140 and 89 respectively.
Using these samples again in the regression analysis does not render different results. Next, I
repeat the exercise adjusting the boundary for uncertainty up (3.2) and down (2.8). The
number of cases in these models is 127 and 104 respectively. Based on these extra analyses,
the overall conclusions do not change and therefore I conclude that the previous findings are
robust to changes in boundary decisions. In these models only the effect for the variable
‘number of steps to the CEO’ changes. In some cases the effect is significant and negative, in
others it is not significant.
Another relevant question to address is whether or not resemblance with the archetype arm’s
length control is always beneficial, regardless of the levels of the contingency variables. In
that case the theory would not be supported (or only partly), because the effects of
resemblance with arm’s length control would not be contingent on the characteristics of the
activities. To answer this question, I test the relationships between misfit with arm’s length
control and MCS effectiveness for the group of cases experiencing high uncertainty (N = 39).
In this subsample misfit is not related to the effectiveness variable (a null result). Also for the
group of cases that experience high levels of asset specificity, misfit is unrelated to MCS
effectiveness. Thus the effect of misfit regarding arm’s length control pertains only to those
observations for which theory predicts that it would. Under the specific circumstances of low
uncertainty and low asset specificity only, MCSs that resemble arm’s length control are more
effective than other governance structures.
37
When I use the initial model for the robustness checks, in two cases the result for misfit becomes less
significant. The worst case scenario returns a p-value of 0.17, two-tailed. The two outliers identified
earlier are responsible for this decline. However, most analyses return results that are significant at
lower confidence levels and the sign of the effect is negative in all cases. Therefore, I conclude that
overall the results are robust.
103
Chapter five
In sum, I find evidence that misfit regarding the archetype arm’s length control has a negative
impact on the perceived effectiveness of the control system. This effect is confined to the
group of cases that experience low levels of both uncertainty and asset specificity. The results
are robust to shifts of the boundaries and hypothesis H1 is being supported.
5.4.3 Result Oriented Machine Control
The second hypothesis pertains to machine control of the result oriented type:
H2:
In case of low uncertainty, high asset specificity, and high measurability of outputs,
resemblance of an MCS with the archetype result oriented machine control is
positively associated with MCS effectiveness.
This group holds 66 observations, which is enough to accommodate regression analysis.
Because of the limited number of cases, not all control variables can enter the model. Table
5.12 shows the descriptive statistics. The share of not-for-profit organizations within this
subsample is much smaller than in the overall sample (14% versus 20%).
Table 5.12 Descriptive statistics: subsample result oriented machine control
1
MCS effectiveness
N
66
min
3.00
max
6.09
mean
4.60
std deviation
0.81
2
misfit result oriented machine control
63
1.00
3.62
2.46
0.66
3
unit effectiveness
66
3.50
6.50
5.10
0.70
4
uncertainty
66
1.29
2.79
2.09
0.36
5
nr of main activity employees
66
4
3200
137
439
6
nr of step to the CEO
66
1
8
2.58
1.38
An examination of the variables leads to the following conclusions: uncertainty shows enough
variation to be incorporated into the model, even though the sample is restricted to cases with
low uncertainty. Overall the scores on misfit regarding result oriented machine control are
relatively low. The maximum value of this variable is 3.62 on a five point scale. However,
previous tests (see 5.3) showed that there is no significant difference between the average
misfit score of this subsample and all other cases. The distribution of variables does not
deviate substantially from the normal distribution (skewness and kurtosis values do not
exceed the generally accepted levels). Finally, no univariate outliers are identified.
Table 5.13 shows the correlations among the variables as they will be entered in the
regression model. Uncertainty is negatively associated with effectiveness of the unit. The
104
Analyses and results
correlation between both variables is quite strong (r = -0.503, p = 0.000). Furthermore,
uncertainty is positively associated with misfit. The strength of this relationship is moderate (r
= 0.376, p = 0.002). Misfit is not related to any of the effectiveness measures. Overall, some
of the correlations between the independent variables are quite strong and multicollinearity
might disturb the regression analysis.
Table 5.13 Bivariate correlations within the group result oriented machine control
1
Pearson correlations
MCS effectiveness
2
misfit result machine control
3
unit effectiveness
1
1
2
3
-0.075
0.532(**)
-0.177
4
uncertainty
-0.295(*)
0.376(**)
-0.503(**)
Note: N ranges from 63 to 66, * = significant at the 5% level (2-tailed), ** = significant at the 1% level (2tailed).
Using the general model I test the relationship between misfit regarding result oriented
machine control and MCS effectiveness. Apart from MCS effectiveness and misfit, I take on
effectiveness of the unit, uncertainty, and the number of employees who work on the main
activity38. Table 5.14 shows the results. The model is significant and explains 28.8% of the
variance in MCS effectiveness (adjusted R2 = 23.9%, N = 63).
Table 5.14 Model for result oriented machine control
B
1.443
std. error
1.258
Beta
misfit with result machine control
.046
.152
effectiveness of the unit
. 606
.150
uncertainty
-.062
nr of main activity employees*
.023
(intercept)
T
1.147
sig.
.256
VIF
.037
.302
.764
1.203
.523
4.049
.000
1.359
.319
-.027
-.193
.848
1.538
.065
.041
.359
.721
1.057
2
2
R = 28.8%, adjusted R = 23.9%, N = 63
* the natural log of this number is used
The results show that effectiveness of the organizational unit has a positive influence on MCS
effectiveness. Moreover, neither misfit nor uncertainty has an effect on MCS effectiveness.
Hypotheses H2 is not being supported.
Mild multicollinearity seems to distort the findings for uncertainty. The variance inflation
factor (VIF) remains below the threshold level of two, but the condition index indicates
possible collinearity problems between effectiveness of the unit and uncertainty (ci = 36.557).
38
As before, some cases have a very small number of employees who work on the main activity.
Rerunning the analyses without these cases does not alter the results.
105
Chapter five
When exploring correlations among the variables (table 5.13), I had already found a fairly
large negative correlation between effectiveness of the unit and uncertainty. The table also
shows that uncertainty and MCS effectiveness are negatively correlated. In a bivariate test I
find a negative effect of uncertainty on MCS effectiveness as expected, but I cannot test for it
in the multivariate model. The regression model for result oriented machine control also
suffers from mild heteroskedasticity, an indication that another variable that explains MCS
effectiveness is missing from the model.
Again, I perform robustness checks on these results. Adjusting the boundary for uncertainty
downward (level = 2.8) does not alter the original sample. Adjusting the boundary upward to
3.2 adds one case to the sample. This is a result of the boundary put on measurability of
outputs. Both types of machine control have activities that exhibit low uncertainty and high
asset specificity. To distinguish between the two types, I chose measurability of outputs as a
fourth contingency variable. This variable represents a form of uncertainty and is as such also
a part of that construct. Within the group of low uncertainty, the variable measurability of
outputs only gets three scores: 3, 4, or 5 (on a five point scale). Those cases with scores 4 and
5 are characterized by highly measurable outputs and thus a lower uncertainty. Those cases
belong to the initial sample for result oriented machine control. For the robustness check, I
can adjust the boundary for measurability of outputs downwards (level > 2). This returns a
sample of 85 cases, which contains all cases for the two types of machine control combined.
Rerunning the regression analysis with this subsample does not alter the conclusions. The
coefficient of misfit is insignificant once more. The next paragraph (5.4.4) will look into the
differences between action and result oriented machine control in more detail.
Adjusting the boundary for asset specificity returns samples with 49 cases (higher boundary
of 3.2) and 86 cases (lower boundary of 2.8) respectively. The results of the regression
analysis reported above are also robust to these changes. The effect of misfit regarding result
oriented machine control is not significant. Other results do not change either.
All in all, I find no evidence of a negative effect of misfit regarding result oriented machine
control for MCS effectiveness. This null result is robust to changes in the boundaries that
determine the relevant archetype. Hypothesis H2 is not supported.
Up to this point the analysis focused on identifying a linear relationship between misfit and
effectiveness for which no evidence has been found. As discussed in chapter two, however,
the form of the relationship between misfit and effectiveness of control remains unclear from
the theory. Therefore, other types of relationships might exist between the two variables. In
106
Analyses and results
this final part of this paragraph I will explore the issue for the subsample result oriented
machine control. Instead of assuming that each increase in misfit has an effect on MCS
performance, it might be the case that only large deviations have an effect on effectiveness,
whereas small deviations do not impact performance directly. This relates to the
complementarities between control instruments that together constitute the packages of
control (see 2.1.2). In case of strong interrelationships minor misfits might impact control
effectiveness directly. However, if the complementarities are less strong, minor misfits might
not cause control problems. This would imply some sort of threshold value of misfit. To
explore whether I can find some evidence for this idea of a threshold misfit level, I compare
two groups of cases within the subsample result oriented machine control. The first group
contains cases that resemble the archetype the most. The misfit value of these cases lies below
the mean minus one standard deviation. The other group contains cases that are furthest away
from the archetype. Their misfit value lies above the mean plus one standard deviation. I
would expect the average effectiveness of the control system to differ between the groups,
and be larger within the first group that resembles the archetype. However, when comparing
the average MCS effectiveness between the two groups I find no significant difference. Once
more, I conclude that misfit does not impact MCS effectiveness for the cases that belong to
result oriented machine control. This initial analysis does not support the idea of a threshold
level of misfit. However, the threshold level might not be reached within this sample. The
descriptive statistics (table 5.12) show that the maximum misfit score is 3.62 on a five point
scale. There are no cases within the sample that show very large deviations from the
archetype. Larger samples would provide more possibilities further to research the issue in
future research projects.
5.4.4 Action Oriented Machine Control
To test the third hypothesis, I study cases that belong to the group action oriented machine
control. Only 19 cases fall into this group. Unfortunately, this number is too small to
accommodate regression analysis39 and I will stick to descriptive statistics and nonparametric
bivariate analyses. The third hypothesis states:
H3:
In case of low uncertainty, high asset specificity, and low measurability of outputs,
resemblance of an MCS with the archetype action oriented machine control is
positively associated with MCS effectiveness.
39
As a threshold level of number of observations per independent variable, Hair et al. (1998; p166)
state the ratio 5 to 1. For each independent variable one should have at least five observations.
However, small samples (less than 20 observations) are only appropriate for simple regression analysis
(p164).
107
Chapter five
To learn about this subgroup, firstly I look at descriptive statistics. Next, correlations provide
a feel for the interrelationships among the variables of interest. Table 5.15 shows descriptive
statistics for the subsample.
Table 5.15 Descriptive statistics: subsample action oriented machine control
1
MCS effectiveness
N
19
min
2.64
max
6.00
mean
4.36
std deviation
0.85
2
misfit action oriented machine control
18
2.48
3.73
3.03
0.39
3
unit effectiveness
19
2.90
6.40
5.13
0.80
4
uncertainty
19
1.70
2.15
1.95
0.13
5
nr of main activity employees
19
4
500
39
113
6
nr of steps to CEO
19
0
4
1.79
1.03
The ratio of profit versus not-for-profit organizations is similar to that in the overall sample
(21% not-for-profit in the subsample versus 20% overall). All variables show enough
variation in scores to enable bivariate nonparametric tests. When comparing the subsamples
previously (5.3), tests showed that the average misfit score regarding action oriented machine
control does not differ between this subgroup and all other cases. However, those cases that
resemble the archetype the most closely are no part of this subsample. In the overall sample
the minimal misfit score is 1.51, within this subsample it is 2.48 (recall that the scale runs
from 1 to 5). This might render less power to the test for effects of misfit on effectiveness
because I cannot compare these cases to cases that show a relatively large fit.
To study the relationships of interest, I calculate correlations between MCS effectiveness,
misfit regarding action oriented machine control, effectiveness of the unit, and uncertainty.
Because of the small sample size, nonparametric tests are appropriate. These tests are better
suited to handle smaller samples, because they are insensitive to outliers (as opposed to
Pearson correlations, for instance) and valid for all population distributions. No underlying
assumptions about the population distribution are made. As a general rule, Newbold (1995)
states that in bivariate analysis samples that holds less than 30 observations are denoted small.
Here I use Spearman’s Rho to provide evidence on the associations among the variables. This
measure is valid for samples of 5 – 30 observations (Newbold, 1995). Its interpretation is
similar to the Pearson correlation coefficient. Table 5.16 shows the results.
108
Analyses and results
Table 5.16 Bivariate correlations within the group action oriented machine control
Spearman’s Rho
1
misfit action oriented
machine control
unit effectiveness
uncertainty
0.243
0.601(**)
-0.384 (^)
MCS effectiveness
N ranges from 18 to 19, ^ = significant at the 10% level (2-tailed), ** = significant at the 1% level (2tailed). Note: other bivariate correlations among the four variables are not significant (not reported)
The relationships among MCS effectiveness, unit effectiveness, and uncertainty are as
expected. MCS effectiveness and effectiveness of the organizational unit correlate positively
(p = 0.007), whereas MCS effectiveness and uncertainty are negatively correlated. This result
is significant at the 10% confidence level. There is no relationship between misfit and
effectiveness. However, within small samples, only strong effects among variables will be
picked up. Testing for other possible bivariate correlations between the four variables returns
no significant results.
To test whether these results are robust to changes in the boundaries that determine the
relevant archetype, I rerun the analysis twice with adjusted boundaries for uncertainty and
asset specificity respectively. When determining which cases belong to action oriented
machine control I set the boundary for uncertainty at 3.2 (N = 28). Next, I set the boundary
for asset specificity at 2.8 (N = 25). Since the sample is already small, I can only adjust the
boundaries such that the sample expands. For both situations, the results for the relationships
between MCS effectiveness, effectiveness of the unit, and uncertainty are stable. However,
the correlation between misfit and MCS effectiveness becomes positive (r = 0.345; p = 0.078
(higher uncertainty boundary); r = 0.366; p = 0.079 (lower asset specificity boundary). The
first result is thus unstable. In a larger sample, misfit regarding action oriented machine
control positively affects MCS performance. This is contrary to the expectations and contrary
to hypothesis H3. Finally, I adjust the boundary for measurability of outputs upwards to
incorporate values 4 or less. This has a large impact on sample size (N = 57). In this sample I
once more find a significant positive effect of misfit on MCS effectiveness (r = 0.404, p =
0.003).
Within the group of cases that have action oriented machine control as their relevant
archetype, I find evidence for a positive relationship between misfit and effectiveness. The
less the MCS resembles the archetype, the more effective it will be. This contradicts
hypothesis H3. The result appears in all samples studied as part of the robustness check. Only
in the initial sample I found no significant result, but that sample is very small and therefore
less likely to pick up any relationships.
109
Chapter five
An interlude: action oriented versus result oriented machine control
The MCtce indicates that both types of machine control are effective in circumstances of low
uncertainty and high asset specificity. To distinguish between the two types, I chose
measurability of outputs as an extra contingency variable. This variable represents a form of
uncertainty and is as such also part of that construct. Within the group of low uncertainty, the
variable measurability of outputs only gets three discrete scores: 3, 4, or 5 (on a five point
scale). Therefore, the division of cases into two groups is rather crude. I might not be able to
distinguish between the two types in a way that best fits the theoretical ideas. Using other
alternatives might have rendered different results. To explore the impact of this decision I run
extra analyses: I combine the groups action oriented machine control and result oriented
machine control and test for the effect of a misfit variable that takes on the score of the lowest
of both misfit variables for each case. I thus compare for each case the scores on both misfit
regarding action oriented and result oriented machine control, and take the lowest. This
variable will be referred to as misfit regarding machine control. This fits the theoretical idea
that the choice between the two types of machine control is a forced choice (Speklé, 2004)
because the situation will enable the one or the other form. The most suitable form will occur,
which is reflected in a lower misfit.
Table 5.17 shows how the two types of machine control make up this new variable and also
shows the initial group for each case. About 75% (N = 61) of the cases shows most
resemblance with the result oriented type of machine control. The other cases resemble the
action oriented type more closely (N = 20). The table also reveals that out of these 61 cases
that resemble the result oriented type more than the action oriented type, 51 indeed belong to
the initial result oriented machine control group. Focusing on the minimal misfit thus renders
a division of the machine control cases that differs from my division based on measurability
of outputs.
Table 5.17 The initial groups versus the groups based on minimum misfit
misfit regarding
machine control:
action oriented
result oriented
total
archetype:
action oriented
8
10
result oriented
12
51
total
20
61
18
63
81
Using the new misfit variable, I estimate the general regression model once more. The results
show that the misfit variable is unrelated to MCS effectiveness (p = 0.923). This different
approach does not render new insights regarding the effects of misfit on effectiveness.
110
Analyses and results
Finally, one analysis is left that might provide a better test of the relationships under study.
There is agreement between the two approaches for 51 cases regarding machine control.
Therefore I can also test the regression model for this group. This renders the same results as
before for result oriented machine control: there is no relationship between misfit and
effectiveness.
In sum, the results I have found for both types of machine control are robust to decisions
regarding the contingency variables.
5.4.5 Exploratory control
The number of cases that falls into the group exploratory control is 24. Once more, the
number is too small to accommodate regression analysis. Hypothesis H4 relates to this group:
H4:
In case of high uncertainty and low ex-post information asymmetry, resemblance of
an MCS with the archetype exploratory control is positively associated with MCS
effectiveness.
To learn about this subgroup, I look at descriptive statistics and calculate correlations among
the variables of interest. Table 5.18 shows descriptive statistics for the subsample.
Table 5.18 Descriptive statistics: subsample exploratory control
1
MCS effectiveness
N
24
min
2.64
max
5.18
mean
3.78
std deviation
0.78
2
misfit exploratory control
22
1.84
3.41
2.70
0.47
3
unit effectiveness
24
2.88
6.33
4.69
0.83
4
uncertainty
24
3.03
3.94
3.29
0.28
5
nr of main activity employees
24
2
500
39
99
6
nr of steps to CEO
24
0
5
2.17
1.09
The share of not-for-profit organizations exceeds that of the overall sample (38% not-forprofit in the subsample versus 20% overall). Previously when comparing this subsample to all
other cases (5.3), I found that within the subgroup, units score on average significantly lower
on misfit regarding exploratory control.
To get insights into the bivariate relationships, I calculate correlation coefficients. As before,
because of the sample size, I rely on nonparametric tests. Table 5.19 shows various significant
relationships. Firstly, a positive relationship between MCS effectiveness and the effectiveness
111
Chapter five
of the unit exists. This effect is significant at the 1% level. Within the other subsamples this
relationship also emerged. Contrary to the theoretical expectations however, a significant
positive relationship exists between misfit and MCS effectiveness (p = 0.10). Within this
subsample, cases that resemble the archetype exploratory control more closely are less
effective than those that do not. This contradicts hypothesis H4.
Table 5.19 Bivariate correlations within the group exploratory control
1
Spearman’s Rho
MCS effectiveness
1
1
2
misfit exploratory control
0.354 (^)
3
unit effectiveness
0.549(**)
2
3
0.367 (^)
4
uncertainty
0.141
0.406 (^)
0.278
Note: N ranges from 22 to 24, ^ = significant at the 10% level (2-tailed), ** = significant at the 1% level
(2-tailed).
Table 5.19 also reveals that there is no relationship between MCS effectiveness and
uncertainty, nor is uncertainty related to unit effectiveness. However, the variable is positively
associated with misfit regarding exploratory control (p = 0.06). This result will be discussed
in paragraph 5.5, which specifically studies the relationships between the misfit variables and
the contingency variables.
Robustness checks show that the results are stable for the relationships among MCS
effectiveness, effectiveness of the unit, and uncertainty. The result for misfit, however, is
unstable. There are two possibilities for adjusting the boundaries: lower values of uncertainty
are taken in (level = 2.8) or higher values of ex-post information asymmetry (level = 4.2).
Adjusting the boundaries in the opposite directions returns either no differences in sample, or
too small a sample. Lowering the boundary for uncertainty returns a sample with 40 cases.
The positive effect of misfit on MCS effectiveness shows up again (r = 0.305; p = 0.062).
When I adjust the boundary of information asymmetry upwards, however, I find no
relationship between misfit and MCS effectiveness (N = 31).
Within the group of cases that have exploratory control as their relevant archetype, I find
some evidence for a positive relationship between misfit and effectiveness and some evidence
that suggests there is no relationship between the variables. The effect is sensitive to decisions
regarding the level of ex-post information asymmetry. When I incorporate cases with higher
levels of information asymmetry, I find no influence of misfit on MCS effectiveness.
However, overall I have found no evidence that meets the theoretical expectations.
Hypothesis H4 is not supported.
112
Analyses and results
5.4.6 Boundary control
The smallest number of cases belongs to the group boundary control. These cases experience
high levels of uncertainty as well as high levels of ex-post information asymmetry.
Hypothesis H5 was formulated for this group:
H5:
In case of high uncertainty and high ex-post information asymmetry, resemblance of
an MCS with the archetype boundary control is positively associated with MCS
effectiveness.
Again, I confine myself to review the descriptive statistics and the correlations between the
variables of interest. Table 5.20 shows descriptive statistics.
Table 5.20 Descriptive statistics: subsample boundary control
1
MCS effectiveness
N
15
min
2.73
max
6.18
mean
4.35
std deviation
0.88
2
misfit boundary control
14
2.35
3.90
3.06
0.42
3
unit effectiveness
15
4.20
6.20
5.18
0.50
4
uncertainty
15
3.04
4.05
3.42
0.36
5
nr of main activity employees
15
2
350
37
88
6
nr of steps to CEO
15
0
5
2.00
1.25
When comparing the share of not-for-profit organizations within the subsample to the overall
sample (5.2), I found their share is much larger (40% versus 20%). Comparing the average
misfit regarding boundary control of cases within this subsample to all other cases, I found no
significant differences. As was the case for action oriented machine control, this sample does
not contain cases with small misfits either.
Table 5.21 reports the correlation coefficients among the variables MCS effectiveness,
effectiveness of the organizational unit, uncertainty, and misfit regarding the archetype
boundary control. Within the group boundary control, there are no significant relationships
between MCS effectiveness and effectiveness of the unit, nor between MCS effectiveness and
uncertainty. Misfit is also unrelated to MCS effectiveness. Note however, the same
reservations made previously for drawing conclusions from small samples are still valid. Only
large effects will be picked up within such a small sample. Of all other possible bivariate
correlations only the association between misfit and effectiveness of the unit is significant.
Both variables are positively correlated (p = 0.01). The theory does not inform us about this
relationship.
113
Chapter five
Table 5.21 Bivariate correlations within the group boundary control
1
Spearman’s Rho
MCS effectiveness
1
1
2
misfit boundary control
0.303
3
unit effectiveness
-0.050
4
uncertainty
0.007
Note: N ranges from 14 to 15, * = significant at the 5% level (2-tailed)
2
3
0.657(*)
-0.121
-0.020
Once more, I perform robustness checks on these results. Testing the relationships for a
sample with a lower uncertainty boundary (level = 2.05) and a lower boundary for
information asymmetry (level = 3.5) confirms the results. Misfit and MCS effectiveness are
unrelated. Sample sizes remain very small, however (N = 20 and N = 23 respectively).
All in all, I find no evidence for a negative correlation between misfit regarding boundary
control and MCS effectiveness. Hypothesis H5 is not being supported.
5.5 Misfit and the contingency variables
Apart from the results that relate directly to the hypotheses, other results deserve our
attention. An issue left unexplored so far, is the relationship between the contingency
variables and misfit. Studying this relationship is beneficial for several reasons. It sheds light
on the underlying assumptions part of the configurational approach, and helps to interpret the
previous findings. Moreover, it can again provide insights into the predictive power of the
theory. As discussed in chapter two (2.3), the contingency approach and the configurational
approach have different expectations about the relationship between contingency variables
and structure variables like the MCSs. In this paragraph I take on an exploratory approach to
study this relationship. Figure 5.4 shows that two steps exist in the theory: contingency
variables determine the relevant archetype, and misfit regarding the relevant archetype relates
to MCS effectiveness. However, the contingency variables might also affect misfit.
Figure 5.4 A two-step analysis
contingency
variables
(UNC, AS, IA)
I
relevant
archetype
and
misfit
misfit
114
II
MCS
effectiveness
Analyses and results
As an initial step in this exploration of the relationships between contingency variables and
misfit variables, I calculate correlations between both types of variables for the complete
sample. Based on the MCtce it is expected that in case of rising asset specificity, for instance,
cases would resemble the two types of machine control more closely, than they would
resemble arm’s length control. Therefore, a positive correlation between misfit regarding
arm’s length control and asset specificity might exist. Correlation tables reveal that both asset
specificity and information asymmetry do not correlate with any of the misfit variables.
Uncertainty, however, correlates with all of them (see table 5.22). Moreover, the direction of
the correlations are as the theory would predict: in case of rising uncertainty MCSs move
away from arm’s length control and result oriented machine control, and towards exploratory
and boundary control.
Table 5.22 Correlations between uncertainty and the misfit variables
Pearson correlations
1
uncertainty
misfit ALC
misfit MR
misfit MA
misfit EC
misfit BC
0.170(**)
0.244(**)
-0.234(**)
-0.344(**)
-0.387(**)
N ranges from 241 to 246, ** = significant at the 1% level (2-tailed). Note: there are no significant
relationships between asset specificity and the misfit variables or ex-post information asymmetry and
the misfit variables.
Of course, this is a very crude way of analysing the situation because I do not take into
account the interactions between the contingency variables. Nevertheless, some insights are
gained.
Apart from studying whether changes in misfit occur with changes in the contingency
variables, it is interesting to assess how the changes take place. As discussed in chapter two
(2.3) different views exist. The first view is that a change in the contingency variables leads to
an adjustment of the MCS and a general tendency to move towards an effective situation.
Gradual adjustments are possible and a fit-line exists (instead a discrete number of fit states).
This is the general way of thinking within contingency theory (Donaldson, 2001). In this view
fit will be re-established through gradual change. An opposed view stems from
configurational thinking and is that the MCS does not change gradually to fit the changing
circumstances, but change occurs as a quantum jump (Gerdin and Greve, 2004). Within
configurational thinking only few states of fit are possible, represented by the archetypes.
Organizational units cannot adjust their MCS gradually because of the complementarities
between control instruments. A small change will disrupt the complete system, and therefore
change is only beneficial if it involves the system as a whole. A one-on-one relationship
between the contingency variables and misfit is thus not assumed.
115
Chapter five
The correlation coefficients calculated above cannot shed light on this issue, because
significant correlations could show up in both cases. To further explore the interrelationships
between the contingency variables and misfit, I run a different analysis. I regress misfit on
uncertainty, asset specificity, and their interaction for the two largest subsamples of arm’s
length control and result oriented machine control. The interaction term enters the model
because the theory pertains to joint effects of the contingency variables, rather than their
individual effects, like standard Transaction Cost Economics (Boerner and Macher, 2001;
David and Han, 2004)40. The product of uncertainty and asset specificity forms the interaction
term. To prevent problems of multicollinearity, I centred (mean adjusted) the scores on
uncertainty and asset specificity before calculating the multiplicative term (cf. Aiken and
West, 1991).
For arm’s length control, the model is not significant (N = 118). There is no relationship
between the contingency variables and misfit regarding arm’s length control. An exploration
of scatter plots that confronts misfit and asset specificity as well as misfit and uncertainty,
does not reveal any other (non-linear) relationships between the variables. Combining this
result with the results found earlier for the relationship between uncertainty and MCS
effectiveness, paints a more complete picture. Recall that within this subsample a direct
negative effect of uncertainty on MCS effectiveness exists (see 5.4.2). Thus an increase in
uncertainty – within the group of cases with low uncertainty- does not influence misfit, but
does have a negative effect on the perceived effectiveness of control. This result indirectly
supports configurational thinking: with rising uncertainty organizational units experience a
decrease in perceived effectiveness of the control system, but apparently they do not change
their control systems accordingly. Adjustments might only be possible to a certain extent.
Adjusting gradually is impossible and further change would involve a quantum jump. Since
quantum jumps involve large changes to all parts of the control system simultaneously, these
will not occur until the contingencies have changed enough to warrant such a large
investment. This would explain why uncertainty does have a negative impact on perceived
effectiveness of the control system, but not on misfit.
However, I have also argued (5.4.1) that a psychological effect of uncertainty might exist that
negatively influences the perceived performance directly. This is an alternative explanation
for the negative effect of uncertainty on MCS effectiveness as shown in the regression.
Unfortunately, I am not able to distinguish between these two effects of uncertainty.
40
Note that in the analyses in paragraph 5.4 in which the hypotheses are tested, the interaction between
the contingency variables are automatically part of the analysis. The combination of the contingency
variables determines the relevant archetype.
116
Analyses and results
Therefore the total effect of uncertainty might represent just one explanation or combine both.
At this point, too little is known about these complex relationships to enable rigorous testing.
I repeat the exercise for the subsample result oriented machine control. A model explaining
misfit from uncertainty, asset specificity, and their interaction is significant and explains
14.5% of the variance in the misfit variable (adj. R2 = 0.10, N = 63). Table 5.23 shows the
results. Again before running the regression analysis variables were checked for normality,
which was acceptable, and for univariate outliers of which none were found. Moreover, there
are no indications of multicollinearity or heteroskedasticity for this model.
Table 5.23 Model for misfit result oriented machine control
B
.438
std. error
1.057
Beta
(intercept)
T
.415
sig.
.680
uncertainty
.689
.229
.369
3.006
.004
asset specificity
.167
.274
.074
.608
.545
interaction uncertainty/asset specificity
-.058
.076
-.094
-.766
.447
2
2
R = 14.5%, adjusted R = 10%, N = 63
The evidence from the analysis of this subsample reveals a completely different picture. Asset
specificity does not have an influence on misfit, nor does the interaction term. Uncertainty,
however, is positively associated with misfit regarding result oriented machine control. An
increase in uncertainty leads to an increased misfit. Again I will combine this result with the
results from the regression model tested earlier (5.4.3). In that model I found no effect of
misfit on MCS effectiveness. This result seems to support contingency thinking: gradual
change might occur in response to changing circumstances and therefore misfit does not
affect effectiveness. If this were true, the misfit that I measure (misfit regarding the archetype)
is not a true misfit, but represents a fit situation because the level of the contingency variable
and the MCS match. The contingency approach assumes that multiple states of fit exist and
that there is a fit-line along which companies can move. The direct impact of uncertainty on
performance could not be assessed because of multicollinearity, but both variables are
significantly negatively correlated (see 5.4.3). Again, it is impossible to distinguish between
the different effects of uncertainty on performance.
The evidence suggests that in some cases the contingency view of fit is appropriate whereas
in other situations configurational forms of fit exist. In terms of arm’s length control versus
result oriented machine control, this implies that complementarities among the control
instruments might be stronger within arm’s length control. Working with a control structure
117
Chapter five
that resembles result oriented machine control might offer more possibilities to adjust without
disrupting the complete system.
This exploration of the effects of the contingency variables on misfit, highlights a few
interesting points. First of all, uncertainty seems to be the most influential contingency
variable in my study and its overall role is complex. For the complete sample the effect of
uncertainty accords with theoretical expectations from the MCtce: when uncertainty increases
organizational units move away from certain archetypes towards others as predicted. A
second point of interest is that the other contingency variables do not play a role in any of the
analyses. Finally, my exploration reveals that the relationship between uncertainty and misfit
differs between the subsample arm’s length control and machine control. This indicates a
possible difference in strength in complementarities between both archetypes.
5.6 Effectiveness of control and organizational effectiveness
Hitherto I have studied several related topics regarding management control system design
and effectiveness, apart from testing specific hypotheses related to the MCtce. This paragraph
also covers a related theme. In this study I have measured perceived effectiveness of both the
management control system and the organizational unit. I have also concluded earlier that
researchers generally measure organizational effectiveness and use it as a proxy for
effectiveness of control rather than measuring control effectiveness directly (cf. Chenhall,
2003). The proxy might be rather crude, because of the many other possible influences on
organizational performance apart from the functioning of the control system. The relationship
between both is implied rather than demonstrated empirically. Consequently, little is known
about the effects of the workings of the MCS on the organization.
In this paragraph the relationship between the two variables is explored. I have argued that the
two variables are likely positively correlated and that causal relationships might exist between
them, the direction of which is unclear (see 5.4.1). In the overall sample, the effectiveness of
the unit positively correlates with MCS effectiveness (r = 0.472, p = 0.01). The strength of
this relationship varies among the different subsamples related to the archetypes. Table 5.24
shows the correlation between both effectiveness measures for the overall sample and for
each of the subsamples.
118
Analyses and results
Table 5.24 Correlation coefficients: MCS effectiveness and unit effectiveness
1
complete sample
N
258
correlation
0.472
sig.
0.01
2
subsample arm’s length control
3
subsample action oriented machine control
120
0.424
0.00
19
0.601*
4
0.00
subsample result oriented machine control
66
0.532
5
0.00
subsample exploratory control
24
0.549*
0.00
6
subsample boundary control
* nonparametric test; Spearman’s Rho
15
-0.050*
0.86
In all but one sample MCS effectiveness and effectiveness of the organizational unit are
positively correlated. Their relationship is quite strong with correlation coefficients ranging
from 0.42 to 0.60. Within the group of cases that belong to boundary control, the variables are
unrelated.
There are two possible explanations for the apparent differences in strength among the
groups. One explanation is that other variables are involved that have an influence on one of
the effectiveness variables, but not the other within the subsamples. Another possibility would
be that the MCS measured only pertains to a small part of the unit. Multiple activities are
executed within one unit and I only measure the MCS of one of them. If differences among
activities exist regarding the control system, the direct link between the MCS mapped and the
effectiveness of the unit might be smaller. To the contrary, if the control system pertains to a
large part of the organizational unit or even to the complete unit, the relationship might be
stronger.
Apart from these general ideas about the differences in the relationship between MCS
effectiveness and effectiveness of the organizational unit among the subsamples, another
possible explanation exists for the boundary control cases. As mentioned before, only very
strong effects would be picked up in small samples and the subsample boundary control only
contains 15 cases. However, in all the other samples the effects are very strong. Therefore, I
suggest another explanation that relates to the specific essence of this control structure. The
MCtce submits that the archetypes are each effective given the circumstances, but they also
represent the best we can do and boundary control is characterized as “a control structure of
last resort” (Speklé, 2001a). If managers indeed perceive this control structure as such to be
less effective than other forms of control, this might explain that lack of correlation between
perceived MCS effectiveness and perceived unit effectiveness. Whether the performance of
the unit is either good or bad, the performance of the MCS is always perceived to be relatively
poor. As a consequence no direct link between both is observed.
119
Chapter five
From these findings I conclude that using organizational effectiveness as a proxy for control
effectiveness is rather crude, and leaves out a lot of information. Focusing on effectiveness of
control directly will render more insights into the workings of control systems. It will be
interesting to study the causal relationship between the two effectiveness variables more
extensively in future research.
5.7 Summary of findings
This chapter discussed the final steps of the methodological framework and showed how I
have determined the relevant archetype for each observation in the sample. This returned five
subsamples, which I firstly compared on general characteristics. Next, I described the testing
of the hypotheses. Two out of five hypotheses could be tested by means of regression
analysis. The others could not be tested this way, but some insights were gained on the
relationships between the variables of interest through bivariate analyses. Finally, three topics
were addressed in an exploratory way: the occurrence of (mis)fit, the relationships between
the contingency variables and misfit, and the relationship between MCS effectiveness and
effectiveness of the organizational unit. In this paragraph I will summarize the results. The
next chapter presents the overall conclusions of this research project, reflections on the
findings, and a discussion.
The allocation of cases to each of the five subsamples returned distinct groups that differ in
terms of the characteristics of their activities. This is important to assure correspondence
between the theoretical and empirical levels of analysis. Because of the variety in activities
within the overall sample, I started by comparing these groups on general characteristics to
learn more about them. The comparison highlighted some differences but did not return
distinct profiles. However, the findings were not counterintuitive either and stressed the
importance of focusing on activities instead of on unit types.
As a second step in exploring differences among the subgroups, I have compared the scores of
the misfit variables. My approach is highly exploratory and provided tentative results. Firstly,
I have compared the average score on misfit regarding each of the archetypes between each of
the subgroups and all other cases. For four subsamples cases on average resemble the relevant
archetype more under the predicted circumstances than under other circumstances. Next, I
compared scores on the misfit variables within each sample to determine which archetype
cases resemble most and least. Almost 35% of cases can be correctly predicted by the theory
in terms of the archetype they resemble most. Only 7% of cases show less resemblance to
their relevant archetype than to any of the other archetypes. The theory helps to explain (part
120
Analyses and results
of) these results. Thus, the theory takes us one step ahead in understanding which MCSs
prevail under what circumstances. However, without reference to MCS effectiveness the
theoretical ideas cannot be tested.
To test the main theoretical claims directly, five hypotheses were set up, two of which could
be tested by means of regression analysis. Based on the regression model for arm’s length
control, I find evidence that misfit regarding this archetype has a negative impact on the
perceived effectiveness of the control system. This effect is confined to the group of cases that
experience low levels of both uncertainty and asset specificity. The results are robust to shifts
of the boundaries and hypothesis H1 is being supported.
To test the second hypothesis, which relates to result oriented machine control again
regression analysis is used. I find no evidence of a negative effect of misfit regarding result
oriented machine control for MCS effectiveness. This null result is robust to changes in the
boundaries that determine the relevant archetype. Hypothesis H2 is not supported.
The other three hypotheses could not be tested by means of regression analysis because of the
small sizes of the subsamples. Therefore, correlations among the variables of interest were
calculated. Within the group of cases that have action oriented machine control as their
relevant archetype, I find evidence for a positive relationship between misfit and
effectiveness. The less the MCS resembles the archetype, the more effective it will be. This
contradicts hypothesis H3. Within the group of cases that have exploratory control as their
relevant archetype, I find some evidence for a positive relationship between misfit and
effectiveness and some evidence that suggests there is no relationship between the variables.
The effect is sensitive to decisions regarding the level of ex-post information asymmetry.
When I incorporate cases with higher levels of information asymmetry, I find no influence of
misfit on MCS effectiveness. However, overall I have found no evidence that meets the
theoretical expectations. Hypothesis H4 is not supported. Finally, within the group boundary
control, I find no evidence for a negative correlation between misfit regarding boundary
control and MCS effectiveness. Therefore hypothesis H5 is not supported either. These
analyses do not provide rigorous tests of the hypotheses but rather tentative results, because of
the small sample sizes and the restriction to bivariate analysis as a consequence thereof.
Apart from the results obtained by direct tests of the MCtce, additional results were obtained.
The exploration of the effects of the contingency variables on misfit revealed that the role of
uncertainty in my study is both influential and complex. For the complete sample the effect of
uncertainty accords with theoretical expectations from the MCtce: when uncertainty increases
121
Chapter five
organizational units move away from certain archetypes towards others as predicted. The
relationship between uncertainty and misfit differs between the groups arm’s length control
and result oriented machine control, which could be an indication of differences in strength of
complementarities of each archetype.
As a final related topic, I studied the relationship between MCS effectiveness and the
effectiveness of the organizational unit. Results indicate that using organizational
effectiveness as a proxy for control effectiveness is rather crude, and leaves out a lot of
information. Focusing on effectiveness of control directly will render more insights into the
workings of control systems. All results will be discussed in the next chapter.
122
Chapter six - Conclusion and discussion
With this research project I contribute to the existing knowledge base by providing an
empirical study on management control systems in their entirety and the effectiveness thereof.
It is well recognized that studying control systems as packages of interrelated control
elements is necessary for understanding the workings of control as well as its effects on
performance (Otley, 1980; Flamholtz, 1983; Holmstrom and Milgrom, 1994; Abernethy and
Chua, 1996). Empirical work that addresses these issues, however, is scarce. The goal of my
research project is to learn about management control system design and effectiveness. My
central research question is: which control systems are effective given the circumstances?
This broad question indicates the nature of this study, which is theory driven yet exploratory
in many respects. Providing an answer to the research question is not the sole contribution of
my research project. Another contribution lies in the specific approach taken on enabling the
study of MCS design and effectiveness as such. My study takes us a step closer to finding
answers to the general problem of design of management control systems and exemplifies the
type of study that can take research within the field forward. It also reveals the complexities
surrounding this research problem.
Studying the research question has led to a discussion of many topics, both theoretical and
methodological. As a point of departure, I build directly from the work by Speklé (2001a,
2001b, 2004) whose Transaction Cost Theory of Management Control gives a theoretical
answer to the research question. By putting the theory up to testing, I have gained insights
into the empirical answer as well. Moreover, operationalizing the theoretical ideas involved
theory refinement. Along the way, several issues of broader importance were addressed
regarding MCS design, the survey method, and scale construction. In this chapter I will
summarise and discuss the findings. Firstly, paragraph 6.1 addresses the research question. It
provides a summary of the main findings and discusses the limitations of my study. Next, I
focus on methodological issues that were central to this project and are of broader importance
(6.2). Finally, in paragraph 6.3 I will discuss ideas for future research based on several
findings that relate to the research topic more broadly and that have been addressed in an
exploratory way.
6.1 The effectiveness of control systems
This paragraph presents the aggregation of findings from this doctoral thesis and formulates
an answer to the research question. I will summarise the results and discuss the limitations of
my study.
123
Chapter six
6.1.1 Summary of findings
To research the central question of this project – which MCSs are effective – I have tested
hypotheses based on the Transaction Cost Theory of Management Control (MCtce; Speklé,
2001a, 2001b, 2004). This theory provides a theoretical answer to the research question. It
submits that there are five effective forms of management control systems referred to as
archetypes of control: arm’s length control, result oriented machine control, action oriented
machine control, exploratory control, and boundary control. Each is effective only for the
control of certain activities, but not for others. In this study I have looked at organizational
units with various activities. Examples are production, human resources, research and
development, and sales. These activities are characterised over three dimensions: the
associated level of uncertainty, asset specificity, and ex-post information asymmetry. The
effectiveness of the archetypes is contingent on these characteristics. This makes each
archetype relevant for the control of certain activities. For instance, arm’s length control is the
most effective control structure for activities with low uncertainty and moderate asset
specificity. The main claim of the theory is: the closer an observed MCS resembles its
relevant archetype, the more effective it will be.
To obtain an empirical answer to the research question, I set out to test this main claim for
each of the archetypes. Data collection by means of a survey returned 258 observations. For
every observation misfit scores regarding each of the archetypes have been calculated. To
enable hypotheses testing the overall sample was split up into five subsamples, one for every
archetype. Before testing the theory’s main claim, I have compared the misfit variables and
the subsamples to explore the predictive power of the theory. Firstly, the results of this initial
analysis will be discussed.
As argued in chapter two (2.3.2), on average cases are expected to resemble the archetypes because that is effective-, yet states of misfit will occur as well. Cases that do show misfits
will be less effective. A first look at the data returned some general findings. First of all,
within the overall sample I have found a move away from arm’s length control and result
oriented machine control towards the other archetypes in case of a rise in the level of
uncertainty. Thus, the average case resembles the archetypes exploratory or boundary control
more in case of high uncertainty. This is as theory would predict. Testing for similar effects of
asset specificity and ex-post information asymmetry returned insignificant results.
I have also compared the average misfit scores among the subsamples. For four out of five
subsamples the average misfit regarding the relevant archetype within the group, is lower
124
Conclusion and discussion
compared to the average misfit of all other cases. Misfit regarding result oriented machine
control forms the exception. For instance, within the subsample arm’s length control, the
average misfit regarding this archetype is significantly lower than the average misfit regarding
arm’s length control among all other cases. This finding once more indicates a link between
the contingency variables and the misfit variables. The circumstances related to misfit seem to
matter in ways that can be expected based on the theory.
Finally, I have compared scores on the misfit variables within each subsample and determined
which archetype cases resemble most and least. The results indicated that almost 35% of
cases can be correctly predicted by the theory in terms of the archetype they resemble the
most. Only 7% of cases show less resemblance to their relevant archetype than to any of the
other archetypes.
Combined these initial findings show a general tendency of management control systems to
resemble their relevant archetype more closely under the specified circumstances. The theory
has some predictive power and takes us one step forward in understanding which MCSs
prevail under what circumstances. This relates directly to the problem of understanding
control structure variety. However, without reference to differences in effectiveness between
cases that do and cases that do not resemble the archetypes, the main claim of the theory
cannot be tested. Thus, to assess the effects of misfit on performance, I have tested hypotheses
directly related to the theory’s main claim.
The subsamples for arm’s length control and result oriented machine control were large
enough to test the impact of misfit on MCS effectiveness by means of regression analysis.
The hypotheses related to these groups are repeated here for ease of reference:
H1:
In case of low uncertainty and moderate asset specificity, resemblance of an MCS
with the archetype arm’s length control is positively associated with MCS
effectiveness.
H2:
In case of low uncertainty, high asset specificity, and high measurability of outputs,
resemblance of an MCS with the archetype result oriented machine control is
positively associated with MCS effectiveness.
The results of the analyses support the first hypothesis. Under the specified circumstances, an
observed MCS that resembles the archetype arm’s length control more closely is relatively
more effective. I have found no support for the second hypothesis. Robustness checks showed
125
Chapter six
that both results are robust against decisions regarding the boundaries set on contingency
variables to determine the relevant archetype for each case.
A first conclusion is that in circumstances of low uncertainty and low asset specificity,
resemblance with the archetype arm’s length control has a positive effect on MCS
effectiveness. Arm’s length control is thus effective for the control of activities that show the
following characteristics: activities are programmable, outputs can be measured, and
performance can be compared to the market. An effective MCS is characterized by use of
market benchmarks and relatively high work autonomy. There is also little discussion related
to the coordination of work. This result pertains to half the cases in my sample. The cases
studied stem from various industries and companies and therefore this conclusion seems to be
valid for a large variety of units as long as their activities have the characteristics as specified.
Another conclusion that follows straight from the analyses is that resemblance with result
oriented machine control does not impact MCS effectiveness. This does not support the
theoretical expectations. Apart from the possibility that the theory might not hold in this
situation, other explanations can be given for this result.
One explanation for the null result I have found for result oriented machine control might lie
in the underlying assumptions of the configurational approach. Recall that the configurational
approach assumes that only few states of fit exist. These are represented by the archetypes.
Complementarities among the elements of control exist within the MCSs and therefore
changing circumstances will not lead to gradual adjustments of the control structure, but a
quantum jump will occur (see 2.3.2). An explanation for the null result would be that the
complementarities within the result oriented machine control structure are rather weak. If this
is true, gradual adjustments are possible up to a certain extent. Put differently, a threshold
level of misfit might exist that only when reached will lead to a decrease in MCS
effectiveness. Some tentative results illustrate this idea. I have tested for a linear relationship
between uncertainty and misfit within the subsample result oriented machine control and
found a negative relationship between both variables. I also examined a scatter plot of
uncertainty and misfit to assess the appropriateness of assuming a linear relationship. The
evidence suggests that gradual adjustments to the MCSs within this subsample are possible
without having negative consequences for the perceived effectiveness of the MCS. It might be
that the complementarities within the control structure result oriented machine control are
indeed relatively weak. This result is limited to the range of misfit scores that occurs within
my sample. I did not find evidence for a threshold level of misfit, but since my sample did not
hold cases that show extreme misfits regarding the archetype the threshold level might not be
126
Conclusion and discussion
reached. This way of thinking has implications for future research. These are discussed in the
third paragraph (6.3).
There is another explanation for the lack of results for misfit regarding result oriented
machine control. It seems that multiple more of less equally effective control systems exist
within the subsample. However, these might differ in terms of their costs. In a situation of low
uncertainty and high asset specificity, managers potentially rely on a diverse set of control
instruments ranging from rules and regulations to specific output targets. With many potential
combinations of control instruments, the same level of control might be established in several
ways, but at different costs. A test of the relationship between misfit and efficiency of control
(instead of effectiveness of control) might therefore have rendered different findings. It could
be that several combinations of control instruments are effective, but only one - the archetype
- is the most efficient. Not measuring the costs of control systems therefore is a limitation of
my study as will be discussed in the next subparagraph (6.1.2).
Apart from arm’s length control and result oriented machine control, I have studied the other
archetypes. Hypotheses H3, H4, and H5 could not be tested by means of regression analysis
because the subsamples are too small. I could only assess bivariate relationships between
misfit and MCS effectiveness within each of the subsamples. Those hypotheses are also
repeated here:
H3:
In case of low uncertainty, high asset specificity, and low measurability of outputs,
resemblance of an MCS with the archetype action oriented machine control is
positively associated with MCS effectiveness.
H4:
In case of high uncertainty and low ex-post information asymmetry, resemblance of
an MCS with the archetype exploratory control is positively associated with MCS
effectiveness.
H5:
In case of high uncertainty and high ex-post information asymmetry, resemblance of
an MCS with the archetype boundary control is positively associated with MCS
effectiveness.
To learn about these archetypes, I firstly studied the misfit variables as such. I have found that
there are no cases in my sample that closely resemble the archetypes action oriented machine
control, exploratory control, or boundary control. The subgroups for these archetypes were
only small, so I might have found cases that resemble the archetypes in larger samples.
127
Chapter six
Bivariate analyses returned the following results. Some of the evidence does not support the
theory and some even contradicts the theory. Within the group of cases that have action
oriented machine control as their relevant archetype, the evidence for the relationship between
misfit and effectiveness contradicts hypothesis three. For the group of cases that have
exploratory control as their relevant archetype, results are unstable. In the initial sample, a
positive effect of misfit on MCS effectiveness showed up, which contradicts hypothesis H4.
Robustness checks, however, showed that this result disappears and the effect becomes
insignificant when I adjust the boundary to incorporate cases with higher levels of ex-post
information asymmetry. The smallest number of cases falls into the group boundary control
(15 cases). I find no evidence for a negative correlation between misfit regarding boundary
control and MCS effectiveness.
Caution is warranted when interpreting these results. The tests of the hypotheses H1 and H2
are convincing in the sense that the samples are relatively large, results are robust to boundary
decisions, and no problems occurred within the models that could disrupt the relationship
between misfit and effectiveness. The results for the other three hypotheses, however, are
based on bivariate analyses and small samples. Recall that the misfit variables represent
profiles with approximately 20 underlying variables. Studying 15 – 25 cases therefore cannot
provide more than tentative results. Moreover, only bivariate relationships between misfit and
MCS effectiveness have been investigated, whereas I expect the interrelationships among the
variables to be more complex. Within the various subsamples multiple correlations between
the variables as specified in the general model exist. MCS effectiveness and effectiveness of
the organizational unit correlate positively, and uncertainty correlates with multiple variables.
Conducting tests for the complete model, takes into account the impact of the relevant
variables simultaneously and therefore provides a more rigorous test of the hypotheses.
Having said that, the evidence for hypotheses H3, H4, and H5 does not support the theoretical
ideas. Regarding my research question, it cannot be concluded that MCSs that resemble the
archetypes action oriented machine control, exploratory control, or boundary control are more
effective than other MCSs in circumstances of high uncertainty or high asset specificity.
All in all, some findings support the theoretical ideas: the overall effect of uncertainty on
misfit regarding the different archetypes, the general tendency to resemble the relevant
archetype more closely than other archetypes (which can be related to the specific
circumstances), and support for the hypothesis related to arm’s length control. Other findings
do not support the theory: resemblance with result oriented machine control does not impact
control effectiveness. The bivariate analyses for action oriented machine control, exploratory
128
Conclusion and discussion
control and boundary control do not support the theory either. However, the effects of
resembling any of the latter archetypes remain unclear due to small sample testing.
Regarding the research question -which MCSs are effective given the circumstances-, I thus
find that management control systems that resemble the archetype arm’s length control are
effective for the control of activities with low uncertainty and low asset specificity.
During this research project, the tests of the theoretical ideas were preceded by theory
refinement. Keating (1995) distinguishes several forms of theory refinement in light of case
study research. One of these forms (theory specification case) describes the process I went
through in my research project: “(to) specify the theory by adding greater precision to
theoretical constructs and propositions and rendering the theory into a refutable form”
(Keating, 1995, p70). Here I refer to theory refinement in a broad sense and address some
points of interest. One element of fine-tuning relates to the distinction between the two types
of machine control. I suggest using the measurability of outputs for differentiating the action
and result oriented type. Moreover, in the process of operationalizing the ideas some of the
theory’s assumptions have been made explicit. For instance, the view that equifinality among
the archetypes is not assumed to exist, which has direct consequences for the way the theory
can be tested. Finally, during the operationalization the substance of the archetypes has been
defined more precisely. All archetypes are defined over the same set of variables. Surely, this
is not the only possible way to define the archetypes (also see 6.1.2), but it forms a basis for
future research and further development of the specific contents of the archetypes.
6.1.2 Limitations
When interpreting the results of this study, a number of limitations should be taken into
account. Some limitations are inherent to survey research, others specifically relate to this
research project. Both will be discussed in this subparagraph.
First of all, I provide no tests of causal relationships. The data are cross-sectional and
therefore both the design of the management control system and its effects (in terms of
effectiveness of control) are measured at one point in time. Longitudinal case studies provide
better means to investigate causal relationships and to study the development of control
systems and the effects of these systems over time. Within survey research it is difficult to
take a time lag into account because it is unknown how long the time lag should be. This
might even differ among the large variety of organizational units within my dataset.
129
Chapter six
Secondly, only two out of five hypotheses could be tested by means of regression analysis
because of small subsamples. Although I tried to select cases according to their activities and
aimed for as much variety as possible, the number of cases for the subsamples action oriented
machine control, exploratory control, and boundary control were limited. More data would
enable a more rigorous test of these hypotheses.
Another limitation relates to the measurement of MCS effectiveness. My objective was to
develop a measure that is generally applicable. Therefore, first of all, to measure effectiveness
I make use of perceptions instead of ‘hard data’. In my case, it is impossible to work with
objective measures, since I am comparing such a large variety of organizational units
throughout the hierarchy. The only way to compare effectiveness of control among them is to
ask the manager directly for an assessment. In chapter three I have argued that this approach
fits my research design, since I study the relative effectiveness of various MCSs. However, a
more fundamental problem exists for the measurement of MCS effectiveness. I have
discussed the notion of equifinality (see 2.3.1) and argued that equifinality cannot be assumed
to exist among the archetypes. The archetypes each represent the most effective structure
given the specific activities, but this does not imply they are equally effective. It might even
be that effectiveness should therefore be measured differently within each of the subsamples
related to the archetypes. Recall that in measuring MCS effectiveness I ask about the
achievement of underlying control goals, such as guiding manager’s behaviour or supporting
managers in their decision making. Some of these might be more relevant for one of the
archetypes, than for another. For instance, the main goal of a boundary control system could
be to set boundaries and not to motivate employees. Because of the small subsamples I was
not able to research such differences. I have two suggestions to improve the measure for
future use: the first one is to attach weights to the goals. Apart from asking for an assessment
of each item, an indication of its importance to the organizational unit might provide more
precise measurement. That complicates the measure a lot though. Another way to go would be
to tailor the measure to a specific situation (when studying a homogenous group of activities,
for instance) or to determine from theory which items to combine for different situations.
As already mentioned when discussing the findings for result oriented machine control, I have
assessed the effectiveness of the control system, not its efficiency. This is also a limitation of
my study. Throughout this research project I have focused on effectiveness of control systems
as my central concern. However, also taking the costs of different control systems into
account could render different and additional insights. For instance, when a manager uses
more control instruments or the same ones more intensively than necessary to establish an
effective control system two things might happen: the system could be distorted and people
130
Conclusion and discussion
might start to obstruct it (1). The effectiveness would decrease accordingly. It could also
return a better effectiveness up to the point of perfect control (2). Both situations will render
excess costs while only one shows a decrease in effectiveness. These aspects could be picked
up by assessing the efficiency of the MCSs. Thus, the costs of MCSs should be taken into
account somehow. An example of an aspect used as a proxy for the costs of control can be
found in Widener (in press). In her study of the levers of control framework, she uses four
items to measure management attention as a proxy of the costs of a control system. This is
only one aspect of the costs of a management control system, but could be a starting point for
the future development of a measure of MCS efficiency.
Another issue to be kept in mind when interpreting the results of this study, is that it could
have been conducted differently. As the author of the MCtce himself put it: “… the application
of this theory is bound to command considerable interpretative efforts from the researcher to
deal with the inevitable shades of grey” (Speklé, 2001, p 439). These shades of grey relate to
the definition of the archetypes, the measurement of misfit, and the assumed linkages between
misfit and effectiveness. The definition of the archetypes could also have been set up
empirically, or through an iterative process confronting theory and empirics. Another example
of choices made concerns the measurement of misfit. In the calculation of misfit, weights
were attached to separate parts of the control structure. Another interpretation of the theory
might have resulted in a different weighting. Also, I have assumed a linear relationship to
exist between misfit and MCS effectiveness whereas other relationships might exist. Although
my choices are consistent with the theory, other possible operationalizations could also have
been consistent with the theory.
A final limitation concerns the measurement of some of the variables 41. Although
considerable effort has been put into designing and testing a survey instrument that delivers
data of high quality, with hindsight I conclude that some variables could have been measured
better. First of all, the measurement of asset specificity can be improved. I was unable to
measure all aspects of this construct and combine them into a MIMIC model that represents
best the underlying structure of the concept. As a consequence I was left with the second best
solution and thus with less precise measurement of asset specificity. I capture only a part of
the construct instead of the whole and therefore the effects of the variable might be bigger
than could be assessed in the analyses. However, it is difficult to assess the true impact on the
findings of this study, because I cannot compare the first and second best solutions. Also
41
Appendix B gives an overview of the questions used in this study. I have added a note to those
questions that might be improved.
131
Chapter six
important is the role of benchmarks. In the questionnaire I only picked up on general use of
benchmarks whereas it would have been better to ask explicitly about their role in
performance evaluation and rewarding as well. This has implications for the distinction
between arm’s length control and result oriented machine control, and could have separated
the two types better. Another aspect that I would have liked to assess in more detail is the
percentage bonus. There was little variety in the answers for this variable and for future use of
the question I suggest adjusting the answer categories to differentiate better between smaller
and larger bonuses.
6.2 Results related to methodology
Throughout this doctoral thesis, several methodological themes have been discussed that are
not only important to this particular study, but to research within management accounting and
control generally. This paragraph presents a summary of the main concerns and implications
for future research. Respectively I will discuss (1) the survey method, (2) measurement of
MCS effectiveness, and (3) measurement of uncertainty and asset specificity or, more
generally, constructs with formative indicators.
The first theme concerns the survey method. Several authors concluded that the survey
method is valuable and valid, but more concern must be given to its correct application (see
for instance Ittner and Larcker, 2001; Zimmerman, 2001). Improper use of the method results
in research that is based on data of poor quality. In chapter three I have showed how the
critique on survey studies within accounting can be dealt with successfully by (1) establishing
as close a match as possible between the theory and operationalization and by (2) putting my
questionnaire through an extensive pre-test. Apart from making use of available methods for
pre-testing, I have introduced and used validity-feedback interviews. This new technique
proves a powerful method to obtain feedback on the content validity of questionnaires. To
improve overall survey research within our field, researchers should make use of the available
techniques for questionnaire design and pre-testing, and not underestimate the intricacies of
the method.
A second recurrent theme has been the measurement of MCS effectiveness. I have developed
a measurement instrument to assess the perceived effectiveness of the management control
system. Starting from the idea that it should be able to pick up the achievement of underlying
goals of control, multiple items have been identified that in combination measure
effectiveness (limitations of this instrument have been discussed in 6.1.2). For two reasons
direct measurement of MCS effectiveness is preferred over measuring the effectiveness of the
132
Conclusion and discussion
organizational unit and using that as a proxy for control effectiveness, which most studies do.
First of all, the proxy is rather crude as illustrated by the correlation coefficients between both
effectiveness measures. The values did not exceed 0.60 (for the exact numbers see table 5.24).
A second reason is that the relationship between MCS effectiveness and effectiveness of the
organizational unit as such deserves our attention. If we believe that the management control
system affects the performance of organizations through its influence on employee behaviour,
empirical evidence helps to prove and understand this relationship. However, little is known
about it and thus several opportunities for future research exist. In the next paragraph I will
discuss some of these opportunities.
The final methodological theme that arose from my research project concerns the construction
of measurement models with formative indicators. Several authors have stressed the
importance of distinguishing formative from reflective indicator models and showed that
wrong conclusions can result from misspecification (see for instance MacKenzie et al., 2005;
Diamantopoulos and Siguaw, 2006; Bisbe et al. (in press)). I suggest a formative approach in
measuring two key TCE variables. For both uncertainty and asset specificity I set up a general
MIMIC model (Jöreskog and Goldberger, 1975). For uncertainty I was able to estimate the
model and use it for scale construction. The construction of a MIMIC model for asset
specificity failed, but improved measurement, especially of the reflective indicators, might
lead to better results. The formative approach fits the essence of both constructs that each
have multiple guises. Within TCE based studies, researchers mostly focus on single aspects of
uncertainty and asset specificity, for instance human asset specificity (Rindfleisch and Heide,
1997), or they take in multiple aspects separately. The measurement of uncertainty has been
very diverse and Boerner and Macher conclude that “empirical findings that relate uncertainty
to organizational form are mixed, partly because of the multitude of uncertainty types
examined” (2001, p6). The measurement of uncertainty has been criticized and researchers
were called upon to pay extra attention to it (Boerner and Macher, 2001; David and Han,
2004). More precise measurement of both asset specificity and uncertainty is called for and I
suggest a formative approach to enable this. A MIMIC model facilitates the study of different
aspects simultaneously and can therefore provide us with a more comprehensive measure of
these constructs.
6.3 Future research - another point of departure
During the search for an answer to the research question several themes emerged that broadly
relate to the study of MCS design and effectiveness. In this paragraph I discuss the main
points of interest and make suggestions for future research. In four subparagraphs I will
133
Chapter six
elaborate on configurational fit and complementarities, the role of uncertainty, the
relationship between MCS effectiveness and effectiveness of the organizational unit, and
finally, the design and effectiveness of MCSs. These topics will be presented in separate
paragraphs, but they are very much intertwined.
6.3.1 Configurational fit and complementarities
Central to this research project is configurational thinking. I study multiple contingency
variables and multiple structure variables (combined in the archetypes), and only a discrete
number of fit states are thought to exist. According to the configurational view, change of
management control systems will occur through quantum jumps rather than gradually (Gerdin
and Greve, 2004). These quantum jumps involve large changes to all parts of the control
system simultaneously, and will not occur until the contingencies have changed enough to
warrant the considerable investments involved. Thus, an MCS will not be adjusted with every
change in the contingencies, which leads to a loss in performance. An opposed view is
provided by the contingency approach. According to this approach change in organizational
structure does accord with changes in contingency variables and will occur gradually.
In chapter five, I have studied the relationship between the contingency variables and the
misfit variables. This analysis returned different results for the largest subsamples: arm’s
length control and result oriented machine control. Within the subsample arm’s length control
I found what I expected adhering to the configurational view: there is no direct relationship
between the contingency variables and misfit, but there is a negative influence of misfit on
performance. To the contrary, for result oriented machine control I have found no effect of
misfit on performance, but a linear relationship between uncertainty and misfit. This would fit
the contingency view better.
Both results seem to be explained best by different views. However, these two views are
positioned as opposing views that cannot be combined (Donaldson, 2001; Gerdin and Greve,
2004). The discussion of these results has led to the idea of differences in strength of
complementarities between MCSs and the possible existence of a threshold level of misfit.
This might be a third view towards fit and effectiveness. If a threshold level of misfit exists,
gradual adjustments are possible up to a certain extent, but at some point the threshold level is
reached and fundamental changes are needed in order to stay effective. My exploratory
analysis did not provide evidence for a threshold level of misfit within the subsample result
oriented machine control, but the level might not have been reached.
134
Conclusion and discussion
Incorporating this idea of differences in strength of complementarities and thus a threshold
level of misfit, provides a slightly different view on fit within the configurational approach.
Figure 6.1 illustrates configurational fit as usually described (panel A; Gerdin and Greve,
2004) and configurational fit with the possibility of differences in strength in
complementarities between different archetypes (panel B). The circles represent states of fit
and the size of the circles indicates the flexibility of each system or rather the weakness of the
complementarities within each system. The stronger the complementarities among the control
instruments are, the smaller the circle is. Within the circles gradual adjustments are possible.
Positions outside the circles always imply states of misfit.
Figure 6.1 Configurational fit: different strengths in complementarities
Panel A
Panel B
S
T
R
U
C
T
U
R
E
S
T
R
U
C
T
U
R
E
CONTINGENCY
CONTINGENCY
Note: the size of the circles indicates the strength of the complementarities: the larger the circle, the
weaker the complementarities within the archetype. Points within the circles represent states of fit;
points outside the circles represent states of misfit. Source panel A: Gerdin and Greve, 2004, p306.
This idea of a threshold level of misfit as a third view to states of fit, misfit, and effectiveness,
might help us to understand the design of organizational structures and to explain empirical
observations. There is little empirical work that focuses on these different views of fit and
studies which approach provides a better explanation for the workings and effectiveness of
organizational structures. Future research might try to compare these different views in a
single study and shed light on the issue.
6.3.2 Uncertainty and the perception of control
In my research project, the role of uncertainty turned out to be both influential and complex.
Apart from its role as a contingency variable used to characterise the activities of
organizational units, the variable seems to influence the perceived effectiveness of the control
system directly. In three out of five subsamples uncertainty correlates negatively with MCS
135
Chapter six
effectiveness. Within the subsamples exploratory control and boundary control I have found
no significant relationship between these variables, but these subsamples are small and only
strong effects would be picked up. There are two potential explanations for this negative
effect of uncertainty on performance: the first relates to changes in the management control
system and states of misfit, the second relates to a psychological effect. Both will be
discussed next.
The first potential explanation stems directly from configurational thinking. As discussed in
the previous paragraph, adhering to the configurational approach an MCS is expected not to
change gradually with changes in contingency variables. Thus, a rise in uncertainty would not
be accompanied by a change in the control structure (as long as the change is not large
enough to justify a quantum jump). However, this change in the level of uncertainty would
cause the MCS no longer to fit the new situation, which in turn leads to a decrease in
performance. In that case a direct negative effect of uncertainty on MCS effectiveness occurs.
As a second potential explanation I have argued that a psychological effect of uncertainty
might exist. This could influence the manager’s assessment of control effectiveness, which is
relevant because I measure perceived effectiveness. The general idea is that in case of high
uncertainty managers feel less comfortable with their management control system even it is
the most effective system given the circumstances. Situations of high uncertainty do not come
with a wide range of options in terms of suitable control instruments. Consider, for instance, a
control structure like boundary control, which hinges on prohibitive behavioural guidelines.
Instead of being able to rely on precise work instructions or pre-set performance targets,
managers cannot but tell their employees what not to do or what to avoid. This likely leaves
them dissatisfied with the effectiveness of the control system even if it is the most effective
given their situation. On the contrary, in a situation of low uncertainty managers could rely on
rules, regulations, pre-set performance targets and the like, and might feel more ‘in control’
than managers that cannot. The evidence I have found within the subsample result oriented
machine control supports this view. Within that subsample I found an effect of uncertainty on
both misfit and MCS effectiveness, but no effect of misfit on MCS effectiveness. Apparently,
the effect of uncertainty does not follow from a state of misfit -like described above-, but
might be a result of the manager’s dissatisfaction with the MCS as such.
Future studies could research this psychological effect. This might enhance our understanding
of the effects of control systems as such. Moreover, if the psychological effect of uncertainty
occurs it should be taken into account when comparing the perceived effectiveness of
different management control systems.
136
Conclusion and discussion
6.3.3 Effectiveness of control and effectiveness of the organization
When discussing the results related to methodology, I have already argued that measuring
both effectiveness of control and effectiveness of the organization or organizational unit is
beneficial. Apart from merely assuming that a positive relationship between the two variables
exists, empirical evidence should be gathered to shed light on this relationship. Two issues
relevant for future research emerged from my study. Firstly, it is unclear what causal
relationship exists between both constructs of effectiveness. Secondly, their relationship
might differ across different situations.
Regarding the first point, I have argued that causality might run both ways. Since the MCS is
supposed to influence employee behaviour in order to accomplish organizational goals, an
effective MCS will likely have a positive influence on the organizational unit. Moreover, a
highly effective organizational unit probably has more resources to spend on its MCS to
improve it. Another effect that might influence the relationship between effectiveness of the
unit and effectiveness of the MCS is again a psychological one. When assessing the perceived
effectiveness of control, a manager of a successful organizational unit might assess the
functioning of his/her control system more positively. This could be due to overall optimism
or to the believe that if one is doing well, the control system must also function well. The
opposite is also true: the MCS will be valued less when the unit is not functioning well. Note
that both effectiveness measures are influenced by many other factors, so that the relations are
probabilistic rather than deterministic.
My analyses showed that MCS effectiveness and effectiveness of the organizational unit are
positively correlated in all but one sample. Their relationship is quite strong with correlation
coefficients ranging from 0.42 to 0.60. Only within the subsample boundary control I did not
find a significant relationship. As a possible explanation I have referred to the essence of
boundary control as a management control structure. In the MCtce the archetype is described
as “a control structure of last resort” (Speklé, 2001a). If managers indeed perceive this control
structure to be less effective than other forms of control, this might account for the lack of
correlation between perceived MCS effectiveness and perceived unit effectiveness within the
subsample boundary control. This triggered the second issue: the relationship between both
effectiveness measures might differ in strength across situations.
Studies that focus specifically on the relationship between the effectiveness of the control
system and unit effectiveness will be highly relevant. They can provide information on the
137
Chapter six
effectiveness of control systems as such as well as on the effects of control systems on the
organization or organizational unit.
6.3.4 MCS design and effectiveness
Several research opportunities have already been discussed, but two issues remain that relate
to MCS design and effectiveness. These will be presented in this final subparagraph.
Research opportunities remain concerning the Transaction Cost Theory of Management
Control. In my study the MCtce provided important insights, but more empirical studies should
test the ideas to enable an assessment of the full impact of this theory. There are several
possibilities. Collecting additional data for the subsamples action oriented machine control,
exploratory control, and boundary control would enable hypotheses testing for these
archetypes. Moreover, the definitions of the archetypes can be compared to empirical profiles
and may be fine-tuned. Apart from a quantitative approach, other methods like case studies
could helpfully be applied to study the archetypes and the activities they control.
Multiple possibilities to contribute exist in the study of management control systems in their
entirety. It is unclear how different control instruments combine to establish control as a set.
Which combinations represent fit states and will thus be effective, and which ones cannot
survive? Issues that deserve our attention are relationships among the elements of control that
make up the system. The exact combinations that are used should be studied. Also, do the
control instruments function as complements or substitutes, and are some systems
characterised by strong complementarities whereas others are not? Would this depend on the
circumstances, or on the structure itself? More empirical work might show how the separate
control instruments within an MCS operate together.
All in all, a lot remains to be learned about management control system design and
effectiveness, leaving us with many exiting opportunities for future research. More studies
that explicitly take on a broad view towards control and relate MCS design to effectiveness
will teach us about this topic that lies at the heart of research within the field.
138
References
Abernethy, M.A., J. Bouwens and L. Van Lent (2004). Determinants of control system design
in divisionalized firms. The Accounting Review, 79 (3), pp545-570.
Abernethy, M.A. and W.F. Chua (1996). A field study of control system ‘Redesign’: the
impact of institutional processes on strategic choice. Contemporary Accounting Research, 13
(2), pp569-606.
Aiken, L.S. and S.G. West (1991). Multiple regression: testing and interpreting interactions.
Sage Publications, Newbury Park.
Anderson, E. (1985). The salesperson as outside agent or employee: a transaction cost
analysis. Marketing Science, 4 (3), pp234-254.
Anderson, E. (1988). Transaction costs as determinants of opportunism in integrated and
independent sales forces. Journal of Economic Behavior and Organization, 9, pp247-264
Anderson, E. and D.C. Schmittlein (1984). Integration of the sales force: an empirical
examination. RAND Journal of Economics, 15, pp385–395.
Beatty, P. (2004). Paradigms of cognitive interviewing practice, and their implications for
developing standards of best practice. ZUMA-Nachrichten Spezial Band 9, Questionnaire
Evaluation Standards, Centre for Surveys, Methods, and Analyses (ZUMA), Mannheim.
Bedford, D. (2005). Management Control Systems Configurations in Practice. University of
Technology, Sydney (working paper).
Bisbe, J., J. Batista-Foguet and R. Chenhall (2006). Defining management accounting
constructs: a methodological note on the risks of conceptual misspecification. Accounting,
Organizations and Society, in press. doi:10.1016/j.aos.2006.09.010.
Boerner, C.S. and J.T. Macher (2001). Transaction cost economics: an assessment of
empirical research in the social sciences. McDonough School of Business (working paper).
Bollen, K. and R. Lennox (1991). Conventional wisdom on measurement: a structural
equation perspective. Psychological Bulletin, 110 (2), pp305-314.
Bouwens, J. and L. van Lent (2004). Assessing the performance of profit center managers.
University of Tilburg (working paper).
Brownell, P. (1995). Research methods in Management Accounting. Coopers & Lybrand,
Melbourne.
Burton, R. M., J. Lauridsen and B. Obel (2002). Return on assets loss from situational and
contingency misfits. Management Science, 48 (11), pp1461-1485.
Chenhall, R.H. (2003). Management control systems design within its organizational context:
findings from contingency-based research and directions for the future. Accounting,
Organizations and Society, 28 (issue 2-3), pp127-168.
139
References
Coles, J.W. and W.S. Hesterly (1998). Transaction costs, quality, and economies of scale:
examining contracting choices in the hospital industry. Journal of Corporate Finance, 4,
pp321–345.
David, R.J. and S.-K. Han (2004). A systematic assessment of the empirical support for
transaction cost economics. Strategic Management Journal, 25, pp39-58.
Diamantopoulos, A. and J.A. Siguaw (2006). Formative versus reflective indicators in
organizational measure development: a comparison and empirical illustration. British Journal
of Management, 17, pp 263-282.
Diamantopoulos, A. and H.M. Winklhofer (2001). Index construction with formative
indicators: an alternative to scale development. Journal of Marketing Research, 38, pp269277.
Dillman, D.A. (2000). Mail and internet surveys. The tailored design method. Second Edition,
John Wiley and Sons, Inc., New York.
Donaldson, L. (2001). The contingency theory of organizations. Sage Publications, Thousand
Oaks.
Doty, D. H. and W. H. Glick (1994). Typologies as a unique form of theory building: towards
improved understanding and modeling. The Academy of Management review 19 (2), pp230251.
Doty, D. H., W.H. Glick and G.P. Huber (1993). Fit, equifinality, and organizational
effectiveness: a test of two configurational theories. Academy of Management Journal, 36 (6),
pp1196-1250.
Drazin, R. and A. H. v. d. Ven. (1985). Alternative forms of fit in contingency theory.
Administrative Science Quarterly 30, pp514-539.
Dunk, A. (1993). The effects of budget emphasis and information asymmetry on the relation
between budgetary participation and slack. The Accounting Review, 68 (2), pp400-410.
Ferreira, A. and D. Otley (2005). The consequences of management control systems design
and use (working paper).
Fisher, J. (1995). Contingency-based research on management control systems: categorization
by level of complexity. Journal of Accounting Literature, 14, pp24-48.
Flamholtz, E.G. (1983). Accounting, budgeting and control systems in their organizational
context: theoretical and empirical perspectives, Accounting, Organizations and Society, 8
(2/3), pp153-169.
Gerdin, J. (2005a). The impact of departmental interdependencies and management
accounting system use on subunit performance. European Accounting Review, 14 (2), pp297327
Gerdin, J. (2005b). The impact of departmental interdependencies and management
accounting system use on subunit performance: a second look. European Accounting Review,
14 (2), pp335-340
Gerdin, J. and J. Greve (2004). Forms of contingency fit in management accounting research a critical review. Accounting, Organizations and Society, 29, pp303-326.
140
References
Gresov, C. (1989). Exploring fit and misfit with multiple contingencies. Administrative
Science Quarterly, 34 (3), pp431-453.
Hair, J.F.(Jr), R.E. Anderson, R.L.Tatham and W.C. Black (1998). Multivariate data analysis.
Fifth Edition, Prentice-Hall, Inc., New Jersey.
Hak, T., K. van der Veer and H. Jansen (2004). The Three-Step Test-Interview (TSTI): an
observational instrument for pretesting self-completion questionnaires. ERIM Report Series
Research in Management.
Hartmann, F.G.H. (2005). The impact of departmental interdependencies and management
accounting system use on subunit performance: a comment. European Accounting Review, 14
(2), pp329-334
Hitt, M.A., R.D. Ireland, B.W. Keats and A. Vianna (1983). Measuring subunit effectiveness.
Decision Sciences, 14 (1), pp87-102.
Hitt, M.A. and R.D. Middlemist (1979). A methodology to develop the criteria and criteria
weightings for assessing subunit effectiveness in organizations. The Academy of Management
Journal, 22 (2), pp356-374.
Hofstede, G. (1981). Management control of public and non-for-profit activities. Accounting,
Organizations and Society, 6 (3), pp193-211.
Holmstrom, B. and P. Milgrom (1994). The firm as an incentive system. The American
Economic Review, 84 (4), pp972-991.
Ittner, C.D. and D.F. Larcker (2001). Assessing empirical research in managerial accounting:
a value-based management perspective. Journal of Accounting and Economics, 32 (1-3),
pp349-410.
Jansen, H. and T. Hak (2005). The productivity of the three-step test-interview (TSTI)
compared to an expert review of a self-administered questionnaire on alcohol consumption.
Journal of Official Statistics, 21 (1), pp103-120.
Jarvis, C.B., S.B. MacKenzie and P.M. Podsakoff (2003). A critical review of construct
indicators and measurement model misspecification in marketing and consumer research.
Journal of Consumer Research, 30 (2), pp199-218.
Jensen, M.C. and W.H. Meckling (1992). Specific and general knowledge, and organizational
structure. In: L. Werin and H. Wijkander (eds.), Contract economics, pp251-274, Basil
Blackwell, Oxford.
Jöreskog, K.G. and A.S. Goldberger (1975). Estimation of a model with multiple causes of a
single latent variable. Journal of the American Statistical Association, 10, pp631-639.
Keating, A.S. (1997). Determinants of divisional performance evaluation practices. Journal of
Accounting and Economics, 24, pp243-273.
Keating, P.J. (1995). A framework for classifying and evaluating the theoretical contributions
of case research in management accounting. Journal of Management Accounting Research, 7
(Fall), pp66-86.
King, G., R.O. Keohane and S. Verba (1994). Designing Social Inquiry. Scientific inference
in qualitative research. Princeton University Press, New Jersey.
141
References
Langfield-Smith, K. and D. Smith (2003). Management control systems and trust in
outsourcing relationships. Management Accounting Research, 14, pp281-307.
Lattin, J., J.D. Carroll and P.E. Green (2003). Analyzing multivariable data. Thomson
Learning, Inc., Pacific Grove.
Luft, J. and M.D. Shields (2003). Mapping management accounting: graphics and guidelines
for theory-consistent empirical research. Accounting, Organizations and Society, 28, pp169249.
MacKenzie, S.B., P.M. Podsakoff and C.B. Jarvis (2005). The problem of measurement
misspecification in behavioral and organizational research and some recommended solutions.
Journal of applied psychology, 90 (4), pp710-730.
Merchant, K.A. (1982). The control function of management. Sloan Management Review,
summer, pp43-55
Milgrom, P. and J. Roberts (1990). The economics of modern manufacturing: technology,
strategy, and organization. The American Economic Review, 80 (3), pp511-528.
Milgrom, P. and J. Roberts (1995). Complementarities and fit strategy, structure, and
organizational change in manufacturing. Journal of Accounting & Economics, 19, pp179-208.
Mintzberg, H. (1983). Structures in fives: designing effective organizations, Prentice Hall,
Englewood Cliffs.
Newbold, P. (1995). Statistics for business & economics. Fourth edition, Prentice-Hall, Inc.,
New Jersey.
Nicholson, B., J. Jones and S. Espenlaub (2006). Transaction costs and control of outsourced
accounting: case evidence from India. Management Accounting Research, 17, pp238-258.
Otley, D.T. (1980). The contingency theory of management accounting: achievement and
prognosis. Accounting, Organizations and Society, 5 (4), pp413-428.
Otley, D., J. Broadbent and A. Berry (1995). Research in management control: an overview of
its development. British Journal of Management, 6 (special issue), S31-S44.
Ouchi, W.G. (1979). A conceptual framework for the design of organizational control
mechanisms. Management Science, 25 (9), pp833-848.
Ouchi, W.G. (1980). Markets, bureaucracies and clans. Administrative Science Quarterly, 25
(1), pp129-141.
Price, J.L. (1997). Handbook of organizational measurement. International Journal of
Manpower, 18 (4/5/6), pp305-558.
Rainey, H.G. (1983). Public agencies and private firms; incentive structures, goals, and
individual roles. Administration and Society, 15, pp207-242.
Rindfleisch, A. and J. Heide (1997). Transaction cost analysis: past, present and future
applications, Journal of Marketing, 61 (October), pp30-54.
Shelanski, H.A. and P.G. Klein (1995). Empirical Research in Transaction Cost Economics: a
Review and Assessment. The Journal of Law, Economics, & Organization, 11, pp335-361.
142
References
Simon, H.A. (1957). Models of man, social and rational: mathematical essays on rational
human behavior in a social setting. John Wiley & Sons Inc., New York.
Simons, R. (1995). Levers of control: How managers use innovative control systems to drive
strategic renewal. Harvard Business School Press, Boston.
Smith, A. (1776). The wealth of nations, edited by Edwin Cannan, 1904. Reprint: New York,
Modern Library, 1973.
Snell, S.A. (1992). Control theory in strategic human resource management: the mediating
effect of administrative information. The Academy of Management Journal, 35 (2), pp292327.
Speklé, R.F. (2001a). Explaining management control structure variety: a transaction cost
economics perspective. Accounting, Organizations and Society, 26, pp419-441.
Speklé, R.F. (2001b). Beyond Generics: a closer look at hybrid and hierarchical governance.
ERIM Ph.D. series Research in Management 8, Erasmus Research Institute of Management,
Erasmus University Rotterdam.
Speklé, R.F. (2004). Configurations of control: a transaction cost approach (working paper).
Speklé, R.F., H.J. Van Elten and A. Kruis (2007). Sourcing of internal auditing: an empirical
study. Management Accounting Research, 18 (1), pp102-124.
Van de Ven, A.H. and D.L. Ferry (1980). Measuring and assessing organizations. John Wiley
& Sons, New York. WWW Reprint 2000, University of Minnesota. http://umn.edu/~avandeve
Van den Bogaard, M.A. and R.F. Speklé (2003). Reinventing the hierarchy: strategy and
control in the Shell Chemicals carve-out. Management Accounting Research, 14, pp79-93.
Van der Stede, W.A., S.M. Young and C.X. Chen (2005). Assessing the quality of evidence in
empirical management accounting research: The case of survey studies. Accounting,
Organizations and Society, 30, pp655-684.
Verbeeten, F.H.M. (2005). Performance management in government and non-profit
organizations: a goal setting theory perspective. Erasmus University, Rotterdam (working
paper).
Widener, S.K. (2004). An empirical investigation of the relation between the use of strategic
human capital and the design of the management control system. Accounting, Organizations
and Society, 29, pp377-399.
Widener, S.K. (2007). An empirical analysis of the levers of control framework. Accounting,
Organizations and Society, in press. doi:10.1016/j.aos.2007.01.001.
Williamson, O.E. (1975). Markets and hierarchies: analysis and antitrust implications. Free
Press, New York.
Williamson, O.E. (1985). The Economic Institutions of Capitalism. Free Press, New York.
Williamson, O.E. (1991). Comparative Economic Organization: the analysis of discrete
structural alternatives. Administrative Science Quarterly, 36 (2), pp269-296.
143
References
Williamson, O.E. (1996). The mechanisms of governance, Oxford University Press, New
York.
Willis, G.B. (2005). Cognitive interviewing: a tool for improving questionnaire design.
Chapter 13, Sage Publications, Thousand Oaks.
Zimmerman, J.L. (2000). Accounting for Decision Making and Control. Third edition,
MacGraw-Hill Companies, Inc., Singapore.
Zimmerman, J.L. (2001). Conjectures regarding empirical managerial accounting research.
Journal of accounting and economics, 32 (1-3), pp411-427.
144
Appendix A - Description of a management control system
In this appendix I describe the management control system of an organizational unit of a
recruitment agency. The agency searches job applicants for their customers, who are
companies in need of personnel with a higher education. This description was used for a
validity-feedback interview (see 3.3) as part of the pre-test of the questionnaire. I asked the
manager to fill out the questionnaire. Next, solely based on his answers, I described the MCS
of the unit and its effectiveness as well as characteristics of the unit’s main activity.
Numerical scores on items were translated back into sentences.
This description is based on the final version of the questionnaire and therefore illustrates the
type of information as well as the amount of detail the questionnaire generates. In the
questionnaire, I refer to organizational units as subunits and therefore this terminology is used
below. Appendix B gives an overview of the exact questions posed.
I
Appendix A
Description: management control system of a recruitment unit
March 2006
Part one – the main activity of the subunit
The main activity of this subunit is ‘recruitment’. This activity is programmable to a large
extent. It is very easy to identify an understandable sequence of steps to be followed in
carrying out the work. It is very easy as well to get an overview of actions to take at the start
of a period. Moreover many of the tasks of employees are the same from day-to-day. It is
quite difficult, however, to think of one best way of doing things.
The activity is also somewhat predictable. Unpredictable variety in the workload occurs
frequently, but exceptions in the work occur rarely. Day-to-day situations are somewhat
different.
It is quite easy to analyse the activities: the manager is sure almost all the time (>90%) about
what the outcomes of work efforts are going to be. It is quite easy for him to assess the work
of employees. Also for the supervisor it is quite easy to assess the manager’s as well as the
employees’ work and to define specific goals for the subunit at the beginning of period. The
outputs of the subunit can be expressed in numbers, but it is quite difficult to reach agreement
on their quality.
There is no goal ambiguity within the unit: unit goals are very clear and specific. Moreover,
goals are very clear to everyone who works in the unit and very easy to explain to outsiders. It
is neither easy nor difficult for a new employee to learn the ins and outs. Tools and equipment
used are unique to a little extent compared to those used by other companies for similar
activities, but the systems used are quite unique. Compared to employees of other companies
that work on similar activities, skills and knowledge of subunit employees are unique to a
little extent. Quite a bit coordination is needed between employees. This unit and other units
within the company depend somewhat on one another.
II
Description of a management control system
Part two – management control within the unit
The control system described below pertains to 22 people.
2.1 Organizational structure (as regards unit employees)
Within the unit coordination of work mainly takes place through standard operating
procedures/work instructions and proposals made by employees. Sometimes general
assignments and planning are used, whereas specific assignments are used little. Work-related
discussions between the manager and the employees occur a couple of times a month,
whereas discussions among employees take place more than once a day.
Employees have quite a bit of freedom in deciding on how to do their job, and how to handle
work exceptions. They also have a say over what happens on their job and take part in making
decisions that affect them. However, employees have little freedom in deciding what they do
on their job. Overall it can be said that employees have quite a bit work autonomy.
2.2 Use of standards for unit employees
Performance targets are most important in guiding employees in their work (rated ‘very
important’). Standard operating procedures/work instructions and behavioural guidelines are
somewhat important in this respect. The budget also acts as a performance target. Prohibitive
guidelines that explicitly specify what is prohibited or what should be avoided are not in use
in this unit.
The only performance targets used for employees’ jobs are revenue targets and these are used
very much. There are no targets related to social or professional skills. Normally, targets are
set at the beginning of a period and do not change much. Whereas market benchmarks are not
used for employees, benchmarks related to internal peer group performance or to the
performance of other units are used quite a bit for employees’ jobs.
2.3 Performance evaluation of unit employees
The unit manager uses subjective judgements somewhat to assess employee performance.
Employees’ contribution to long term subunit and long term company performance is
considered to be quite important in performance evaluation. Employees have some
information advantage compared to the unit manager (score 2.4 on a five point scale). They
are significantly better informed about the impact of both internal and external factors on their
activities. The have quite some information advantage as regards the type of activities they
III
Appendix A
have undertaken and their (potential) performance. They have no advantage regarding internal
processes or technical aspects of the work.
In order for employees to obtain a positive evaluation achievement of performance targets is
very important. Showing good social and professional skills is quite important. Compliance
with work instructions or behavioural guidelines is less relevant in this respect. The same
holds true for being evaluated negatively.
2.4 Rewards and incentives for unit employees
Employees receive bonuses based on both their individual performance and on the
performance of their subunit. Bonuses can make up 16–30% of their total financial reward.
The only criterion for being granted a bonus (rated ‘very important’) is the achievement of
performance targets.
When performing well, increased responsibilities and recognition from both peers and
superiors are important non-financial rewards. Apart from this, employees receive some
increased autonomy. Achievement of performance targets and professional skills are very
important for getting a non-financial reward. Showing good social skills is quite important,
but compliance with instructions/behavioural guidelines not at all. For employees’ career
prospects, again achievement of performance targets is most important. Skills are quite
important and compliance with work instructions and other behavioural guidelines is of little
importance for employees’ careers.
There is little focus on the group within the unit (or much focus on the individual). Employees
always compete over performance targets. Sometimes they make someone an outsider whose
work is below that of the others or encourage each other.
Part three - control of the unit manager
3.1 Organizational structure (as regards the manager)
Of all time spent on managerial activities, the subunit manager spends most time on staffing,
directing and motivating employees, as well as on interacting with other managers. Least time
is spent on establishing strategies for the unit. Work related discussions with other managers
or supervisors occur a couple of times a month.
IV
Description of a management control system
Overall the manager has quite some work autonomy (average score: 4.4 on a 5 point scale).
He has very much influence in investment, marketing, production, and human resource
decisions as well as in setting the agenda for the unit. He has also very much influence in
deciding what tasks are to be performed within the unit. He has quite a bit influence in setting
targets for subunit performance and designing the reward system. However, he has no
influence in deciding upon criteria for employee performance appraisal.
3.2 Use of standards for the unit manager
Whereas for employees’ jobs one type of performance targets was in use, for the manager
different types occur. The most important ones are profit, revenue, and cost targets. Apart
from these, non-financial quantitative targets are used quite a bit. There are no qualitative
targets or targets related to skills. Also for the manager the budget serves as a performance
target.
Normally targets are set at the beginning of a period and do not change much. Market
benchmarks as well as benchmarks related to internal peer group performance or to the
performance of other units are somewhat in use. Behavioural guidelines that specify explicitly
what should be avoided or what is prohibited do not exist for the manager either.
3.3 Manager’s performance evaluation
The unit supervisor uses subjective judgements quite a bit to assess the manager’s
performance. The supervisor considers the manager’s contribution to both long term subunit
and long term company performance to be quite important in performance evaluation. At the
end of a period the manager has quite some information advantage compared to the supervisor
(score 3.1 on a five item scale), especially regarding the impact of internal and external
factors on his activities, and the type of activities undertaken in the unit. He is also better
informed about the internal processes and about what has been achieved in the unit. He has no
advantage regarding technical aspects of the work.
In order for the manager to obtain a positive evaluation, achievement of performance targets
is considered to be very important. Showing good skills is little important and compliance
with behavioural guidelines and work instructions not at all. For a negative evaluation the
picture is somewhat different: achievement of performance targets remains the most
important aspect, but lack of professional skills and violation of rules/behavioural guidelines
are quite important as well. Lack of social skills is somewhat important.
V
Appendix A
3.4 Reward structure for the manager
The unit manager receives stocks/options/profit shares. These amount to 31-45% of his total
financial reward. The only thing that is important to get rewarded is achievement of
performance targets. For career prospects, however, professional and social skills are
important too. Compliance with work instructions/behavioural guidelines is somewhat
important for the manager’s career.
Part four - effectiveness
Overall the effectiveness of the subunit is more than satisfactory (average score 5.5 on a 1-7
scale). The quantity of work produced gets the highest possible score and is thus ‘very good’.
Other aspects, for example satisfaction of external customers, employee satisfaction, and the
problem-solving ability of the unit, are graded more than satisfactory or good. Satisfaction of
internal customers and compliance with instructions/guidelines get the lowest scores and are
rated satisfactory.
The effectiveness of the management control system (MCS) of the unit is also assessed. The
MCS is the whole of all aspects of organizational structure, use of standards, performance
evaluation, and reward structure as described above.
When assessing different aspects of the MCS we see that about half the aspects are rated more
than satisfactory whereas the other half scores less than satisfactory. Accordingly, the overall
score is satisfactory (4.3 on a 7 point scale).
Items that get high scores are motivating employees and managers, perceived fairness of the
system, amount of work autonomy employees and managers have, and use of financial
rewards. Items that are rated less than satisfactory are the coordination of work, guiding
employees and managers, use of non-financial rewards and behavioural guidelines, and
monitoring.
The contribution of the MCS to overall subunit performance is also perceived to be poor. This
judgement is underlined when confronting strategic goals of the unit with the support of the
MCS to achieve these. It can be concluded that although being flexible and efficient are both
quite important, the MCS does not facilitate these aspects at all. The same holds true for being
innovative and improving internal business processes. Both are somewhat important, but not
supported by the MCS.
VI
Appendix B - Overview of survey questions
This appendix holds an overview of all questions that underlie the analyses in this study. The
questions pertain to management control systems and their effectiveness as well as to the
main activity and effectiveness of organizational units. Questions are identified by their
original number and ordered by construct/variable. For each I indicate the scale, the items,
and the original source. When no source is stated, the question has been newly developed for
this study. Occasionally, a remark on use is added as well. The questions are part of a larger
questionnaire that addresses management control system design and effectiveness both at the
level of the organizational unit and within organizational units. A copy of the complete
questionnaire is to be found at http://www.nyenrode.nl/faculty/faculty.cfm?faculty_id=201.
VII
Appendix B
General information
1) A subunit is defined as a manager and all employees reporting directly or indirectly
to him/her. What is the name of your subunit?
2) Please state the industry that your subunit is in:
3) What is the appropriate description of your subunit?
items: Division, Business unit, Production department, Marketing or sales, Research and
development, Treasury, Finance & accounting, ICT, Human resources, Project unit/
team, Other support, Other
4) Please indicate the number of steps up the hierarchy from your position to the CEO.
Example: in the picture this would be 3.
note: a copy of the picture referred to is to be found in chapter three (3.2.2)
5) Please indicate the number of steps down the hierarchy of your unit from your
position to the bottom level. Example: in the picture this would be 2.
note: idem question 4
46) Including yourself, how many employees work for this subunit? Please recall that
the subunit consists of you and all employees that report directly or indirectly to you.
You may include people hired from other companies if they report to you.
47) Question 25 asked you to choose one activity as the main activity of your subunit.
How many employees within your subunit work on the main activity?
Activities of the organizational unit
24) The following questions are about what employees in your subunit actually do. We
refer to this as the activities of your subunit. Examples of activities are R&D, human
resources, or production of a specific good or service. Recall that your unit comprises
you as a manager and all employees that report directly or indirectly to you. Your
subunit might clearly have one main activity. It might also have more activities that are
equally important. Please name the main activities of your subunit:
25) Which activity is the most important or the most characteristic one for your unit?
Please choose one activity:
VIII
Overview of survey questions
Management Control Systems
7) Are the following performance targets used for your job?
scale: not at all, little, somewhat, quite a bit, very much
items: Stock price related targets, Return targets (examples: ROI, EVA, shareholder value
added), Profit targets (examples: net income, profit margin, operating profit),
Revenue targets, Cost targets, Non-financial quantitative targets (examples: market
share, output quota), Qualitative targets (examples: customer satisfaction, service
quality)
source: types of performance targets based on Bouwens and Van Lent (2004, p15)
8) Please indicate whether the following types of performance benchmarks are used for
your job.
scale: not at all, little, somewhat, quite a bit, very much
items: a. Market benchmarks or benchmarks based on performance of external peer
groups
b. Benchmarks based on performance of internal peer groups or the performance of
other subunits
note: to learn about use of benchmarks more broadly this question does not suffice. It
informs us only about whether or not benchmarks are being used as such. No
information on how these are being used is obtained nor on their role in, for
instance, performance evaluation.
9) Which of the following statements describes best how your performance targets come
about? Please check one box.
items: a. Performance targets are set at the beginning of a period and will not change
much under normal circumstances
b. Performance targets come about steadily over time/ evolve during a period
11) The subunit’s budget can play several roles. How important are the following roles
of the budget? Please assign a number 1 to 3, where 1 implies most important.
items: a. The budget serves as a performance target to be met
b. The budget gives guidance about the way to go
c. The budget sets limits to what can be done
12) The next two questions (12 & 13) are about the things your supervisor takes into
consideration when evaluating your performance. Some things might be crucial for
either a positive or a negative evaluation, other things might be less important. How
important are the following things for you to get a positive evaluation?
scale: not at all important, little important, somewhat important, quite important, very
important
items: Social skills, Professional skills, Achievement of performance targets, Budget
versus actuals, Compliance with standard operating procedures/ work instructions,
Compliance with other behavioural guidelines
source: inspired by Bouwens and Van Lent (2004), Verbeeten (2005), and Van de Ven and
Ferry (1980)
IX
Appendix B
13) How important are the following things for you to get a negative evaluation?
scale: not at all important, little important, somewhat important, quite important, very
important
items: Lack of social skills, Lack of professional skills, Not achieving performance targets,
Budget versus actuals, Violating standard operating procedures/ work instructions,
Violating other behavioural guidelines
source: idem question 12
14) How much does your supervisor use subjective judgements to evaluate your
performance?
scale: not at all, very little, somewhat, quite a bit, very much
15) When evaluating your performance, does your supervisor take into consideration
your contribution to long term subunit and long term company performance?
scale: not at all, little, somewhat, quite a bit, very much
items: Long term subunit performance, Long term company performance
16) Two types of behavioural guidelines might be in use for your job. Guidelines that
specify what should be done, and guidelines that specify what should be avoided or what
is prohibited. We refer to the latter as prohibitive guidelines. Do such prohibitive
guidelines apply to you?
items: no (skip to 19), yes
17) (If yes) General prohibitive guidelines hold for all employees of the company.
Examples are general ethical norms, or rules about private use of the Internet. Subunitspecific prohibitive guidelines are relevant to the performance of your subunit.
Examples are authorisation levels and procedures, or policy restrictions. Do subunitspecific prohibitive guidelines apply to you?
items: no (skip to 19), yes
18) (If yes) Please indicate whether you agree or disagree with the following statement:
“Violating subunit-specific prohibitive guidelines will always have serious consequences
for my career.”
scale: disagree strongly, disagree somewhat, neutral, agree somewhat, agree strongly
19) Do you receive any financial rewards apart from a fixed salary?
items: no (skip to 23), yes
21) What percentage of your financial reward is variable?
items: 1 – 15%, 16 – 30%, 31 – 45%, 46 – 60%, more than 60%
note: in my sample variation in the answers to this question were limited. For future use
of this question, I recommend adjusting the answer categories to incorporate smaller
ranges.
X
Overview of survey questions
22) When do you receive a variable financial reward? Please indicate the importance of
the following things for this to happen.
scale: not at all important, little important, somewhat important, quite important, very
important
items: Social skills, Professional skills, Achievement of performance targets, Budget
versus actuals, Compliance with standard operating procedures/ work instructions,
Compliance with other behavioural guidelines
source: idem question 12
23) Please indicate the importance of the following things for your career prospects.
scale: not at all important, little important, somewhat important, quite important, very
important
items: Social skills, Professional skills, Achievement of performance targets, Budget
versus actuals, Compliance with standard operating procedures/ work instructions,
Compliance with other behavioural guidelines
source: idem question 12
42) Please also indicate for the following decisions how much say or influence you have.
scale: none, little, some, quite a bit, very much
items: a. Deciding what work or tasks are to be performed in your unit
b. Setting standards/ targets for subunit performance
c. Deciding upon criteria for performance appraisal of employees
d. Designing the reward system
e. Deciding upon standard operating procedures/ work instructions
f. Deciding upon other behavioural guidelines
source: items a-d; Van de Ven and Ferry, 1980, Q25 (p435), Q57 (p445), Q63 (p446), Q79
(p450); modified
56) During the past 3 months, how often did discussions related to the coordination of
work (face-to-face, by telephone or e-mail) occur on a one-to-one basis?
scale: once a month or less, couple of times a month, about once a week, every day, more
than once a day
item: C (one item used): between you and other managers or supervisors
source: Van de Ven and Ferry, 1980, Q86 (p453); scale modified
ASSET SPECIFICITY
Physical asset specificity
29) How much of the equipment in your subunit is especially designed or unique
compared to equipment used for similar activities in other companies? Examples of
equipment are machinery, tools, or warehousing.
scale: none, some, about half, most, all
source: Coles and Hesterly, 1998; modified
30) How unique are the systems used in your subunit compared to systems used by other
companies for similar activities? Examples are communication systems or software.
scale: not at all unique, unique to a little extent, somewhat unique, quite unique, very
unique
XI
Appendix B
Interdependencies
35) To what extent do your subunit’s actions impact on work carried out in other
subunits of your firm?
scale: to no extent, little extent, some extent, great extent, very great extent
source: Keating, 1997
36) To what extent do actions of other subunits of the firm impact on work carried out
in your particular subunit?
scale: to no extent, little extent, some extent, great extent, very great extent
source: Keating, 1997
Asset specificity in general
39) Please respond to the following statement by ticking one box.
“My company could go to other companies to buy the products/ services my subunit
delivers.” (reverse scored)
items: a. It is very difficult/ impossible to obtain these products or services from other
companies
b. Possible, but price and quality conditions would be worse (quality would be less
and/ or the price higher)
c. It would be no problem to obtain the goods/ services elsewhere at about the
same price and quality
d. Possible, price and quality conditions would be better (at a better quality and/ or
lower price)
40) Please indicate whether you agree or disagree with the following statement.
“Outsourcing the activities of this subunit would imply a major strategic change for the
company.”
scale: disagree strongly, disagree somewhat, neutral, agree somewhat, agree strongly
note: in my sample this question is unable to discriminate between respondents: there is a
lack of variety in the answers
Human asset specificity
49) How easy is it for a new subunit employee to learn the ins and outs he/ she needs to
know to be effective?
scale: very easy, quite easy, neither easy nor difficult, quite difficult, very difficult
source: Anderson and Schmittlein, 1984; modified
50) Consider a new employee who has experience in a similar job/ profession. How
much extra training would that person need? Please estimate the number of weeks of
training, including training on-the-job.
source: Anderson and Schmittlein, 1984; modified
52) How unique are the skills and knowledge of subunit employees compared to skills
and knowledge of employees of other companies who work on similar activities?
scale: not at all unique, unique to a little extent, somewhat unique, quite unique, very
unique
XII
Overview of survey questions
UNCERTAINTY
Programmability
26) At the start of a period, how easy is it to get an overview of actions to take in order to
carry out successfully the subunit’s main activity?
scale: very easy, quite easy, neither easy nor difficult, quite difficult, very difficult
Goal ambiguity
31) How clearly defined are the goals of this subunit? (reverse scored)
scale: very unclear, quite unclear, somewhat clear, quite clear, very clear
source: Rainy, 1983
32) How specific are the goals of this subunit?
scale: very general, mostly general, somewhat specific, quite specific, very specific
33) How easy is it to explain the goals of this subunit to outsiders? (reverse scored)
scale: very easy, quite easy, neither easy nor difficult, quite difficult, very difficult
source: Rainy, 1983
34) Please indicate whether you agree or disagree with the following statement. “The
goals of this subunit are clear to almost everyone who works in the subunit.”
(reverse scored)
scale: disagree strongly, disagree somewhat, neutral, agree somewhat, agree strongly
source: Rainy, 1983
Unpredictability
37) How often do the following things happen?
scale: never, rarely, sometimes, frequently, always
items: a. Unpredictably, the amount of work varies greatly over time. (example: peak
demands)
b. Exceptions arise in your subunit that require substantially different methods or
procedures
source: item B; Van de Ven and Ferry, 1980, Q6 (p433)
38) How much the same are day-to-day situations, problems or issues you encounter in
your subunit? (reverse scored)
scale: completely different, quite a bit different, somewhat different, mostly the same, very
much the same
source: Van de Ven and Ferry, 1980, Q3 (p432)
XIII
Appendix B
Uncertainty in general
44) How easy is it for your supervisor to know whether you and your employees do the
work correctly?
scale: very easy, quite easy, neither easy nor difficult, quite difficult, very difficult
source: Van de Ven and Ferry, 1980, Q4 (p433); modified
83) How easy is it to reach agreement with superiors when judging/ appraising the
quality of subunit performance?
scale: very easy, quite easy, neither easy nor difficult, quite difficult, very difficult
Measurability of outputs
82) Can the outputs of the subunit be measured objectively and expressed in a number?
Please keep in mind to focus on the main activity.
scale: not at all, very little, somewhat, quite a bit, very much
note: this measure as such picks up ‘measurability of outputs’; reverse scored is it a part
of uncertainty
Information asymmetry
45) Please compare the amount of information you have relative to your supervisor at
the end of a period. If you have no advantage or a disadvantage, tick “no advantage”.
scale: no advantage, some, quite some, significant, very significant advantage
items: a. How much better informed are you about the type of activities that have been
undertaken in your subunit?
b. How much more familiar are you with the type of input output relations inherent
in the internal operations of your subunit?
c. How much more certain are you about whether the performance potential of
your subunit has been realized?
d. How much more familiar are you with the technical aspects of the work in your
subunit?
e. How much better are you able to assess the impact of internal factors on your
activities?
f. How much better are you able to assess the impact of external factors on your
activities?
g. How much better do you understand what has been achieved in your subunit?
source: all items except item f: based on Dunk, 1993; modified
XIV
Overview of survey questions
Effectiveness of the organizational unit
84) How would you rate the performance of your subunit on the following aspects?
Please feel free to write “don’t know” or “not applicable” after an item when
appropriate.
scale: 1 (very poor), 2, 3, 4 (satisfactory), 5, 6, 7 (very good)
items: a. Achievement of subunit goals
b. Cooperation with other subunits
c. Satisfaction of internal customers of your subunit
d. Satisfaction of external customers of your subunit
e. Employee satisfaction
f. Compliance with standards and behavioural guidelines
g. Problem-solving ability of the subunit
h. The quantity or amount of work produced
i. The quality or accuracy of work produced
j. Overall subunit performance
source: items a, h, i: Van de Ven and Ferry, 1980, Q93 (p455); items a, b, e, f, g: based on
Hitt and Middlemist, 1979 and Hitt et al. 1983
Effectiveness of the management control system
Note: see 6.1.2. This measure does not take into account differences between control
structures regarding the importance of control goals. I have two suggestions to improve it: the
first one is to attach weights to the goals. Apart from asking for an assessment of each item,
an indication of its importance to the organizational unit might provide more precise
measurement. That complicates the measure a lot though. Another suggestion is to tailor the
measure to a specific situation (when studying a homogenous group of cases, for instance) or
to determine from theory which items/goals are relevant for different situations.
85) The organizational structure, use of performance targets or behavioural guidelines,
and the reward and performance evaluation system combined form the management
control system. How would you rate the performance of the management control system
of your subunit on the following aspects?
scale: 1 (very poor), 2, 3, 4 (satisfactory), 5, 6, 7 (very good)
items: a. Motivating employees
b. Motivating managers
c. Supporting managers in decision making
d. Guiding employees’ behaviour
e. Guiding managers’ behaviour
f. Coordination of work
g. Perceived fairness
h. Contribution to overall subunit performance
source: item h: Ferreira and Otley, 2005; modified
86) How would you rate the following aspects of your subunit?
scale: 1 (very poor), 2, 3, 4 (satisfactory), 5, 6, 7 (very good)
items: a. Amount of work autonomy employees have
b. Amount of work autonomy managers have
c. Use of financial rewards
d. Use of non-financial rewards
e. Performance evaluation
f. Use of standards / (behavioural) guidelines
g. Monitoring
XV
Samenvatting
Dit proefschrift beschrijft de opzet, uitvoering en bevindingen van een empirische studie naar
het ontwerp en de effectiviteit van management control systemen. Management control
systemen bestaan uit een pakket van control instrumenten die gebruikt worden om het werk te
coördineren en prikkels te geven door middel van belonen en straffen. Voorbeelden van
control instrumenten zijn regels en procedures, budgetten, prestatiemaatstaven en bonussen.
Om control systemen op te zetten, kunnen organisaties kiezen uit een variëteit aan
instrumenten. Niet iedere mogelijke combinatie van control instrumenten zal echter effectief
zijn. De werking van een bepaald instrument hangt namelijk af van de combinatie met de
andere instrumenten. Er is onderlinge samenhang.
Vele onderzoekers hebben zich gericht op individuele instrumenten van control, maar er is
weinig empirisch werk dat control systemen in hun totaliteit bestudeert als pakket van control
instrumenten. Om die reden is er nog weinig bekend over de verschijningsvormen van control
systemen en nog minder over de effectiviteit ervan. Met deze studie draag ik bij aan de
wetenschappelijke kennis door mij specifiek te richten op management control systemen in
hun totaliteit en op hun relatieve effectiviteit. Mijn centrale onderzoeksvraag is: welke
management control systemen zijn -gegeven de omstandigheden- effectief?
De Transactiekosten Theorie van Management Control (Speklé, 2001a, 2001b, 2004)
verschaft inzichten in deze materie en vormt de theoretische basis van dit proefschrift. De
theorie claimt dat er vijf effectieve control vormen (ofwel archetypen) bestaan: arm’s length
control, result oriented machine control, action oriented machine control, exploratory control,
en boundary control. Elk van deze archetypen biedt een effectieve control oplossing voor
bepaalde activiteiten, maar niet voor andere. Voorbeelden van activiteiten zijn de productie
van een goed of dienst, human resources, of research en development. Activiteiten worden
gekenmerkt door hun mate van onzekerheid, de specificiteit van gedane investeringen, en ex
post informatie asymmetrie. De theorie specificeert bijvoorbeeld dat het archetype arm’s
length control het meest effectief is voor de control van activiteiten met een lage onzekerheid
en een lage mate van specificiteit. De theorie claimt dat een management control systeem van
een organisatieonderdeel dat lijkt op zijn relevante archetype, effectiever zal zijn in de control
van zijn activiteiten dan andere control systemen. Welk archetype relevant is, hangt af van de
activiteiten van dat organisatieonderdeel. Ik toets deze claims empirisch voor 258
organisatieonderdelen waarvoor met behulp van een vragenlijst gegevens zijn verzameld.
XVII
Om de hypothesen te kunnen toetsen, moet ik voor elke waarneming eerst bepalen welk
archetype relevant is. Daarom worden alle waarnemingen aan een van vijf subgroepen (een
voor elk archetype) toegewezen op basis van de karakteristieken van hun activiteiten. De
resultaten van een regressieanalyse tonen aan dat gelijkenis van het management control
systeem met het archetype arm’s length control, de effectiviteit van dat control systeem
positief beïnvloedt. Zoals door de theorie voorspeld, treedt dit effect alleen op bij activiteiten
met een lage mate van onzekerheid en een lage mate van specificiteit. Voor de waarnemingen
die in de groep result oriented machine control vallen, vind ik een nul resultaat: er is geen
verband tussen de gelijkenis van een management control systeem met het archetype en de
effectiviteit van het control systeem. Voor action oriented machine control, exploratory
control en boundary control zijn de subgroepen te klein om een regressieanalyse uit te kunnen
voeren. De bivariate analyses die ik in plaats daarvan heb uitgevoerd, leveren slechts
indicatieve resultaten op die de hypothesen niet bevestigen.
Andere resultaten zijn meer algemeen aan het onderwerp gerelateerd. In de eerste plaats vind
ik een positief verband tussen de effectiviteit van het management control systeem en de
effectiviteit van het organisatieonderdeel. De sterkte van deze relatie verschilt voor de
subgroepen die ieder een van de archetypen representeren. Daarnaast vind ik bewijs voor een
direct negatief effect van onzekerheid op de gepercipieerde effectiviteit van het control
systeem. Het totale effect van onzekerheid is complex omdat de variabele het ontwerp van
control systemen ook direct beïnvloedt.
Met mijn studie lever ik ook een methodologische bijdrage. In dit proefschrift beschrijf ik alle
stappen die nodig zijn voor het operationaliseren van de theorie en voor het in kaart brengen
van management control systemen. Daarmee geef ik een voorbeeld van hoe een studie naar
management control systemen in hun totaliteit en hun effectiviteit opgezet kan worden. Ik
introduceer ook ‘validity-feedback’ interviews als een krachtig instrument om de kwaliteit
van vragenlijsten te testen. Het instrument is algemeen bruikbaar en levert extra informatie
over de kwaliteit van vragenlijsten boven informatie die andere pre-test methoden opleveren.
Ten slotte, om het meten van de variabele onzekerheid te verbeteren heb ik een algemeen
model opgezet met formatieve indicatoren (in plaats van reflectieve). Dit heeft een positief
effect op de validiteit. Deze aanpak kan ook de toekomstige meting van de variabele
‘specificiteit van gedane investeringen’ verbeteren.
XVIII
Curriculum Vitae
After graduating from the Stedelijk Gymnasium in Leiden, Anne-Marie Kruis (1977) studied
both general and business economics at Erasmus University Rotterdam. She specialised in
international economics and business. Anne-Marie has spent one semester studying in
Münster, Germany during which time she grew attached to the people, the language, and the
place, which she still visits regularly. After graduating in 2002, she decided to apply for a
PhD position at Erasmus University where she started working on her PhD project one year
later. She enjoyed her time at Erasmus very much, but decided after two and a half years to
switch to Nyenrode Business University where she continued working on her thesis.
She currently holds a postdoc position at Nyenrode Business University and works on several
research projects. One of them involves an international cooperation with a colleague from
Rice University in Houston, Texas. Anne-Marie’s research interests are in the field of
management control systems and organizational design. She also has great interest in
methodological problems. Although most her time is dedicated to scientific research, she
enthusiastically teaches courses on management accounting and control as well as on
statistics.
XIX