Download Presentation

Conceptualizing and Measuring Institutions:
A View from Political Science
December 15, 2007
Marcus Kurtz
Ohio State University
Andrew Schrank
University of New Mexico
Outline
• Measuring governance: objective v. subjective indicators.
• Limits to subjective indicators:
--Conceptualization: a moving and blurry target.
--Operationalization: pro-business bias and systematic error.
--Measurement: non-independence and random error.
• Objective alternatives?
Objective v. Subjective Measures of Governance
Objective measures of concrete outputs
Tax ratios (Organski & Kugler 1980)
Tax structures (Kling 1968; Krasner 1985)
Public service provision (Migdal 1988; Putnam 1992)
Subjective (perceptions-based) measures of institutions
Bureaucratic quality from ICRG (commercial risk analysts)
Property rights indices from Heritage Foundation (NGO staff)
Bribery data from the IMD (firm surveys)
Corruption perceptions from TI (aggregated assessments)
Aggregate perceptions-based measures: the state of the art
World Governance Indicators (World Bank)
Objective v. Subjective Measures of Governance
Objective measures of concrete output
Tax ratios (Organski & Kugler 1980)
Tax structures (Kling 1968; Krasner 1985)
Public service provision (Migdal 1988; Putnam 1992)
Subjective (perceptions-based) measures of institutions
Bureaucratic quality from ICRG (expert assessments)
Property rights indices from Heritage Foundation (NGO staff)
Bribery data from the IMD (firm surveys)
Corruption perceptions from TI (aggregated assessment)
Aggregate perceptions-based measures: the state of the art
World Governance Indicators (World Bank)
Our focus:
WGI rule of
law (RL)
Conceptualization: a moving target?
What is the rule of law?
Citizen respect for the state and its rules (WGI 1999)
Quality of contract enforcement, the police, and the courts and the
likelihood of crime and violence (WGI 2005)
Extent to which agents have confidence in and abide by the rules of
society and the above (WGI 2006).
Norms of limited governance (Kaufmann et al. 2007)
Does shifting conceptualization have consequences?
The Italian example
Problems of multiple understandings of the term
Evaluating measurement hinges on conceptualization.
Measurement: a pro-business bias?
Questions: What factors into RL?
Crime, courts, cops and property rights.
Crime: Expert and survey data tend to
privilege property crime
Property rights: Public officials use
expropriation to fight crime and
corruption (i.e., Korea; RICO)
Courts and cops: Indicators ignore
other law enforcement agencies (i.e.,
regulatory authorities)
Sources: Whose opinions matter? Most
sources are businesses or their
advisers.
Why is this crucial?
Are businesspeople’s opinions
representative?
Measurement: a pro-business bias?
Questions: What factors into RL?
Crime, courts, cops and property rights.
Crime: Expert and survey data tend to
privilege property crime
Property rights: Public officials use
expropriation to fight crime and
corruption (i.e., Korea; RICO)
Courts and cops: Indicators ignore
other law enforcement agencies (i.e.,
regulatory authorities)
Sources: Whose opinions matter? Most
sources are businesses or their
advisers.
Why is this crucial?
Are businesspeople’s opinions
representative?
Businesspeople differ from others in
evaluating the Rule of Law.
Question
Responses
Business
Trust in the
judiciary
1 = much
trust;
2= some
trust;
3= little trust;
4 = no trust
Odds ratio =
1.15
(p < .010)
Trust in the
police
Odds ratio =
1.23
(p < .001)
Source: Latinobarometer data used in WGI
rule of law indicator.
Ordered logit models; country dummies
suppressed.
NO
Measurement: Non-Independence and Noise
Worldwide Governance Indicators provide estimates of uncertainty
What is the reliability of the rule of law (RL) measure?
Measurement: Non-Independence and Noise
Worldwide Governance Indicators provide estimates of uncertainty
What is the reliability of the rule of law (RL) measure?
All Countries
Rule of Law, 1996
KKM SEs
Rule of Law, 1996
1.5*KKM SEs
Rule of Law, 1996
2.0*KKM SEs
Correlation
0.909 (median)
0.818 (median)
0.716 (median)
Confidence Interval
[0.876 – 0.934]
[0.752 – 0.867]
[0.607 –0.788]
Below median income
Correlation
0.724 (median)
0.538 (median)
0.398 (median)
Confidence Interval
[0.581 – 0.821]
[0.343 – 0.701]
[0.148 –0.600]
What are the consequences of noise for growth models?
Conclusion: bringing objective reality back in?
• Subjective indicators and the
problem of causal mechanisms.
Analogy with self-reported health
data.
• An objective alternative would:
--be issue specific;
--look at measurable outputs;
--include data on inputs;
--differentiate rule of the game
from their application.
• Example: Labor law enforcement
in Latin America.
Conclusion: bringing objective reality back in?
• Subjective indicators and the
problem of causal mechanisms.
Analogy with self-reported health
data.
15
Paraguay
Bolivia
10
Mexico
e(Informality|GDP/cap, job security)
• An objective alternative would:
--be issue specific;
--look at measurable outputs;
--include data on inputs;
--differentiate rule of the game
from their application.
Fig. 4: Labor law enforcement in Latin America: quantity and quality
Peru
Argentina
5
Venezuela
Brazil
Ecuador
0
Uruguay
Nicaragua
0
0.00005
Colombia
0.0001
0.00015
0.0002
0.00025
Panama
Honduras
-5
El Salvador
Dom. Rep.
-10
Costa Rica
e(Informality|GDP/cap + job sec) = -4.3186ln(Inspectors/worker) - 45.5
R2 = 0.4093; t(Inspectors/worker) = -3.22; p < .001
-15
Inspectors per worker
• Example: Labor law enforcement
in Latin America.
Explaining residual informality by
the intensity of enforcement:
Slope as ROI;
residuals as
quality of
enforcement
Chile
0.0003