Download temperature

EVALUATING PUBLIC POLICY -- PERFORMANCE INDICATORS AND CAUSAL REASONING
=> DAVID M. REINER “CAUSAL REASONING AND GOAL SETTING” PHD DISSERTATION 2002
A. CONCRETE ACTIONS -- DEFINE GOALS IN TERMS OF BEHAVIOR
B. INTERVENING STATES OR LINKS -- DEFINE GOALS IN TERMS OF OBSERVABLES ON CAUSAL PATH
C. ULTIMATE OBJECTIVES -- DEFINE GOALS IN TERMS OF CONSUMMATORY VALUES
STRENGTH
WEAKNESS
AIR QUALITY
ANTITRUST
FOOD
SAFETY
EDUCATION
BEHAVIORAL
INDICATORS
ASSIGNING
RESPONSIBILITY
LEGITIMATING
POLICY
INTERVENING
INDICATORS
NOMINAL MEASURABILITY AND
OSTENSIBLE OBJECTIVITY
RELEVANCE TO LEGITIMACY
AND RESPONSIBILITY
CONSUMMATORY
INDICATORS
LEGITIMATING
POLICY
ASSIGNING
RESPONSIBILITY
BEHAVIORAL
INDICATORS
FUEL FORMULA
ENGINE SYS OBD
CATALYST
COLLUSION
TEMPERATURE
HANDLING
HACCP PROCESS
CLASS HOURS
CURRICULUM
TEACHER/PUPIL
SPENDING/PUPIL
TEACHER PAY
INTERVENING
INDICATORS
LDV NOX EMISSIONS
AMBIENT NOX LEVEL
OZONE LEVELS
PRICE CORRELATIONS
E.COLI COUNTS
FOOD POISONING INCIDENCE
CONSUMMATORY
INDICATORS
MORTALITY
HEALTH
ACCEPTANCE&YIELD RATE
AVERAGE SAT (SOME TAKE)
AVERAGE MCAS (ALL TAKE)
DROP OUT PERCENT
COLLEGE BOUND PERCENT
WELFARE/INNOVATION
MORTALITY
HEALTH
QUALITY OF LIFE
WORKFORCE PROD
INTELLECT QUAL
POL DELIBERATION
QUERY: HOW CHOOSE INDICATOR IF CAUSAL STRUCTURES ARE SIMPLE WITH CLEAR LINKS?
A
B
C
QUERY: HOW CHOOSE INDICATOR IF CAUSAL STRUCTURES ARE COMPLEX WITH MULTIPLE
BEHAVIORAL SOURCES, INTERVENING STATES, AND ULTIMATE EFFECTS WITH UNCERTAIN LINKS?
A1
B1
C1
A2
B2
C2
A3
B3
C3
PERVERSE EXAMPLES -- UNINTENDED EFFECTS OF INDICATOR SELECTION
INTERACTION EFFECTS = RELATIONSHIP ACROSS COMPONENTS NOT LINEAR
NOX EMISSIONS REDUCTIONS / VOC CONSTRAINED ENVIRONMENT => OZONE WEEKEND EFFECT
SULFUR IN FUEL / OBD / 3WAY CATALYST => HIGHER NOX AND NITROUS OXIDE
ELECTIONS / ETHNIC TENSION => VIOLENCE
SELECTION EFFECTS = IMPROVE INDICATORS BY CHANGING COMPOSITION OF POPULATION
FOCUS ON SAT SCORES => REDUCE POOL OF STUDENTS TAKING SATS
FOCUS ON MCAS AND OTHER UNIVERSAL TESTS => DUMP SPECIAL NEEDS AND METCO STUDENTS
FOCUS ON FIRM ENVIRO PERFORMANCE => MOVE DIRTY AND DANGEROUS WORK OUT OF FIRM
FOCUS ON RIGHTS OF IMMIGRANTS => KEEP IMMIGRANTS OFFSHORE
PERVERSE EFFECTS OF INCREASING ACCOUNTABILITY -- GAMBLING FOR RESURRECTION
ACTIONS
INTERVENING STATES
EMISSIONS
AMBIENT LEVELS
FUEL ECONOMY STANDARD
ECONOMIC
SECURITY
CO2
TRAPS AND FILTERS
ULTIMATE GOALS
PARTICULATES
WATER QUALITY
FUEL STANDARD OXYGENATE
CO
FUEL STANDARD SULFUR
SOX
ENVIRONMENTAL
QUALITY
CLIMATE CHANGE
OBD AND SENSORS
HUMAN HEALTH
ADVANCED CATALYSTS
NITROGEN OXIDES
AIR QUALITY
CRANKCASE VENTILATION
OZONE
VOCS
PERC DRY CLEANING
IRAQ PERFORMANCE INDICATORS SLIDE OMITTED . . .
BEHAVIORAL
INDICATORS
CORPORATE
* BOARD SELECT
GOVERNANCE * AUDIT STANDARD
INTERVENING
INDICATORS
* PRICE-EARNINGS RATIOS
* COMPENSATION-PE RATIO
* STOCK PRICE CHANGES
POLITICAL
* FRANCHISE
* CORRUPTION SURVEYS
GOVERNANCE. * SECRET BALLOT * KLEPTOCRACY MEASURES
* BID PROCEDURES * ELECTORAL PARTICIPATION
* PRESS (PAPER)
* NUMBER OF PARTIES
* TV (OWNERSHIP) * POL LEGITIMACY SURVEYS
* CONSTIT RIGHTS * ARREST / INCARCERATION
* CIVIL SERVICE
* VIOLENCE INDICES
JUDICIARY
* INFLATION RATE
CONSUMMATORY
INDICATORS
* BERLE-MEANS
* FREEDOM
* SECURITY
* PROSPERITY
IMPROVING PUBLIC POLICY -- FOSTERING ADAPTION AND SELF CORRECTION 12/9/2003
OVERVIEW
CORE PROBLEM
1 PUBLIC POLICIES ARE (NECESSARILY) IMPERFECT EXPERIMENTS
-- Do not have reliable information on critical aspects of problems at the outset
2 ADAPTATION OF POLICY TO INFORMATION AFTER INITIAL POLICIES ARE SET IS CRITICAL
-- Side effects, interaction effects, technological change, and mass reactions revealed after the fact
3 INITIAL PUBLIC POLICIES (AND PRIVATE STRATEGIES) TEND TO LOCK IN
-- Deadhand produced by bureaucratic interests, business interests, and ideational inertia
SOME RECOMMENDATIONS
CONVENTIONAL ADVICE: FOCUS ON GETTING IT RIGHT INITIALLY SINCE CANNOT CHANGE
-- given irreversibility of public policies (and private business strategies) in the face of revealed information . . .
-- technocratic emphasis on formal integrated assessment, cost benefit, and risk assessment
UNCONVENTIONAL ADVICE: FOCUS ON IMPROVING CAPACITY FOR ADAPTION
-- emphasis on identifying bureaucratic and business interests in adaptation
-- emphasis on strengthening incentives for surfacing and using information elicited by policy experiments
UNEXPECTED COLLATERAL COSTS/BENEFITS
+ SOX reduction for acid rain on forests and lakes => large human health benefits
+ GI Bill educational benefits => enhanced national economic productivity and stimulus
+ elimination of lead in gasoline to save simple auto catalysts => large human health benefits
- ban on CFCs to preserve ozone => occupational health and safety problems on HCFCs
- oxygenate standard => MTBE for air quality => water contamination
- lead paint abatement programs => housing discrimination against families with children
- fire safety regulations => use of asbestos => adverse heath effects
- asbestos abatement programs => elevated exposures from removal
UNEXPECTED INTERACTION EFFECTS
- cost of emissions control equipment AND improvements in auto quality => reduced turnover => aging fleet
- advanced three way catalysts AND fuels with sulfur => elevated nitrous oxide emissions
- mandate scrubbers AND low sulfur coals in World Bank financed power plants => raise total SOX
- NOX reductions AND VOC constrained environment => perverse effect on ozone production
UNANTICIPATED EFFECTS OF SCIENTIFIC AND TECHNOLOGICAL CHANGE
Development of much better technologies => lock in effects on old strategies predicated on old technologies?
* machine guns - retention of infantry tactics in early phases of WWI
* aircraft carriers - retention of emphasis on battleships through part of WWII
Lower then expected costs of compliance => biases in initial calculation of regulatory costs?
* reduction of dioxins in paper production - companies cut chlorine at much lower cost than expected
* reduction of NOX emissions - crankcase ventilation and EFI cheaper than expected + performance benefit
QUESTIONS -WHICH EXAMPLES OF REVEALED INFORMATION SHOULD HAVE BEEN EXPECTED?
WHAT FACTORS HAVE LIMITED USE OF REVEALED INFORMATION TO CORRECT POLICIES?
"DEADHAND" -- IRREVERSIBILITY OF INITIAL POLICY IN FACE OF REVEALED INFORMATION
ORGANIZATIONAL AND BUREAUCRATIC INTERESTS
* selling once is hard enough . . . selling again is harder
* resistance to changing SOPS within organizations
* resistance to reallocating resources within organizations and across organizations
COGNITIVE AND KNOWLEDGE BASED INTERESTS
* post hoc correction difficult without delegitimating policy – Jasanoff reconstruction plausible basis for policy
* commitment to experts and expertise -- defense mechanisms for epistemic communities
INDUSTRIAL STRUCTURE
* regulations create new business environment
* environment selects for firms -- some die, some survive, and some thrive
* which firms win and which firms lose from changing policies?
FOSTERING ADAPTATION TO INFO REVEALED AFTER INITIAL POLICIES SET
=> BRIAN ZUCKERMAN TMP PHD 2001 – CRITICAL SOURCE OF FIGURES, TEXT, AND ARGUMENT
AMBIENT AIR QUALITY
TRENDS AND PROBLEMS IN AIR QUALITY -- FOLLOWUP ON THURSDAY CAUSAL DIAGRAM
* MECHANISMS FOR SURFACING INFORMATION ON SIDE EFFECTS AND INTERACTION EFFECTS
CARB GATHERS WEEKEND INFO THOUGH INCENTIVES ARE WEAK
EPA SLOW IN UPDATING MODELS TO REFLECT INFORMATION
FIRMS NOT SHARING DATA ON FUEL-CATALYST AFTERTREATMENT INTERACTION EFFECTS
* WHAT INCENTIVES FOR FIRMS AND GOVTS TO DECOUPLE PROBLEMS OR CONSIDER JOINTLY?
* WHAT FIRMS AND GOVTS HAVE INTEREST IN RESISTING OR SUPPORTING CHANGES?
BANKING POLICY
1930s CRISIS => BAILOUT PLUS REGULATION
1980s DEREGULATION with BAILOUT => S&l AND BANKING CRISIS => LIMITED REREGULATION
FOOD SAFETY -- TECH CHANGE, OLD INSPECTION, AND HAACP
* INFORMATION REVEALED THROUGH COMPLAINTS
* IN RETROSPECT, WHICH FIRMS HAVE INTEREST IN RESISTING OR SUPPORTING CHANGES?
CAUTION: DO NOT INTERPRET TREND LINES AS DIRECT INDICES OF EFFICACY OF PUBLIC
ADAPTATION. FIRMS LEARN AND ADAPT, CONSUMERS LEARN AND CHANGE BEHAVIOR, AND WEAK
FIRMS DROP OUT UNDER PRESSURE . . .
Figure 1-1 shows the relative progress toward meeting the national ambient ozone standard as compared with two of
the other criteria air pollutants; carbon monoxide and sulfur dioxide. Progress toward meeting the national goal of
smog-free air has taken longer than expected to attain. Comparing ozone with carbon monoxide, especially, highlights
the relative intractability of the ozone problem; although during the 1970s more municipalities exceeded the carbon
monoxide standard by a greater amount than the ozone standard, by the 1990s almost all locations had met the
carbon monoxide goal. Moreover, smog levels have remained roughly constant during the 1990s, despite the array of
new programs launched by the 1990 amendments to the Clean Air Act. Progress has been limited by difficulties in
incorporating scientific findings into implementation policies, and by unintended consequences and perverse outcomes
stemming from incremental regulation of an non-linear, inter-linked problem.
140%
120%
100%
Ozone
CO
SO2
Standard
80%
60%
40%
20%
0%
1975
1979
1983
1987
1991
1995
Figure 1-2 shows that the incidence of salmonella, the most accurately measured foodborne contaminant, grew
steadily from the 1960s through the 1980s. As discussed above, a combination of trends in food production and
consumption interacted to raise the risk of foodborne illness. Nevertheless, food safety regulators moved slowly
toward introducing new inspection procedures that were better based in science and that responded to the emergence
of new pathogens. USDA officials, under pressure from industry to allow increased productivity and from Congress to
reduce budgets, acted to reorganize their inspector force and tighten command-and-control regulations in ways that did
not effectively reduce growing food safety risks, and may have contributed to them.
30
25
20
15
10
5
0
1960
1965
1970
1975
1980
1985
1990
Salmonella Cases per 100,000
1995
Figure 1-3 shows the number of commercial banks that failed between 1934, the first year after the introduction of
deposit insurance, and 1995. The commercial banking system experienced severe difficulties in the late 1980s and
early 1990s stemming from volatile economic conditions and growing competition in the financial services sector. The
spike during the 1980s and early 1990s reflects not only the failure of a large number of banks, but also the collapse of
several large financial institutions. The number and severity of failures forced Congress to recapitalize the insurance
system in 1991. Despite the presence of an elite corps of bank supervisors using procedures newly updated in order
to provide earlier warning of bank failures, regulators often were surprised by the magnitude and breadth of the
industry’s problems.
300
250
200
150
100
50
Number of Commercial Bank Failures
1994
1989
1984
1979
1974
1969
1964
1959
1954
1949
1944
1939
1934
0
FOOD AND DRUG - CERTIFICATION AND GUIDELINES
* DIETARY SUPPLEMENTS AND HERBAL CURES
* GMO FOODS
* BUTTER-MARGARINE
* DRUGS - CERTIFICATIONS AND GUIDELINES - TRIAZOLAM FAILURE
* ACCUTANE – TORTS SURFACE INFORMATION ON BIRTH DEFECTS, COMPANY IMPROVES CONSENT
TIRE-SUV SAFETY and BOSTON CHURCH
* INFORMATION REVEALED THROUGH COMPLAINTS AND LAW SUITS
* INCENTIVES TO KEEP INFORMATION AND SETTLEMENTS SECRET
* REGULATORY AND ENFORCEMENT AGENCIES DID NOT ACQUIRE INFORMATION UNTIL LATE
* PRESS PLAYED KEY ROLE IN SURFACING INFORMATION ON PROBLEM
* IN RETROSPECT, WHAT INTERESTS IN EARLIER SELF CORRECTION BY FIRMS/CHURCH/GOVT?
HOW WOULD BASIC RECOMMENDATIONS WE STARTED WITH HAVE WORKED?
CONVENTIONAL ADVICE: FOCUS ON GETTING IT RIGHT INITIALLY SINCE CANNOT CHANGE
-- given irreversibility of public policies (and private business strategies) in the face of revealed information . . .
-- technocratic emphasis on formal integrated assessment, cost benefit, and risk assessment
UNCONVENTIONAL ADVICE: FOCUS ON IMPROVING CAPACITY FOR ADAPTION
-- identify bureaucratic and business interests in adaptation – recognize interests in change?
-- strengthen incentives for surfacing and using information elicited by experiments -- what info, and by whom?
-- use compensation to buy off opposition – how set standards?
-- use consummatory/aggregated over instrumental/compartmentalized indicators – even if less precise
….other ideas on reducing tendencies toward lock in and improving performance?