Download Measuring the benefits of drug law enforcement

Measuring the benefits of drug law enforcement:
The development of the Australian Federal Police’s Drug Harm Index
1
Robyn G. Attewell , Michael McFadden
2
(3rd Annual Conference of the International Society for the Study of Drug Policy,
March 2009)
1
2
Internal Audit and Business Analysis, Australian Federal Police, Australia
University of Queensland Social Research Centre, Australia
(Word count 3531)
Abstract
This paper outlines the historical development of the Australian Federal Police (AFP)
Drug Harm Index (DHI) and its application in the evaluation of strategic approaches
to combating the importation of illicit drugs into Australia. The AFP DHI was
developed to provide a single measure that encapsulates the potential value to the
Australian community of drug seizures. The index represents the dollar value of harm
that would have ensued had the drugs seized by the AFP reached the community. It is
based on estimates of the social, health and financial cost of drug abuse.
The DHI is a Key Performance Indicator used both internally and externally to report
on drug trafficking investigations conducted by the AFP’s Border and International
Network. The development and refinement of the index over almost a decade has
been in response to the expanding literature on estimating the community cost of
illicit drug use and the changes in drug types, drug production, availability and
consumption. The most recent review proposes purity adjustments, precursor
conversions and the inclusion of sedatives. The paper also outlines the future
refinement and proposed broader application of the index.
The DHI enables economic evaluations to be performed by providing the basis for the
benefit estimate in a benefit cost analysis. It is estimated that, overall, the Australian
community receives approximately $5 to $6 of benefit for every dollar invested in
federal drug law enforcement. The performance of three AFP drug policy strategies
(partnerships with other agencies, focus on serious and complex crime, and emphasis
on intelligence) were evaluated and it is concluded that higher rates of economic
return were achieved for all three.
Background
The impetus for the development of an index of performance regarding importation of
illicit drugs was driven by the accountability reporting requirements resulting from
the introduction of an outcomes and outputs framework by the Australian
Government (Department of Finance and Administration, 2004). Each year, all
Australian federal agencies are required to define and report on the outcomes
expected to be achieved through their outputs. The DHI provided the AFP with a
mechanism for translating data on total seizures of various drugs, into a single dollar
figure of social impact which was simple to calculate and report.
1
The AFP provides a federal law enforcement capacity across a large range of national
interests. It enforces a range of federal laws covering border crime (such as drug
importation and people smuggling), economic crime (including fraud, money
laundering, tax offences identity crime and corruption), and ‘high tech’ or cyber crime
(including online child sex exploitation). Other AFP outputs include counter
terrorism, aviation security, the protection of high office holders and an international
deployment group which provides capacity building programs and offshore law
enforcement initiatives in the Pacific region and elsewhere. It should be noted that
general community policing responsibilities, including most drug-related offences, are
provided by separate law enforcement agencies specific to each Australian State and
Territory. The exception is the Australian Capital Territory (ACT) where such
services are provided by the AFP through a contracted service with the ACT
Government. Thus, the law enforcement activity referred to in this paper relates
primarily to preventing the importation of large to medium size consignments of illicit
drugs (or their precursors) into Australia, and not to domestic production, trafficking
or possession of drugs.
Details of the previous versions of the AFP DHI have been published (McFadden,
Mwesigye and Williamson, 2002 and McFadden, 2006). This paper provides details
of the most recent update of the index and provides an overview of its development.
This paper also provide examples of benefit cost analysis based on the AFP DHI
(McFadden, Mwesigye and Williamson, 2002 and McFadden, 2008).
Methodology
Drug Harm Index methodology
The basic notion of the AFP DHI is that the primary benefit from drug seizures is that
the drugs are prevented from entering the community. Thus the various costs that
would have been associated with the use of these drugs are avoided. The AFP DHI is
defined simply as the dollar estimate harm avoided per kilogram (c) multiplied by the
seizure weight in kilograms (w). However, the relative harm differs for various
classes of drugs, so this needs to be repeated for each drug type and then summed
across the different drug classes. Mathematically, this can be written as
DHI = Σi ciwi where i = 1 .. n, n is the number of different drug classes and ci and
wi are the costs and seizure weights for drug class i.
Both the complexity and evolution of the DHI are associated with deriving the most
valid, accurate and up-to-date estimates of social cost per kilogram, and the choice of
the most appropriate and comprehensive groupings of drugs to which these estimates
can be applied. The AFP DHI was first derived in 2001, revised in 2003, an interim
adjustment made in 2006, and another major revision is currently being finalised.
AFP DHI 2001 version
In the absence of a comprehensive estimate of the social cost of drug abuse in
Australia, the original index was based on street price converted to cost per kilogram.
McFadden, Mwesigye and Williamson (2002) reasoned that street price could be
substituted for the economic value since estimates from separate studies in the United
States differed by no more than 5%. Regionally based Australian street prices were
2
then obtained for heroin, cocaine, amphetamines and cannabis and used in the DHI
formula after an adjustment for a difference in purity between drugs seized at the
border and drugs on the street.
AFP DHI 2003 version
The primary improvement in 2003 was basing the index on Australian data. Full
details are available in McFadden (2006). The harm value per kilogram was derived
from dividing the total (annual) cost of harm estimated by Collins and Lapsley (2002)
by an estimate of total (annual) consumption of drugs from Australian surveys (Hall
2000 and AIHW 1999). The Collins and Lapsley study was one of a series of studies
commissioned by the Commonwealth Department of Health and Ageing to measure
the social costs of drug abuse. Total social costs were obtained by summing separate
component estimates of tangible and intangible costs (Figure 1). Crime was the
largest cost component (Figure 1). This breakdown was important since these
component costs were further disaggregated by drug type (opioids, stimulants and
cannabis) by McFadden (2006). It should be noted that precursor chemical seizures
were included in the stimulant group.
Figure 1. Estimates of social harm attributable to illicit drug use divided into tangible and intangible costs in
Australia, 1998, from Collins and Lapsley (2002). Note health costs include road crashes and resources denotes
resources used in abusive consumption
Pain &
suffering Labour
Loss of life
costs
Health care
Resources
Crime
Tangible
Intangible
AFP DHI 2006 version
An interim update occurred in 2006 due to peer review and literature review which
suggested that there was an overestimate of the harm associated with heroin and an
underestimate of harm associated with amphetamines. Hence the heroin and
amphetamine weightings were adjusted within the constraint of keeping the total
estimated harm consistent with Collins and Lapsley (2002). The effect is illustrated in
Figure 2.
AFP DHI 2009 review
The AFP has commissioned a review of the DHI by the University of Queensland
Social Research Centre (UQSRC). At the time of preparing this paper, the AFP had
yet to endorse the recommendations contained in the UQSRC submission.
3
The main issues prompting the review are the availability of new source data
(updating from 1998 to 2004) and the need to evaluate the impact of an additional
drug type (sedatives) which was not previously addressed but which has become more
prominent in recent AFP seizures.
Despite an update in the Collins and Lapsley series (2008), Moore (2007) was used as
the starting point for the index since separate specific estimates of social cost per
kilogram for different drug types were provided and other limitations of Collins and
Lapsley were addressed. The estimates were extrapolated to other drug types through
using the relative harm ratings from a UK survey reported by the Advisory Council on
the Misuse of Drugs (Commons Scientific and Technology Committee, 2006) (Table
1). Total consumption of the different drug types was calculated consistent with the
previous version, by applying estimates of average consumption from the 2007 World
Drug Report (UNODC, 2007) to estimates of the total number of users in Australia
from the 2007 National Drug Strategy Household Survey (AIHW 2008) and the
population census (ABS, 2008).
In a final step, conversion ratios from precursor to final product were sourced from
the UNODC (2006) and applied to amphetamine costs for ATS precursors ephedrine
and pseudoephedrine (0.70) and to MDMA costs for MDMA precursors (0.10)
according to the following formula:
Cost per precursor kg = Cost per product kg x conversion rate
Table 1. Expert panel ratings of overall harm (Commons Science and Technology Committee, 2006), inclusion in
Moore (2007) and correspondence with AFP and Australian Bureau of Statistics (ABS) drug classifications.
Overall Harm
UK Drug of Interest
Rating(0 = no risk,
3 = extreme)
Corresponding
Included in
Corresponding AFP
Moore (2007)
Drug Group
Analgesic
ABS Drug
Classification
Heroin
2.77
Yes
Heroin (Opioids)
Cocaine
2.30
Yes
Cocaine
Stimulants
Barbiturates
2.08
No
Other
Sedatives
Street methadone
1.94
No
Heroin (Opioids)
Analgesic
Ketamine
1.74
No
Other
Sedatives
Benzodiazepines
1.70
No
Other
Sedatives
Amphetamines
1.66
Yes
Amphetamine
Stimulants
Buprenorphine
1.58
No
Other
Analgesic
Cannabis
1.33
Yes
Cannabis
Stimulants
4-MTA
1.27
No
Other
Stimulants
LSD
1.23
No
Hallucinogen
Stimulants
Methylphenidate
1.18
No
Other
Stimulants
Steroids
1.15
No
Other
Anabolic Agents
GHB
1.12
No
Other
Sedatives
Ecstasy
1.09
No
MDMA
Stimulants
4
Table 2. Aggregation up to the Final Social cost per kilogram by broad drug class from estimates for specific
drugs with weighting by prevalence (AIHW, 2008).
AFP DHI
2009
Final
Social
$ per kg
AFP Drug
Group
$ per kg
UK drugs of
interest
Stimulants
Sedatives
Cannabis
1,008,522
Heroin
1,008,522
2007
prevalence
%
1,148,914
0.2
Street methadone
802,307
0.05
Buprenorphine
653,169
0.05
Heroin
Opioids
$ per kg
Amphetamines
333,472
Amphetamines
333,472
2.3
Cocaine
461,369
Cocaine
461,369
1.6
Hallucinogen
246,421
LSD
246,421
0.6
MDMA
218,966
Ecstasy
218,966
3.5
Barbiturates
417,844
0.1
Ketamine
350,212
0.2
GHB
224,323
0.1
7,658
9.1
302,426
335,648
7,658
Sedatives
335,648
Cannabis
7,658
Cannabis
Note: Street methadone and buprenorphine are combined in the AIHW survey (0.1%). They were allocated equal
portions of that estimate of prevalence of recent us (0.05% each).
Table 3. Overview of the development of the AFP DHI in the period 2001 to 2009 (noting that the 2009 update
has not been implemented and remains a proposal at this time)
Year of
update/
review
2001
Year of
primary
source data
1999
2003
1998
2006
1998
2009
2004
Drug classes
Heroin, Cocaine,
Amphetamines,
Cannabis
Opioids,
Amphetamines,
Cannabis
Opioids,
Amphetamines,
Cannabis, Precursors
Opioids,
Amphetamines,
Cannabis, Precursors,
Sedatives
Purity
adjustment
Yes
Precursor
conversion
No
No
No
No
Nominal
Yes
Yes
Primary limitations
Based on US research
Based on 1998 source data;
amphetamine harm
underestimated
Based on 1998 source data;
excludes sedatives, only
nominal inclusion of precursors
Based on 2004 source data; cost
estimates interpolated for
certain drugs. Precursor
conversion and purity estimates
are incomplete.
Potential impact on AFP DHI results
Table 3 summarises the methodological development of the AFP DHI over the past
decade. Figure 2 illustrates the change in relative weights for each primary drug class
over the time. It should be noted that the most recent update has not yet been
implemented and is a proposal at this stage.
McFadden (2006) made detailed comparisons on the result of calculating the original
AFP DHI (2001) and the AFP DHI (2003) on AFP seizures in the period 1987 to
2003. It was concluded that the results were comparable (in fact, only 3% different)
and trends in annual values were similar. The potential impact is greater in the most
recent review, but this is largely due to large GHB seizures in 2008. The total savings
5
to the community from AFP drug seizures during the period July 1999 to December
2008 were estimated to be $7.8 billion using AFP DHI 2006, but increase to $9.8
billion using the proposed AFP DHI 2009; a 27% increase. A breakdown of costs
indicates that 15% of the increase is based on the inclusion of sedatives, 2% to CPI
increases and 10% to changes in the cost estimates.
Figure 2. Estimated social cost per kilogram of drugs of different types in successive versions of the AFP DHI
($AUD) (noting that the 2009 DHI remains a proposal at this time).
$1,200,000
AUD
$1,000,000
$800,000
DHI2003
$600,000
DHI2006
$400,000
DHI2009
$200,000
s
ive
Se
da
t
is
Pr
ec
ur
so
rs
Ca
nn
ab
ul
an
ts
St
im
O
pi
o
id
s
$0
Return on Investment Methodology
Benefit-cost analysis provides a way of quantifying the economic performance of a
program. Firstly, benefits and costs are estimated in dollar terms. Then they are
quantitatively compared by calculating either a ratio (of benefit to cost) or a
difference (benefit minus cost). The ratio is termed the Return on Investment (ROI).
It is interpreted as the return achieved for each dollar spent. The difference is termed
the net present value (NPV). It is the net return after costs are taken into account.
The AFP has applied this type of analysis in evaluating drug law enforcement for the
1999-00 to 2000-01 period (McFadden, Mwesigye and Williamson, 2002) and a
slightly more comprehensive version for the 2000-01 to 2004-05 period (McFadden,
2008). On the benefits side of the equation the AFP DHI 2001 and AFP DHI 2006
were used, respectively, in the two studies, to estimate the direct impact of making the
seizures. In the latter study, a further deterrence benefit was included. This was
estimated to be an additional 10% of the direct impact. On the costs side of the
equation, the original analysis only included estimates of policing and border control
costs incurred by the AFP and the Australian Customs and Border Protection Service
(Customs), respectively. This was expanded to include legal costs (Director of Public
Prosecutions costs and court costs) and prison costs (based on Productivity
Commission estimates of costs per prisoner per day) (Figure 3). A crude estimate of
variability in the final ROI figure was obtained through applying a notional variability
factor to the components. This ranged from 5% for costs estimated directly to 20%
for costs estimated from secondary sources.
6
Return on Investment Results
Despite the additional sophistication of the second model and the slightly different
time periods, the ROI estimates from both studies were similar; a $5.20 return for the
first and $5.80 for the second. The second study was based on unit record data of
2,716 drug investigations in the five year period. Three specific strategies relevant to
this period were able to be evaluated by recalculating the ROI after restricting the
analysis to those cases impacted by the policy, such as those involving domestic and
international partners, those concentrating on serious or complex crime and those
involving intelligence. For example, 1,936 out of the 2,716 cases were referred to the
AFP by Customs. These cases resulted in an ROI estimate of $6.00 compared to the
overall estimate of $5.80. Figure 4 shows the full set of results. Each policy related
estimate corresponded to a larger return than the overall estimate and thus provides an
evidence base for recommending further implementation of the policies. This marked
an important step forward in the use of the Drug Harm Index from being primarily a
reporting and an accountability tool to an essential component of the AFP’s ability to
monitor, adjust and refine specific operational strategies in drug law enforcement.
The AFP is currently in the process of updating the records in the second study to
include cases completed by December 2008. The number of drug investigations
available for analysis is 3,641. Preliminary analysis using the AFP DHI 2006
indicates an overall ROI of 5.7 which is nearly the same as the earlier estimate.
However, if the recommendations of the UQSRC report into the 2009 review of the
DHI are accepted, the ROI increases to 6.8. The availability of more unit records will
allow the AFP to conduct more sensitive and detailed analyses of the efficiency and
effectiveness of its drug law enforcement strategies. It is again indicative of an
approach of ongoing improvement and refinement to our techniques for assessing our
own performance. A detailed report should be available later in the year.
Figure 3. Estimates of cost and benefit of federal Australian drug law enforcement associated with illicit drug
seizures in the period 2000-01 to 2004-05 (McFadden 2008)
$2,000
Costs
$1,750
AFP
AUD $m
$1,500
Customs
$1,250
Legal
$1,000
Prisons
$750
$500
Benefits
$250
Direct (AFP DHI)
$0
Costs
Benefits
Deterrence
7
Figure 4. Estimates of return on investment in drug law enforcement associated with illicit drug seizures in the
period 2000-01 to 2004-05 (McFadden 2008). Note that error bars provide some indication of variability in the
component costs. The percentages shown are the proportion of the total investigations for each subset. However,
the subsets are not mutually exclusive.
Return on Investment (ROI)
$0
$1 $2 $3
$4 $5 $6
$ 7 $ 8 $ 9 $ 10 $ 11
All AFP drug investigations
Subset involving:
Domestic partner 71%
International partner 12%
Serious & complex crime 14%
Intelligence-led policing 19%
Discussion
The simplicity of the AFP DHI as an aggregation of social cost by seizure weight
across different drug types belies the difficulty in producing a valid and accurate
index. The estimation of social cost of illicit drugs is only truly applicable in the
timeframe and region in which source data are collected. However, the expense of
conducting such studies makes it unlikely they would ever be conducted annually.
The same limitation applies to estimating consumption. In addition, consumption
estimates are notoriously difficult and non-response and underestimation will always
be a problem when posing questions about illegal activity. Further limitations of the
methodology include the assumption that harm is constant by weight and over time.
Adjustment for dependant users and market supply may address these issues. Some
consideration should also be given to environmental costs such as those associated
with the production of synthetic drugs as in the recent report on the economic cost of
methamphetamine use (Nicosia, Liccardo Pacula, Kilmer, Lundberg and Chiesa,
2009). Finally, although some sensitivity analysis was conducted on the economic
evaluation, the statistical precision of the estimation needs to be considered in more
detail.
Conclusion
Despite the limitations resulting from the paucity of data and the complexity required
to keep pace with the ever-changing illicit drug landscape, the DHI is based on sound
methodology and has proven to have great utility as a performance measure within the
AFP. Since it is based on the premise of estimating benefit through reduction in
consumption, it is also applicable beyond law enforcement, for example in evaluation
of drug treatment programs. Further development of this type is being addressed in
Australia by a coalition of researchers, policy makers and policy implementers who
have sought funding to develop a more broadly applicable Australian Drug Harm
Index which will be suitable for use in both law enforcement and health fields. The
AFP is a keen supporter of this and is one of the partners in the proposed project. It is
expected that the collaboration would result in better access to contemporary research
through pooling resources, higher level of scientific rigour in the methodology, and
negotiation and agreement on the best approaches to the identified limitations.
8
Finally, an Australian DHI would result in uniformity of reporting to government
which would be advantageous in the current period of economic restraint.
References
Australian Bureau of Statistics (2008) 3235.0 - Population by Age and Sex, Regions of
Australia, 2007, Commonwealth of Australia.
Australian Institute of Health and Welfare (1999) National Drug Strategy Household
Survey 1998, Commonwealth of Australia.
Australian Institute of Health and Welfare (2008) 2007 National Drug Strategy
Household Survey: first results. Drug Statistics Series number 20, Cat. no. PHE 98,
Canberra: AIHW.
Collins D & Lapsley H (2002) Counting the Cost: Estimates of the Social Costs of
Drug Abuse in Australia in 1998-9, Canberra: Commonwealth Department of Health
and Ageing.
Collins D & Lapsley H (2008) The costs of tobacco, alcohol and illicit drug abuse to
Australian society in 2004/05, Commonwealth of Australia.
Commons Science and Technology Committee (2006) Drug classification: making a
hash of it? House of Commons.
Department of Finance and Administration (2004) Structuring Outcomes and Outputs,
Commonwealth of Australia.
Hall W, Ross J, Lynskey M, Law M & Degenhardt L (2000) How many dependent
heroin users are there in Australia? Monograph 44. Sydney: National Drug and
Alcohol Research Centre.
McFadden M (2006) The Australian Federal Police Drug Harm Index: A new
methodology for quantifying success in combating drug use, Australian Journal of
Public Administration, 65 (4), 68-81.
McFadden M, Mwesigye S & Williamson G (2002) Pricing outputs: A case study in
law enforcement, Australian Journal of Public Administration, 61 (4), 80-88.
9
McFadden M (submitted 2008) Measuring the Benefits of Drug Law Enforcement,
International Journal of Public Sector Management.
Moore T (2007) Working estimates of the social cost per gram and per user for
cannabis, cocaine, opiates and amphetamines, Drug Policy Modelling Program
Monograph 14, Turning Point.
Nicosia N, Liccardo Pacula R, Kilmer B, Lundberg R and Chiesa J (2009) The
economic cost of methamphetamine use in the United States, 2005, RAND
Corporation.
United Nations Office on Drugs and Crime (2006) Precursor control at a glance,
United Nations Publication, RAS938.
10