Download (PBAC) and

Quality of decision-making by the Pharmaceutical Benefits Advisory
Committee (PBAC) and the impact on outcomes
Neville A M, Lloyd J M.
Pretium (Pharmaceutical and Health Economic Consultants) Sydney, Australia.
Introduction
In Australia, medicines are registered and approved for use by the Therapeutic
Goods Administration (TGA) and the Australian Drug Evaluation Committee (ADEC).
The TGA carries out a range of assessment and monitoring activities to ensure that
all therapeutic goods available in Australia are of an acceptable standard. At the
same time, the TGA aims to ensure that the Australian community has access, within
a reasonable time, to therapeutic advances. The ADEC was formed in 1963 and
given the role of providing independent, scientific advice on new drugs, within the
policy framework of the time, to the Federal Government. The ADEC is appointed
by the Minister for Health and Ageing and provides advice to the Minister and the
Secretary of the Commonwealth Department of Health and Ageing through the TGA,
on a number of matters including medical and scientific evaluations of applications
for registration of prescription drugs (e.g. new chemical entities, new forms of
previously registered drugs and therapeutic variations to registered drugs). Figure 1
displays the process involved in submitting for and obtaining registration and
marketing approval in Australia.
Figure 1
The processes involved in submitting for and obtaining
registration and marketing approval in Australia.
Given the complexity of the analyses that support submissions to the PBAC,
we sought to determine the quality of the PBAC evaluations (as determined by
the sponsors of the applications), their effect on decision-making and the outcomes
of PBAC meetings.
The questionnaire was divided into three sections; each explored one area of
interest. The first section (Section A) was designed to elicit examples of good
decision-making by the PBAC. The survey asked respondents to provide examples
of a good decision that was experienced by their company. Detail was requested on:
Glossary of Terms
ADEC
DUSC
ESC
PBAC
PBS
PES
PSC
RWC
TGA
Australian Drug Evaluation Committee
Drug Utilisation Sub-Committee (of the PBAC)
Economics Sub-Committee (of the PBAC)
Pharmaceutical Benefits Advisory Committee
Pharmaceutical Benefits Scheme
Pharmaceutical Evaluation Section
Pharmaceutical Sub-Committee (of ADEC)
Restriction Working Committee
Therapeutic Goods Administration
1st submissions
Resubmissions
Total
Good decisions
26
(67%)
6
(33%)
32
(56%)
Poor decisions
13
(33%)
12
(67%)
25
(44%)
39
18
Total
the type of economic argument used in the submission,
whether a positive or a negative recommendation was the outcome,
the reason/s for the decision qualifying as good, and
if the price agreement post recommendation equalled the submission price.
The same detail was requested in the second section (Section B), but focussing
on poor decision-making by the PBAC.
Respondents were asked in Section C to quantify the decisions made by the
PBAC for all submissions made during the 6 meetings of interest. Specifically,
quantitative detail was requested on:
1st submissions,
From sections A and B of the survey, detail was requested about one or two
individual submissions, one good and one poor. From this detail (from 12 companies)
it is hypothesised that the quality of the PBAC decision was related to the form
of economic argument taken. From the feedback received the likelihood of the
evaluation being good was 69% (9/13) if the submission took a cost minimisation
approach. In contrast, the likelihood of an evaluation being good given an
incremental cost effectiveness approach was only 38% (3/8), Table 3.
Table 3
Quality of decision making by type of economic argument
used (numbers and percentages)
resubmissions, and
the maximum number of submissions until a medicine was successfully listed
on the PBS.
Results
Source: Australian Draft Regulatory Guidelines for Prescription Medicines, December 2003
Figure 2
The processes involved in submitting for and
obtaining public subsidy in Australia.
Sponsor – lodges Major Submission
Preparation of DUSC
Secretariat Commentary
Preparation of PBAC
Secretariat Overview
A total of 35 questionnaires were sent and 17 replies were received, a response
rate of 48%. Five of these respondents had not prepared any submissions to the
PBAC in the period of study. Thus 12 companies were able to provide details of their
submissions. The 17 companies had sales in the period of analysis that represented
47% of total pharmaceutical sales [as assessed by IMS data, API and AHI].
From section C of the questionnaire, the total number of submissions made to
the PBAC within the survey period was 57, which is approximately 40-45% of all
submissions. Of these, 39 were first submissions and 18 were resubmissions. Of the
total of 57, 35 submissions resulted in a PBS listing (61% successful). Of the total
submissions, 39 were first submissions (68% of the total). Of the total successful
submissions, 26 were first submissions, a success rate of 67% for this group.
Resubmission had a lower success rate – 9 of 18 submissions resulted in PBS
listing (50%).
Preparation of PES
Commentary
Poor Decisions
Total
Cost minimisation
9
(69%)
4
(31%)
13
Cost effectiveness
3
(38%)
5
(63%)
8
Total
12
(57%)
9
(43%)
PBAC Secretariat
Table 4
26
(66%)
9
(50%)
35
(61%)
Unsuccessful submission
13
(34%)
8
(50%)
22
(39%)
39
18
Sponsor – for Pre-sub-committee Response
Total
Successful
submission
Unsuccessful
submission
Cost minimisation
12
(92%)
1
(8%)
13
Cost effectiveness
5
(63%)
3
(38%)
8
17
4
Total
Total
Reasons for decisions being poor
DUSC Meeting
for DUSC Adviser
Restriction
Working Group
for RWC Advice
ESC Meeting
for ESC Advice
PBAC Secretariat
Sponsor – for Pre-PBAC Response
PBAC Secretariat
PBAC Meeting
Source: Source and flow of documents for major submissions to the PBAC (17 week cycle)
(www.health.gov.au/pbs/pharm/listing/subchart.htm)
From the detail in Section C of the questionnaire, which formed a sample of 57
submissions, industry considered that good decisions were made by the PBAC
56% of the time (32/57). This figure ranged from 67% for first submissions to
only for 33% for resubmissions. Thus, industry considered that poor decisions were
made by the PBAC in 44% of submissions, 33% for first submissions and 67%
of the time for resubmissions, Table 2. The low quality of decision-making in the
resubmission process resulted in many repeated resubmissions. One company
reported that it had taken 6 submissions until a medicine was successfully listed
on the PBS. The sample median was 2, where a figure was provided.
“Conflicting advice [was received from the Secretariat] on appropriate
comparator that led to delays in listing.”
“We were forbidden from using the market leader as the comparison.
We used another product as comparator; our data were not believed,
then we resubmitted using a common comparator to another agent
and, ultimately, were given listing at the market leader price which
was not the comparator in our submission.”
“PBB advised [the sponsor] that drug A was the appropriate comparator.
PBAC stated that the comparator could equally have been another type
of inhibitor or dual inhibitor regimens even though none of these have
regulatory approval.”
2. Inter Agency Inconsistency
Another major issue of concern was that decisions were incorrectly interpreted
results in the favour of the Pharmaceutical Benefits Branch, even ignoring the
PBAC’s own published guidelines. There were instances where very large nonrandomised trials were not considered in the evaluation, valid epidemiological data
was ignored or not accepted or there were problems regarding the definition of
‘therapeutic equivalence’. These flaws could result in outcomes that contradicted
Therapeutic Goods Administration findings.
“TGA/ADEC had accepted that the drug was effective making some
allowance for the Orphan Drug status, but on the same data the
PBAC did not accept that the drug was effective.”
“The quality of submission was criticised because only the preliminary
report of the pivotal study was provided. This was in spite of the fact
that the product was fast-track evaluated by the TGA and the same
report was used to approve product for marketing.”
Two main issues were apparent where the policy set out by the Department was
problematic and not always optimal. The use of an ‘evidence-based’ approach to
developing and evaluating the submissions could require evidence that would be
impossible to prove. Also, the uncertainty over the value of Quality of Life measures
and the related issue of cost utility analyses is an area that needs improvement and
clarification. Others cited costing issues as a concern.
“The pack size recommended was not consistent with evidence
based practice.”
“Quality of life results were compared to baseline levels rather than the
difference between treatment groups (when the guidelines stipulate
comparison should be between treatment groups).”
4. Issues of Expertise
Total
Successful submission
“The comparator’s dose was not accepted – PBAC wanted bd versus
tid (with no evidence cited by PBAC).”
3. Issues of Policy and Process
It is also hypothesised that the success of a first submission was related
to the form of economic argument taken. From the feedback received the
likelihood of the submission being successful was 92% (12/13) if the
submission took a cost minimisation approach. In contrast, the likelihood
of an evaluation being successful given an incremental cost effectiveness
approach was 63% (5/8), Table 4.
Type of decision made by submission / resubmission
(numbers analysed and percentages)
1st submissions Resubmissions
PBAC Secretariat
Good Decisions
Probability of success by type of economic argument used
(numbers and percentages)
Table 1
Method
A good decision was defined as one that incorporates the evidence within a
sound analytical framework. In this case, the framework for analysis is the Guidelines
for the Pharmaceutical Industry on Preparation of Submissions to the PBAC2.
The following criteria were used to define why, when and how the decisions made
were good in quality:
evidence was systematically included by the evaluators, and
A poor decision was defined as one where the Guidelines are not followed or
were poorly interpreted. The reasons why the applicant considered the evaluation
(with specific examples where possible) were provided.
PBAC Secretariat
A survey was conducted to determine pharmaceutical industry’s experience
regarding PBAC decision-making over a period of six PBAC meetings. The survey
period was stipulated as the 4 meetings of the PBAC in 2000 and the first two
(March and June) of 2001. The membership of the PBAC changed after the June
2001 meeting, which would allow ongoing survey work to compare the changes
over time and by different Committees. The questionnaire was designed to elicit
information on good and poor decisions, and information needed to quantify the
issues and their effects on submission outcomes.
Type of decision made by submission / resubmission
(numbers analysed and percentages)
comparator/s were within guidelines and current practice.
However, the Federal Government will only subsidise a drug if the Pharmaceutical
Benefits Advisory Committee (PBAC) evaluation has determined the drug to be
cost effective. The PBAC was established in 1954 under section 101 of the National
Health Act (1953) to make recommendations and give advice to the Federal Minister
of Health about which drugs and medicinal preparations should be made available
as pharmaceutical benefits on the Pharmaceutical Benefits Scheme (PBS). The PBS
is a national (taxpayer funded) scheme that provides subsidised access to necessary
medicines. Note that medicines/medicinal preparations can only be listed on the
PBS for their registered indications (as determined by the TGA & ADEC). Following
an amendment to the Act in 1987, which took effect in 1993, the Committee has
been required by the Act to consider the effectiveness and cost of a proposed drug
or medicinal preparation compared to alternative therapies. Guidelines to assist
those preparing submissions for consideration by the PBAC (in most cases, the
local pharmaceutical industry) have been in existence since 1991 (with subsequent
revisions). Figure 2 displays processes involved in submitting for and obtaining public
subsidy (PBS listing) in Australia.
The content of such submissions and their outcome (recommended, rejected or
deferred) had not, until recently, been made public. However, the government
reviewed the quality of pharmaceutical industry pharmacoeconomic analyses
included in PBAC submission documents and reported that significant problems
existed1. It is likely, however, that, given the complexity of many of the interventions
and the analyses that support the submission, the PBAC evaluation process itself
could contain errors. This is problematic as there is no process for appeal decisions
with which sponsors disagree. Rather, resubmissions to the same Committee,
following the same processes, is required.
Table 2
guidelines were followed by the Pharmaceutical Benefits Branch in the
evaluation of submissions,
A number of reasons were identified as problematic and lowering the quality of
evaluations and decision-making. These have been grouped into 4 areas by the
researchers and are:
Companies recognised that the submissions were sometimes complex and
required the development of considerable expertise internally to understand and
then present the issues. However, the same level of expertise was not always
demonstrable by the evaluators. An example of when this becomes overt is in
the choice of ‘old’ or out-dated surrogate outcomes that are not the best measure
of the benefit of new interventions.
“The primary outcome of a pivotal study was the percent of patients
with undetectable virus. This is recognised by most experts, including
US treatment guidelines, as the best surrogate indicator of future
disease outcomes. PBAC asked the sponsor to justify why an older
and now outdated marker should not have been used instead.”
“[The PBAC] did not accept the comparator dosage which we submitted
(tds dose) because they felt that the most commonly used dosage was
bd (no evidence cited); they did not allow a cost offset for pathology
testing for drug prophylaxis because they felt that such testing is not
necessary (no evidence cited); they approved a pack size (5 injections)
which is less than the minimum duration of treatment (6 days).”
1. Comparator Issues
Many respondents cited comparator issues as a source of concern. This group
ranges from issues over the choice of comparator made by the evaluators,
which was often neither pragmatic nor evidence-based, to the age of comparators
(when the low price of generic comparators would result in returns being inadequate
given the costs of developing innovative new products). Confusion could arise
when current comparator usage may not reflect approved usage, or doses were
not accepted. Companies often received conflicting advice from the PBAC on the
appropriate comparator to use over the course of a submission and subsequent
resubmission. Also, opinion over the most appropriate comparator could differ
between the PBAC and the PBPA.
Conclusions
Government subsidy decision-making is of variable quality, which varies depending
on the method of economic argument used and affects the probability of success
for submissions. A process to appeal decisions or have them reviewed by an
independent body should be considered.
1 Problems With the Interpretation of Pharmacoeconomic Analyses A Review of Submissions to
the Australian Pharmaceutical Benefits Scheme. Hill, SR et al. JAMA. 2000;283:2116-2121.
2 Commonwealth Department of Health and Ageing, Guidelines for the Pharmaceutical Industry on
Preparation of Submissions to the Pharmaceutical Benefits Advisory Committee, September 2002.